[edk2-devel] Runtime Page Granularity on ARM64

[edk2-devel] Runtime Page Granularity on ARM64

[Oliver Smith-Denny]

Hi Ard,

I just sent out a patch (MdeModulePkg: HeapGuard: Don't Assume Pool Head
Allocated In First Page) to fix HeapGuard GuardAlignedToTail behavior on
ARM64. However, this raised a question of why ARM64 sets
RUNTIME_PAGE_ALLOCATION_GRANULARITY to 64k when X64 does not. You added
this in ProcessorBind.h for ARM64, so I am hoping to get some additional
context from you (or anyone on the mailing list who has insight).

I understand that on ARM64 we can have 64k pages in the OS, but what I
do not understand is why we need to map in 64k chunks in UEFI. I see the
UEFI spec says that ARM allows for 64K pages and that if runtime code
or data is within a 64KB page then all 4k pages within that 64K page
need the same memory attributes, which makes sense.

Is this runtime granularity just to satisfy that requirement that the
memory attributes are the same or is there an additional reason that
we need to use the 64k granularity on ARM64?

In any case, I am confused why this is not an issue on X64 or if we
have 2MB pages in the OS? I'm not as familiar with the mechanisms an
OS will use to map runtime services within its space, but they will be
virtualized and the OS will have its own page tables, so it doesn't
quite follow to me why the OS cares all that much what UEFI has done.

Any light you can shed here would be greatly appreciated.

Thanks,
Oliver



-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.
View/Reply Online (#107657): https://edk2.groups.io/g/devel/message/107657
Mute This Topic: https://groups.io/mt/100652665/7686176
Group Owner: devel+owner@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub [rebecca@openfw.io]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [edk2-devel] Runtime Page Granularity on ARM64

[Ard Biesheuvel]

Hi,

On Wed, 9 Aug 2023 at 23:35, Oliver Smith-Denny
<osde@linux.microsoft.com> wrote:
>
> Hi Ard,
>
> I just sent out a patch (MdeModulePkg: HeapGuard: Don't Assume Pool Head
> Allocated In First Page) to fix HeapGuard GuardAlignedToTail behavior on
> ARM64. However, this raised a question of why ARM64 sets
> RUNTIME_PAGE_ALLOCATION_GRANULARITY to 64k when X64 does not. You added
> this in ProcessorBind.h for ARM64, so I am hoping to get some additional
> context from you (or anyone on the mailing list who has insight).
>
> I understand that on ARM64 we can have 64k pages in the OS, but what I
> do not understand is why we need to map in 64k chunks in UEFI. I see the
> UEFI spec says that ARM allows for 64K pages and that if runtime code
> or data is within a 64KB page then all 4k pages within that 64K page
> need the same memory attributes, which makes sense.
>
> Is this runtime granularity just to satisfy that requirement that the
> memory attributes are the same or is there an additional reason that
> we need to use the 64k granularity on ARM64?
>
> In any case, I am confused why this is not an issue on X64 or if we
> have 2MB pages in the OS? I'm not as familiar with the mechanisms an
> OS will use to map runtime services within its space, but they will be
> virtualized and the OS will have its own page tables, so it doesn't
> quite follow to me why the OS cares all that much what UEFI has done.
>
> Any light you can shed here would be greatly appreciated.
>

It is not about how UEFI maps them at boot time at all, and we happily
use 1GB or 2MB mappings for runtime regions if the permission
attributes allow it. It is simply about the granularity of regions
that the OS needs to care about, i.e., those with the
EFI_MEMORY_RUNTIME attribute set in the EFI memory map, as well as
memory used for ACPI tables or mapped at runtime by the AML
interpreter.

There are two important differences with X64:
- arm64 supports 16k and 64k pages
- arm64 does not tolerate aliases with mismatched attributes

If a EFI_MEMORY_RUNTIME region is not aligned to 64k, the OS is
generally forced to round outwards if it is running with a page size >
4k.This means that some adjacent physical pages will be covered by
[typically] a writeback cacheable mapping even though the memory map
may describe them differently. For instance, a EFI reserved region
used by the ACPI parking protocol requires non-cacheable mappings, and
even if such a mapping is created elsewhere in the OS's virtual
address space, if it overlaps with a cacheable mapping of the same
physical 64k, the result is unpredictable. (The uncached accesses are
likely to hit in the cache inadvertently if a [speculative] access via
the cached alias pulled in the data)

Whether or not the architecture or OS supports 2 MB pages is not
really relevant, given that it will never be forced to use those for
regions that are not aligned to 2 MB. A 64k pagesize OS simply has no
smaller granule available, and so it has to round outwards (*)

The least impactful way to achieve this in EDK2 was to increase the
page allocation granularity for runtime memory types (and rework the
pool allocator to make better use of memory that is allocated in
larger chunks). I imagine there might be other ways to ensure that
EFI_MEMORY_RUNTIME regions are aligned sufficiently, e.g., by
reasoning about whether or not adjacent regions may require different
attributes, and permitting misalignment if they don't. But this will
be a lot more hassle, and a lot more room for error.

-- 
Ard.


(*) Theoretically, we could always map EFI memory with 4k granularity
in Linux, given that we use TTBR0 mappings for it, whose granule size
is independent from the granule size used by the OS for its own VA
space, but this is not straight-forward in terms of implementation.


-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.
View/Reply Online (#107659): https://edk2.groups.io/g/devel/message/107659
Mute This Topic: https://groups.io/mt/100652665/7686176
Group Owner: devel+owner@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub [rebecca@openfw.io]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [edk2-devel] Runtime Page Granularity on ARM64

[Andrew Fish via groups.io]

[-- Attachment #1: Type: text/plain, Size: 2677 bytes --]

Oliver,

My understanding is that AArch64 (ARMv8) supports 3 page table granules: 4KB, 16KB, and 64KB [1]. These granules represent the smallest range for a page table, and this granule changes which bits of the VA index into what level of paging. On x86 this indexing was fixed an a 2 MiB page table just liveds higher up in the same hierarchy. 

While AArch64 supports all 3 granule sizes, how many granule sizes are supported by a given CPU is implementation defined. So it is legal for an AArch64 CPU to only support 64KB pages. You can always do 4K pages on Intel, so that is the difference. 

[1] https://developer.arm.com/documentation/101811/0103/Translation-granule

Thanks,

Andrew Fish

> On Aug 9, 2023, at 5:35 PM, Oliver Smith-Denny <osde@linux.microsoft.com> wrote:
> 
> Hi Ard,
> 
> I just sent out a patch (MdeModulePkg: HeapGuard: Don't Assume Pool Head
> Allocated In First Page) to fix HeapGuard GuardAlignedToTail behavior on
> ARM64. However, this raised a question of why ARM64 sets
> RUNTIME_PAGE_ALLOCATION_GRANULARITY to 64k when X64 does not. You added
> this in ProcessorBind.h for ARM64, so I am hoping to get some additional
> context from you (or anyone on the mailing list who has insight).
> 
> I understand that on ARM64 we can have 64k pages in the OS, but what I
> do not understand is why we need to map in 64k chunks in UEFI. I see the
> UEFI spec says that ARM allows for 64K pages and that if runtime code
> or data is within a 64KB page then all 4k pages within that 64K page
> need the same memory attributes, which makes sense.
> 
> Is this runtime granularity just to satisfy that requirement that the
> memory attributes are the same or is there an additional reason that
> we need to use the 64k granularity on ARM64?
> 
> In any case, I am confused why this is not an issue on X64 or if we
> have 2MB pages in the OS? I'm not as familiar with the mechanisms an
> OS will use to map runtime services within its space, but they will be
> virtualized and the OS will have its own page tables, so it doesn't
> quite follow to me why the OS cares all that much what UEFI has done.
> 
> Any light you can shed here would be greatly appreciated.
> 
> Thanks,
> Oliver
> 
> 
> 
> 
> 
> 



-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.
View/Reply Online (#107660): https://edk2.groups.io/g/devel/message/107660
Mute This Topic: https://groups.io/mt/100652665/7686176
Group Owner: devel+owner@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/leave/12367111/7686176/1913456212/xyzzy [rebecca@openfw.io]
-=-=-=-=-=-=-=-=-=-=-=-



[-- Attachment #2: Type: text/html, Size: 3651 bytes --]

Re: [edk2-devel] Runtime Page Granularity on ARM64

[Oliver Smith-Denny]

Thanks Ard and Andrew, I appreciate the info! That clears things up for
me.

Oliver

On 8/9/2023 3:56 PM, Ard Biesheuvel wrote:
> Hi,
> 
> On Wed, 9 Aug 2023 at 23:35, Oliver Smith-Denny
> <osde@linux.microsoft.com> wrote:
>>
>> Hi Ard,
>>
>> I just sent out a patch (MdeModulePkg: HeapGuard: Don't Assume Pool Head
>> Allocated In First Page) to fix HeapGuard GuardAlignedToTail behavior on
>> ARM64. However, this raised a question of why ARM64 sets
>> RUNTIME_PAGE_ALLOCATION_GRANULARITY to 64k when X64 does not. You added
>> this in ProcessorBind.h for ARM64, so I am hoping to get some additional
>> context from you (or anyone on the mailing list who has insight).
>>
>> I understand that on ARM64 we can have 64k pages in the OS, but what I
>> do not understand is why we need to map in 64k chunks in UEFI. I see the
>> UEFI spec says that ARM allows for 64K pages and that if runtime code
>> or data is within a 64KB page then all 4k pages within that 64K page
>> need the same memory attributes, which makes sense.
>>
>> Is this runtime granularity just to satisfy that requirement that the
>> memory attributes are the same or is there an additional reason that
>> we need to use the 64k granularity on ARM64?
>>
>> In any case, I am confused why this is not an issue on X64 or if we
>> have 2MB pages in the OS? I'm not as familiar with the mechanisms an
>> OS will use to map runtime services within its space, but they will be
>> virtualized and the OS will have its own page tables, so it doesn't
>> quite follow to me why the OS cares all that much what UEFI has done.
>>
>> Any light you can shed here would be greatly appreciated.
>>
> 
> It is not about how UEFI maps them at boot time at all, and we happily
> use 1GB or 2MB mappings for runtime regions if the permission
> attributes allow it. It is simply about the granularity of regions
> that the OS needs to care about, i.e., those with the
> EFI_MEMORY_RUNTIME attribute set in the EFI memory map, as well as
> memory used for ACPI tables or mapped at runtime by the AML
> interpreter.
> 
> There are two important differences with X64:
> - arm64 supports 16k and 64k pages
> - arm64 does not tolerate aliases with mismatched attributes
> 
> If a EFI_MEMORY_RUNTIME region is not aligned to 64k, the OS is
> generally forced to round outwards if it is running with a page size >
> 4k.This means that some adjacent physical pages will be covered by
> [typically] a writeback cacheable mapping even though the memory map
> may describe them differently. For instance, a EFI reserved region
> used by the ACPI parking protocol requires non-cacheable mappings, and
> even if such a mapping is created elsewhere in the OS's virtual
> address space, if it overlaps with a cacheable mapping of the same
> physical 64k, the result is unpredictable. (The uncached accesses are
> likely to hit in the cache inadvertently if a [speculative] access via
> the cached alias pulled in the data)
> 
> Whether or not the architecture or OS supports 2 MB pages is not
> really relevant, given that it will never be forced to use those for
> regions that are not aligned to 2 MB. A 64k pagesize OS simply has no
> smaller granule available, and so it has to round outwards (*)
> 
> The least impactful way to achieve this in EDK2 was to increase the
> page allocation granularity for runtime memory types (and rework the
> pool allocator to make better use of memory that is allocated in
> larger chunks). I imagine there might be other ways to ensure that
> EFI_MEMORY_RUNTIME regions are aligned sufficiently, e.g., by
> reasoning about whether or not adjacent regions may require different
> attributes, and permitting misalignment if they don't. But this will
> be a lot more hassle, and a lot more room for error.
> 


-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.
View/Reply Online (#107661): https://edk2.groups.io/g/devel/message/107661
Mute This Topic: https://groups.io/mt/100652665/7686176
Group Owner: devel+owner@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub [rebecca@openfw.io]
-=-=-=-=-=-=-=-=-=-=-=-