Re: "practical" memory allocation limit?

[Andrew Fish <afish@apple.com>]

For X64 (x86-64) systems it is common for PEI to run in 32-bit mode with paging disabled. This means the DXE Core gets loaded under 4GB and and the HOBs and such are < 4GB. So having the initial memory map < 4GB helps with preventing fragmentation. 

There is also architectural cruft on x86. Historically some of the PCI devices did not support DMA > 4GB and the system needed memory under for DMA buffers. The other problem is processor mode transitions as real mode code needs to be < 1MB and 32-bit protected mode needs to be under 4GB. To get into the 64-bit long mode you start in real mode, transition to protected mode, and then enter long mode. Even in 2017 there are still real mode entry points required for x86, mainly the reset vector, and IPI for the AP (how you wake up the other CPUs even in the OS). If you are using a CSM and need to get into real mode I think the thunk code assumes it runs < 4G (at least it did a long time ago). 

Not to mention just general cruft in the code that has assumptions about running < 4G (things like AsmThunk16(), CPU drivers etc.). 

Thanks,

Andrew Fish 


> On Aug 28, 2017, at 8:31 AM, Shi, Steven <steven.shi@intel.com> wrote:
> 
> OK, got it. Thanks.
> 
> 
> 
> For the why the 64bits DXE usually prefer allocations below 4GB, there is a good white paper elaborate the memory service initialization flows and can explain the reason. Please see the page 23 in below white paper (I have mark the answer in red).
> 
> 
> 
> https://github.com/tianocore-docs/Docs/blob/master/White_Papers/A_Tour_Beyond_BIOS_Securiy_Enhancement_to_Mitigate_Buffer_Overflow_in_UEFI.pdf
> 
> Heap Management in EDK II
> In UEFI, the DxeCore maintains the heap usage. The UEFI driver or application may call
> AllocatePages/FreePages/AllocatePool/FreePool to allocate or free the resource, or call
> GetMemoryMap() to review all of the memory usage.
> [Heap Initialization]
> When DxeIpl transfers control to the DxeCore, all of the resource information is reported in a
> Hand-off-Block (HOB) [PI] list. The DxeCore constructs the heap based upon the HOB
> information. See figure 4-2 Heap Initialization.
> 1) The DxeCore needs to find one region to serve as the initial memory in
> CoreInitializeMemoryServices()
> (https://github.com/tianocore/edk2/blob/master/MdeModulePkg/Core/Dxe/Gcd/G
> cd.c). The function is responsible for priming the memory map so that memory allocations
> and resource allocations can be made. If the memory region described by the PHIT HOB is
> big enough to hold BIN and minimum initial memory, this memory region is used as
> highest priority. It can make the memory BIN allocation to be at the same memory region
> with PHIT that has better compatibility to avoid memory fragmentation. Usually the BIN
> size is already considered by platform PEIM when the platform PEIM calls
> InstallPeiMemory() to PEI core.
> 
> 
> 
> 
> 
> 
> 
> 
> Steven Shi
> 
> Intel\SSG\STO\UEFI Firmware
> 
> 
> 
> Tel: +86 021-61166522
> 
> iNet: 821-6522
> 
> 
> 
> 
> 
>> -----Original Message-----
> 
>> From: Laszlo Ersek [mailto:lersek@redhat.com]
> 
>> Sent: Monday, August 28, 2017 10:40 PM
> 
>> To: Shi, Steven <steven.shi@intel.com>; edk2-devel-01 <edk2-
> 
>> devel@lists.01.org>
> 
>> Cc: Kinney, Michael D <michael.d.kinney@intel.com>; Gao, Liming
> 
>> <liming.gao@intel.com>; Ard Biesheuvel <ard.biesheuvel@linaro.org>
> 
>> Subject: Re: [edk2] "practical" memory allocation limit?
> 
>> 
> 
>> On 08/28/17 16:25, Shi, Steven wrote:
> 
>>> Hi Laszlo,
> 
>>> 
> 
>>> I happen to have a question about how to correctly get the system memory
> 
>> size on Qemu.
> 
>>> 
> 
>>> In the QemuInitializeRam() of OvmfPkg\PlatformPei\MemDetect.c, I add
> 
>> debug info as below to trace the GetSystemMemorySizeBelow4gb() and
> 
>> GetSystemMemorySizeAbove4gb() output. But the output results seems not
> 
>> right if my input memory size > 4GB. See my commands and trace outputs in
> 
>> below.
> 
>>> 
> 
>>> 
> 
>>> 
> 
>>> MemDetect.c  Line 602:
> 
>>> 
> 
>>>  //
> 
>>> 
> 
>>>  // Determine total memory size available
> 
>>> 
> 
>>>  //
> 
>>> 
> 
>>>  LowerMemorySize = GetSystemMemorySizeBelow4gb ();
> 
>>> 
> 
>>>  UpperMemorySize = GetSystemMemorySizeAbove4gb ();
> 
>>> 
> 
>>>  DEBUG ((EFI_D_INFO, "LowerMemorySize= 0x%x\n", LowerMemorySize));
> 
>>> 
> 
>>>  DEBUG ((EFI_D_INFO, "UpperMemorySize= 0x%x\n", UpperMemorySize));
> 
>>> 
> 
>>> 
> 
>>> 
> 
>>> My test commands and trace outputs:
> 
>>> 
> 
>>> $ /opt/qemu/bin/qemu-system-x86_64 -m 5120 -enable-kvm -hda
> 
>> /home/jshi19/workspace/simics5-project/images/luv-
> 
>> v2.1_diskboot_gpt_x86_64_.img -machine pc-q35-2.9 -bios OVMF.fd -serial
> 
>> file:serial.log
> 
>>> 
> 
>>> LowerMemorySize= 0x80000000 //2GB, but should not it be 4GB?
> 
>>> 
> 
>>> UpperMemorySize= 0xC0000000 //3GB, but should not it be 1GB?
> 
>> 
> 
>> No, this is correct; the values returned are system memory *amounts*.
> 
>> (On the QEMU command line, the "-m" switch says how much system
> 
>> memory
> 
>> you have, not the highest address in system memory.)
> 
>> 
> 
>> In the 4GB address space, you don't have *just* system memory, you have
> 
>> a whole bunch of MMIO too. (Most of that is used for 32-bit MMIO PCI BAR
> 
>> allocation, some is used for the pflash chip range, LAPICs, etc.) On Q35
> 
>> you also have 256MB assigned to the MMCONFIG / ECAM area (which
> 
>> provides
> 
>> direct MMIO access to PCI config space for PCI 256 buses).
> 
>> 
> 
>> The physical memory map also depends on the board model; on i440fx, the
> 
>> 32-bit system memory can go up to 3GB, while on Q35, it only goes up to
> 
>> 2GB.
> 
>> 
> 
>> So, if you pass 5GB to a Q35 machine, 2GB of that are placed in the
> 
>> address space at [0, 2G), and the other 3GB are placed in the address
> 
>> space at [4G, 7G).
> 
>> 
> 
>>> $ /opt/qemu/bin/qemu-system-x86_64 -m 6144 -enable-kvm -hda
> 
>> /home/jshi19/workspace/simics5-project/images/luv-
> 
>> v2.1_diskboot_gpt_x86_64_.img -machine pc-q35-2.9 -bios OVMF.fd -serial
> 
>> file:serial.log
> 
>>> 
> 
>>> LowerMemorySize= 0x80000000 //2GB, but should not it be 4GB?
> 
>>> 
> 
>>> UpperMemorySize= 0x0                 // 0GB, but should not it be 2GB?
> 
>> 
> 
>> With 6G DRAM, 2G go low, and 4G go high.
> 
>> 
> 
>> (Your output is misleading because you used the %x format specifier in
> 
>> your debug message -- %Lx would be correct, for printing a UINT64 value.)
> 
>> 
> 
>> Laszlo
> _______________________________________________
> edk2-devel mailing list
> edk2-devel@lists.01.org
> https://lists.01.org/mailman/listinfo/edk2-devel