Re: [edk2-devel] [PATCH 4/4] OvmfPkg/PlatformPei: fix MTRR for low-RAM sizes that have many bits clear

["Laszlo Ersek" <lersek@redhat.com>]

Hi Phil,

On 05/08/19 09:33, Philippe Mathieu-Daudé wrote:
> On 5/4/19 2:07 AM, Laszlo Ersek wrote:
>> Assume that we boot OVMF in a QEMU guest with 1025 MB of RAM. The
>> following assertion will fire:
>>
>>> ASSERT_EFI_ERROR (Status = Out of Resources)
>>> ASSERT OvmfPkg/PlatformPei/MemDetect.c(696): !EFI_ERROR (Status)
>>
>> That's because the range [1025 MB, 4 GB) that we try to mark as
>> uncacheable with MTRRs has size 3071 MB:
>>
>>    0x1_0000_0000
>>   -0x0_4010_0000
>>   --------------
>>    0x0_BFF0_0000
>>
>> The integer that stands for the uncacheable area size has 11 (eleven) bits
>> set to 1. As a result, covering this size requires 11 variable MTRRs (each
>> MTRR must cover a naturally aligned, power-of-two sized area). But, if we
>> need more variable MTRRs than the CPU can muster (such as 8), then
>> MtrrSetMemoryAttribute() fails, and we refuse to continue booting (which
>> is justified, in itself).
>>
>> Unfortunately, this is not difficult to trigger, and the error message is
>> well-hidden from end-users, in the OVMF debug log. The following
>> mitigation is inspired by SeaBIOS:
>>
>> Truncate the uncacheable area size to a power-of-two, while keeping the
>> end fixed at 4 GB. Such an interval can be covered by just one variable
>> MTRR.
>>
>> This may leave such an MMIO gap, between the end of low-RAM and the start
>> of the uncacheable area, that is marked as WB (through the MTRR default).
>> Raise the base of the 32-bit PCI MMIO aperture accordingly -- the gap will
>> not be used for anything.
> 
> I had to draw it to be sure I understood correctly:
> 
> +-------------+               +-------------+  <-- 4GB
> |             |               |             |
> |             |               |             |
> |             |               |  PCI MMIO   |
> |             |               |             |
> |             |               | uncacheable |
> | uncacheable |               |             |
> |             |               |             |
> |             |       ---->   +-------------+  <-- mQemuUc32Base
> |             |      |        |             |      (pow2 aligned)
> |             |      |        |     GAP     |
> |             |      |        | (cacheable) |
> +-------------+  ----         +-------------+  <-- TopOfLowRam
> |             |               |             |    (not pow2 aligned)
> |             |               |             |
> |             |               |             |
> |             |               |             |
> | LowerMemory |               | LowerMemory |
> | (cacheable) |               | (cacheable) |
> |             |               |             |
> |             |               |             |
> |             |               |             |
> +-------------+               +-------------+

Correct. "mQemuUc32Base" is not itself a whole power of two, but it is
pow2 aligned, where the alignment must not be smaller than "size" *is*.

This natural alignment is ensured because 4GB is itself a power of two
(2^32). Thus, if "size" is 2^m, then

  mQemuUc32Base == (4GB - size) == (2^32 - 2^m) == 2^m * (2^(32-m) - 1)

and that is divisible by 2^m. (We know for sure that (m < 32).)
Therefore the natural alignment for the base is satisfied.

For example, consider base=3GB, size=1GB. Then m=30.

> 
>> On Q35, the minimal 32-bit PCI MMIO aperture (triggered by RAM size 2815
>> MB) shrinks from
>>
>>   0xE000_0000 - 0xAFF0_0000 = 769 MB
>>
>> to
>>
>>   0xE000_0000 - 0xC000_0000 = 512 MB
>>
>> On i440fx, the minimal 32-bit PCI MMIO aperture (triggered by RAM size
>> 3583 MB) shrinks from
>>
>>   0xFC00_0000 - 0xDFF0_0000 = 449 MB
>>
>> to
>>
>>   0xFC00_0000 - 0xE000_0000 = 448 MB
>>
>> Cc: Ard Biesheuvel <ard.biesheuvel@linaro.org>
>> Cc: Gerd Hoffmann <kraxel@redhat.com>
>> Cc: Jordan Justen <jordan.l.justen@intel.com>
>> Ref: https://bugzilla.redhat.com/show_bug.cgi?id=1666941
>> Ref: https://bugzilla.redhat.com/show_bug.cgi?id=1701710
>> Signed-off-by: Laszlo Ersek <lersek@redhat.com>
>> ---
>>  OvmfPkg/PlatformPei/Platform.h  |  2 ++
>>  OvmfPkg/PlatformPei/MemDetect.c | 23 +++++++++++++++++---
>>  OvmfPkg/PlatformPei/Platform.c  |  4 +---
>>  3 files changed, 23 insertions(+), 6 deletions(-)
>>
>> diff --git a/OvmfPkg/PlatformPei/Platform.h b/OvmfPkg/PlatformPei/Platform.h
>> index 81af8b71480f..4476ddd871cd 100644
>> --- a/OvmfPkg/PlatformPei/Platform.h
>> +++ b/OvmfPkg/PlatformPei/Platform.h
>> @@ -114,4 +114,6 @@ extern UINT32 mMaxCpuCount;
>>  
>>  extern UINT16 mHostBridgeDevId;
>>  
>> +extern UINT32 mQemuUc32Base;
>> +
>>  #endif // _PLATFORM_PEI_H_INCLUDED_
>> diff --git a/OvmfPkg/PlatformPei/MemDetect.c b/OvmfPkg/PlatformPei/MemDetect.c
>> index e890e36408a6..ae73c63d27d5 100644
>> --- a/OvmfPkg/PlatformPei/MemDetect.c
>> +++ b/OvmfPkg/PlatformPei/MemDetect.c
>> @@ -42,6 +42,8 @@ STATIC UINT32 mS3AcpiReservedMemorySize;
>>  
>>  STATIC UINT16 mQ35TsegMbytes;
>>  
>> +UINT32 mQemuUc32Base;
>> +
>>  VOID
>>  Q35TsegMbytesInitialization (
>>    VOID
>> @@ -663,6 +665,8 @@ QemuInitializeRam (
>>    // cover it exactly.
>>    //
>>    if (IsMtrrSupported ()) {
>> +    UINT32 Uc32Size;
>> +
>>      MtrrGetAllMtrrs (&MtrrSettings);
>>  
>>      //
>> @@ -689,11 +693,24 @@ QemuInitializeRam (
>>  
>>      //
>>      // Set memory range from the "top of lower RAM" (RAM below 4GB) to 4GB as
>> -    // uncacheable
>> +    // uncacheable. Make sure one variable MTRR suffices by truncating the size
>> +    // to a whole power of two. This will round the base *up*, and a gap (not
>> +    // used for either RAM or MMIO) may stay in the middle, marked as
>> +    // cacheable-by-default.
>>      //
>> -    Status = MtrrSetMemoryAttribute (LowerMemorySize,
>> -               SIZE_4GB - LowerMemorySize, CacheUncacheable);
>> +    Uc32Size = GetPowerOfTwo32 ((UINT32)(SIZE_4GB - LowerMemorySize));
>> +    mQemuUc32Base = (UINT32)(SIZE_4GB - Uc32Size);
>> +    if (mQemuUc32Base != LowerMemorySize) {
>> +      DEBUG ((DEBUG_VERBOSE, "%a: rounded UC32 base from 0x%x up to 0x%x, for "
>> +        "an UC32 size of 0x%x\n", __FUNCTION__, (UINT32)LowerMemorySize,
>> +        mQemuUc32Base, Uc32Size));
>> +    }
>> +
>> +    Status = MtrrSetMemoryAttribute (mQemuUc32Base, Uc32Size,
>> +               CacheUncacheable);
>>      ASSERT_EFI_ERROR (Status);
>> +  } else {
>> +    mQemuUc32Base = (UINT32)LowerMemorySize;
>>    }
>>  }
>>  
>> diff --git a/OvmfPkg/PlatformPei/Platform.c b/OvmfPkg/PlatformPei/Platform.c
>> index fd8eccaf3e50..c064b4ed9b8f 100644
>> --- a/OvmfPkg/PlatformPei/Platform.c
>> +++ b/OvmfPkg/PlatformPei/Platform.c
>> @@ -174,14 +174,12 @@ MemMapInitialization (
>>    AddIoMemoryRangeHob (0x0A0000, BASE_1MB);
>>  
>>    if (!mXen) {
>> -    UINT32  TopOfLowRam;
>>      UINT64  PciExBarBase;
>>      UINT32  PciBase;
>>      UINT32  PciSize;
>>  
>> -    TopOfLowRam = GetSystemMemorySizeBelow4gb ();
>>      PciExBarBase = 0;
>> -    PciBase = (TopOfLowRam < BASE_2GB) ? BASE_2GB : TopOfLowRam;
>> +    PciBase = (mQemuUc32Base < BASE_2GB) ? BASE_2GB : mQemuUc32Base;
>>      if (mHostBridgeDevId == INTEL_Q35_MCH_DEVICE_ID) {
>>        //
>>        // The 32-bit PCI host aperture is expected to fall between the top of
>>
> 
> Reviewed-by: Philippe Mathieu-Daude <philmd@redhat.com>
> 

Thank you!
Laszlo