[PATCH v3 0/3] OvmfPkg/PlatformInitLib: move mmconfig to 0xe0000000

[PATCH v3 0/3] OvmfPkg/PlatformInitLib: move mmconfig to 0xe0000000

[Gerd Hoffmann]

v3:
 - drop OvmfXen.dsc changes (Anthony).
 - add rb tags.
 - rebase to latest master.
v2:
 - rebase to latest master.

Gerd Hoffmann (3):
  OvmfPkg/PlatformInitLib: update address space layout comment
  OvmfPkg/PlatformInitLib: move mmconfig to 0xe0000000
  OvmfPkg/PlatformInitLib: simplify mtrr setup

 OvmfPkg/AmdSev/AmdSevX64.dsc                |  2 +-
 OvmfPkg/IntelTdx/IntelTdxX64.dsc            |  2 +-
 OvmfPkg/OvmfPkgIa32.dsc                     |  2 +-
 OvmfPkg/OvmfPkgIa32X64.dsc                  |  2 +-
 OvmfPkg/OvmfPkgX64.dsc                      |  2 +-
 OvmfPkg/Library/PlatformInitLib/MemDetect.c | 36 ++++++++-----------
 OvmfPkg/Library/PlatformInitLib/Platform.c  | 38 +++++++++++----------
 7 files changed, 40 insertions(+), 44 deletions(-)

-- 
2.39.2

[PATCH v3 1/3] OvmfPkg/PlatformInitLib: update address space layout comment

[Gerd Hoffmann]

Move the commment up so it is placed just before the address space
calculations start.  Also add q35 memory layout.

Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
Reviewed-by: Anthony PERARD <anthony.perard@citrix.com>
---
 OvmfPkg/Library/PlatformInitLib/Platform.c | 36 ++++++++++++----------
 1 file changed, 19 insertions(+), 17 deletions(-)

diff --git a/OvmfPkg/Library/PlatformInitLib/Platform.c b/OvmfPkg/Library/PlatformInitLib/Platform.c
index 9fee6e481038..678e8e329023 100644
--- a/OvmfPkg/Library/PlatformInitLib/Platform.c
+++ b/OvmfPkg/Library/PlatformInitLib/Platform.c
@@ -152,26 +152,12 @@ PlatformMemMapInitialization (
     return;
   }
 
-  PlatformGetSystemMemorySizeBelow4gb (PlatformInfoHob);
-  PciExBarBase = 0;
-  if (PlatformInfoHob->HostBridgeDevId == INTEL_Q35_MCH_DEVICE_ID) {
-    //
-    // The MMCONFIG area is expected to fall between the top of low RAM and
-    // the base of the 32-bit PCI host aperture.
-    //
-    PciExBarBase = PcdGet64 (PcdPciExpressBaseAddress);
-    ASSERT (PlatformInfoHob->LowMemory <= PciExBarBase);
-    ASSERT (PciExBarBase <= MAX_UINT32 - SIZE_256MB);
-    PciBase = (UINT32)(PciExBarBase + SIZE_256MB);
-  } else {
-    ASSERT (PlatformInfoHob->LowMemory <= PlatformInfoHob->Uc32Base);
-    PciBase = PlatformInfoHob->Uc32Base;
-  }
-
   //
   // address       purpose   size
   // ------------  --------  -------------------------
-  // max(top, 2g)  PCI MMIO  0xFC000000 - max(top, 2g)
+  // max(top, 2g)  PCI MMIO  0xFC000000 - max(top, 2g)  (pc)
+  // 0xB0000000    MMCONFIG                     256 MB  (q35)
+  // 0xC0000000    PCI MMIO                     960 MB  (q35)
   // 0xFC000000    gap                           44 MB
   // 0xFEC00000    IO-APIC                        4 KB
   // 0xFEC01000    gap                         1020 KB
@@ -181,6 +167,22 @@ PlatformMemMapInitialization (
   // 0xFED20000    gap                          896 KB
   // 0xFEE00000    LAPIC                          1 MB
   //
+  PlatformGetSystemMemorySizeBelow4gb (PlatformInfoHob);
+  PciExBarBase = 0;
+  if (PlatformInfoHob->HostBridgeDevId == INTEL_Q35_MCH_DEVICE_ID) {
+    //
+    // The MMCONFIG area is expected to fall between the top of low RAM and
+    // the base of the 32-bit PCI host aperture.
+    //
+    PciExBarBase = PcdGet64 (PcdPciExpressBaseAddress);
+    ASSERT (PlatformInfoHob->LowMemory <= PciExBarBase);
+    ASSERT (PciExBarBase <= MAX_UINT32 - SIZE_256MB);
+    PciBase = (UINT32)(PciExBarBase + SIZE_256MB);
+  } else {
+    ASSERT (PlatformInfoHob->LowMemory <= PlatformInfoHob->Uc32Base);
+    PciBase = PlatformInfoHob->Uc32Base;
+  }
+
   PciSize = 0xFC000000 - PciBase;
   PlatformAddIoMemoryBaseSizeHob (PciBase, PciSize);
 
-- 
2.39.2

[PATCH v3 2/3] OvmfPkg/PlatformInitLib: move mmconfig to 0xe0000000

[Gerd Hoffmann]

Also swap the ordering of 32bit PCI MMIO window on q35, i.e. use the
room between end of low memory and the start of the mmconfig bar.

With a typical configuration on modern qemu with gigabyte-aligned memory
the MMIO window start at 0x8000000, sized 1532 MB.  In case there is
memory present above 0x80000000 the window will start at 0xc0000000
instead, with 512 MB size.

This depends on qemu commit 4a4418369d6d ("q35: fix mmconfig and
PCI0._CRS"), so it raises the bar for the lowest supported version
to qemu 4.1 (released Aug 2019).

Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
Reviewed-by: Anthony PERARD <anthony.perard@citrix.com>
---
 OvmfPkg/AmdSev/AmdSevX64.dsc               |  2 +-
 OvmfPkg/IntelTdx/IntelTdxX64.dsc           |  2 +-
 OvmfPkg/OvmfPkgIa32.dsc                    |  2 +-
 OvmfPkg/OvmfPkgIa32X64.dsc                 |  2 +-
 OvmfPkg/OvmfPkgX64.dsc                     |  2 +-
 OvmfPkg/Library/PlatformInitLib/Platform.c | 10 +++++-----
 6 files changed, 10 insertions(+), 10 deletions(-)

diff --git a/OvmfPkg/AmdSev/AmdSevX64.dsc b/OvmfPkg/AmdSev/AmdSevX64.dsc
index 654c3af0928b..c005e474dd82 100644
--- a/OvmfPkg/AmdSev/AmdSevX64.dsc
+++ b/OvmfPkg/AmdSev/AmdSevX64.dsc
@@ -443,7 +443,7 @@ [PcdsFixedAtBuild]
   #
   # On Q35 machine types that QEMU intends to support in the long term, QEMU
   # never lets the RAM below 4 GB exceed 2816 MB.
-  gEfiMdePkgTokenSpaceGuid.PcdPciExpressBaseAddress|0xB0000000
+  gEfiMdePkgTokenSpaceGuid.PcdPciExpressBaseAddress|0xE0000000
 
 !if $(SOURCE_DEBUG_ENABLE) == TRUE
   gEfiSourceLevelDebugPkgTokenSpaceGuid.PcdDebugLoadImageMethod|0x2
diff --git a/OvmfPkg/IntelTdx/IntelTdxX64.dsc b/OvmfPkg/IntelTdx/IntelTdxX64.dsc
index e6d85286e002..5c56858d063b 100644
--- a/OvmfPkg/IntelTdx/IntelTdxX64.dsc
+++ b/OvmfPkg/IntelTdx/IntelTdxX64.dsc
@@ -448,7 +448,7 @@ [PcdsFixedAtBuild]
   #
   # On Q35 machine types that QEMU intends to support in the long term, QEMU
   # never lets the RAM below 4 GB exceed 2816 MB.
-  gEfiMdePkgTokenSpaceGuid.PcdPciExpressBaseAddress|0xB0000000
+  gEfiMdePkgTokenSpaceGuid.PcdPciExpressBaseAddress|0xE0000000
 
   #
   # The NumberOfPages values below are ad-hoc. They are updated sporadically at
diff --git a/OvmfPkg/OvmfPkgIa32.dsc b/OvmfPkg/OvmfPkgIa32.dsc
index 9e4705f29b46..a6db902f54ec 100644
--- a/OvmfPkg/OvmfPkgIa32.dsc
+++ b/OvmfPkg/OvmfPkgIa32.dsc
@@ -559,7 +559,7 @@ [PcdsFixedAtBuild]
   #
   # On Q35 machine types that QEMU intends to support in the long term, QEMU
   # never lets the RAM below 4 GB exceed 2816 MB.
-  gEfiMdePkgTokenSpaceGuid.PcdPciExpressBaseAddress|0xB0000000
+  gEfiMdePkgTokenSpaceGuid.PcdPciExpressBaseAddress|0xE0000000
 
 !if $(SOURCE_DEBUG_ENABLE) == TRUE
   gEfiSourceLevelDebugPkgTokenSpaceGuid.PcdDebugLoadImageMethod|0x2
diff --git a/OvmfPkg/OvmfPkgIa32X64.dsc b/OvmfPkg/OvmfPkgIa32X64.dsc
index 86940e9af54c..076fc0353de0 100644
--- a/OvmfPkg/OvmfPkgIa32X64.dsc
+++ b/OvmfPkg/OvmfPkgIa32X64.dsc
@@ -565,7 +565,7 @@ [PcdsFixedAtBuild]
   #
   # On Q35 machine types that QEMU intends to support in the long term, QEMU
   # never lets the RAM below 4 GB exceed 2816 MB.
-  gEfiMdePkgTokenSpaceGuid.PcdPciExpressBaseAddress|0xB0000000
+  gEfiMdePkgTokenSpaceGuid.PcdPciExpressBaseAddress|0xE0000000
 
 !if $(SOURCE_DEBUG_ENABLE) == TRUE
   gEfiSourceLevelDebugPkgTokenSpaceGuid.PcdDebugLoadImageMethod|0x2
diff --git a/OvmfPkg/OvmfPkgX64.dsc b/OvmfPkg/OvmfPkgX64.dsc
index 00f7005fca40..4a276e1ec0d9 100644
--- a/OvmfPkg/OvmfPkgX64.dsc
+++ b/OvmfPkg/OvmfPkgX64.dsc
@@ -586,7 +586,7 @@ [PcdsFixedAtBuild]
   #
   # On Q35 machine types that QEMU intends to support in the long term, QEMU
   # never lets the RAM below 4 GB exceed 2816 MB.
-  gEfiMdePkgTokenSpaceGuid.PcdPciExpressBaseAddress|0xB0000000
+  gEfiMdePkgTokenSpaceGuid.PcdPciExpressBaseAddress|0xE0000000
 
 !if $(SOURCE_DEBUG_ENABLE) == TRUE
   gEfiSourceLevelDebugPkgTokenSpaceGuid.PcdDebugLoadImageMethod|0x2
diff --git a/OvmfPkg/Library/PlatformInitLib/Platform.c b/OvmfPkg/Library/PlatformInitLib/Platform.c
index 678e8e329023..5cf8af825a2f 100644
--- a/OvmfPkg/Library/PlatformInitLib/Platform.c
+++ b/OvmfPkg/Library/PlatformInitLib/Platform.c
@@ -156,8 +156,8 @@ PlatformMemMapInitialization (
   // address       purpose   size
   // ------------  --------  -------------------------
   // max(top, 2g)  PCI MMIO  0xFC000000 - max(top, 2g)  (pc)
-  // 0xB0000000    MMCONFIG                     256 MB  (q35)
-  // 0xC0000000    PCI MMIO                     960 MB  (q35)
+  // max(top, 2g)  PCI MMIO  0xE0000000 - max(top, 2g)  (q35)
+  // 0xE0000000    MMCONFIG                     256 MB  (q35)
   // 0xFC000000    gap                           44 MB
   // 0xFEC00000    IO-APIC                        4 KB
   // 0xFEC01000    gap                         1020 KB
@@ -168,6 +168,7 @@ PlatformMemMapInitialization (
   // 0xFEE00000    LAPIC                          1 MB
   //
   PlatformGetSystemMemorySizeBelow4gb (PlatformInfoHob);
+  PciBase      = PlatformInfoHob->Uc32Base;
   PciExBarBase = 0;
   if (PlatformInfoHob->HostBridgeDevId == INTEL_Q35_MCH_DEVICE_ID) {
     //
@@ -177,13 +178,12 @@ PlatformMemMapInitialization (
     PciExBarBase = PcdGet64 (PcdPciExpressBaseAddress);
     ASSERT (PlatformInfoHob->LowMemory <= PciExBarBase);
     ASSERT (PciExBarBase <= MAX_UINT32 - SIZE_256MB);
-    PciBase = (UINT32)(PciExBarBase + SIZE_256MB);
+    PciSize = (UINT32)(PciExBarBase - PciBase);
   } else {
     ASSERT (PlatformInfoHob->LowMemory <= PlatformInfoHob->Uc32Base);
-    PciBase = PlatformInfoHob->Uc32Base;
+    PciSize = 0xFC000000 - PciBase;
   }
 
-  PciSize = 0xFC000000 - PciBase;
   PlatformAddIoMemoryBaseSizeHob (PciBase, PciSize);
 
   PlatformInfoHob->PcdPciMmio32Base = PciBase;
-- 
2.39.2

[PATCH v3 3/3] OvmfPkg/PlatformInitLib: simplify mtrr setup

[Gerd Hoffmann]

With the new mmconfig location at 0xe0000000 above the 32-bit PCI MMIO
window we don't have to special-case the mmconfig xbar any more.  We'll
just add a mtrr uncachable entry starting at MMIO window base and ending
at 4GB.

Update comments to match reality.

Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
Reviewed-by: Anthony PERARD <anthony.perard@citrix.com>
---
 OvmfPkg/Library/PlatformInitLib/MemDetect.c | 36 +++++++++------------
 1 file changed, 15 insertions(+), 21 deletions(-)

diff --git a/OvmfPkg/Library/PlatformInitLib/MemDetect.c b/OvmfPkg/Library/PlatformInitLib/MemDetect.c
index 38cece9173e8..f85a63ac5130 100644
--- a/OvmfPkg/Library/PlatformInitLib/MemDetect.c
+++ b/OvmfPkg/Library/PlatformInitLib/MemDetect.c
@@ -61,33 +61,20 @@ PlatformQemuUc32BaseInitialization (
     return;
   }
 
+  ASSERT (
+    PlatformInfoHob->HostBridgeDevId == INTEL_Q35_MCH_DEVICE_ID ||
+    PlatformInfoHob->HostBridgeDevId == INTEL_82441_DEVICE_ID
+    );
+
   PlatformGetSystemMemorySizeBelow4gb (PlatformInfoHob);
 
   if (PlatformInfoHob->HostBridgeDevId == INTEL_Q35_MCH_DEVICE_ID) {
     ASSERT (PcdGet64 (PcdPciExpressBaseAddress) <= MAX_UINT32);
     ASSERT (PcdGet64 (PcdPciExpressBaseAddress) >= PlatformInfoHob->LowMemory);
-
-    if (PlatformInfoHob->LowMemory <= BASE_2GB) {
-      // Newer qemu with gigabyte aligned memory,
-      // 32-bit pci mmio window is 2G -> 4G then.
-      PlatformInfoHob->Uc32Base = BASE_2GB;
-    } else {
-      //
-      // On q35, the 32-bit area that we'll mark as UC, through variable MTRRs,
-      // starts at PcdPciExpressBaseAddress. The platform DSC is responsible for
-      // setting PcdPciExpressBaseAddress such that describing the
-      // [PcdPciExpressBaseAddress, 4GB) range require a very small number of
-      // variable MTRRs (preferably 1 or 2).
-      //
-      PlatformInfoHob->Uc32Base = (UINT32)PcdGet64 (PcdPciExpressBaseAddress);
-    }
-
-    return;
   }
 
-  ASSERT (PlatformInfoHob->HostBridgeDevId == INTEL_82441_DEVICE_ID);
   //
-  // On i440fx, start with the [LowerMemorySize, 4GB) range. Make sure one
+  // Start with the [LowerMemorySize, 4GB) range. Make sure one
   // variable MTRR suffices by truncating the size to a whole power of two,
   // while keeping the end affixed to 4GB. This will round the base up.
   //
@@ -1027,6 +1014,13 @@ PlatformQemuInitializeRam (
   // practically any alignment, and we may not have enough variable MTRRs to
   // cover it exactly.
   //
+  // Because of that PlatformQemuUc32BaseInitialization() will round
+  // up PlatformInfoHob->LowMemory to make sure a single mtrr register
+  // is enough.  The the result will be stored in
+  // PlatformInfoHob->Uc32Base.  On a typical qemu configuration with
+  // gigabyte-alignment being used LowMemory will be 2 or 3 GB and no
+  // rounding is needed, so LowMemory and Uc32Base will be identical.
+  //
   if (IsMtrrSupported () && (PlatformInfoHob->HostBridgeDevId != CLOUDHV_DEVICE_ID)) {
     MtrrGetAllMtrrs (&MtrrSettings);
 
@@ -1056,8 +1050,8 @@ PlatformQemuInitializeRam (
     ASSERT_EFI_ERROR (Status);
 
     //
-    // Set the memory range from the start of the 32-bit MMIO area (32-bit PCI
-    // MMIO aperture on i440fx, PCIEXBAR on q35) to 4GB as uncacheable.
+    // Set the memory range from the start of the 32-bit PCI MMIO
+    // aperture to 4GB as uncacheable.
     //
     Status = MtrrSetMemoryAttribute (
                PlatformInfoHob->Uc32Base,
-- 
2.39.2

Re: [PATCH v3 0/3] OvmfPkg/PlatformInitLib: move mmconfig to 0xe0000000

[Ard Biesheuvel]

On Tue, 28 Mar 2023 at 09:34, Gerd Hoffmann <kraxel@redhat.com> wrote:
>
> v3:
>  - drop OvmfXen.dsc changes (Anthony).
>  - add rb tags.
>  - rebase to latest master.
> v2:
>  - rebase to latest master.
>
> Gerd Hoffmann (3):
>   OvmfPkg/PlatformInitLib: update address space layout comment
>   OvmfPkg/PlatformInitLib: move mmconfig to 0xe0000000
>   OvmfPkg/PlatformInitLib: simplify mtrr setup
>

Merged as #4195 - thanks.

>  OvmfPkg/AmdSev/AmdSevX64.dsc                |  2 +-
>  OvmfPkg/IntelTdx/IntelTdxX64.dsc            |  2 +-
>  OvmfPkg/OvmfPkgIa32.dsc                     |  2 +-
>  OvmfPkg/OvmfPkgIa32X64.dsc                  |  2 +-
>  OvmfPkg/OvmfPkgX64.dsc                      |  2 +-
>  OvmfPkg/Library/PlatformInitLib/MemDetect.c | 36 ++++++++-----------
>  OvmfPkg/Library/PlatformInitLib/Platform.c  | 38 +++++++++++----------
>  7 files changed, 40 insertions(+), 44 deletions(-)
>
> --
> 2.39.2
>