[PATCH v2 0/3] OvmfPkg/PlatformInitLib: move mmconfig to 0xe0000000

[PATCH v2 0/3] OvmfPkg/PlatformInitLib: move mmconfig to 0xe0000000

[Gerd Hoffmann]

v2:
 - rebase to latest master.

Gerd Hoffmann (3):
  OvmfPkg/PlatformInitLib: update address space layout comment
  OvmfPkg/PlatformInitLib: move mmconfig to 0xe0000000
  OvmfPkg/PlatformInitLib: simplify mtrr setup

 OvmfPkg/AmdSev/AmdSevX64.dsc                |  2 +-
 OvmfPkg/IntelTdx/IntelTdxX64.dsc            |  2 +-
 OvmfPkg/OvmfPkgIa32.dsc                     |  2 +-
 OvmfPkg/OvmfPkgIa32X64.dsc                  |  2 +-
 OvmfPkg/OvmfPkgX64.dsc                      |  2 +-
 OvmfPkg/OvmfXen.dsc                         |  2 +-
 OvmfPkg/Library/PlatformInitLib/MemDetect.c | 36 ++++++++-----------
 OvmfPkg/Library/PlatformInitLib/Platform.c  | 38 +++++++++++----------
 8 files changed, 41 insertions(+), 45 deletions(-)

-- 
2.39.2

[PATCH v2 1/3] OvmfPkg/PlatformInitLib: update address space layout comment

[Gerd Hoffmann]

Move the commment up so it is placed just before the address space
calculations start.  Also add q35 memory layout.

Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
---
 OvmfPkg/Library/PlatformInitLib/Platform.c | 36 ++++++++++++----------
 1 file changed, 19 insertions(+), 17 deletions(-)

diff --git a/OvmfPkg/Library/PlatformInitLib/Platform.c b/OvmfPkg/Library/PlatformInitLib/Platform.c
index 9fee6e481038..678e8e329023 100644
--- a/OvmfPkg/Library/PlatformInitLib/Platform.c
+++ b/OvmfPkg/Library/PlatformInitLib/Platform.c
@@ -152,26 +152,12 @@ PlatformMemMapInitialization (
     return;
   }
 
-  PlatformGetSystemMemorySizeBelow4gb (PlatformInfoHob);
-  PciExBarBase = 0;
-  if (PlatformInfoHob->HostBridgeDevId == INTEL_Q35_MCH_DEVICE_ID) {
-    //
-    // The MMCONFIG area is expected to fall between the top of low RAM and
-    // the base of the 32-bit PCI host aperture.
-    //
-    PciExBarBase = PcdGet64 (PcdPciExpressBaseAddress);
-    ASSERT (PlatformInfoHob->LowMemory <= PciExBarBase);
-    ASSERT (PciExBarBase <= MAX_UINT32 - SIZE_256MB);
-    PciBase = (UINT32)(PciExBarBase + SIZE_256MB);
-  } else {
-    ASSERT (PlatformInfoHob->LowMemory <= PlatformInfoHob->Uc32Base);
-    PciBase = PlatformInfoHob->Uc32Base;
-  }
-
   //
   // address       purpose   size
   // ------------  --------  -------------------------
-  // max(top, 2g)  PCI MMIO  0xFC000000 - max(top, 2g)
+  // max(top, 2g)  PCI MMIO  0xFC000000 - max(top, 2g)  (pc)
+  // 0xB0000000    MMCONFIG                     256 MB  (q35)
+  // 0xC0000000    PCI MMIO                     960 MB  (q35)
   // 0xFC000000    gap                           44 MB
   // 0xFEC00000    IO-APIC                        4 KB
   // 0xFEC01000    gap                         1020 KB
@@ -181,6 +167,22 @@ PlatformMemMapInitialization (
   // 0xFED20000    gap                          896 KB
   // 0xFEE00000    LAPIC                          1 MB
   //
+  PlatformGetSystemMemorySizeBelow4gb (PlatformInfoHob);
+  PciExBarBase = 0;
+  if (PlatformInfoHob->HostBridgeDevId == INTEL_Q35_MCH_DEVICE_ID) {
+    //
+    // The MMCONFIG area is expected to fall between the top of low RAM and
+    // the base of the 32-bit PCI host aperture.
+    //
+    PciExBarBase = PcdGet64 (PcdPciExpressBaseAddress);
+    ASSERT (PlatformInfoHob->LowMemory <= PciExBarBase);
+    ASSERT (PciExBarBase <= MAX_UINT32 - SIZE_256MB);
+    PciBase = (UINT32)(PciExBarBase + SIZE_256MB);
+  } else {
+    ASSERT (PlatformInfoHob->LowMemory <= PlatformInfoHob->Uc32Base);
+    PciBase = PlatformInfoHob->Uc32Base;
+  }
+
   PciSize = 0xFC000000 - PciBase;
   PlatformAddIoMemoryBaseSizeHob (PciBase, PciSize);
 
-- 
2.39.2

Re: [PATCH v2 1/3] OvmfPkg/PlatformInitLib: update address space layout comment

[Anthony PERARD]

On Tue, Mar 07, 2023 at 07:54:52AM +0100, Gerd Hoffmann wrote:
> Move the commment up so it is placed just before the address space
> calculations start.  Also add q35 memory layout.
> 
> Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>

Reviewed-by: Anthony PERARD <anthony.perard@citrix.com>

Thanks,

-- 
Anthony PERARD

[PATCH v2 2/3] OvmfPkg/PlatformInitLib: move mmconfig to 0xe0000000

[Gerd Hoffmann]

Also swap the ordering of 32bit PCI MMIO window on q35, i.e. use the
room between end of low memory and the start of the mmconfig bar.

With a typical configuration on modern qemu with gigabyte-aligned memory
the MMIO window start at 0x8000000, sized 1532 MB.  In case there is
memory present above 0x80000000 the window will start at 0xc0000000
instead, with 512 MB size.

This depends on qemu commit 4a4418369d6d ("q35: fix mmconfig and
PCI0._CRS"), so it raises the bar for the lowest supported version
to qemu 4.1 (released Aug 2019).

Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
---
 OvmfPkg/AmdSev/AmdSevX64.dsc               |  2 +-
 OvmfPkg/IntelTdx/IntelTdxX64.dsc           |  2 +-
 OvmfPkg/OvmfPkgIa32.dsc                    |  2 +-
 OvmfPkg/OvmfPkgIa32X64.dsc                 |  2 +-
 OvmfPkg/OvmfPkgX64.dsc                     |  2 +-
 OvmfPkg/OvmfXen.dsc                        |  2 +-
 OvmfPkg/Library/PlatformInitLib/Platform.c | 10 +++++-----
 7 files changed, 11 insertions(+), 11 deletions(-)

diff --git a/OvmfPkg/AmdSev/AmdSevX64.dsc b/OvmfPkg/AmdSev/AmdSevX64.dsc
index 1cebd6b4bcc2..e8481ba8cdf3 100644
--- a/OvmfPkg/AmdSev/AmdSevX64.dsc
+++ b/OvmfPkg/AmdSev/AmdSevX64.dsc
@@ -443,7 +443,7 @@ [PcdsFixedAtBuild]
   #
   # On Q35 machine types that QEMU intends to support in the long term, QEMU
   # never lets the RAM below 4 GB exceed 2816 MB.
-  gEfiMdePkgTokenSpaceGuid.PcdPciExpressBaseAddress|0xB0000000
+  gEfiMdePkgTokenSpaceGuid.PcdPciExpressBaseAddress|0xE0000000
 
 !if $(SOURCE_DEBUG_ENABLE) == TRUE
   gEfiSourceLevelDebugPkgTokenSpaceGuid.PcdDebugLoadImageMethod|0x2
diff --git a/OvmfPkg/IntelTdx/IntelTdxX64.dsc b/OvmfPkg/IntelTdx/IntelTdxX64.dsc
index 95b9594ddce0..6fea54bb0b6b 100644
--- a/OvmfPkg/IntelTdx/IntelTdxX64.dsc
+++ b/OvmfPkg/IntelTdx/IntelTdxX64.dsc
@@ -448,7 +448,7 @@ [PcdsFixedAtBuild]
   #
   # On Q35 machine types that QEMU intends to support in the long term, QEMU
   # never lets the RAM below 4 GB exceed 2816 MB.
-  gEfiMdePkgTokenSpaceGuid.PcdPciExpressBaseAddress|0xB0000000
+  gEfiMdePkgTokenSpaceGuid.PcdPciExpressBaseAddress|0xE0000000
 
   #
   # The NumberOfPages values below are ad-hoc. They are updated sporadically at
diff --git a/OvmfPkg/OvmfPkgIa32.dsc b/OvmfPkg/OvmfPkgIa32.dsc
index 22dc29330d2d..fff3e13151ad 100644
--- a/OvmfPkg/OvmfPkgIa32.dsc
+++ b/OvmfPkg/OvmfPkgIa32.dsc
@@ -560,7 +560,7 @@ [PcdsFixedAtBuild]
   #
   # On Q35 machine types that QEMU intends to support in the long term, QEMU
   # never lets the RAM below 4 GB exceed 2816 MB.
-  gEfiMdePkgTokenSpaceGuid.PcdPciExpressBaseAddress|0xB0000000
+  gEfiMdePkgTokenSpaceGuid.PcdPciExpressBaseAddress|0xE0000000
 
 !if $(SOURCE_DEBUG_ENABLE) == TRUE
   gEfiSourceLevelDebugPkgTokenSpaceGuid.PcdDebugLoadImageMethod|0x2
diff --git a/OvmfPkg/OvmfPkgIa32X64.dsc b/OvmfPkg/OvmfPkgIa32X64.dsc
index 6b539814bdb0..169b9b9c2beb 100644
--- a/OvmfPkg/OvmfPkgIa32X64.dsc
+++ b/OvmfPkg/OvmfPkgIa32X64.dsc
@@ -565,7 +565,7 @@ [PcdsFixedAtBuild]
   #
   # On Q35 machine types that QEMU intends to support in the long term, QEMU
   # never lets the RAM below 4 GB exceed 2816 MB.
-  gEfiMdePkgTokenSpaceGuid.PcdPciExpressBaseAddress|0xB0000000
+  gEfiMdePkgTokenSpaceGuid.PcdPciExpressBaseAddress|0xE0000000
 
 !if $(SOURCE_DEBUG_ENABLE) == TRUE
   gEfiSourceLevelDebugPkgTokenSpaceGuid.PcdDebugLoadImageMethod|0x2
diff --git a/OvmfPkg/OvmfPkgX64.dsc b/OvmfPkg/OvmfPkgX64.dsc
index e3c64456dfef..12b88b71ab7f 100644
--- a/OvmfPkg/OvmfPkgX64.dsc
+++ b/OvmfPkg/OvmfPkgX64.dsc
@@ -586,7 +586,7 @@ [PcdsFixedAtBuild]
   #
   # On Q35 machine types that QEMU intends to support in the long term, QEMU
   # never lets the RAM below 4 GB exceed 2816 MB.
-  gEfiMdePkgTokenSpaceGuid.PcdPciExpressBaseAddress|0xB0000000
+  gEfiMdePkgTokenSpaceGuid.PcdPciExpressBaseAddress|0xE0000000
 
 !if $(SOURCE_DEBUG_ENABLE) == TRUE
   gEfiSourceLevelDebugPkgTokenSpaceGuid.PcdDebugLoadImageMethod|0x2
diff --git a/OvmfPkg/OvmfXen.dsc b/OvmfPkg/OvmfXen.dsc
index c328987e8432..18144d9a6d94 100644
--- a/OvmfPkg/OvmfXen.dsc
+++ b/OvmfPkg/OvmfXen.dsc
@@ -434,7 +434,7 @@ [PcdsFixedAtBuild]
   #
   # On Q35 machine types that QEMU intends to support in the long term, QEMU
   # never lets the RAM below 4 GB exceed 2816 MB.
-  gEfiMdePkgTokenSpaceGuid.PcdPciExpressBaseAddress|0xB0000000
+  gEfiMdePkgTokenSpaceGuid.PcdPciExpressBaseAddress|0xE0000000
 
 !if $(SOURCE_DEBUG_ENABLE) == TRUE
   gEfiSourceLevelDebugPkgTokenSpaceGuid.PcdDebugLoadImageMethod|0x2
diff --git a/OvmfPkg/Library/PlatformInitLib/Platform.c b/OvmfPkg/Library/PlatformInitLib/Platform.c
index 678e8e329023..5cf8af825a2f 100644
--- a/OvmfPkg/Library/PlatformInitLib/Platform.c
+++ b/OvmfPkg/Library/PlatformInitLib/Platform.c
@@ -156,8 +156,8 @@ PlatformMemMapInitialization (
   // address       purpose   size
   // ------------  --------  -------------------------
   // max(top, 2g)  PCI MMIO  0xFC000000 - max(top, 2g)  (pc)
-  // 0xB0000000    MMCONFIG                     256 MB  (q35)
-  // 0xC0000000    PCI MMIO                     960 MB  (q35)
+  // max(top, 2g)  PCI MMIO  0xE0000000 - max(top, 2g)  (q35)
+  // 0xE0000000    MMCONFIG                     256 MB  (q35)
   // 0xFC000000    gap                           44 MB
   // 0xFEC00000    IO-APIC                        4 KB
   // 0xFEC01000    gap                         1020 KB
@@ -168,6 +168,7 @@ PlatformMemMapInitialization (
   // 0xFEE00000    LAPIC                          1 MB
   //
   PlatformGetSystemMemorySizeBelow4gb (PlatformInfoHob);
+  PciBase      = PlatformInfoHob->Uc32Base;
   PciExBarBase = 0;
   if (PlatformInfoHob->HostBridgeDevId == INTEL_Q35_MCH_DEVICE_ID) {
     //
@@ -177,13 +178,12 @@ PlatformMemMapInitialization (
     PciExBarBase = PcdGet64 (PcdPciExpressBaseAddress);
     ASSERT (PlatformInfoHob->LowMemory <= PciExBarBase);
     ASSERT (PciExBarBase <= MAX_UINT32 - SIZE_256MB);
-    PciBase = (UINT32)(PciExBarBase + SIZE_256MB);
+    PciSize = (UINT32)(PciExBarBase - PciBase);
   } else {
     ASSERT (PlatformInfoHob->LowMemory <= PlatformInfoHob->Uc32Base);
-    PciBase = PlatformInfoHob->Uc32Base;
+    PciSize = 0xFC000000 - PciBase;
   }
 
-  PciSize = 0xFC000000 - PciBase;
   PlatformAddIoMemoryBaseSizeHob (PciBase, PciSize);
 
   PlatformInfoHob->PcdPciMmio32Base = PciBase;
-- 
2.39.2

Re: [PATCH v2 2/3] OvmfPkg/PlatformInitLib: move mmconfig to 0xe0000000

[Anthony PERARD]

On Tue, Mar 07, 2023 at 07:54:53AM +0100, Gerd Hoffmann wrote:
> Also swap the ordering of 32bit PCI MMIO window on q35, i.e. use the
> room between end of low memory and the start of the mmconfig bar.
> 
> With a typical configuration on modern qemu with gigabyte-aligned memory
> the MMIO window start at 0x8000000, sized 1532 MB.  In case there is
> memory present above 0x80000000 the window will start at 0xc0000000
> instead, with 512 MB size.
> 
> This depends on qemu commit 4a4418369d6d ("q35: fix mmconfig and
> PCI0._CRS"), so it raises the bar for the lowest supported version
> to qemu 4.1 (released Aug 2019).

Maybe this new version requirement should be noted in OvmfPkg/README,
but I'm not sure were as it affect only machine started with Q35.



> diff --git a/OvmfPkg/OvmfXen.dsc b/OvmfPkg/OvmfXen.dsc
> index c328987e8432..18144d9a6d94 100644
> --- a/OvmfPkg/OvmfXen.dsc
> +++ b/OvmfPkg/OvmfXen.dsc
> @@ -434,7 +434,7 @@ [PcdsFixedAtBuild]
>    #
>    # On Q35 machine types that QEMU intends to support in the long term, QEMU
>    # never lets the RAM below 4 GB exceed 2816 MB.
> -  gEfiMdePkgTokenSpaceGuid.PcdPciExpressBaseAddress|0xB0000000
> +  gEfiMdePkgTokenSpaceGuid.PcdPciExpressBaseAddress|0xE0000000

Could you remove this change? OvmfXen doesn't use PlatformInitLib, and
we still have worked to do to be able to use Q35 in Xen guest.


I think the rest looks fine, so:
Reviewed-by: Anthony PERARD <anthony.perard@citrix.com>

Thanks,

-- 
Anthony PERARD

[PATCH v2 3/3] OvmfPkg/PlatformInitLib: simplify mtrr setup

[Gerd Hoffmann]

With the new mmconfig location at 0xe0000000 above the 32-bit PCI MMIO
window we don't have to special-case the mmconfig xbar any more.  We'll
just add a mtrr uncachable entry starting at MMIO window base and ending
at 4GB.

Update comments to match reality.

Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
---
 OvmfPkg/Library/PlatformInitLib/MemDetect.c | 36 +++++++++------------
 1 file changed, 15 insertions(+), 21 deletions(-)

diff --git a/OvmfPkg/Library/PlatformInitLib/MemDetect.c b/OvmfPkg/Library/PlatformInitLib/MemDetect.c
index 38cece9173e8..f85a63ac5130 100644
--- a/OvmfPkg/Library/PlatformInitLib/MemDetect.c
+++ b/OvmfPkg/Library/PlatformInitLib/MemDetect.c
@@ -61,33 +61,20 @@ PlatformQemuUc32BaseInitialization (
     return;
   }
 
+  ASSERT (
+    PlatformInfoHob->HostBridgeDevId == INTEL_Q35_MCH_DEVICE_ID ||
+    PlatformInfoHob->HostBridgeDevId == INTEL_82441_DEVICE_ID
+    );
+
   PlatformGetSystemMemorySizeBelow4gb (PlatformInfoHob);
 
   if (PlatformInfoHob->HostBridgeDevId == INTEL_Q35_MCH_DEVICE_ID) {
     ASSERT (PcdGet64 (PcdPciExpressBaseAddress) <= MAX_UINT32);
     ASSERT (PcdGet64 (PcdPciExpressBaseAddress) >= PlatformInfoHob->LowMemory);
-
-    if (PlatformInfoHob->LowMemory <= BASE_2GB) {
-      // Newer qemu with gigabyte aligned memory,
-      // 32-bit pci mmio window is 2G -> 4G then.
-      PlatformInfoHob->Uc32Base = BASE_2GB;
-    } else {
-      //
-      // On q35, the 32-bit area that we'll mark as UC, through variable MTRRs,
-      // starts at PcdPciExpressBaseAddress. The platform DSC is responsible for
-      // setting PcdPciExpressBaseAddress such that describing the
-      // [PcdPciExpressBaseAddress, 4GB) range require a very small number of
-      // variable MTRRs (preferably 1 or 2).
-      //
-      PlatformInfoHob->Uc32Base = (UINT32)PcdGet64 (PcdPciExpressBaseAddress);
-    }
-
-    return;
   }
 
-  ASSERT (PlatformInfoHob->HostBridgeDevId == INTEL_82441_DEVICE_ID);
   //
-  // On i440fx, start with the [LowerMemorySize, 4GB) range. Make sure one
+  // Start with the [LowerMemorySize, 4GB) range. Make sure one
   // variable MTRR suffices by truncating the size to a whole power of two,
   // while keeping the end affixed to 4GB. This will round the base up.
   //
@@ -1027,6 +1014,13 @@ PlatformQemuInitializeRam (
   // practically any alignment, and we may not have enough variable MTRRs to
   // cover it exactly.
   //
+  // Because of that PlatformQemuUc32BaseInitialization() will round
+  // up PlatformInfoHob->LowMemory to make sure a single mtrr register
+  // is enough.  The the result will be stored in
+  // PlatformInfoHob->Uc32Base.  On a typical qemu configuration with
+  // gigabyte-alignment being used LowMemory will be 2 or 3 GB and no
+  // rounding is needed, so LowMemory and Uc32Base will be identical.
+  //
   if (IsMtrrSupported () && (PlatformInfoHob->HostBridgeDevId != CLOUDHV_DEVICE_ID)) {
     MtrrGetAllMtrrs (&MtrrSettings);
 
@@ -1056,8 +1050,8 @@ PlatformQemuInitializeRam (
     ASSERT_EFI_ERROR (Status);
 
     //
-    // Set the memory range from the start of the 32-bit MMIO area (32-bit PCI
-    // MMIO aperture on i440fx, PCIEXBAR on q35) to 4GB as uncacheable.
+    // Set the memory range from the start of the 32-bit PCI MMIO
+    // aperture to 4GB as uncacheable.
     //
     Status = MtrrSetMemoryAttribute (
                PlatformInfoHob->Uc32Base,
-- 
2.39.2

Re: [PATCH v2 3/3] OvmfPkg/PlatformInitLib: simplify mtrr setup

[Anthony PERARD]

On Tue, Mar 07, 2023 at 07:54:54AM +0100, Gerd Hoffmann wrote:
> With the new mmconfig location at 0xe0000000 above the 32-bit PCI MMIO
> window we don't have to special-case the mmconfig xbar any more.  We'll
> just add a mtrr uncachable entry starting at MMIO window base and ending
> at 4GB.
> 
> Update comments to match reality.
> 
> Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>

Reviewed-by: Anthony PERARD <anthony.perard@citrix.com>

Thanks,

-- 
Anthony PERARD

Re: [PATCH v2 0/3] OvmfPkg/PlatformInitLib: move mmconfig to 0xe0000000

[Gerd Hoffmann]

On Tue, Mar 07, 2023 at 07:54:51AM +0100, Gerd Hoffmann wrote:
> v2:
>  - rebase to latest master.
> 
> Gerd Hoffmann (3):
>   OvmfPkg/PlatformInitLib: update address space layout comment
>   OvmfPkg/PlatformInitLib: move mmconfig to 0xe0000000
>   OvmfPkg/PlatformInitLib: simplify mtrr setup

Ping.  Any comments on this patch series?

thanks,
  Gerd