[PATCH v6 0/6] OvmfPkg/Microvm/pcie: add pcie support

[PATCH v6 0/6] OvmfPkg/Microvm/pcie: add pcie support

[Gerd Hoffmann]

Needs two little tweaks in PCI code because microvm supports mmio only.
Other than that just wire up the existing code (the PCIe host adapter
used by microvm is the same (virtual) hardware used by the arm/aarch64
virtual machines).

v6:
 - codestyle fix (Abner Chang).

v5:
 - codestyle (uncrustify) fix.

v4:
 - update PciHostBridge check (Abner Chang).

v3:
 - rebase to latest master, adapt to PlatformInitLib.
 - rework PhysMemAddressWidth handling for microvm.

v2:
 - rebase to latest master
 - pick up review tags

Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=3777

Gerd Hoffmann (6):
  MdeModulePkg/PciHostBridge: io range is not mandatory
  OvmfPkg/FdtPciHostBridgeLib: io range is not mandatory
  OvmfPkg/Platform: unfix PcdPciExpressBaseAddress
  OvmfPkg/Microvm/pcie: no vbeshim please
  OvmfPkg/Microvm/pcie: mPhysMemAddressWidth tweak
  OvmfPkg/Microvm/pcie: add pcie support

 OvmfPkg/Microvm/MicrovmX64.dsc                | 40 ++++++++++-------
 .../PlatformInitLib/PlatformInitLib.inf       |  4 +-
 OvmfPkg/PlatformPei/PlatformPei.inf           |  2 +-
 .../Bus/Pci/PciHostBridgeDxe/PciHostBridge.c  |  3 ++
 .../FdtPciHostBridgeLib/FdtPciHostBridgeLib.c | 45 ++++++++++---------
 OvmfPkg/Library/PlatformInitLib/MemDetect.c   | 45 ++++++++++++++++++-
 OvmfPkg/Library/PlatformInitLib/Platform.c    |  4 +-
 OvmfPkg/PlatformPei/Platform.c                |  2 +-
 OvmfPkg/QemuVideoDxe/VbeShim.c                |  2 +
 OvmfPkg/Microvm/README                        |  2 +-
 10 files changed, 104 insertions(+), 45 deletions(-)

-- 
2.35.1

[PATCH v6 1/6] MdeModulePkg/PciHostBridge: io range is not mandatory

[Gerd Hoffmann]

io range is not mandatory according to pcie spec,
so allow bridge configurations without io address
space assigned.

Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
Reviewed-by: Ard Biesheuvel <ardb@kernel.org>
---
 MdeModulePkg/Bus/Pci/PciHostBridgeDxe/PciHostBridge.c | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/MdeModulePkg/Bus/Pci/PciHostBridgeDxe/PciHostBridge.c b/MdeModulePkg/Bus/Pci/PciHostBridgeDxe/PciHostBridge.c
index b20bcd310ad5..712662707931 100644
--- a/MdeModulePkg/Bus/Pci/PciHostBridgeDxe/PciHostBridge.c
+++ b/MdeModulePkg/Bus/Pci/PciHostBridgeDxe/PciHostBridge.c
@@ -1085,6 +1085,9 @@ NotifyPhase (
               RootBridge->ResAllocNode[Index].Base   = BaseAddress;
               RootBridge->ResAllocNode[Index].Status = ResAllocated;
               DEBUG ((DEBUG_INFO, "Success\n"));
+            } else if ((Index == TypeIo) && (RootBridge->Io.Base == MAX_UINT64)) {
+              /* optional on PCIe */
+              DEBUG ((DEBUG_INFO, "PCI Root Bridge does not provide IO Resources.\n"));
             } else {
               ReturnStatus = EFI_OUT_OF_RESOURCES;
               DEBUG ((DEBUG_ERROR, "Out Of Resource!\n"));
-- 
2.35.1

[PATCH v6 2/6] OvmfPkg/FdtPciHostBridgeLib: io range is not mandatory

[Gerd Hoffmann]

io range is not mandatory according to pcie spec,
so allow host bridges without io address space.

Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
---
 .../FdtPciHostBridgeLib/FdtPciHostBridgeLib.c | 45 ++++++++++---------
 1 file changed, 23 insertions(+), 22 deletions(-)

diff --git a/OvmfPkg/Fdt/FdtPciHostBridgeLib/FdtPciHostBridgeLib.c b/OvmfPkg/Fdt/FdtPciHostBridgeLib/FdtPciHostBridgeLib.c
index 98828e0b262b..14b41a533e96 100644
--- a/OvmfPkg/Fdt/FdtPciHostBridgeLib/FdtPciHostBridgeLib.c
+++ b/OvmfPkg/Fdt/FdtPciHostBridgeLib/FdtPciHostBridgeLib.c
@@ -292,13 +292,8 @@ ProcessPciHost (
     }
   }
 
-  if ((*IoSize == 0) || (*Mmio32Size == 0)) {
-    DEBUG ((
-      DEBUG_ERROR,
-      "%a: %a space empty\n",
-      __FUNCTION__,
-      (*IoSize == 0) ? "IO" : "MMIO32"
-      ));
+  if (*Mmio32Size == 0) {
+    DEBUG ((DEBUG_ERROR, "%a: MMIO32 space empty\n", __FUNCTION__));
     return EFI_PROTOCOL_ERROR;
   }
 
@@ -333,13 +328,15 @@ ProcessPciHost (
     return Status;
   }
 
-  //
-  // Map the MMIO window that provides I/O access - the PCI host bridge code
-  // is not aware of this translation and so it will only map the I/O view
-  // in the GCD I/O map.
-  //
-  Status = MapGcdMmioSpace (*IoBase + IoTranslation, *IoSize);
-  ASSERT_EFI_ERROR (Status);
+  if (*IoSize != 0) {
+    //
+    // Map the MMIO window that provides I/O access - the PCI host bridge code
+    // is not aware of this translation and so it will only map the I/O view
+    // in the GCD I/O map.
+    //
+    Status = MapGcdMmioSpace (*IoBase + IoTranslation, *IoSize);
+    ASSERT_EFI_ERROR (Status);
+  }
 
   return Status;
 }
@@ -413,17 +410,21 @@ PciHostBridgeGetRootBridges (
 
   AllocationAttributes = EFI_PCI_HOST_BRIDGE_COMBINE_MEM_PMEM;
 
-  Io.Base   = IoBase;
-  Io.Limit  = IoBase + IoSize - 1;
+  if (IoSize != 0) {
+    Io.Base  = IoBase;
+    Io.Limit = IoBase + IoSize - 1;
+  } else {
+    Io.Base  = MAX_UINT64;
+    Io.Limit = 0;
+  }
+
   Mem.Base  = Mmio32Base;
   Mem.Limit = Mmio32Base + Mmio32Size - 1;
 
-  if (sizeof (UINTN) == sizeof (UINT64)) {
-    MemAbove4G.Base  = Mmio64Base;
-    MemAbove4G.Limit = Mmio64Base + Mmio64Size - 1;
-    if (Mmio64Size > 0) {
-      AllocationAttributes |= EFI_PCI_HOST_BRIDGE_MEM64_DECODE;
-    }
+  if ((sizeof (UINTN) == sizeof (UINT64)) && (Mmio64Size != 0)) {
+    MemAbove4G.Base       = Mmio64Base;
+    MemAbove4G.Limit      = Mmio64Base + Mmio64Size - 1;
+    AllocationAttributes |= EFI_PCI_HOST_BRIDGE_MEM64_DECODE;
   } else {
     //
     // UEFI mandates a 1:1 virtual-to-physical mapping, so on a 32-bit
-- 
2.35.1

[PATCH v6 3/6] OvmfPkg/Platform: unfix PcdPciExpressBaseAddress

[Gerd Hoffmann]

Will be set by FdtPciHostBridgeLib, so it can't be an fixed when we
want use that library.

Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
---
 OvmfPkg/Library/PlatformInitLib/PlatformInitLib.inf | 4 +++-
 OvmfPkg/PlatformPei/PlatformPei.inf                 | 2 +-
 OvmfPkg/Library/PlatformInitLib/MemDetect.c         | 4 ++--
 OvmfPkg/Library/PlatformInitLib/Platform.c          | 4 ++--
 4 files changed, 8 insertions(+), 6 deletions(-)

diff --git a/OvmfPkg/Library/PlatformInitLib/PlatformInitLib.inf b/OvmfPkg/Library/PlatformInitLib/PlatformInitLib.inf
index d0fab5cc1f4f..d2a0bec43452 100644
--- a/OvmfPkg/Library/PlatformInitLib/PlatformInitLib.inf
+++ b/OvmfPkg/Library/PlatformInitLib/PlatformInitLib.inf
@@ -54,8 +54,10 @@ [LibraryClasses]
 [LibraryClasses.X64]
   TdxLib
 
-[FixedPcd]
+[Pcd]
   gEfiMdePkgTokenSpaceGuid.PcdPciExpressBaseAddress
+
+[FixedPcd]
   gUefiOvmfPkgTokenSpaceGuid.PcdOvmfWorkAreaBase
   gUefiOvmfPkgTokenSpaceGuid.PcdOvmfWorkAreaSize
   gUefiOvmfPkgTokenSpaceGuid.PcdOvmfLockBoxStorageBase
diff --git a/OvmfPkg/PlatformPei/PlatformPei.inf b/OvmfPkg/PlatformPei/PlatformPei.inf
index 00372fa0ebb5..3cd83e6ec3e5 100644
--- a/OvmfPkg/PlatformPei/PlatformPei.inf
+++ b/OvmfPkg/PlatformPei/PlatformPei.inf
@@ -95,6 +95,7 @@ [Pcd]
   gUefiOvmfPkgTokenSpaceGuid.PcdXenPvhStartOfDayStructPtr
   gUefiOvmfPkgTokenSpaceGuid.PcdXenPvhStartOfDayStructPtrSize
   gEfiMdePkgTokenSpaceGuid.PcdGuidedExtractHandlerTableAddress
+  gEfiMdePkgTokenSpaceGuid.PcdPciExpressBaseAddress
   gEfiMdeModulePkgTokenSpaceGuid.PcdFlashNvStorageFtwSpareSize
   gEfiMdeModulePkgTokenSpaceGuid.PcdFlashNvStorageVariableSize
   gEfiMdeModulePkgTokenSpaceGuid.PcdEmuVariableNvStoreReserved
@@ -118,7 +119,6 @@ [Pcd]
 [FixedPcd]
   gUefiOvmfPkgTokenSpaceGuid.PcdOvmfCpuidBase
   gUefiOvmfPkgTokenSpaceGuid.PcdOvmfCpuidSize
-  gEfiMdePkgTokenSpaceGuid.PcdPciExpressBaseAddress
   gEmbeddedTokenSpaceGuid.PcdMemoryTypeEfiACPIMemoryNVS
   gEmbeddedTokenSpaceGuid.PcdMemoryTypeEfiACPIReclaimMemory
   gEmbeddedTokenSpaceGuid.PcdMemoryTypeEfiReservedMemoryType
diff --git a/OvmfPkg/Library/PlatformInitLib/MemDetect.c b/OvmfPkg/Library/PlatformInitLib/MemDetect.c
index 4c1dedf863c3..83a7b6726bb7 100644
--- a/OvmfPkg/Library/PlatformInitLib/MemDetect.c
+++ b/OvmfPkg/Library/PlatformInitLib/MemDetect.c
@@ -61,8 +61,8 @@ PlatformQemuUc32BaseInitialization (
     // [PcdPciExpressBaseAddress, 4GB) range require a very small number of
     // variable MTRRs (preferably 1 or 2).
     //
-    ASSERT (FixedPcdGet64 (PcdPciExpressBaseAddress) <= MAX_UINT32);
-    PlatformInfoHob->Uc32Base = (UINT32)FixedPcdGet64 (PcdPciExpressBaseAddress);
+    ASSERT (PcdGet64 (PcdPciExpressBaseAddress) <= MAX_UINT32);
+    PlatformInfoHob->Uc32Base = (UINT32)PcdGet64 (PcdPciExpressBaseAddress);
     return;
   }
 
diff --git a/OvmfPkg/Library/PlatformInitLib/Platform.c b/OvmfPkg/Library/PlatformInitLib/Platform.c
index 101074f6100d..60a30a01f3b5 100644
--- a/OvmfPkg/Library/PlatformInitLib/Platform.c
+++ b/OvmfPkg/Library/PlatformInitLib/Platform.c
@@ -154,7 +154,7 @@ PlatformMemMapInitialization (
     // The MMCONFIG area is expected to fall between the top of low RAM and
     // the base of the 32-bit PCI host aperture.
     //
-    PciExBarBase = FixedPcdGet64 (PcdPciExpressBaseAddress);
+    PciExBarBase = PcdGet64 (PcdPciExpressBaseAddress);
     ASSERT (TopOfLowRam <= PciExBarBase);
     ASSERT (PciExBarBase <= MAX_UINT32 - SIZE_256MB);
     PciBase = (UINT32)(PciExBarBase + SIZE_256MB);
@@ -278,7 +278,7 @@ PciExBarInitialization (
   // determined in AddressWidthInitialization(), i.e., 36 bits, will suffice
   // for DXE's page tables to cover the MMCONFIG area.
   //
-  PciExBarBase.Uint64 = FixedPcdGet64 (PcdPciExpressBaseAddress);
+  PciExBarBase.Uint64 = PcdGet64 (PcdPciExpressBaseAddress);
   ASSERT ((PciExBarBase.Uint32[1] & MCH_PCIEXBAR_HIGHMASK) == 0);
   ASSERT ((PciExBarBase.Uint32[0] & MCH_PCIEXBAR_LOWMASK) == 0);
 
-- 
2.35.1

[PATCH v6 4/6] OvmfPkg/Microvm/pcie: no vbeshim please

[Gerd Hoffmann]

Those old windows versions which need the vbeshim hack
will not run on microvm anyway.

Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
---
 OvmfPkg/QemuVideoDxe/VbeShim.c | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/OvmfPkg/QemuVideoDxe/VbeShim.c b/OvmfPkg/QemuVideoDxe/VbeShim.c
index 8faa146b6cce..2a048211a823 100644
--- a/OvmfPkg/QemuVideoDxe/VbeShim.c
+++ b/OvmfPkg/QemuVideoDxe/VbeShim.c
@@ -156,6 +156,8 @@ InstallVbeShim (
     case INTEL_Q35_MCH_DEVICE_ID:
       Pam1Address = DRAMC_REGISTER_Q35 (MCH_PAM1);
       break;
+    case MICROVM_PSEUDO_DEVICE_ID:
+      return;
     default:
       DEBUG ((
         DEBUG_ERROR,
-- 
2.35.1

[PATCH v6 5/6] OvmfPkg/Microvm/pcie: mPhysMemAddressWidth tweak

[Gerd Hoffmann]

microvm places the 64bit mmio space at the end of the physical address
space.  So mPhysMemAddressWidth must be correct, otherwise the pci host
bridge setup throws an error because it thinks the 64bit mmio window is
not addressable.

On microvm we can simply use standard cpuid to figure the address width
because the host-phys-bits option (-cpu ${name},host-phys-bits=on) is
forced to be enabled.  Side note: For 'pc' and 'q35' this is not the
case for backward compatibility reasons.

Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
---
 OvmfPkg/Library/PlatformInitLib/MemDetect.c | 41 +++++++++++++++++++++
 OvmfPkg/PlatformPei/Platform.c              |  2 +-
 2 files changed, 42 insertions(+), 1 deletion(-)

diff --git a/OvmfPkg/Library/PlatformInitLib/MemDetect.c b/OvmfPkg/Library/PlatformInitLib/MemDetect.c
index 83a7b6726bb7..c28d7601f87e 100644
--- a/OvmfPkg/Library/PlatformInitLib/MemDetect.c
+++ b/OvmfPkg/Library/PlatformInitLib/MemDetect.c
@@ -491,6 +491,42 @@ PlatformGetFirstNonAddress (
   return FirstNonAddress;
 }
 
+/*
+ * Use CPUID to figure physical address width.  Does *not* work
+ * reliable on qemu.  For historical reasons qemu returns phys-bits=40
+ * even in case the host machine supports less than that.
+ *
+ * qemu has a cpu property (host-phys-bits={on,off}) to change that
+ * and make sure guest phys-bits are not larger than host phys-bits.,
+ * but it is off by default.  Exception: microvm machine type
+ * hard-wires that property to on.
+ */
+VOID
+EFIAPI
+PlatformAddressWidthFromCpuid (
+  IN OUT EFI_HOB_PLATFORM_INFO  *PlatformInfoHob
+  )
+{
+  UINT32  RegEax;
+
+  AsmCpuid (0x80000000, &RegEax, NULL, NULL, NULL);
+  if (RegEax >= 0x80000008) {
+    AsmCpuid (0x80000008, &RegEax, NULL, NULL, NULL);
+    PlatformInfoHob->PhysMemAddressWidth = (UINT8)RegEax;
+  } else {
+    PlatformInfoHob->PhysMemAddressWidth = 36;
+  }
+
+  PlatformInfoHob->FirstNonAddress = LShiftU64 (1, PlatformInfoHob->PhysMemAddressWidth);
+
+  DEBUG ((
+    DEBUG_INFO,
+    "%a: cpuid: phys-bits is %d\n",
+    __FUNCTION__,
+    PlatformInfoHob->PhysMemAddressWidth
+    ));
+}
+
 /**
   Initialize the PhysMemAddressWidth field in PlatformInfoHob based on guest RAM size.
 **/
@@ -503,6 +539,11 @@ PlatformAddressWidthInitialization (
   UINT64  FirstNonAddress;
   UINT8   PhysMemAddressWidth;
 
+  if (PlatformInfoHob->HostBridgeDevId == 0xffff /* microvm */) {
+    PlatformAddressWidthFromCpuid (PlatformInfoHob);
+    return;
+  }
+
   //
   // As guest-physical memory size grows, the permanent PEI RAM requirements
   // are dominated by the identity-mapping page tables built by the DXE IPL.
diff --git a/OvmfPkg/PlatformPei/Platform.c b/OvmfPkg/PlatformPei/Platform.c
index f006755d5fdb..009db67ee60a 100644
--- a/OvmfPkg/PlatformPei/Platform.c
+++ b/OvmfPkg/PlatformPei/Platform.c
@@ -357,12 +357,12 @@ InitializePlatform (
 
   S3Verification ();
   BootModeInitialization (&mPlatformInfoHob);
-  AddressWidthInitialization (&mPlatformInfoHob);
 
   //
   // Query Host Bridge DID
   //
   mPlatformInfoHob.HostBridgeDevId = PciRead16 (OVMF_HOSTBRIDGE_DID);
+  AddressWidthInitialization (&mPlatformInfoHob);
 
   MaxCpuCountInitialization (&mPlatformInfoHob);
 
-- 
2.35.1

[PATCH v6 6/6] OvmfPkg/Microvm/pcie: add pcie support

[Gerd Hoffmann]

Link in pcie and host bridge bits.  Enables support for PCIe in microvm
(qemu-system-x86_64 -M microvm,pcie=on).

Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=3777
Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
---
 OvmfPkg/Microvm/MicrovmX64.dsc | 40 +++++++++++++++++++++-------------
 OvmfPkg/Microvm/README         |  2 +-
 2 files changed, 26 insertions(+), 16 deletions(-)

diff --git a/OvmfPkg/Microvm/MicrovmX64.dsc b/OvmfPkg/Microvm/MicrovmX64.dsc
index c9c843e116a9..82e3f2195aff 100644
--- a/OvmfPkg/Microvm/MicrovmX64.dsc
+++ b/OvmfPkg/Microvm/MicrovmX64.dsc
@@ -336,7 +336,9 @@ [LibraryClasses.common.DXE_RUNTIME_DRIVER]
 !endif
   UefiRuntimeLib|MdePkg/Library/UefiRuntimeLib/UefiRuntimeLib.inf
   BaseCryptLib|CryptoPkg/Library/BaseCryptLib/RuntimeCryptLib.inf
-  PciLib|OvmfPkg/Library/DxePciLibI440FxQ35/DxePciLibI440FxQ35.inf
+#  PciLib|MdePkg/Library/BasePciLibPciExpress/BasePciLibPciExpress.inf
+#  PciPcdProducerLib|OvmfPkg/Fdt/FdtPciPcdProducerLib/FdtPciPcdProducerLib.inf
+#  PciExpressLib|OvmfPkg/Library/BaseCachingPciExpressLib/BaseCachingPciExpressLib.inf
   QemuFwCfgS3Lib|OvmfPkg/Library/QemuFwCfgS3Lib/DxeQemuFwCfgS3LibFwCfg.inf
   VariablePolicyLib|MdeModulePkg/Library/VariablePolicyLib/VariablePolicyLibRuntimeDxe.inf
 
@@ -353,7 +355,9 @@ [LibraryClasses.common.UEFI_DRIVER]
   DebugLib|OvmfPkg/Library/PlatformDebugLibIoPort/PlatformDebugLibIoPort.inf
 !endif
   UefiScsiLib|MdePkg/Library/UefiScsiLib/UefiScsiLib.inf
-  PciLib|OvmfPkg/Library/DxePciLibI440FxQ35/DxePciLibI440FxQ35.inf
+  PciLib|MdePkg/Library/BasePciLibPciExpress/BasePciLibPciExpress.inf
+  PciPcdProducerLib|OvmfPkg/Fdt/FdtPciPcdProducerLib/FdtPciPcdProducerLib.inf
+  PciExpressLib|OvmfPkg/Library/BaseCachingPciExpressLib/BaseCachingPciExpressLib.inf
 
 [LibraryClasses.common.DXE_DRIVER]
   PcdLib|MdePkg/Library/DxePcdLib/DxePcdLib.inf
@@ -375,7 +379,9 @@ [LibraryClasses.common.DXE_DRIVER]
 !if $(SOURCE_DEBUG_ENABLE) == TRUE
   DebugAgentLib|SourceLevelDebugPkg/Library/DebugAgent/DxeDebugAgentLib.inf
 !endif
-  PciLib|OvmfPkg/Library/DxePciLibI440FxQ35/DxePciLibI440FxQ35.inf
+  PciLib|MdePkg/Library/BasePciLibPciExpress/BasePciLibPciExpress.inf
+  PciPcdProducerLib|OvmfPkg/Fdt/FdtPciPcdProducerLib/FdtPciPcdProducerLib.inf
+  PciExpressLib|OvmfPkg/Library/BaseCachingPciExpressLib/BaseCachingPciExpressLib.inf
   MpInitLib|UefiCpuPkg/Library/MpInitLib/DxeMpInitLib.inf
   QemuFwCfgS3Lib|OvmfPkg/Library/QemuFwCfgS3Lib/DxeQemuFwCfgS3LibFwCfg.inf
   QemuLoadImageLib|OvmfPkg/Library/X86QemuLoadImageLib/X86QemuLoadImageLib.inf
@@ -391,7 +397,9 @@ [LibraryClasses.common.UEFI_APPLICATION]
 !else
   DebugLib|OvmfPkg/Library/PlatformDebugLibIoPort/PlatformDebugLibIoPort.inf
 !endif
-  PciLib|OvmfPkg/Library/DxePciLibI440FxQ35/DxePciLibI440FxQ35.inf
+  PciLib|MdePkg/Library/BasePciLibPciExpress/BasePciLibPciExpress.inf
+  PciPcdProducerLib|OvmfPkg/Fdt/FdtPciPcdProducerLib/FdtPciPcdProducerLib.inf
+  PciExpressLib|OvmfPkg/Library/BaseCachingPciExpressLib/BaseCachingPciExpressLib.inf
 
 [LibraryClasses.common.DXE_SMM_DRIVER]
   PcdLib|MdePkg/Library/DxePcdLib/DxePcdLib.inf
@@ -412,7 +420,9 @@ [LibraryClasses.common.DXE_SMM_DRIVER]
   DebugAgentLib|SourceLevelDebugPkg/Library/DebugAgent/SmmDebugAgentLib.inf
 !endif
   BaseCryptLib|CryptoPkg/Library/BaseCryptLib/SmmCryptLib.inf
-  PciLib|OvmfPkg/Library/DxePciLibI440FxQ35/DxePciLibI440FxQ35.inf
+  PciLib|MdePkg/Library/BasePciLibPciExpress/BasePciLibPciExpress.inf
+  PciPcdProducerLib|OvmfPkg/Fdt/FdtPciPcdProducerLib/FdtPciPcdProducerLib.inf
+  PciExpressLib|OvmfPkg/Library/BaseCachingPciExpressLib/BaseCachingPciExpressLib.inf
 
 [LibraryClasses.common.SMM_CORE]
   PcdLib|MdePkg/Library/DxePcdLib/DxePcdLib.inf
@@ -428,7 +438,9 @@ [LibraryClasses.common.SMM_CORE]
 !else
   DebugLib|OvmfPkg/Library/PlatformDebugLibIoPort/PlatformDebugLibIoPort.inf
 !endif
-  PciLib|OvmfPkg/Library/DxePciLibI440FxQ35/DxePciLibI440FxQ35.inf
+  PciLib|MdePkg/Library/BasePciLibPciExpress/BasePciLibPciExpress.inf
+  PciPcdProducerLib|OvmfPkg/Fdt/FdtPciPcdProducerLib/FdtPciPcdProducerLib.inf
+  PciExpressLib|OvmfPkg/Library/BaseCachingPciExpressLib/BaseCachingPciExpressLib.inf
 
 ################################################################################
 #
@@ -503,14 +515,6 @@ [PcdsFixedAtBuild]
   gEfiMdePkgTokenSpaceGuid.PcdDebugPropertyMask|0x2F
 !endif
 
-  # This PCD is used to set the base address of the PCI express hierarchy. It
-  # is only consulted when OVMF runs on Q35. In that case it is programmed into
-  # the PCIEXBAR register.
-  #
-  # On Q35 machine types that QEMU intends to support in the long term, QEMU
-  # never lets the RAM below 4 GB exceed 2816 MB.
-  gEfiMdePkgTokenSpaceGuid.PcdPciExpressBaseAddress|0xB0000000
-
 !if $(SOURCE_DEBUG_ENABLE) == TRUE
   gEfiSourceLevelDebugPkgTokenSpaceGuid.PcdDebugLoadImageMethod|0x2
 !endif
@@ -576,6 +580,12 @@ [PcdsDynamicDefault]
   gEfiMdePkgTokenSpaceGuid.PcdFSBClock|1000000000
   gEfiMdePkgTokenSpaceGuid.PcdPlatformBootTimeOut|0
 
+  # set PcdPciExpressBaseAddress to MAX_UINT64, which signifies that this
+  # PCD and PcdPciDisableBusEnumeration below have not been assigned yet
+  gEfiMdePkgTokenSpaceGuid.PcdPciExpressBaseAddress|0xFFFFFFFFFFFFFFFF
+  gEfiMdePkgTokenSpaceGuid.PcdPciIoTranslation|0x0
+  gEfiMdeModulePkgTokenSpaceGuid.PcdPciDisableBusEnumeration|TRUE
+
   # Set video resolution for text setup.
   gEfiMdeModulePkgTokenSpaceGuid.PcdSetupVideoHorizontalResolution|640
   gEfiMdeModulePkgTokenSpaceGuid.PcdSetupVideoVerticalResolution|480
@@ -676,7 +686,7 @@ [Components]
   OvmfPkg/PciHotPlugInitDxe/PciHotPlugInit.inf
   MdeModulePkg/Bus/Pci/PciHostBridgeDxe/PciHostBridgeDxe.inf {
     <LibraryClasses>
-      PciHostBridgeLib|MdeModulePkg/Library/PciHostBridgeLibNull/PciHostBridgeLibNull.inf
+      PciHostBridgeLib|OvmfPkg/Fdt/FdtPciHostBridgeLib/FdtPciHostBridgeLib.inf
       PciHostBridgeUtilityLib|OvmfPkg/Library/PciHostBridgeUtilityLib/PciHostBridgeUtilityLib.inf
       NULL|OvmfPkg/Library/PlatformHasIoMmuLib/PlatformHasIoMmuLib.inf
   }
diff --git a/OvmfPkg/Microvm/README b/OvmfPkg/Microvm/README
index 540d39f2ec21..813920d92a60 100644
--- a/OvmfPkg/Microvm/README
+++ b/OvmfPkg/Microvm/README
@@ -29,7 +29,7 @@ features
  [working] serial console
  [working] direct kernel boot
  [working] virtio-mmio support
- [in progress] pcie support
+ [working] pcie support
 
 known limitations
 -----------------
-- 
2.35.1

Re: [PATCH v6 0/6] OvmfPkg/Microvm/pcie: add pcie support

[Ard Biesheuvel]

On Mon, 25 Apr 2022 at 12:34, Gerd Hoffmann <kraxel@redhat.com> wrote:
>
> Needs two little tweaks in PCI code because microvm supports mmio only.
> Other than that just wire up the existing code (the PCIe host adapter
> used by microvm is the same (virtual) hardware used by the arm/aarch64
> virtual machines).
>
> v6:
>  - codestyle fix (Abner Chang).
>
> v5:
>  - codestyle (uncrustify) fix.
>
> v4:
>  - update PciHostBridge check (Abner Chang).
>
> v3:
>  - rebase to latest master, adapt to PlatformInitLib.
>  - rework PhysMemAddressWidth handling for microvm.
>
> v2:
>  - rebase to latest master
>  - pick up review tags
>
> Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=3777
>
> Gerd Hoffmann (6):
>   MdeModulePkg/PciHostBridge: io range is not mandatory
>   OvmfPkg/FdtPciHostBridgeLib: io range is not mandatory
>   OvmfPkg/Platform: unfix PcdPciExpressBaseAddress
>   OvmfPkg/Microvm/pcie: no vbeshim please
>   OvmfPkg/Microvm/pcie: mPhysMemAddressWidth tweak
>   OvmfPkg/Microvm/pcie: add pcie support
>

Thanks Gerd. I'm happy to take these, but not without an ack from the
MdeModulePkg maintainers.



>  OvmfPkg/Microvm/MicrovmX64.dsc                | 40 ++++++++++-------
>  .../PlatformInitLib/PlatformInitLib.inf       |  4 +-
>  OvmfPkg/PlatformPei/PlatformPei.inf           |  2 +-
>  .../Bus/Pci/PciHostBridgeDxe/PciHostBridge.c  |  3 ++
>  .../FdtPciHostBridgeLib/FdtPciHostBridgeLib.c | 45 ++++++++++---------
>  OvmfPkg/Library/PlatformInitLib/MemDetect.c   | 45 ++++++++++++++++++-
>  OvmfPkg/Library/PlatformInitLib/Platform.c    |  4 +-
>  OvmfPkg/PlatformPei/Platform.c                |  2 +-
>  OvmfPkg/QemuVideoDxe/VbeShim.c                |  2 +
>  OvmfPkg/Microvm/README                        |  2 +-
>  10 files changed, 104 insertions(+), 45 deletions(-)
>
> --
> 2.35.1
>

Re: [PATCH v6 2/6] OvmfPkg/FdtPciHostBridgeLib: io range is not mandatory

[Abner Chang]

Thanks.

Reviewed-by: Abner Chang <abner.chang@hpe.com>

> -----Original Message-----
> From: Gerd Hoffmann <kraxel@redhat.com>
> Sent: Monday, April 25, 2022 6:35 PM
> To: devel@edk2.groups.io
> Cc: Jiewen Yao <jiewen.yao@intel.com>; Oliver Steffen
> <osteffen@redhat.com>; Leif Lindholm <quic_llindhol@quicinc.com>; Pawel
> Polawski <ppolawsk@redhat.com>; Hao A Wu <hao.a.wu@intel.com>;
> Chang, Abner (HPS SW/FW Technologist) <abner.chang@hpe.com>; Liming
> Gao <gaoliming@byosoft.com.cn>; Ray Ni <ray.ni@intel.com>; Jordan Justen
> <jordan.l.justen@intel.com>; Jian J Wang <jian.j.wang@intel.com>; Ard
> Biesheuvel <ardb+tianocore@kernel.org>; Gerd Hoffmann
> <kraxel@redhat.com>
> Subject: [PATCH v6 2/6] OvmfPkg/FdtPciHostBridgeLib: io range is not
> mandatory
> 
> io range is not mandatory according to pcie spec,
> so allow host bridges without io address space.
> 
> Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
> ---
>  .../FdtPciHostBridgeLib/FdtPciHostBridgeLib.c | 45 ++++++++++---------
>  1 file changed, 23 insertions(+), 22 deletions(-)
> 
> diff --git a/OvmfPkg/Fdt/FdtPciHostBridgeLib/FdtPciHostBridgeLib.c
> b/OvmfPkg/Fdt/FdtPciHostBridgeLib/FdtPciHostBridgeLib.c
> index 98828e0b262b..14b41a533e96 100644
> --- a/OvmfPkg/Fdt/FdtPciHostBridgeLib/FdtPciHostBridgeLib.c
> +++ b/OvmfPkg/Fdt/FdtPciHostBridgeLib/FdtPciHostBridgeLib.c
> @@ -292,13 +292,8 @@ ProcessPciHost (
>      }
>    }
> 
> -  if ((*IoSize == 0) || (*Mmio32Size == 0)) {
> -    DEBUG ((
> -      DEBUG_ERROR,
> -      "%a: %a space empty\n",
> -      __FUNCTION__,
> -      (*IoSize == 0) ? "IO" : "MMIO32"
> -      ));
> +  if (*Mmio32Size == 0) {
> +    DEBUG ((DEBUG_ERROR, "%a: MMIO32 space empty\n",
> __FUNCTION__));
>      return EFI_PROTOCOL_ERROR;
>    }
> 
> @@ -333,13 +328,15 @@ ProcessPciHost (
>      return Status;
>    }
> 
> -  //
> -  // Map the MMIO window that provides I/O access - the PCI host bridge
> code
> -  // is not aware of this translation and so it will only map the I/O view
> -  // in the GCD I/O map.
> -  //
> -  Status = MapGcdMmioSpace (*IoBase + IoTranslation, *IoSize);
> -  ASSERT_EFI_ERROR (Status);
> +  if (*IoSize != 0) {
> +    //
> +    // Map the MMIO window that provides I/O access - the PCI host bridge
> code
> +    // is not aware of this translation and so it will only map the I/O view
> +    // in the GCD I/O map.
> +    //
> +    Status = MapGcdMmioSpace (*IoBase + IoTranslation, *IoSize);
> +    ASSERT_EFI_ERROR (Status);
> +  }
> 
>    return Status;
>  }
> @@ -413,17 +410,21 @@ PciHostBridgeGetRootBridges (
> 
>    AllocationAttributes = EFI_PCI_HOST_BRIDGE_COMBINE_MEM_PMEM;
> 
> -  Io.Base   = IoBase;
> -  Io.Limit  = IoBase + IoSize - 1;
> +  if (IoSize != 0) {
> +    Io.Base  = IoBase;
> +    Io.Limit = IoBase + IoSize - 1;
> +  } else {
> +    Io.Base  = MAX_UINT64;
> +    Io.Limit = 0;
> +  }
> +
>    Mem.Base  = Mmio32Base;
>    Mem.Limit = Mmio32Base + Mmio32Size - 1;
> 
> -  if (sizeof (UINTN) == sizeof (UINT64)) {
> -    MemAbove4G.Base  = Mmio64Base;
> -    MemAbove4G.Limit = Mmio64Base + Mmio64Size - 1;
> -    if (Mmio64Size > 0) {
> -      AllocationAttributes |= EFI_PCI_HOST_BRIDGE_MEM64_DECODE;
> -    }
> +  if ((sizeof (UINTN) == sizeof (UINT64)) && (Mmio64Size != 0)) {
> +    MemAbove4G.Base       = Mmio64Base;
> +    MemAbove4G.Limit      = Mmio64Base + Mmio64Size - 1;
> +    AllocationAttributes |= EFI_PCI_HOST_BRIDGE_MEM64_DECODE;
>    } else {
>      //
>      // UEFI mandates a 1:1 virtual-to-physical mapping, so on a 32-bit
> --
> 2.35.1

Re: [PATCH v6 1/6] MdeModulePkg/PciHostBridge: io range is not mandatory

[Abner Chang]

Reviewed-by: Abner Chang <abner.chang@hpe.com>

> -----Original Message-----
> From: Gerd Hoffmann <kraxel@redhat.com>
> Sent: Monday, April 25, 2022 6:35 PM
> To: devel@edk2.groups.io
> Cc: Jiewen Yao <jiewen.yao@intel.com>; Oliver Steffen
> <osteffen@redhat.com>; Leif Lindholm <quic_llindhol@quicinc.com>; Pawel
> Polawski <ppolawsk@redhat.com>; Hao A Wu <hao.a.wu@intel.com>; Chang,
> Abner (HPS SW/FW Technologist) <abner.chang@hpe.com>; Liming Gao
> <gaoliming@byosoft.com.cn>; Ray Ni <ray.ni@intel.com>; Jordan Justen
> <jordan.l.justen@intel.com>; Jian J Wang <jian.j.wang@intel.com>; Ard
> Biesheuvel <ardb+tianocore@kernel.org>; Gerd Hoffmann
> <kraxel@redhat.com>; Ard Biesheuvel <ardb@kernel.org>
> Subject: [PATCH v6 1/6] MdeModulePkg/PciHostBridge: io range is not
> mandatory
> 
> io range is not mandatory according to pcie spec,
> so allow bridge configurations without io address
> space assigned.
> 
> Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
> Reviewed-by: Ard Biesheuvel <ardb@kernel.org>
> ---
>  MdeModulePkg/Bus/Pci/PciHostBridgeDxe/PciHostBridge.c | 3 +++
>  1 file changed, 3 insertions(+)
> 
> diff --git a/MdeModulePkg/Bus/Pci/PciHostBridgeDxe/PciHostBridge.c
> b/MdeModulePkg/Bus/Pci/PciHostBridgeDxe/PciHostBridge.c
> index b20bcd310ad5..712662707931 100644
> --- a/MdeModulePkg/Bus/Pci/PciHostBridgeDxe/PciHostBridge.c
> +++ b/MdeModulePkg/Bus/Pci/PciHostBridgeDxe/PciHostBridge.c
> @@ -1085,6 +1085,9 @@ NotifyPhase (
>                RootBridge->ResAllocNode[Index].Base   = BaseAddress;
>                RootBridge->ResAllocNode[Index].Status = ResAllocated;
>                DEBUG ((DEBUG_INFO, "Success\n"));
> +            } else if ((Index == TypeIo) && (RootBridge->Io.Base == MAX_UINT64)) {
> +              /* optional on PCIe */
> +              DEBUG ((DEBUG_INFO, "PCI Root Bridge does not provide IO
> Resources.\n"));
>              } else {
>                ReturnStatus = EFI_OUT_OF_RESOURCES;
>                DEBUG ((DEBUG_ERROR, "Out Of Resource!\n"));
> --
> 2.35.1