[PATCH 0/4] Prepare bhyve's OVMF for GPU-Passthrough

[PATCH 0/4] Prepare bhyve's OVMF for GPU-Passthrough

[c.koehne]

Hi,

GPU-Passthrough for bhyve requires a few patches to work properly.
These patches will allow GPU-Passthrough for bhyve.
It will work for dedicated AMD GPUs and integrated Intel GPUs.


Best Regards,
Corvin



Corvin Köhne (4):
  OvmfPkg/Bhyve: enable bus enumeration
  Revert "OvmfPkg/Bhyve: consume PciHostBridgeLibScan"
  OvmfPkg/Bhyve: add USB support
  OvmfPkg/Bhyve: use static PCI32Base address

 OvmfPkg/Bhyve/BhyveX64.dsc           | 19 +++++++++++++++----
 OvmfPkg/Bhyve/BhyveX64.fdf           | 10 ++++++++++
 OvmfPkg/Bhyve/PlatformPei/Platform.c |  4 +++-
 3 files changed, 28 insertions(+), 5 deletions(-)

-- 
2.11.0

Beckhoff Automation GmbH & Co. KG | Managing Director: Dipl. Phys. Hans Beckhoff
Registered office: Verl, Germany | Register court: Guetersloh HRA 7075

[PATCH 1/4] OvmfPkg/Bhyve: enable bus enumeration

[Corvin Köhne]

Neccessary for GPU Passthrough of dedicated AMD GPUs. For Linux
guests, AMD GPUs require that their PCI ROM is processed by UEFI.
Enable bus enumeration to process the PCI ROM of all devices.

Signed-off-by: Corvin Köhne <c.koehne@beckhoff.com>
---
 OvmfPkg/Bhyve/BhyveX64.dsc | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/OvmfPkg/Bhyve/BhyveX64.dsc b/OvmfPkg/Bhyve/BhyveX64.dsc
index d8792812ab..ef331d556e 100644
--- a/OvmfPkg/Bhyve/BhyveX64.dsc
+++ b/OvmfPkg/Bhyve/BhyveX64.dsc
@@ -431,7 +431,7 @@
   gEfiMdeModulePkgTokenSpaceGuid.PcdInstallAcpiSdtProtocol|TRUE
 
 [PcdsFixedAtBuild]
-  gEfiMdeModulePkgTokenSpaceGuid.PcdPciDisableBusEnumeration|TRUE
+  gEfiMdeModulePkgTokenSpaceGuid.PcdPciDisableBusEnumeration|FALSE
   gEfiMdeModulePkgTokenSpaceGuid.PcdStatusCodeUseSerial|FALSE
   gEfiMdeModulePkgTokenSpaceGuid.PcdStatusCodeUseMemory|TRUE
   gEfiMdeModulePkgTokenSpaceGuid.PcdStatusCodeMemorySize|1
-- 
2.11.0

Beckhoff Automation GmbH & Co. KG | Managing Director: Dipl. Phys. Hans Beckhoff
Registered office: Verl, Germany | Register court: Guetersloh HRA 7075

Re: [edk2-devel] [PATCH 1/4] OvmfPkg/Bhyve: enable bus enumeration

[Laszlo Ersek]

On 06/14/21 11:00, Corvin Köhne wrote:
> Neccessary for GPU Passthrough of dedicated AMD GPUs. For Linux
> guests, AMD GPUs require that their PCI ROM is processed by UEFI.
> Enable bus enumeration to process the PCI ROM of all devices.
> 
> Signed-off-by: Corvin Köhne <c.koehne@beckhoff.com>
> ---
>  OvmfPkg/Bhyve/BhyveX64.dsc | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/OvmfPkg/Bhyve/BhyveX64.dsc b/OvmfPkg/Bhyve/BhyveX64.dsc
> index d8792812ab..ef331d556e 100644
> --- a/OvmfPkg/Bhyve/BhyveX64.dsc
> +++ b/OvmfPkg/Bhyve/BhyveX64.dsc
> @@ -431,7 +431,7 @@
>    gEfiMdeModulePkgTokenSpaceGuid.PcdInstallAcpiSdtProtocol|TRUE
>  
>  [PcdsFixedAtBuild]
> -  gEfiMdeModulePkgTokenSpaceGuid.PcdPciDisableBusEnumeration|TRUE
> +  gEfiMdeModulePkgTokenSpaceGuid.PcdPciDisableBusEnumeration|FALSE
>    gEfiMdeModulePkgTokenSpaceGuid.PcdStatusCodeUseSerial|FALSE
>    gEfiMdeModulePkgTokenSpaceGuid.PcdStatusCodeUseMemory|TRUE
>    gEfiMdeModulePkgTokenSpaceGuid.PcdStatusCodeMemorySize|1
> 

The right approach -- assuming you reach an agreement with Peter -- is
to revert commist c2f24ba3218a and 8c8f886f2755 first (in this order),
then remove the PcdPciDisableBusEnumeration setting from the DSC
altogether (given the DEC default of the PCD being FALSE).

Independently, please use "python3
$EDK_TOOLS_PATH/Scripts/GetMaintainer.py", for determining the propert
set of CC's for every patch in a series.

I have no comments on the other patches (especially the 4th one), as
they are tied to bhyve platform internals, which I'm unfamiliar with.

Thanks
Laszlo

[PATCH 2/4] Revert "OvmfPkg/Bhyve: consume PciHostBridgeLibScan"

[Corvin Köhne]

This reverts commit c2f24ba3218ae91a8d5a1a31c31dad3417850d0c.

Revert this commit to enable bus enumeration properly.

Signed-off-by: Corvin Köhne <c.koehne@beckhoff.com>
---
 OvmfPkg/Bhyve/BhyveX64.dsc | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/OvmfPkg/Bhyve/BhyveX64.dsc b/OvmfPkg/Bhyve/BhyveX64.dsc
index ef331d556e..951282c7d0 100644
--- a/OvmfPkg/Bhyve/BhyveX64.dsc
+++ b/OvmfPkg/Bhyve/BhyveX64.dsc
@@ -666,7 +666,7 @@
   OvmfPkg/PciHotPlugInitDxe/PciHotPlugInit.inf
   MdeModulePkg/Bus/Pci/PciHostBridgeDxe/PciHostBridgeDxe.inf {
     <LibraryClasses>
-      PciHostBridgeLib|OvmfPkg/Library/PciHostBridgeLibScan/PciHostBridgeLibScan.inf
+      PciHostBridgeLib|OvmfPkg/Library/PciHostBridgeLib/PciHostBridgeLib.inf
       PciHostBridgeUtilityLib|OvmfPkg/Library/PciHostBridgeUtilityLib/PciHostBridgeUtilityLib.inf
       NULL|OvmfPkg/Library/PlatformHasIoMmuLib/PlatformHasIoMmuLib.inf
   }
-- 
2.11.0

Beckhoff Automation GmbH & Co. KG | Managing Director: Dipl. Phys. Hans Beckhoff
Registered office: Verl, Germany | Register court: Guetersloh HRA 7075

[PATCH 3/4] OvmfPkg/Bhyve: add USB support

[Corvin Köhne]

An USB driver is required to use a keyboard or mouse while installing
an OS or while in a bootloader menu like grub when using GPU + USB
Passthrough.

Signed-off-by: Corvin Köhne <c.koehne@beckhoff.com>
---
 OvmfPkg/Bhyve/BhyveX64.dsc | 11 +++++++++++
 OvmfPkg/Bhyve/BhyveX64.fdf | 10 ++++++++++
 2 files changed, 21 insertions(+)

diff --git a/OvmfPkg/Bhyve/BhyveX64.dsc b/OvmfPkg/Bhyve/BhyveX64.dsc
index 951282c7d0..c35bf18449 100644
--- a/OvmfPkg/Bhyve/BhyveX64.dsc
+++ b/OvmfPkg/Bhyve/BhyveX64.dsc
@@ -163,6 +163,7 @@
   FileHandleLib|MdePkg/Library/UefiFileHandleLib/UefiFileHandleLib.inf
   UefiCpuLib|UefiCpuPkg/Library/BaseUefiCpuLib/BaseUefiCpuLib.inf
   SecurityManagementLib|MdeModulePkg/Library/DxeSecurityManagementLib/DxeSecurityManagementLib.inf
+  UefiUsbLib|MdePkg/Library/UefiUsbLib/UefiUsbLib.inf
   SerializeVariablesLib|OvmfPkg/Library/SerializeVariablesLib/SerializeVariablesLib.inf
   QemuFwCfgLib|OvmfPkg/Library/QemuFwCfgLib/QemuFwCfgLibNull.inf
   QemuFwCfgS3Lib|OvmfPkg/Library/QemuFwCfgS3Lib/BaseQemuFwCfgS3LibNull.inf
@@ -777,6 +778,16 @@
 !endif
   OvmfPkg/VirtioNetDxe/VirtioNet.inf
 
+  #
+  # Usb Support
+  #
+  MdeModulePkg/Bus/Pci/UhciDxe/UhciDxe.inf
+  MdeModulePkg/Bus/Pci/EhciDxe/EhciDxe.inf
+  MdeModulePkg/Bus/Pci/XhciDxe/XhciDxe.inf
+  MdeModulePkg/Bus/Usb/UsbBusDxe/UsbBusDxe.inf
+  MdeModulePkg/Bus/Usb/UsbKbDxe/UsbKbDxe.inf
+  MdeModulePkg/Bus/Usb/UsbMassStorageDxe/UsbMassStorageDxe.inf
+
 !ifdef $(CSM_ENABLE)
   IntelFrameworkModulePkg/Csm/BiosThunk/VideoDxe/VideoDxe.inf {
     <LibraryClasses>
diff --git a/OvmfPkg/Bhyve/BhyveX64.fdf b/OvmfPkg/Bhyve/BhyveX64.fdf
index 3eff36dac1..f081b82137 100644
--- a/OvmfPkg/Bhyve/BhyveX64.fdf
+++ b/OvmfPkg/Bhyve/BhyveX64.fdf
@@ -291,6 +291,16 @@ INF MdeModulePkg/Logo/LogoDxe.inf
 !include NetworkPkg/Network.fdf.inc
   INF  OvmfPkg/VirtioNetDxe/VirtioNet.inf
 
+#
+# Usb Support
+#
+INF  MdeModulePkg/Bus/Pci/UhciDxe/UhciDxe.inf
+INF  MdeModulePkg/Bus/Pci/EhciDxe/EhciDxe.inf
+INF  MdeModulePkg/Bus/Pci/XhciDxe/XhciDxe.inf
+INF  MdeModulePkg/Bus/Usb/UsbBusDxe/UsbBusDxe.inf
+INF  MdeModulePkg/Bus/Usb/UsbKbDxe/UsbKbDxe.inf
+INF  MdeModulePkg/Bus/Usb/UsbMassStorageDxe/UsbMassStorageDxe.inf
+
 !ifdef $(CSM_ENABLE)
 INF  IntelFrameworkModulePkg/Csm/BiosThunk/VideoDxe/VideoDxe.inf
 !endif
-- 
2.11.0

Beckhoff Automation GmbH & Co. KG | Managing Director: Dipl. Phys. Hans Beckhoff
Registered office: Verl, Germany | Register court: Guetersloh HRA 7075

[PATCH 4/4] OvmfPkg/Bhyve: use static PCI32Base address

[Corvin Köhne]

It's neccessary to allocate a Graphics Stolen Memory area to enable
GPU-Passthrough for integrated Intel GPUs. Therefore, use a new
memory layout with a static Pci32Baseaddress.

Old layout:
	[...        , lowmemlimit] RAM
	[lowmemlimit, 0xE000 0000] PCI Space
New layout:
	[...        , lowmemlimit] RAM
	[lowmemlimit, gsmbase    ] Memory hole (may be absent)
	[gsmbase    , 0xC000 0000] GSM (may be absent)
	[0xC000 0000, 0xE000 0000] PCI Space

Signed-off-by: Corvin Köhne <c.koehne@beckhoff.com>
---
 OvmfPkg/Bhyve/BhyveX64.dsc           | 4 ++--
 OvmfPkg/Bhyve/PlatformPei/Platform.c | 4 +++-
 2 files changed, 5 insertions(+), 3 deletions(-)

diff --git a/OvmfPkg/Bhyve/BhyveX64.dsc b/OvmfPkg/Bhyve/BhyveX64.dsc
index c35bf18449..e5d1dbccff 100644
--- a/OvmfPkg/Bhyve/BhyveX64.dsc
+++ b/OvmfPkg/Bhyve/BhyveX64.dsc
@@ -537,8 +537,8 @@
   gUefiOvmfPkgTokenSpaceGuid.PcdOvmfHostBridgePciDevId|0
   gUefiOvmfPkgTokenSpaceGuid.PcdPciIoBase|0x0
   gUefiOvmfPkgTokenSpaceGuid.PcdPciIoSize|0x0
-  gUefiOvmfPkgTokenSpaceGuid.PcdPciMmio32Base|0x0
-  gUefiOvmfPkgTokenSpaceGuid.PcdPciMmio32Size|0x0
+  gUefiOvmfPkgTokenSpaceGuid.PcdPciMmio32Base|0xC0000000
+  gUefiOvmfPkgTokenSpaceGuid.PcdPciMmio32Size|0x20000000
   gUefiOvmfPkgTokenSpaceGuid.PcdPciMmio64Base|0x0
   gUefiOvmfPkgTokenSpaceGuid.PcdPciMmio64Size|0x800000000
 
diff --git a/OvmfPkg/Bhyve/PlatformPei/Platform.c b/OvmfPkg/Bhyve/PlatformPei/Platform.c
index 3a414ffcb7..f38e74ccfc 100644
--- a/OvmfPkg/Bhyve/PlatformPei/Platform.c
+++ b/OvmfPkg/Bhyve/PlatformPei/Platform.c
@@ -191,7 +191,9 @@ MemMapInitialization (
       ASSERT (PciExBarBase <= MAX_UINT32 - SIZE_256MB);
       PciBase = (UINT32)(PciExBarBase + SIZE_256MB);
     } else {
-      PciBase = (TopOfLowRam < BASE_2GB) ? BASE_2GB : TopOfLowRam;
+      PciBase = PcdGet64(PcdPciMmio32Base);
+      if (PciBase == 0)
+        PciBase = (TopOfLowRam < BASE_2GB) ? BASE_2GB : TopOfLowRam;
     }
 
     //
-- 
2.11.0

Beckhoff Automation GmbH & Co. KG | Managing Director: Dipl. Phys. Hans Beckhoff
Registered office: Verl, Germany | Register court: Guetersloh HRA 7075

Re: [edk2-devel] [PATCH 0/4] Prepare bhyve's OVMF for GPU-Passthrough

[Peter Grehan]

Hi Corvin,

> GPU-Passthrough for bhyve requires a few patches to work properly.
> These patches will allow GPU-Passthrough for bhyve.
> It will work for dedicated AMD GPUs and integrated Intel GPUs.

  I have no issue with adding USB support or fixing the PCI32 base: that 
either codifies existing behaviour or adds functionality.

  However, flipping the switch over to bus enumeration being in EFI is a 
policy change in how bhyve has always worked - can that be discussed on 
the freebsd-virtualization first ?

later,

Peter.