[PATCH v2 0/2] Prepare bhyve for GPU-Passthrough

[PATCH v2 0/2] Prepare bhyve for GPU-Passthrough

[Corvin Köhne]

Hi,

these patches could be merged to improve GPU-Passthrough support.

If I get an agreement with Peter about BusEnumeration, I'll create a new patch series


Best Regards
Corvin


Corvin Köhne (2):
  OvmfPkg/Bhyve: add USB support
  OvmfPkg/Bhyve: use static PCI32Base address

 OvmfPkg/Bhyve/BhyveX64.dsc           | 15 +++++++++++++--
 OvmfPkg/Bhyve/BhyveX64.fdf           | 10 ++++++++++
 OvmfPkg/Bhyve/PlatformPei/Platform.c |  4 +++-
 3 files changed, 26 insertions(+), 3 deletions(-)

-- 
2.11.0

Beckhoff Automation GmbH & Co. KG | Managing Director: Dipl. Phys. Hans Beckhoff
Registered office: Verl, Germany | Register court: Guetersloh HRA 7075

[PATCH v2 1/2] OvmfPkg/Bhyve: add USB support

[Corvin Köhne]

An USB driver is required to use a keyboard or mouse while installing
an OS or while in a bootloader menu like grub when using GPU + USB
Passthrough.
---
 OvmfPkg/Bhyve/BhyveX64.dsc | 11 +++++++++++
 OvmfPkg/Bhyve/BhyveX64.fdf | 10 ++++++++++
 2 files changed, 21 insertions(+)

diff --git a/OvmfPkg/Bhyve/BhyveX64.dsc b/OvmfPkg/Bhyve/BhyveX64.dsc
index d8792812ab..22a6f11069 100644
--- a/OvmfPkg/Bhyve/BhyveX64.dsc
+++ b/OvmfPkg/Bhyve/BhyveX64.dsc
@@ -163,6 +163,7 @@
   FileHandleLib|MdePkg/Library/UefiFileHandleLib/UefiFileHandleLib.inf
   UefiCpuLib|UefiCpuPkg/Library/BaseUefiCpuLib/BaseUefiCpuLib.inf
   SecurityManagementLib|MdeModulePkg/Library/DxeSecurityManagementLib/DxeSecurityManagementLib.inf
+  UefiUsbLib|MdePkg/Library/UefiUsbLib/UefiUsbLib.inf
   SerializeVariablesLib|OvmfPkg/Library/SerializeVariablesLib/SerializeVariablesLib.inf
   QemuFwCfgLib|OvmfPkg/Library/QemuFwCfgLib/QemuFwCfgLibNull.inf
   QemuFwCfgS3Lib|OvmfPkg/Library/QemuFwCfgS3Lib/BaseQemuFwCfgS3LibNull.inf
@@ -777,6 +778,16 @@
 !endif
   OvmfPkg/VirtioNetDxe/VirtioNet.inf
 
+  #
+  # Usb Support
+  #
+  MdeModulePkg/Bus/Pci/UhciDxe/UhciDxe.inf
+  MdeModulePkg/Bus/Pci/EhciDxe/EhciDxe.inf
+  MdeModulePkg/Bus/Pci/XhciDxe/XhciDxe.inf
+  MdeModulePkg/Bus/Usb/UsbBusDxe/UsbBusDxe.inf
+  MdeModulePkg/Bus/Usb/UsbKbDxe/UsbKbDxe.inf
+  MdeModulePkg/Bus/Usb/UsbMassStorageDxe/UsbMassStorageDxe.inf
+
 !ifdef $(CSM_ENABLE)
   IntelFrameworkModulePkg/Csm/BiosThunk/VideoDxe/VideoDxe.inf {
     <LibraryClasses>
diff --git a/OvmfPkg/Bhyve/BhyveX64.fdf b/OvmfPkg/Bhyve/BhyveX64.fdf
index 3eff36dac1..f081b82137 100644
--- a/OvmfPkg/Bhyve/BhyveX64.fdf
+++ b/OvmfPkg/Bhyve/BhyveX64.fdf
@@ -291,6 +291,16 @@ INF MdeModulePkg/Logo/LogoDxe.inf
 !include NetworkPkg/Network.fdf.inc
   INF  OvmfPkg/VirtioNetDxe/VirtioNet.inf
 
+#
+# Usb Support
+#
+INF  MdeModulePkg/Bus/Pci/UhciDxe/UhciDxe.inf
+INF  MdeModulePkg/Bus/Pci/EhciDxe/EhciDxe.inf
+INF  MdeModulePkg/Bus/Pci/XhciDxe/XhciDxe.inf
+INF  MdeModulePkg/Bus/Usb/UsbBusDxe/UsbBusDxe.inf
+INF  MdeModulePkg/Bus/Usb/UsbKbDxe/UsbKbDxe.inf
+INF  MdeModulePkg/Bus/Usb/UsbMassStorageDxe/UsbMassStorageDxe.inf
+
 !ifdef $(CSM_ENABLE)
 INF  IntelFrameworkModulePkg/Csm/BiosThunk/VideoDxe/VideoDxe.inf
 !endif
-- 
2.11.0

Beckhoff Automation GmbH & Co. KG | Managing Director: Dipl. Phys. Hans Beckhoff
Registered office: Verl, Germany | Register court: Guetersloh HRA 7075

[PATCH v2 2/2] OvmfPkg/Bhyve: use static PCI32Base address

[Corvin Köhne]

It's neccessary to allocate a Graphics Stolen Memory area to enable
GPU-Passthrough for integrated Intel GPUs. Therefore, use a new
memory layout with a static Pci32Baseaddress.

Old layout:
	[...        , lowmemlimit] RAM
	[lowmemlimit, 0xE000 0000] PCI Space
New layout:
	[...        , lowmemlimit] RAM
	[lowmemlimit, gsmbase    ] Memory hole (may be absent)
	[gsmbase    , 0xC000 0000] GSM (may be absent)
	[0xC000 0000, 0xE000 0000] PCI Space
---
 OvmfPkg/Bhyve/BhyveX64.dsc           | 4 ++--
 OvmfPkg/Bhyve/PlatformPei/Platform.c | 4 +++-
 2 files changed, 5 insertions(+), 3 deletions(-)

diff --git a/OvmfPkg/Bhyve/BhyveX64.dsc b/OvmfPkg/Bhyve/BhyveX64.dsc
index 22a6f11069..b678029b40 100644
--- a/OvmfPkg/Bhyve/BhyveX64.dsc
+++ b/OvmfPkg/Bhyve/BhyveX64.dsc
@@ -537,8 +537,8 @@
   gUefiOvmfPkgTokenSpaceGuid.PcdOvmfHostBridgePciDevId|0
   gUefiOvmfPkgTokenSpaceGuid.PcdPciIoBase|0x0
   gUefiOvmfPkgTokenSpaceGuid.PcdPciIoSize|0x0
-  gUefiOvmfPkgTokenSpaceGuid.PcdPciMmio32Base|0x0
-  gUefiOvmfPkgTokenSpaceGuid.PcdPciMmio32Size|0x0
+  gUefiOvmfPkgTokenSpaceGuid.PcdPciMmio32Base|0xC0000000
+  gUefiOvmfPkgTokenSpaceGuid.PcdPciMmio32Size|0x20000000
   gUefiOvmfPkgTokenSpaceGuid.PcdPciMmio64Base|0x0
   gUefiOvmfPkgTokenSpaceGuid.PcdPciMmio64Size|0x800000000
 
diff --git a/OvmfPkg/Bhyve/PlatformPei/Platform.c b/OvmfPkg/Bhyve/PlatformPei/Platform.c
index 3a414ffcb7..f38e74ccfc 100644
--- a/OvmfPkg/Bhyve/PlatformPei/Platform.c
+++ b/OvmfPkg/Bhyve/PlatformPei/Platform.c
@@ -191,7 +191,9 @@ MemMapInitialization (
       ASSERT (PciExBarBase <= MAX_UINT32 - SIZE_256MB);
       PciBase = (UINT32)(PciExBarBase + SIZE_256MB);
     } else {
-      PciBase = (TopOfLowRam < BASE_2GB) ? BASE_2GB : TopOfLowRam;
+      PciBase = PcdGet64(PcdPciMmio32Base);
+      if (PciBase == 0)
+        PciBase = (TopOfLowRam < BASE_2GB) ? BASE_2GB : TopOfLowRam;
     }
 
     //
-- 
2.11.0

Beckhoff Automation GmbH & Co. KG | Managing Director: Dipl. Phys. Hans Beckhoff
Registered office: Verl, Germany | Register court: Guetersloh HRA 7075

Re: [PATCH v2 1/2] OvmfPkg/Bhyve: add USB support

[Peter Grehan]

Hi Corvin,

> +  #
> +  # Usb Support
> +  #
> +  MdeModulePkg/Bus/Pci/UhciDxe/UhciDxe.inf
> +  MdeModulePkg/Bus/Pci/EhciDxe/EhciDxe.inf
> +  MdeModulePkg/Bus/Pci/XhciDxe/XhciDxe.inf
> +  MdeModulePkg/Bus/Usb/UsbBusDxe/UsbBusDxe.inf
> +  MdeModulePkg/Bus/Usb/UsbKbDxe/UsbKbDxe.inf
> +  MdeModulePkg/Bus/Usb/UsbMassStorageDxe/UsbMassStorageDxe.inf

  bhyve doesn't (and will likely never) support Uhci/Ehci controllers so 
those lines can be removed.

later,

Peter.

Re: [PATCH v2 1/2] OvmfPkg/Bhyve: add USB support

[Corvin Köhne]

Hi Peter,

>  bhyve doesn't (and will likely never) support Uhci/Ehci controllers so those lines can be removed.

Is it possible to pass such a controller by PCI-Passthrough to a VM?
If it's possible, I would keep these lines.


Best regards
Corvin

Beckhoff Automation GmbH & Co. KG | Managing Director: Dipl. Phys. Hans Beckhoff Registered office: Verl, Germany | Register court: Guetersloh HRA 7075

Re: [edk2-devel] [PATCH v2 1/2] OvmfPkg/Bhyve: add USB support

[Peter Grehan]

Hi Corvin,

>>   bhyve doesn't (and will likely never) support Uhci/Ehci controllers so those lines can be removed.
> 
> Is it possible to pass such a controller by PCI-Passthrough to a VM?
> If it's possible, I would keep these lines.

  Uhci/Ehci only support legacy interrupts and that isn't supported by 
bhyve PCI passthru.

later,

Peter.

Re: [edk2-devel] [PATCH v2 1/2] OvmfPkg/Bhyve: add USB support

[Michael Brown]

On 01/07/2021 11:32, Peter Grehan wrote:
>>>   bhyve doesn't (and will likely never) support Uhci/Ehci controllers 
>>> so those lines can be removed.
>>
>> Is it possible to pass such a controller by PCI-Passthrough to a VM?
>> If it's possible, I would keep these lines.
> 
>   Uhci/Ehci only support legacy interrupts and that isn't supported by 
> bhyve PCI passthru.

Is that a detail of the current implementation, or a fundamental 
limitation in the bhyve architecture?

Thanks,

Michael

Re: [edk2-devel] [PATCH v2 1/2] OvmfPkg/Bhyve: add USB support

[Peter Grehan]

Hi Michael,

>>>>   bhyve doesn't (and will likely never) support Uhci/Ehci 
>>>> controllers so those lines can be removed.
>>>
>>> Is it possible to pass such a controller by PCI-Passthrough to a VM?
>>> If it's possible, I would keep these lines.
>>
>>   Uhci/Ehci only support legacy interrupts and that isn't supported by 
>> bhyve PCI passthru.
> 
> Is that a detail of the current implementation, or a fundamental 
> limitation in the bhyve architecture?

  Only two choices ? :) Maybe half way between those points.

later,

Peter.

Re: [edk2-devel] [PATCH v2 1/2] OvmfPkg/Bhyve: add USB support

[Michael Brown]

On 02/07/2021 00:31, Peter Grehan wrote:
>>>>>   bhyve doesn't (and will likely never) support Uhci/Ehci 
>>>>> controllers so those lines can be removed.
>>>>
>>>> Is it possible to pass such a controller by PCI-Passthrough to a VM?
>>>> If it's possible, I would keep these lines.
>>>
>>>   Uhci/Ehci only support legacy interrupts and that isn't supported 
>>> by bhyve PCI passthru.
>>
>> Is that a detail of the current implementation, or a fundamental 
>> limitation in the bhyve architecture?
> 
>   Only two choices ? :) Maybe half way between those points.

When there is no fundamental limitation, i.e. when a future version of 
the hypervisor may be able to support the feature with no changes to the 
firmware, then it would be good practice to leave the drivers enabled. 
Doing so avoids creating an unnecessarily tight coupling between the 
hypervisor and firmware versions.

More importantly: does it even matter that the hypervisor doesn't 
support passthrough of PCI legacy interrupts?  UEFI operates on a 
polling basis, with the only active interrupt being some kind of 
periodic timer.  Where do you see any requirement for legacy interrupts 
in the UHCI/EHCI drivers?

Thanks,

Michael

Re: [edk2-devel] [PATCH v2 1/2] OvmfPkg/Bhyve: add USB support

[Peter Grehan]

>>> Is that a detail of the current implementation, or a fundamental 
>>> limitation in the bhyve architecture?
>>
>>   Only two choices ? :) Maybe half way between those points.
> 
> When there is no fundamental limitation,

  It's not so much a fundamental technical limitation as in a large 
amount of work in FreeBSD for almost zero return - maybe a fundamental 
resource limitation.

> i.e. when a future version of 
> the hypervisor may be able to support the feature with no changes to the 
> firmware, then it would be good practice to leave the drivers enabled. 
> Doing so avoids creating an unnecessarily tight coupling between the 
> hypervisor and firmware versions.
  Well versed in those issues :)

> More importantly: does it even matter that the hypervisor doesn't 
> support passthrough of PCI legacy interrupts?  UEFI operates on a 
> polling basis, with the only active interrupt being some kind of 
> periodic timer.  Where do you see any requirement for legacy interrupts 
> in the UHCI/EHCI drivers?

  It creates a case where the o/s booted by EFI isn't able to use those 
controllers as expected.

  Anyways, this is more a discussion for the freebsd-virtualization 
mailing list if you'd like to chat about it more.

later,

Peter.