Re: [Help] UEFI boot KVM4T vm hang On TianoCore

[Laszlo Ersek <lersek@redhat.com>]

Hi,

sorry about the delayed response.

CC'ing Igor, and commenting below:

On 10/31/17 15:29, Hangaohuai wrote:
> Hi, Laszlo Ersek;
>
> I have tested the uefi booting KVM vm with the configuration(xml); but
> start hang.
> Enable the memoryhotplug, with usb3.0 config. The config as Config1
> below.
>
> Tested branches:
> UDK2017     eea98eea4ccbb1d640657770bccb5497fddc6064
> Master       76fd5a660d704538a1b14a58d03a4eef9682b01c
>
> Both hang on the snapshot TianoCore in VNC
>
> Try to shoot the problem;
> I find the early version can boot success with the same config.
> Maybe the patch1 below cause the problem;
>
> Try to ignore the patch does, the master/ UDK2017 both can boot
> success.
> But I don't know why. Hope for your help,thanks.
>
> Patch1
> ***********************************************
> commit 4f5eff8193096eb847639f090a7dfae3cff95fde
> Author: Laszlo Ersek <lersek@redhat.com>
> Date:   Fri Mar 4 20:06:26 2016 +0100
>
> OvmfPkg: PciHostBridgeLib: install 64-bit PCI host aperture
>
> diff --git a/OvmfPkg/Library/PciHostBridgeLib/PciHostBridgeLib.c b/OvmfPkg/Library/PciHostBridgeLib/PciHostBridgeLib
> index 3e02778..1d3d10a 100644
> --- a/OvmfPkg/Library/PciHostBridgeLib/PciHostBridgeLib.c
> +++ b/OvmfPkg/Library/PciHostBridgeLib/PciHostBridgeLib.c
> @@ -132,6 +132,13 @@ InitRootBridge (
>    RootBus->MemAbove4G.Base      = 0;
>    RootBus->MemAbove4G.Limit     = 0;
> +  if (PcdGet64 (PcdPciMmio64Size) > 0) {
> +    RootBus->AllocationAttributes |= EFI_PCI_HOST_BRIDGE_MEM64_DECODE;
> +    RootBus->MemAbove4G.Base       = PcdGet64 (PcdPciMmio64Base);
> +    RootBus->MemAbove4G.Limit      = PcdGet64 (PcdPciMmio64Base) +
> +                                     (PcdGet64 (PcdPciMmio64Size) - 1);
> +  }
> +
>    RootBus->Bus.Base  = RootBusNumber;
>    RootBus->Bus.Limit = MaxSubBusNumber;
>    RootBus->Io.Base   = PcdGet64 (PcdPciIoBase);
> diff --git a/OvmfPkg/Library/PciHostBridgeLib/PciHostBridgeLib.inf b/OvmfPkg/Library/PciHostBridgeLib/PciHostBridgeL
> index bbec746..7a964c7 100644
> --- a/OvmfPkg/Library/PciHostBridgeLib/PciHostBridgeLib.inf
> +++ b/OvmfPkg/Library/PciHostBridgeLib/PciHostBridgeLib.inf
> @@ -51,4 +51,6 @@
>    gUefiOvmfPkgTokenSpaceGuid.PcdPciIoSize
>    gUefiOvmfPkgTokenSpaceGuid.PcdPciMmio32Base
>    gUefiOvmfPkgTokenSpaceGuid.PcdPciMmio32Size
> +  gUefiOvmfPkgTokenSpaceGuid.PcdPciMmio64Base
> +  gUefiOvmfPkgTokenSpaceGuid.PcdPciMmio64Size
>    gUefiOvmfPkgTokenSpaceGuid.PcdOvmfHostBridgePciDevId
> ***********************************************
>
>
> Config1
> ***********************************************
> <maxMemory slots='256' unit='KiB'>4294967296</maxMemory>
>   <memory unit='KiB'>4294967</memory>
>   <currentMemory unit='KiB'>4294967</currentMemory>
> Xxxx
> <controller type='usb' index='2' model='nec-xhci'>
>     <address type='pci' domain='0x0000' bus='0x00' slot='0x1c' function='0x0'/>
>  </controller>
> ***********************************************

The patch that you identified is responsible for exposing the 64-bit
MMIO aperture for PCI MMIO BAR allocation purposes.

Reverting this patch might mask the issue, but I don't expect the actual
problem to be in this patch.

Instead, the base address and the size of the aperture in question is
determined in the file

  OvmfPkg/PlatformPei/MemDetect.c

in the function

  GetFirstNonAddress()

This function takes the memory (DIMM) hotplug range in question, the end
of which is presented by QEMU in the fw_cfg file

  etc/reserved-memory-end

To my knowledge, this fw_cfg file is an 8-byte integer in LE encoding.

Quoting the code:

>   //
>   // If QEMU presents an E820 map, then get the highest exclusive >=4GB RAM
>   // address from it. This can express an address >= 4GB+1TB.
>   //
>   // Otherwise, get the flat size of the memory above 4GB from the CMOS (which
>   // can only express a size smaller than 1TB), and add it to 4GB.
>   //
>   Status = ScanOrAdd64BitE820Ram (&FirstNonAddress);
>   if (EFI_ERROR (Status)) {
>     FirstNonAddress = BASE_4GB + GetSystemMemorySizeAbove4gb ();
>   }
>
> ...
>
>   //
>   // The "etc/reserved-memory-end" fw_cfg file, when present, contains an
>   // absolute, exclusive end address for the memory hotplug area. This area
>   // starts right at the end of the memory above 4GB. The 64-bit PCI host
>   // aperture must be placed above it.
>   //
>   Status = QemuFwCfgFindFile ("etc/reserved-memory-end", &FwCfgItem,
>              &FwCfgSize);
>   if (!EFI_ERROR (Status) && FwCfgSize == sizeof HotPlugMemoryEnd) {
>     QemuFwCfgSelectItem (FwCfgItem);
>     QemuFwCfgReadBytes (FwCfgSize, &HotPlugMemoryEnd);
>
>     ASSERT (HotPlugMemoryEnd >= FirstNonAddress);
>     FirstNonAddress = HotPlugMemoryEnd;
>   }

Can you check if the ASSERT fires for you?

For that, please capture the OVMF debug log like this:

(1) build OVMF with the following flag:

  --pcd=gEfiMdePkgTokenSpaceGuid.PcdDebugPrintErrorLevel=0x8040004F

(2) modify your domain XML as follows:

  <domain type='kvm' xmlns:qemu='http://libvirt.org/schemas/domain/qemu/1.0'>
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^   <--- don't forget this attribute!
    <qemu:commandline>
      <qemu:arg value='-global'/>
      <qemu:arg value='isa-debugcon.iobase=0x402'/>
      <qemu:arg value='-debugcon'/>
      <qemu:arg value='file:/tmp/ovmf.log'/>
    </qemu:commandline>
  </domain>

Then the file "/tmp/ovmf.log" should contain a verbose OVMF log --
please attach it.

Thanks
Laszlo