Re: [edk2-devel] [BUG] Extremely slow boot times with CPU and GPU passthrough and host phys-bits > 40

["mitchell.augustin via groups.io" <mitchell.augustin=canonical.com@groups.io>]

[-- Attachment #1.1: Type: text/plain, Size: 2269 bytes --]

Hi,

Thanks for the suggestion. I'm not necessarily saying this patch itself has an issue, just that it is the point in the git history at which this slow boot time issue manifests for us. This may be because the patch does actually fix the other issue I described above related to BAR assignment not working correctly in versions before that patch, despite boot being faster back then. (in those earlier versions, the PCI devices for the GPUs were passed through, but the BAR assignment was erroneous, so we couldn't actually use them - the Nvidia GPU driver would just throw errors.)

After I initially posted here, we also discovered another kernel issue that was contributing to the boot times for this config exceeding 5 minutes - so with that isolated, I can say that my config only takes about a 5 minutes for a full boot: 1-2 minutes for `virsh start` (which scales with guest memory allocation), and about 2-3 minutes of time spent on PCIe initialization / BAR assignment for 2 to 4 GPUs (attached). This was still the case when I tried with my GPUs attached in the way you suggested. I'll attach the xml config for that and for my original VM in case I may have configured something incorrectly there.

With that said, I have a more basic question - do you expect that it should take upwards of 30 seconds after `virsh start` completes before I see any output in `virsh console`, or that PCI devices' memory window assignments in the VM should take 45-90 seconds per passed-through GPU? (given that when the same kernel on the host initializes these devices, it doesn't take nearly this long?)

I'm going to attempt to profile ovmf next to see what part of the code path is taking up the most time, but if you already have an idea of what that might be (and whether it is actually a bug or expected to take that long), that insight would be appreciated.

Thanks,
Mitchell Augustin


-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.
View/Reply Online (#120801): https://edk2.groups.io/g/devel/message/120801
Mute This Topic: https://groups.io/mt/109651206/7686176
Group Owner: devel+owner@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub [rebecca@openfw.io]
-=-=-=-=-=-=-=-=-=-=-=-



[-- Attachment #1.2: Type: text/html, Size: 3129 bytes --]

[-- Attachment #2: 4gpu-virsh-pxb-pcie.txt --]
[-- Type: text/plain, Size: 14008 bytes --]

<domain type='kvm' id='2'>
  <name>4gpu-vm-2</name>
  <uuid>e3dca4cb-028b-4ca2-8bf7-f87a394551db</uuid>
  <metadata>
    <libosinfo:libosinfo xmlns:libosinfo="http://libosinfo.org/xmlns/libvirt/domain/1.0">
      <libosinfo:os id="http://ubuntu.com/ubuntu/22.04"/>
    </libosinfo:libosinfo>
  </metadata>
  <memory unit='KiB'>966262784</memory>
  <currentMemory unit='KiB'>966262784</currentMemory>
  <vcpu placement='static'>16</vcpu>
  <iothreads>1</iothreads>
  <iothreadids>
    <iothread id='1'/>
  </iothreadids>
  <cputune>
    <vcpupin vcpu='0' cpuset='16'/>
    <vcpupin vcpu='1' cpuset='128'/>
    <vcpupin vcpu='2' cpuset='17'/>
    <vcpupin vcpu='3' cpuset='129'/>
    <vcpupin vcpu='4' cpuset='18'/>
    <vcpupin vcpu='5' cpuset='130'/>
    <vcpupin vcpu='6' cpuset='19'/>
    <vcpupin vcpu='7' cpuset='131'/>
    <vcpupin vcpu='8' cpuset='20'/>
    <vcpupin vcpu='9' cpuset='132'/>
    <vcpupin vcpu='10' cpuset='21'/>
    <vcpupin vcpu='11' cpuset='133'/>
    <vcpupin vcpu='12' cpuset='22'/>
    <vcpupin vcpu='13' cpuset='134'/>
    <vcpupin vcpu='14' cpuset='23'/>
    <vcpupin vcpu='15' cpuset='135'/>
    <emulatorpin cpuset='55,167'/>
    <iothreadpin iothread='1' cpuset='54,166'/>
  </cputune>
  <numatune>
    <memory mode='strict' nodeset='0'/>
  </numatune>
  <resource>
    <partition>/machine</partition>
  </resource>
  <os firmware='efi'>
    <type arch='x86_64' machine='pc-q35-8.2'>hvm</type>
    <firmware>
      <feature enabled='no' name='enrolled-keys'/>
      <feature enabled='no' name='secure-boot'/>
    </firmware>
    <loader readonly='yes' type='pflash'>/usr/share/OVMF/OVMF_CODE_4M.fd</loader>
    <nvram template='/usr/share/OVMF/OVMF_VARS_4M.fd'>/var/lib/libvirt/qemu/nvram/4gpu-vm-2_VARS.fd</nvram>
    <boot dev='hd'/>
  </os>
  <features>
    <acpi/>
    <apic/>
  </features>
  <cpu mode='host-passthrough' check='none' migratable='on'>
    <cache mode='passthrough'/>
    <numa>
      <cell id='0' cpus='0-15' memory='966262784' unit='KiB'/>
    </numa>
  </cpu>
  <clock offset='utc'>
    <timer name='rtc' tickpolicy='catchup'/>
    <timer name='pit' tickpolicy='delay'/>
    <timer name='hpet' present='no'/>
  </clock>
  <on_poweroff>destroy</on_poweroff>
  <on_reboot>restart</on_reboot>
  <on_crash>destroy</on_crash>
  <pm>
    <suspend-to-mem enabled='no'/>
    <suspend-to-disk enabled='no'/>
  </pm>
  <devices>
    <emulator>/usr/bin/qemu-system-x86_64</emulator>
    <disk type='file' device='disk'>
      <driver name='qemu' type='qcow2'/>
      <source file='/vms/testbox01-vda.qcow2' index='2'/>
      <backingStore type='file' index='3'>
        <format type='qcow2'/>
        <source file='/vms/jammy-server-cloudimg-amd64.img'/>
        <backingStore/>
      </backingStore>
      <target dev='vda' bus='virtio'/>
      <alias name='virtio-disk0'/>
      <address type='pci' domain='0x0000' bus='0x04' slot='0x00' function='0x0'/>
    </disk>
    <disk type='file' device='disk'>
      <driver name='qemu' type='qcow2' iothread='1' queues='16'/>
      <source file='/vms/testbox01-seed.qcow2' index='1'/>
      <backingStore/>
      <target dev='vdb' bus='virtio'/>
      <alias name='virtio-disk1'/>
      <address type='pci' domain='0x0000' bus='0x05' slot='0x00' function='0x0'/>
    </disk>
    <controller type='usb' index='0' model='qemu-xhci' ports='15'>
      <alias name='usb'/>
      <address type='pci' domain='0x0000' bus='0x02' slot='0x00' function='0x0'/>
    </controller>
    <controller type='pci' index='0' model='pcie-root'>
      <alias name='pcie.0'/>
    </controller>
    <controller type='pci' index='1' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='1' port='0x8'/>
      <alias name='pci.1'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x0' multifunction='on'/>
    </controller>
    <controller type='pci' index='2' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='2' port='0x9'/>
      <alias name='pci.2'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x1'/>
    </controller>
    <controller type='pci' index='3' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='3' port='0xa'/>
      <alias name='pci.3'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x2'/>
    </controller>
    <controller type='pci' index='4' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='4' port='0xb'/>
      <alias name='pci.4'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x3'/>
    </controller>
    <controller type='pci' index='5' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='5' port='0xc'/>
      <alias name='pci.5'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x4'/>
    </controller>
    <controller type='pci' index='6' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='6' port='0xd'/>
      <alias name='pci.6'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x5'/>
    </controller>
    <controller type='pci' index='7' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='7' port='0xe'/>
      <alias name='pci.7'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x6'/>
    </controller>
    <controller type='pci' index='8' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='8' port='0xf'/>
      <alias name='pci.8'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x7'/>
    </controller>
    <controller type='pci' index='9' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='9' port='0x10'/>
      <alias name='pci.9'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0' multifunction='on'/>
    </controller>
    <controller type='pci' index='10' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='10' port='0x11'/>
      <alias name='pci.10'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x1'/>
    </controller>
    <controller type='pci' index='11' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='11' port='0x12'/>
      <alias name='pci.11'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x2'/>
    </controller>
    <controller type='pci' index='12' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='12' port='0x13'/>
      <alias name='pci.12'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x3'/>
    </controller>
    <controller type='pci' index='13' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='13' port='0x14'/>
      <alias name='pci.13'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x4'/>
    </controller>
    <controller type='pci' index='14' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='14' port='0x15'/>
      <alias name='pci.14'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x5'/>
    </controller>
    <controller type='pci' index='15' model='pcie-expander-bus'>
      <model name='pxb-pcie'/>
      <target busNr='64'>
        <node>0</node>
      </target>
      <alias name='pci.15'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x0f' function='0x0'/>
    </controller>
    <controller type='pci' index='16' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='16' port='0x17'/>
      <alias name='pci.16'/>
      <address type='pci' domain='0x0000' bus='0x0f' slot='0x01' function='0x5'/>
    </controller>
    <controller type='pci' index='17' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='17' port='0x18'/>
      <alias name='pci.17'/>
      <address type='pci' domain='0x0000' bus='0x0f' slot='0x01' function='0x6'/>
    </controller>
    <controller type='pci' index='18' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='18' port='0x19'/>
      <alias name='pci.18'/>
      <address type='pci' domain='0x0000' bus='0x0f' slot='0x01' function='0x7'/>
    </controller>
    <controller type='pci' index='19' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='19' port='0x20'/>
      <alias name='pci.19'/>
      <address type='pci' domain='0x0000' bus='0x0f' slot='0x02' function='0x0' multifunction='on'/>
    </controller>
    <controller type='pci' index='20' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='20' port='0x16'/>
      <alias name='pci.20'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x6'/>
    </controller>
    <controller type='pci' index='21' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='21' port='0x17'/>
      <alias name='pci.21'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x7'/>
    </controller>
    <controller type='pci' index='22' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='22' port='0x18'/>
      <alias name='pci.22'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0' multifunction='on'/>
    </controller>
    <controller type='pci' index='23' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='23' port='0x19'/>
      <alias name='pci.23'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x1'/>
    </controller>
    <controller type='pci' index='24' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='24' port='0x1a'/>
      <alias name='pci.24'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x2'/>
    </controller>
    <controller type='pci' index='25' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='25' port='0x1b'/>
      <alias name='pci.25'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x3'/>
    </controller>
    <controller type='sata' index='0'>
      <alias name='ide'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x1f' function='0x2'/>
    </controller>
    <controller type='virtio-serial' index='0'>
      <alias name='virtio-serial0'/>
      <address type='pci' domain='0x0000' bus='0x03' slot='0x00' function='0x0'/>
    </controller>
    <interface type='network'>
      <mac address='52:54:00:31:c6:77'/>
      <source network='default' portid='7475442e-da4f-4f83-9eb1-9a7e15107c07' bridge='virbr0'/>
      <target dev='vnet1'/>
      <model type='virtio'/>
      <alias name='net0'/>
      <address type='pci' domain='0x0000' bus='0x01' slot='0x00' function='0x0'/>
    </interface>
    <serial type='pty'>
      <target type='isa-serial' port='0'>
        <model name='isa-serial'/>
      </target>
      <alias name='serial0'/>
    </serial>
    <console type='pty'>
      <target type='serial' port='0'/>
      <alias name='serial0'/>
    </console>
    <channel type='unix'>
      <source mode='bind' path='/run/libvirt/qemu/channel/2-4gpu-vm-2/org.qemu.guest_agent.0'/>
      <target type='virtio' name='org.qemu.guest_agent.0'/>
      <alias name='channel0'/>
      <address type='virtio-serial' controller='0' bus='0' port='1'/>
    </channel>
    <input type='mouse' bus='ps2'>
      <alias name='input0'/>
    </input>
    <input type='keyboard' bus='ps2'>
      <alias name='input1'/>
    </input>
    <tpm model='tpm-crb'>
      <backend type='emulator' version='2.0'/>
      <alias name='tpm0'/>
    </tpm>
    <audio id='1' type='none'/>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x1b' slot='0x00' function='0x0'/>
      </source>
      <alias name='hostdev0'/>
      <address type='pci' domain='0x0000' bus='0x16' slot='0x00' function='0x0'/>
    </hostdev>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x61' slot='0x00' function='0x0'/>
      </source>
      <alias name='hostdev1'/>
      <address type='pci' domain='0x0000' bus='0x17' slot='0x00' function='0x0'/>
    </hostdev>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0xc3' slot='0x00' function='0x0'/>
      </source>
      <alias name='hostdev2'/>
      <address type='pci' domain='0x0000' bus='0x18' slot='0x00' function='0x0'/>
    </hostdev>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0xdf' slot='0x00' function='0x0'/>
      </source>
      <alias name='hostdev3'/>
      <address type='pci' domain='0x0000' bus='0x19' slot='0x00' function='0x0'/>
    </hostdev>
    <watchdog model='itco' action='reset'>
      <alias name='watchdog0'/>
    </watchdog>
    <memballoon model='virtio'>
      <alias name='balloon0'/>
      <address type='pci' domain='0x0000' bus='0x0a' slot='0x00' function='0x0'/>
    </memballoon>
    <rng model='virtio'>
      <backend model='random'>/dev/urandom</backend>
      <alias name='rng0'/>
      <address type='pci' domain='0x0000' bus='0x0b' slot='0x00' function='0x0'/>
    </rng>
  </devices>
  <seclabel type='dynamic' model='apparmor' relabel='yes'>
    <label>libvirt-e3dca4cb-028b-4ca2-8bf7-f87a394551db</label>
    <imagelabel>libvirt-e3dca4cb-028b-4ca2-8bf7-f87a394551db</imagelabel>
  </seclabel>
  <seclabel type='dynamic' model='dac' relabel='yes'>
    <label>+64055:+993</label>
    <imagelabel>+64055:+993</imagelabel>
  </seclabel>
</domain>


[-- Attachment #3: 2gpu-virsh-pxb-pcie.txt --]
[-- Type: text/plain, Size: 8122 bytes --]

<domain type='kvm'>
  <name>numa</name>
  <uuid>9fd1c950-a8cd-4459-8775-5fa918507d8c</uuid>
  <metadata>
    <libosinfo:libosinfo xmlns:libosinfo="http://libosinfo.org/xmlns/libvirt/domain/1.0">
      <libosinfo:os id="http://ubuntu.com/ubuntu/22.04"/>
    </libosinfo:libosinfo>
  </metadata>
  <memory unit='KiB'>134217728</memory>
  <currentMemory unit='KiB'>134217728</currentMemory>
  <vcpu placement='static' cpuset='0-39,56-95'>80</vcpu>
  <os>
    <type arch='x86_64' machine='pc-q35-6.2'>hvm</type>
    <loader readonly='yes' type='pflash'>/usr/share/OVMF/OVMF_NEW.fd</loader>
    <nvram>/var/lib/libvirt/qemu/nvram/numa_VARS.fd</nvram>
    <boot dev='hd'/>
  </os>
  <features>
    <acpi/>
    <apic/>
  </features>
  <cpu mode='host-passthrough' check='none' migratable='on'>
    <cache mode='passthrough'/>
    <numa>
      <cell id='0' cpus='0-39' memory='67108864' unit='KiB'/>
      <cell id='1' cpus='40-79' memory='67108864' unit='KiB'/>
    </numa>
  </cpu>
  <clock offset='utc'>
    <timer name='rtc' tickpolicy='catchup'/>
    <timer name='pit' tickpolicy='delay'/>
    <timer name='hpet' present='no'/>
  </clock>
  <on_poweroff>destroy</on_poweroff>
  <on_reboot>restart</on_reboot>
  <on_crash>destroy</on_crash>
  <pm>
    <suspend-to-mem enabled='no'/>
    <suspend-to-disk enabled='no'/>
  </pm>
  <devices>
    <emulator>/usr/bin/qemu-system-x86_64</emulator>
    <disk type='file' device='disk'>
      <driver name='qemu' type='qcow2'/>
      <source file='/vms/numa/numa-vda.qcow2'/>
      <backingStore type='file'>
        <format type='qcow2'/>
        <source file='/vms/images/jammy-server-cloudimg-amd64.img'/>
        <backingStore/>
      </backingStore>
      <target dev='vda' bus='virtio'/>
      <address type='pci' domain='0x0000' bus='0x06' slot='0x00' function='0x0'/>
    </disk>
    <controller type='usb' index='0' model='qemu-xhci' ports='15'>
      <address type='pci' domain='0x0000' bus='0x04' slot='0x00' function='0x0'/>
    </controller>
    <controller type='pci' index='0' model='pcie-root'/>
    <controller type='pci' index='1' model='pcie-expander-bus'>
      <model name='pxb-pcie'/>
      <target busNr='128'>
        <node>1</node>
      </target>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x0'/>
    </controller>
    <controller type='pci' index='2' model='pcie-expander-bus'>
      <model name='pxb-pcie'/>
      <target busNr='64'>
        <node>0</node>
      </target>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>
    </controller>
    <controller type='pci' index='3' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='3' port='0x10'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0' multifunction='on'/>
    </controller>
    <controller type='pci' index='4' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='4' port='0x11'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x1'/>
    </controller>
    <controller type='pci' index='5' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='5' port='0x12'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x2'/>
    </controller>
    <controller type='pci' index='6' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='6' port='0x13'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x3'/>
    </controller>
    <controller type='pci' index='7' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='7' port='0x14'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x4'/>
    </controller>
    <controller type='pci' index='8' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='8' port='0x15'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x5'/>
    </controller>
    <controller type='pci' index='9' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='9' port='0x16'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x6'/>
    </controller>
    <controller type='pci' index='10' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='10' port='0x17'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x7'/>
    </controller>
    <controller type='pci' index='11' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='11' port='0x18'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/>
    </controller>
    <controller type='pci' index='12' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='12' port='0x19'/>
      <address type='pci' domain='0x0000' bus='0x01' slot='0x00' function='0x0' multifunction='on'/>
    </controller>
    <controller type='pci' index='13' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='13' port='0x1a'/>
      <address type='pci' domain='0x0000' bus='0x01' slot='0x00' function='0x1'/>
    </controller>
    <controller type='pci' index='14' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='14' port='0x1b'/>
      <address type='pci' domain='0x0000' bus='0x02' slot='0x00' function='0x0' multifunction='on'/>
    </controller>
    <controller type='pci' index='15' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='15' port='0x1c'/>
      <address type='pci' domain='0x0000' bus='0x02' slot='0x00' function='0x1'/>
    </controller>
    <controller type='sata' index='0'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x1f' function='0x2'/>
    </controller>
    <controller type='virtio-serial' index='0'>
      <address type='pci' domain='0x0000' bus='0x05' slot='0x00' function='0x0'/>
    </controller>
    <interface type='network'>
      <mac address='52:54:00:a2:fa:f7'/>
      <source network='default'/>
      <model type='virtio'/>
      <address type='pci' domain='0x0000' bus='0x03' slot='0x00' function='0x0'/>
    </interface>
    <serial type='pty'>
      <target type='isa-serial' port='0'>
        <model name='isa-serial'/>
      </target>
    </serial>
    <console type='pty'>
      <target type='serial' port='0'/>
    </console>
    <channel type='unix'>
      <target type='virtio' name='org.qemu.guest_agent.0'/>
      <address type='virtio-serial' controller='0' bus='0' port='1'/>
    </channel>
    <input type='tablet' bus='usb'>
      <address type='usb' bus='0' port='1'/>
    </input>
    <input type='mouse' bus='ps2'/>
    <input type='keyboard' bus='ps2'/>
    <tpm model='tpm-crb'>
      <backend type='emulator' version='2.0'/>
    </tpm>
    <graphics type='vnc' port='-1' autoport='yes' listen='0.0.0.0'>
      <listen type='address' address='0.0.0.0'/>
    </graphics>
    <audio id='1' type='none'/>
    <video>
      <model type='bochs' vram='16384' heads='1' primary='yes'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/>
    </video>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x52' slot='0x00' function='0x0'/>
      </source>
      <address type='pci' domain='0x0000' bus='0x0c' slot='0x00' function='0x0'/>
    </hostdev>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x61' slot='0x00' function='0x0'/>
      </source>
      <address type='pci' domain='0x0000' bus='0x0e' slot='0x00' function='0x0'/>
    </hostdev>
    <memballoon model='virtio'>
      <address type='pci' domain='0x0000' bus='0x08' slot='0x00' function='0x0'/>
    </memballoon>
    <rng model='virtio'>
      <backend model='random'>/dev/urandom</backend>
      <address type='pci' domain='0x0000' bus='0x09' slot='0x00' function='0x0'/>
    </rng>
  </devices>
</domain>

[-- Attachment #4: this-part-takes-2-3-minutes.txt --]
[-- Type: text/plain, Size: 8493 bytes --]

[    4.783965] PCI host bridge to bus 0000:00
[    4.784357] pci_bus 0000:00: root bus resource [bus 00-ff]
[    4.784877] pci_bus 0000:00: root bus resource [io  0x0000-0x0cf7 window]
[    4.785524] pci_bus 0000:00: root bus resource [io  0x0d00-0xffff window]
[    4.786169] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]
[    4.786193] pci_bus 0000:00: root bus resource [mem 0x80000000-0xdfffffff window]
[    4.786906] pci_bus 0000:00: root bus resource [mem 0xf0000000-0xfebfffff window]
[    4.787615] pci_bus 0000:00: root bus resource [mem 0x380000000000-0x386fffffffff window]
[    4.788455] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000
[    4.789650] pci 0000:00:01.0: [1b36:000c] type 01 class 0x060400
[    4.793329] pci 0000:00:01.0: reg 0x10: [mem 0x81c8e000-0x81c8efff]
[    4.798179] pci 0000:00:01.1: [1b36:000c] type 01 class 0x060400
[    4.800468] pci 0000:00:01.1: reg 0x10: [mem 0x81c8d000-0x81c8dfff]
[    4.805304] pci 0000:00:01.2: [1b36:000c] type 01 class 0x060400
[    4.807075] pci 0000:00:01.2: reg 0x10: [mem 0x81c8c000-0x81c8cfff]
[    4.813525] pci 0000:00:01.3: [1b36:000c] type 01 class 0x060400
[    4.815140] pci 0000:00:01.3: reg 0x10: [mem 0x81c8b000-0x81c8bfff]
[    4.818208] pci 0000:00:01.4: [1b36:000c] type 01 class 0x060400
[    4.820029] pci 0000:00:01.4: reg 0x10: [mem 0x81c8a000-0x81c8afff]
[    4.826976] pci 0000:00:01.5: [1b36:000c] type 01 class 0x060400
[    4.828745] pci 0000:00:01.5: reg 0x10: [mem 0x81c89000-0x81c89fff]
[    4.831537] pci 0000:00:01.6: [1b36:000c] type 01 class 0x060400
[    4.833289] pci 0000:00:01.6: reg 0x10: [mem 0x81c88000-0x81c88fff]
[    4.838964] pci 0000:00:01.7: [1b36:000c] type 01 class 0x060400
[    4.840706] pci 0000:00:01.7: reg 0x10: [mem 0x81c87000-0x81c87fff]
[    4.843826] pci 0000:00:02.0: [1b36:000c] type 01 class 0x060400
[    4.845512] pci 0000:00:02.0: reg 0x10: [mem 0x81c86000-0x81c86fff]
[    4.850960] pci 0000:00:02.1: [1b36:000c] type 01 class 0x060400
[    4.852667] pci 0000:00:02.1: reg 0x10: [mem 0x81c85000-0x81c85fff]
[    4.855532] pci 0000:00:02.2: [1b36:000c] type 01 class 0x060400
[    4.857172] pci 0000:00:02.2: reg 0x10: [mem 0x81c84000-0x81c84fff]
[    4.862296] pci 0000:00:02.3: [1b36:000c] type 01 class 0x060400
[    4.864129] pci 0000:00:02.3: reg 0x10: [mem 0x81c83000-0x81c83fff]
[    4.867013] pci 0000:00:02.4: [1b36:000c] type 01 class 0x060400
[    4.868680] pci 0000:00:02.4: reg 0x10: [mem 0x81c82000-0x81c82fff]
[    4.871544] pci 0000:00:02.5: [1b36:000c] type 01 class 0x060400
[    4.875434] pci 0000:00:02.5: reg 0x10: [mem 0x81c81000-0x81c81fff]
[    4.888520] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100
[    4.889440] pci 0000:00:1f.0: quirk: [io  0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO
[    4.890380] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601
[    4.894195] pci 0000:00:1f.2: reg 0x20: [io  0x6040-0x605f]
[    4.895386] pci 0000:00:1f.2: reg 0x24: [mem 0x81c80000-0x81c80fff]
[    4.898909] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500
[    4.901232] pci 0000:00:1f.3: reg 0x20: [io  0x6000-0x603f]
[    4.903078] acpiphp: Slot [0] registered
[    4.903583] pci 0000:01:00.0: [1af4:1041] type 00 class 0x020000
[    4.905510] pci 0000:01:00.0: reg 0x14: [mem 0x81a00000-0x81a00fff]
[    4.908440] pci 0000:01:00.0: reg 0x20: [mem 0x380000000000-0x380000003fff 64bit pref]
[    4.910982] pci 0000:01:00.0: reg 0x30: [mem 0xfff80000-0xffffffff pref]
[    4.921546] pci 0000:00:01.0: PCI bridge to [bus 01]
[    4.922044] pci 0000:00:01.0:   bridge window [mem 0x81a00000-0x81bfffff]
[    4.922212] pci 0000:00:01.0:   bridge window [mem 0x380000000000-0x3807ffffffff 64bit pref]
[    4.923574] acpiphp: Slot [0-2] registered
[    4.924070] pci 0000:02:00.0: [1b36:000d] type 00 class 0x0c0330
[    4.925076] pci 0000:02:00.0: reg 0x10: [mem 0x81800000-0x81803fff 64bit]
[    4.937005] pci 0000:00:01.1: PCI bridge to [bus 02]
[    4.937527] pci 0000:00:01.1:   bridge window [mem 0x81800000-0x819fffff]
[    4.938211] pci 0000:00:01.1:   bridge window [mem 0x380800000000-0x380fffffffff 64bit pref]
[    4.939617] acpiphp: Slot [0-3] registered
[    4.940141] pci 0000:03:00.0: [1af4:1043] type 00 class 0x078000
[    4.942063] pci 0000:03:00.0: reg 0x14: [mem 0x81600000-0x81600fff]
[    4.944106] pci 0000:03:00.0: reg 0x20: [mem 0x381000000000-0x381000003fff 64bit pref]
[    4.955436] pci 0000:00:01.2: PCI bridge to [bus 03]
[    4.955965] pci 0000:00:01.2:   bridge window [mem 0x81600000-0x817fffff]
[    4.956659] pci 0000:00:01.2:   bridge window [mem 0x381000000000-0x3817ffffffff 64bit pref]
[    4.958098] acpiphp: Slot [0-4] registered
[    4.958295] pci 0000:04:00.0: [1af4:1042] type 00 class 0x010000
[    4.960222] pci 0000:04:00.0: reg 0x14: [mem 0x81400000-0x81400fff]
[    4.962536] pci 0000:04:00.0: reg 0x20: [mem 0x381800000000-0x381800003fff 64bit pref]
[    4.976182] pci 0000:00:01.3: PCI bridge to [bus 04]
[    4.976723] pci 0000:00:01.3:   bridge window [mem 0x81400000-0x815fffff]
[    4.977417] pci 0000:00:01.3:   bridge window [mem 0x381800000000-0x381fffffffff 64bit pref]
[    4.978724] acpiphp: Slot [0-5] registered
[    4.979241] pci 0000:05:00.0: [1af4:1042] type 00 class 0x010000
[    4.981201] pci 0000:05:00.0: reg 0x14: [mem 0x81200000-0x81200fff]
[    4.983803] pci 0000:05:00.0: reg 0x20: [mem 0x382000000000-0x382000003fff 64bit pref]
[    4.995249] pci 0000:00:01.4: PCI bridge to [bus 05]
[    4.995783] pci 0000:00:01.4:   bridge window [mem 0x81200000-0x813fffff]
[    4.996509] pci 0000:00:01.4:   bridge window [mem 0x382000000000-0x3827ffffffff 64bit pref]
[    4.998741] acpiphp: Slot [0-6] registered
[    4.999244] pci 0000:06:00.0: [1af4:1045] type 00 class 0x00ff00
[    5.003387] pci 0000:06:00.0: reg 0x20: [mem 0x382800000000-0x382800003fff 64bit pref]
[    5.014724] pci 0000:00:01.5: PCI bridge to [bus 06]
[    5.015227] pci 0000:00:01.5:   bridge window [mem 0x81000000-0x811fffff]
[    5.015921] pci 0000:00:01.5:   bridge window [mem 0x382800000000-0x382fffffffff 64bit pref]
[    5.017424] acpiphp: Slot [0-7] registered
[    5.017944] pci 0000:07:00.0: [1af4:1044] type 00 class 0x00ff00
[    5.019675] pci 0000:07:00.0: reg 0x14: [mem 0x80e00000-0x80e00fff]
[    5.024024] pci 0000:07:00.0: reg 0x20: [mem 0x383000000000-0x383000003fff 64bit pref]
[    5.035826] pci 0000:00:01.6: PCI bridge to [bus 07]
[    5.036343] pci 0000:00:01.6:   bridge window [mem 0x80e00000-0x80ffffff]
[    5.037041] pci 0000:00:01.6:   bridge window [mem 0x383000000000-0x3837ffffffff 64bit pref]
[    5.038317] acpiphp: Slot [0-8] registered
[    5.047620] pci 0000:00:01.7: PCI bridge to [bus 08]
[    5.048142] pci 0000:00:01.7:   bridge window [mem 0x80c00000-0x80dfffff]
[    5.048836] pci 0000:00:01.7:   bridge window [mem 0x383800000000-0x383fffffffff 64bit pref]
[    5.050222] acpiphp: Slot [0-9] registered
[    5.059539] pci 0000:00:02.0: PCI bridge to [bus 09]
[    5.060045] pci 0000:00:02.0:   bridge window [mem 0x80a00000-0x80bfffff]
[    5.060741] pci 0000:00:02.0:   bridge window [mem 0x384000000000-0x3847ffffffff 64bit pref]
[    5.062056] acpiphp: Slot [0-10] registered
[    5.071077] pci 0000:00:02.1: PCI bridge to [bus 0a]
[    5.071589] pci 0000:00:02.1:   bridge window [mem 0x80800000-0x809fffff]
[    5.072277] pci 0000:00:02.1:   bridge window [mem 0x384800000000-0x384fffffffff 64bit pref]
[    5.073620] acpiphp: Slot [0-11] registered
[    5.082936] pci 0000:00:02.2: PCI bridge to [bus 0b]
[    5.083454] pci 0000:00:02.2:   bridge window [mem 0x80600000-0x807fffff]
[    5.084144] pci 0000:00:02.2:   bridge window [mem 0x385000000000-0x3857ffffffff 64bit pref]
[    5.085489] acpiphp: Slot [0-12] registered
[    5.094805] pci 0000:00:02.3: PCI bridge to [bus 0c]
[    5.096290] pci 0000:00:02.3:   bridge window [mem 0x80400000-0x805fffff]
[    5.096988] pci 0000:00:02.3:   bridge window [mem 0x385800000000-0x385fffffffff 64bit pref]
[    5.098318] acpiphp: Slot [0-13] registered
[    5.107624] pci 0000:00:02.4: PCI bridge to [bus 0d]
[    5.108132] pci 0000:00:02.4:   bridge window [mem 0x80200000-0x803fffff]
[    5.108829] pci 0000:00:02.4:   bridge window [mem 0x386000000000-0x3867ffffffff 64bit pref]
[    5.110174] acpiphp: Slot [0-14] registered
[    5.119076] pci 0000:00:02.5: PCI bridge to [bus 0e]
[    5.119596] pci 0000:00:02.5:   bridge window [mem 0x80000000-0x801fffff]
[    5.120282] pci 0000:00:02.5:   bridge window [mem 0x386800000000-0x386fffffffff 64bit pref]