Re: [edk2-devel] [BUG] Extremely slow boot times with CPU and GPU passthrough and host phys-bits > 40

["Brian J. Johnson via groups.io" <brian.johnson=hpe.com@groups.io>]

On 11/22/24 04:35, Gerd Hoffmann via groups.io wrote:
> On Thu, Nov 21, 2024 at 04:23:32PM -0800, mitchell.augustin via groups.io wrote:
>> Thanks for the suggestions, Gerd.
>>
>>> '-global ICH9-LPC.acpi-pci-hotplug-with-bridge-support=off'
>>
>> Unfortunatly, this did not result in any change in boot time.
>>
>>> '-cpu $name,host-phys-bits=on,host-phys-bits-limit=$bits'
>>
>> -cpu host,host-phys-bits=on,host-phys-bits-limit=40 does result in the desired very fast boot time, but I get errors when I try to load the GPU driver:
>>
>> NVRM: BAR0 is 0M @ 0x0 (PCI:0000:06:00.0)
> 
> OVMF scales the pcie root port bridge windows with the address space.
> 
> So you can try increase host-phys-bits-limit.  Values between 40 and 46
> should have an effect.
> 
> The opt/ovmf/X-PciMmio64Mb setting already mentioned elsewhere in this
> thread should also work with the dynamic mmio window.
> 
>> * *I ask this because I notice that when I disable host passthrough *(which works with my GPUs and results in a fast boot if `pci=realloc pci=nocrs` are used)* , my VM shows "Address sizes: 40 bits physical, 48 bits virtual", but when I use host passthrough and only override the physbits (which fails as described above), my VM shows "Address sizes: 40 bits physical, 57 bits virtual". (so I'm wondering if that could be a factor here.)*
> 
> It's 5-level paging support, and the qemu option name for that is la57
> (-cpu $name,la57={on|off})
> 
> take care,
>    Gerd

I'm not familiar with passing through devices, but I'll mention that 
900+ GB of memory will take 40 PA bits just for that, with little room 
left above the RAM for PCI BARs in MMIOH.  So I wouldn't be surprised if 
you need more PA bits, which would push you into 5-level paging.

Gerd's suggestion to check if the MTRRs and page attributes are getting 
set properly is a good one.  That has caused this type of problem in the 
past.
-- 
Brian J. Johnson
Hewlett Packard Enterprise

brian.johnson@hpe.com


-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.
View/Reply Online (#120818): https://edk2.groups.io/g/devel/message/120818
Mute This Topic: https://groups.io/mt/109651206/7686176
Group Owner: devel+owner@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub [rebecca@openfw.io]
-=-=-=-=-=-=-=-=-=-=-=-