Re: [edk2-devel] [PATCH V1 1/1] UefiCpuPkg/ResetVector: Cache Disable should not be set by default in CR0

["Laszlo Ersek" <lersek@redhat.com>]

On 1/23/24 17:11, Gerd Hoffmann wrote:
>   Hi,
> 
>>>>> Well, it's OVMF in a virtual machine.  No boot guard involved.
>>>>> So we could probably go for a OVMF-specific patch here.
>>>>>
>>>>> But I'd prefer to figure what exactly is happening here before going
>>>>> down that route.  An extreme slowdown just because we flip that bit
>>>>> doesn't make sense to me.
>>>>>
>>>>>> Why is boot time increasing?
>>>>>
>>>>> Not clear.  It seems to be the lzma uncompress of the firmware volume
>>>>> in rom / pflash which is very slow.  Also it is apparently only
>>>>> triggered in case pci device assignment is used.
>>>>
>>>> I've seen extreme slowness on physical platforms when we've mixed up the
>>>> MTRRs or page tables, causing code to be mapped uncached.
>>>>
>>>> Lzma uncompress of ROM could be pretty slow as well, if the ROM is being
>>>> read uncached.  Lzma probably reads the data a byte at a time, which is the
>>>> worst case for uncached accesses.  Since this is a VM, it's not actually
>>>> uncached at the hardware level, but I don't know how QEMU/KVM handles
>>>> uncached guest mappings.... It may be doing a VMEXIT for every byte.
>>>>
>>>> Anyway, I suggest double-checking your page tables and MTRRs.
>>>
>>> It happens very early at boot, before MTRRs are setup, running on the
>>> initial page tables created by the OVMF reset vector.  The initial page
>>> tables have just 'accessed', 'dirty', 'read/write' and 'present' bits
>>> set for the 0-4G identity mapping.
>>>
>>> It seems to have something to do with EPT.  It does not happen on AMD
>>> processors.  It also does not happen when disabling EPT support in kvm
>>> on the host machine.
>>>
>>> looked at kvm kernel traces, I don't see excessive vmexits.
>>
>> This discussion evokes vague memories in me. I'll dump them here, but I
>> have no idea if they will be useful. (They probably won't.)
>>
>> - edk2 commit 98f378a7be12 ("OvmfPkg/ResetVector: enable caching in
>> initial page tables", 2013-09-24)
>>
>> - Linux (host) commit 879ae1880449 ("KVM: x86: obey
>> KVM_X86_QUIRK_CD_NW_CLEARED in kvm_set_cr0()", 2015-11-04)
> 
> I actually waded through the source code in both places ;)
> 
> Turned out kvm propagates guest MTRR settings to EPT memory types,
> but only in case kvm_arch_has_noncoherent_dma() is true, which why
> this triggers only with a mdev device assigned.

... I should not have stopped myself. :)

(I'm aware that this is on edk2-devel, but a public reference to
virt-staff cannot hurt.)

So, yesterday I read your status on virt-staff, and I found an entry in
it that resembled this upstream thread pretty closely. However, your
status was the *only* mention of "mdev" specifically, and so I wasn't
sure if *mdev* meant the same thing as the more generic upstream
expression "pci device assignment" (see it above in the context).

Furthermore, I saw kvm_arch_has_noncoherent_dma() in my linux commit
879ae1880449, which superficially resembled device assignment, but... I
dismissed it. In the end, I only managed (and even that, only
reluctantly) the above pointers... Thanks for tracking it down!

But then, next question: why has this problem *not* been reported
repeatedly? There's a whole bunch of users (gamers) that run Windows
guests with device (GPU) assignment. I'm sure they'd absolutely complain
about very slow OVMF boot (like they actually have, in the past, about
similar LZMA slowdowns due to improper caching setup).

Something must be special about Min's assigned device.

Laszlo



-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.
View/Reply Online (#114285): https://edk2.groups.io/g/devel/message/114285
Mute This Topic: https://groups.io/mt/100367559/7686176
Group Owner: devel+owner@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/leave/12367111/7686176/1913456212/xyzzy [rebecca@openfw.io]
-=-=-=-=-=-=-=-=-=-=-=-