Re: [edk2-devel] [RFC PATCH 01/28] OvmfPkg/Sec: Enable cache early to speed up booting

["Jordan Justen" <jordan.l.justen@intel.com>]

On 2019-08-22 06:46:07, Laszlo Ersek wrote:
> On 08/21/19 23:51, Jordan Justen wrote:
> > On 2019-08-21 07:21:25, Laszlo Ersek wrote:
> >> On 08/19/19 23:35, Lendacky, Thomas wrote:
> >>> From: Tom Lendacky <thomas.lendacky@amd.com>
> >>>
> >>> +  //
> >>> +  // Enable caching
> >>> +  //
> >>> +  AsmEnableCache ();
> >>> +
> >>>    DEBUG ((EFI_D_INFO,
> >>>      "SecCoreStartupWithStack(0x%x, 0x%x)\n",
> >>>      (UINT32)(UINTN)BootFv,
> >>>
> >>
> >> This makes me uncomfortable. There used to be problems related to
> >> caching when VFIO device assignment were used. My concern is admittedly
> >> vague, but this is a very brittle area of OVMF-on-KVM. If you asked me
> >> "well what could break here", I'd answer "you never know, and the burden
> >> of proof is not on me". :) Can we make this change conditional on SEV-ES?
> > 
> > This was also raised as an issue by Peter for the ACRN hypervisor and
> > Scott for the bhyve hypervisor.
> > 
> > I think it is rare for a platform to enable cache at this early of a
> > stage, but it is also rare to decompress a firmware volume at this
> > point.
> > 
> > It appears that it could be helpful to figure out how to safely enable
> > cache by default here, since it does seem to be impacting several
> > hypervisors.
> 
> I can't think of anything better than "trial and error".

Maybe we could try to detect kvm, and enable caching if !kvm.

Maybe we could enable it during the decompress of the PEI FV and
disable it afterward?

> The issues that
> used to pop up in the past, due to host kernel (KVM) changes,
> particularly in connection with VFIO device assignment, have been
> completely obscure and unpenetrable to me.

Don't we eventually enable caching during the boot, so how is VFIO not
affected by that?

> Even though I've contributed
> at least one KVM patch to mitigate those problems, they remain a mistery
> to me, and I remain unable to *reason* about the problems or the fixes.

If VFIO requires uncached access, then what mechanisms does kvm
support for accessing memory ranges uncached? I thought kvm simply
ignored the cache setting and always enabled caching, because this
section of boot is not particularly slow with kvm. But, if enabling
caching causes issues, then I guess it does something.

Does kvm support mtrr to uncache i/o ranges? I didn't think kvm
supported mtrrs in the past.

I hope we wouldn't have to use paging to disable caching for the
affected regions.

-Jordan

> So I think we could only flip the behavior (enable cache by default) and
> collect bug reports. But that's extremely annoying for end-users, and I
> see "no regressions" as one of my top responsibilities.
> 
> Even if we provided an fw_cfg knob to disable the change, using
> QemuFwCfgSecLib, similarly to commit ab081a50e565 ("OvmfPkg:
> PlatformPei: take no-exec DXE settings from the QEMU command line",
> 2015-09-15), such fw_cfg knobs are difficult to use through layered
> products, such as libvirt, proxmox, etc. And of course fw_cfg is only
> available on QEMU.
> 
> Laszlo