From mboxrd@z Thu Jan 1 00:00:00 1970 Authentication-Results: mx.groups.io; dkim=missing; spf=pass (domain: redhat.com, ip: 209.132.183.28, mailfrom: lersek@redhat.com) Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by groups.io with SMTP; Fri, 23 Aug 2019 06:32:46 -0700 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 1F5BE300CB6A; Fri, 23 Aug 2019 13:32:46 +0000 (UTC) Received: from lacos-laptop-7.usersys.redhat.com (ovpn-117-39.ams2.redhat.com [10.36.117.39]) by smtp.corp.redhat.com (Postfix) with ESMTP id C57462636D; Fri, 23 Aug 2019 13:32:42 +0000 (UTC) Subject: Re: [edk2-devel] [RFC PATCH 01/28] OvmfPkg/Sec: Enable cache early to speed up booting To: Jordan Justen , devel@edk2.groups.io, thomas.lendacky@amd.com Cc: Ard Biesheuvel , Michael D Kinney , Liming Gao , Eric Dong , Ray Ni , "Singh, Brijesh" , "Fang, Peter" , D Scott Phillips References: <6a37c84f4989304b21205d6263c6491f81da3233.1566250534.git.thomas.lendacky@amd.com> <6d3442d5-46ab-2b99-6100-0e5c56477735@redhat.com> <156642425970.26211.8321620974236559246@jljusten-skl> <9659350e-fb81-fa3a-d86a-5a76d73c3ce1@redhat.com> <156650666283.13750.10666423055754200944@jljusten-skl> From: "Laszlo Ersek" Message-ID: <04a030c5-6739-ab0f-9f1e-e87f6decb766@redhat.com> Date: Fri, 23 Aug 2019 15:32:41 +0200 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Thunderbird/52.9.1 MIME-Version: 1.0 In-Reply-To: <156650666283.13750.10666423055754200944@jljusten-skl> X-Scanned-By: MIMEDefang 2.84 on 10.5.11.23 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.46]); Fri, 23 Aug 2019 13:32:46 +0000 (UTC) Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 7bit On 08/22/19 22:44, Jordan Justen wrote: > On 2019-08-22 06:46:07, Laszlo Ersek wrote: >> On 08/21/19 23:51, Jordan Justen wrote: >>> On 2019-08-21 07:21:25, Laszlo Ersek wrote: >>>> On 08/19/19 23:35, Lendacky, Thomas wrote: >>>>> From: Tom Lendacky >>>>> >>>>> + // >>>>> + // Enable caching >>>>> + // >>>>> + AsmEnableCache (); >>>>> + >>>>> DEBUG ((EFI_D_INFO, >>>>> "SecCoreStartupWithStack(0x%x, 0x%x)\n", >>>>> (UINT32)(UINTN)BootFv, >>>>> >>>> >>>> This makes me uncomfortable. There used to be problems related to >>>> caching when VFIO device assignment were used. My concern is admittedly >>>> vague, but this is a very brittle area of OVMF-on-KVM. If you asked me >>>> "well what could break here", I'd answer "you never know, and the burden >>>> of proof is not on me". :) Can we make this change conditional on SEV-ES? >>> >>> This was also raised as an issue by Peter for the ACRN hypervisor and >>> Scott for the bhyve hypervisor. >>> >>> I think it is rare for a platform to enable cache at this early of a >>> stage, but it is also rare to decompress a firmware volume at this >>> point. >>> >>> It appears that it could be helpful to figure out how to safely enable >>> cache by default here, since it does seem to be impacting several >>> hypervisors. >> >> I can't think of anything better than "trial and error". > > Maybe we could try to detect kvm, and enable caching if !kvm. > > Maybe we could enable it during the decompress of the PEI FV and > disable it afterward? > >> The issues that >> used to pop up in the past, due to host kernel (KVM) changes, >> particularly in connection with VFIO device assignment, have been >> completely obscure and unpenetrable to me. > > Don't we eventually enable caching during the boot, so how is VFIO not > affected by that? > >> Even though I've contributed >> at least one KVM patch to mitigate those problems, they remain a mistery >> to me, and I remain unable to *reason* about the problems or the fixes. > > If VFIO requires uncached access, then what mechanisms does kvm > support for accessing memory ranges uncached? I thought kvm simply > ignored the cache setting and always enabled caching, because this > section of boot is not particularly slow with kvm. But, if enabling > caching causes issues, then I guess it does something. > > Does kvm support mtrr to uncache i/o ranges? I didn't think kvm > supported mtrrs in the past. > > I hope we wouldn't have to use paging to disable caching for the > affected regions. All good questions, and I'm not equipped to answer them, unfortunately. I'm happy to review and regression-test -- including GPU assignment, on my workstation that's dedicated to that use case -- OvmfPkg patches, if someone posts them. This stuff is in constant flux in KVM. Recent example: https://www.redhat.com/archives/vfio-users/2019-August/msg00016.html Thanks Laszlo >> So I think we could only flip the behavior (enable cache by default) and >> collect bug reports. But that's extremely annoying for end-users, and I >> see "no regressions" as one of my top responsibilities. >> >> Even if we provided an fw_cfg knob to disable the change, using >> QemuFwCfgSecLib, similarly to commit ab081a50e565 ("OvmfPkg: >> PlatformPei: take no-exec DXE settings from the QEMU command line", >> 2015-09-15), such fw_cfg knobs are difficult to use through layered >> products, such as libvirt, proxmox, etc. And of course fw_cfg is only >> available on QEMU. >> >> Laszlo