From: "Min Xu" <min.m.xu@intel.com>
To: "devel@edk2.groups.io" <devel@edk2.groups.io>
Cc: "Johnson, Brian" <brian.johnson@hpe.com>,
"West, Catharine" <catharine.west@intel.com>,
"Ni, Ray" <ray.ni@intel.com>,
"Wu, MingliangX" <mingliangx.wu@intel.com>,
"Yao, Jiewen" <jiewen.yao@intel.com>,
"Xue, Shengfeng" <xueshengfeng@byosoft.com.cn>,
"Dong, Eric" <eric.dong@intel.com>,
"Kumar, Rahul R" <rahul.r.kumar@intel.com>,
"De, Debkumar" <debkumar.de@intel.com>,
"Sun, Yi Y" <yi.y.sun@intel.com>,
"Huang, Jiaqing" <jiaqing.huang@intel.com>,
Gerd Hoffmann <kraxel@redhat.com>,
Laszlo Ersek <lersek@redhat.com>
Subject: Re: [edk2-devel] [PATCH V1 1/1] UefiCpuPkg/ResetVector: Cache Disable should not be set by default in CR0
Date: Wed, 24 Jan 2024 03:06:18 +0000 [thread overview]
Message-ID: <PH0PR11MB5064CF50F9D8EBE8F758BD38C57B2@PH0PR11MB5064.namprd11.prod.outlook.com> (raw)
In-Reply-To: <3ea2zwl64ktnxhchys2x3yqndz35gx2ppssvkn5zeg23jt5x7e@qm2jpmw2zveb>
Add intel linux guys in CC list.
> -----Original Message-----
> From: Gerd Hoffmann <kraxel@redhat.com>
> Sent: Wednesday, January 24, 2024 12:12 AM
> To: Laszlo Ersek <lersek@redhat.com>
> Cc: devel@edk2.groups.io; Johnson, Brian <brian.johnson@hpe.com>; West,
> Catharine <catharine.west@intel.com>; Xu, Min M <min.m.xu@intel.com>; Ni,
> Ray <ray.ni@intel.com>; Wu, MingliangX <mingliangx.wu@intel.com>; Yao,
> Jiewen <jiewen.yao@intel.com>; Xue, Shengfeng
> <xueshengfeng@byosoft.com.cn>; Dong, Eric <eric.dong@intel.com>; Kumar,
> Rahul R <rahul.r.kumar@intel.com>; De, Debkumar
> <debkumar.de@intel.com>
> Subject: Re: Re: [edk2-devel] [PATCH V1 1/1] UefiCpuPkg/ResetVector: Cache
> Disable should not be set by default in CR0
>
> Hi,
>
> > >>> Well, it's OVMF in a virtual machine. No boot guard involved.
> > >>> So we could probably go for a OVMF-specific patch here.
> > >>>
> > >>> But I'd prefer to figure what exactly is happening here before
> > >>> going down that route. An extreme slowdown just because we flip
> > >>> that bit doesn't make sense to me.
> > >>>
> > >>>> Why is boot time increasing?
> > >>>
> > >>> Not clear. It seems to be the lzma uncompress of the firmware
> > >>> volume in rom / pflash which is very slow. Also it is apparently
> > >>> only triggered in case pci device assignment is used.
> > >>
> > >> I've seen extreme slowness on physical platforms when we've mixed
> > >> up the MTRRs or page tables, causing code to be mapped uncached.
> > >>
> > >> Lzma uncompress of ROM could be pretty slow as well, if the ROM is
> > >> being read uncached. Lzma probably reads the data a byte at a
> > >> time, which is the worst case for uncached accesses. Since this is
> > >> a VM, it's not actually uncached at the hardware level, but I don't
> > >> know how QEMU/KVM handles uncached guest mappings.... It may be
> doing a VMEXIT for every byte.
> > >>
> > >> Anyway, I suggest double-checking your page tables and MTRRs.
> > >
> > > It happens very early at boot, before MTRRs are setup, running on
> > > the initial page tables created by the OVMF reset vector. The
> > > initial page tables have just 'accessed', 'dirty', 'read/write' and
> > > 'present' bits set for the 0-4G identity mapping.
> > >
> > > It seems to have something to do with EPT. It does not happen on
> > > AMD processors. It also does not happen when disabling EPT support
> > > in kvm on the host machine.
> > >
> > > looked at kvm kernel traces, I don't see excessive vmexits.
> >
> > This discussion evokes vague memories in me. I'll dump them here, but
> > I have no idea if they will be useful. (They probably won't.)
> >
> > - edk2 commit 98f378a7be12 ("OvmfPkg/ResetVector: enable caching in
> > initial page tables", 2013-09-24)
> >
> > - Linux (host) commit 879ae1880449 ("KVM: x86: obey
> > KVM_X86_QUIRK_CD_NW_CLEARED in kvm_set_cr0()", 2015-11-04)
>
> I actually waded through the source code in both places ;)
>
> Turned out kvm propagates guest MTRR settings to EPT memory types, but
> only in case kvm_arch_has_noncoherent_dma() is true, which why this
> triggers only with a mdev device assigned.
>
> MTRR disabled gets translated to UNCACHABLE, this is where the slowdown
> comes from. Test patch below fixes it for me.
>
> take care,
> Gerd
>
> ----------------------------- cut here ---------------------------- commit
> eb9f40ffd8afad03ac1fb6ac0e2a9af12ae78152
> Author: Gerd Hoffmann <kraxel@redhat.com>
> Date: Tue Jan 23 15:33:51 2024 +0100
>
> OvmfPkg/Sec: early mtrr setup
>
> Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
>
> diff --git a/OvmfPkg/Library/PlatformInitLib/MemDetect.c
> b/OvmfPkg/Library/PlatformInitLib/MemDetect.c
> index f042517bb64a..14f39236e44d 100644
> --- a/OvmfPkg/Library/PlatformInitLib/MemDetect.c
> +++ b/OvmfPkg/Library/PlatformInitLib/MemDetect.c
> @@ -1081,12 +1081,14 @@ PlatformQemuInitializeRam (
> if (IsMtrrSupported () && (PlatformInfoHob->HostBridgeDevId !=
> CLOUDHV_DEVICE_ID)) {
> MtrrGetAllMtrrs (&MtrrSettings);
>
> +#if 0
> //
> // MTRRs disabled, fixed MTRRs disabled, default type is uncached
> //
> ASSERT ((MtrrSettings.MtrrDefType & BIT11) == 0);
> ASSERT ((MtrrSettings.MtrrDefType & BIT10) == 0);
> ASSERT ((MtrrSettings.MtrrDefType & 0xFF) == 0);
> +#endif
>
> //
> // flip default type to writeback
> diff --git a/OvmfPkg/Sec/SecMain.c b/OvmfPkg/Sec/SecMain.c index
> 9bd1b9c95227..2820be1bab7c 100644
> --- a/OvmfPkg/Sec/SecMain.c
> +++ b/OvmfPkg/Sec/SecMain.c
> @@ -30,6 +30,7 @@
> #include <Ppi/MpInitLibDep.h>
> #include <Library/TdxHelperLib.h>
> #include <Library/CcProbeLib.h>
> +#include <Register/Intel/ArchitecturalMsr.h>
> #include "AmdSev.h"
>
> #define SEC_IDT_ENTRY_COUNT 34
> @@ -956,6 +957,19 @@ SecCoreStartupWithStack (
> InitializeApicTimer (0, MAX_UINT32, TRUE, 5);
> DisableApicTimerInterrupt ();
>
> + //
> + // Early MTRR setup (enable + set sefault) // {
> + MSR_IA32_MTRR_DEF_TYPE_REGISTER DefType;
> +
> + DefType.Uint64 = 0;
> + DefType.Bits.Type = 6; /* write back */
> + DefType.Bits.E = 1; /* enable */
> + AsmWriteMsr64 (MSR_IA32_MTRR_DEF_TYPE, DefType.Uint64);
> + DEBUG ((DEBUG_ERROR, "%a:%d early mtrr: %lx\n", __func__, __LINE__,
> + DefType.Uint64)); }
> +
> //
> // Initialize Debug Agent to support source level debug in SEC/PEI phases
> before memory ready.
> //
-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.
View/Reply Online (#114249): https://edk2.groups.io/g/devel/message/114249
Mute This Topic: https://groups.io/mt/100367559/7686176
Group Owner: devel+owner@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub [rebecca@openfw.io]
-=-=-=-=-=-=-=-=-=-=-=-
next prev parent reply other threads:[~2024-01-24 3:06 UTC|newest]
Thread overview: 16+ messages / expand[flat|nested] mbox.gz Atom feed top
2023-07-26 9:47 [edk2-devel] [PATCH V1 1/1] UefiCpuPkg/ResetVector: Cache Disable should not be set by default in CR0 xueshengfeng via groups.io
2023-07-26 9:55 ` Ni, Ray
[not found] ` <177562550EF0534C.27380@groups.io>
2023-08-03 8:14 ` Ni, Ray
2024-01-10 7:51 ` Min Xu
2024-01-10 16:43 ` West, Catharine
2024-01-18 15:46 ` Gerd Hoffmann
2024-01-22 19:11 ` Brian J. Johnson
2024-01-23 5:01 ` Min Xu
2024-01-23 10:52 ` Gerd Hoffmann
2024-01-23 14:13 ` Laszlo Ersek
2024-01-23 16:11 ` Gerd Hoffmann
2024-01-24 3:06 ` Min Xu [this message]
2024-01-24 12:49 ` Laszlo Ersek
2024-01-24 13:26 ` Gerd Hoffmann
2024-01-24 14:45 ` Laszlo Ersek
2024-01-24 17:11 ` Gerd Hoffmann
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-list from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=PH0PR11MB5064CF50F9D8EBE8F758BD38C57B2@PH0PR11MB5064.namprd11.prod.outlook.com \
--to=devel@edk2.groups.io \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox