public inbox for devel@edk2.groups.io
 help / color / mirror / Atom feed
From: "Gerd Hoffmann" <kraxel@redhat.com>
To: Laszlo Ersek <lersek@redhat.com>
Cc: devel@edk2.groups.io, "Brian J. Johnson" <brian.johnson@hpe.com>,
	 "West, Catharine" <catharine.west@intel.com>,
	"Xu, Min M" <min.m.xu@intel.com>, "Ni, Ray" <ray.ni@intel.com>,
	 "Wu, MingliangX" <mingliangx.wu@intel.com>,
	"Yao, Jiewen" <jiewen.yao@intel.com>,
	 "Xue, Shengfeng" <xueshengfeng@byosoft.com.cn>,
	"Dong, Eric" <eric.dong@intel.com>,
	 "Kumar, Rahul R" <rahul.r.kumar@intel.com>,
	"De, Debkumar" <debkumar.de@intel.com>
Subject: Re: [edk2-devel] [PATCH V1 1/1] UefiCpuPkg/ResetVector: Cache Disable should not be set by default in CR0
Date: Tue, 23 Jan 2024 17:11:52 +0100	[thread overview]
Message-ID: <3ea2zwl64ktnxhchys2x3yqndz35gx2ppssvkn5zeg23jt5x7e@qm2jpmw2zveb> (raw)
In-Reply-To: <1708ba2b-c969-ee8a-2cbe-fdc9acd31998@redhat.com>

  Hi,

> >>> Well, it's OVMF in a virtual machine.  No boot guard involved.
> >>> So we could probably go for a OVMF-specific patch here.
> >>>
> >>> But I'd prefer to figure what exactly is happening here before going
> >>> down that route.  An extreme slowdown just because we flip that bit
> >>> doesn't make sense to me.
> >>>
> >>>> Why is boot time increasing?
> >>>
> >>> Not clear.  It seems to be the lzma uncompress of the firmware volume
> >>> in rom / pflash which is very slow.  Also it is apparently only
> >>> triggered in case pci device assignment is used.
> >>
> >> I've seen extreme slowness on physical platforms when we've mixed up the
> >> MTRRs or page tables, causing code to be mapped uncached.
> >>
> >> Lzma uncompress of ROM could be pretty slow as well, if the ROM is being
> >> read uncached.  Lzma probably reads the data a byte at a time, which is the
> >> worst case for uncached accesses.  Since this is a VM, it's not actually
> >> uncached at the hardware level, but I don't know how QEMU/KVM handles
> >> uncached guest mappings.... It may be doing a VMEXIT for every byte.
> >>
> >> Anyway, I suggest double-checking your page tables and MTRRs.
> > 
> > It happens very early at boot, before MTRRs are setup, running on the
> > initial page tables created by the OVMF reset vector.  The initial page
> > tables have just 'accessed', 'dirty', 'read/write' and 'present' bits
> > set for the 0-4G identity mapping.
> > 
> > It seems to have something to do with EPT.  It does not happen on AMD
> > processors.  It also does not happen when disabling EPT support in kvm
> > on the host machine.
> > 
> > looked at kvm kernel traces, I don't see excessive vmexits.
> 
> This discussion evokes vague memories in me. I'll dump them here, but I
> have no idea if they will be useful. (They probably won't.)
> 
> - edk2 commit 98f378a7be12 ("OvmfPkg/ResetVector: enable caching in
> initial page tables", 2013-09-24)
> 
> - Linux (host) commit 879ae1880449 ("KVM: x86: obey
> KVM_X86_QUIRK_CD_NW_CLEARED in kvm_set_cr0()", 2015-11-04)

I actually waded through the source code in both places ;)

Turned out kvm propagates guest MTRR settings to EPT memory types,
but only in case kvm_arch_has_noncoherent_dma() is true, which why
this triggers only with a mdev device assigned.

MTRR disabled gets translated to UNCACHABLE, this is where the
slowdown comes from.  Test patch below fixes it for me.

take care,
  Gerd

----------------------------- cut here ----------------------------
commit eb9f40ffd8afad03ac1fb6ac0e2a9af12ae78152
Author: Gerd Hoffmann <kraxel@redhat.com>
Date:   Tue Jan 23 15:33:51 2024 +0100

    OvmfPkg/Sec: early mtrr setup
    
    Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>

diff --git a/OvmfPkg/Library/PlatformInitLib/MemDetect.c b/OvmfPkg/Library/PlatformInitLib/MemDetect.c
index f042517bb64a..14f39236e44d 100644
--- a/OvmfPkg/Library/PlatformInitLib/MemDetect.c
+++ b/OvmfPkg/Library/PlatformInitLib/MemDetect.c
@@ -1081,12 +1081,14 @@ PlatformQemuInitializeRam (
   if (IsMtrrSupported () && (PlatformInfoHob->HostBridgeDevId != CLOUDHV_DEVICE_ID)) {
     MtrrGetAllMtrrs (&MtrrSettings);
 
+#if 0
     //
     // MTRRs disabled, fixed MTRRs disabled, default type is uncached
     //
     ASSERT ((MtrrSettings.MtrrDefType & BIT11) == 0);
     ASSERT ((MtrrSettings.MtrrDefType & BIT10) == 0);
     ASSERT ((MtrrSettings.MtrrDefType & 0xFF) == 0);
+#endif
 
     //
     // flip default type to writeback
diff --git a/OvmfPkg/Sec/SecMain.c b/OvmfPkg/Sec/SecMain.c
index 9bd1b9c95227..2820be1bab7c 100644
--- a/OvmfPkg/Sec/SecMain.c
+++ b/OvmfPkg/Sec/SecMain.c
@@ -30,6 +30,7 @@
 #include <Ppi/MpInitLibDep.h>
 #include <Library/TdxHelperLib.h>
 #include <Library/CcProbeLib.h>
+#include <Register/Intel/ArchitecturalMsr.h>
 #include "AmdSev.h"
 
 #define SEC_IDT_ENTRY_COUNT  34
@@ -956,6 +957,19 @@ SecCoreStartupWithStack (
   InitializeApicTimer (0, MAX_UINT32, TRUE, 5);
   DisableApicTimerInterrupt ();
 
+  //
+  // Early MTRR setup (enable + set sefault)
+  //
+  {
+    MSR_IA32_MTRR_DEF_TYPE_REGISTER  DefType;
+
+    DefType.Uint64    = 0;
+    DefType.Bits.Type = 6; /* write back */
+    DefType.Bits.E    = 1; /* enable */
+    AsmWriteMsr64 (MSR_IA32_MTRR_DEF_TYPE, DefType.Uint64);
+    DEBUG ((DEBUG_ERROR, "%a:%d early mtrr: %lx\n", __func__, __LINE__, DefType.Uint64));
+  }
+
   //
   // Initialize Debug Agent to support source level debug in SEC/PEI phases before memory ready.
   //



-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.
View/Reply Online (#114214): https://edk2.groups.io/g/devel/message/114214
Mute This Topic: https://groups.io/mt/100367559/7686176
Group Owner: devel+owner@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub [rebecca@openfw.io]
-=-=-=-=-=-=-=-=-=-=-=-



  reply	other threads:[~2024-01-23 16:12 UTC|newest]

Thread overview: 16+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2023-07-26  9:47 [edk2-devel] [PATCH V1 1/1] UefiCpuPkg/ResetVector: Cache Disable should not be set by default in CR0 xueshengfeng via groups.io
2023-07-26  9:55 ` Ni, Ray
     [not found] ` <177562550EF0534C.27380@groups.io>
2023-08-03  8:14   ` Ni, Ray
2024-01-10  7:51     ` Min Xu
2024-01-10 16:43       ` West, Catharine
2024-01-18 15:46         ` Gerd Hoffmann
2024-01-22 19:11           ` Brian J. Johnson
2024-01-23  5:01             ` Min Xu
2024-01-23 10:52             ` Gerd Hoffmann
2024-01-23 14:13               ` Laszlo Ersek
2024-01-23 16:11                 ` Gerd Hoffmann [this message]
2024-01-24  3:06                   ` Min Xu
2024-01-24 12:49                   ` Laszlo Ersek
2024-01-24 13:26                     ` Gerd Hoffmann
2024-01-24 14:45                       ` Laszlo Ersek
2024-01-24 17:11                         ` Gerd Hoffmann

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-list from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=3ea2zwl64ktnxhchys2x3yqndz35gx2ppssvkn5zeg23jt5x7e@qm2jpmw2zveb \
    --to=devel@edk2.groups.io \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox