From mboxrd@z Thu Jan 1 00:00:00 1970
Return-Path: <bounce+27952+114214+7686176+12367111@groups.io>
Received: from mail02.groups.io (mail02.groups.io [66.175.222.108])
by spool.mail.gandi.net (Postfix) with ESMTPS id CCCCFAC171E
for <rebecca@openfw.io>; Tue, 23 Jan 2024 16:12:02 +0000 (UTC)
DKIM-Signature: a=rsa-sha256; bh=I+f3OgrRo29rCZ0dDyuB8M3uZHG3A3pfwDp0feGljJ4=;
c=relaxed/simple; d=groups.io;
h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version:In-Reply-To:Precedence:List-Subscribe:List-Help:Sender:List-Id:Mailing-List:Delivered-To:Reply-To:List-Unsubscribe-Post:List-Unsubscribe:Content-Type:Content-Disposition;
s=20140610; t=1706026321; v=1;
b=TcDIoCYsSDK47RekN7cmyviPODqb+oL3FYQZueap9O1TAwBWJkDhWzy4MTwRQpKI9vH1YbZA
n0LUBX9GBDGsA/L6erLAsWU0nSqa5rFxSUIguaKSG+00FRyITbUBq14Fw8YEaPXX0zZOhxwXmud
OVOUhCLdP8H1seWUMA1qx8QU=
X-Received: by 127.0.0.2 with SMTP id hSNDYY7687511xlumwR4YPgE; Tue, 23 Jan 2024 08:12:01 -0800
X-Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124])
by mx.groups.io with SMTP id smtpd.web10.16621.1706026320674639163
for <devel@edk2.groups.io>;
Tue, 23 Jan 2024 08:12:00 -0800
X-Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com
[66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS
(version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id
us-mta-155-mOn63ffWNyaEOdj9EpXV0Q-1; Tue, 23 Jan 2024 11:11:55 -0500
X-MC-Unique: mOn63ffWNyaEOdj9EpXV0Q-1
X-Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.rdu2.redhat.com [10.11.54.2])
(using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256)
(No client certificate requested)
by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 34D1688B7A1;
Tue, 23 Jan 2024 16:11:54 +0000 (UTC)
X-Received: from sirius.home.kraxel.org (unknown [10.39.193.66])
by smtp.corp.redhat.com (Postfix) with ESMTPS id D403140C1430;
Tue, 23 Jan 2024 16:11:53 +0000 (UTC)
X-Received: by sirius.home.kraxel.org (Postfix, from userid 1000)
id 5CBAB180038B; Tue, 23 Jan 2024 17:11:52 +0100 (CET)
Date: Tue, 23 Jan 2024 17:11:52 +0100
From: "Gerd Hoffmann" <kraxel@redhat.com>
To: Laszlo Ersek <lersek@redhat.com>
Cc: devel@edk2.groups.io, "Brian J. Johnson" <brian.johnson@hpe.com>,
"West, Catharine" <catharine.west@intel.com>, "Xu, Min M" <min.m.xu@intel.com>, "Ni, Ray" <ray.ni@intel.com>,
"Wu, MingliangX" <mingliangx.wu@intel.com>, "Yao, Jiewen" <jiewen.yao@intel.com>,
"Xue, Shengfeng" <xueshengfeng@byosoft.com.cn>, "Dong, Eric" <eric.dong@intel.com>,
"Kumar, Rahul R" <rahul.r.kumar@intel.com>, "De, Debkumar" <debkumar.de@intel.com>
Subject: Re: [edk2-devel] [PATCH V1 1/1] UefiCpuPkg/ResetVector: Cache Disable should not be set by default in CR0
Message-ID: <3ea2zwl64ktnxhchys2x3yqndz35gx2ppssvkn5zeg23jt5x7e@qm2jpmw2zveb>
References: <20230726094754.171-1-xueshengfeng@byosoft.com.cn>
<177562550EF0534C.27380@groups.io>
<MN6PR11MB8244ED31046A2B89AC6BBB478C08A@MN6PR11MB8244.namprd11.prod.outlook.com>
<PH0PR11MB50645603F9F6CE4639FF1A7AC5692@PH0PR11MB5064.namprd11.prod.outlook.com>
<SN7PR11MB7591D19E5BF0096ABDF4AB9A9B692@SN7PR11MB7591.namprd11.prod.outlook.com>
<iat7urnhdfjbs4z5msw3flhmxz2altbe6piixqv2av43xj3ayf@v56pxdihr3lk>
<3505f62e-cc54-490e-983f-7b4312e41509@hpe.com>
<3lxerlg6g5gbzsxyh2v4qqqxru34ewytbge2wm6s7quyx3itx6@xlajojgm73qe>
<1708ba2b-c969-ee8a-2cbe-fdc9acd31998@redhat.com>
MIME-Version: 1.0
In-Reply-To: <1708ba2b-c969-ee8a-2cbe-fdc9acd31998@redhat.com>
X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.2
X-Mimecast-Spam-Score: 0
X-Mimecast-Originator: redhat.com
Precedence: Bulk
List-Subscribe: <mailto:devel+subscribe@edk2.groups.io>
List-Help: <mailto:devel+help@edk2.groups.io>
Sender: devel@edk2.groups.io
List-Id: <devel.edk2.groups.io>
Mailing-List: list devel@edk2.groups.io; contact devel+owner@edk2.groups.io
Reply-To: devel@edk2.groups.io,kraxel@redhat.com
List-Unsubscribe-Post: List-Unsubscribe=One-Click
List-Unsubscribe: <https://edk2.groups.io/g/devel/leave/12367111/7686176/1913456212/plugh>
X-Gm-Message-State: p7K8tYLcYAikL9fOZhBbcjmex7686176AA=
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
X-GND-Status: LEGIT
Authentication-Results: spool.mail.gandi.net;
dkim=pass header.d=groups.io header.s=20140610 header.b=TcDIoCYs;
dmarc=fail reason="SPF not aligned (relaxed), DKIM not aligned (relaxed)" header.from=redhat.com (policy=none);
spf=pass (spool.mail.gandi.net: domain of bounce@groups.io designates 66.175.222.108 as permitted sender) smtp.mailfrom=bounce@groups.io
Hi,
> >>> Well, it's OVMF in a virtual machine. No boot guard involved.
> >>> So we could probably go for a OVMF-specific patch here.
> >>>
> >>> But I'd prefer to figure what exactly is happening here before going
> >>> down that route. An extreme slowdown just because we flip that bit
> >>> doesn't make sense to me.
> >>>
> >>>> Why is boot time increasing?
> >>>
> >>> Not clear. It seems to be the lzma uncompress of the firmware volume
> >>> in rom / pflash which is very slow. Also it is apparently only
> >>> triggered in case pci device assignment is used.
> >>
> >> I've seen extreme slowness on physical platforms when we've mixed up the
> >> MTRRs or page tables, causing code to be mapped uncached.
> >>
> >> Lzma uncompress of ROM could be pretty slow as well, if the ROM is being
> >> read uncached. Lzma probably reads the data a byte at a time, which is the
> >> worst case for uncached accesses. Since this is a VM, it's not actually
> >> uncached at the hardware level, but I don't know how QEMU/KVM handles
> >> uncached guest mappings.... It may be doing a VMEXIT for every byte.
> >>
> >> Anyway, I suggest double-checking your page tables and MTRRs.
> >
> > It happens very early at boot, before MTRRs are setup, running on the
> > initial page tables created by the OVMF reset vector. The initial page
> > tables have just 'accessed', 'dirty', 'read/write' and 'present' bits
> > set for the 0-4G identity mapping.
> >
> > It seems to have something to do with EPT. It does not happen on AMD
> > processors. It also does not happen when disabling EPT support in kvm
> > on the host machine.
> >
> > looked at kvm kernel traces, I don't see excessive vmexits.
>
> This discussion evokes vague memories in me. I'll dump them here, but I
> have no idea if they will be useful. (They probably won't.)
>
> - edk2 commit 98f378a7be12 ("OvmfPkg/ResetVector: enable caching in
> initial page tables", 2013-09-24)
>
> - Linux (host) commit 879ae1880449 ("KVM: x86: obey
> KVM_X86_QUIRK_CD_NW_CLEARED in kvm_set_cr0()", 2015-11-04)
I actually waded through the source code in both places ;)
Turned out kvm propagates guest MTRR settings to EPT memory types,
but only in case kvm_arch_has_noncoherent_dma() is true, which why
this triggers only with a mdev device assigned.
MTRR disabled gets translated to UNCACHABLE, this is where the
slowdown comes from. Test patch below fixes it for me.
take care,
Gerd
----------------------------- cut here ----------------------------
commit eb9f40ffd8afad03ac1fb6ac0e2a9af12ae78152
Author: Gerd Hoffmann <kraxel@redhat.com>
Date: Tue Jan 23 15:33:51 2024 +0100
OvmfPkg/Sec: early mtrr setup
Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
diff --git a/OvmfPkg/Library/PlatformInitLib/MemDetect.c b/OvmfPkg/Library/PlatformInitLib/MemDetect.c
index f042517bb64a..14f39236e44d 100644
--- a/OvmfPkg/Library/PlatformInitLib/MemDetect.c
+++ b/OvmfPkg/Library/PlatformInitLib/MemDetect.c
@@ -1081,12 +1081,14 @@ PlatformQemuInitializeRam (
if (IsMtrrSupported () && (PlatformInfoHob->HostBridgeDevId != CLOUDHV_DEVICE_ID)) {
MtrrGetAllMtrrs (&MtrrSettings);
+#if 0
//
// MTRRs disabled, fixed MTRRs disabled, default type is uncached
//
ASSERT ((MtrrSettings.MtrrDefType & BIT11) == 0);
ASSERT ((MtrrSettings.MtrrDefType & BIT10) == 0);
ASSERT ((MtrrSettings.MtrrDefType & 0xFF) == 0);
+#endif
//
// flip default type to writeback
diff --git a/OvmfPkg/Sec/SecMain.c b/OvmfPkg/Sec/SecMain.c
index 9bd1b9c95227..2820be1bab7c 100644
--- a/OvmfPkg/Sec/SecMain.c
+++ b/OvmfPkg/Sec/SecMain.c
@@ -30,6 +30,7 @@
#include <Ppi/MpInitLibDep.h>
#include <Library/TdxHelperLib.h>
#include <Library/CcProbeLib.h>
+#include <Register/Intel/ArchitecturalMsr.h>
#include "AmdSev.h"
#define SEC_IDT_ENTRY_COUNT 34
@@ -956,6 +957,19 @@ SecCoreStartupWithStack (
InitializeApicTimer (0, MAX_UINT32, TRUE, 5);
DisableApicTimerInterrupt ();
+ //
+ // Early MTRR setup (enable + set sefault)
+ //
+ {
+ MSR_IA32_MTRR_DEF_TYPE_REGISTER DefType;
+
+ DefType.Uint64 = 0;
+ DefType.Bits.Type = 6; /* write back */
+ DefType.Bits.E = 1; /* enable */
+ AsmWriteMsr64 (MSR_IA32_MTRR_DEF_TYPE, DefType.Uint64);
+ DEBUG ((DEBUG_ERROR, "%a:%d early mtrr: %lx\n", __func__, __LINE__, DefType.Uint64));
+ }
+
//
// Initialize Debug Agent to support source level debug in SEC/PEI phases before memory ready.
//
-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.
View/Reply Online (#114214): https://edk2.groups.io/g/devel/message/114214
Mute This Topic: https://groups.io/mt/100367559/7686176
Group Owner: devel+owner@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub [rebecca@openfw.io]
-=-=-=-=-=-=-=-=-=-=-=-