From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail02.groups.io (mail02.groups.io [66.175.222.108]) by spool.mail.gandi.net (Postfix) with ESMTPS id CCCCFAC171E for ; Tue, 23 Jan 2024 16:12:02 +0000 (UTC) DKIM-Signature: a=rsa-sha256; bh=I+f3OgrRo29rCZ0dDyuB8M3uZHG3A3pfwDp0feGljJ4=; c=relaxed/simple; d=groups.io; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version:In-Reply-To:Precedence:List-Subscribe:List-Help:Sender:List-Id:Mailing-List:Delivered-To:Reply-To:List-Unsubscribe-Post:List-Unsubscribe:Content-Type:Content-Disposition; s=20140610; t=1706026321; v=1; b=TcDIoCYsSDK47RekN7cmyviPODqb+oL3FYQZueap9O1TAwBWJkDhWzy4MTwRQpKI9vH1YbZA n0LUBX9GBDGsA/L6erLAsWU0nSqa5rFxSUIguaKSG+00FRyITbUBq14Fw8YEaPXX0zZOhxwXmud OVOUhCLdP8H1seWUMA1qx8QU= X-Received: by 127.0.0.2 with SMTP id hSNDYY7687511xlumwR4YPgE; Tue, 23 Jan 2024 08:12:01 -0800 X-Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by mx.groups.io with SMTP id smtpd.web10.16621.1706026320674639163 for ; Tue, 23 Jan 2024 08:12:00 -0800 X-Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-155-mOn63ffWNyaEOdj9EpXV0Q-1; Tue, 23 Jan 2024 11:11:55 -0500 X-MC-Unique: mOn63ffWNyaEOdj9EpXV0Q-1 X-Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.rdu2.redhat.com [10.11.54.2]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 34D1688B7A1; Tue, 23 Jan 2024 16:11:54 +0000 (UTC) X-Received: from sirius.home.kraxel.org (unknown [10.39.193.66]) by smtp.corp.redhat.com (Postfix) with ESMTPS id D403140C1430; Tue, 23 Jan 2024 16:11:53 +0000 (UTC) X-Received: by sirius.home.kraxel.org (Postfix, from userid 1000) id 5CBAB180038B; Tue, 23 Jan 2024 17:11:52 +0100 (CET) Date: Tue, 23 Jan 2024 17:11:52 +0100 From: "Gerd Hoffmann" To: Laszlo Ersek Cc: devel@edk2.groups.io, "Brian J. Johnson" , "West, Catharine" , "Xu, Min M" , "Ni, Ray" , "Wu, MingliangX" , "Yao, Jiewen" , "Xue, Shengfeng" , "Dong, Eric" , "Kumar, Rahul R" , "De, Debkumar" Subject: Re: [edk2-devel] [PATCH V1 1/1] UefiCpuPkg/ResetVector: Cache Disable should not be set by default in CR0 Message-ID: <3ea2zwl64ktnxhchys2x3yqndz35gx2ppssvkn5zeg23jt5x7e@qm2jpmw2zveb> References: <20230726094754.171-1-xueshengfeng@byosoft.com.cn> <177562550EF0534C.27380@groups.io> <3505f62e-cc54-490e-983f-7b4312e41509@hpe.com> <3lxerlg6g5gbzsxyh2v4qqqxru34ewytbge2wm6s7quyx3itx6@xlajojgm73qe> <1708ba2b-c969-ee8a-2cbe-fdc9acd31998@redhat.com> MIME-Version: 1.0 In-Reply-To: <1708ba2b-c969-ee8a-2cbe-fdc9acd31998@redhat.com> X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.2 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Precedence: Bulk List-Subscribe: List-Help: Sender: devel@edk2.groups.io List-Id: Mailing-List: list devel@edk2.groups.io; contact devel+owner@edk2.groups.io Reply-To: devel@edk2.groups.io,kraxel@redhat.com List-Unsubscribe-Post: List-Unsubscribe=One-Click List-Unsubscribe: X-Gm-Message-State: p7K8tYLcYAikL9fOZhBbcjmex7686176AA= Content-Type: text/plain; charset=us-ascii Content-Disposition: inline X-GND-Status: LEGIT Authentication-Results: spool.mail.gandi.net; dkim=pass header.d=groups.io header.s=20140610 header.b=TcDIoCYs; dmarc=fail reason="SPF not aligned (relaxed), DKIM not aligned (relaxed)" header.from=redhat.com (policy=none); spf=pass (spool.mail.gandi.net: domain of bounce@groups.io designates 66.175.222.108 as permitted sender) smtp.mailfrom=bounce@groups.io Hi, > >>> Well, it's OVMF in a virtual machine. No boot guard involved. > >>> So we could probably go for a OVMF-specific patch here. > >>> > >>> But I'd prefer to figure what exactly is happening here before going > >>> down that route. An extreme slowdown just because we flip that bit > >>> doesn't make sense to me. > >>> > >>>> Why is boot time increasing? > >>> > >>> Not clear. It seems to be the lzma uncompress of the firmware volume > >>> in rom / pflash which is very slow. Also it is apparently only > >>> triggered in case pci device assignment is used. > >> > >> I've seen extreme slowness on physical platforms when we've mixed up the > >> MTRRs or page tables, causing code to be mapped uncached. > >> > >> Lzma uncompress of ROM could be pretty slow as well, if the ROM is being > >> read uncached. Lzma probably reads the data a byte at a time, which is the > >> worst case for uncached accesses. Since this is a VM, it's not actually > >> uncached at the hardware level, but I don't know how QEMU/KVM handles > >> uncached guest mappings.... It may be doing a VMEXIT for every byte. > >> > >> Anyway, I suggest double-checking your page tables and MTRRs. > > > > It happens very early at boot, before MTRRs are setup, running on the > > initial page tables created by the OVMF reset vector. The initial page > > tables have just 'accessed', 'dirty', 'read/write' and 'present' bits > > set for the 0-4G identity mapping. > > > > It seems to have something to do with EPT. It does not happen on AMD > > processors. It also does not happen when disabling EPT support in kvm > > on the host machine. > > > > looked at kvm kernel traces, I don't see excessive vmexits. > > This discussion evokes vague memories in me. I'll dump them here, but I > have no idea if they will be useful. (They probably won't.) > > - edk2 commit 98f378a7be12 ("OvmfPkg/ResetVector: enable caching in > initial page tables", 2013-09-24) > > - Linux (host) commit 879ae1880449 ("KVM: x86: obey > KVM_X86_QUIRK_CD_NW_CLEARED in kvm_set_cr0()", 2015-11-04) I actually waded through the source code in both places ;) Turned out kvm propagates guest MTRR settings to EPT memory types, but only in case kvm_arch_has_noncoherent_dma() is true, which why this triggers only with a mdev device assigned. MTRR disabled gets translated to UNCACHABLE, this is where the slowdown comes from. Test patch below fixes it for me. take care, Gerd ----------------------------- cut here ---------------------------- commit eb9f40ffd8afad03ac1fb6ac0e2a9af12ae78152 Author: Gerd Hoffmann Date: Tue Jan 23 15:33:51 2024 +0100 OvmfPkg/Sec: early mtrr setup Signed-off-by: Gerd Hoffmann diff --git a/OvmfPkg/Library/PlatformInitLib/MemDetect.c b/OvmfPkg/Library/PlatformInitLib/MemDetect.c index f042517bb64a..14f39236e44d 100644 --- a/OvmfPkg/Library/PlatformInitLib/MemDetect.c +++ b/OvmfPkg/Library/PlatformInitLib/MemDetect.c @@ -1081,12 +1081,14 @@ PlatformQemuInitializeRam ( if (IsMtrrSupported () && (PlatformInfoHob->HostBridgeDevId != CLOUDHV_DEVICE_ID)) { MtrrGetAllMtrrs (&MtrrSettings); +#if 0 // // MTRRs disabled, fixed MTRRs disabled, default type is uncached // ASSERT ((MtrrSettings.MtrrDefType & BIT11) == 0); ASSERT ((MtrrSettings.MtrrDefType & BIT10) == 0); ASSERT ((MtrrSettings.MtrrDefType & 0xFF) == 0); +#endif // // flip default type to writeback diff --git a/OvmfPkg/Sec/SecMain.c b/OvmfPkg/Sec/SecMain.c index 9bd1b9c95227..2820be1bab7c 100644 --- a/OvmfPkg/Sec/SecMain.c +++ b/OvmfPkg/Sec/SecMain.c @@ -30,6 +30,7 @@ #include #include #include +#include #include "AmdSev.h" #define SEC_IDT_ENTRY_COUNT 34 @@ -956,6 +957,19 @@ SecCoreStartupWithStack ( InitializeApicTimer (0, MAX_UINT32, TRUE, 5); DisableApicTimerInterrupt (); + // + // Early MTRR setup (enable + set sefault) + // + { + MSR_IA32_MTRR_DEF_TYPE_REGISTER DefType; + + DefType.Uint64 = 0; + DefType.Bits.Type = 6; /* write back */ + DefType.Bits.E = 1; /* enable */ + AsmWriteMsr64 (MSR_IA32_MTRR_DEF_TYPE, DefType.Uint64); + DEBUG ((DEBUG_ERROR, "%a:%d early mtrr: %lx\n", __func__, __LINE__, DefType.Uint64)); + } + // // Initialize Debug Agent to support source level debug in SEC/PEI phases before memory ready. // -=-=-=-=-=-=-=-=-=-=-=- Groups.io Links: You receive all messages sent to this group. View/Reply Online (#114214): https://edk2.groups.io/g/devel/message/114214 Mute This Topic: https://groups.io/mt/100367559/7686176 Group Owner: devel+owner@edk2.groups.io Unsubscribe: https://edk2.groups.io/g/devel/unsub [rebecca@openfw.io] -=-=-=-=-=-=-=-=-=-=-=-