From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail02.groups.io (mail02.groups.io [66.175.222.108]) by spool.mail.gandi.net (Postfix) with ESMTPS id AD52294177E for ; Wed, 24 Jan 2024 12:50:04 +0000 (UTC) DKIM-Signature: a=rsa-sha256; bh=eRY9PlFUcXCCkUhSdtZ0ORPATXI5me9I63UML6dzNYI=; c=relaxed/simple; d=groups.io; h=Message-ID:Date:MIME-Version:Subject:To:Cc:References:From:In-Reply-To:Precedence:List-Subscribe:List-Help:Sender:List-Id:Mailing-List:Delivered-To:Reply-To:List-Unsubscribe-Post:List-Unsubscribe:Content-Language:Content-Type:Content-Transfer-Encoding; s=20140610; t=1706100603; v=1; b=n5Eq9svhpIzjKb4JrTroYEGahBe8WrQW87cYTAH8pZF0zU1z/XuRjEwPkPxB2P7QpJ91AUWT 0aQ3Wlaj4KeUFFumkULqiXtCta2Lwz9WetQai9w4sOdgXqk5m7dh0XmC5KcTeu6L+LJ53E64cQM eG5DWMtYuW1gGr+7cbGSzlS4= X-Received: by 127.0.0.2 with SMTP id vDOYYY7687511xxT3z8TrTEM; Wed, 24 Jan 2024 04:50:03 -0800 X-Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by mx.groups.io with SMTP id smtpd.web10.21529.1706100602782180135 for ; Wed, 24 Jan 2024 04:50:02 -0800 X-Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-675-BWGUc4zjNZul1DcrnwiBGw-1; Wed, 24 Jan 2024 07:49:57 -0500 X-MC-Unique: BWGUc4zjNZul1DcrnwiBGw-1 X-Received: from smtp.corp.redhat.com (int-mx09.intmail.prod.int.rdu2.redhat.com [10.11.54.9]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 2C304108C22A; Wed, 24 Jan 2024 12:49:56 +0000 (UTC) X-Received: from [10.39.195.127] (unknown [10.39.195.127]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 0A586492BCB; Wed, 24 Jan 2024 12:49:53 +0000 (UTC) Message-ID: <70393bba-23a8-0c26-e245-55cc075e9002@redhat.com> Date: Wed, 24 Jan 2024 13:49:52 +0100 MIME-Version: 1.0 Subject: Re: [edk2-devel] [PATCH V1 1/1] UefiCpuPkg/ResetVector: Cache Disable should not be set by default in CR0 To: Gerd Hoffmann Cc: devel@edk2.groups.io, "Brian J. Johnson" , "West, Catharine" , "Xu, Min M" , "Ni, Ray" , "Wu, MingliangX" , "Yao, Jiewen" , "Xue, Shengfeng" , "Dong, Eric" , "Kumar, Rahul R" , "De, Debkumar" References: <20230726094754.171-1-xueshengfeng@byosoft.com.cn> <177562550EF0534C.27380@groups.io> <3505f62e-cc54-490e-983f-7b4312e41509@hpe.com> <3lxerlg6g5gbzsxyh2v4qqqxru34ewytbge2wm6s7quyx3itx6@xlajojgm73qe> <1708ba2b-c969-ee8a-2cbe-fdc9acd31998@redhat.com> <3ea2zwl64ktnxhchys2x3yqndz35gx2ppssvkn5zeg23jt5x7e@qm2jpmw2zveb> From: "Laszlo Ersek" In-Reply-To: <3ea2zwl64ktnxhchys2x3yqndz35gx2ppssvkn5zeg23jt5x7e@qm2jpmw2zveb> X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.9 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Precedence: Bulk List-Subscribe: List-Help: Sender: devel@edk2.groups.io List-Id: Mailing-List: list devel@edk2.groups.io; contact devel+owner@edk2.groups.io Reply-To: devel@edk2.groups.io,lersek@redhat.com List-Unsubscribe-Post: List-Unsubscribe=One-Click List-Unsubscribe: X-Gm-Message-State: YTghx8vj92hprEueR9Pxn2cwx7686176AA= Content-Language: en-US Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-GND-Status: LEGIT Authentication-Results: spool.mail.gandi.net; dkim=pass header.d=groups.io header.s=20140610 header.b=n5Eq9svh; dmarc=fail reason="SPF not aligned (relaxed), DKIM not aligned (relaxed)" header.from=redhat.com (policy=none); spf=pass (spool.mail.gandi.net: domain of bounce@groups.io designates 66.175.222.108 as permitted sender) smtp.mailfrom=bounce@groups.io On 1/23/24 17:11, Gerd Hoffmann wrote: > Hi, >=20 >>>>> Well, it's OVMF in a virtual machine. No boot guard involved. >>>>> So we could probably go for a OVMF-specific patch here. >>>>> >>>>> But I'd prefer to figure what exactly is happening here before going >>>>> down that route. An extreme slowdown just because we flip that bit >>>>> doesn't make sense to me. >>>>> >>>>>> Why is boot time increasing? >>>>> >>>>> Not clear. It seems to be the lzma uncompress of the firmware volume >>>>> in rom / pflash which is very slow. Also it is apparently only >>>>> triggered in case pci device assignment is used. >>>> >>>> I've seen extreme slowness on physical platforms when we've mixed up t= he >>>> MTRRs or page tables, causing code to be mapped uncached. >>>> >>>> Lzma uncompress of ROM could be pretty slow as well, if the ROM is bei= ng >>>> read uncached. Lzma probably reads the data a byte at a time, which i= s the >>>> worst case for uncached accesses. Since this is a VM, it's not actual= ly >>>> uncached at the hardware level, but I don't know how QEMU/KVM handles >>>> uncached guest mappings.... It may be doing a VMEXIT for every byte. >>>> >>>> Anyway, I suggest double-checking your page tables and MTRRs. >>> >>> It happens very early at boot, before MTRRs are setup, running on the >>> initial page tables created by the OVMF reset vector. The initial page >>> tables have just 'accessed', 'dirty', 'read/write' and 'present' bits >>> set for the 0-4G identity mapping. >>> >>> It seems to have something to do with EPT. It does not happen on AMD >>> processors. It also does not happen when disabling EPT support in kvm >>> on the host machine. >>> >>> looked at kvm kernel traces, I don't see excessive vmexits. >> >> This discussion evokes vague memories in me. I'll dump them here, but I >> have no idea if they will be useful. (They probably won't.) >> >> - edk2 commit 98f378a7be12 ("OvmfPkg/ResetVector: enable caching in >> initial page tables", 2013-09-24) >> >> - Linux (host) commit 879ae1880449 ("KVM: x86: obey >> KVM_X86_QUIRK_CD_NW_CLEARED in kvm_set_cr0()", 2015-11-04) >=20 > I actually waded through the source code in both places ;) >=20 > Turned out kvm propagates guest MTRR settings to EPT memory types, > but only in case kvm_arch_has_noncoherent_dma() is true, which why > this triggers only with a mdev device assigned. ... I should not have stopped myself. :) (I'm aware that this is on edk2-devel, but a public reference to virt-staff cannot hurt.) So, yesterday I read your status on virt-staff, and I found an entry in it that resembled this upstream thread pretty closely. However, your status was the *only* mention of "mdev" specifically, and so I wasn't sure if *mdev* meant the same thing as the more generic upstream expression "pci device assignment" (see it above in the context). Furthermore, I saw kvm_arch_has_noncoherent_dma() in my linux commit 879ae1880449, which superficially resembled device assignment, but... I dismissed it. In the end, I only managed (and even that, only reluctantly) the above pointers... Thanks for tracking it down! But then, next question: why has this problem *not* been reported repeatedly? There's a whole bunch of users (gamers) that run Windows guests with device (GPU) assignment. I'm sure they'd absolutely complain about very slow OVMF boot (like they actually have, in the past, about similar LZMA slowdowns due to improper caching setup). Something must be special about Min's assigned device. Laszlo -=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D- Groups.io Links: You receive all messages sent to this group. View/Reply Online (#114285): https://edk2.groups.io/g/devel/message/114285 Mute This Topic: https://groups.io/mt/100367559/7686176 Group Owner: devel+owner@edk2.groups.io Unsubscribe: https://edk2.groups.io/g/devel/leave/12367111/7686176/19134562= 12/xyzzy [rebecca@openfw.io] -=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-