From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by mx.groups.io with SMTP id smtpd.web10.26453.1673450627651981094 for ; Wed, 11 Jan 2023 07:23:48 -0800 Authentication-Results: mx.groups.io; dkim=pass header.i=@redhat.com header.s=mimecast20190719 header.b=EI/M7okt; spf=pass (domain: redhat.com, ip: 170.10.129.124, mailfrom: kraxel@redhat.com) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1673450626; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=5L3ockXo++XniVlX3sQCYPlsbG8vQhxKhB9PVhHejMc=; b=EI/M7okt8fVnt6CjZeStzYC4Zm+Lr1ZRIZaN9Z8o47hpefc64ibB5edOHJIArIBqWwcVB8 SksCKI4xunPzbPXauXALo1emtkR1rEtqdCy0CJVRWkw7eZM493maFKNa0zI1tUa2SA03u5 bN5URyPXiOEWpzfFYQg8Tnvd++fSm14= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-43-4SY9bB6wMYqXedrj36DtFQ-1; Wed, 11 Jan 2023 10:23:43 -0500 X-MC-Unique: 4SY9bB6wMYqXedrj36DtFQ-1 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.rdu2.redhat.com [10.11.54.6]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 766DB100F902; Wed, 11 Jan 2023 15:23:43 +0000 (UTC) Received: from sirius.home.kraxel.org (unknown [10.39.192.238]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 422112166B26; Wed, 11 Jan 2023 15:23:43 +0000 (UTC) Received: by sirius.home.kraxel.org (Postfix, from userid 1000) id D87661800613; Wed, 11 Jan 2023 16:23:41 +0100 (CET) Date: Wed, 11 Jan 2023 16:23:41 +0100 From: "Gerd Hoffmann" To: Laszlo Ersek Cc: devel@edk2.groups.io, Pawel Polawski , Jiewen Yao , Oliver Steffen , Jordan Justen , Ard Biesheuvel Subject: Re: [PATCH v2 2/4] OvmfPkg/PlatformInitLib: Add PlatformGetLowMemoryCB Message-ID: <20230111152341.d3p6iy5pml7shfvk@sirius.home.kraxel.org> References: <20230110082123.159521-1-kraxel@redhat.com> <20230110082123.159521-3-kraxel@redhat.com> <043b03d6-0a6b-c533-255b-24a7805d5cca@redhat.com> <20230111072925.l47t4ahgynqjyegq@sirius.home.kraxel.org> <4ace3789-7192-0c53-b4b2-f62f907176d0@redhat.com> MIME-Version: 1.0 In-Reply-To: <4ace3789-7192-0c53-b4b2-f62f907176d0@redhat.com> X-Scanned-By: MIMEDefang 3.1 on 10.11.54.6 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset=us-ascii Content-Disposition: inline > > root@fedora ~# cat /proc/iomem > > [ ... ] > > 7ebfe000-7effffff : System RAM > > 7f000000-7fffffff : Reserved > > 80000000-afffffff : PCI Bus 0000:00 > > b0000000-bfffffff : PCI MMCONFIG 0000 [bus 00-ff] > > b0000000-bfffffff : Reserved > > b0000000-bfffffff : pnp 00:04 > > c0000000-febfffff : PCI Bus 0000:00 > > [ ... ] > > root@fedora ~# cat /proc/mtrr > > reg00: base=0x080000000 ( 2048MB), size= 2048MB, count=1: uncachable > > reg01: base=0x800000000 (32768MB), size=32768MB, count=1: uncachable > > Ugh, what? :) > > I was about to point out a contradiction between (a) the following from > commit 2a0bd3bffc80: > > + // Newer qemu with gigabyte aligned memory, > + // 32-bit pci mmio window is 2G -> 4G then. > > and (b) your confirmation that the PCIEXBAR location does not change. > Namely, I was about to point out that PCIEXBAR -- *config space* > expressed as MMIO -- would then overlap the 32-bit MMIO aperture, the > one that's assignable to BARs as MMIO space. > > But then your /proc/iomem quote actually confirms this is what happens > in practice -- and apparently works??? In Linux anyways? > > FWIW I don't see how this is safe with regard to the firmware. Even if > QEMU is capable of generating a set of discontiguous resource > descriptors in the DSDT / _CRS, and Linux is capable of dealing with > that, I don't understand how the firmware does it. It doesn't. It still operates with the 0xc0000000+ range as 32bit mmio window. Which is why the 80000000-afffffff range is unused. Linux could map hotplug device resources there, but that's it. > > Bus: 0 - FF Translation=0 > > Io: 6000 - FFFF Translation=0 > > Mem: C0000000 - FBFFFFFF Translation=0 > > MemAbove4G: 800000000 - FFFFFFFFF Translation=0 > > PMem: FFFFFFFFFFFFFFFF - 0 Translation=0 > > PMemAbove4G: FFFFFFFFFFFFFFFF - 0 Translation=0 > > Note "Mem: C0000000 - FBFFFFFF Translation=0". Yes. > Therefore, I also don't understand where the requirement comes (from > Linux? where?) that the firmware mark the "gap" between 2048 MB and > 2816 MB as uncached. The firmware does not use it for anything, so why > does the Linux kernel do? And if the Linux kernel does, then why does > it not reprogram the MTRRs as well? Some test case complained because the 80000000-afffffff range is io address space (according to /proc/iomem) but not tagged as uncachable in mtrr registers. > The commit message from commit 2a0bd3bffc80 states, "Which effectively > makes the 32-bit pci mmio window start at 0x80000000". .. according to the guest os view because qemu generates _CRS resources with 80000000-afffffff included. > I'm precisely after that "effectively" adverb here: placing the 32-bit > MMIO aperture at 2048 MB is not *at all* what the firmware does. Yes. > I've filed a new TianoCore BZ about clarifying the comments please: > > https://bugzilla.tianocore.org/show_bug.cgi?id=4289 OK. > > With gigabyte-alignment being the common case these days it might make > > sense to place the MMCONFIG area at 0xe0000000 instead ... > > I feel really unsafe about complicating this code even further. I think it should actually simplify things. All the inconsistencies we have (as you outlined above) due to the hole punching and edk2 supporting only a single range for 32bit mmio should go away, and we will have less address space layout differences between q35 and pc. We'll set LowMemory -> 4G to UC via mtrr (both pc and q35) We'll use LowMemory -> 0xFBFFFFFF (pc) or LowMemory -> 0xdfffffff (q35) as 32bit mmio window. We'll use 0xe0000000 -> 0xeffffffff for mmconfig (q35 only). Qemu will add 0xf0000000 -> 0xFBFFFFFF to the PCI bus _CRS so linux could use it but the firmware wouldn't do anything with it (q35 only). take care, Gerd