Re: [PATCH v2 2/4] OvmfPkg/PlatformInitLib: Add PlatformGetLowMemoryCB

["Gerd Hoffmann" <kraxel@redhat.com>]

> > root@fedora ~# cat /proc/iomem
> > [ ... ]
> > 7ebfe000-7effffff : System RAM
> > 7f000000-7fffffff : Reserved
> > 80000000-afffffff : PCI Bus 0000:00
> > b0000000-bfffffff : PCI MMCONFIG 0000 [bus 00-ff]
> >   b0000000-bfffffff : Reserved
> >     b0000000-bfffffff : pnp 00:04
> > c0000000-febfffff : PCI Bus 0000:00
> > [ ... ]
> > root@fedora ~# cat /proc/mtrr
> > reg00: base=0x080000000 ( 2048MB), size= 2048MB, count=1: uncachable
> > reg01: base=0x800000000 (32768MB), size=32768MB, count=1: uncachable
> 
> Ugh, what? :)
> 
> I was about to point out a contradiction between (a) the following from
> commit 2a0bd3bffc80:
> 
> +      // Newer qemu with gigabyte aligned memory,
> +      // 32-bit pci mmio window is 2G -> 4G then.
> 
> and (b) your confirmation that the PCIEXBAR location does not change.
> Namely, I was about to point out that PCIEXBAR -- *config space*
> expressed as MMIO -- would then overlap the 32-bit MMIO aperture, the
> one that's assignable to BARs as MMIO space.
> 
> But then your /proc/iomem quote actually confirms this is what happens
> in practice -- and apparently works??? In Linux anyways?
> 
> FWIW I don't see how this is safe with regard to the firmware. Even if
> QEMU is capable of generating a set of discontiguous resource
> descriptors in the DSDT / _CRS, and Linux is capable of dealing with
> that, I don't understand how the firmware does it.

It doesn't.  It still operates with the 0xc0000000+ range as 32bit mmio
window.  Which is why the 80000000-afffffff range is unused.  Linux
could map hotplug device resources there, but that's it.

> >            Bus: 0 - FF Translation=0
> >             Io: 6000 - FFFF Translation=0
> >            Mem: C0000000 - FBFFFFFF Translation=0
> >     MemAbove4G: 800000000 - FFFFFFFFF Translation=0
> >           PMem: FFFFFFFFFFFFFFFF - 0 Translation=0
> >    PMemAbove4G: FFFFFFFFFFFFFFFF - 0 Translation=0
> 
>   Note "Mem: C0000000 - FBFFFFFF Translation=0".

Yes.

>   Therefore, I also don't understand where the requirement comes (from
>   Linux? where?) that the firmware mark the "gap" between 2048 MB and
>   2816 MB as uncached. The firmware does not use it for anything, so why
>   does the Linux kernel do? And if the Linux kernel does, then why does
>   it not reprogram the MTRRs as well?

Some test case complained because the 80000000-afffffff range is io
address space (according to /proc/iomem) but not tagged as uncachable
in mtrr registers.

>   The commit message from commit 2a0bd3bffc80 states, "Which effectively
>   makes the 32-bit pci mmio window start at 0x80000000".

.. according to the guest os view because qemu generates _CRS resources
with 80000000-afffffff included.

>   I'm precisely after that "effectively" adverb here: placing the 32-bit
>   MMIO aperture at 2048 MB is not *at all* what the firmware does.

Yes.

> I've filed a new TianoCore BZ about clarifying the comments please:
> 
>   https://bugzilla.tianocore.org/show_bug.cgi?id=4289

OK.

> > With gigabyte-alignment being the common case these days it might make
> > sense to place the MMCONFIG area at 0xe0000000 instead ...
> 
> I feel really unsafe about complicating this code even further.

I think it should actually simplify things.  All the inconsistencies we
have (as you outlined above) due to the hole punching and edk2
supporting only a single range for 32bit mmio should go away, and we
will have less address space layout differences between q35 and pc.

We'll set LowMemory  -> 4G to UC via mtrr (both pc and q35)

We'll use LowMemory  -> 0xFBFFFFFF (pc) or
          LowMemory  -> 0xdfffffff (q35) as 32bit mmio window.

We'll use 0xe0000000 -> 0xeffffffff for mmconfig (q35 only).

Qemu will add 0xf0000000 -> 0xFBFFFFFF to the PCI bus _CRS so
linux could use it but the firmware wouldn't do anything with
it (q35 only).

take care,
  Gerd