Re: [edk2-devel] [PATCH edk2-platforms v2 0/3] SbsaQemu: support multiple PCI Express buses

["Ard Biesheuvel" <ardb@kernel.org>]

On Tue, 28 May 2024 at 12:31, Marcin Juszkiewicz
<marcin.juszkiewicz@linaro.org> wrote:
>
> QEMU allows to have NUMA setup where each node has own cpus, memory and
> i/o. We already handle cpus and memory. This patchset adds support for
> having multiple PCI Express buses.
>
> SbsaQemu assumed that there is only bus 0. First patch does PCIe bus
> scan to find all host bridges (bus 0 one and additional 'pxb-pcie'
> ones).
>
> Second patch moves description of PCIe from DSDT to SSDT (one per each
> PCIe bus). So Operating System will know about all of them.
>
> Third patch moves generation of MCFG table to C. It is preparation to
> move PCIe Pcds from being fixed to dynamic ones.
>
> There are some booting issues with assigning resources for cards:
>
> pci 0000:00:03.0: BAR 15: no space for [mem size 0x00200000 64bit pref]
> pci 0000:00:03.0: BAR 15: failed to assign [mem size 0x00200000 64bit pref]
> pci 0000:00:01.0: BAR 6: no space for [mem size 0x00040000 pref]
> pci 0000:00:01.0: BAR 6: failed to assign [mem size 0x00040000 pref]
> pci 0000:00:03.0: BAR 13: no space for [io  size 0x1000]
> pci 0000:00:03.0: BAR 13: failed to assign [io  size 0x1000]
>
> Boot log (Linux + lspci + ACPI tables dump):
> https://people.linaro.org/~marcin.juszkiewicz/sbsa-ref/boot-linux-with-numa-multiple-pcie-buses.txt
>
> I am wondering where I made mistakes in handling PCIe buses.
>

I would expect each host bridge to have its own separate resource
windows for config space, buses and MMIO regions.

So each host bridge gets a different segment number, and each segment
is associated with a different ECAM region. That also means the bus
range can start at 0x0 for each segment, as they are completely
disjoint.

This is a more accurate representation of the physical topology, given
that each host bridge has its own link to the CPU side interconnect,
and so things like peer-to-peer DMA between endpoints does not
generally work unless the endpoints share a segment, especially in the
presence of SMMUs.


-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.
View/Reply Online (#119308): https://edk2.groups.io/g/devel/message/119308
Mute This Topic: https://groups.io/mt/106345969/7686176
Group Owner: devel+owner@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub [rebecca@openfw.io]
-=-=-=-=-=-=-=-=-=-=-=-