From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail05.groups.io (mail05.groups.io [45.79.224.7]) by spool.mail.gandi.net (Postfix) with ESMTPS id EDE16740040 for ; Tue, 28 May 2024 14:31:22 +0000 (UTC) DKIM-Signature: a=rsa-sha256; bh=4WcIzRGeqyruTMuBVMZNA5LGm4ZjhtHYUNvsmbHL+3M=; c=relaxed/simple; d=groups.io; h=MIME-Version:References:In-Reply-To:From:Date:Message-ID:Subject:To:Cc:Precedence:List-Subscribe:List-Help:Sender:List-Id:Mailing-List:Delivered-To:Resent-Date:Resent-From:Reply-To:List-Unsubscribe-Post:List-Unsubscribe:Content-Type; s=20240206; t=1716906682; v=1; b=sNh7V0KzlpMDN17ftMesM0XpXUp7931fkXDXBwo+vS5nriJl1tP2yVob0Rbuu6PO9MsHE7RG Epz5oHgJca4p3n2cU0kM0B2RuKmM1QjbYXReVSguxYyMJRbuORmB2IgwNq2TUDSu++MUfCGCJ5h qr7i8PhKpVAxxj5sp5ecBJt0e1oZQ5N+mDbOtJgRFEMMFCLMoEh4i/JLgN2elg7OdtNBg09gpmU aA+f8Lfs3lqLnIYA9B7vkFGLObm2aC2j+Lzhsf1CXTF2wr6snXZt6k+WAs4QkktGpDnp58XINzl KMvUeiD+T4F07nVKGGqbA7g3ERwIKAKFJdmj/EIMgSUpQ== X-Received: by 127.0.0.2 with SMTP id IZBEYY7687511xAosufj5i3A; Tue, 28 May 2024 07:31:21 -0700 X-Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by mx.groups.io with SMTP id smtpd.web10.23479.1716906680656282108 for ; Tue, 28 May 2024 07:31:20 -0700 X-Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by dfw.source.kernel.org (Postfix) with ESMTP id 26EBD62127 for ; Tue, 28 May 2024 14:31:20 +0000 (UTC) X-Received: by smtp.kernel.org (Postfix) with ESMTPSA id D4E3EC32789 for ; Tue, 28 May 2024 14:31:19 +0000 (UTC) X-Received: by mail-lj1-f172.google.com with SMTP id 38308e7fff4ca-2e96f29884dso9260601fa.0 for ; Tue, 28 May 2024 07:31:19 -0700 (PDT) X-Gm-Message-State: o1h2D4AFTd5zfxLBcIRMP335x7686176AA= X-Google-Smtp-Source: AGHT+IH9yMyqLlZGOrseAtMKH+jraHmqMSo0e1BaKphgnH3B10mMzMUDwY6awhzUNw9rs3hOKNoT6+nollaXhD7BUSs= X-Received: by 2002:a05:651c:234:b0:2e9:84ee:ae7a with SMTP id 38308e7fff4ca-2e984eeb084mr16666851fa.48.1716906678105; Tue, 28 May 2024 07:31:18 -0700 (PDT) MIME-Version: 1.0 References: <20240528-review-multiple-pcie-0425-v2-0-e2ec9f098a78@linaro.org> In-Reply-To: <20240528-review-multiple-pcie-0425-v2-0-e2ec9f098a78@linaro.org> From: "Ard Biesheuvel" Date: Tue, 28 May 2024 16:31:06 +0200 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: [edk2-devel] [PATCH edk2-platforms v2 0/3] SbsaQemu: support multiple PCI Express buses To: devel@edk2.groups.io, marcin.juszkiewicz@linaro.org Cc: Leif Lindholm , Graeme Gregory , Ray Ni Precedence: Bulk List-Subscribe: List-Help: Sender: devel@edk2.groups.io List-Id: Mailing-List: list devel@edk2.groups.io; contact devel+owner@edk2.groups.io Resent-Date: Tue, 28 May 2024 07:31:20 -0700 Resent-From: ardb@kernel.org Reply-To: devel@edk2.groups.io,ardb@kernel.org List-Unsubscribe-Post: List-Unsubscribe=One-Click List-Unsubscribe: Content-Type: text/plain; charset="UTF-8" X-GND-Status: LEGIT Authentication-Results: spool.mail.gandi.net; dkim=pass header.d=groups.io header.s=20240206 header.b=sNh7V0Kz; dmarc=fail reason="SPF not aligned (relaxed), DKIM not aligned (relaxed)" header.from=kernel.org (policy=none); spf=pass (spool.mail.gandi.net: domain of bounce@groups.io designates 45.79.224.7 as permitted sender) smtp.mailfrom=bounce@groups.io On Tue, 28 May 2024 at 12:31, Marcin Juszkiewicz wrote: > > QEMU allows to have NUMA setup where each node has own cpus, memory and > i/o. We already handle cpus and memory. This patchset adds support for > having multiple PCI Express buses. > > SbsaQemu assumed that there is only bus 0. First patch does PCIe bus > scan to find all host bridges (bus 0 one and additional 'pxb-pcie' > ones). > > Second patch moves description of PCIe from DSDT to SSDT (one per each > PCIe bus). So Operating System will know about all of them. > > Third patch moves generation of MCFG table to C. It is preparation to > move PCIe Pcds from being fixed to dynamic ones. > > There are some booting issues with assigning resources for cards: > > pci 0000:00:03.0: BAR 15: no space for [mem size 0x00200000 64bit pref] > pci 0000:00:03.0: BAR 15: failed to assign [mem size 0x00200000 64bit pref] > pci 0000:00:01.0: BAR 6: no space for [mem size 0x00040000 pref] > pci 0000:00:01.0: BAR 6: failed to assign [mem size 0x00040000 pref] > pci 0000:00:03.0: BAR 13: no space for [io size 0x1000] > pci 0000:00:03.0: BAR 13: failed to assign [io size 0x1000] > > Boot log (Linux + lspci + ACPI tables dump): > https://people.linaro.org/~marcin.juszkiewicz/sbsa-ref/boot-linux-with-numa-multiple-pcie-buses.txt > > I am wondering where I made mistakes in handling PCIe buses. > I would expect each host bridge to have its own separate resource windows for config space, buses and MMIO regions. So each host bridge gets a different segment number, and each segment is associated with a different ECAM region. That also means the bus range can start at 0x0 for each segment, as they are completely disjoint. This is a more accurate representation of the physical topology, given that each host bridge has its own link to the CPU side interconnect, and so things like peer-to-peer DMA between endpoints does not generally work unless the endpoints share a segment, especially in the presence of SMMUs. -=-=-=-=-=-=-=-=-=-=-=- Groups.io Links: You receive all messages sent to this group. View/Reply Online (#119308): https://edk2.groups.io/g/devel/message/119308 Mute This Topic: https://groups.io/mt/106345969/7686176 Group Owner: devel+owner@edk2.groups.io Unsubscribe: https://edk2.groups.io/g/devel/unsub [rebecca@openfw.io] -=-=-=-=-=-=-=-=-=-=-=-