From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail.windriver.com (mail.windriver.com [147.11.1.11]) (using TLSv1.1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by ml01.01.org (Postfix) with ESMTPS id ADCE821A6F106 for ; Tue, 30 May 2017 10:30:02 -0700 (PDT) Received: from huisne.wrs.com (huisne.wrs.com [147.11.217.26]) by mail.windriver.com (8.15.2/8.15.1) with ESMTP id v4UHUrXt002460; Tue, 30 May 2017 10:30:53 -0700 (PDT) Received: from ala-wpaul-lx1.wrs.com (ala-wpaul-lx1.corp.ad.wrs.com [147.11.157.242]) by huisne.wrs.com (8.9.1/8.9.1) with ESMTP id KAA05605; Tue, 30 May 2017 10:30:53 -0700 (PDT) From: Bill Paul Organization: Wind River Systems To: edk2-devel@ml01.01.org Date: Tue, 30 May 2017 10:34:28 -0700 User-Agent: KMail/1.13.5 (Linux/2.6.32-28-generic; KDE/4.4.5; x86_64; ; ) Cc: Vladimir Olovyannikov , Ard Biesheuvel , edk2-devel@ml01.01.org References: <4220315aed43c05b37b1b71a9eff432e@mail.gmail.com> <730d8b33c76d52366585fd6055562d88@mail.gmail.com> In-Reply-To: <730d8b33c76d52366585fd6055562d88@mail.gmail.com> MIME-Version: 1.0 Message-Id: <201705301034.28519.wpaul@windriver.com> Subject: Re: Using a generic PciHostBridgeDxe driver for a multi-PCIe-domain platform X-BeenThere: edk2-devel@lists.01.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: EDK II Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 30 May 2017 17:30:02 -0000 Content-Type: Text/Plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Of all the gin joints in all the towns in all the world, Vladimir Olovyannikov had to walk into mine at 09:49:16 on Tuesday 30 May 2017 and say: > > -----Original Message----- > > From: Ard Biesheuvel [mailto:ard.biesheuvel@linaro.org] > > Sent: May-30-17 9:35 AM > > To: Vladimir Olovyannikov > > Cc: edk2-devel@lists.01.org > > Subject: Re: Using a generic PciHostBridgeDxe driver for a > > multi-PCIe-domain > > platform > > > > On 30 May 2017 at 16:23, Vladimir Olovyannikov > > > > wrote: > > > Hi, > > > > > > I've started PCIe stack implementation design for an armv8 aarch64 > > > platform. > > > The platform's PCIe represents several host bridges, and each > > > hostbridge has one rootbridge. > > > They do not share any resources between each other. > > > Looking into the PciHostBridgeDxe implementation I can see that it > > > supports only one hostbridge, and there is a comment: > > > // Most systems in the world including complex servers have only one > > > Host Bridge. > > > > > > So in my case should I create my own PciHostBridgeDxe driver > > > supporting multiple hostbridges and do not use the Industry standard > > > > driver? > > > > > I am very new to it, and will appreciate any help or idea. > > > > As far as I can tell, PciHostBridgeLib allows you to return an arbitrary > > number > > of PCI host bridges, each with their own segment number. I haven't tried > > it > > myself, but it is worth a try whether returning an array of all host > > bridges on > > your platform works as expected. > > Thank you Ard, > Right, but PciHostBridgeDxe seems to work with one hostbridge. > I am confused that > > // Make sure all root bridges share the same ResourceAssigned value > > The rootbridges are independent on the platform, and should not share > anything. Or am I missing anything? > Anyway, I will try to return multiple hostbridges in the PciHostBridgeLib. This may be an Intel-ism. Note that for PCIe, I typically refer to "host bridges" as root complexes. On PowerPC SoCs that I've worked with (e.g. Freescale/NXP MPC8548, P2020, P4080, T4240) there are often several root complexes. A typical board design may have several PCIe slots where each slot is connected to one root complex in the SoC. Each root complex is therefore the parent of a separate "segment" which has its own unique bus/dev/func space. Each root complex has its own bank of registers to control it, including a separate set of configuration space access registers. This means you can have multiple PCIe trees each with its own bus0/dev0/func0 root. There can therefore be several devices with the same bus/dev/func tuple, but which reside on separate segments. The ARMv8 board you're working with is probably set up the same way. I've only worked with ARM Cortex-A boards and those have all had just one PCIe root complex, but it stands to reason those that have multiple root complexes would follow the same pattern as the PPC devices. Intel systems can (and often do) also have multiple PCIe root complexes, however for the sake of backwards compatibility, they all end up sharing the same configuration space access registers (0xCF8/0xCFC or memory mapped extended config space) and share a single unified bus/dev/func tree. Note that the tree is not always contiguous. For example, I've seen one Intel board where there was a special PCIe device on bus 128. In the ACPI tables, there were two PCI "segments" described, the second of which corresponded to bus 128. There was no switch or bridge to connect bus 128 to the tree rooted at bus0/dev0/func0, so it would not be possible to automatically discover it by just walking the bus0/dev0/func0 tree and all its branches: you needed to use the ACPI hint to know it was there. I have also seen cases like this with pre-PCIe systems. For example, I've seen a Dell server that had both 32-bit and 64-bit PCI buses, where the 64-bit bus was at bus 1, but was not directly bridged to bus 0 (the 32-bit bus). There was a reason for this: 64-bit PCIe buses are usually clocked at 66MHz, but will fall back to 33MHz if you connect a 32-bit PCI device to them (this is supported for backward compatibility). Reducing the bus clock reduces performance, so to avoid that it's necessary to keep the 32-bit and 64-bit buses separate and thus give each one its own host bridge. As with the previous example, all the devices shared the same bus/dev/func space, but the only way to learn about the other segment was to probe the ACPI tables. It sounds as if the UEFI PCI host bridge code may be biased towards the Intel PCI behavior, though I'm not sure to what extent. So the comment that you found that says: // Most systems in the world including complex servers have only one Host Bridge. Should probably be amended: it should probably say "Most Intel systems" and even those systems probably do have more than one host bridge (root complex), it's just that it doesn't look like it. -Bill > Thank you, > Vladimir > _______________________________________________ > edk2-devel mailing list > edk2-devel@lists.01.org > https://lists.01.org/mailman/listinfo/edk2-devel -- ============================================================================= -Bill Paul (510) 749-2329 | Senior Member of Technical Staff, wpaul@windriver.com | Master of Unix-Fu - Wind River Systems ============================================================================= "I put a dollar in a change machine. Nothing changed." - George Carlin =============================================================================