From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from smtp-fw-33001.amazon.com (smtp-fw-33001.amazon.com [207.171.190.10]) by mx.groups.io with SMTP id smtpd.web10.7817.1643121404678725345 for ; Tue, 25 Jan 2022 06:36:44 -0800 Authentication-Results: mx.groups.io; dkim=pass header.i=@amazon.com header.s=amazon201209 header.b=blOwZ30q; spf=pass (domain: amazon.de, ip: 207.171.190.10, mailfrom: prvs=0179bbfd8=ncoleon@amazon.de) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amazon.com; i=@amazon.com; q=dns/txt; s=amazon201209; t=1643121404; x=1674657404; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version; bh=PLyHgnALXsmWr0AuZdNwca02q3/6m9dGfZzemCqYXFo=; b=blOwZ30qfMr/rq21sb55EztfR+4V0jot+zBuHDL90U9mGxyhJDqVWlou xxcCsaSGebyPWixChLCjNUByvOB656GKiL8+JlsbIh80Nkkga5ZPrjjmU KoC9ASTFxY39K3MQYNxL+BizU4GJrvoaNof4pUNmYD902/PL3xiWJneTi 8=; X-IronPort-AV: E=Sophos;i="5.88,315,1635206400"; d="scan'208";a="172034600" Received: from iad12-co-svc-p1-lb1-vlan2.amazon.com (HELO email-inbound-relay-pdx-2a-cb1ffea5.us-west-2.amazon.com) ([10.43.8.2]) by smtp-border-fw-33001.sea14.amazon.com with ESMTP; 25 Jan 2022 14:36:29 +0000 Received: from EX13D49EUC003.ant.amazon.com (pdx1-ws-svc-p6-lb9-vlan2.pdx.amazon.com [10.236.137.194]) by email-inbound-relay-pdx-2a-cb1ffea5.us-west-2.amazon.com (Postfix) with ESMTPS id E2B4360ECC; Tue, 25 Jan 2022 14:36:28 +0000 (UTC) Received: from ub4014a598e6c52.ant.amazon.com (10.43.161.126) by EX13D49EUC003.ant.amazon.com (10.43.164.91) with Microsoft SMTP Server (TLS) id 15.0.1497.28; Tue, 25 Jan 2022 14:36:24 +0000 From: "Ojeda Leon, Nicolas" To: CC: , Nicolas Ojeda Leon , Alexander Graf , Gerd Hoffmann Subject: [PATCH v3 5/8] OvmfPkg/PciHostBridgeUtilityLib: Initialize RootBridges apertures with spec Date: Tue, 25 Jan 2022 15:35:55 +0100 Message-ID: X-Mailer: git-send-email 2.17.1 In-Reply-To: References: MIME-Version: 1.0 X-Originating-IP: [10.43.161.126] X-ClientProxiedBy: EX13D35UWB004.ant.amazon.com (10.43.161.230) To EX13D49EUC003.ant.amazon.com (10.43.164.91) Content-Type: text/plain Consume the host-provided specification of PCI host bridges if available. Using the DxeHardwareInfoLib, populate a list of hardware descriptors based on the content of the "hardware-info" fw-cfg file, if provided. In the affirmative case, use the resources and attributes specified by the hypervisor for each Host Bridge to create the RootBridge elements. In Ovmf platforms, the host can provide the specification of non-discoverable hardware resources like PCI host bridges. If the proper fw-cfg file is found, parse the contents provided by the host into a linked list by using the Hardware Info library. Then, using the list of PCI host bridges' descriptions, populate the PCI_ROOT_BRIDGES array with the resources and attributes specified by the host. If the file is not provided or no Host Bridge is found in it, fold back to the legacy method based on pre-defined apertures and rules. In some use cases, the host requires additional control over the hardware resources' configurations in the guest for performance and discoverability reasons. For instance, to disclose information about the PCI hierarchy to the guest so that this can profit from optimized accesses. In this case, the host can decide to describe multiple PCI Host Bridges and provide a specific set of resources (e.g. MMIO apertures) so that the guest uses the values provided. Using the provided values may entitle the guest to added performance, for example by using specific MMIO mappings that can enable peer-to-peer communication across the PCI hierarchy or by allocating memory closer to a device for faster DMA transactions. Cc: Alexander Graf Cc: Gerd Hoffmann Signed-off-by: Nicolas Ojeda Leon --- .../PciHostBridgeUtilityLib.c | 345 ++++++++++++++---- .../PciHostBridgeUtilityLib.inf | 1 + 2 files changed, 268 insertions(+), 78 deletions(-) diff --git a/OvmfPkg/Library/PciHostBridgeUtilityLib/PciHostBridgeUtilityLib.c b/OvmfPkg/Library/PciHostBridgeUtilityLib/PciHostBridgeUtilityLib.c index 92e1ea812f..b0e3b5e3cf 100644 --- a/OvmfPkg/Library/PciHostBridgeUtilityLib/PciHostBridgeUtilityLib.c +++ b/OvmfPkg/Library/PciHostBridgeUtilityLib/PciHostBridgeUtilityLib.c @@ -12,13 +12,16 @@ #include #include +#include #include #include #include +#include #include #include #include #include +#include #pragma pack(1) typedef struct { @@ -234,14 +237,29 @@ PciHostBridgeUtilityGetRootBridges ( IN PCI_ROOT_BRIDGE_APERTURE *PMemAbove4G ) { - EFI_STATUS Status; - FIRMWARE_CONFIG_ITEM FwCfgItem; - UINTN FwCfgSize; - UINT64 ExtraRootBridges; - PCI_ROOT_BRIDGE *Bridges; - UINTN Initialized; - UINTN LastRootBridgeNumber; - UINTN RootBridgeNumber; + EFI_STATUS Status; + FIRMWARE_CONFIG_ITEM FwCfgItem; + UINTN FwCfgSize; + UINT64 ExtraRootBridges; + PCI_ROOT_BRIDGE *Bridges; + UINTN Initialized; + UINTN LastRootBridgeNumber; + UINTN RootBridgeNumber; + UINTN PciHostBridgeCount; + UINT8 *HardwareInfoBlob; + LIST_ENTRY HwInfoList; + LIST_ENTRY *HwLink; + HARDWARE_INFO *HwInfo; + UINT64 HwInfoAttributes; + UINT64 HwInfoAllocationAttributes; + BOOLEAN HwInfoDmaAbove4G; + BOOLEAN HwInfoNoExtendedConfigSpace; + BOOLEAN HwInfoCombineMemPMem; + PCI_ROOT_BRIDGE_APERTURE HwInfoIo; + PCI_ROOT_BRIDGE_APERTURE HwInfoMem; + PCI_ROOT_BRIDGE_APERTURE HwInfoMemAbove4G; + PCI_ROOT_BRIDGE_APERTURE HwInfoPMem; + PCI_ROOT_BRIDGE_APERTURE HwInfoPMemAbove4G; *Count = 0; @@ -294,103 +312,266 @@ PciHostBridgeUtilityGetRootBridges ( )); } + // + // Initialize the Hardware Info list head to start with an empty but valid + // list head. + // + InitializeListHead (&HwInfoList); + HardwareInfoBlob = NULL; + Initialized = 0; + Bridges = NULL; + PciHostBridgeCount = 0; + + // + // Hypervisor can provide the specifications (resources) for one or more + // PCI host bridges. Such information comes through fw-cfg as part of + // the hardware-info file. + // + Status = QemuFwCfgFindFile ("etc/hardware-info", &FwCfgItem, &FwCfgSize); + + if (!EFI_ERROR (Status)) { + HardwareInfoBlob = AllocatePool (FwCfgSize); + + if (HardwareInfoBlob == NULL) { + DEBUG (( + DEBUG_ERROR, + "%a: Failed to allocate memory for hardware resources info\n", + __FUNCTION__ + )); + return NULL; + } + + QemuFwCfgSelectItem (FwCfgItem); + QemuFwCfgReadBytes (FwCfgSize, HardwareInfoBlob); + + // + // Create the list of hardware info devices filtering for PCI host + // bridges + // + Status = CreateHardwareInfoList ( + HardwareInfoBlob, + FwCfgSize, + HardwareInfoTypeHostBridge, + &HwInfoList + ); + + if (EFI_ERROR (Status)) { + DEBUG (( + DEBUG_ERROR, + "%a: Failed to create hardware info list to retrieve host " + "bridges information\n", + __FUNCTION__ + )); + + goto FreeBridges; + } + + PciHostBridgeCount = GetHardwareInfoCountByType ( + &HwInfoList, + HardwareInfoTypeHostBridge, + sizeof (HOST_BRIDGE_INFO) + ); + + if ((1 + ExtraRootBridges) != PciHostBridgeCount) { + DEBUG (( + DEBUG_ERROR, + "%a: Invalid number of host bridges in hardware info list. " + "Expected %ld, got %ld\n", + __FUNCTION__, + (1 + ExtraRootBridges), + PciHostBridgeCount + )); + goto FreeBridges; + } + + DEBUG (( + DEBUG_INFO, + "%a: Resources for %Lu root buses found in fw-cfg\n", + __FUNCTION__, + PciHostBridgeCount + )); + } + // // Allocate the "main" root bridge, and any extra root bridges. // Bridges = AllocatePool ((1 + (UINTN)ExtraRootBridges) * sizeof *Bridges); if (Bridges == NULL) { DEBUG ((DEBUG_ERROR, "%a: %r\n", __FUNCTION__, EFI_OUT_OF_RESOURCES)); - return NULL; + goto FreeBridges; } - Initialized = 0; + if (!IsListEmpty (&HwInfoList)) { + // + // If Host Bridges' specification was obtained from fw-cfg, the list + // contains information to populate all root bridges in the system + // including resources and attributes. + // + HwLink = GetFirstHardwareInfoByType ( + &HwInfoList, + HardwareInfoTypeHostBridge, + sizeof (HOST_BRIDGE_INFO) + ); + + while (!EndOfHardwareInfoList (&HwInfoList, HwLink)) { + HwInfo = HARDWARE_INFO_FROM_LINK (HwLink); + + HardwareInfoPciHostBridgeGet ( + HwInfo->Data.PciHostBridge, + HwInfo->Header.Size, + &RootBridgeNumber, + &LastRootBridgeNumber, + &HwInfoAttributes, + &HwInfoDmaAbove4G, + &HwInfoNoExtendedConfigSpace, + &HwInfoCombineMemPMem, + &HwInfoIo, + &HwInfoMem, + &HwInfoMemAbove4G, + &HwInfoPMem, + &HwInfoPMemAbove4G, + NULL + ); - // - // The "main" root bus is always there. - // - LastRootBridgeNumber = BusMin; + HwInfoAllocationAttributes = 0; + if (HwInfoCombineMemPMem) { + HwInfoAllocationAttributes |= EFI_PCI_HOST_BRIDGE_COMBINE_MEM_PMEM; + } - // - // Scan all other root buses. If function 0 of any device on a bus returns a - // VendorId register value different from all-bits-one, then that bus is - // alive. - // - for (RootBridgeNumber = BusMin + 1; - RootBridgeNumber <= BusMax && Initialized < ExtraRootBridges; - ++RootBridgeNumber) - { - UINTN Device; - - for (Device = 0; Device <= PCI_MAX_DEVICE; ++Device) { - if (PciRead16 ( - PCI_LIB_ADDRESS ( - RootBridgeNumber, - Device, - 0, - PCI_VENDOR_ID_OFFSET - ) - ) != MAX_UINT16) + if ((HwInfoMemAbove4G.Limit > HwInfoMemAbove4G.Base) || + (HwInfoPMemAbove4G.Limit > HwInfoPMemAbove4G.Base)) { - break; + HwInfoAllocationAttributes |= EFI_PCI_HOST_BRIDGE_MEM64_DECODE; } - } - if (Device <= PCI_MAX_DEVICE) { - // - // Found the next root bus. We can now install the *previous* one, - // because now we know how big a bus number range *that* one has, for any - // subordinate buses that might exist behind PCI bridges hanging off it. - // Status = PciHostBridgeUtilityInitRootBridge ( - Attributes, - Attributes, - AllocationAttributes, - DmaAbove4G, - NoExtendedConfigSpace, - (UINT8)LastRootBridgeNumber, - (UINT8)(RootBridgeNumber - 1), - Io, - Mem, - MemAbove4G, - PMem, - PMemAbove4G, + HwInfoAttributes, + HwInfoAttributes, + HwInfoAllocationAttributes, + HwInfoDmaAbove4G, + HwInfoNoExtendedConfigSpace, + RootBridgeNumber, + LastRootBridgeNumber, + &HwInfoIo, + &HwInfoMem, + &HwInfoMemAbove4G, + &HwInfoPMem, + &HwInfoPMemAbove4G, &Bridges[Initialized] ); + if (EFI_ERROR (Status)) { goto FreeBridges; } ++Initialized; - LastRootBridgeNumber = RootBridgeNumber; + + HwLink = GetNextHardwareInfoByType ( + &HwInfoList, + HwLink, + HardwareInfoTypeHostBridge, + sizeof (HOST_BRIDGE_INFO) + ); } + } else { + Initialized = 0; + + // + // The "main" root bus is always there. + // + LastRootBridgeNumber = BusMin; + + // + // Scan all other root buses. If function 0 of any device on a bus returns a + // VendorId register value different from all-bits-one, then that bus is + // alive. + // + for (RootBridgeNumber = BusMin + 1; + RootBridgeNumber <= BusMax && Initialized < ExtraRootBridges; + ++RootBridgeNumber) + { + UINTN Device; + + for (Device = 0; Device <= PCI_MAX_DEVICE; ++Device) { + if (PciRead16 ( + PCI_LIB_ADDRESS ( + RootBridgeNumber, + Device, + 0, + PCI_VENDOR_ID_OFFSET + ) + ) != MAX_UINT16) + { + break; + } + } + + if (Device <= PCI_MAX_DEVICE) { + // + // Found the next root bus. We can now install the *previous* one, + // because now we know how big a bus number range *that* one has, for any + // subordinate buses that might exist behind PCI bridges hanging off it. + // + Status = PciHostBridgeUtilityInitRootBridge ( + Attributes, + Attributes, + AllocationAttributes, + DmaAbove4G, + NoExtendedConfigSpace, + (UINT8)LastRootBridgeNumber, + (UINT8)(RootBridgeNumber - 1), + Io, + Mem, + MemAbove4G, + PMem, + PMemAbove4G, + &Bridges[Initialized] + ); + if (EFI_ERROR (Status)) { + goto FreeBridges; + } + + ++Initialized; + LastRootBridgeNumber = RootBridgeNumber; + } + } + + // + // Install the last root bus (which might be the only, ie. main, root bus, if + // we've found no extra root buses). + // + Status = PciHostBridgeUtilityInitRootBridge ( + Attributes, + Attributes, + AllocationAttributes, + DmaAbove4G, + NoExtendedConfigSpace, + (UINT8)LastRootBridgeNumber, + (UINT8)BusMax, + Io, + Mem, + MemAbove4G, + PMem, + PMemAbove4G, + &Bridges[Initialized] + ); + if (EFI_ERROR (Status)) { + goto FreeBridges; + } + + ++Initialized; } + *Count = Initialized; + + // If resources were allocated for host bridges info, release them // - // Install the last root bus (which might be the only, ie. main, root bus, if - // we've found no extra root buses). - // - Status = PciHostBridgeUtilityInitRootBridge ( - Attributes, - Attributes, - AllocationAttributes, - DmaAbove4G, - NoExtendedConfigSpace, - (UINT8)LastRootBridgeNumber, - (UINT8)BusMax, - Io, - Mem, - MemAbove4G, - PMem, - PMemAbove4G, - &Bridges[Initialized] - ); - if (EFI_ERROR (Status)) { - goto FreeBridges; + if (HardwareInfoBlob) { + FreePool (HardwareInfoBlob); } - ++Initialized; + FreeHardwareInfoList (&HwInfoList); - *Count = Initialized; return Bridges; FreeBridges: @@ -399,7 +580,15 @@ FreeBridges: PciHostBridgeUtilityUninitRootBridge (&Bridges[Initialized]); } - FreePool (Bridges); + if (Bridges) { + FreePool (Bridges); + } + + if (HardwareInfoBlob) { + FreePool (HardwareInfoBlob); + } + + FreeHardwareInfoList (&HwInfoList); return NULL; } diff --git a/OvmfPkg/Library/PciHostBridgeUtilityLib/PciHostBridgeUtilityLib.inf b/OvmfPkg/Library/PciHostBridgeUtilityLib/PciHostBridgeUtilityLib.inf index 83a734c172..e4fc903121 100644 --- a/OvmfPkg/Library/PciHostBridgeUtilityLib/PciHostBridgeUtilityLib.inf +++ b/OvmfPkg/Library/PciHostBridgeUtilityLib/PciHostBridgeUtilityLib.inf @@ -38,6 +38,7 @@ BaseMemoryLib DebugLib DevicePathLib + DxeHardwareInfoLib MemoryAllocationLib PciLib QemuFwCfgLib -- 2.17.1 Amazon Development Center Germany GmbH Krausenstr. 38 10117 Berlin Geschaeftsfuehrung: Christian Schlaeger, Jonathan Weiss Eingetragen am Amtsgericht Charlottenburg unter HRB 149173 B Sitz: Berlin Ust-ID: DE 289 237 879