From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from smtp-fw-80006.amazon.com (smtp-fw-80006.amazon.com [99.78.197.217]) by mx.groups.io with SMTP id smtpd.web09.191.1655847693835176719 for ; Tue, 21 Jun 2022 14:41:34 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=@amazon.com header.s=amazon201209 header.b=YJKSmalm; spf=pass (domain: amazon.de, ip: 99.78.197.217, mailfrom: prvs=1645a0c96=ncoleon@amazon.de) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amazon.com; i=@amazon.com; q=dns/txt; s=amazon201209; t=1655847694; x=1687383694; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version; bh=CwcBg8Tzo5Bo+Cv1R5yBeiH43ZnIWlaXkSj6pm+R6ys=; b=YJKSmalmG2/j/kwM3wfMV2uxg+FWKp4U4yO9G0zkPHbf81LXw+zjk0ex HsRsmR0gm89/ct2zvmqtkxz7qsZepU9dGFmbugwqV+nWaHzt9cdJZFwO2 SFyzYLi+XdQTWach9PK2+5Nv7xJKrSn23IfTRdAlOaSl9YwXQi9Dw2oI5 Y=; X-IronPort-AV: E=Sophos;i="5.92,210,1650931200"; d="scan'208";a="100442823" Received: from pdx4-co-svc-p1-lb2-vlan3.amazon.com (HELO email-inbound-relay-pdx-2c-b09ea7fa.us-west-2.amazon.com) ([10.25.36.214]) by smtp-border-fw-80006.pdx80.corp.amazon.com with ESMTP; 21 Jun 2022 21:41:19 +0000 Received: from EX13D49EUC003.ant.amazon.com (pdx1-ws-svc-p6-lb9-vlan3.pdx.amazon.com [10.236.137.198]) by email-inbound-relay-pdx-2c-b09ea7fa.us-west-2.amazon.com (Postfix) with ESMTPS id 5ACDA432F2; Tue, 21 Jun 2022 21:41:18 +0000 (UTC) Received: from ub4014a598e6c52.ant.amazon.com (10.43.161.183) by EX13D49EUC003.ant.amazon.com (10.43.164.91) with Microsoft SMTP Server (TLS) id 15.0.1497.36; Tue, 21 Jun 2022 21:41:14 +0000 From: "Ojeda Leon, Nicolas" To: CC: , , , Nicolas Ojeda Leon , Alexander Graf , Gerd Hoffmann Subject: [PATCH v6 5/5] OvmfPkg/PciHostBridgeUtilityLib: Initialize RootBridges apertures with spec Date: Tue, 21 Jun 2022 23:40:48 +0200 Message-ID: <32aacc5e6c8804b6d18da904b43dce27da484a38.1655841172.git.ncoleon@amazon.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: References: MIME-Version: 1.0 X-Originating-IP: [10.43.161.183] X-ClientProxiedBy: EX13D22UWC001.ant.amazon.com (10.43.162.192) To EX13D49EUC003.ant.amazon.com (10.43.164.91) Content-Type: text/plain Consume the host-provided specification of PCI host bridges if available. Using the DxeHardwareInfoLib, populate a list of hardware descriptors based on the content of the "hardware-info" fw-cfg file, if provided. In the affirmative case, use the resources and attributes specified by the hypervisor for each Host Bridge to create the RootBridge elements. In Ovmf platforms, the host can provide the specification of non-discoverable hardware resources like PCI host bridges. If the proper fw-cfg file is found, parse the contents provided by the host into a linked list by using the Hardware Info library. Then, using the list of PCI host bridges' descriptions, populate the PCI_ROOT_BRIDGES array with the resources and attributes specified by the host. If the file is not provided or no Host Bridge is found in it, fold back to the legacy method based on pre-defined apertures and rules. In some use cases, the host requires additional control over the hardware resources' configurations in the guest for performance and discoverability reasons. For instance, to disclose information about the PCI hierarchy to the guest so that this can profit from optimized accesses. In this case, the host can decide to describe multiple PCI Host Bridges and provide a specific set of resources (e.g. MMIO apertures) so that the guest uses the values provided. Using the provided values may entitle the guest to added performance, for example by using specific MMIO mappings that can enable peer-to-peer communication across the PCI hierarchy or by allocating memory closer to a device for faster DMA transactions. Cc: Alexander Graf Cc: Gerd Hoffmann Acked-by: Gerd Hoffmann Signed-off-by: Nicolas Ojeda Leon --- Notes: v6: Preparation for merge: - Verify the status output of HardwareInfoPciHostBridgeGet and abort the host-provided PCI root bridges parsing if an error happened. - Verify the validity of bus number range (start - end) extracted from host-provided info to avoid creating root bridges that have an invalid range. Abort parsing if error detected. - Cast bus number values down to UINT8 after having verified values are in range. v5: - Added "Acked-by" tag --- .../PciHostBridgeUtilityLib.c | 332 +++++++++++++++++- .../PciHostBridgeUtilityLib.inf | 1 + 2 files changed, 328 insertions(+), 5 deletions(-) diff --git a/OvmfPkg/Library/PciHostBridgeUtilityLib/PciHostBridgeUtilityLib.c b/OvmfPkg/Library/PciHostBridgeUtilityLib/PciHostBridgeUtilityLib.c index 92e1ea812f..cad2bd6c96 100644 --- a/OvmfPkg/Library/PciHostBridgeUtilityLib/PciHostBridgeUtilityLib.c +++ b/OvmfPkg/Library/PciHostBridgeUtilityLib/PciHostBridgeUtilityLib.c @@ -12,13 +12,16 @@ #include #include +#include #include #include #include +#include #include #include #include #include +#include #pragma pack(1) typedef struct { @@ -189,7 +192,9 @@ PciHostBridgeUtilityUninitRootBridge ( } /** - Utility function to return all the root bridge instances in an array. + Utility function to scan PCI root bridges and create instances for those + that are found not empty. Populate their resources from the default + provided parameters and return all the root bridge instances in an array. @param[out] Count The number of root bridge instances. @@ -217,9 +222,9 @@ PciHostBridgeUtilityUninitRootBridge ( @return All the root bridge instances in an array. **/ +STATIC PCI_ROOT_BRIDGE * -EFIAPI -PciHostBridgeUtilityGetRootBridges ( +PciHostBridgeUtilityGetRootBridgesBusScan ( OUT UINTN *Count, IN UINT64 Attributes, IN UINT64 AllocationAttributes, @@ -243,8 +248,6 @@ PciHostBridgeUtilityGetRootBridges ( UINTN LastRootBridgeNumber; UINTN RootBridgeNumber; - *Count = 0; - if ((BusMin > BusMax) || (BusMax > PCI_MAX_BUS)) { DEBUG (( DEBUG_ERROR, @@ -403,6 +406,325 @@ FreeBridges: return NULL; } +/** + Utility function to read root bridges information from host-provided fw-cfg + file and return them in an array. + + @param[out] Count The number of root bridge instances. + + @return All the root bridge instances in an array parsed from + host-provided fw-cfg file (hardware-info). +**/ +STATIC +PCI_ROOT_BRIDGE * +PciHostBridgeUtilityGetRootBridgesHostProvided ( + OUT UINTN *Count + ) +{ + EFI_STATUS Status; + FIRMWARE_CONFIG_ITEM FwCfgItem; + UINTN FwCfgSize; + PCI_ROOT_BRIDGE *Bridges; + UINTN Initialized; + UINTN LastRootBridgeNumber; + UINTN RootBridgeNumber; + UINTN PciHostBridgeCount; + UINT8 *HardwareInfoBlob; + LIST_ENTRY HwInfoList; + LIST_ENTRY *HwLink; + HARDWARE_INFO *HwInfo; + UINT64 Attributes; + UINT64 AllocationAttributes; + BOOLEAN DmaAbove4G; + BOOLEAN NoExtendedConfigSpace; + BOOLEAN CombineMemPMem; + PCI_ROOT_BRIDGE_APERTURE Io; + PCI_ROOT_BRIDGE_APERTURE Mem; + PCI_ROOT_BRIDGE_APERTURE MemAbove4G; + PCI_ROOT_BRIDGE_APERTURE PMem; + PCI_ROOT_BRIDGE_APERTURE PMemAbove4G; + + // + // Initialize the Hardware Info list head to start with an empty but valid + // list head. + // + InitializeListHead (&HwInfoList); + HardwareInfoBlob = NULL; + Initialized = 0; + Bridges = NULL; + PciHostBridgeCount = 0; + + // + // Hypervisor can provide the specifications (resources) for one or more + // PCI host bridges. Such information comes through fw-cfg as part of + // the hardware-info file. + // + Status = QemuFwCfgFindFile ("etc/hardware-info", &FwCfgItem, &FwCfgSize); + + if (EFI_ERROR (Status)) { + return NULL; + } + + HardwareInfoBlob = AllocatePool (FwCfgSize); + + if (HardwareInfoBlob == NULL) { + DEBUG (( + DEBUG_ERROR, + "%a: Failed to allocate memory for hardware resources info\n", + __FUNCTION__ + )); + return NULL; + } + + QemuFwCfgSelectItem (FwCfgItem); + QemuFwCfgReadBytes (FwCfgSize, HardwareInfoBlob); + + // + // Create the list of hardware info devices filtering for PCI host + // bridges + // + Status = CreateHardwareInfoList ( + HardwareInfoBlob, + FwCfgSize, + HardwareInfoTypeHostBridge, + &HwInfoList + ); + + if (EFI_ERROR (Status)) { + DEBUG (( + DEBUG_ERROR, + "%a: Failed to create hardware info list to retrieve host " + "bridges information from fw-cfg\n", + __FUNCTION__ + )); + + goto FreeBridges; + } + + PciHostBridgeCount = GetHardwareInfoCountByType ( + &HwInfoList, + HardwareInfoTypeHostBridge, + sizeof (HOST_BRIDGE_INFO) + ); + + if (PciHostBridgeCount == 0) { + goto FreeBridges; + } + + DEBUG (( + DEBUG_INFO, + "%a: Host provided description for %Lu root bridges\n", + __FUNCTION__, + PciHostBridgeCount + )); + + // + // Allocate the root bridges + // + Bridges = AllocatePool (((UINTN)PciHostBridgeCount) * sizeof *Bridges); + if (Bridges == NULL) { + DEBUG ((DEBUG_ERROR, "%a: %r\n", __FUNCTION__, EFI_OUT_OF_RESOURCES)); + goto FreeBridges; + } + + // + // If Host Bridges' specification was obtained from fw-cfg, the list + // contains information to populate all root bridges in the system + // including resources and attributes. + // + HwLink = GetFirstHardwareInfoByType ( + &HwInfoList, + HardwareInfoTypeHostBridge, + sizeof (HOST_BRIDGE_INFO) + ); + + while (!EndOfHardwareInfoList (&HwInfoList, HwLink)) { + HwInfo = HARDWARE_INFO_FROM_LINK (HwLink); + + Status = HardwareInfoPciHostBridgeGet ( + HwInfo->Data.PciHostBridge, + (UINTN)HwInfo->Header.Size, + &RootBridgeNumber, + &LastRootBridgeNumber, + &Attributes, + &DmaAbove4G, + &NoExtendedConfigSpace, + &CombineMemPMem, + &Io, + &Mem, + &MemAbove4G, + &PMem, + &PMemAbove4G, + NULL + ); + + if (EFI_ERROR (Status)) { + goto FreeBridges; + } + + if ((RootBridgeNumber > LastRootBridgeNumber) || (LastRootBridgeNumber > PCI_MAX_BUS)) { + DEBUG (( + DEBUG_ERROR, + "%a: invalid bus range with BusMin %Lu and BusMax " + "%Lu\n", + __FUNCTION__, + (UINT64)RootBridgeNumber, + (UINT64)LastRootBridgeNumber + )); + goto FreeBridges; + } + + AllocationAttributes = 0; + if (CombineMemPMem) { + AllocationAttributes |= EFI_PCI_HOST_BRIDGE_COMBINE_MEM_PMEM; + } + + if ((MemAbove4G.Limit > MemAbove4G.Base) || + (PMemAbove4G.Limit > PMemAbove4G.Base)) + { + AllocationAttributes |= EFI_PCI_HOST_BRIDGE_MEM64_DECODE; + } + + Status = PciHostBridgeUtilityInitRootBridge ( + Attributes, + Attributes, + AllocationAttributes, + DmaAbove4G, + NoExtendedConfigSpace, + (UINT8)RootBridgeNumber, + (UINT8)LastRootBridgeNumber, + &Io, + &Mem, + &MemAbove4G, + &PMem, + &PMemAbove4G, + &Bridges[Initialized] + ); + + if (EFI_ERROR (Status)) { + goto FreeBridges; + } + + ++Initialized; + + HwLink = GetNextHardwareInfoByType ( + &HwInfoList, + HwLink, + HardwareInfoTypeHostBridge, + sizeof (HOST_BRIDGE_INFO) + ); + } + + *Count = Initialized; + + // + // If resources were allocated for host bridges info, release them + // + if (HardwareInfoBlob) { + FreePool (HardwareInfoBlob); + } + + FreeHardwareInfoList (&HwInfoList); + return Bridges; + +FreeBridges: + while (Initialized > 0) { + --Initialized; + PciHostBridgeUtilityUninitRootBridge (&Bridges[Initialized]); + } + + if (Bridges) { + FreePool (Bridges); + } + + if (HardwareInfoBlob) { + FreePool (HardwareInfoBlob); + } + + FreeHardwareInfoList (&HwInfoList); + return NULL; +} + +/** + Utility function to return all the root bridge instances in an array. + + @param[out] Count The number of root bridge instances. + + @param[in] Attributes Initial attributes. + + @param[in] AllocAttributes Allocation attributes. + + @param[in] DmaAbove4G DMA above 4GB memory. + + @param[in] NoExtendedConfigSpace No Extended Config Space. + + @param[in] BusMin Minimum Bus number, inclusive. + + @param[in] BusMax Maximum Bus number, inclusive. + + @param[in] Io IO aperture. + + @param[in] Mem MMIO aperture. + + @param[in] MemAbove4G MMIO aperture above 4G. + + @param[in] PMem Prefetchable MMIO aperture. + + @param[in] PMemAbove4G Prefetchable MMIO aperture above 4G. + + @return All the root bridge instances in an array. +**/ +PCI_ROOT_BRIDGE * +EFIAPI +PciHostBridgeUtilityGetRootBridges ( + OUT UINTN *Count, + IN UINT64 Attributes, + IN UINT64 AllocationAttributes, + IN BOOLEAN DmaAbove4G, + IN BOOLEAN NoExtendedConfigSpace, + IN UINTN BusMin, + IN UINTN BusMax, + IN PCI_ROOT_BRIDGE_APERTURE *Io, + IN PCI_ROOT_BRIDGE_APERTURE *Mem, + IN PCI_ROOT_BRIDGE_APERTURE *MemAbove4G, + IN PCI_ROOT_BRIDGE_APERTURE *PMem, + IN PCI_ROOT_BRIDGE_APERTURE *PMemAbove4G + ) +{ + PCI_ROOT_BRIDGE *Bridges; + + *Count = 0; + + // + // First attempt to get the host provided descriptions of the Root Bridges + // if available. + // + Bridges = PciHostBridgeUtilityGetRootBridgesHostProvided (Count); + + // + // If host did not provide Root Bridge information, scan the buses and + // auto populate them with default resources. + // + if (Bridges == NULL) { + Bridges = PciHostBridgeUtilityGetRootBridgesBusScan ( + Count, + Attributes, + AllocationAttributes, + DmaAbove4G, + NoExtendedConfigSpace, + BusMin, + BusMax, + Io, + Mem, + MemAbove4G, + PMem, + PMemAbove4G + ); + } + + return Bridges; +} + /** Utility function to free root bridge instances array from PciHostBridgeUtilityGetRootBridges(). diff --git a/OvmfPkg/Library/PciHostBridgeUtilityLib/PciHostBridgeUtilityLib.inf b/OvmfPkg/Library/PciHostBridgeUtilityLib/PciHostBridgeUtilityLib.inf index 83a734c172..e4fc903121 100644 --- a/OvmfPkg/Library/PciHostBridgeUtilityLib/PciHostBridgeUtilityLib.inf +++ b/OvmfPkg/Library/PciHostBridgeUtilityLib/PciHostBridgeUtilityLib.inf @@ -38,6 +38,7 @@ BaseMemoryLib DebugLib DevicePathLib + DxeHardwareInfoLib MemoryAllocationLib PciLib QemuFwCfgLib -- 2.17.1 Amazon Development Center Germany GmbH Krausenstr. 38 10117 Berlin Geschaeftsfuehrung: Christian Schlaeger, Jonathan Weiss Eingetragen am Amtsgericht Charlottenburg unter HRB 149173 B Sitz: Berlin Ust-ID: DE 289 237 879