From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from smtp-fw-9103.amazon.com (smtp-fw-9103.amazon.com [207.171.188.200]) by mx.groups.io with SMTP id smtpd.web12.11904.1630766560946125987 for ; Sat, 04 Sep 2021 07:42:41 -0700 Authentication-Results: mx.groups.io; dkim=fail reason="signature has expired" header.i=@amazon.com header.s=amazon201209 header.b=kNh3lhbv; spf=pass (domain: amazon.de, ip: 207.171.188.200, mailfrom: prvs=874eef02e=ncoleon@amazon.de) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amazon.com; i=@amazon.com; q=dns/txt; s=amazon201209; t=1630766560; x=1662302560; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version; bh=vJkbG2qjEji5l8WDDQuvCM34FFkvN/ACrSHTsRBZ92c=; b=kNh3lhbv3B9pAwWR95+IyklohNgxbuKfCkwm1cmQVM4eEr03VvEkbmD+ aIndhUYO6V6D5OXi5t6LR/+xfCr+d/rONwdQTIKJpcckZ0CB6zyPSkzvb 9mGl8/QVftGUlnrs4FerZCxH8ypQE6n2iIn/hLUfyKyt3WmV3GHFkstit o=; X-IronPort-AV: E=Sophos;i="5.85,268,1624320000"; d="scan'208";a="955537417" Received: from pdx4-co-svc-p1-lb2-vlan2.amazon.com (HELO email-inbound-relay-1e-42f764a0.us-east-1.amazon.com) ([10.25.36.210]) by smtp-border-fw-9103.sea19.amazon.com with ESMTP; 04 Sep 2021 14:42:33 +0000 Received: from EX13D49EUC003.ant.amazon.com (iad12-ws-svc-p26-lb9-vlan3.iad.amazon.com [10.40.163.38]) by email-inbound-relay-1e-42f764a0.us-east-1.amazon.com (Postfix) with ESMTPS id A7D5BC019D for ; Sat, 4 Sep 2021 14:42:32 +0000 (UTC) Received: from ub4014a598e6c52.ant.amazon.com (10.43.162.52) by EX13D49EUC003.ant.amazon.com (10.43.164.91) with Microsoft SMTP Server (TLS) id 15.0.1497.23; Sat, 4 Sep 2021 14:42:29 +0000 From: Nicolas Ojeda Leon To: CC: Nicolas Ojeda Leon , Alexander Graf Subject: [PATCH 5/5] MdeModulePkg/PciBusDxe: Handling of pre-populated PCI BARs Date: Sat, 4 Sep 2021 16:37:56 +0200 Message-ID: X-Mailer: git-send-email 2.17.1 In-Reply-To: References: MIME-Version: 1.0 X-Originating-IP: [10.43.162.52] X-ClientProxiedBy: EX13D16UWC001.ant.amazon.com (10.43.162.117) To EX13D49EUC003.ant.amazon.com (10.43.164.91) Content-Type: text/plain Extend the PCI BAR placement logic in order to consider pre-populated resources first, if indicated by the Pcd token PcdPciPreservePopulatedMappings. The PCI_BAR type is augmented by one field for mapping the absolute address of prepopulated BARs into a root bridge relative offset. As part of the CreateResourceMap stage of the PCI resource allocation process, all the resources belonging to a root bridge are analyzed and if a BaseAddress (absolute address read from the PCI device during enumeration) is present, it is translated into a root-bridge relative offset and the space occupied by it is marked as used in the root bridge's aperture. This process is performed before the regular placement logic is executed. The remaining unassigned BARs are then placed according to default logic starting after the last pre-populated resource. The motivation behind respecting pre-populated BARs is that the hypervisor can decide on placement of some resources and the guest must use the same values to benefit from performance optimizations or specific requirements. Signed-off-by: Nicolas Ojeda Leon Cc: Alexander Graf --- MdeModulePkg/Bus/Pci/PciBusDxe/PciBus.h | 1 + .../Bus/Pci/PciBusDxe/PciResourceSupport.h | 20 ++ .../Bus/Pci/PciBusDxe/PciEnumeratorSupport.c | 1 + MdeModulePkg/Bus/Pci/PciBusDxe/PciLib.c | 11 + .../Bus/Pci/PciBusDxe/PciResourceSupport.c | 281 +++++++++++++++++- 5 files changed, 312 insertions(+), 2 deletions(-) diff --git a/MdeModulePkg/Bus/Pci/PciBusDxe/PciBus.h b/MdeModulePkg/Bus/Pci/PciBusDxe/PciBus.h index a619a68526..06e78ac40b 100644 --- a/MdeModulePkg/Bus/Pci/PciBusDxe/PciBus.h +++ b/MdeModulePkg/Bus/Pci/PciBusDxe/PciBus.h @@ -95,6 +95,7 @@ typedef enum { // struct _PCI_BAR { UINT64 BaseAddress; + UINT64 BaseAddressOffset; UINT64 Length; UINT64 Alignment; PCI_BAR_TYPE BarType; diff --git a/MdeModulePkg/Bus/Pci/PciBusDxe/PciResourceSupport.h b/MdeModulePkg/Bus/Pci/PciBusDxe/PciResourceSupport.h index 9f2de291ba..5f4886d6ba 100644 --- a/MdeModulePkg/Bus/Pci/PciBusDxe/PciResourceSupport.h +++ b/MdeModulePkg/Bus/Pci/PciBusDxe/PciResourceSupport.h @@ -111,6 +111,26 @@ CalculateApertureIo16 ( IN PCI_RESOURCE_NODE *Bridge ); +/** + Populate a root bridge's resource aperture according to + initial conditions ruled by individual pre-populated + resources + + @note Pre-populated PIO BARs are not supported. + + @param Bridge PCI resource node for given bridge device. + @param Base Resource aperture base address + +**/ +VOID +PopulateResourceAperture ( + IN UINT64 Mem32Base, + IN UINT64 PMem32Base, + IN UINT64 Mem64Base, + IN UINT64 PMem64Base, + IN OUT PCI_IO_DEVICE *Bridge + ); + /** This function is used to calculate the resource aperture for a given bridge device. diff --git a/MdeModulePkg/Bus/Pci/PciBusDxe/PciEnumeratorSupport.c b/MdeModulePkg/Bus/Pci/PciBusDxe/PciEnumeratorSupport.c index db1b35f8ef..c76b37ab05 100644 --- a/MdeModulePkg/Bus/Pci/PciBusDxe/PciEnumeratorSupport.c +++ b/MdeModulePkg/Bus/Pci/PciBusDxe/PciEnumeratorSupport.c @@ -1789,6 +1789,7 @@ PciParseBar ( PciIoDevice->PciBar[BarIndex].BarTypeFixed = FALSE; PciIoDevice->PciBar[BarIndex].Offset = (UINT8) Offset; + PciIoDevice->PciBar[BarIndex].BaseAddressOffset = 0; if ((Value & 0x01) != 0) { // // Device I/Os diff --git a/MdeModulePkg/Bus/Pci/PciBusDxe/PciLib.c b/MdeModulePkg/Bus/Pci/PciBusDxe/PciLib.c index 82147a4946..bf82d8fdaf 100644 --- a/MdeModulePkg/Bus/Pci/PciBusDxe/PciLib.c +++ b/MdeModulePkg/Bus/Pci/PciBusDxe/PciLib.c @@ -571,6 +571,17 @@ PciHostBridgeResourceAllocator ( &PMem64Base ); + // + // Handle pre-populated resources + // + PopulateResourceAperture ( + Mem32Base, + PMem32Base, + Mem64Base, + PMem64Base, + RootBridgeDev + ); + // // Get the max ROM size that the root bridge can process // Insert to resource map so that there will be dedicate MEM32 resource range for Option ROM. diff --git a/MdeModulePkg/Bus/Pci/PciBusDxe/PciResourceSupport.c b/MdeModulePkg/Bus/Pci/PciBusDxe/PciResourceSupport.c index 4969ee0f64..ad8db9a0ca 100644 --- a/MdeModulePkg/Bus/Pci/PciBusDxe/PciResourceSupport.c +++ b/MdeModulePkg/Bus/Pci/PciBusDxe/PciResourceSupport.c @@ -325,6 +325,270 @@ CalculateApertureIo16 ( Bridge->Length = MAX (Bridge->Length, PaddingAperture); } +/** + Analyze a single BAR and if it is pre-populated (its baseAddress is already + set), calculate the offset inside the root bridge. + + @param Bar PCI BAR to be placed if pre-populated + @param Base Base address of the root bridge + +**/ +STATIC +VOID +PrePopulateBAR ( + IN OUT PCI_BAR *Bar, + IN UINT64 Base + ) +{ + if (Base == gAllOne) { + return; + } + + // + // Pre-populated IO space BARs are not supported + // + if (Bar->BarType == PciBarTypeIo16 || Bar->BarType == PciBarTypeIo32) { + return ; + } + + if (Bar->BaseAddress < Base) { + return ; + } + + if (Bar->BaseAddress != 0) { + Bar->BaseAddressOffset = Bar->BaseAddress - Base; + } +} + + + +/** + Calculate and set the address offset inside the root bridge of device + BARs that are pre-populated. + + @param Dev PCI device whose BARs are to be analyzed + @param Base Base address of the root bridge under which the device + is located. + + */ +STATIC +VOID +PopulateDeviceBars ( + IN UINT64 Mem32Base, + IN UINT64 PMem32Base, + IN UINT64 Mem64Base, + IN UINT64 PMem64Base, + IN PCI_IO_DEVICE *Dev + ) +{ + UINTN Index; + + for (Index = 0; Index < PCI_MAX_BAR; Index++) + { + switch (Dev->PciBar[Index].BarType) { + case PciBarTypeMem32: + PrePopulateBAR(&Dev->PciBar[Index], Mem32Base); + break; + + case PciBarTypePMem32: + PrePopulateBAR(&Dev->PciBar[Index], PMem32Base); + break; + + case PciBarTypeMem64: + PrePopulateBAR(&Dev->PciBar[Index], Mem64Base); + break; + + case PciBarTypePMem64: + PrePopulateBAR(&Dev->PciBar[Index], PMem64Base); + break; + + default: + break; + } + } +} + +/** + Considering the Bridge attributes specifying the supported resources, + merge the base addresses to fake the non-suported types belonging to + the alternative type. + + @param Mem32Base 32-bit base address + @param PMem32Base 32-bit prefetchable base address + @param Mem64Base 64-bit base address + @param PMem64Base 64-bit prefetchable base address + @param Bridge PCI resource node for given bridge device. + + */ +STATIC +VOID +MergeBridgeResourceBases ( + IN OUT UINT64 *Mem32Base, + IN OUT UINT64 *PMem32Base, + IN OUT UINT64 *Mem64Base, + IN OUT UINT64 *PMem64Base, + IN PCI_IO_DEVICE *Bridge + ) +{ + // + // if root bridge supports combined Pmem Mem decoding + // merge these two type of resource + // + if (BridgeSupportResourceDecode (Bridge, EFI_BRIDGE_PMEM_MEM_COMBINE_SUPPORTED)) { + *PMem32Base = *Mem32Base; + *PMem64Base = *Mem64Base; + } + + // + // If bridge doesn't support Pmem32 + // degrade it to mem32 + // + if (!BridgeSupportResourceDecode (Bridge, EFI_BRIDGE_PMEM32_DECODE_SUPPORTED)) { + *PMem32Base = *Mem32Base; + } + + // + // If firmware is in 32-bit mode, + // then degrade PMEM64/MEM64 requests + // + if (sizeof (UINTN) <= 4) { + *Mem64Base = *Mem32Base ; + *PMem64Base = *PMem32Base; + } else { + // + // if the bridge does not support MEM64, degrade MEM64 to MEM32 + // + if (!BridgeSupportResourceDecode (Bridge, EFI_BRIDGE_MEM64_DECODE_SUPPORTED)) { + *Mem64Base = *Mem32Base; + } + + // + // if the bridge does not support PMEM64, degrade PMEM64 to PMEM32 + // + if (!BridgeSupportResourceDecode (Bridge, EFI_BRIDGE_PMEM64_DECODE_SUPPORTED)) { + *PMem64Base = *PMem32Base; + } + } +} + +/** + Populate a root bridge's resource aperture according to + initial conditions ruled by individual pre-populated + resources + + @param Bridge PCI resource node for given bridge device. + @param Base Resource aperture base address + +**/ +VOID +PopulateResourceAperture ( + IN UINT64 Mem32Base, + IN UINT64 PMem32Base, + IN UINT64 Mem64Base, + IN UINT64 PMem64Base, + IN OUT PCI_IO_DEVICE *Bridge + ) +{ + PCI_IO_DEVICE *Temp; + LIST_ENTRY *CurrentLink; + + // + // Verify if pre-populated BARs need to be respected, otherwise + // there is no need to pre-populate any resource + // + if (! PcdGetBool(PcdPciPreservePopulatedMappings)) { + return; + } + + CurrentLink = Bridge->ChildList.ForwardLink; + + // + // Merge base addresses of the different types depending on resource + // decoding supported by the bridge + // + MergeBridgeResourceBases( + &Mem32Base, + &PMem32Base, + &Mem64Base, + &PMem64Base, + Bridge + ); + + while (CurrentLink != NULL && CurrentLink != &Bridge->ChildList) { + + Temp = PCI_IO_DEVICE_FROM_LINK (CurrentLink); + + PopulateDeviceBars( + Mem32Base, + PMem32Base, + Mem64Base, + PMem64Base, + Temp + ); + + // + // Recursive call to analyze the hierarchical tree under a bridge + // + if (IS_PCI_BRIDGE (&Temp->Pci)) { + PopulateResourceAperture ( + Mem32Base, + PMem32Base, + Mem64Base, + PMem64Base, + Temp + ); + } + + CurrentLink = CurrentLink->ForwardLink; + } +} + +/** + Calculate the current used resource of the bridge provided that some + of the resources under it might have been pre-populated. + + @param Bridge PCI resource node for given bridge device. + +**/ +STATIC +UINT64 +GetBridgePopulatedAperture ( + IN PCI_RESOURCE_NODE *Bridge + ) +{ + UINT64 Aperture; + UINT64 EndAddress; + LIST_ENTRY *CurrentLink; + PCI_RESOURCE_NODE *Node; + PCI_BAR *Bar; + + Aperture = 0; + + // + // Analyze all resource nodes looking for the prepopulated with the + // highest address used. + // + for ( CurrentLink = GetFirstNode (&Bridge->ChildList) + ; !IsNull (&Bridge->ChildList, CurrentLink) + ; CurrentLink = GetNextNode (&Bridge->ChildList, CurrentLink) + ) { + Node = RESOURCE_NODE_FROM_LINK (CurrentLink); + + Bar = &Node->PciDev->PciBar[Node->Bar]; + + if (Bar->BaseAddress != 0) { + EndAddress = Bar->BaseAddressOffset + Bar->Length; + + if (EndAddress > Aperture) { + Aperture = EndAddress; + } + } + + } + + return Aperture; +} + /** This function is used to calculate the resource aperture for a given bridge device. @@ -338,6 +602,7 @@ CalculateResourceAperture ( ) { UINT64 Aperture[2]; + UINT64 InitialAperture; LIST_ENTRY *CurrentLink; PCI_RESOURCE_NODE *Node; @@ -351,8 +616,14 @@ CalculateResourceAperture ( return ; } - Aperture[PciResUsageTypical] = 0; - Aperture[PciResUsagePadding] = 0; + // + // Initialize apertures at bridge's current resource usage + // which might be occupied by pre-populated resources. + // + InitialAperture = GetBridgePopulatedAperture(Bridge); + Aperture[PciResUsageTypical] = InitialAperture; + Aperture[PciResUsagePadding] = InitialAperture; + // // Assume the bridge is aligned // @@ -362,6 +633,12 @@ CalculateResourceAperture ( ) { Node = RESOURCE_NODE_FROM_LINK (CurrentLink); + PCI_BAR *Bar = &Node->PciDev->PciBar[Node->Bar]; + if (PcdGetBool(PcdPciPreservePopulatedMappings) && Bar->BaseAddress) { + Node->Offset = Bar->BaseAddressOffset; + continue; + } + // // It's possible for a bridge to contain multiple padding resource // nodes due to DegradeResource(). -- 2.17.1 Amazon Development Center Germany GmbH Krausenstr. 38 10117 Berlin Geschaeftsfuehrung: Christian Schlaeger, Jonathan Weiss Eingetragen am Amtsgericht Charlottenburg unter HRB 149173 B Sitz: Berlin Ust-ID: DE 289 237 879