From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-wr1-f47.google.com (mail-wr1-f47.google.com [209.85.221.47]) by mx.groups.io with SMTP id smtpd.web08.11635.1635252550114598245 for ; Tue, 26 Oct 2021 05:49:10 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=@nuviainc-com.20210112.gappssmtp.com header.s=20210112 header.b=45xTyzrF; spf=pass (domain: nuviainc.com, ip: 209.85.221.47, mailfrom: leif@nuviainc.com) Received: by mail-wr1-f47.google.com with SMTP id i9so3195105wrc.3 for ; Tue, 26 Oct 2021 05:49:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nuviainc-com.20210112.gappssmtp.com; s=20210112; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to; bh=SkCh3TJiNqqy9w3/VuGGIxKWSoEJJ86ANWV9Dy4Ec5M=; b=45xTyzrFdGaNM9Di2SpNp++YH1fSXyGg1GiTV6axkLwiyNqiE49z6q+KD8z08nbeIO VzTBgCMiDO7h0dOBliAmWh+x4WzuQEkEMJhJ3Wu+iJ6hYR0XkDHiC0LpGT2ONlB85jBE 0yYjiA67peEYDvuK4rWqAbNmiDCdZeyhIx4ikeEaec9sUJy9T+/Hapia5q0IXho9pscP aFDICkef7MENIfRRioLGUsI1ppy8jwvB8lG1haVrcMq0iphUX4xGs4Fm93yjp95MWLeF nElrJaInwMtzhx1G4WtvbZhEO25w62tYLyOj/LAfulDBbPLOdecV/ZAhp0+jHMoMrFFj vjUQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to; bh=SkCh3TJiNqqy9w3/VuGGIxKWSoEJJ86ANWV9Dy4Ec5M=; b=VLOLY848Ptgi4/RMrNgL0rOv+bqZ7nln+UHYteR4xar6SpQyvvJOCZ28TlOXxB8X51 tQBGRNeDDMbnTmsM0SnbRCKRGPxjdnMHt9ZJ69BsUJ/OGpRLIcO9llhVU1ZWbRCUxS8F cikWwpQZ+JIHSF60J5iPS+OE4/Crg5NaRHneEGieFOHZHpCNv8IcOYPmpmFpsC+H0kwq UTD77cL6cdESCTD/ZrJywQBUHMPnH6SP6Y2XRnzAGcTxcAyfiD35s21vIgg3fUpmhfcx R7H+avFWK9ROt2RK1zgk7fT5XmBjg1HSgndUtFH20I6u1gPcKloT2DF3WWlzgwUOS//1 slsQ== X-Gm-Message-State: AOAM532Tdgd928Zw4YkZAV3tJ9WYKzV+2jfRWd2uWS4QY6bh2RcHbixw L7Ej+Nm2m0utsadEqcl6W2+Ptw== X-Google-Smtp-Source: ABdhPJyxTGQ9Ns+8QUbBv24sCDT7/iIJPSi23Igdit3knsfH6175P/1u4vx3ISvIvUMNiYlMQMbF4A== X-Received: by 2002:a05:6000:247:: with SMTP id m7mr24366879wrz.106.1635252548572; Tue, 26 Oct 2021 05:49:08 -0700 (PDT) Return-Path: Received: from leviathan (cpc92314-cmbg19-2-0-cust559.5-4.cable.virginm.net. [82.11.186.48]) by smtp.gmail.com with ESMTPSA id x6sm1221215wro.63.2021.10.26.05.49.07 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 26 Oct 2021 05:49:07 -0700 (PDT) Date: Tue, 26 Oct 2021 13:49:06 +0100 From: "Leif Lindholm" To: Nhi Pham Cc: devel@edk2.groups.io, patches@amperecomputing.com, vunguyen@os.amperecomputing.com, Thang Nguyen , Chuong Tran , Phong Vo , Michael D Kinney , Ard Biesheuvel , Nate DeSimone Subject: Re: [edk2-platforms][PATCH v4 14/31] AmpereAltraPkg: Add driver to initialize PCIe Root Complex Message-ID: <20211026124906.x4i7namhhvylwjf2@leviathan> References: <20211022061809.31087-1-nhi@os.amperecomputing.com> <20211022061809.31087-15-nhi@os.amperecomputing.com> MIME-Version: 1.0 In-Reply-To: <20211022061809.31087-15-nhi@os.amperecomputing.com> Content-Type: text/plain; charset=us-ascii Content-Disposition: inline On Fri, Oct 22, 2021 at 13:17:52 +0700, Nhi Pham wrote: > From: Vu Nguyen > > This driver takes responsibility for: > - Parsing platform settings to build the Root Complex info HOB > - Initializing each Root Complex > > Cc: Thang Nguyen > Cc: Chuong Tran > Cc: Phong Vo > Cc: Leif Lindholm > Cc: Michael D Kinney > Cc: Ard Biesheuvel > Cc: Nate DeSimone > > Signed-off-by: Nhi Pham Reviewed-by: Leif Lindholm > --- > Silicon/Ampere/AmpereAltraPkg/AmpereAltraPkg.dsc.inc | 1 + > Platform/Ampere/JadePkg/Jade.fdf | 1 + > Silicon/Ampere/AmpereAltraPkg/Drivers/PcieInitPei/PcieInitPei.inf | 40 ++ > Silicon/Ampere/AmpereAltraPkg/Drivers/PcieInitPei/RootComplexNVParam.h | 28 ++ > Silicon/Ampere/AmpereAltraPkg/Include/Platform/Ac01.h | 45 ++ > Silicon/Ampere/AmpereAltraPkg/Drivers/PcieInitPei/PcieInitPei.c | 156 ++++++ > Silicon/Ampere/AmpereAltraPkg/Drivers/PcieInitPei/RootComplexNVParam.c | 514 ++++++++++++++++++++ > 7 files changed, 785 insertions(+) > > diff --git a/Silicon/Ampere/AmpereAltraPkg/AmpereAltraPkg.dsc.inc b/Silicon/Ampere/AmpereAltraPkg/AmpereAltraPkg.dsc.inc > index a6f7d87769fe..edf3e4988625 100644 > --- a/Silicon/Ampere/AmpereAltraPkg/AmpereAltraPkg.dsc.inc > +++ b/Silicon/Ampere/AmpereAltraPkg/AmpereAltraPkg.dsc.inc > @@ -534,6 +534,7 @@ [Components.common] > Silicon/Ampere/AmpereAltraPkg/Drivers/ATFHobPei/ATFHobPeim.inf > ArmPlatformPkg/MemoryInitPei/MemoryInitPeim.inf > Silicon/Ampere/AmpereAltraPkg/Drivers/FlashPei/FlashPei.inf > + Silicon/Ampere/AmpereAltraPkg/Drivers/PcieInitPei/PcieInitPei.inf > ArmPkg/Drivers/CpuPei/CpuPei.inf > UefiCpuPkg/CpuIoPei/CpuIoPei.inf > MdeModulePkg/Universal/FaultTolerantWritePei/FaultTolerantWritePei.inf > diff --git a/Platform/Ampere/JadePkg/Jade.fdf b/Platform/Ampere/JadePkg/Jade.fdf > index 122a5b446302..b8293c096c4c 100644 > --- a/Platform/Ampere/JadePkg/Jade.fdf > +++ b/Platform/Ampere/JadePkg/Jade.fdf > @@ -162,6 +162,7 @@ [FV.FVMAIN_COMPACT] > INF MdeModulePkg/Universal/Variable/Pei/VariablePei.inf > INF MdeModulePkg/Universal/ReportStatusCodeRouter/Pei/ReportStatusCodeRouterPei.inf > INF MdeModulePkg/Universal/StatusCodeHandler/Pei/StatusCodeHandlerPei.inf > + INF Silicon/Ampere/AmpereAltraPkg/Drivers/PcieInitPei/PcieInitPei.inf > > # > # Print platform information before passing control into the Driver Execution Environment (DXE) phase > diff --git a/Silicon/Ampere/AmpereAltraPkg/Drivers/PcieInitPei/PcieInitPei.inf b/Silicon/Ampere/AmpereAltraPkg/Drivers/PcieInitPei/PcieInitPei.inf > new file mode 100644 > index 000000000000..17ac1672dac8 > --- /dev/null > +++ b/Silicon/Ampere/AmpereAltraPkg/Drivers/PcieInitPei/PcieInitPei.inf > @@ -0,0 +1,40 @@ > +## @file > +# > +# Copyright (c) 2020 - 2021, Ampere Computing LLC. All rights reserved.
> +# > +# SPDX-License-Identifier: BSD-2-Clause-Patent > +# > +## > + > +[Defines] > + INF_VERSION = 0x0001001B > + BASE_NAME = PcieInitPei > + FILE_GUID = CD3F92A7-9AE4-42F9-B2CC-B47A8615B85B > + MODULE_TYPE = PEIM > + VERSION_STRING = 1.0 > + ENTRY_POINT = PcieInitEntry > + > +[Sources] > + PcieInitPei.c > + RootComplexNVParam.c > + RootComplexNVParam.h > + > +[Packages] > + MdePkg/MdePkg.dec > + Silicon/Ampere/AmpereAltraPkg/AmpereAltraPkg.dec > + > +[LibraryClasses] > + Ac01PcieLib > + AmpereCpuLib > + BaseMemoryLib > + BoardPcieLib > + DebugLib > + HobLib > + PeimEntryPoint > + > +[Guids] > + gRootComplexInfoHobGuid > + gPlatformInfoHobGuid > + > +[Depex] > + TRUE > diff --git a/Silicon/Ampere/AmpereAltraPkg/Drivers/PcieInitPei/RootComplexNVParam.h b/Silicon/Ampere/AmpereAltraPkg/Drivers/PcieInitPei/RootComplexNVParam.h > new file mode 100644 > index 000000000000..008a8db69f2c > --- /dev/null > +++ b/Silicon/Ampere/AmpereAltraPkg/Drivers/PcieInitPei/RootComplexNVParam.h > @@ -0,0 +1,28 @@ > +/** @file > + > + Copyright (c) 2020 - 2021, Ampere Computing LLC. All rights reserved.
> + > + SPDX-License-Identifier: BSD-2-Clause-Patent > + > +**/ > + > +#ifndef GET_ROOT_COMPLEX_INFO_ > +#define GET_ROOT_COMPLEX_INFO_ > + > +#define BITS_PER_BYTE 8 > +#define BYTE_MASK 0xFF > +#define PCIE_ERRATA_SPEED1 0x0001 // Limited speed errata > + > +#ifndef BIT > +#define BIT(nr) (1 << (nr)) > +#endif > + > +#define PCIE_GET_MAX_WIDTH(Pcie, Max) \ > + !((Pcie).MaxWidth) ? (Max) : MIN ((Pcie).MaxWidth, (Max)) > + > +VOID > +ParseRootComplexNVParamData ( > + AC01_ROOT_COMPLEX *RootComplex > + ); > + > +#endif /* GET_ROOT_COMPLEX_INFO_ */ > diff --git a/Silicon/Ampere/AmpereAltraPkg/Include/Platform/Ac01.h b/Silicon/Ampere/AmpereAltraPkg/Include/Platform/Ac01.h > index 2310e4e1ce98..f3e6b4041737 100644 > --- a/Silicon/Ampere/AmpereAltraPkg/Include/Platform/Ac01.h > +++ b/Silicon/Ampere/AmpereAltraPkg/Include/Platform/Ac01.h > @@ -229,4 +229,49 @@ > // > #define PLATFORM_CPM_UID_BIT_OFFSET 8 > > +// > +// Max number for AC01 PCIE Root Complexes > +// > +#define AC01_PCIE_MAX_ROOT_COMPLEX 16 > + > +// > +// Max number for AC01 PCIE Root Complexes per socket > +// > +#define AC01_PCIE_MAX_RCS_PER_SOCKET 8 > + > +// > +// The base address of {TCU, CSR, MMCONFIG} Registers > +// > +#define AC01_PCIE_CSR_BASE_LIST 0x33FFE0000000, 0x37FFE0000000, 0x3BFFE0000000, 0x3FFFE0000000, 0x23FFE0000000, 0x27FFE0000000, 0x2BFFE0000000, 0x2FFFE0000000, 0x73FFE0000000, 0x77FFE0000000, 0x7BFFE0000000, 0x7FFFE0000000, 0x63FFE0000000, 0x67FFE0000000, 0x6BFFE0000000, 0x6FFFE0000000 > + > +// > +// The base address of MMIO Registers > +// > +#define AC01_PCIE_MMIO_BASE_LIST 0x300000000000, 0x340000000000, 0x380000000000, 0x3C0000000000, 0x200000000000, 0x240000000000, 0x280000000000, 0x2C0000000000, 0x700000000000, 0x740000000000, 0x780000000000, 0x7C0000000000, 0x600000000000, 0x640000000000, 0x680000000000, 0x6C0000000000 > + > +// > +// The size of MMIO space > +// > +#define AC01_PCIE_MMIO_SIZE_LIST 0x3FFE0000000, 0x3FFE0000000, 0x3FFE0000000, 0x3FFE0000000, 0x3FFE0000000, 0x3FFE0000000, 0x3FFE0000000, 0x3FFE0000000, 0x3FFE0000000, 0x3FFE0000000, 0x3FFE0000000, 0x3FFE0000000, 0x3FFE0000000, 0x3FFE0000000, 0x3FFE0000000, 0x3FFE0000000 > + > +// > +// The base address of MMIO32 Registers > +// > +#define AC01_PCIE_MMIO32_BASE_LIST 0x000020000000, 0x000028000000, 0x000030000000, 0x000038000000, 0x000004000000, 0x000008000000, 0x000010000000, 0x000018000000, 0x000060000000, 0x000068000000, 0x000070000000, 0x000078000000, 0x000040000000, 0x000048000000, 0x000050000000, 0x000058000000 > + > +// > +// The size of MMIO32 space > +// > +#define AC01_PCIE_MMIO32_SIZE_LIST 0x8000000, 0x8000000, 0x8000000, 0x8000000, 0x4000000, 0x8000000, 0x8000000, 0x8000000, 0x8000000, 0x8000000, 0x8000000, 0x8000000, 0x8000000, 0x8000000, 0x8000000, 0x8000000 > + > +// > +// The base address of MMIO32 Registers > +// > +#define AC01_PCIE_MMIO32_BASE_1P_LIST 0x000040000000, 0x000050000000, 0x000060000000, 0x000070000000, 0x000008000000, 0x000010000000, 0x000020000000, 0x000030000000, 0, 0, 0, 0, 0, 0, 0, 0 > + > +// > +// The size of MMIO32 1P space > +// > +#define AC01_PCIE_MMIO32_SIZE_1P_LIST 0x10000000, 0x10000000, 0x10000000, 0x10000000, 0x8000000, 0x10000000, 0x10000000, 0x10000000, 0, 0, 0, 0, 0, 0, 0, 0 > + > #endif /* PLATFORM_AC01_H_ */ > diff --git a/Silicon/Ampere/AmpereAltraPkg/Drivers/PcieInitPei/PcieInitPei.c b/Silicon/Ampere/AmpereAltraPkg/Drivers/PcieInitPei/PcieInitPei.c > new file mode 100644 > index 000000000000..76cbb76f38f0 > --- /dev/null > +++ b/Silicon/Ampere/AmpereAltraPkg/Drivers/PcieInitPei/PcieInitPei.c > @@ -0,0 +1,156 @@ > +/** @file > + > + Copyright (c) 2020 - 2021, Ampere Computing LLC. All rights reserved.
> + > + SPDX-License-Identifier: BSD-2-Clause-Patent > + > +**/ > + > +#include > + > +#include > +#include > +#include > +#include > +#include > +#include > +#include > +#include > + > +#include "RootComplexNVParam.h" > + > +#define TCU_OFFSET 0 > +#define SNPSRAM_OFFSET 0x9000 > +#define HB_CSR_OFFSET 0x01000000 > +#define PCIE0_CSR_OFFSET 0x01010000 > +#define SERDES_CSR_OFFSET 0x01200000 > +#define MMCONFIG_OFFSET 0x10000000 > + > +#define PCIE_CSR_SIZE 0x10000 > + > +STATIC AC01_ROOT_COMPLEX mRootComplexList[AC01_PCIE_MAX_ROOT_COMPLEX]; > +STATIC UINT64 mCsrBase[AC01_PCIE_MAX_ROOT_COMPLEX] = { AC01_PCIE_CSR_BASE_LIST }; > +STATIC UINT64 mMmio32Base[AC01_PCIE_MAX_ROOT_COMPLEX] = { AC01_PCIE_MMIO32_BASE_LIST }; > +STATIC UINT64 mMmio32Base1P[AC01_PCIE_MAX_ROOT_COMPLEX] = { AC01_PCIE_MMIO32_BASE_1P_LIST }; > +STATIC UINT64 mMmio32Size[AC01_PCIE_MAX_ROOT_COMPLEX] = { AC01_PCIE_MMIO32_SIZE_LIST }; > +STATIC UINT64 mMmio32Size1P[AC01_PCIE_MAX_ROOT_COMPLEX] = { AC01_PCIE_MMIO32_SIZE_1P_LIST }; > +STATIC UINT64 mMmioBase[AC01_PCIE_MAX_ROOT_COMPLEX] = { AC01_PCIE_MMIO_BASE_LIST }; > +STATIC UINT64 mMmioSize[AC01_PCIE_MAX_ROOT_COMPLEX] = { AC01_PCIE_MMIO_SIZE_LIST }; > + > +VOID > +BuildRootComplexData ( > + VOID > + ) > +{ > + AC01_ROOT_COMPLEX *RootComplex; > + UINT8 RCIndex; > + UINT8 PcieIndex; > + > + ZeroMem (&mRootComplexList, sizeof (AC01_ROOT_COMPLEX) * AC01_PCIE_MAX_ROOT_COMPLEX); > + > + // > + // Adjust Root Complex MMIO32 base address in 1P or 2P configuration > + // > + if (!IsSlaveSocketAvailable ()) { > + CopyMem ((VOID *)&mMmio32Base, (VOID *)&mMmio32Base1P, sizeof (mMmio32Base1P)); > + CopyMem ((VOID *)&mMmio32Size, (VOID *)&mMmio32Size1P, sizeof (mMmio32Size1P)); > + } > + > + for (RCIndex = 0; RCIndex < AC01_PCIE_MAX_ROOT_COMPLEX; RCIndex++) { > + RootComplex = &mRootComplexList[RCIndex]; > + RootComplex->Active = TRUE; > + RootComplex->DevMapLow = 0; > + RootComplex->DevMapHigh = 0; > + RootComplex->Socket = RCIndex / AC01_PCIE_MAX_RCS_PER_SOCKET; > + RootComplex->ID = RCIndex % AC01_PCIE_MAX_RCS_PER_SOCKET; > + RootComplex->CsrBase = mCsrBase[RCIndex]; > + RootComplex->TcuBase = RootComplex->CsrBase + TCU_OFFSET; > + RootComplex->HostBridgeBase = RootComplex->CsrBase + HB_CSR_OFFSET; > + RootComplex->SerdesBase = RootComplex->CsrBase + SERDES_CSR_OFFSET; > + RootComplex->MmcfgBase = RootComplex->CsrBase + MMCONFIG_OFFSET; > + RootComplex->MmioBase = mMmioBase[RCIndex]; > + RootComplex->MmioSize = mMmioSize[RCIndex]; > + RootComplex->Mmio32Base = mMmio32Base[RCIndex]; > + RootComplex->Mmio32Size = mMmio32Size[RCIndex]; > + RootComplex->Type = (RootComplex->ID < MaxRootComplexA) ? RootComplexTypeA : RootComplexTypeB; > + RootComplex->MaxPcieController = (RootComplex->Type == RootComplexTypeB) ? MaxPcieControllerB : MaxPcieControllerA; > + RootComplex->Logical = BoardPcieGetSegmentNumber (RootComplex); > + > + for (PcieIndex = 0; PcieIndex < RootComplex->MaxPcieController; PcieIndex++) { > + RootComplex->Pcie[PcieIndex].ID = PcieIndex; > + RootComplex->Pcie[PcieIndex].CsrBase = RootComplex->CsrBase + PCIE0_CSR_OFFSET + PcieIndex * PCIE_CSR_SIZE; > + RootComplex->Pcie[PcieIndex].SnpsRamBase = RootComplex->Pcie[PcieIndex].CsrBase + SNPSRAM_OFFSET; > + RootComplex->Pcie[PcieIndex].DevNum = PcieIndex + 1; > + } > + > + ParseRootComplexNVParamData (RootComplex); > + > + DEBUG (( > + DEBUG_INFO, > + " + S%d - RootComplex%a%d, MMCfgBase:0x%lx, MmioBase:0x%lx, Mmio32Base:0x%lx, Enabled:%a\n", > + RootComplex->Socket, > + (RootComplex->Type == RootComplexTypeA) ? "A" : "B", > + RootComplex->ID, > + RootComplex->MmcfgBase, > + RootComplex->MmioBase, > + RootComplex->Mmio32Base, > + (RootComplex->Active) ? "Y" : "N" > + )); > + > + DEBUG ((DEBUG_INFO, " + DevMapLo/Hi: 0x%x/0x%x\n", RootComplex->DevMapLow, RootComplex->DevMapHigh)); > + for (PcieIndex = 0; PcieIndex < RootComplex->MaxPcieController; PcieIndex++) { > + DEBUG (( > + DEBUG_INFO, > + " + PCIE%d:0x%lx - Enabled:%a - DevNum:0x%x\n", > + PcieIndex, > + RootComplex->Pcie[PcieIndex].CsrBase, > + (RootComplex->Pcie[PcieIndex].Active) ? "Y" : "N", > + RootComplex->Pcie[PcieIndex].DevNum > + )); > + } > + } > +} > + > +EFI_STATUS > +EFIAPI > +PcieInitEntry ( > + IN EFI_PEI_FILE_HANDLE FileHandle, > + IN CONST EFI_PEI_SERVICES **PeiServices > + ) > +{ > + AC01_ROOT_COMPLEX *RootComplex; > + EFI_STATUS Status; > + UINT8 Index; > + > + BuildRootComplexData (); > + > + // > + // Initialize Root Complex and underneath controllers > + // > + for (Index = 0; Index < AC01_PCIE_MAX_ROOT_COMPLEX; Index++) { > + RootComplex = &mRootComplexList[Index]; > + if (!RootComplex->Active) { > + continue; > + } > + > + Status = Ac01PcieCoreSetupRC (RootComplex, FALSE, 0); > + if (EFI_ERROR (Status)) { > + DEBUG ((DEBUG_ERROR, "RootComplex[%d]: Init Failed\n", Index)); > + RootComplex->Active = FALSE; > + continue; > + } > + } > + > + Ac01PcieCorePostSetupRC (mRootComplexList); > + > + // > + // Build Root Complex info Hob > + // > + BuildGuidDataHob ( > + &gRootComplexInfoHobGuid, > + (VOID *)&mRootComplexList, > + sizeof (mRootComplexList) > + ); > + > + return EFI_SUCCESS; > +} > diff --git a/Silicon/Ampere/AmpereAltraPkg/Drivers/PcieInitPei/RootComplexNVParam.c b/Silicon/Ampere/AmpereAltraPkg/Drivers/PcieInitPei/RootComplexNVParam.c > new file mode 100644 > index 000000000000..bffb480798fa > --- /dev/null > +++ b/Silicon/Ampere/AmpereAltraPkg/Drivers/PcieInitPei/RootComplexNVParam.c > @@ -0,0 +1,514 @@ > +/** @file > + Parsing NVPARAM board settings for bifurcation programming. > + > + NVPARAM board settings is spec-ed within Firmware Interface Requirement. > + Bifurcation devmap is programmed before at SCP following the rule > + > + Root Complex Type-A devmap settings (RP == Root Port) > + ----------------------------------------- > + | RP0 | RP1 | RP2 | RP3 | Devmap | > + | (x16) | (x4) | (x8) | (x4) | (output) | > + ------------------------------------------- > + | Y | N | N | N | 0 | > + | Y | N | Y | N | 1 | > + | Y | N | Y | Y | 2 | > + | Y | Y | Y | Y | 3 | > + ----------------------------------------- > + > + Root Complex Type-B LOW (aka RCBxA) devmap settings (RP == Root Port) > + ---------------------------------------- > + | RP0 | RP1 | RP2 | RP3 | Devmap | > + | (x8) | (x2) | (x4) | (x3) | (output) | > + ---------------------------------------- > + | Y | N | N | N | 0 | > + | Y | N | Y | N | 1 | > + | Y | N | Y | Y | 2 | > + | Y | Y | Y | Y | 3 | > + ---------------------------------------- > + > + Root Complex Type-B HIGH (aka RCBxB) devmap settings (RP == Root Port) > + ---------------------------------------- > + | RP4 | RP5 | RP6 | RP7 | Devmap | > + | (x8) | (x2) | (x4) | (x3) | (output) | > + ---------------------------------------- > + | Y | N | N | N | 0 | > + | Y | N | Y | N | 1 | > + | Y | N | Y | Y | 2 | > + | Y | Y | Y | Y | 3 | > + ---------------------------------------- > + > + Copyright (c) 2020 - 2021, Ampere Computing LLC. All rights reserved.
> + > + SPDX-License-Identifier: BSD-2-Clause-Patent > + > +**/ > + > +#include > + > +#include > +#include > +#include > +#include > +#include > +#include > +#include > + > +#include "RootComplexNVParam.h" > + > +STATIC > +BOOLEAN > +IsEmptyRC ( > + AC01_ROOT_COMPLEX *RootComplex > + ) > +{ > + UINT8 Idx; > + > + for (Idx = PcieController0; Idx < MaxPcieController; Idx++) { > + if (RootComplex->Pcie[Idx].Active) { > + return FALSE; > + } > + } > + > + return TRUE; > +} > + > +VOID > +SetRootComplexBifurcation ( > + IN AC01_ROOT_COMPLEX *RootComplex, > + IN UINT8 RPStart, > + IN DEV_MAP_MODE DevMap > + ) > +{ > + UINT8 MaxWidth; > + > + if (RPStart != PcieController0 && RPStart != PcieController4) { > + return; > + } > + > + if (RootComplex->Type != RootComplexTypeB && RPStart == PcieController4) { > + return; > + } > + > + if (RootComplex->Type == RootComplexTypeA && RootComplex->Pcie[RPStart].MaxWidth == LINK_WIDTH_X16) { > + RootComplex->Pcie[RPStart + 1].MaxWidth = LINK_WIDTH_X4; > + RootComplex->Pcie[RPStart + 2].MaxWidth = LINK_WIDTH_X8; > + RootComplex->Pcie[RPStart + 3].MaxWidth = LINK_WIDTH_X4; > + } > + > + if (RootComplex->Type == RootComplexTypeB && RootComplex->Pcie[RPStart].MaxWidth == LINK_WIDTH_X8) { > + RootComplex->Pcie[RPStart + 1].MaxWidth = LINK_WIDTH_X2; > + RootComplex->Pcie[RPStart + 2].MaxWidth = LINK_WIDTH_X4; > + RootComplex->Pcie[RPStart + 3].MaxWidth = LINK_WIDTH_X2; > + } > + > + switch (DevMap) { > + case DevMapMode2: > + MaxWidth = (RootComplex->Type == RootComplexTypeA) ? LINK_WIDTH_X8 : LINK_WIDTH_X4; > + RootComplex->Pcie[RPStart].MaxWidth = PCIE_GET_MAX_WIDTH (RootComplex->Pcie[RPStart], MaxWidth); > + RootComplex->Pcie[RPStart + 1].Active = FALSE; > + RootComplex->Pcie[RPStart + 2].MaxWidth = PCIE_GET_MAX_WIDTH (RootComplex->Pcie[RPStart + 2], MaxWidth); > + RootComplex->Pcie[RPStart + 2].Active = TRUE; > + RootComplex->Pcie[RPStart + 3].Active = FALSE; > + break; > + > + case DevMapMode3: > + MaxWidth = (RootComplex->Type == RootComplexTypeA) ? LINK_WIDTH_X8 : LINK_WIDTH_X4; > + RootComplex->Pcie[RPStart].MaxWidth = PCIE_GET_MAX_WIDTH (RootComplex->Pcie[RPStart], MaxWidth); > + RootComplex->Pcie[RPStart + 1].Active = FALSE; > + MaxWidth = (RootComplex->Type == RootComplexTypeA) ? LINK_WIDTH_X4 : LINK_WIDTH_X2; > + RootComplex->Pcie[RPStart + 2].MaxWidth = PCIE_GET_MAX_WIDTH (RootComplex->Pcie[RPStart + 2], MaxWidth); > + RootComplex->Pcie[RPStart + 2].Active = TRUE; > + RootComplex->Pcie[RPStart + 3].MaxWidth = PCIE_GET_MAX_WIDTH (RootComplex->Pcie[RPStart + 3], MaxWidth); > + RootComplex->Pcie[RPStart + 3].Active = TRUE; > + break; > + > + case DevMapMode4: > + MaxWidth = (RootComplex->Type == RootComplexTypeA) ? LINK_WIDTH_X4 : LINK_WIDTH_X2; > + RootComplex->Pcie[RPStart].MaxWidth = PCIE_GET_MAX_WIDTH (RootComplex->Pcie[RPStart], MaxWidth); > + RootComplex->Pcie[RPStart + 1].MaxWidth = PCIE_GET_MAX_WIDTH (RootComplex->Pcie[RPStart + 1], MaxWidth); > + RootComplex->Pcie[RPStart + 1].Active = TRUE; > + RootComplex->Pcie[RPStart + 2].MaxWidth = PCIE_GET_MAX_WIDTH (RootComplex->Pcie[RPStart + 2], MaxWidth); > + RootComplex->Pcie[RPStart + 2].Active = TRUE; > + RootComplex->Pcie[RPStart + 3].MaxWidth = PCIE_GET_MAX_WIDTH (RootComplex->Pcie[RPStart + 3], MaxWidth); > + RootComplex->Pcie[RPStart + 3].Active = TRUE; > + break; > + > + case DevMapMode1: > + default: > + MaxWidth = (RootComplex->Type == RootComplexTypeA) ? LINK_WIDTH_X16 : LINK_WIDTH_X8; > + RootComplex->Pcie[RPStart].MaxWidth = PCIE_GET_MAX_WIDTH (RootComplex->Pcie[RPStart], MaxWidth); > + RootComplex->Pcie[RPStart + 1].Active = FALSE; > + RootComplex->Pcie[RPStart + 2].Active = FALSE; > + RootComplex->Pcie[RPStart + 3].Active = FALSE; > + break; > + } > +} > + > +VOID > +GetDefaultDevMap ( > + AC01_ROOT_COMPLEX *RootComplex > + ) > +{ > + if (RootComplex->Pcie[PcieController0].Active > + && RootComplex->Pcie[PcieController1].Active > + && RootComplex->Pcie[PcieController2].Active > + && RootComplex->Pcie[PcieController3].Active) { > + RootComplex->DefaultDevMapLow = DevMapMode4; > + } else if (RootComplex->Pcie[PcieController0].Active > + && RootComplex->Pcie[PcieController2].Active > + && RootComplex->Pcie[PcieController3].Active) { > + RootComplex->DefaultDevMapLow = DevMapMode3; > + } else if (RootComplex->Pcie[PcieController0].Active > + && RootComplex->Pcie[PcieController2].Active) { > + RootComplex->DefaultDevMapLow = DevMapMode2; > + } else { > + RootComplex->DefaultDevMapLow = DevMapMode1; > + } > + > + if (RootComplex->Pcie[PcieController4].Active > + && RootComplex->Pcie[PcieController5].Active > + && RootComplex->Pcie[PcieController6].Active > + && RootComplex->Pcie[PcieController7].Active) { > + RootComplex->DefaultDevMapHigh = DevMapMode4; > + } else if (RootComplex->Pcie[PcieController4].Active > + && RootComplex->Pcie[PcieController6].Active > + && RootComplex->Pcie[PcieController7].Active) { > + RootComplex->DefaultDevMapHigh = DevMapMode3; > + } else if (RootComplex->Pcie[PcieController4].Active > + && RootComplex->Pcie[PcieController6].Active) { > + RootComplex->DefaultDevMapHigh = DevMapMode2; > + } else { > + RootComplex->DefaultDevMapHigh = DevMapMode1; > + } > + > + if (RootComplex->DevMapLow == 0) { > + RootComplex->DevMapLow = RootComplex->DefaultDevMapLow; > + } > + > + if (RootComplex->Type == RootComplexTypeB && RootComplex->DevMapHigh == 0) { > + RootComplex->DevMapHigh = RootComplex->DefaultDevMapHigh; > + } > + > + SetRootComplexBifurcation (RootComplex, PcieController0, RootComplex->DevMapLow); > + if (RootComplex->Type == RootComplexTypeB) { > + SetRootComplexBifurcation (RootComplex, PcieController4, RootComplex->DevMapHigh); > + } > +} > + > +VOID > +GetLaneAllocation ( > + AC01_ROOT_COMPLEX *RootComplex > + ) > +{ > + EFI_STATUS Status; > + INTN RPIndex; > + NVPARAM NvParamOffset; > + UINT32 Value, Width; > + > + // Retrieve lane allocation and capabilities for each controller > + if (RootComplex->Type == RootComplexTypeA) { > + NvParamOffset = (RootComplex->Socket == 0) ? NV_SI_RO_BOARD_S0_RCA0_CFG : NV_SI_RO_BOARD_S1_RCA0_CFG; > + NvParamOffset += RootComplex->ID * NV_PARAM_ENTRYSIZE; > + } else { > + // > + // There're two NVParam entries per RootComplexTypeB > + // > + NvParamOffset = (RootComplex->Socket == 0) ? NV_SI_RO_BOARD_S0_RCB0_LO_CFG : NV_SI_RO_BOARD_S1_RCB0_LO_CFG; > + NvParamOffset += (RootComplex->ID - MaxRootComplexA) * (NV_PARAM_ENTRYSIZE * 2); > + } > + > + Status = NVParamGet (NvParamOffset, NV_PERM_ALL, &Value); > + if (EFI_ERROR (Status)) { > + Value = 0; > + } > + > + for (RPIndex = 0; RPIndex < MaxPcieControllerA; RPIndex++) { > + Width = (Value >> (RPIndex * BITS_PER_BYTE)) & BYTE_MASK; > + switch (Width) { > + case 1: > + case 2: > + case 3: > + case 4: > + RootComplex->Pcie[RPIndex].MaxWidth = 1 << Width; > + RootComplex->Pcie[RPIndex].MaxGen = LINK_SPEED_GEN3; > + RootComplex->Pcie[RPIndex].Active = TRUE; > + break; > + > + case 0: > + default: > + RootComplex->Pcie[RPIndex].MaxWidth = LINK_WIDTH_NONE; > + RootComplex->Pcie[RPIndex].MaxGen = LINK_SPEED_NONE; > + RootComplex->Pcie[RPIndex].Active = FALSE; > + break; > + } > + } > + > + if (RootComplex->Type == RootComplexTypeB) { > + NvParamOffset += NV_PARAM_ENTRYSIZE; > + Status = NVParamGet (NvParamOffset, NV_PERM_ALL, &Value); > + if (EFI_ERROR (Status)) { > + Value = 0; > + } > + > + for (RPIndex = MaxPcieControllerA; RPIndex < MaxPcieController; RPIndex++) { > + Width = (Value >> ((RPIndex - MaxPcieControllerA) * BITS_PER_BYTE)) & BYTE_MASK; > + switch (Width) { > + case 1: > + case 2: > + case 3: > + case 4: > + RootComplex->Pcie[RPIndex].MaxWidth = 1 << Width; > + RootComplex->Pcie[RPIndex].MaxGen = LINK_SPEED_GEN3; > + RootComplex->Pcie[RPIndex].Active = TRUE; > + break; > + > + case 0: > + default: > + RootComplex->Pcie[RPIndex].MaxWidth = LINK_WIDTH_NONE; > + RootComplex->Pcie[RPIndex].MaxGen = LINK_SPEED_NONE; > + RootComplex->Pcie[RPIndex].Active = FALSE; > + break; > + } > + } > + } > + > + // Do not proceed if no Root Port enabled > + if (IsEmptyRC (RootComplex)) { > + RootComplex->Active = FALSE; > + } > +} > + > +NVPARAM > +GetGen3PresetNvParamOffset ( > + AC01_ROOT_COMPLEX *RootComplex > + ) > +{ > + NVPARAM NvParamOffset; > + > + if (RootComplex->Socket == 0) { > + if (RootComplex->Type == RootComplexTypeA) { > + if (RootComplex->ID < MaxRootComplexA) { > + NvParamOffset = NV_SI_RO_BOARD_S0_RCA0_TXRX_G3PRESET + RootComplex->ID * NV_PARAM_ENTRYSIZE; > + } else { > + NvParamOffset = NV_SI_RO_BOARD_S0_RCA4_TXRX_G3PRESET + (RootComplex->ID - MaxRootComplexA) * NV_PARAM_ENTRYSIZE; > + } > + } else { > + // > + // There're two NVParam entries per RootComplexTypeB > + // > + NvParamOffset = NV_SI_RO_BOARD_S0_RCB0A_TXRX_G3PRESET + (RootComplex->ID - MaxRootComplexA) * (NV_PARAM_ENTRYSIZE * 2); > + } > + } else if (RootComplex->Type == RootComplexTypeA) { > + if (RootComplex->ID < MaxRootComplexA) { > + NvParamOffset = NV_SI_RO_BOARD_S1_RCA2_TXRX_G3PRESET + (RootComplex->ID - 2) * NV_PARAM_ENTRYSIZE; > + } else { > + NvParamOffset = NV_SI_RO_BOARD_S1_RCA4_TXRX_G3PRESET + (RootComplex->ID - MaxRootComplexA) * NV_PARAM_ENTRYSIZE; > + } > + } else { > + // > + // There're two NVParam entries per RootComplexTypeB > + // > + NvParamOffset = NV_SI_RO_BOARD_S1_RCB0A_TXRX_G3PRESET + (RootComplex->ID - MaxRootComplexA) * (NV_PARAM_ENTRYSIZE * 2); > + } > + > + return NvParamOffset; > +} > + > +NVPARAM > +GetGen4PresetNvParamOffset ( > + AC01_ROOT_COMPLEX *RootComplex > + ) > +{ > + NVPARAM NvParamOffset; > + > + if (RootComplex->Socket == 0) { > + if (RootComplex->Type == RootComplexTypeA) { > + if (RootComplex->ID < MaxRootComplexA) { > + NvParamOffset = NV_SI_RO_BOARD_S0_RCA0_TXRX_G4PRESET + RootComplex->ID * NV_PARAM_ENTRYSIZE; > + } else { > + NvParamOffset = NV_SI_RO_BOARD_S0_RCA4_TXRX_G4PRESET + (RootComplex->ID - MaxRootComplexA) * NV_PARAM_ENTRYSIZE; > + } > + } else { > + // > + // There're two NVParam entries per RootComplexTypeB > + // > + NvParamOffset = NV_SI_RO_BOARD_S0_RCB0A_TXRX_G4PRESET + (RootComplex->ID - MaxRootComplexA) * (NV_PARAM_ENTRYSIZE * 2); > + } > + } else if (RootComplex->Type == RootComplexTypeA) { > + if (RootComplex->ID < MaxRootComplexA) { > + NvParamOffset = NV_SI_RO_BOARD_S1_RCA2_TXRX_G4PRESET + (RootComplex->ID - 2) * NV_PARAM_ENTRYSIZE; > + } else { > + NvParamOffset = NV_SI_RO_BOARD_S1_RCA4_TXRX_G4PRESET + (RootComplex->ID - MaxRootComplexA) * NV_PARAM_ENTRYSIZE; > + } > + } else { > + // > + // There're two NVParam entries per RootComplexTypeB > + // > + NvParamOffset = NV_SI_RO_BOARD_S1_RCB0A_TXRX_G4PRESET + (RootComplex->ID - MaxRootComplexA) * (NV_PARAM_ENTRYSIZE * 2); > + } > + > + return NvParamOffset; > +} > + > +VOID > +GetPresetSetting ( > + AC01_ROOT_COMPLEX *RootComplex > + ) > +{ > + EFI_STATUS Status; > + INTN Index; > + NVPARAM NvParamOffset; > + UINT32 Value; > + > + // Load default value > + for (Index = 0; Index < MaxPcieControllerB; Index++) { > + RootComplex->PresetGen3[Index] = PRESET_INVALID; > + RootComplex->PresetGen4[Index] = PRESET_INVALID; > + } > + > + NvParamOffset = GetGen3PresetNvParamOffset (RootComplex); > + > + Status = NVParamGet (NvParamOffset, NV_PERM_ALL, &Value); > + if (!EFI_ERROR (Status)) { > + for (Index = 0; Index < MaxPcieControllerA; Index++) { > + RootComplex->PresetGen3[Index] = (Value >> (Index * BITS_PER_BYTE)) & BYTE_MASK; > + } > + } > + > + if (RootComplex->Type == RootComplexTypeB) { > + NvParamOffset += NV_PARAM_ENTRYSIZE; > + Status = NVParamGet (NvParamOffset, NV_PERM_ALL, &Value); > + if (!EFI_ERROR (Status)) { > + for (Index = MaxPcieControllerA; Index < MaxPcieController; Index++) { > + RootComplex->PresetGen3[Index] = (Value >> ((Index - MaxPcieControllerA) * BITS_PER_BYTE)) & BYTE_MASK; > + } > + } > + } > + > + NvParamOffset = GetGen4PresetNvParamOffset (RootComplex); > + > + Status = NVParamGet (NvParamOffset, NV_PERM_ALL, &Value); > + if (!EFI_ERROR (Status)) { > + for (Index = 0; Index < MaxPcieControllerA; Index++) { > + RootComplex->PresetGen4[Index] = (Value >> (Index * BITS_PER_BYTE)) & BYTE_MASK; > + } > + } > + > + if (RootComplex->Type == RootComplexTypeB) { > + NvParamOffset += NV_PARAM_ENTRYSIZE; > + Status = NVParamGet (NvParamOffset, NV_PERM_ALL, &Value); > + if (!EFI_ERROR (Status)) { > + for (Index = MaxPcieControllerA; Index < MaxPcieController; Index++) { > + RootComplex->PresetGen4[Index] = (Value >> ((Index - MaxPcieControllerA) * BITS_PER_BYTE)) & BYTE_MASK; > + } > + } > + } > +} > + > +VOID > +GetMaxSpeedGen ( > + AC01_ROOT_COMPLEX *RootComplex > + ) > +{ > + UINT8 MaxSpeedGen[MaxPcieControllerA] = { LINK_SPEED_GEN4, LINK_SPEED_GEN4, LINK_SPEED_GEN4, LINK_SPEED_GEN4 }; // Bifurcation 0: RootComplexTypeA x16 / RootComplexTypeB x8 > + UINT8 ErrataSpeedDevMap3[MaxPcieControllerA] = { LINK_SPEED_GEN4, LINK_SPEED_GEN4, LINK_SPEED_GEN1, LINK_SPEED_GEN1 }; // Bifurcation 2: x8 x4 x4 (PCIE_ERRATA_SPEED1) > + UINT8 ErrataSpeedDevMap4[MaxPcieControllerA] = { LINK_SPEED_GEN1, LINK_SPEED_GEN1, LINK_SPEED_GEN1, LINK_SPEED_GEN1 }; // Bifurcation 3: x4 x4 x4 x4 (PCIE_ERRATA_SPEED1) > + UINT8 ErrataSpeedRcb[MaxPcieControllerA] = { LINK_SPEED_GEN1, LINK_SPEED_GEN1, LINK_SPEED_GEN1, LINK_SPEED_GEN1 }; // RootComplexTypeB PCIE_ERRATA_SPEED1 > + UINT8 Idx; > + UINT8 *MaxGen; > + > + ASSERT (MaxPcieControllerA == 4); > + ASSERT (MaxPcieController == 8); > + > + // > + // Due to hardware errata, for A0/A1* > + // RootComplexTypeB is limited to Gen1 speed. > + // RootComplexTypeA x16, x8 port supports up to Gen4, > + // RootComplexTypeA x4 port only supports Gen1. > + // > + MaxGen = MaxSpeedGen; > + if (RootComplex->Type == RootComplexTypeB) { > + if (RootComplex->Flags & PCIE_ERRATA_SPEED1) { > + MaxGen = ErrataSpeedRcb; > + } > + } else { > + switch (RootComplex->DevMapLow) { > + case DevMapMode3: /* x8 x4 x4 */ > + if (RootComplex->Flags & PCIE_ERRATA_SPEED1) { > + MaxGen = ErrataSpeedDevMap3; > + } > + break; > + > + case DevMapMode4: /* x4 x4 x4 x4 */ > + if (RootComplex->Flags & PCIE_ERRATA_SPEED1) { > + MaxGen = ErrataSpeedDevMap4; > + } > + break; > + > + case DevMapMode2: /* x8 x8 */ > + case DevMapMode1: /* x16 */ > + default: > + break; > + } > + } > + > + for (Idx = 0; Idx < MaxPcieControllerA; Idx++) { > + RootComplex->Pcie[Idx].MaxGen = RootComplex->Pcie[Idx].Active ? MaxGen[Idx] : LINK_SPEED_NONE; > + } > + > + if (RootComplex->Type == RootComplexTypeB) { > + for (Idx = MaxPcieControllerA; Idx < MaxPcieController; Idx++) { > + RootComplex->Pcie[Idx].MaxGen = RootComplex->Pcie[Idx].Active ? > + MaxGen[Idx - MaxPcieControllerA] : LINK_SPEED_NONE; > + } > + } > +} > + > +VOID > +ParseRootComplexNVParamData ( > + IN OUT AC01_ROOT_COMPLEX *RootComplex > + ) > +{ > + PLATFORM_INFO_HOB *PlatformHob; > + UINT32 EFuse; > + UINT8 RootComplexID; > + VOID *Hob; > + > + EFuse = 0; > + Hob = GetFirstGuidHob (&gPlatformInfoHobGuid); > + if (Hob != NULL) { > + PlatformHob = (PLATFORM_INFO_HOB *)GET_GUID_HOB_DATA (Hob); > + EFuse = PlatformHob->RcDisableMask[0] | (PlatformHob->RcDisableMask[1] << AC01_PCIE_MAX_RCS_PER_SOCKET); > + DEBUG (( > + DEBUG_INFO, > + "RcDisableMask[0]: 0x%x [1]: 0x%x\n", > + PlatformHob->RcDisableMask[0], > + PlatformHob->RcDisableMask[1] > + )); > + > + // Update errata flags for Ampere Altra > + if ((PlatformHob->ScuProductId[0] & 0xff) == 0x01) { > + if (PlatformHob->AHBCId[0] == 0x20100 > + || PlatformHob->AHBCId[0] == 0x21100 > + || (IsSlaveSocketActive () > + && (PlatformHob->AHBCId[1] == 0x20100 > + || PlatformHob->AHBCId[1] == 0x21100))) > + { > + RootComplex->Flags |= PCIE_ERRATA_SPEED1; > + DEBUG ((DEBUG_INFO, "RootComplex[%d]: Flags 0x%x\n", RootComplex->ID, RootComplex->Flags)); > + } > + } > + } > + > + RootComplexID = RootComplex->Socket * AC01_PCIE_MAX_RCS_PER_SOCKET + RootComplex->ID; > + RootComplex->DefaultActive = !(EFuse & BIT (RootComplexID)) ? TRUE : FALSE; > + if (!IsSlaveSocketActive () && RootComplex->Socket == 1) { > + RootComplex->DefaultActive = FALSE; > + } > + RootComplex->Active = RootComplex->Active && RootComplex->DefaultActive; > + > + GetPresetSetting (RootComplex); > + GetLaneAllocation (RootComplex); > + GetDefaultDevMap (RootComplex); > + GetMaxSpeedGen (RootComplex); > +} > -- > 2.17.1 >