Re: [PATCH edk2-platforms 2/3] Silicon/NXP: Add support for reserving a chunk from RAM

["Leif Lindholm" <leif@nuviainc.com>]

On Wed, Jul 08, 2020 at 00:19:32 -0500, Pankaj Bansal wrote:
> From: Pankaj Bansal <pankaj.bansal@nxp.com>
> 
> Some NXP SOCs have some specialized IP blocks (like MC), which
> require DDR memory to operate. This DDR memory should not be managed
> by OS or UEFI.
> 
> Moreover to ensure that these IP blocks always get memory, and maximum
> contiguous RAM is available for UEFI and OS to use, add the support for
> reserving a chunk from RAM before reporting available RAM to UEFI.

I can't shake the feeling this code has the wrong level of
complexity. It's reserving *one* memory window, at a given alignment.

> Signed-off-by: Pankaj Bansal <pankaj.bansal@nxp.com>
> ---
>  Silicon/NXP/NxpQoriqLs.dec                                |  10 ++
>  Silicon/NXP/LX2160A/LX2160A.dsc.inc                       |   4 +
>  Silicon/NXP/Library/MemoryInitPeiLib/MemoryInitPeiLib.inf |   3 +
>  Silicon/NXP/Library/MemoryInitPeiLib/MemoryInitPeiLib.c   | 142 +++++++++++++++++++-
>  4 files changed, 157 insertions(+), 2 deletions(-)
> 
> diff --git a/Silicon/NXP/NxpQoriqLs.dec b/Silicon/NXP/NxpQoriqLs.dec
> index 188a9fe1f382..0e762066e547 100644
> --- a/Silicon/NXP/NxpQoriqLs.dec
> +++ b/Silicon/NXP/NxpQoriqLs.dec
> @@ -41,3 +41,13 @@ [PcdsDynamic.common]
>    gNxpQoriqLsTokenSpaceGuid.PcdPciCfgShiftEnable|FALSE|BOOLEAN|0x00000600
>    gNxpQoriqLsTokenSpaceGuid.PcdPciLsGen4Ctrl|FALSE|BOOLEAN|0x00000601
>    gNxpQoriqLsTokenSpaceGuid.PcdPciHideRootPort|FALSE|BOOLEAN|0x00000602
> +
> +  # Reserved RAM Base address alignment. This number ought to be Power of two
> +  # in case no alignment is needed, this number should be 1.

(As has been pointed out to me in the past, 1 is also a power of 2...)

> +  gNxpQoriqLsTokenSpaceGuid.PcdReservedMemAlignment|0x1|UINT64|0x00000603
> +  # Size of the RAM to be reserved. This RAM region is neither reported to UEFI
> +  # nor to OS

"Reported" is an inaccurate description of the mechanism involved.

> +  gNxpQoriqLsTokenSpaceGuid.PcdReservedMemSize|0x0|UINT64|0x00000604
> +  # Reserved RAM Base address which is calculated based on PcdReservedMemSize
> +  # and PcdReservedMemAlignment
> +  gNxpQoriqLsTokenSpaceGuid.PcdReservedMemBase|0x0|UINT64|0x00000605

Why  use a Pcd for something that is calculated at runtime and needs
to be used immediately? Moreover, I see this Pcd getting set, but I
then don't see it getting used ever?

> diff --git a/Silicon/NXP/LX2160A/LX2160A.dsc.inc b/Silicon/NXP/LX2160A/LX2160A.dsc.inc
> index 43e361464c8e..755ca169f213 100644
> --- a/Silicon/NXP/LX2160A/LX2160A.dsc.inc
> +++ b/Silicon/NXP/LX2160A/LX2160A.dsc.inc
> @@ -29,6 +29,10 @@ [PcdsDynamicDefault.common]
>    gArmTokenSpaceGuid.PcdGicRedistributorsBase|0x6200000
>    gArmTokenSpaceGuid.PcdGicInterruptInterfaceBase|0xC0C0000
>  
> +[PcdsDynamicHii]
> +  gNxpQoriqLsTokenSpaceGuid.PcdReservedMemAlignment|L"ReservedMemAlignment"|gEfiGlobalVariableGuid|0x0|0x20000000|NV,BS
> +  gNxpQoriqLsTokenSpaceGuid.PcdReservedMemSize|L"ReservedMemSize"|gEfiGlobalVariableGuid|0x0|0x20000000|NV,BS
> +

And here we get Hii involved, but I see no connection to a user
interface form.

>  [PcdsFixedAtBuild.common]
>    gArmTokenSpaceGuid.PcdGenericWatchdogControlBase|0x23A0000
>    gArmTokenSpaceGuid.PcdGenericWatchdogRefreshBase|0x2390000
> diff --git a/Silicon/NXP/Library/MemoryInitPeiLib/MemoryInitPeiLib.inf b/Silicon/NXP/Library/MemoryInitPeiLib/MemoryInitPeiLib.inf
> index a33f8cd3f743..ed23a86b43d9 100644
> --- a/Silicon/NXP/Library/MemoryInitPeiLib/MemoryInitPeiLib.inf
> +++ b/Silicon/NXP/Library/MemoryInitPeiLib/MemoryInitPeiLib.inf
> @@ -49,6 +49,9 @@ [FixedPcd]
>  [Pcd]
>    gArmTokenSpaceGuid.PcdSystemMemoryBase
>    gArmTokenSpaceGuid.PcdSystemMemorySize
> +  gNxpQoriqLsTokenSpaceGuid.PcdReservedMemAlignment
> +  gNxpQoriqLsTokenSpaceGuid.PcdReservedMemBase
> +  gNxpQoriqLsTokenSpaceGuid.PcdReservedMemSize
>  
>  [Depex]
>    TRUE
> diff --git a/Silicon/NXP/Library/MemoryInitPeiLib/MemoryInitPeiLib.c b/Silicon/NXP/Library/MemoryInitPeiLib/MemoryInitPeiLib.c
> index 11d1f1260b35..b416323a4ced 100644
> --- a/Silicon/NXP/Library/MemoryInitPeiLib/MemoryInitPeiLib.c
> +++ b/Silicon/NXP/Library/MemoryInitPeiLib/MemoryInitPeiLib.c
> @@ -118,6 +118,127 @@ GetDramRegionsInfo (
>    return EFI_BUFFER_TOO_SMALL;
>  }
>  
> +/**
> +  Calculate the base address of Reserved RAM.
> +  Reserved RAM is not reported to either UEFI or OS.

Here is the word "reported" again.
A region is included in the memory map or not.
If we are setting parts of the memory map aside, then we should do
that by marking them as EfiReservedMemoryType.

> +
> +  @param[in, out] DramRegions  Array of type DRAM_REGION_INFO. The size of this
> +                               array must be one more (+ 1) than the maximum
> +                               regions supported on platform. This is because,
> +                               if due to Reserved RAM alignment requirements a
> +                               hole is created in any DRAM region, then the RAM
> +                               after hole gets reported to UEFI and then
> +                               subsequently to OS. which is why, the last entry
> +                               of this array will not be parsed while
> +                               calculating Reserved RAM base address. Caller
> +                               must ensure that last entry of this array is zero
> +                               initialized.
> +  @param[in] NumRegions        Size of DramRegions array (including +1 for hole)
> +  @param[in] ReservedMemSize   Size of RAM to be reserved.
> +
> +  @return  if successful Address of the Reserved RAM region, 0 otherwise.
> +**/
> +STATIC
> +UINTN
> +CalculateReservedMemBase (
> +  IN DRAM_REGION_INFO *DramRegions,
> +  IN UINT32           NumRegions,
> +  IN UINTN            ReservedMemSize
> +)
> +{
> +  UINTN                 ReservedMemAlignment;
> +  EFI_PHYSICAL_ADDRESS  AlignmentMask;
> +  UINTN                 RegionBaseAddress;
> +  UINTN                 RegionSize;
> +  UINTN                 ReservedBaseAddress;
> +  INTN                  Index;
> +  INTN                  Index2;
> +
> +  ReservedMemAlignment = PcdGet64 (PcdReservedMemAlignment);
> +  //
> +  // Compute alignment bit mask
> +  //
> +  if (ReservedMemAlignment) {
> +    AlignmentMask = LShiftU64 (1, LowBitSet64(ReservedMemAlignment)) - 1;
> +  } else {
> +    AlignmentMask = 0;
> +  }
> +
> +  // The DRAM region info is sorted based on the RAM address is SOC memory map.
> +  // i.e. DramRegions[0] is at lower address, as compared to DramRegions[1].
> +  // The goal to start from last region is to find the topmost RAM region that
> +  // can contain Reserved RAM region i.e. PcdReservedMemSize.
> +  // Since this RAM is not reported to either UEFI or OS, This ensures that
> +  // maximum amount of lower RAM (32 bit addresses) are left
> +  // for OS to allocate to devices that can only work with 32bit physical
> +  // addresses. E.g. legacy devices that need to DMA to 32bit addresses.
> +  for (Index = NumRegions - 2; Index >=0; Index--) {
> +    RegionBaseAddress = DramRegions[Index].BaseAddress;
> +    RegionSize = DramRegions[Index].Size;
> +
> +    if (ReservedMemSize > RegionSize) {
> +      continue;
> +    }
> +
> +    ReservedBaseAddress = (RegionBaseAddress + RegionSize - ReservedMemSize);
> +    ReservedBaseAddress &= (~AlignmentMask);
> +    if (ReservedBaseAddress < RegionBaseAddress) {
> +      continue;
> +    }
> +
> +    // found the region from which reserved mem is to be carved out
> +    // Need to modify the region size and create/delete region if need be
> +
> +    RegionSize -= ReservedMemSize;
> +    if (RegionSize == 0) {
> +      // delete the region but maintain the sorted list of regions
> +      for (Index2 = Index; Index2 < NumRegions; Index2++) {
> +        CopyMem (
> +          &DramRegions[Index2],
> +          &DramRegions[Index2 + 1],
> +          sizeof (DRAM_REGION_INFO)
> +        );
> +      }
> +      break;
> +    }
> +
> +    if (ReservedBaseAddress - RegionBaseAddress) {
> +      DramRegions[Index].Size = ReservedBaseAddress - RegionBaseAddress;
> +      RegionSize -= DramRegions[Index].Size;
> +    } else {
> +      DramRegions[Index].BaseAddress = ReservedBaseAddress + ReservedMemSize;
> +      DramRegions[Index].Size = RegionSize;
> +      RegionSize = 0;
> +    }
> +
> +    if (RegionSize == 0) {
> +      break;
> +    }
> +
> +    // A hole has been created in DRAM regions due to Reserved RAM alignment
> +    // requirements. create a new DRAM region for DRAM memory after hole.
> +    // Maintain the sorted list of regions
> +    for (Index2 = NumRegions; Index2 > (Index + 1); Index2--) {
> +      CopyMem (
> +        &DramRegions[Index2],
> +        &DramRegions[Index2 - 1],
> +        sizeof (DRAM_REGION_INFO)
> +      );
> +    }
> +    DramRegions[Index2].BaseAddress = ReservedBaseAddress + ReservedMemSize;
> +    DramRegions[Index2].Size = RegionSize;
> +    RegionSize = 0;
> +
> +    break;
> +  }
> +
> +  if (Index == -1) {
> +    return 0;
> +  } else {
> +    return ReservedBaseAddress;
> +  }
> +}
> +
>  /**
>    Get the installed RAM information.
>    Initialize Memory HOBs (Resource Descriptor HOBs)
> @@ -135,7 +256,9 @@ MemoryInitPeiLibConstructor (
>    UINTN                         BaseAddress;
>    UINTN                         Size;
>    UINTN                         Top;
> -  DRAM_REGION_INFO              DramRegions[MAX_DRAM_REGIONS];
> +  // Extra region gets created if we want to reserve a memory region and that
> +  // creates a memory hole because of alignment requirements
> +  DRAM_REGION_INFO              DramRegions[MAX_DRAM_REGIONS + 1];

No. Just no.

>    EFI_RESOURCE_ATTRIBUTE_TYPE   ResourceAttributes;
>    UINTN                         FdBase;
>    UINTN                         FdTop;
> @@ -155,6 +278,21 @@ MemoryInitPeiLibConstructor (
>  
>    (VOID)GetDramRegionsInfo (DramRegions, ARRAY_SIZE (DramRegions));
>  
> +  // Get the reserved memory size from non volatile storage
> +  Size = PcdGet64 (PcdReservedMemSize);
> +  if (Size) {
> +    BaseAddress = CalculateReservedMemBase (
> +                    DramRegions,
> +                    ARRAY_SIZE (DramRegions),
> +                    Size
> +                  );
> +    if (BaseAddress) {
> +      DEBUG ((DEBUG_INFO, "ReservedMem: start 0x%lx, size 0x%lx\n",
> +        BaseAddress, Size));
> +      PcdSet64S (PcdReservedMemBase, BaseAddress);

PcdReservedMemBase set but never used.

And how do the other controllers find out about what region has ended
up being set aside for them? At what point during the boot process do
they need it?

/
    Leif

> +    }
> +  }
> +
>    FdBase = (UINTN)PcdGet64 (PcdFdBaseAddress);
>    FdTop = FdBase + (UINTN)PcdGet32 (PcdFdSize);
>  
> @@ -168,7 +306,7 @@ MemoryInitPeiLibConstructor (
>    // This ensures that maximum amount of lower RAM (32 bit addresses) are left
>    // for OS to allocate to devices that can only work with 32bit physical
>    // addresses. E.g. legacy devices that need to DMA to 32bit addresses.
> -  for (Index = MAX_DRAM_REGIONS - 1; Index >= 0; Index--) {
> +  for (Index = MAX_DRAM_REGIONS; Index >= 0; Index--) {
>      if (DramRegions[Index].Size == 0) {
>        continue;
>      }
> -- 
> 2.17.1
>