* [edk2-devel] [edk2-platforms][PATCH v1 0/4] add support for enabling cxl-type3 device as memory expander on Arm/RD platform
@ 2023-10-10 11:38 Sayanta Pattanayak
2023-10-10 11:38 ` [edk2-devel] [edk2-platforms][PATCH v1 1/4] Platform/Sgi: add remote memory descriptor Sayanta Pattanayak
` (3 more replies)
0 siblings, 4 replies; 5+ messages in thread
From: Sayanta Pattanayak @ 2023-10-10 11:38 UTC (permalink / raw)
To: devel; +Cc: Ard Biesheuvel, Sami Mujawar
This patch series is about enabling CXL-Type3 capability on Arm reference
design platform.
* Designated 8GB address space is reserved and configured for using as extended
CXL memory.
* During PCIe enumeration process, a CXL device with DOE capability is looked
for. Once discovered, using DOE method CDAT information is fetched and
details like CXL device memory range, memory type collected.
* Thereafter using a registered notifier, SRAT and HMAT tables are updated with
remote CXL memory information.
These additions will enable CXL.Memory device to be used as memory expander on
Arm Reference Design platform.
Link to the branch with this patch series -
https://gitlab.arm.com/infra-solutions/reference-design/platsw/edk2-platforms/-/tree/topics/cxl_v2
Sayanta Pattanayak (4):
Platform/Sgi: add remote memory descriptor
Platform/Sgi: prepare SRAT, HMAT table
Platform/ARM: add CXL driver
Platform/Sgi: add CXL driver support
Platform/ARM/Drivers/CxlDxe/CxlDxe.dec | 21 +
Platform/ARM/SgiPkg/SgiPlatform.dec | 8 +
Platform/ARM/SgiPkg/SgiPlatform.dsc.inc | 12 +
Platform/ARM/SgiPkg/RdN2Cfg1/RdN2Cfg1.dsc | 7 +
Platform/ARM/SgiPkg/SgiPlatform.fdf | 14 +-
Platform/ARM/Drivers/CxlDxe/CxlDxe.inf | 50 ++
Platform/ARM/SgiPkg/Library/AcpiTableGeneratorLib/AcpiTableGenerator.inf | 68 +++
Platform/ARM/SgiPkg/Library/PlatformLib/PlatformLib.inf | 4 +
Platform/ARM/Drivers/CxlDxe/CxlDxe.h | 163 ++++++
Platform/ARM/Include/Protocol/Cxl.h | 62 +++
Platform/ARM/SgiPkg/Library/AcpiTableGeneratorLib/AcpiTableGenerator.h | 39 ++
Platform/ARM/Drivers/CxlDxe/CxlDxe.c | 575 ++++++++++++++++++++
Platform/ARM/SgiPkg/Library/AcpiTableGeneratorLib/AcpiTableGenerator.c | 106 ++++
Platform/ARM/SgiPkg/Library/AcpiTableGeneratorLib/HmatTableGenerator.c | 120 ++++
Platform/ARM/SgiPkg/Library/AcpiTableGeneratorLib/SratTableGenerator.c | 312 +++++++++++
Platform/ARM/SgiPkg/Library/PlatformLib/PlatformLibMem.c | 19 +-
16 files changed, 1577 insertions(+), 3 deletions(-)
create mode 100644 Platform/ARM/Drivers/CxlDxe/CxlDxe.dec
create mode 100644 Platform/ARM/Drivers/CxlDxe/CxlDxe.inf
create mode 100644 Platform/ARM/SgiPkg/Library/AcpiTableGeneratorLib/AcpiTableGenerator.inf
create mode 100644 Platform/ARM/Drivers/CxlDxe/CxlDxe.h
create mode 100644 Platform/ARM/Include/Protocol/Cxl.h
create mode 100644 Platform/ARM/SgiPkg/Library/AcpiTableGeneratorLib/AcpiTableGenerator.h
create mode 100644 Platform/ARM/Drivers/CxlDxe/CxlDxe.c
create mode 100644 Platform/ARM/SgiPkg/Library/AcpiTableGeneratorLib/AcpiTableGenerator.c
create mode 100644 Platform/ARM/SgiPkg/Library/AcpiTableGeneratorLib/HmatTableGenerator.c
create mode 100644 Platform/ARM/SgiPkg/Library/AcpiTableGeneratorLib/SratTableGenerator.c
--
2.25.1
-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.
View/Reply Online (#109488): https://edk2.groups.io/g/devel/message/109488
Mute This Topic: https://groups.io/mt/101872993/7686176
Group Owner: devel+owner@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub [rebecca@openfw.io]
-=-=-=-=-=-=-=-=-=-=-=-
^ permalink raw reply [flat|nested] 5+ messages in thread
* [edk2-devel] [edk2-platforms][PATCH v1 1/4] Platform/Sgi: add remote memory descriptor
2023-10-10 11:38 [edk2-devel] [edk2-platforms][PATCH v1 0/4] add support for enabling cxl-type3 device as memory expander on Arm/RD platform Sayanta Pattanayak
@ 2023-10-10 11:38 ` Sayanta Pattanayak
2023-10-10 11:38 ` [edk2-devel] [edk2-platforms][PATCH v1 2/4] Platform/Sgi: prepare SRAT, HMAT table Sayanta Pattanayak
` (2 subsequent siblings)
3 siblings, 0 replies; 5+ messages in thread
From: Sayanta Pattanayak @ 2023-10-10 11:38 UTC (permalink / raw)
To: devel; +Cc: Ard Biesheuvel, Sami Mujawar
8GB address region starting at 0x3FE00000000 is reserved for extended
Remote Memory use. One of the use case, for extended remote memory
region, is using it as CXL.Mem region.
Remote memory region, with Normal memory attributes, is included into
the Translation table entries based on the PcdRemoteCxlMemory flag
status.
Resource descriptor for Remote CXL device memory is not created in
PlatformLibMem, it will be added at later execution phase after the
discovery of CXL Mem device.
This patch adds a Pcd, PcdRemoteCxlMemory for identifying a platform
with CXL enabled configuration.
Also marked Local memory block count, for creating SRAT tables in
subsequent patch.
Signed-off-by: Sayanta Pattanayak <sayanta.pattanayak@arm.com>
---
Platform/ARM/SgiPkg/SgiPlatform.dec | 8 ++++++++
Platform/ARM/SgiPkg/SgiPlatform.dsc.inc | 4 ++++
Platform/ARM/SgiPkg/RdN2Cfg1/RdN2Cfg1.dsc | 7 +++++++
Platform/ARM/SgiPkg/Library/PlatformLib/PlatformLib.inf | 4 ++++
Platform/ARM/SgiPkg/Library/PlatformLib/PlatformLibMem.c | 19 +++++++++++++++++--
5 files changed, 40 insertions(+), 2 deletions(-)
diff --git a/Platform/ARM/SgiPkg/SgiPlatform.dec b/Platform/ARM/SgiPkg/SgiPlatform.dec
index 1613cc0198..a147e22f42 100644
--- a/Platform/ARM/SgiPkg/SgiPlatform.dec
+++ b/Platform/ARM/SgiPkg/SgiPlatform.dec
@@ -102,5 +102,13 @@
gArmSgiTokenSpaceGuid.PcdIoVirtSocExpBlk0Base|0|UINT64|0x0000002B
gArmSgiTokenSpaceGuid.PcdIoVirtSocExpBlkUartEnable|0|UINT32|0x0000002C
+ # Number of Local Memory blocks
+ gArmSgiTokenSpaceGuid.PcdNumLocalMemBlock|1|UINT32|0x00001005
+
+ # Expanded Remote Memory Base and Size
+ gArmSgiTokenSpaceGuid.PcdRemoteCxlMemory|FALSE|BOOLEAN|0x00001006
+ gArmSgiTokenSpaceGuid.PcdRemoteMemoryBase|0|UINT64|0x00001007
+ gArmSgiTokenSpaceGuid.PcdRemoteMemorySize|0|UINT64|0x00001008
+
[Ppis]
gNtFwConfigDtInfoPpiGuid = { 0x6f606eb3, 0x9123, 0x4e15, { 0xa8, 0x9b, 0x0f, 0xac, 0x66, 0xef, 0xd0, 0x17 } }
diff --git a/Platform/ARM/SgiPkg/SgiPlatform.dsc.inc b/Platform/ARM/SgiPkg/SgiPlatform.dsc.inc
index 26ecd9ed59..26b9a11a24 100644
--- a/Platform/ARM/SgiPkg/SgiPlatform.dsc.inc
+++ b/Platform/ARM/SgiPkg/SgiPlatform.dsc.inc
@@ -111,6 +111,10 @@
gArmSgiTokenSpaceGuid.PcdDramBlock2Base|0x8080000000
gArmSgiTokenSpaceGuid.PcdDramBlock2Size|0x180000000
+!if $(EDK2_ENABLE_REMOTE_CXL_MEM) == TRUE
+ gArmSgiTokenSpaceGuid.PcdRemoteCxlMemory|TRUE
+!endif
+
!if $(LPI_EN) == TRUE
# Allow use of LPI in the response to _OSC method call
gArmSgiTokenSpaceGuid.PcdOscLpiEnable|1
diff --git a/Platform/ARM/SgiPkg/RdN2Cfg1/RdN2Cfg1.dsc b/Platform/ARM/SgiPkg/RdN2Cfg1/RdN2Cfg1.dsc
index c26f652cb9..37bae5c392 100644
--- a/Platform/ARM/SgiPkg/RdN2Cfg1/RdN2Cfg1.dsc
+++ b/Platform/ARM/SgiPkg/RdN2Cfg1/RdN2Cfg1.dsc
@@ -47,6 +47,13 @@
gArmPlatformTokenSpaceGuid.PcdCoreCount|1
gArmPlatformTokenSpaceGuid.PcdClusterCount|8
+ # Number of local memory blocks
+ gArmSgiTokenSpaceGuid.PcdNumLocalMemBlock|3
+
+ # Expanded Remote Memory Region, used as CXL Memory
+ gArmSgiTokenSpaceGuid.PcdRemoteMemoryBase|0x3FE00000000
+ gArmSgiTokenSpaceGuid.PcdRemoteMemorySize|0x200000000
+
################################################################################
#
# Components Section - list of all EDK II Modules needed by this Platform
diff --git a/Platform/ARM/SgiPkg/Library/PlatformLib/PlatformLib.inf b/Platform/ARM/SgiPkg/Library/PlatformLib/PlatformLib.inf
index 020bde0d1f..2cdf9377e9 100644
--- a/Platform/ARM/SgiPkg/Library/PlatformLib/PlatformLib.inf
+++ b/Platform/ARM/SgiPkg/Library/PlatformLib/PlatformLib.inf
@@ -51,6 +51,10 @@
gArmTokenSpaceGuid.PcdSystemMemoryBase
gArmTokenSpaceGuid.PcdSystemMemorySize
+ gArmSgiTokenSpaceGuid.PcdRemoteCxlMemory
+ gArmSgiTokenSpaceGuid.PcdRemoteMemoryBase
+ gArmSgiTokenSpaceGuid.PcdRemoteMemorySize
+
gArmSgiTokenSpaceGuid.PcdChipCount
gArmTokenSpaceGuid.PcdGicDistributorBase
diff --git a/Platform/ARM/SgiPkg/Library/PlatformLib/PlatformLibMem.c b/Platform/ARM/SgiPkg/Library/PlatformLib/PlatformLibMem.c
index fa3cfbc730..13ba2a5960 100644
--- a/Platform/ARM/SgiPkg/Library/PlatformLib/PlatformLibMem.c
+++ b/Platform/ARM/SgiPkg/Library/PlatformLib/PlatformLibMem.c
@@ -50,6 +50,7 @@ ArmPlatformGetVirtualMemoryMap (
UINTN Index;
ARM_MEMORY_REGION_DESCRIPTOR *VirtualMemoryTable;
EFI_RESOURCE_ATTRIBUTE_TYPE ResourceAttributes;
+ UINTN TotalMemoryMapDescirptors;
ResourceAttributes =
EFI_RESOURCE_ATTRIBUTE_PRESENT |
@@ -111,9 +112,15 @@ ArmPlatformGetVirtualMemoryMap (
ASSERT (VirtualMemoryMap != NULL);
Index = 0;
+ TotalMemoryMapDescirptors = MAX_VIRTUAL_MEMORY_MAP_DESCRIPTORS;
+
+#if (FixedPcdGetBool (PcdRemoteCxlMemory))
+ TotalMemoryMapDescirptors += 1;
+#endif
+
VirtualMemoryTable = (ARM_MEMORY_REGION_DESCRIPTOR*)AllocatePages
(EFI_SIZE_TO_PAGES (sizeof (ARM_MEMORY_REGION_DESCRIPTOR) *
- MAX_VIRTUAL_MEMORY_MAP_DESCRIPTORS));
+ TotalMemoryMapDescirptors));
if (VirtualMemoryTable == NULL) {
return;
}
@@ -221,6 +228,14 @@ ArmPlatformGetVirtualMemoryMap (
VirtualMemoryTable[Index].Length = PcdGet64 (PcdDramBlock2Size);
VirtualMemoryTable[Index].Attributes = ARM_MEMORY_REGION_ATTRIBUTE_WRITE_BACK;
+#if (FixedPcdGetBool (PcdRemoteCxlMemory))
+ // Expanded Remote memory region
+ VirtualMemoryTable[++Index].PhysicalBase = FixedPcdGet64 (PcdRemoteMemoryBase);
+ VirtualMemoryTable[Index].VirtualBase = FixedPcdGet64 (PcdRemoteMemoryBase);
+ VirtualMemoryTable[Index].Length = FixedPcdGet64 (PcdRemoteMemorySize);
+ VirtualMemoryTable[Index].Attributes = ARM_MEMORY_REGION_ATTRIBUTE_WRITE_BACK;
+#endif
+
#if (FixedPcdGet32 (PcdChipCount) > 1)
// Chip 1 DDR Block 1 - (2GB)
VirtualMemoryTable[++Index].PhysicalBase = SYSTEM_MEMORY_BASE_REMOTE (1),
@@ -283,6 +298,6 @@ ArmPlatformGetVirtualMemoryMap (
VirtualMemoryTable[Index].Length = 0;
VirtualMemoryTable[Index].Attributes = (ARM_MEMORY_REGION_ATTRIBUTES)0;
- ASSERT ((Index + 1) <= MAX_VIRTUAL_MEMORY_MAP_DESCRIPTORS);
+ ASSERT ((Index + 1) <= TotalMemoryMapDescirptors);
*VirtualMemoryMap = VirtualMemoryTable;
}
--
2.25.1
-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.
View/Reply Online (#109489): https://edk2.groups.io/g/devel/message/109489
Mute This Topic: https://groups.io/mt/101872994/7686176
Group Owner: devel+owner@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub [rebecca@openfw.io]
-=-=-=-=-=-=-=-=-=-=-=-
^ permalink raw reply related [flat|nested] 5+ messages in thread
* [edk2-devel] [edk2-platforms][PATCH v1 2/4] Platform/Sgi: prepare SRAT, HMAT table
2023-10-10 11:38 [edk2-devel] [edk2-platforms][PATCH v1 0/4] add support for enabling cxl-type3 device as memory expander on Arm/RD platform Sayanta Pattanayak
2023-10-10 11:38 ` [edk2-devel] [edk2-platforms][PATCH v1 1/4] Platform/Sgi: add remote memory descriptor Sayanta Pattanayak
@ 2023-10-10 11:38 ` Sayanta Pattanayak
2023-10-10 11:38 ` [edk2-devel] [edk2-platforms][PATCH v1 3/4] Platform/ARM: add CXL driver Sayanta Pattanayak
2023-10-10 11:38 ` [edk2-devel] [edk2-platforms][PATCH v1 4/4] Platform/Sgi: add CXL driver support Sayanta Pattanayak
3 siblings, 0 replies; 5+ messages in thread
From: Sayanta Pattanayak @ 2023-10-10 11:38 UTC (permalink / raw)
To: devel; +Cc: Ard Biesheuvel, Sami Mujawar
The primary objective of this patch is to create SRAT, HMAT table at
runtime based on configuration data found by discovering remote CXL Mem
device. In SRAT table, the Localmemory, GICC structure information
remain Static, but only configuration data about Remote memory node is
updated at runtime. In HMAT the latency numbers are not tuned to actual
performance values in current solution. After consolidating all
structure data, SRAT and HMAT tables are installed.
This patch creates and registers notifier event in AcpiTableGenerator
DXE, and the event depends on CXL protocol interface, so that remote
memory node population in SRAT, HMAT tables can be performed after CXL
device discovery and necessary configuration.
While preparing SRAT table, CXL memory information (number of memory
nodes, remote memory size) is fetched from CXL Dxe using CXLprotocol
interfaces. If remote CXL memory is present then from the reserved Host
address space an area equal to remote CXL memory size will be added to
EFI System Memory space. The memory is EfiGcdMemoryTypeSystemMemory and
has Normal memory attributes.
In Single-Chip scenario, one of the primary use case, of having extended
remote memory area and SRAT,HMAT table, is to avail CXL.Memory device as
memory expander.
Signed-off-by: Sayanta Pattanayak <sayanta.pattanayak@arm.com>
---
Platform/ARM/SgiPkg/SgiPlatform.dsc.inc | 7 +
Platform/ARM/SgiPkg/SgiPlatform.fdf | 9 +-
Platform/ARM/SgiPkg/Library/AcpiTableGeneratorLib/AcpiTableGenerator.inf | 68 +++++
Platform/ARM/SgiPkg/Library/AcpiTableGeneratorLib/AcpiTableGenerator.h | 39 +++
Platform/ARM/SgiPkg/Library/AcpiTableGeneratorLib/AcpiTableGenerator.c | 106 +++++++
Platform/ARM/SgiPkg/Library/AcpiTableGeneratorLib/HmatTableGenerator.c | 120 ++++++++
Platform/ARM/SgiPkg/Library/AcpiTableGeneratorLib/SratTableGenerator.c | 312 ++++++++++++++++++++
7 files changed, 660 insertions(+), 1 deletion(-)
diff --git a/Platform/ARM/SgiPkg/SgiPlatform.dsc.inc b/Platform/ARM/SgiPkg/SgiPlatform.dsc.inc
index 26b9a11a24..fbe5b0b705 100644
--- a/Platform/ARM/SgiPkg/SgiPlatform.dsc.inc
+++ b/Platform/ARM/SgiPkg/SgiPlatform.dsc.inc
@@ -310,6 +310,13 @@
#
Platform/ARM/SgiPkg/Drivers/PlatformDxe/PlatformDxe.inf
+ #
+ # SRAT/HMAT Table generator
+ #
+!if $(EDK2_ENABLE_REMOTE_CXL_MEM) == TRUE
+ Platform/ARM/SgiPkg/Library/AcpiTableGeneratorLib/AcpiTableGenerator.inf
+!endif
+
#
# FAT filesystem + GPT/MBR partitioning
#
diff --git a/Platform/ARM/SgiPkg/SgiPlatform.fdf b/Platform/ARM/SgiPkg/SgiPlatform.fdf
index 7e55214c20..b1a227f22e 100644
--- a/Platform/ARM/SgiPkg/SgiPlatform.fdf
+++ b/Platform/ARM/SgiPkg/SgiPlatform.fdf
@@ -1,5 +1,5 @@
#
-# Copyright (c) 2018-2021, ARM Limited. All rights reserved.
+# Copyright (c) 2018 - 2022, Arm Limited. All rights reserved.<BR>
#
# SPDX-License-Identifier: BSD-2-Clause-Patent
#
@@ -194,6 +194,13 @@ READ_LOCK_STATUS = TRUE
INF MdeModulePkg/Universal/SetupBrowserDxe/SetupBrowserDxe.inf
INF MdeModulePkg/Universal/BdsDxe/BdsDxe.inf
+ #
+ # SRAT/HMAT Table generator
+ #
+!if $(EDK2_ENABLE_REMOTE_CXL_MEM) == TRUE
+ INF Platform/ARM/SgiPkg/Library/AcpiTableGeneratorLib/AcpiTableGenerator.inf
+!endif
+
#
# Networking stack
#
diff --git a/Platform/ARM/SgiPkg/Library/AcpiTableGeneratorLib/AcpiTableGenerator.inf b/Platform/ARM/SgiPkg/Library/AcpiTableGeneratorLib/AcpiTableGenerator.inf
new file mode 100644
index 0000000000..484e05f269
--- /dev/null
+++ b/Platform/ARM/SgiPkg/Library/AcpiTableGeneratorLib/AcpiTableGenerator.inf
@@ -0,0 +1,68 @@
+## @file
+# ACPI table generator sources.
+#
+# SRAT, HMAT table generator sources prepare respective ACPI tables, that
+# help OSPM to identify device domains, memory attributes etc.
+#
+# Copyright (c) 2023, Arm Limited. All rights reserved.<BR>
+#
+# SPDX-License-Identifier: BSD-2-Clause-Patent
+##
+
+[Defines]
+ INF_VERSION = 0x0001001B
+ BASE_NAME = AcpiTableGeneratorLib
+ FILE_GUID = fd0e015b-bbbf-474c-8b68-9ea1b555f913
+ MODULE_TYPE = DXE_DRIVER
+ VERSION_STRING = 1.0
+ ENTRY_POINT = AcpiTableGeneratorEntryPoint
+
+[Sources.common]
+ AcpiTableGenerator.c
+ HmatTableGenerator.c
+ SratTableGenerator.c
+
+[Packages]
+ ArmPkg/ArmPkg.dec
+ ArmPlatformPkg/ArmPlatformPkg.dec
+ DynamicTablesPkg/DynamicTablesPkg.dec
+ EmbeddedPkg/EmbeddedPkg.dec
+ MdeModulePkg/MdeModulePkg.dec
+ MdePkg/MdePkg.dec
+ Platform/ARM/Drivers/CxlDxe/CxlDxe.dec
+ Platform/ARM/SgiPkg/SgiPlatform.dec
+
+
+[LibraryClasses]
+ ArmPlatformLib
+ BaseLib
+ DebugLib
+ DxeServicesTableLib
+ HobLib
+ IoLib
+ MemoryAllocationLib
+ UefiDriverEntryPoint
+
+[Protocols]
+ gEfiAcpiTableProtocolGuid ## CONSUMES
+ gCxlPlatformProtocolGuid ## CONSUMES
+
+[Depex]
+ TRUE
+
+[FixedPcd]
+ gArmPlatformTokenSpaceGuid.PcdClusterCount
+ gArmPlatformTokenSpaceGuid.PcdCoreCount
+
+ gArmSgiTokenSpaceGuid.PcdChipCount
+ gArmSgiTokenSpaceGuid.PcdDramBlock2Base
+ gArmSgiTokenSpaceGuid.PcdDramBlock2Size
+ gArmSgiTokenSpaceGuid.PcdGicSize
+ gArmSgiTokenSpaceGuid.PcdNumLocalMemBlock
+ gArmSgiTokenSpaceGuid.PcdRemoteMemoryBase
+ gArmSgiTokenSpaceGuid.PcdRemoteMemorySize
+
+ gArmTokenSpaceGuid.PcdMmBufferBase
+ gArmTokenSpaceGuid.PcdMmBufferSize
+ gArmTokenSpaceGuid.PcdSystemMemoryBase
+ gArmTokenSpaceGuid.PcdSystemMemorySize
diff --git a/Platform/ARM/SgiPkg/Library/AcpiTableGeneratorLib/AcpiTableGenerator.h b/Platform/ARM/SgiPkg/Library/AcpiTableGeneratorLib/AcpiTableGenerator.h
new file mode 100644
index 0000000000..d9f5b396aa
--- /dev/null
+++ b/Platform/ARM/SgiPkg/Library/AcpiTableGeneratorLib/AcpiTableGenerator.h
@@ -0,0 +1,39 @@
+/** @file
+ Function declarations for AcpiTableGenerator.
+
+ Function declarations are needed for creating ACPI tables at runtime.
+ Macros for Remote memory nodes.
+
+ Copyright (c) 2023, Arm Limited. All rights reserved.<BR>
+ SPDX-License-Identifier: BSD-2-Clause-Patent
+**/
+
+#ifndef ACPI_TABLE_GENERATOR_H_
+#define ACPI_TABLE_GENERATOR_H_
+
+#include <Library/AcpiLib.h>
+#include <Library/AmlLib/AmlLib.h>
+#include <Library/BaseLib.h>
+#include <Library/BaseMemoryLib.h>
+#include <Library/DebugLib.h>
+#include <Library/IoLib.h>
+#include <Library/MemoryAllocationLib.h>
+#include <Library/PcdLib.h>
+#include <Library/UefiBootServicesTableLib.h>
+#include <Protocol/AcpiTable.h>
+#include <Protocol/Cxl.h>
+#include <SgiAcpiHeader.h>
+#include <SgiPlatform.h>
+
+#define LOWER_BYTES_MASK 0xFFFFF000
+#define LOWER_BYTES_SHIFT 32
+
+EFI_STATUS EFIAPI SratTableGenerator (
+ EFI_ACPI_TABLE_PROTOCOL *mAcpiTableProtocol
+ );
+
+EFI_STATUS EFIAPI HmatTableGenerator (
+ EFI_ACPI_TABLE_PROTOCOL *mAcpiTableProtocol
+ );
+
+#endif /* ACPI_TABLE_GENERATOR_H_ */
diff --git a/Platform/ARM/SgiPkg/Library/AcpiTableGeneratorLib/AcpiTableGenerator.c b/Platform/ARM/SgiPkg/Library/AcpiTableGeneratorLib/AcpiTableGenerator.c
new file mode 100644
index 0000000000..3faacfde00
--- /dev/null
+++ b/Platform/ARM/SgiPkg/Library/AcpiTableGeneratorLib/AcpiTableGenerator.c
@@ -0,0 +1,106 @@
+/** @file
+ ACPI Table Generator Entrypoint. It invokes functions to
+ generate SRAT, HMAT tables.
+
+ Copyright (c) 2023, Arm Limited. All rights reserved.<BR>
+ SPDX-License-Identifier: BSD-2-Clause-Patent
+**/
+
+#include "AcpiTableGenerator.h"
+
+VOID *mCxlProtocolRegistration;
+
+STATIC EFI_ACPI_TABLE_PROTOCOL *mAcpiTableProtocol = NULL;
+
+// Protocol event handler for creating SRAT, HMAT tables
+VOID
+EFIAPI
+AcpiTableGenerator (
+ IN EFI_EVENT Event,
+ IN VOID* Context
+ )
+{
+ EFI_STATUS Status;
+
+ Status = gBS->LocateProtocol (
+ &gEfiAcpiTableProtocolGuid,
+ NULL,
+ (VOID **)&mAcpiTableProtocol
+ );
+ if (EFI_ERROR (Status)) {
+ DEBUG ((
+ DEBUG_ERROR,
+ "%a: Failed to locate ACPI table protocol, status: %r\n",
+ __func__,
+ Status
+ ));
+ return;
+ }
+
+ Status = SratTableGenerator (mAcpiTableProtocol);
+ if (EFI_ERROR (Status)) {
+ DEBUG ((
+ DEBUG_ERROR,
+ "%a: Failed to create SRAT table: %r\n",
+ __func__,
+ Status
+ ));
+ return;
+ }
+
+ Status = HmatTableGenerator (mAcpiTableProtocol);
+ if (EFI_ERROR (Status)) {
+ DEBUG ((
+ DEBUG_ERROR,
+ "%a: Failed to create HMAT table: %r\n",
+ __func__,
+ Status
+ ));
+ return;
+ }
+
+ gBS->CloseEvent (Event);
+ return;
+}
+
+/**
+ Initialize function for the driver.
+
+ Locate ACPI Table protocol and installs SRAT, HMAT tables.
+
+ @param[in] ImageHandle Handle to image.
+ @param[in] SystemTable Pointer to System table.
+
+ @retval EFI_SUCCESS On successful installation of SRAT, HMAT ACPI tables.
+ @retval Other Failure in installing ACPI tables.
+
+**/
+EFI_STATUS
+EFIAPI
+AcpiTableGeneratorEntryPoint (
+ IN EFI_HANDLE ImageHandle,
+ IN EFI_SYSTEM_TABLE *SystemTable
+ )
+{
+ EFI_STATUS Status;
+ EFI_EVENT CxlProtocolEvent;
+
+ Status = gBS->CreateEvent (
+ EVT_NOTIFY_SIGNAL,
+ TPL_CALLBACK,
+ AcpiTableGenerator,
+ NULL,
+ &CxlProtocolEvent
+ );
+
+ //
+ // Register for protocol notifications on this event
+ //
+ Status = gBS->RegisterProtocolNotify (
+ &gCxlPlatformProtocolGuid,
+ CxlProtocolEvent,
+ &mCxlProtocolRegistration
+ );
+
+ return Status;
+}
diff --git a/Platform/ARM/SgiPkg/Library/AcpiTableGeneratorLib/HmatTableGenerator.c b/Platform/ARM/SgiPkg/Library/AcpiTableGeneratorLib/HmatTableGenerator.c
new file mode 100644
index 0000000000..b21ac86e25
--- /dev/null
+++ b/Platform/ARM/SgiPkg/Library/AcpiTableGeneratorLib/HmatTableGenerator.c
@@ -0,0 +1,120 @@
+/** @file
+ Heterogeneous Memory Attribute Table (HMAT) Table Generator.
+
+ The (HMAT) describes the memory attributes, such as bandwidth and latency
+ details, related to Memory Proximity Domains. The software is expected
+ to use this information as a hint for optimization, or when the system has
+ heterogeneous memory.
+
+ Copyright (c) 2023, Arm Limited. All rights reserved.<BR>
+ SPDX-License-Identifier: BSD-2-Clause-Patent
+
+ @par Specification Reference:
+ - ACPI 6.4, Chapter 5.2.27 Heterogeneous Memory Attribute Table (HMAT)
+**/
+
+#include "AcpiTableGenerator.h"
+
+#define CHIP_CNT 2
+#define INITATOR_PROXIMITY_DOMAIN_CNT 2
+#define TARGET_PROXIMITY_DOMAIN_CNT 2
+
+/* HMAT Table */
+typedef struct InitiatorTargetProximityMatrix {
+ UINT32 InitatorProximityDomain[INITATOR_PROXIMITY_DOMAIN_CNT];
+ UINT32 TargetProximityDomain[TARGET_PROXIMITY_DOMAIN_CNT];
+ UINT16 MatrixEntry[INITATOR_PROXIMITY_DOMAIN_CNT *
+ TARGET_PROXIMITY_DOMAIN_CNT];
+} INITIATOR_TARGET_PROXIMITY_MATRIX;
+
+typedef struct {
+ EFI_ACPI_6_4_HETEROGENEOUS_MEMORY_ATTRIBUTE_TABLE_HEADER Header;
+ EFI_ACPI_6_4_HMAT_STRUCTURE_MEMORY_PROXIMITY_DOMAIN_ATTRIBUTES
+ Proximity[CHIP_CNT];
+ EFI_ACPI_6_4_HMAT_STRUCTURE_SYSTEM_LOCALITY_LATENCY_AND_BANDWIDTH_INFO
+ LatencyInfo;
+ INITIATOR_TARGET_PROXIMITY_MATRIX Matrix;
+} EFI_ACPI_HETEROGENEOUS_MEMORY_ATTRIBUTE_TABLE;
+
+EFI_ACPI_HETEROGENEOUS_MEMORY_ATTRIBUTE_TABLE Hmat = {
+ // Header
+ {
+ ARM_ACPI_HEADER (
+ EFI_ACPI_6_4_HETEROGENEOUS_MEMORY_ATTRIBUTE_TABLE_SIGNATURE,
+ EFI_ACPI_HETEROGENEOUS_MEMORY_ATTRIBUTE_TABLE,
+ EFI_ACPI_6_4_HETEROGENEOUS_MEMORY_ATTRIBUTE_TABLE_REVISION
+ ),
+ {
+ EFI_ACPI_RESERVED_BYTE,
+ EFI_ACPI_RESERVED_BYTE,
+ EFI_ACPI_RESERVED_BYTE,
+ EFI_ACPI_RESERVED_BYTE
+ },
+ },
+
+ // Memory Proximity Domain
+ {
+ EFI_ACPI_6_4_HMAT_STRUCTURE_MEMORY_PROXIMITY_DOMAIN_ATTRIBUTES_INIT (
+ 1, 0x0, 0x0),
+ EFI_ACPI_6_4_HMAT_STRUCTURE_MEMORY_PROXIMITY_DOMAIN_ATTRIBUTES_INIT (
+ 1, 0x0, 0x1),
+ },
+
+ // Latency Info
+ EFI_ACPI_6_4_HMAT_STRUCTURE_SYSTEM_LOCALITY_LATENCY_AND_BANDWIDTH_INFO_INIT (
+ 0, 0, 0, INITATOR_PROXIMITY_DOMAIN_CNT, TARGET_PROXIMITY_DOMAIN_CNT, 100),
+ {
+ {0, 1},
+ {0, 1},
+ {
+ //
+ // The latencies mentioned in this table are hypothetical values and
+ // represents typical latency between two chips. These values are
+ // applicable only for RD-N1-Edge dual-chip fixed virtual platform and
+ // should not be reused for other platforms.
+ //
+ 10, 20,
+ 20, 10,
+ }
+ }
+};
+
+/**
+ Installs the HMAT table.
+
+ @param[in] mAcpiTableProtocol Handle to AcpiTableProtocol.
+
+ @retval EFI_SUCCESS On successful installation of HMAT table.
+ @retval Other Failure in installing HMAT table.
+**/
+EFI_STATUS
+EFIAPI
+HmatTableGenerator (
+ IN EFI_ACPI_TABLE_PROTOCOL *mAcpiTableProtocol
+ )
+{
+ EFI_STATUS Status;
+ UINTN AcpiTableHandle;
+
+ Status = mAcpiTableProtocol->InstallAcpiTable (
+ mAcpiTableProtocol,
+ &Hmat,
+ sizeof (Hmat),
+ &AcpiTableHandle
+ );
+ if (EFI_ERROR (Status)) {
+ DEBUG ((
+ DEBUG_ERROR,
+ "%a: HMAT table installation failed, status: %r\n",
+ __func__,
+ Status
+ ));
+ } else {
+ DEBUG ((
+ DEBUG_INFO,
+ "Installed HMAT table \n"
+ ));
+ }
+
+ return Status;
+}
diff --git a/Platform/ARM/SgiPkg/Library/AcpiTableGeneratorLib/SratTableGenerator.c b/Platform/ARM/SgiPkg/Library/AcpiTableGeneratorLib/SratTableGenerator.c
new file mode 100644
index 0000000000..50a26b8fca
--- /dev/null
+++ b/Platform/ARM/SgiPkg/Library/AcpiTableGeneratorLib/SratTableGenerator.c
@@ -0,0 +1,312 @@
+/** @file
+ SRAT Table Generator.
+
+ SRAT table provides information that allows OSPM to associate devices such as
+ processors with system locality / proximity domains and clock domains.
+
+ Copyright (c) 2023, Arm Limited. All rights reserved.<BR>
+ SPDX-License-Identifier: BSD-2-Clause-Patent
+
+ @par Specification Reference:
+ - ACPI 6.4, Chapter 5.2.16 System Resource Affinity Table (SRAT)
+**/
+
+#include <Library/DxeServicesTableLib.h>
+#include <Library/HobLib.h>
+#include "AcpiTableGenerator.h"
+
+STATIC EFI_ACPI_6_4_MEMORY_AFFINITY_STRUCTURE *RemoteMemory;
+
+EFI_ACPI_6_4_SYSTEM_RESOURCE_AFFINITY_TABLE_HEADER SratHeader = {
+ ARM_ACPI_HEADER (
+ EFI_ACPI_6_4_SYSTEM_RESOURCE_AFFINITY_TABLE_SIGNATURE,
+ EFI_ACPI_6_4_SYSTEM_RESOURCE_AFFINITY_TABLE_HEADER,
+ EFI_ACPI_6_4_SYSTEM_RESOURCE_AFFINITY_TABLE_REVISION
+ ),
+ 0x00000001,
+ EFI_ACPI_RESERVED_QWORD
+};
+
+EFI_ACPI_6_4_GICC_AFFINITY_STRUCTURE Gicc[8] = {
+ EFI_ACPI_6_4_GICC_AFFINITY_STRUCTURE_INIT (
+ 0x0, 0x00000000, 0x00000001, 0x00000000),
+ EFI_ACPI_6_4_GICC_AFFINITY_STRUCTURE_INIT (
+ 0x0, 0x00000001, 0x00000001, 0x00000000),
+ EFI_ACPI_6_4_GICC_AFFINITY_STRUCTURE_INIT (
+ 0x0, 0x00000002, 0x00000001, 0x00000000),
+ EFI_ACPI_6_4_GICC_AFFINITY_STRUCTURE_INIT (
+ 0x0, 0x00000003, 0x00000001, 0x00000000),
+ EFI_ACPI_6_4_GICC_AFFINITY_STRUCTURE_INIT (
+ 0x0, 0x00000004, 0x00000001, 0x00000000),
+ EFI_ACPI_6_4_GICC_AFFINITY_STRUCTURE_INIT (
+ 0x0, 0x00000005, 0x00000001, 0x00000000),
+ EFI_ACPI_6_4_GICC_AFFINITY_STRUCTURE_INIT (
+ 0x0, 0x00000006, 0x00000001, 0x00000000),
+ EFI_ACPI_6_4_GICC_AFFINITY_STRUCTURE_INIT (
+ 0x0, 0x00000007, 0x00000001, 0x00000000),
+#if ((CORE_COUNT * CLUSTER_COUNT) > 8)
+ EFI_ACPI_6_4_GICC_AFFINITY_STRUCTURE_INIT (
+ 0x0, 0x00000008, 0x00000001, 0x00000000),
+ EFI_ACPI_6_4_GICC_AFFINITY_STRUCTURE_INIT (
+ 0x0, 0x00000009, 0x00000001, 0x00000000),
+ EFI_ACPI_6_4_GICC_AFFINITY_STRUCTURE_INIT (
+ 0x0, 0x0000000A, 0x00000001, 0x00000000),
+ EFI_ACPI_6_4_GICC_AFFINITY_STRUCTURE_INIT (
+ 0x0, 0x0000000B, 0x00000001, 0x00000000),
+ EFI_ACPI_6_4_GICC_AFFINITY_STRUCTURE_INIT (
+ 0x0, 0x0000000C, 0x00000001, 0x00000000),
+ EFI_ACPI_6_4_GICC_AFFINITY_STRUCTURE_INIT (
+ 0x0, 0x0000000D, 0x00000001, 0x00000000),
+ EFI_ACPI_6_4_GICC_AFFINITY_STRUCTURE_INIT (
+ 0x0, 0x0000000E, 0x00000001, 0x00000000),
+ EFI_ACPI_6_4_GICC_AFFINITY_STRUCTURE_INIT (
+ 0x0, 0x0000000F, 0x00000001, 0x00000000),
+#endif
+};
+
+EFI_ACPI_6_4_MEMORY_AFFINITY_STRUCTURE LocalMemory[3] = {
+ // Memory at 32-bit address space
+ EFI_ACPI_6_4_MEMORY_AFFINITY_STRUCTURE_INIT (
+ 0x0, FixedPcdGet64 (PcdSystemMemoryBase),
+ FixedPcdGet64 (PcdSystemMemorySize), 0x00000001),
+ // Memory at 64-bit address space
+ EFI_ACPI_6_4_MEMORY_AFFINITY_STRUCTURE_INIT (
+ 0x0, FixedPcdGet64 (PcdDramBlock2Base),
+ FixedPcdGet64 (PcdDramBlock2Size), 0x00000001),
+ // MmBuffer region
+ EFI_ACPI_6_4_MEMORY_AFFINITY_STRUCTURE_INIT (
+ 0x0, FixedPcdGet64 (PcdMmBufferBase),
+ FixedPcdGet64 (PcdMmBufferSize), 0x00000001),
+};
+
+/**
+ Fetch the details of Remote Memory Node, using CXL protocol interfaces.
+
+ By using CXL platform protocol interfaces, fetch number CXL remote memory
+ nodes and their corresponding configurations(Base address, length).
+
+ @param[out] RemoteMemCount Number of Remote CXL memory nodes.
+
+ @retval RemoteMem Remote memory configuraiton on successful fetching
+ of remote memory configuration.
+ @retval Zero Returns Zero on failure.
+**/
+STATIC
+REMOTE_MEMORY_CONFIG *
+FetchRemoteCxlMem (OUT UINT32 *RemoteMemCount)
+{
+ EFI_STATUS Status;
+ CXL_PLATFORM_PROTOCOL *CxlProtocol;
+ REMOTE_MEMORY_CONFIG *RemoteMem;
+ UINT32 RemoteMemNumber;
+
+ Status = gBS->LocateProtocol (
+ &gCxlPlatformProtocolGuid,
+ NULL,
+ (VOID **)&CxlProtocol
+ );
+ if (EFI_ERROR (Status)) {
+ DEBUG ((
+ DEBUG_ERROR,
+ "%a: Failed to locate CXL Protocol, status: %r\n",
+ __func__,
+ Status
+ ));
+ return 0;
+ }
+
+ RemoteMemNumber = CxlProtocol->CxlGetRemoteMemCount ();
+ if (RemoteMemNumber) {
+ RemoteMem = (REMOTE_MEMORY_CONFIG *) AllocateZeroPool (
+ sizeof (REMOTE_MEMORY_CONFIG) *
+ RemoteMemNumber
+ );
+ if (RemoteMem == NULL) {
+ DEBUG ((DEBUG_WARN, "No memory for Remote Memory affinity structure:\n"));
+ return 0;
+ }
+ } else {
+ DEBUG ((DEBUG_INFO, "No Remote Memory node exists:\n"));
+ return 0;
+ }
+
+ Status = CxlProtocol->CxlGetRemoteMem (RemoteMem, &RemoteMemNumber);
+ if (EFI_ERROR (Status)) {
+ DEBUG ((
+ DEBUG_ERROR,
+ "%a: Failed to get CXL Remote memory details: %r\n",
+ __func__,
+ Status
+ ));
+ FreePool(RemoteMem);
+ return 0;
+ }
+
+ *RemoteMemCount = RemoteMemNumber;
+
+ return RemoteMem;
+}
+
+/**
+ Fetch the details of all Remote Memory Node, using CXL protocol interfaces.
+ Prepare SRAT table structure by combining LocalMemoryNode and
+ RemoteMemoryNode information. Thereafter installs the SRAT table.
+
+ @param[in] mAcpiTableProtocol Handle to AcpiTableProtocol.
+
+ @retval EFI_SUCCESS On successful installation of SRAT table.
+ @retval Other Failure in installing SRAT table.
+**/
+EFI_STATUS
+EFIAPI
+SratTableGenerator (
+ IN EFI_ACPI_TABLE_PROTOCOL *mAcpiTableProtocol
+ )
+{
+ EFI_STATUS Status;
+ UINTN AcpiTableHandle;
+ UINTN MemoryNodeCount;
+ UINTN TableSize;
+ UINT8 Idx;
+ VOID *Srat, *SratDataNext;
+ REMOTE_MEMORY_CONFIG *RemoteMem;
+ UINT32 RemoteMemCount;
+ UINT64 HostPhysicalBase;
+ UINT64 MemDevicePhysicalBase;
+
+ RemoteMem = FetchRemoteCxlMem (&RemoteMemCount);
+
+ if (RemoteMemCount) {
+ RemoteMemory = AllocateZeroPool (
+ sizeof (EFI_ACPI_6_4_MEMORY_AFFINITY_STRUCTURE) *
+ RemoteMemCount
+ );
+
+ if (RemoteMemory == NULL) {
+ DEBUG ((DEBUG_WARN, "No memory for Remote Memory affinity structure:\n"));
+ RemoteMemCount = 0;
+ } else {
+ HostPhysicalBase = FixedPcdGet64 (PcdRemoteMemoryBase);
+
+ for (Idx = 0; Idx < RemoteMemCount; Idx++) {
+ RemoteMemory[Idx].Type = 1;
+ RemoteMemory[Idx].Length =
+ sizeof (EFI_ACPI_6_4_MEMORY_AFFINITY_STRUCTURE);
+ RemoteMemory[Idx].ProximityDomain = 1;
+ RemoteMemory[Idx].Reserved1 = EFI_ACPI_RESERVED_WORD;
+
+ MemDevicePhysicalBase = HostPhysicalBase + RemoteMem[Idx].DpaAddress;
+ RemoteMemory[Idx].AddressBaseLow =
+ MemDevicePhysicalBase & LOWER_BYTES_MASK;
+ RemoteMemory[Idx].AddressBaseHigh =
+ MemDevicePhysicalBase >> LOWER_BYTES_SHIFT;
+
+ RemoteMemory[Idx].LengthLow =
+ RemoteMem[Idx].DpaLength & LOWER_BYTES_MASK;
+ RemoteMemory[Idx].LengthHigh =
+ RemoteMem[Idx].DpaLength >> LOWER_BYTES_SHIFT;
+
+ RemoteMemory[Idx].Reserved2 = EFI_ACPI_RESERVED_WORD;
+ RemoteMemory[Idx].Flags = 0x1;
+ RemoteMemory[Idx].Reserved3 = EFI_ACPI_RESERVED_WORD;
+
+ HostPhysicalBase += RemoteMem[Idx].DpaLength;
+ }
+
+ Status = gDS->AddMemorySpace (
+ EfiGcdMemoryTypeSystemMemory,
+ FixedPcdGet64 (PcdRemoteMemoryBase),
+ (HostPhysicalBase - FixedPcdGet64 (PcdRemoteMemoryBase)),
+ EFI_MEMORY_WC | EFI_MEMORY_WT | EFI_MEMORY_WB);
+
+ Status = gDS->SetMemorySpaceAttributes (
+ FixedPcdGet64 (PcdRemoteMemoryBase),
+ (HostPhysicalBase - FixedPcdGet64 (PcdRemoteMemoryBase)),
+ EFI_MEMORY_WB);
+ if (EFI_ERROR (Status)) {
+ DEBUG ((
+ DEBUG_ERROR,
+ "%a: Failed to set memory attributes: %r\n",
+ __func__,
+ Status
+ ));
+ FreePool (RemoteMem);
+ return 0;
+ }
+ }
+
+ FreePool (RemoteMem);
+ }
+
+ MemoryNodeCount = FixedPcdGet32 (PcdNumLocalMemBlock);
+ MemoryNodeCount += RemoteMemCount;
+ TableSize = sizeof (Gicc) +
+ sizeof (SratHeader) +
+ (MemoryNodeCount *
+ sizeof (EFI_ACPI_6_4_MEMORY_AFFINITY_STRUCTURE));
+
+ Srat = AllocatePool (TableSize);
+
+ if (Srat == NULL) {
+ DEBUG ((
+ DEBUG_ERROR,
+ "%a: Failed to allocate memory for SRAT table\n",
+ __func__
+ ));
+
+ FreePool(RemoteMemory);
+ return EFI_OUT_OF_RESOURCES;
+ }
+
+ CopyMem (
+ Srat,
+ &SratHeader,
+ sizeof (SratHeader)
+ );
+
+ SratDataNext = ((char *)Srat + sizeof (SratHeader));
+ CopyMem (
+ SratDataNext,
+ Gicc,
+ sizeof (Gicc)
+ );
+
+ SratDataNext = ((char *)SratDataNext + sizeof (Gicc));
+ CopyMem (
+ SratDataNext,
+ LocalMemory,
+ sizeof (LocalMemory)
+ );
+
+ SratDataNext = ((char *)SratDataNext + sizeof (LocalMemory));
+ if (RemoteMemCount) {
+ CopyMem (
+ SratDataNext,
+ RemoteMemory,
+ sizeof (EFI_ACPI_6_4_MEMORY_AFFINITY_STRUCTURE) *
+ RemoteMemCount
+ );
+ }
+
+ ((EFI_ACPI_DESCRIPTION_HEADER *)Srat)->Length = TableSize;
+
+ Status = mAcpiTableProtocol->InstallAcpiTable (
+ mAcpiTableProtocol,
+ (EFI_ACPI_6_4_SYSTEM_RESOURCE_AFFINITY_TABLE_HEADER *)Srat,
+ TableSize,
+ &AcpiTableHandle
+ );
+
+ if (EFI_ERROR (Status)) {
+ DEBUG ((
+ DEBUG_ERROR,
+ "%a: SRAT table installation failed, status: %r\n",
+ __func__,
+ Status
+ ));
+ } else {
+ DEBUG ((DEBUG_INFO, "Installed SRAT table \n"));
+ }
+
+ return Status;
+}
--
2.25.1
-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.
View/Reply Online (#109490): https://edk2.groups.io/g/devel/message/109490
Mute This Topic: https://groups.io/mt/101872995/7686176
Group Owner: devel+owner@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub [rebecca@openfw.io]
-=-=-=-=-=-=-=-=-=-=-=-
^ permalink raw reply related [flat|nested] 5+ messages in thread
* [edk2-devel] [edk2-platforms][PATCH v1 3/4] Platform/ARM: add CXL driver
2023-10-10 11:38 [edk2-devel] [edk2-platforms][PATCH v1 0/4] add support for enabling cxl-type3 device as memory expander on Arm/RD platform Sayanta Pattanayak
2023-10-10 11:38 ` [edk2-devel] [edk2-platforms][PATCH v1 1/4] Platform/Sgi: add remote memory descriptor Sayanta Pattanayak
2023-10-10 11:38 ` [edk2-devel] [edk2-platforms][PATCH v1 2/4] Platform/Sgi: prepare SRAT, HMAT table Sayanta Pattanayak
@ 2023-10-10 11:38 ` Sayanta Pattanayak
2023-10-10 11:38 ` [edk2-devel] [edk2-platforms][PATCH v1 4/4] Platform/Sgi: add CXL driver support Sayanta Pattanayak
3 siblings, 0 replies; 5+ messages in thread
From: Sayanta Pattanayak @ 2023-10-10 11:38 UTC (permalink / raw)
To: devel; +Cc: Ard Biesheuvel, Sami Mujawar
This patch introduces CXL Dxe, which primarily discovers a PCIe device
with CXL capability and configures the same.
This patch creates and registers event notifier based on
gEfiPciEnumerationCompleteProtocolGuid. Once PCIe enumeration is
completed then CXL Dxe triggers function to discover CXL capable device
with memory expander capability and reads out memory range details.
In addition to find out a PCIe device with CXL Mem extended capability,
the module also looks for CXL device with DOE capability. Once DOE
capability is found, then execute DOE operation to fetch CDAT
structures(DSMAS), which carry information about CXL Device Memory
range, type of memory etc. It stores the remote CXL memory details in
local data structure.
Later it installs CXL Protocol interface, which will trigger
AcpiTableGenerator module to fetch CXL memory details using CXL protocol
interfaces for preparation of ACPI tables.
Signed-off-by: Sayanta Pattanayak <sayanta.pattanayak@arm.com>
---
Platform/ARM/Drivers/CxlDxe/CxlDxe.dec | 21 +
Platform/ARM/Drivers/CxlDxe/CxlDxe.inf | 50 ++
Platform/ARM/Drivers/CxlDxe/CxlDxe.h | 163 ++++++
Platform/ARM/Include/Protocol/Cxl.h | 62 +++
Platform/ARM/Drivers/CxlDxe/CxlDxe.c | 575 ++++++++++++++++++++
5 files changed, 871 insertions(+)
diff --git a/Platform/ARM/Drivers/CxlDxe/CxlDxe.dec b/Platform/ARM/Drivers/CxlDxe/CxlDxe.dec
new file mode 100644
index 0000000000..930b0cc9da
--- /dev/null
+++ b/Platform/ARM/Drivers/CxlDxe/CxlDxe.dec
@@ -0,0 +1,21 @@
+## @file
+#
+# Copyright (c) 2023, Arm Limited. All rights reserved.
+# SPDX-License-Identifier: BSD-2-Clause-Patent
+#
+##
+
+[Defines]
+ DEC_SPECIFICATION = 0x0001001B
+ PACKAGE_NAME = CxlDxe
+ PACKAGE_GUID = cbeba70a-4351-4ff6-a074-6854337b39be
+ PACKAGE_VERSION = 0.1
+
+[Guids.common]
+ gArmRemoteCxlMemDescriptorGuid = { 0x29e54f5e, 0x365a, 0x4235, { 0xba, 0x62, 0xc8, 0x32, 0xd7, 0x66, 0x08, 0x52 } }
+
+[Protocols]
+ gCxlPlatformProtocolGuid = { 0x88c7bb9c, 0x6175, 0x4bfb, { 0x96, 0x40, 0xd2, 0x53, 0xd3, 0xd8, 0x7c, 0x17 } }
+
+[Includes.common]
+ Platform/ARM/Include
diff --git a/Platform/ARM/Drivers/CxlDxe/CxlDxe.inf b/Platform/ARM/Drivers/CxlDxe/CxlDxe.inf
new file mode 100644
index 0000000000..f2416a0768
--- /dev/null
+++ b/Platform/ARM/Drivers/CxlDxe/CxlDxe.inf
@@ -0,0 +1,50 @@
+## @file
+# Discovers CXL capable device and reads out device properties.
+#
+# Copyright (c) 2023, Arm Limited. All rights reserved.<BR>
+# SPDX-License-Identifier: BSD-2-Clause-Patent
+#
+##
+
+[Defines]
+ INF_VERSION = 0x0001001B
+ BASE_NAME = CxlDxe
+ FILE_GUID = 29e54f5e-365a-4235-ba62-c832d7660852
+ MODULE_TYPE = DXE_DRIVER
+ VERSION_STRING = 1.0
+ ENTRY_POINT = CxlDxeEntryPoint
+
+[Sources.common]
+ CxlDxe.c
+
+[Guids]
+ gArmRemoteCxlMemDescriptorGuid
+
+[Packages]
+ EmbeddedPkg/EmbeddedPkg.dec
+ MdeModulePkg/MdeModulePkg.dec
+ MdePkg/MdePkg.dec
+ ArmPkg/ArmPkg.dec
+ Platform/ARM/ARM.dec
+ Platform/ARM/Drivers/CxlDxe/CxlDxe.dec
+
+[LibraryClasses]
+ ArmMmuLib
+ ArmLib
+ BaseLib
+ DebugLib
+ DevicePathLib
+ DxeServicesTableLib
+ PcdLib
+ UefiBootServicesTableLib
+ UefiDriverEntryPoint
+ UefiLib
+ UefiRuntimeServicesTableLib
+
+[Protocols]
+ gCxlPlatformProtocolGuid ## PRODUCES
+ gEfiPciIoProtocolGuid ##CONSUMES
+ gEfiPciEnumerationCompleteProtocolGuid
+
+[Depex]
+ TRUE
diff --git a/Platform/ARM/Drivers/CxlDxe/CxlDxe.h b/Platform/ARM/Drivers/CxlDxe/CxlDxe.h
new file mode 100644
index 0000000000..02c60fa784
--- /dev/null
+++ b/Platform/ARM/Drivers/CxlDxe/CxlDxe.h
@@ -0,0 +1,163 @@
+/** @file
+ CXL driver file.
+
+ Defines CXL specific structures and macros.
+
+ Copyright (c) 2023, Arm Limited. All rights reserved.<BR>
+ SPDX-License-Identifier: BSD-2-Clause-Patent
+
+ @par Specification Reference:
+ - CXL Specificiation Revision 3.0, Version 0.7
+**/
+
+#ifndef CXL_DXE_H_
+#define CXL_DXE_H_
+
+#define PCIE_EXTENDED_CAP_OFFSET 0x100
+#define PCIE_EXTENDED_CAP_ID_MASK 0xFFFF
+#define PCIE_EXTENDED_NEXT_CAP_OFFSET_MASK 0xFFF
+#define PCIE_EXTENDED_NEXT_CAP_OFFSET_SHIFT 20
+
+#define PCIE_EXT_CAP_DOE_ID 0x2E
+
+#define PCIE_EXTENDED_CAP_NEXT(n) ((n)>>(PCIE_EXTENDED_NEXT_CAP_OFFSET_SHIFT))
+#define IS_CXL_DVSEC(n) (((n)&(0xFFFF)) == 0x1E98)
+
+#define DOE_DATA_OBJECT_VID 0x0000ffff
+#define DOE_DATA_OBJECT_TYPE 0x00ff0000
+#define DOE_DATA_OBJECT_LENGTH 0x0003ffff
+
+#define CXL_DOE_TABLE_ENTRY_HANDLE 0xffff0000
+
+#define CXL_DOE_TABLE_ENTRY_HANDLE_LAST 0xffff
+
+#define DVSEC_CXL_VENDOR_ID 0x1E98
+
+#define DOE_DATA_OBJ_HEADER_1 0x0
+#define DOE_DATA_OBJ_HEADER_2 0x4
+
+#define DOE_CAPABILITIES_REG 0x4
+#define DOE_CONTROL_REG 0x8
+#define DOE_STATUS_REG 0xC
+#define DOE_WRITE_DATA_MAILBOX_REG 0x10
+#define DOE_READ_DATA_MAILBOX_REG 0x14
+
+#define DOE_STAT_DOE_BUSY 0x1
+#define DOE_STAT_DATA_OBJ_READY ((0x1) << 31)
+#define DOE_CTRL_DOE_GO ((0x1) << 31)
+
+#define IS_DOE_SUPPORTED(n) \
+ (((n)&(PCIE_EXTENDED_CAP_ID_MASK))==(PCIE_EXT_CAP_DOE_ID))
+
+typedef enum {
+ PCIE_EXT_CAP_HEADER,
+ PCIE_DVSEC_HEADER_1,
+ PCIE_DVSEC_HEADER_2,
+ PCIE_DVSEC_HEADER_MAX
+} PCIE_DVSEC_HEADER_ENUM;
+
+/**
+ * Data Object Header
+ *
+ * Data Object Exchange(DOE) Header1 and Header2 put together.
+ * Reference taken from PCI-SIG ECN and
+ * CXL Specification (Revision 3.0, Version 0.7).
+**/
+typedef struct {
+ UINT16 VendorId;
+ UINT8 DataObjType;
+ UINT8 Reserved;
+ UINT32 Length;
+} DOE_HEADER;
+
+#define DOE_DATA_OBJ_TYPE_COMPLIANCE 0x0
+#define DOE_DATA_OBJ_TYPE_CDAT 0x2
+
+/**
+ * DOE read request data
+ *
+ * DOE read request data structure. For CXL DOE requests are made
+ * to read CDAT tables.
+ * Reference taken from CXL Specification (Revision 3.0, Version 0.7).
+**/
+typedef struct {
+ DOE_HEADER Header;
+ UINT8 ReqCode;
+ UINT8 TableType;
+ UINT16 EntryHandle;
+} CXL_CDAT_READ_ENTRY_REQ;
+
+#define CXL_CDAT_DOE_ENTRYHANDLE_LAST_ENTRY 0xFFFF
+
+/* Size in DW(4 Bytes) */
+#define CDAT_READ_ENTRY_REQ_SIZE 3
+
+/**
+ * DOE read response data
+ *
+ * DOE read response data structure. For CXL, DOE responses hold
+ * information about CDAT tables.
+ * Reference taken from CXL Specification (Revision 3.0, Version 0.7).
+**/
+typedef struct {
+ DOE_HEADER Header;
+ UINT8 RspCode;
+ UINT8 TableType;
+ UINT16 EntryHandle;
+ UINT32 CdatTable[32];
+} CXL_CDAT_READ_ENTRY_RESP;
+
+/* Size in DW(4 Bytes) */
+#define CDAT_READ_ENTRY_RESP_SIZE 3
+
+/**
+ * Coherent Device Attribute Table(CDAT) Header
+ *
+ * CDAT header, which is followed by variable number of CDAT structures.
+ * Reference taken from CDAT Specification (Revision 1.02).
+**/
+typedef struct {
+ UINT32 Length;
+ UINT8 Revision;
+ UINT8 Checksum;
+ UINT8 Reserved[6];
+ UINT32 Sequence;
+} CDAT_TABLE_HEADER;
+
+/* Size in DW(4 Bytes) */
+#define CDAT_TABLE_HEADER_SIZE 4
+
+/* Total CDAT table size. It can be increased further. */
+#define TOTAL_CDAT_ENTRY 24
+
+/**
+ * Device Scoped Memory Affinity Structure (DSMAS)
+ *
+ * DSMAS returns Device Physical Address(DPA) range and it's attributes.
+ * Reference taken from CDAT Specification (Revision 1.02).
+**/
+typedef struct {
+ UINT8 Type;
+ UINT8 Reserved_1;
+ UINT16 Length;
+ UINT8 DsmadHandle;
+ UINT8 Flags;
+ UINT16 Reserved_2;
+ UINT64 DpaBase;
+ UINT64 DpaLength;
+} CDAT_DSMAS;
+
+/* Size in DW(4 Bytes) */
+#define CDAT_STRUCTURE_DSMAS_SIZE 6
+
+typedef enum {
+ CDAT_STRUCTURE_DSMAS,
+ CDAT_STRUCTURE_DSLBIS,
+ CDAT_STRUCTURE_DSMSCIS,
+ CDAT_STRUCTURE_DSIS,
+ CDAT_STRUCTURE_DSEMTS,
+ CDAT_STRUCTURE_SSLBIS,
+ CDAT_STRUCTURE_COUNT
+} CDAT_STRUCTURE_TYPE;
+
+#endif /* CXL_DXE_H_ */
diff --git a/Platform/ARM/Include/Protocol/Cxl.h b/Platform/ARM/Include/Protocol/Cxl.h
new file mode 100644
index 0000000000..b2bd9d95e9
--- /dev/null
+++ b/Platform/ARM/Include/Protocol/Cxl.h
@@ -0,0 +1,62 @@
+/** @file
+ Interface API of CXL Platform protocol.
+
+ Declares the CXL Platform protocol interfaces, which are used by other
+ platform drivers for collecting information regarding discovered remote
+ memory nodes.
+
+ Copyright (c) 2023, Arm Limited. All rights reserved.<BR>
+ SPDX-License-Identifier: BSD-2-Clause-Patent
+**/
+
+#ifndef CXL_H_
+#define CXL_H_
+
+/** Remote memory details
+ *
+ * Remote memory region address in device address space and length of the
+ * region. These informations are passed using ACPI tables, where addressbase
+ * will be mapped to Host system address space.
+**/
+typedef struct {
+ UINT64 DpaAddress; /// Remote memory base in device address space
+ UINT64 DpaLength; /// Remote memory length lower bytes
+} REMOTE_MEMORY_CONFIG;
+
+/**
+ Update Remote memory information
+
+ Update the Remote memory details, Base address and Length, for number of
+ Remote memory nodes discovered from the CXL devices.
+
+ @param[out] RemoteMem Array for updating Remote memory config.
+ @param[in,out] MemCount Number of supported remote memory nodes.
+
+ @retval EFI_SUCCES Memory is updated successfully
+**/
+typedef
+EFI_STATUS
+(EFIAPI *CXL_GET_REMOTE_MEM) (
+ OUT REMOTE_MEMORY_CONFIG *RemoteMem,
+ IN OUT UINT32 *MemCount
+ );
+
+/**
+ Return number of remote memory nodes discovered from CXL Mem devices.
+
+ @retval UINT32 Number of supported remote memory nodes.
+**/
+typedef UINT32 (EFIAPI *CXL_GET_REMOTE_MEM_COUNT) (VOID);
+
+/**
+ * CXL Platform Protocol
+ *
+ * This protocol enables platform drivers to get number of memory range count
+ * and associated memory configurations.
+**/
+typedef struct {
+ CXL_GET_REMOTE_MEM CxlGetRemoteMem;
+ CXL_GET_REMOTE_MEM_COUNT CxlGetRemoteMemCount;
+} CXL_PLATFORM_PROTOCOL;
+
+#endif /* CXL_H_ */
diff --git a/Platform/ARM/Drivers/CxlDxe/CxlDxe.c b/Platform/ARM/Drivers/CxlDxe/CxlDxe.c
new file mode 100644
index 0000000000..409602189e
--- /dev/null
+++ b/Platform/ARM/Drivers/CxlDxe/CxlDxe.c
@@ -0,0 +1,575 @@
+/** @file
+ Discovers CXL capable device and reads out device capabilities.
+
+ This driver locates PciIo Protocol and discovers PCIe devices with CXL.Mem
+ capability. If a device with CXL.Mem capability is found then DOE capability
+ is looked for. Once DOE capability is found, CDAT structures are fetched from
+ the respective device.
+ It also installs CXL Platform portocol, which can be used by other
+ Platform drivers for capturing remote memory configurations and attributes.
+
+ Copyright (c) 2023, Arm Limited. All rights reserved.<BR>
+ SPDX-License-Identifier: BSD-2-Clause-Patent
+
+ @par Specification Reference:
+ - CXL Specificiation Revision 3.0, Version 0.7, Chapter 8.1.11
+**/
+
+#include <Library/BaseLib.h>
+#include <Library/BaseMemoryLib.h>
+#include <Library/DebugLib.h>
+#include <Library/MemoryAllocationLib.h>
+#include <Library/UefiBootServicesTableLib.h>
+#include <Library/UefiLib.h>
+#include <Protocol/Cxl.h>
+#include <Protocol/PciIo.h>
+#include <IndustryStandard/Pci.h>
+#include <IndustryStandard/Pci22.h>
+
+#include "CxlDxe.h"
+
+STATIC EFI_EVENT CxlEvent;
+VOID *mPciRegistration;
+STATIC UINT32 RemoteMemCount;
+//TODO: For now considered maximum 5 remote memory ranges.
+// In future it will be made dynamic.
+STATIC REMOTE_MEMORY_CONFIG RemoteMemConfig[5];
+
+/**
+ Check whether device is ready to receive new data through DOE request.
+
+ @param[in] Pci PCI IO Protocol handle.
+ @param[in] DoeBase Base offset of DOE status registers in PCIe device
+ config space.
+
+ @retval EFI_SUCCESS Successful read operation.
+ @retval Other Device not ready or failed to check device status.
+**/
+STATIC
+EFI_STATUS
+IsDoeBusy (
+ IN EFI_PCI_IO_PROTOCOL *Pci,
+ IN UINT32 DoeBase
+ )
+{
+ EFI_STATUS Status;
+ UINT32 DoeStatVal;
+
+ Status = Pci->Pci.Read (Pci, EfiPciIoWidthUint32, DoeBase, 1, &DoeStatVal);
+ if (EFI_ERROR (Status))
+ return Status;
+
+ if (DoeStatVal & DOE_STAT_DOE_BUSY)
+ return EFI_ALREADY_STARTED;
+
+ return Status;
+}
+
+/**
+ Read out CDAT structure data for host memory configuration.
+
+ From the DOE response data, various CDAT structure data are filtered out
+ for host platform configuration.
+
+ @param[in] DoeRespCdatDat Response data from DOE operation.
+ @param[in] Length DOE response data length in bytes.
+**/
+STATIC
+VOID
+UpdateCdatData (
+ IN UINT32 *DoeRespCdatData,
+ IN UINT16 Length
+ )
+{
+ UINT32 Index;
+ CDAT_DSMAS *Dsmas;
+
+ // Skipping the CDAT header.
+ Index = CDAT_TABLE_HEADER_SIZE;
+
+ while (Index < Length) {
+ switch (DoeRespCdatData[Index] & 0xff) {
+ case CDAT_STRUCTURE_DSMAS:
+ Dsmas = (CDAT_DSMAS *)(& (DoeRespCdatData [Index]));
+ RemoteMemConfig[RemoteMemCount].DpaAddress = Dsmas->DpaBase;
+ RemoteMemConfig[RemoteMemCount].DpaLength = Dsmas->DpaLength;
+ RemoteMemCount ++;
+ Index += CDAT_STRUCTURE_DSMAS_SIZE;
+ break;
+ default:
+ break;
+ }
+ }
+
+ return;
+}
+
+/**
+ Receive DOE response.
+
+ For CXL, DOE responses carry CDAT structures that hold information about
+ remote memory ranges and associated attributes.
+ System firmware polls the Data Object Ready bit and, provided it is Set, reads
+ data from the DOE Read Data Mailbox and writes 1 to the DOE Read Data Mailbox
+ to indicate a successful read.In the read process, a DWORD is read at a time.
+ Data Object Header2 holds number of DW to be transferred for capturing the
+ entire DOE response.
+
+ @param[in] Pci PCI IO Protocol handle.
+ @param[in] DoeBase Base offset of DOE registers in PCIe device config
+ space.
+ @param[out] EntryHandle Value of next entry in table. For CXL, table type
+ is CDAT.
+
+ @retval EFI_SUCCESS Successful receiving of DOE response.
+ @retval Other Failed receiving of DOE response.
+**/
+STATIC
+EFI_STATUS
+DoeReceiveResponse (
+ IN EFI_PCI_IO_PROTOCOL *Pci,
+ IN UINT32 DoeBase,
+ OUT UINT16 *EntryHandle
+ )
+{
+ EFI_STATUS Status;
+ UINT32 DoeReadMbData = 1;
+ UINT32 DoeRespCdatData[TOTAL_CDAT_ENTRY] = {};
+ UINT32 DoeStat;
+ UINT32 Index = 0;
+ UINT16 Length;
+ UINT64 Reg;
+ UINT32 Val;
+
+ Reg = DoeBase + DOE_STATUS_REG;
+ Status = Pci->Pci.Read ( Pci, EfiPciIoWidthUint32, Reg, 1, &DoeStat);
+ if (EFI_ERROR (Status))
+ return Status;
+
+ if ((DoeStat & DOE_STAT_DATA_OBJ_READY) != 0) {
+ Index = 0;
+ Reg = DoeBase + DOE_READ_DATA_MAILBOX_REG;
+
+ // Read the DOE header.
+ Status = Pci->Pci.Read (Pci, EfiPciIoWidthUint32, Reg, 1, &Val);
+ if (EFI_ERROR (Status))
+ return Status;
+
+ // Write 1 to DOE Read Data Mailbox to indicate successful Read.
+ Status = Pci->Pci.Write (Pci, EfiPciIoWidthUint32, Reg, 1, &DoeReadMbData);
+ if (EFI_ERROR (Status))
+ return Status;
+
+ // Read the DOE Header 2 for data length.
+ Status = Pci->Pci.Read (Pci, EfiPciIoWidthUint32, Reg, 1, &Val);
+ if (EFI_ERROR (Status))
+ return Status;
+
+ Length = Val & DOE_DATA_OBJECT_LENGTH;
+ if (Length < 2) {
+ DEBUG ((DEBUG_ERROR, " DOE data read error\n"));
+ return EFI_PROTOCOL_ERROR;
+ }
+
+ // Write 1 to DOE Read Data Mailbox to indicate successful Read.
+ Status = Pci->Pci.Write (Pci, EfiPciIoWidthUint32, Reg, 1, &DoeReadMbData);
+ if (EFI_ERROR (Status))
+ return Status;
+
+ // Read DOE read entry response header.
+ Status = Pci->Pci.Read (Pci, EfiPciIoWidthUint32, Reg, 1, &Val);
+ if (EFI_ERROR (Status))
+ return Status;
+
+ *EntryHandle = ((Val & CXL_DOE_TABLE_ENTRY_HANDLE) >> 16);
+ // Write 1 to DOE Read Data Mailbox to indicate successful Read.
+ Status = Pci->Pci.Write (Pci, EfiPciIoWidthUint32, Reg, 1, &DoeReadMbData);
+ if (EFI_ERROR (Status))
+ return Status;
+
+ // Discount the length of 2DW DOE Header and 1DW Read entry response
+ Length -= 3;
+ while (Index < Length) {
+ Status = Pci->Pci.Read (Pci, EfiPciIoWidthUint32, Reg, 1, &Val);
+ if (EFI_ERROR (Status))
+ return Status;
+
+ DoeRespCdatData[Index] = Val;
+ Index++;
+ // Write 1 to DOE Read Data Mailbox to indicate successful Read.
+ Status = Pci->Pci.Write (
+ Pci,
+ EfiPciIoWidthUint32,
+ Reg,
+ 1,
+ &DoeReadMbData
+ );
+ if (EFI_ERROR (Status))
+ return Status;
+ }
+
+ UpdateCdatData (DoeRespCdatData, Length);
+ }
+
+ return Status;
+}
+
+/**
+ Make DOE request to fetch CDAT structures and receive DOE response.
+
+ This function requests for DOE objects and receives response for the same.
+ The steps include -
+ 1. System firmware checks that the DOE Busy bit is Clear.
+ 2. System firmware writes entire data object a DWORD(4 Bytes) at a time via
+ DOE Write Data Mailbox register.
+ 3. System firmware writes 1b to the DOE Go bit.
+ 4. The DOE instance consumes the DOE request from the DOE mailbox.
+ 5. The DOE instance generates a DOE Response and Sets Data Object Ready bit.
+ 6. System firmware polls the Data Object Ready bit and, provided it is Set,
+ reads data from the DOE Read Data Mailbox and writes 1 to the DOE Read
+ Data Mailbox to indicate a successful read, a DWORD at a time until the
+ entire DOE Response is read.
+ 7: DOE requests are made until response for last CDAT table entry is received.
+
+ @param[in] Pci PCI IO Protocol handle.
+ @param[in] DoeBase Base offset of DOE registers in PCIe device config space.
+
+ @retval EFI_SUCCESS Successful DOE request and response receive.
+ @retval Other Failed DOE request or response receive.
+**/
+STATIC
+EFI_STATUS
+SendDoeCommand (
+ IN EFI_PCI_IO_PROTOCOL *Pci,
+ IN UINT32 DoeBase
+ )
+{
+ EFI_STATUS Status;
+ UINT32 Val;
+ UINT64 Reg;
+ CXL_CDAT_READ_ENTRY_REQ CxlDoeReq;
+ UINT32 Index = 0;
+
+ // CDAT DOE Request header & Read entry request object.
+ CxlDoeReq.Header.VendorId = DVSEC_CXL_VENDOR_ID;
+ CxlDoeReq.Header.DataObjType = DOE_DATA_OBJ_TYPE_CDAT;
+ CxlDoeReq.Header.Length = CDAT_READ_ENTRY_REQ_SIZE;
+
+ // 0 indicates that it's a read request.
+ CxlDoeReq.ReqCode = 0;
+
+ // 0 indicates that table type is CDAT.
+ CxlDoeReq.TableType = 0;
+
+ // 0 represents very first entry in the table.
+ CxlDoeReq.EntryHandle = 0;
+
+ Reg = DoeBase + DOE_WRITE_DATA_MAILBOX_REG;
+
+ do {
+ Status = IsDoeBusy (Pci, DoeBase + DOE_STATUS_REG);
+
+ if (EFI_ERROR (Status)) {
+ DEBUG ((DEBUG_WARN, "Device busy or DOE request can't be made\n"));
+ return Status;
+ }
+
+ while (Index < CDAT_READ_ENTRY_REQ_SIZE) {
+ Val = *((UINT32 *) (&CxlDoeReq) + Index);
+ Status = Pci->Pci.Write (Pci, EfiPciIoWidthUint32, Reg, 1, &Val);
+ if (EFI_ERROR (Status)) {
+ DEBUG ((DEBUG_WARN, "Error while making DOE request\n"));
+ return Status;
+ }
+ Index++;
+ }
+
+ Reg = DoeBase + DOE_CONTROL_REG;
+ Status = Pci->Pci.Read (Pci, EfiPciIoWidthUint32, Reg, 1, &Val);
+ if (EFI_ERROR (Status)) {
+ DEBUG ((DEBUG_WARN, "Error while reading DOE control reg\n"));
+ return Status;
+ }
+
+ Val |= DOE_CTRL_DOE_GO;
+ Status = Pci->Pci.Write (Pci, EfiPciIoWidthUint32, Reg, 1, &Val);
+ if (EFI_ERROR (Status)) {
+ DEBUG ((DEBUG_WARN, "Error while writing into DOE control reg\n"));
+ return Status;
+ }
+
+ Status = DoeReceiveResponse (Pci, DoeBase, &CxlDoeReq.EntryHandle);
+ if (EFI_ERROR (Status)) {
+ DEBUG ((DEBUG_WARN, "Error while receiving DOE response\n"));
+ return Status;
+ }
+ } while (CxlDoeReq.EntryHandle < CXL_DOE_TABLE_ENTRY_HANDLE_LAST);
+
+ return Status;
+}
+
+/**
+ Return number of remote memory nodes discovered from CXL Mem devices.
+
+ @retval UINT32 Number of supported remote memory nodes
+**/
+STATIC UINT32 EFIAPI CxlGetRemoteMemCount (VOID)
+{
+ return RemoteMemCount;
+}
+
+/**
+ Update Remote memory information
+
+ Update the Remote memory details, Base address and Length, for number of
+ Remote memory nodes discovered from the CXL devices. If the update request
+ for number of memory nodes is more than discovered remote memory nodes number,
+ then MemCount is modified to number of discovered remote memory nodes.
+
+ @param[out] RemoteMem Array for updating Remote memory config.
+ @param[in,out] MemCount Number of supported remote memory nodes.
+
+ @retval EFI_SUCCES Memory is updated successfully
+**/
+STATIC
+EFI_STATUS
+EFIAPI
+CxlGetRemoteMem (
+ OUT REMOTE_MEMORY_CONFIG *RemoteMemInfo,
+ IN OUT UINT32 *MemCount
+ )
+{
+
+ if ((*MemCount) > RemoteMemCount) {
+ DEBUG ((DEBUG_WARN, "Requested for more than max. Remote Memory node\n"));
+ *MemCount = RemoteMemCount;
+ }
+
+ CopyMem (
+ RemoteMemInfo,
+ RemoteMemConfig,
+ sizeof (REMOTE_MEMORY_CONFIG) * (*MemCount)
+ );
+
+ return EFI_SUCCESS;
+}
+
+/**
+ Installs the CXL platform protocol.
+
+ CXL platform protocol has interfaces for providing CXL mem device
+ configurations. A Platform driver can fetch such configurations
+ using these protocl interfaces.
+
+ @retval EFI_SUCCESS On successful installation of protocol interfaces.
+ @retval Other On failure of protocol installation.
+**/
+STATIC
+EFI_STATUS
+CxlInstallProtocol (VOID)
+{
+ EFI_STATUS Status;
+ EFI_HANDLE *CxlPlatformHandle;
+ CXL_PLATFORM_PROTOCOL *CxlPlatformProtocol;
+ STATIC BOOLEAN CxlProtocolInstalled = FALSE;
+
+ if (CxlProtocolInstalled == TRUE) {
+ DEBUG ((DEBUG_INFO, "Protocol already installed. \n"));
+ return EFI_SUCCESS;
+ }
+
+ CxlPlatformHandle = (EFI_HANDLE *) AllocateZeroPool (sizeof(EFI_HANDLE));
+
+ CxlPlatformProtocol =
+ (CXL_PLATFORM_PROTOCOL *) AllocateZeroPool (sizeof(CXL_PLATFORM_PROTOCOL));
+
+ if (!CxlPlatformProtocol) {
+ DEBUG ((
+ DEBUG_ERROR,
+ "CxlInstallProtocol: Failed to allocate memory for CxlPlatformProtocol\n"
+ ));
+ return EFI_OUT_OF_RESOURCES;
+ }
+
+ CxlPlatformProtocol->CxlGetRemoteMem = CxlGetRemoteMem;
+ CxlPlatformProtocol->CxlGetRemoteMemCount = CxlGetRemoteMemCount;
+
+ Status = gBS->InstallProtocolInterface (
+ CxlPlatformHandle,
+ &gCxlPlatformProtocolGuid,
+ EFI_NATIVE_INTERFACE,
+ CxlPlatformProtocol
+ );
+
+ if (EFI_ERROR(Status)) {
+ DEBUG ((
+ DEBUG_ERROR,
+ "CxlInstallProtocol: Failed to install CxlPlatformProtocol: 0x%08x\n",
+ Status
+ ));
+
+ return Status;
+ }
+
+ CxlProtocolInstalled = TRUE;
+
+ DEBUG ((DEBUG_INFO, "Installed protocol: %p\n", CxlPlatformProtocol));
+ return EFI_SUCCESS;
+}
+
+VOID
+EFIAPI
+PciBusEvent (
+ IN EFI_EVENT Event,
+ IN VOID* Context
+ )
+{
+
+ EFI_STATUS Status;
+ EFI_PCI_IO_PROTOCOL *Pci;
+ UINTN Seg, Bus, Dev, Func;
+ EFI_HANDLE *HandleBuffer;
+ UINTN HandleCount, Index;
+ UINT32 ExtCapOffset, NextExtCapOffset;
+ UINT32 NextDoeExtCapOffset;
+ UINT32 PcieExtCapAndDvsecHeader[3];
+ UINT32 PcieExtCapHeader;
+
+ Status = gBS->LocateHandleBuffer (
+ ByProtocol,
+ &gEfiPciIoProtocolGuid,
+ NULL,
+ &HandleCount,
+ &HandleBuffer
+ );
+ if (EFI_ERROR(Status)) {
+ DEBUG ((DEBUG_ERROR, "Failed to locate any PciIo protocols\n"));
+ return;
+ }
+
+ for (Index = 0; Index < HandleCount; Index++) {
+ Status = gBS->HandleProtocol (
+ HandleBuffer[Index],
+ &gEfiPciIoProtocolGuid,
+ (VOID **)&Pci
+ );
+ if (EFI_ERROR (Status)) {
+ DEBUG ((DEBUG_ERROR, "Failed to get Pci handle\n"));
+ return;
+ }
+
+ Pci->GetLocation (Pci, &Seg, &Bus, &Dev, &Func);
+ NextExtCapOffset = PCIE_EXTENDED_CAP_OFFSET;
+
+ do {
+ ExtCapOffset = NextExtCapOffset;
+ Status = Pci->Pci.Read (
+ Pci,
+ EfiPciIoWidthUint32,
+ ExtCapOffset,
+ PCIE_DVSEC_HEADER_MAX,
+ PcieExtCapAndDvsecHeader
+ );
+ if (EFI_ERROR (Status)) {
+ DEBUG ((DEBUG_ERROR, "Failed to read PCI IO for Ext. capability\n"));
+ return;
+ }
+
+ /* Check whether this is a CXL device */
+ if (IS_CXL_DVSEC (PcieExtCapAndDvsecHeader[PCIE_DVSEC_HEADER_1])) {
+ DEBUG ((DEBUG_INFO, "Found CXL Device\n"));
+
+ NextDoeExtCapOffset = PCIE_EXTENDED_CAP_OFFSET;
+ do {
+ ExtCapOffset = NextDoeExtCapOffset;
+ Status = Pci->Pci.Read (
+ Pci,
+ EfiPciIoWidthUint32,
+ ExtCapOffset,
+ 1,
+ &PcieExtCapHeader
+ );
+ if (EFI_ERROR (Status)) {
+ DEBUG ((DEBUG_ERROR, "Failed to read PCI Ext. capability\n"));
+ return;
+ }
+
+ if (IS_DOE_SUPPORTED (PcieExtCapHeader)) {
+ DEBUG ((DEBUG_INFO, "Found DOE Capability\n"));
+ Status = SendDoeCommand (Pci, ExtCapOffset);
+
+ if (EFI_ERROR (Status)) {
+ DEBUG ((DEBUG_WARN, "Not Found DOE Capability\n"));
+ } else {
+ Status = CxlInstallProtocol();
+ if (EFI_ERROR (Status))
+ return;
+ }
+
+ NextExtCapOffset = 0;
+ break;
+ }
+
+ NextDoeExtCapOffset = PCIE_EXTENDED_CAP_NEXT (PcieExtCapHeader);
+ } while(NextDoeExtCapOffset);
+ }
+
+ if (NextExtCapOffset == 0)
+ break;
+
+ NextExtCapOffset = PCIE_EXTENDED_CAP_NEXT (
+ PcieExtCapAndDvsecHeader[PCIE_EXT_CAP_HEADER]
+ );
+
+ } while (NextExtCapOffset);
+ }
+
+ gBS->CloseEvent (CxlEvent);
+ CxlEvent = NULL;
+
+ return;
+}
+
+/**
+ Entry point for CXL DXE.
+
+ This Dxe depends on gEfiPciEnumerationCompleteProtocolGuid. It locates
+ PciIo Protocol and discovers PCIe devices with CXL.Mem capability. If a
+ device with CXL.Mem capability is found then DOE capability is looked for.
+ After that, CXL.Mem device configurations are fetched, and thereafter CXL
+ Platform portocol is installed.
+
+ @param[in] ImageHandle Handle to the Efi image.
+ @param[in] SystemTable A pointer to the Efi System Table.
+
+ @retval EFI_SUCCESS On successful execution of mentioned functionlities.
+ @retval Other On failure.
+**/
+EFI_STATUS
+EFIAPI
+CxlDxeEntryPoint (
+ IN EFI_HANDLE ImageHandle,
+ IN EFI_SYSTEM_TABLE *SystemTable
+ )
+{
+ EFI_STATUS Status;
+
+ Status = gBS->CreateEvent (
+ EVT_NOTIFY_SIGNAL,
+ TPL_CALLBACK,
+ PciBusEvent,
+ NULL,
+ &CxlEvent
+ );
+
+ //
+ // Register for protocol notifications on this event
+ //
+ Status = gBS->RegisterProtocolNotify (
+ &gEfiPciEnumerationCompleteProtocolGuid,
+ CxlEvent,
+ &mPciRegistration
+ );
+
+ return Status;
+}
--
2.25.1
-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.
View/Reply Online (#109491): https://edk2.groups.io/g/devel/message/109491
Mute This Topic: https://groups.io/mt/101872996/7686176
Group Owner: devel+owner@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub [rebecca@openfw.io]
-=-=-=-=-=-=-=-=-=-=-=-
^ permalink raw reply related [flat|nested] 5+ messages in thread
* [edk2-devel] [edk2-platforms][PATCH v1 4/4] Platform/Sgi: add CXL driver support
2023-10-10 11:38 [edk2-devel] [edk2-platforms][PATCH v1 0/4] add support for enabling cxl-type3 device as memory expander on Arm/RD platform Sayanta Pattanayak
` (2 preceding siblings ...)
2023-10-10 11:38 ` [edk2-devel] [edk2-platforms][PATCH v1 3/4] Platform/ARM: add CXL driver Sayanta Pattanayak
@ 2023-10-10 11:38 ` Sayanta Pattanayak
3 siblings, 0 replies; 5+ messages in thread
From: Sayanta Pattanayak @ 2023-10-10 11:38 UTC (permalink / raw)
To: devel; +Cc: Ard Biesheuvel, Sami Mujawar
CxlDxe discovers CXL capable device with memory expander capability
and reads out memory range details. Platform drivers fetch the remote
memory details and include as part of system memory through ACPI
tables.
Signed-off-by: Sayanta Pattanayak <sayanta.pattanayak@arm.com>
---
Platform/ARM/SgiPkg/SgiPlatform.dsc.inc | 1 +
Platform/ARM/SgiPkg/SgiPlatform.fdf | 11 ++++++++---
2 files changed, 9 insertions(+), 3 deletions(-)
diff --git a/Platform/ARM/SgiPkg/SgiPlatform.dsc.inc b/Platform/ARM/SgiPkg/SgiPlatform.dsc.inc
index fbe5b0b705..d1d833d9e6 100644
--- a/Platform/ARM/SgiPkg/SgiPlatform.dsc.inc
+++ b/Platform/ARM/SgiPkg/SgiPlatform.dsc.inc
@@ -314,6 +314,7 @@
# SRAT/HMAT Table generator
#
!if $(EDK2_ENABLE_REMOTE_CXL_MEM) == TRUE
+ edk2-platforms/Platform/ARM/Drivers/CxlDxe/CxlDxe.inf
Platform/ARM/SgiPkg/Library/AcpiTableGeneratorLib/AcpiTableGenerator.inf
!endif
diff --git a/Platform/ARM/SgiPkg/SgiPlatform.fdf b/Platform/ARM/SgiPkg/SgiPlatform.fdf
index b1a227f22e..ae48719979 100644
--- a/Platform/ARM/SgiPkg/SgiPlatform.fdf
+++ b/Platform/ARM/SgiPkg/SgiPlatform.fdf
@@ -194,10 +194,15 @@ READ_LOCK_STATUS = TRUE
INF MdeModulePkg/Universal/SetupBrowserDxe/SetupBrowserDxe.inf
INF MdeModulePkg/Universal/BdsDxe/BdsDxe.inf
- #
- # SRAT/HMAT Table generator
- #
!if $(EDK2_ENABLE_REMOTE_CXL_MEM) == TRUE
+ #
+ # Cxl Device driver
+ #
+ INF edk2-platforms/Platform/ARM/Drivers/CxlDxe/CxlDxe.inf
+
+ #
+ # SRAT/HMAT Table generator
+ #
INF Platform/ARM/SgiPkg/Library/AcpiTableGeneratorLib/AcpiTableGenerator.inf
!endif
--
2.25.1
-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.
View/Reply Online (#109492): https://edk2.groups.io/g/devel/message/109492
Mute This Topic: https://groups.io/mt/101872997/7686176
Group Owner: devel+owner@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub [rebecca@openfw.io]
-=-=-=-=-=-=-=-=-=-=-=-
^ permalink raw reply related [flat|nested] 5+ messages in thread
end of thread, other threads:[~2023-10-10 11:38 UTC | newest]
Thread overview: 5+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2023-10-10 11:38 [edk2-devel] [edk2-platforms][PATCH v1 0/4] add support for enabling cxl-type3 device as memory expander on Arm/RD platform Sayanta Pattanayak
2023-10-10 11:38 ` [edk2-devel] [edk2-platforms][PATCH v1 1/4] Platform/Sgi: add remote memory descriptor Sayanta Pattanayak
2023-10-10 11:38 ` [edk2-devel] [edk2-platforms][PATCH v1 2/4] Platform/Sgi: prepare SRAT, HMAT table Sayanta Pattanayak
2023-10-10 11:38 ` [edk2-devel] [edk2-platforms][PATCH v1 3/4] Platform/ARM: add CXL driver Sayanta Pattanayak
2023-10-10 11:38 ` [edk2-devel] [edk2-platforms][PATCH v1 4/4] Platform/Sgi: add CXL driver support Sayanta Pattanayak
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox