[edk2-devel] [PATCH v3 0/1] support multi memory nodes

[edk2-devel] [PATCH v3 0/1] support multi memory nodes

[Xiong Yining]

When SaSbQemu platform is configured with multi memory nodes, like 
numa architecture, os will ignore any memory node in the device tree 
except the first one. The kernel reads UEFI memory map for memory 
information when booting via UEFI. 

However UEFI only allocates the first memory node memory space for 
SbsaQemu platform, in this scenario we can refer to the implement 
of "OvmfPkg/Fdt/HighMemDxe" and use the GCD services to add memory 
spaces for high memory node.

Instead of using FdtClientDxe driver to get the informatin of memory, 
we get the hardware information of memory via SMC calls, which is proposed
on get rid of DeviceTree from SbsaQemu:
https://openfw.io/edk2-devel/20240131132400.3022662-1-xiongyining1480@phytium.com.cn/

Changes in v2:
- get the information of memory via SMC rather than FdtClientDxe.
- add a new driver rather than use HighMemDxe. 

Changes in v3:
- when the memory node is the first one, there is no need to add it.

Xiong Yining (1):
  SbsaQemu: add memory space for the high memory nodes

 Platform/Qemu/SbsaQemu/SbsaQemu.dsc           |   3 +-
 Platform/Qemu/SbsaQemu/SbsaQemu.fdf           |   1 +
 .../SbsaQemuHighMemDxe/SbsaQemuHighMemDxe.inf |  45 ++++++
 .../SbsaQemuHighMemDxe/SbsaQemuHighMemDxe.c   | 134 ++++++++++++++++++
 4 files changed, 182 insertions(+), 1 deletion(-)
 create mode 100644 Silicon/Qemu/SbsaQemu/Drivers/SbsaQemuHighMemDxe/SbsaQemuHighMemDxe.inf
 create mode 100644 Silicon/Qemu/SbsaQemu/Drivers/SbsaQemuHighMemDxe/SbsaQemuHighMemDxe.c

-- 
2.34.1



-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.
View/Reply Online (#115637): https://edk2.groups.io/g/devel/message/115637
Mute This Topic: https://groups.io/mt/104463711/7686176
Group Owner: devel+owner@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub [rebecca@openfw.io]
-=-=-=-=-=-=-=-=-=-=-=-

[edk2-devel] [PATCH v3 1/1] SbsaQemu: add memory space for the high memory nodes

[Xiong Yining]

To support more memory nodes, we refer to the implement of
"OvmfPkg/Fdt/HighMemDxe" to add memory space for the high memory nodes
except the first one.

Signed-off-by: Xiong Yining <xiongyining1480@phytium.com.cn>
Signed-off-by: Chen Baozi <chenbaozi@phytium.com.cn>
---
 Platform/Qemu/SbsaQemu/SbsaQemu.dsc           |   1 +
 Platform/Qemu/SbsaQemu/SbsaQemu.fdf           |   1 +
 .../SbsaQemuHighMemDxe/SbsaQemuHighMemDxe.inf |  45 ++++++
 .../SbsaQemuHighMemDxe/SbsaQemuHighMemDxe.c   | 134 ++++++++++++++++++
 4 files changed, 182 insertions(+), 1 deletion(-)
 create mode 100644 Silicon/Qemu/SbsaQemu/Drivers/SbsaQemuHighMemDxe/SbsaQemuHighMemDxe.inf
 create mode 100644 Silicon/Qemu/SbsaQemu/Drivers/SbsaQemuHighMemDxe/SbsaQemuHighMemDxe.c

diff --git a/Platform/Qemu/SbsaQemu/SbsaQemu.dsc b/Platform/Qemu/SbsaQemu/SbsaQemu.dsc
index bde61651da2e..e48bb8eb0174 100644
--- a/Platform/Qemu/SbsaQemu/SbsaQemu.dsc
+++ b/Platform/Qemu/SbsaQemu/SbsaQemu.dsc
@@ -675,6 +675,7 @@ DEFINE NETWORK_HTTP_BOOT_ENABLE       = FALSE
   ArmPkg/Drivers/TimerDxe/TimerDxe.inf
   OvmfPkg/VirtNorFlashDxe/VirtNorFlashDxe.inf
   MdeModulePkg/Universal/WatchdogTimerDxe/WatchdogTimer.inf
+  Silicon/Qemu/SbsaQemu/Drivers/SbsaQemuHighMemDxe/SbsaQemuHighMemDxe.inf
 
   #
   # FAT filesystem + GPT/MBR partitioning
diff --git a/Platform/Qemu/SbsaQemu/SbsaQemu.fdf b/Platform/Qemu/SbsaQemu/SbsaQemu.fdf
index 6fcfd25faaeb..b35f42e11aa4 100644
--- a/Platform/Qemu/SbsaQemu/SbsaQemu.fdf
+++ b/Platform/Qemu/SbsaQemu/SbsaQemu.fdf
@@ -161,6 +161,7 @@ READ_LOCK_STATUS   = TRUE
 
   INF MdeModulePkg/Core/Dxe/DxeMain.inf
   INF MdeModulePkg/Universal/PCD/Dxe/Pcd.inf
+  INF Silicon/Qemu/SbsaQemu/Drivers/SbsaQemuHighMemDxe/SbsaQemuHighMemDxe.inf
 
   #
   # PI DXE Drivers producing Architectural Protocols (EFI Services)
diff --git a/Silicon/Qemu/SbsaQemu/Drivers/SbsaQemuHighMemDxe/SbsaQemuHighMemDxe.inf b/Silicon/Qemu/SbsaQemu/Drivers/SbsaQemuHighMemDxe/SbsaQemuHighMemDxe.inf
new file mode 100644
index 000000000000..85bfb722affa
--- /dev/null
+++ b/Silicon/Qemu/SbsaQemu/Drivers/SbsaQemuHighMemDxe/SbsaQemuHighMemDxe.inf
@@ -0,0 +1,45 @@
+## @file
+#  High memory node enumeration DXE driver for SbsaQemu
+#
+#  Copyright (c) 2023, Linaro Ltd. All rights reserved.
+#
+#  SPDX-License-Identifier: BSD-2-Clause-Patent
+#
+##
+
+[Defines]
+  INF_VERSION                    = 0x00010005
+  BASE_NAME                      = SbsaQemuHighMemDxe
+  FILE_GUID                      = 9E749C5E-C282-32F8-7CC3-E5A3DDE15329
+  MODULE_TYPE                    = DXE_DRIVER
+  VERSION_STRING                 = 1.0
+
+  ENTRY_POINT                    = InitializeHighMemDxe
+
+[Sources]
+  SbsaQemuHighMemDxe.c
+
+[Packages]
+  MdePkg/MdePkg.dec
+  MdeModulePkg/MdeModulePkg.dec
+  ArmPkg/ArmPkg.dec
+  Silicon/Qemu/SbsaQemu/SbsaQemu.dec
+
+[LibraryClasses]
+  BaseLib
+  DebugLib
+  DxeServicesTableLib
+  PcdLib
+  UefiBootServicesTableLib
+  UefiDriverEntryPoint
+  SbsaQemuHardwareInfoLib
+
+[Protocols]
+  gEfiCpuArchProtocolGuid                 ## CONSUMES
+
+[Pcd]
+  gEfiMdeModulePkgTokenSpaceGuid.PcdDxeNxMemoryProtectionPolicy
+  gArmTokenSpaceGuid.PcdSystemMemoryBase
+
+[Depex]
+  gEfiCpuArchProtocolGuid
diff --git a/Silicon/Qemu/SbsaQemu/Drivers/SbsaQemuHighMemDxe/SbsaQemuHighMemDxe.c b/Silicon/Qemu/SbsaQemu/Drivers/SbsaQemuHighMemDxe/SbsaQemuHighMemDxe.c
new file mode 100644
index 000000000000..35ccf8eb942b
--- /dev/null
+++ b/Silicon/Qemu/SbsaQemu/Drivers/SbsaQemuHighMemDxe/SbsaQemuHighMemDxe.c
@@ -0,0 +1,134 @@
+/** @file
+*  High memory node enumeration DXE driver for SbsaQemu
+*  Virtual Machines
+*
+*  Copyright (c) 2023, Linaro Ltd. All rights reserved.
+*
+*  SPDX-License-Identifier: BSD-2-Clause-Patent
+*
+**/
+
+#include <Library/BaseLib.h>
+#include <Library/DebugLib.h>
+#include <Library/DxeServicesTableLib.h>
+#include <Library/PcdLib.h>
+#include <Library/UefiBootServicesTableLib.h>
+#include <Library/SbsaQemuHardwareInfoLib.h>
+
+#include <Protocol/Cpu.h>
+
+EFI_STATUS
+EFIAPI
+InitializeHighMemDxe (
+  IN EFI_HANDLE        ImageHandle,
+  IN EFI_SYSTEM_TABLE  *SystemTable
+  )
+{
+  EFI_CPU_ARCH_PROTOCOL            *Cpu;
+  EFI_STATUS                       Status;
+  UINT32                           NumMemNodes;
+  UINT32                           index;
+  UINT64                           CurBase;
+  UINT64                           CurSize;
+  UINT64                           Attributes;
+  MemoryInfo                       MemInfo;
+  EFI_GCD_MEMORY_SPACE_DESCRIPTOR  GcdDescriptor;
+
+  Status = gBS->LocateProtocol (
+                  &gEfiCpuArchProtocolGuid,
+                  NULL,
+                  (VOID **)&Cpu
+                  );
+  ASSERT_EFI_ERROR (Status);
+
+  //
+  // Check for memory node and add the memory spaces except the lowest one
+  //
+  NumMemNodes = SbsaQemuGetMemNodeCount();
+  for (index = 0; index < NumMemNodes; index++){
+    MemInfo = SbsaQemuGetMemInfo(index);
+    CurBase = MemInfo.AddressBase;
+    CurSize = MemInfo.AddressSize;
+
+    if (CurBase > PcdGet64 (PcdSystemMemoryBase)) {
+      Status = gDS->GetMemorySpaceDescriptor (CurBase, &GcdDescriptor);
+      if (EFI_ERROR (Status)) {
+        DEBUG ((
+          DEBUG_WARN,
+          "%a: Region 0x%lx - 0x%lx not found in the GCD memory space map\n",
+          __func__,
+          CurBase,
+          CurBase + CurSize - 1
+          ));
+        continue;
+      }
+
+      if (GcdDescriptor.GcdMemoryType == EfiGcdMemoryTypeNonExistent) {
+        Status = gDS->AddMemorySpace (
+                        EfiGcdMemoryTypeSystemMemory,
+                        CurBase,
+                        CurSize,
+                        EFI_MEMORY_WB
+                        );
+
+        if (EFI_ERROR (Status)) {
+          DEBUG ((
+            DEBUG_ERROR,
+            "%a: Failed to add System RAM @ 0x%lx - 0x%lx (%r)\n",
+            __func__,
+            CurBase,
+            CurBase + CurSize - 1,
+            Status
+            ));
+          continue;
+        }
+
+        Status = gDS->SetMemorySpaceAttributes (
+                        CurBase,
+                        CurSize,
+                        EFI_MEMORY_WB
+                        );
+        if (EFI_ERROR (Status)) {
+          DEBUG ((
+            DEBUG_WARN,
+            "%a: gDS->SetMemorySpaceAttributes() failed on region 0x%lx - 0x%lx (%r)\n",
+            __func__,
+            CurBase,
+            CurBase + CurSize - 1,
+            Status
+            ));
+        }
+
+        Attributes = EFI_MEMORY_WB;
+        if ((PcdGet64 (PcdDxeNxMemoryProtectionPolicy) &
+              (1U << (UINT32)EfiConventionalMemory)) != 0)
+        {
+          Attributes |= EFI_MEMORY_XP;
+        }
+
+        Status = Cpu->SetMemoryAttributes (Cpu, CurBase, CurSize, Attributes);
+
+        if (EFI_ERROR (Status)) {
+          DEBUG ((
+            DEBUG_ERROR,
+            "%a: Failed to set System RAM @ 0x%lx - 0x%lx attribute (%r)\n",
+            __func__,
+            CurBase,
+            CurBase + CurSize - 1,
+            Status
+            ));
+        } else {
+          DEBUG ((
+            DEBUG_INFO,
+            "%a: Add System RAM @ 0x%lx - 0x%lx\n",
+            __func__,
+            CurBase,
+            CurBase + CurSize - 1
+            ));
+        }
+      }
+    }
+  }
+
+  return EFI_SUCCESS;
+}
-- 
2.34.1



-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.
View/Reply Online (#115636): https://edk2.groups.io/g/devel/message/115636
Mute This Topic: https://groups.io/mt/104463710/7686176
Group Owner: devel+owner@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub [rebecca@openfw.io]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [edk2-devel] [PATCH v3 1/1] SbsaQemu: add memory space for the high memory nodes

[Marcin Juszkiewicz]

W dniu 20.02.2024 o 8:33 AM, Xiong Yining pisze:
> To support more memory nodes, we refer to the implement of
> "OvmfPkg/Fdt/HighMemDxe" to add memory space for the high memory nodes
> except the first one.
> 
> Signed-off-by: Xiong Yining<xiongyining1480@phytium.com.cn>
> Signed-off-by: Chen Baozi<chenbaozi@phytium.com.cn>

Tested-by: Marcin Juszkiewicz <marcin.juszkiewicz@linaro.org>

EDK2 reported memory from both NUMA nodes, Linux got whole memory too 
(with SRAT table patch).



-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.
View/Reply Online (#115725): https://edk2.groups.io/g/devel/message/115725
Mute This Topic: https://groups.io/mt/104463710/7686176
Group Owner: devel+owner@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub [rebecca@openfw.io]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [edk2-devel] [PATCH v3 1/1] SbsaQemu: add memory space for the high memory nodes

[Marcin Juszkiewicz]

W dniu 20.02.2024 o 08:33, Xiong Yining pisze:
> To support more memory nodes, we refer to the implement of
> "OvmfPkg/Fdt/HighMemDxe" to add memory space for the high memory nodes
> except the first one.
> 
> Signed-off-by: Xiong Yining<xiongyining1480@phytium.com.cn>
> Signed-off-by: Chen Baozi<chenbaozi@phytium.com.cn>

Needs rebase on top of "get rid of DeviceTree from SbsaQemu" series.

Rename SbsaQemuHardwareInfoLib to HardwareInfoLib and use newer function 
names.


-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.
View/Reply Online (#117120): https://edk2.groups.io/g/devel/message/117120
Mute This Topic: https://groups.io/mt/104463710/7686176
Group Owner: devel+owner@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub [rebecca@openfw.io]
-=-=-=-=-=-=-=-=-=-=-=-