> 
> Implement Cache Management Operations (CMO) defined by
> RISC-V spec https://github.com/riscv/riscv-CMOs.
> 
> Notes:
> 1. CMO only supports block based Operations. Meaning cache
> flush/invd/clean Operations are not available for the entire
> range. In that case we fallback on fence.i instructions.
> 2. Operations are implemented using Opcodes to make them compiler
> independent. binutils 2.39+ compilers support CMO instructions.
> 
> Test:
> 1. Ensured correct instructions are refelecting in asm
> 2. Not able to verify actual instruction in HW as Qemu ignores
> any actual cache operations.
> 
> Cc: Michael D Kinney <michael.d.kinney@...>
> Cc: Liming Gao <gaoliming@...>
> Cc: Zhiguang Liu <zhiguang.liu@...>
> Cc: Sunil V L <sunilvl@...>
> Cc: Daniel Schaefer <git@...>
> Cc: Laszlo Ersek <lersek@...>
> 
> Signed-off-by: Dhaval Sharma <dhaval@...>
> Reviewed-by: Laszlo Ersek <lersek@...>
> ---
> 

I verified this CMO framework on an actual HW platform.

SW:
edk2: https://github.com/rivosinc/edk2/tree/dev-rv-cmo-v7 branch: dev-rv-cmo-v7
edk2-platforms: https://github.com/sophgo/edk2-platforms branch: sg2042-dev

HW:
Milk-V Pioneer Box, a developer motherboard based on SG2042 with 64-Core T-HEAD C920.

Attention:
The T-HEAD C920 implemented its own CMO Extension and is different from the standard CMO Extension.

Test steps:
1. Modified the opcodes in RiscVasm.inc to accommodate the C920 CMO feature.
diff --git a/MdePkg/Include/RiscV64/RiscVasm.inc b/MdePkg/Include/RiscV64/RiscVasm.inc
index 29de735885..5df85fdb31 100644
--- a/MdePkg/Include/RiscV64/RiscVasm.inc
+++ b/MdePkg/Include/RiscV64/RiscVasm.inc
@@ -7,13 +7,13 @@
*/

.macro RISCVCMOFLUSH
-    .word 0x25200f
+    .long 0x0275000b^M
.endm

.macro RISCVCMOINVALIDATE
-    .word 0x05200f
+    .long 0x0265000b^M
.endm

.macro RISCVCMOCLEAN
-    .word 0x15200f
+    .long 0x0275000b^M
.endm

2. We enable the CMO during the PCIe devices with DMA access to the memory, just focus on the implementation of CpuFlushCpuDataCache based on the EFI_CPU_ARCH_PROTOCOL. Except for PCIe, in other words, except for the cpu->FlushDataCache, we do not use CMO. And the PCIe inbound only relates to datacache.clean and datacache.invalidate.
diff --git a/UefiCpuPkg/CpuDxeRiscV64/CpuDxe.c b/UefiCpuPkg/CpuDxeRiscV64/CpuDxe.c
index 2af3b62234..cf50bc5f92 100644
--- a/UefiCpuPkg/CpuDxeRiscV64/CpuDxe.c
+++ b/UefiCpuPkg/CpuDxeRiscV64/CpuDxe.c
@@ -9,6 +9,8 @@
**/

#include "CpuDxe.h"
+#include <Library/CacheMaintenanceLib.h>^M
+#include <Library/PcdLib.h>^M

//
// Global Variables
@@ -59,7 +61,7 @@ EFI_CPU_ARCH_PROTOCOL  gCpu = {
CpuGetTimerValue,
CpuSetMemoryAttributes,
1,                          // NumberOfTimers
-  4                           // DmaBufferAlignment
+  64                           // DmaBufferAlignment^M
};

//
@@ -90,6 +92,21 @@ CpuFlushCpuDataCache (
IN EFI_CPU_FLUSH_TYPE     FlushType
)
{
+  PatchPcdSet64 (PcdRiscVFeatureOverride, 0x1);^M
+  switch (FlushType) {^M
+    case EfiCpuFlushTypeWriteBack:^M
+      WriteBackDataCacheRange ((VOID *)(UINTN)Start, (UINTN)Length);^M
+      break;^M
+    case EfiCpuFlushTypeInvalidate:^M
+      InvalidateInstructionCacheRange ((VOID *)(UINTN)Start, (UINTN)Length);^M
+      break;^M
+    case EfiCpuFlushTypeWriteBackInvalidate:^M
+      WriteBackInvalidateDataCacheRange ((VOID *)(UINTN)Start, (UINTN)Length);^M
+      break;^M
+    default:^M
+      return EFI_INVALID_PARAMETER;^M
+  }^M
+^M
return EFI_SUCCESS;
}

diff --git a/Platform/Sophgo/SG2042_EVB_Board/SG2042.dsc b/Platform/Sophgo/SG2042_EVB_Board/SG2042.dsc
index 51ff89678c..e2e44ad619 100644
--- a/Platform/Sophgo/SG2042_EVB_Board/SG2042.dsc
+++ b/Platform/Sophgo/SG2042_EVB_Board/SG2042.dsc
@@ -389,6 +389,7 @@

[PcdsPatchableInModule]
gSophgoSG2042PlatformPkgTokenSpaceGuid.PcdSG2042PhyAddrToVirAddr|0
+  gEfiMdePkgTokenSpaceGuid.PcdRiscVFeatureOverride|0

################################################################################
#
@@ -500,7 +501,7 @@
# RISC-V Core module
#
UefiCpuPkg/CpuTimerDxeRiscV64/CpuTimerDxeRiscV64.inf
-  Silicon/Sophgo/SG2042Pkg/Override/UefiCpuPkg/CpuDxeRiscV64/CpuDxeRiscV64.inf
+  UefiCpuPkg/CpuDxeRiscV64/CpuDxeRiscV64.inf
MdeModulePkg/Universal/ResetSystemRuntimeDxe/ResetSystemRuntimeDxe.inf

MdeModulePkg/Universal/FaultTolerantWriteDxe/FaultTolerantWriteDxe.inf

diff --git a/Platform/Sophgo/SG2042_EVB_Board/SG2042.fdf b/Platform/Sophgo/SG2042_EVB_Board/SG2042.fdf
index 844fc3eac0..9cbb1d3f65 100644
--- a/Platform/Sophgo/SG2042_EVB_Board/SG2042.fdf
+++ b/Platform/Sophgo/SG2042_EVB_Board/SG2042.fdf
@@ -77,7 +77,7 @@ INF  Silicon/Sophgo/SG2042Pkg/Drivers/SdHostDxe/SdHostDxe.inf

# RISC-V Core Drivers
INF  UefiCpuPkg/CpuTimerDxeRiscV64/CpuTimerDxeRiscV64.inf
-INF  Silicon/Sophgo/SG2042Pkg/Override/UefiCpuPkg/CpuDxeRiscV64/CpuDxeRiscV64.inf
+INF  UefiCpuPkg/CpuDxeRiscV64/CpuDxeRiscV64.inf

INF  MdeModulePkg/Universal/FaultTolerantWriteDxe/FaultTolerantWriteDxe.inf
INF  MdeModulePkg/Universal/Variable/RuntimeDxe/VariableRuntimeDxe.inf

3. Now the PCIe devices are in work order on PioneerBox. The CMO instructions are executed as expected.
Reviewed-by: Jingyu Li <jingyu.li01@sophgo.com>

Thanks,
Jingyu


-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.
View/Reply Online (#110428): https://edk2.groups.io/g/devel/message/110428
Mute This Topic: https://groups.io/mt/102256466/7686176
Group Owner: devel+owner@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub [rebecca@openfw.io]
-=-=-=-=-=-=-=-=-=-=-=-