On Tue, Oct 31, 2023 at 05:55 PM, Dhaval Sharma wrote:

I am posting an update on behalf of Jingyu as he had trouble with posting. CC'ing him here:
In summary what we have verified so far:

I have verified that instructions/op codes are okay. I have also verified on Qemu that functionally it seems to be calling correct instructions. Ensured with negative test cases that any other op codes do cause exceptions as expected.

Jingyu was able to verify the CpuFlushCpuDataCache function with this framework (he had to use custom op code based on his soc implementation) on SG2042. There is one issue that he is debugging now which is related to other cache instructions and he will get back with more data. P.S. SG2042 does not implement the exact same CMO opcodes but equivalent ones. So this experiment is just an additional data point that helps verify the framework and not CMO itself.

In general it sounds like framework flows are alright and as long as instructions do their job as claimed in the spec, it is lower risk.

Guess this is what we have so far. If it makes sense to everyone, we could go ahead with merging with this *feature disabled by default* after Jingyu provides clarity reg failures on SG2042 platform. Otherwise we can wait until newer Si is available where these exact instructions can be tested and then upstreamed.

[From Jingyu]

I verified this CMO framework on an actual HW platform.

SW:
edk2: https://github.com/rivosinc/edk2/tree/dev-rv-cmo-v7 branch: dev-rv-cmo-v7
edk2-platforms: https://github.com/sophgo/edk2-platforms branch: sg2042-dev

HW:
Milk-V Pioneer Box, a developer motherboard based on SG2042 with 64-Core T-HEAD C920.

Attention:
The T-HEAD C920 implemented its own CMO Extension and is different from the standard CMO Extension.

Test steps:
1. Modified the opcodes in RiscVasm.inc to accommodate the C920 CMO feature.

Update the test status.
The InvalidateInstructionCacheRange and the InvalidateDataCacheRange execute the same instruction "cbo.inval", but the T-HEAD C920 executes different instructions for the two functions. Please see the form below for details.
I replaced ".long 0x02a5000b" with "fence.i", which solved unexpected exceptions that occurred yesterday.

	RISC-V standard CMO Extension	C920 CMO Extension
WriteBackInvalidateDataCache
WriteBackInvalidateDataCacheRange	cbo.flush	".long 0x02b5000b"
WriteBackDataCache
WriteBackDataCacheRange	cbo.clean	".long 0x02b5000b"
InvalidateDataCache	fence	fence
InvalidateDataCacheRange	cbo.inval	".long 0x02a5000b"
InvalidateInstructionCache	fence.i	fence.i
InvalidateInstructionCacheRange	cbo.inval	fence.i

Now, I enable CMO in the entire edk2 phase, not just during PCIe inbound. No exceptions found so far. Hope to give you some reference.

Update the patches:

diff --git a/MdePkg/Include/RiscV64/RiscVasm.inc b/MdePkg/Include/RiscV64/RiscVasm.inc

index 29de735885..c2e573eb3d 100644

--- a/MdePkg/Include/RiscV64/RiscVasm.inc

+++ b/MdePkg/Include/RiscV64/RiscVasm.inc

@@ -7,13 +7,13 @@

.macro RISCVCMOFLUSH

- .word 0x25200f

+ .long 0x02b5000b^M

.endm

.macro RISCVCMOINVALIDATE

- .word 0x05200f

+ .long 0x02a5000b^M

.endm

.macro RISCVCMOCLEAN

- .word 0x15200f

+ .long 0x02b5000b^M

.endm

diff --git a/MdePkg/Library/BaseCacheMaintenanceLib/RiscVCache.c b/MdePkg/Library/BaseCacheMaintenanceLib/RiscVCache.c

index 5b3104afb6..ee85d0548c 100644

--- a/MdePkg/Library/BaseCacheMaintenanceLib/RiscVCache.c

+++ b/MdePkg/Library/BaseCacheMaintenanceLib/RiscVCache.c

@@ -89,7 +89,7 @@ CacheOpCacheRange (

Start &= ~((UINTN)CacheLineSize - 1);

DEBUG (

- (DEBUG_INFO,

+ (DEBUG_VERBOSE,^M

"CacheOpCacheRange:\

Performing Cache Management Operation %d \n", Op)

);

@@ -163,7 +163,8 @@ InvalidateInstructionCacheRange (

)

{

if (RiscVIsCMOEnabled ()) {

- CacheOpCacheRange (Address, Length, Invld);

+ // CacheOpCacheRange (Address, Length, Invld);^M

+ InvalidateInstructionCache ();^M

} else {

DEBUG (

(DEBUG_VERBOSE,

diff --git a/UefiCpuPkg/CpuDxeRiscV64/CpuDxe.c b/UefiCpuPkg/CpuDxeRiscV64/CpuDxe.c

index 2af3b62234..824667bc87 100644

--- a/UefiCpuPkg/CpuDxeRiscV64/CpuDxe.c

+++ b/UefiCpuPkg/CpuDxeRiscV64/CpuDxe.c

@@ -9,6 +9,7 @@

**/

#include "CpuDxe.h"

+#include <Library/CacheMaintenanceLib.h>^M

// Global Variables

@@ -59,7 +60,7 @@ EFI_CPU_ARCH_PROTOCOL gCpu = {

CpuGetTimerValue,

CpuSetMemoryAttributes,

1, // NumberOfTimers

- 4 // DmaBufferAlignment

+ 64 // DmaBufferAlignment^M

};

@@ -90,6 +91,20 @@ CpuFlushCpuDataCache (

IN EFI_CPU_FLUSH_TYPE FlushType

)

{

+ switch (FlushType) {^M

+ case EfiCpuFlushTypeWriteBack:^M

+ WriteBackDataCacheRange ((VOID *)(UINTN)Start, (UINTN)Length);^M

+ break;^M

+ case EfiCpuFlushTypeInvalidate:^M

+ InvalidateDataCacheRange ((VOID *)(UINTN)Start, (UINTN)Length);^M

+ break;^M

+ case EfiCpuFlushTypeWriteBackInvalidate:^M

+ WriteBackInvalidateDataCacheRange ((VOID *)(UINTN)Start, (UINTN)Length);^M

+ break;^M

+ default:^M

+ return EFI_INVALID_PARAMETER;^M

+ }^M

+^M

return EFI_SUCCESS;

}

diff --git a/Platform/Sophgo/SG2042_EVB_Board/SG2042.fdf b/Platform/Sophgo/SG2042_EVB_Board/SG2042.fdf

index 844fc3eac0..9cbb1d3f65 100644

--- a/Platform/Sophgo/SG2042_EVB_Board/SG2042.fdf

+++ b/Platform/Sophgo/SG2042_EVB_Board/SG2042.fdf

@@ -77,7 +77,7 @@ INF Silicon/Sophgo/SG2042Pkg/Drivers/SdHostDxe/SdHostDxe.inf

# RISC-V Core Drivers

INF UefiCpuPkg/CpuTimerDxeRiscV64/CpuTimerDxeRiscV64.inf

-INF Silicon/Sophgo/SG2042Pkg/Override/UefiCpuPkg/CpuDxeRiscV64/CpuDxeRiscV64.inf

+INF UefiCpuPkg/CpuDxeRiscV64/CpuDxeRiscV64.inf

INF MdeModulePkg/Universal/FaultTolerantWriteDxe/FaultTolerantWriteDxe.inf

INF MdeModulePkg/Universal/Variable/RuntimeDxe/VariableRuntimeDxe.inf

diff --git a/Platform/Sophgo/SG2042_EVB_Board/SG2042.dsc b/Platform/Sophgo/SG2042_EVB_Board/SG2042.dsc

index 51ff89678c..bc3f96e21b 100644

--- a/Platform/Sophgo/SG2042_EVB_Board/SG2042.dsc

+++ b/Platform/Sophgo/SG2042_EVB_Board/SG2042.dsc

@@ -389,6 +389,7 @@

[PcdsPatchableInModule]

gSophgoSG2042PlatformPkgTokenSpaceGuid.PcdSG2042PhyAddrToVirAddr|0

+ gEfiMdePkgTokenSpaceGuid.PcdRiscVFeatureOverride|0x1

################################################################################

@@ -500,7 +501,7 @@

# RISC-V Core module

UefiCpuPkg/CpuTimerDxeRiscV64/CpuTimerDxeRiscV64.inf

- Silicon/Sophgo/SG2042Pkg/Override/UefiCpuPkg/CpuDxeRiscV64/CpuDxeRiscV64.inf

+ UefiCpuPkg/CpuDxeRiscV64/CpuDxeRiscV64.inf

MdeModulePkg/Universal/ResetSystemRuntimeDxe/ResetSystemRuntimeDxe.inf

MdeModulePkg/Universal/FaultTolerantWriteDxe/FaultTolerantWriteDxe.inf

Thanks,
Jingyu

_._,_._,_

Groups.io Links:

You receive all messages sent to this group.

_._,_._,_