From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail02.groups.io (mail02.groups.io [66.175.222.108]) by spool.mail.gandi.net (Postfix) with ESMTPS id EE69D7803EB for ; Tue, 31 Oct 2023 15:36:38 +0000 (UTC) DKIM-Signature: a=rsa-sha256; bh=lSQcDihFy9L7tBucXSAM4oKmJx9k9PFUmphsTcsZzB4=; c=relaxed/simple; d=groups.io; h=Subject:To:From:User-Agent:MIME-Version:Date:References:In-Reply-To:Message-ID:Precedence:List-Subscribe:List-Help:Sender:List-Id:Mailing-List:Delivered-To:Reply-To:List-Unsubscribe-Post:List-Unsubscribe:Content-Type; s=20140610; t=1698766597; v=1; b=gOBnPvZM2TfapAst6fVJWpgssuriBdKfDq1QJ9fnkjooqr25FNXgD7kzcVy2G/E2mov8xNXE SDnZ3rWvtQ2/gQ+w3zdb/PuWVvi/aMilhDBs8GIwcB+b+uHQGyNCSuk56oYYj5rL2hi+mRein8N vp2VZHy3gCcqLDqJQE7fhe/I= X-Received: by 127.0.0.2 with SMTP id I7PHYY7687511x2tq2HFoL0c; Tue, 31 Oct 2023 08:36:37 -0700 Subject: Re: [edk2-devel] [PATCH v7 3/5] MdePkg: Implement RISC-V Cache Management Operations To: Dhaval Sharma ,devel@edk2.groups.io From: "Jingyu Li via groups.io" X-Originating-Location: Central, Central and Western District, HK (103.68.183.115) X-Originating-Platform: Windows Chrome 118 User-Agent: GROUPS.IO Web Poster MIME-Version: 1.0 Date: Mon, 30 Oct 2023 23:45:41 -0700 References: <20231029144613.150580-4-dhaval@rivosinc.com> In-Reply-To: <20231029144613.150580-4-dhaval@rivosinc.com> Message-ID: <22939.1698734741183117459@groups.io> Precedence: Bulk List-Subscribe: List-Help: Sender: devel@edk2.groups.io List-Id: Mailing-List: list devel@edk2.groups.io; contact devel+owner@edk2.groups.io Reply-To: devel@edk2.groups.io,jingyu.li01@sophgo.com List-Unsubscribe-Post: List-Unsubscribe=One-Click List-Unsubscribe: X-Gm-Message-State: ECB3c5U0IiuxjcVkFpV9airlx7686176AA= Content-Type: multipart/alternative; boundary="0m67kCXcdFX2Zkt563J1" X-GND-Status: LEGIT Authentication-Results: spool.mail.gandi.net; dkim=pass header.d=groups.io header.s=20140610 header.b=gOBnPvZM; dmarc=none; spf=pass (spool.mail.gandi.net: domain of bounce@groups.io designates 66.175.222.108 as permitted sender) smtp.mailfrom=bounce@groups.io --0m67kCXcdFX2Zkt563J1 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable >=20 > Implement Cache Management Operations (CMO) defined by > RISC-V spec https://github.com/riscv/riscv-CMOs. >=20 > Notes: > 1. CMO only supports block based Operations. Meaning cache > flush/invd/clean Operations are not available for the entire > range. In that case we fallback on fence.i instructions. > 2. Operations are implemented using Opcodes to make them compiler > independent. binutils 2.39+ compilers support CMO instructions. >=20 > Test: > 1. Ensured correct instructions are refelecting in asm > 2. Not able to verify actual instruction in HW as Qemu ignores > any actual cache operations. >=20 > Cc: Michael D Kinney > Cc: Liming Gao > Cc: Zhiguang Liu > Cc: Sunil V L > Cc: Daniel Schaefer > Cc: Laszlo Ersek >=20 > Signed-off-by: Dhaval Sharma > Reviewed-by: Laszlo Ersek > --- >=20 I verified this CMO framework on an actual HW platform. SW: edk2: https://github.com/rivosinc/edk2/tree/dev-rv-cmo-v7 branch: dev-rv-cm= o-v7 edk2-platforms: https://github.com/sophgo/edk2-platforms branch: sg2042-dev HW: Milk-V Pioneer Box, a developer motherboard based on SG2042 with 64-Core T-= HEAD C920. Attention: The T-HEAD C920 implemented its own CMO Extension and is different from the= standard CMO Extension. Test steps: 1. Modified the opcodes in RiscVasm.inc to accommodate the C920 CMO feature= . diff --git a/MdePkg/Include/RiscV64/RiscVasm.inc b/MdePkg/Include/RiscV64/R= iscVasm.inc index 29de735885..5df85fdb31 100644 --- a/MdePkg/Include/RiscV64/RiscVasm.inc +++ b/MdePkg/Include/RiscV64/RiscVasm.inc @@ -7,13 +7,13 @@ */ .macro RISCVCMOFLUSH -=C2=A0 =C2=A0 .word 0x25200f +=C2=A0 =C2=A0 .long 0x0275000b^M .endm .macro RISCVCMOINVALIDATE -=C2=A0 =C2=A0 .word 0x05200f +=C2=A0 =C2=A0 .long 0x0265000b^M .endm .macro RISCVCMOCLEAN -=C2=A0 =C2=A0 .word 0x15200f +=C2=A0 =C2=A0 .long 0x0275000b^M .endm 2. We enable the CMO during the PCIe devices with DMA access to the memory,= just focus on the implementation of CpuFlushCpuDataCache based on the EFI_= CPU_ARCH_PROTOCOL.=C2=A0Except for PCIe, in other words, except for the cpu= ->FlushDataCache, we do not use CMO. And the PCIe inbound only relates to d= atacache.clean and datacache.invalidate. diff --git a/UefiCpuPkg/CpuDxeRiscV64/CpuDxe.c b/UefiCpuPkg/CpuDxeRiscV64/C= puDxe.c index 2af3b62234..cf50bc5f92 100644 --- a/UefiCpuPkg/CpuDxeRiscV64/CpuDxe.c +++ b/UefiCpuPkg/CpuDxeRiscV64/CpuDxe.c @@ -9,6 +9,8 @@ **/ #include "CpuDxe.h" +#include ^M +#include ^M // // Global Variables @@ -59,7 +61,7 @@ EFI_CPU_ARCH_PROTOCOL=C2=A0 gCpu =3D { CpuGetTimerValue, CpuSetMemoryAttributes, 1,=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 // NumberOfTimers -=C2=A0 4=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0// DmaBufferAlignment +=C2=A0 64=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0// DmaBufferAlignment^M }; // @@ -90,6 +92,21 @@ CpuFlushCpuDataCache ( IN EFI_CPU_FLUSH_TYPE=C2=A0 =C2=A0 =C2=A0FlushType ) { +=C2=A0 PatchPcdSet64 (PcdRiscVFeatureOverride, 0x1);^M +=C2=A0 switch (FlushType) {^M +=C2=A0 =C2=A0 case EfiCpuFlushTypeWriteBack:^M +=C2=A0 =C2=A0 =C2=A0 WriteBackDataCacheRange ((VOID *)(UINTN)Start, (UINTN= )Length);^M +=C2=A0 =C2=A0 =C2=A0 break;^M +=C2=A0 =C2=A0 case EfiCpuFlushTypeInvalidate:^M +=C2=A0 =C2=A0 =C2=A0 InvalidateInstructionCacheRange ((VOID *)(UINTN)Start= , (UINTN)Length);^M +=C2=A0 =C2=A0 =C2=A0 break;^M +=C2=A0 =C2=A0 case EfiCpuFlushTypeWriteBackInvalidate:^M +=C2=A0 =C2=A0 =C2=A0 WriteBackInvalidateDataCacheRange ((VOID *)(UINTN)Sta= rt, (UINTN)Length);^M +=C2=A0 =C2=A0 =C2=A0 break;^M +=C2=A0 =C2=A0 default:^M +=C2=A0 =C2=A0 =C2=A0 return EFI_INVALID_PARAMETER;^M +=C2=A0 }^M +^M return EFI_SUCCESS; } diff --git a/Platform/Sophgo/SG2042_EVB_Board/SG2042.dsc b/Platform/Sophgo/= SG2042_EVB_Board/SG2042.dsc index 51ff89678c..e2e44ad619 100644 --- a/Platform/Sophgo/SG2042_EVB_Board/SG2042.dsc +++ b/Platform/Sophgo/SG2042_EVB_Board/SG2042.dsc @@ -389,6 +389,7 @@ [PcdsPatchableInModule] gSophgoSG2042PlatformPkgTokenSpaceGuid.PcdSG2042PhyAddrToVirAddr|0 +=C2=A0 gEfiMdePkgTokenSpaceGuid.PcdRiscVFeatureOverride|0 ###########################################################################= ##### # @@ -500,7 +501,7 @@ # RISC-V Core module # UefiCpuPkg/CpuTimerDxeRiscV64/CpuTimerDxeRiscV64.inf -=C2=A0 Silicon/Sophgo/SG2042Pkg/Override/UefiCpuPkg/CpuDxeRiscV64/CpuDxeRi= scV64.inf +=C2=A0 UefiCpuPkg/CpuDxeRiscV64/CpuDxeRiscV64.inf MdeModulePkg/Universal/ResetSystemRuntimeDxe/ResetSystemRuntimeDxe.inf MdeModulePkg/Universal/FaultTolerantWriteDxe/FaultTolerantWriteDxe.inf diff --git a/Platform/Sophgo/SG2042_EVB_Board/SG2042.fdf b/Platform/Sophgo/= SG2042_EVB_Board/SG2042.fdf index 844fc3eac0..9cbb1d3f65 100644 --- a/Platform/Sophgo/SG2042_EVB_Board/SG2042.fdf +++ b/Platform/Sophgo/SG2042_EVB_Board/SG2042.fdf @@ -77,7 +77,7 @@ INF=C2=A0 Silicon/Sophgo/SG2042Pkg/Drivers/SdHostDxe/SdHo= stDxe.inf # RISC-V Core Drivers INF=C2=A0 UefiCpuPkg/CpuTimerDxeRiscV64/CpuTimerDxeRiscV64.inf -INF=C2=A0 Silicon/Sophgo/SG2042Pkg/Override/UefiCpuPkg/CpuDxeRiscV64/CpuDx= eRiscV64.inf +INF=C2=A0 UefiCpuPkg/CpuDxeRiscV64/CpuDxeRiscV64.inf INF=C2=A0 MdeModulePkg/Universal/FaultTolerantWriteDxe/FaultTolerantWriteDx= e.inf INF=C2=A0 MdeModulePkg/Universal/Variable/RuntimeDxe/VariableRuntimeDxe.inf 3. Now the PCIe devices are in work order on PioneerBox. The=C2=A0CMO instr= uctions are executed as expected. Reviewed-by: Jingyu Li Thanks, Jingyu -=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D- Groups.io Links: You receive all messages sent to this group. View/Reply Online (#110428): https://edk2.groups.io/g/devel/message/110428 Mute This Topic: https://groups.io/mt/102256466/7686176 Group Owner: devel+owner@edk2.groups.io Unsubscribe: https://edk2.groups.io/g/devel/unsub [rebecca@openfw.io] -=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D- --0m67kCXcdFX2Zkt563J1 Content-Type: text/html; charset="utf-8" Content-Transfer-Encoding: quoted-printable
Implement Cache= Management Operations (CMO) defined by
RISC-V s= pec https://github.com/riscv/riscv-CMOs.

Notes:
1. CMO only s= upports block based Operations. Meaning cache
f= lush/invd/clean Operations are not available for the entire
range. In that case we fallback on fence.i instructions.
2. Operations are implemented using Opcodes to make t= hem compiler
independent. binutils 2.39+ compil= ers support CMO instructions.

Test:
1. Ensured c= orrect instructions are refelecting in asm
2. No= t able to verify actual instruction in HW as Qemu ignores
any actual cache operations.

Cc: Michael D Kinney <michael.= d.kinney@...>
Cc: Liming Gao <gaoliming@..= .>
Cc: Zhiguang Liu <zhiguang.liu@...><= /span>
Cc: Sunil V L <sunilvl@...>
Cc: Daniel Schaefer <git@...>
Cc: Laszlo Ersek <lersek@...>

Signed-off-by: Dhaval Sharma <dhava= l@...>
Reviewed-by: Laszlo Ersek <lersek@.= ..>
---
I verified this CMO framework on an actual HW platform.

SW:
edk2: https://github.com/rivosin= c/edk2/tree/dev-rv-cmo-v7 branch: dev-rv-cmo-v7
edk2-platfo= rms: https://github.com/sophgo/edk2-platforms  b= ranch: sg2042-dev

HW:
Milk-V Pioneer Box, a developer moth= erboard based on SG2042 with 64-Core T-HEAD C920.

Attention:
The T-HEAD C920 implemented its own CMO Extension and = is different from the standard CMO Extension.

Test steps:
1. Modified the opcodes in RiscVasm.inc to accommodate the C920 CMO feature= .
diff --git a/MdePkg/Include/RiscV64/RiscVasm.inc b/MdePkg/Include/Risc= V64/RiscVasm.inc
index 29de735885..5df85fdb31 100644
--- a/MdePkg/Include/RiscV64/RiscVasm.inc
+++ b/MdePkg/Include/RiscV64/RiscVasm.inc
@@ -7,13 +7,13 @@
  */
 
 .macro RISCVCMOFLUSH
-    .word 0x25200f
+    .long 0x0275000b^M
 .endm
 
 .macro RISCVCMOINVALIDATE
-    .word 0x05200f
+    .long 0x0265000b^M
 .endm
 
 .macro RISCVCMOCLEAN
-    .word 0x15200f
+    .long 0x0275000b^M
 .endm
 
2. We enable the CMO during the PCIe devices with DMA access to the memory,= just focus on the implementation of CpuFlushCpuDataCache based on the EFI_= CPU_ARCH_PROTOCOL. Except for PCIe, in other words, except for the cpu= ->FlushDataCache, we do not use CMO. And the PCIe inbound only relates t= o datacache.clean and datacache.invalidate.
diff --git a/UefiCpuPkg/CpuDxeRiscV64/CpuDxe.c b/UefiCpuPkg/CpuDxeRisc= V64/CpuDxe.c
index 2af3b62234..cf50bc5f92 100644
--- a/UefiCpuPkg/CpuDxeRiscV64/CpuDxe.c
+++ b/UefiCpuPkg/CpuDxeRiscV64/CpuDxe.c
@@ -9,6 +9,8 @@
 **/
 
 #include "CpuDxe.h"
+#include <Library/CacheMaintenanceLib.h>^M
+#include <Library/PcdLib.h>^M
 
 //
 // Global Variables
@@ -59,7 +61,7 @@ EFI_CPU_ARCH_PROTOCOL  gCpu =3D {
   CpuGetTimerValue,
   CpuSetMemoryAttributes,
   1,               =           // NumberOfTimers
-  4                 = ;          // DmaBufferAlignment
+  64                &nbs= p;          // DmaBufferAlignment^M
 };
 
 //
@@ -90,6 +92,21 @@ CpuFlushCpuDataCache (
   IN EFI_CPU_FLUSH_TYPE     FlushType
   )
 {
+  PatchPcdSet64 (PcdRiscVFeatureOverride, 0x1);^M
+  switch (FlushType) {^M
+    case EfiCpuFlushTypeWriteBack:^M
+      WriteBackDataCacheRange ((VOID *)(UINTN)Start, (= UINTN)Length);^M
+      break;^M
+    case EfiCpuFlushTypeInvalidate:^M
+      InvalidateInstructionCacheRange ((VOID *)(UINTN)= Start, (UINTN)Length);^M
+      break;^M
+    case EfiCpuFlushTypeWriteBackInvalidate:^M
+      WriteBackInvalidateDataCacheRange ((VOID *)(UINT= N)Start, (UINTN)Length);^M
+      break;^M
+    default:^M
+      return EFI_INVALID_PARAMETER;^M
+  }^M
+^M
   return EFI_SUCCESS;
 }

diff --git a/Platform/Sophgo/SG2042_EVB_Board/SG2042.dsc b/Platform/So= phgo/SG2042_EVB_Board/SG2042.dsc
index 51ff89678c..e2e44ad619 100644
--- a/Platform/Sophgo/SG2042_EVB_Board/SG2042.dsc
+++ b/Platform/Sophgo/SG2042_EVB_Board/SG2042.dsc
@@ -389,6 +389,7 @@
 
 [PcdsPatchableInModule]
   gSophgoSG2042PlatformPkgTokenSpaceGuid.PcdSG2042PhyAddrTo= VirAddr|0
+  gEfiMdePkgTokenSpaceGuid.PcdRiscVFeatureOverride|0
 
 ################################################################= ################
 #
@@ -500,7 +501,7 @@
   # RISC-V Core module
   #
   UefiCpuPkg/CpuTimerDxeRiscV64/CpuTimerDxeRiscV64.inf
-  Silicon/Sophgo/SG2042Pkg/Override/UefiCpuPkg/CpuDxeRiscV64/Cpu= DxeRiscV64.inf
+  UefiCpuPkg/CpuDxeRiscV64/CpuDxeRiscV64.inf
   MdeModulePkg/Universal/ResetSystemRuntimeDxe/ResetSystemR= untimeDxe.inf
 
   MdeModulePkg/Universal/FaultTolerantWriteDxe/FaultToleran= tWriteDxe.inf

diff --git a/Platform/Sophgo/SG2042_EVB_Board/SG2042.fdf b/Platform/So= phgo/SG2042_EVB_Board/SG2042.fdf
index 844fc3eac0..9cbb1d3f65 100644
--- a/Platform/Sophgo/SG2042_EVB_Board/SG2042.fdf
+++ b/Platform/Sophgo/SG2042_EVB_Board/SG2042.fdf
@@ -77,7 +77,7 @@ INF  Silicon/Sophgo/SG2042Pkg/Drivers/SdHostDxe= /SdHostDxe.inf
 
 # RISC-V Core Drivers
 INF  UefiCpuPkg/CpuTimerDxeRiscV64/CpuTimerDxeRiscV64.inf
-INF  Silicon/Sophgo/SG2042Pkg/Override/UefiCpuPkg/CpuDxeRiscV64/= CpuDxeRiscV64.inf
+INF  UefiCpuPkg/CpuDxeRiscV64/CpuDxeRiscV64.inf
 
 INF  MdeModulePkg/Universal/FaultTolerantWriteDxe/FaultTole= rantWriteDxe.inf
 INF  MdeModulePkg/Universal/Variable/RuntimeDxe/VariableRun= timeDxe.inf
 
3. Now the PCIe devices are in work order on PioneerBox. The CMO instr= uctions are executed as expected.

Reviewed-by: Jingyu Li <jingyu.li01@sophgo.com><= /span>

= Thanks,
Jingyu
_._,_._,_
Groups.io Links:

=20 You receive all messages sent to this group. =20 =20

View/Reply Online (#110428) | =20 | Mute= This Topic | New Topic
Your Subscriptio= n | Contact Group Owner | Unsubscribe [rebecca@openfw.io]

_._,_._,_
--0m67kCXcdFX2Zkt563J1--