From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-pf1-f180.google.com (mail-pf1-f180.google.com [209.85.210.180]) by mx.groups.io with SMTP id smtpd.web10.4365.1689240819177824291 for ; Thu, 13 Jul 2023 02:33:39 -0700 Authentication-Results: mx.groups.io; dkim=fail reason="signature has expired" header.i=@rivosinc-com.20221208.gappssmtp.com header.s=20221208 header.b=UpdvLehP; spf=pass (domain: rivosinc.com, ip: 209.85.210.180, mailfrom: dhaval@rivosinc.com) Received: by mail-pf1-f180.google.com with SMTP id d2e1a72fcca58-666eef03ebdso246859b3a.1 for ; Thu, 13 Jul 2023 02:33:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rivosinc-com.20221208.gappssmtp.com; s=20221208; t=1689240818; x=1691832818; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=SJP68EfkbcjUHafF4nK+VOiGqsmFJi/76h5ezDv+zYk=; b=UpdvLehPjZQx0PlkiHi5Zm1DHrhVjIdxNF8d4ygysOVmhVKf0Jo7OY8QToNWa2hX5E yEumqQ+8CyGbfzLUEUQJhyT5ueRDNVFm7UJtC72AlUcO6F7j6LtjvT/+PREQKQztEK5G WG78zFAGoju7ubSqCdTnVEcQ1v3Xp47FAgl9fwLbpfibupEEHO+TEodgTJ/uggLwm91s Jj9bKRGTDuISsFTF0GC6tdfYB4b/vvfFfnD6nk4rGjNZeBcA7Aqre5KPRFz2peiJ9CbK jKAKPCQF/a/r/RWEkFlO6/HW7y+3+DBC6FWJcBiHf9z361EyO8sqvxqzBxpIbIrswG9z 416g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1689240818; x=1691832818; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=SJP68EfkbcjUHafF4nK+VOiGqsmFJi/76h5ezDv+zYk=; b=XJQqXAOZv2o1rfyaJP48HX78veb9N2kigLgH6hV+XqhULUA7W4ovHlAJBkpjOcWxON bNemQI/oRGHKRXRjN6fPsoSsJPEE4M5hX59/unh/EDeeO2oTBK2bPNpIxltqRLtkXg3P TXvq73K2pxRkVnK7mO0HbhHuiYEog0M+e+o5DHGy/90nMHtchd3D4u6OLqVDujXDq3iu 1vk4zYOmdWjHV5+N/qo85PMNzqwHBWhIUELNiGZXwpc7IjcOAav15BEk+yFgcZ6oJec/ ob2fI94GKjAgZBUwdShrMwSMhp0wNZUXGxNzk4AjlrK1BS7fYcesVt8KzlArGNfnbeN+ x9PA== X-Gm-Message-State: ABy/qLYzADX6scqAxRBONpV955IxK0ZyEeYGvUrXKdc69gRYjxYxpHJk bxeLxafhJEFZyoyCsVAisdCAc7aKJvavI85F7rs= X-Google-Smtp-Source: APBJJlHXK8jlVTIDn1tdKCEwgI89CiSdCiJSAjGJ0tA62qnQL0Xn+e3cP2+pqnS7T4Mf64xrMczZzg== X-Received: by 2002:a05:6a20:4406:b0:131:b3fa:eaaa with SMTP id ce6-20020a056a20440600b00131b3faeaaamr453363pzb.61.1689240818388; Thu, 13 Jul 2023 02:33:38 -0700 (PDT) Return-Path: Received: from dhaval.blr.rivosinc.com ([49.249.129.34]) by smtp.gmail.com with ESMTPSA id d11-20020a170902b70b00b001b898595be7sm5431178pls.291.2023.07.13.02.33.36 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 13 Jul 2023 02:33:38 -0700 (PDT) From: "Dhaval Sharma" To: devel@edk2.groups.io Cc: Ard Biesheuvel , Jiewen Yao , Jordan Justen , Gerd Hoffmann , Sunil V L , Andrei Warkentin Subject: [PATCH v4 1/1] MdePkg:Implement RISCV CMO Date: Thu, 13 Jul 2023 15:03:31 +0530 Message-Id: <20230713093331.267164-2-dhaval@rivosinc.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230713093331.267164-1-dhaval@rivosinc.com> References: <20230713093331.267164-1-dhaval@rivosinc.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable From: Dhaval Sharma Implementing code to support Cache Management Operations (CMO) defined by RV spec https://github.com/riscv/riscv-CMOs Notes: 1. CMO only supports block based Operations. Meaning complete cache flush/invd/clean Operations are not available. In that case we fallback on fence.i instructions. 2. Rely on the fact that platform init has initialized CMO and this implementation just checks if it is enabled. 3. In order to avoid compiler dependency injecting byte code. Test: 1. Ensured correct instructions are refelecting in asm 2. Able to boot platform with RiscVVirtQemu config 3. Not able to verify actual instruction in HW as Qemu ignores any actual cache operations. Cc: Ard Biesheuvel Cc: Jiewen Yao Cc: Jordan Justen Cc: Gerd Hoffmann Cc: Sunil V L Cc: Andrei Warkentin Signed-off-by: Dhaval Sharma --- Notes: v4: - Removed CMO specific directory in Base Lib - Implemented compiler independent code for CMO - Merged CMO implementation with fence.i - Added logic to confirm CMO is enabled MdePkg/Library/BaseLib/BaseLib.inf | 2 +- MdePkg/Include/Register/RiscV64/RiscVEncoding.h | 4 + MdePkg/Library/BaseCacheMaintenanceLib/RiscVCache.c | 200 ++++++++++++++++= ++-- MdePkg/Library/BaseLib/RiscV64/FlushCache.S | 21 -- MdePkg/Library/BaseLib/RiscV64/RiscVCacheMgmt.S | 64 +++++++ 5 files changed, 254 insertions(+), 37 deletions(-) diff --git a/MdePkg/Library/BaseLib/BaseLib.inf b/MdePkg/Library/BaseLib/Ba= seLib.inf index 03c7b02e828b..53389389448c 100644 --- a/MdePkg/Library/BaseLib/BaseLib.inf +++ b/MdePkg/Library/BaseLib/BaseLib.inf @@ -400,7 +400,7 @@ [Sources.RISCV64] RiscV64/RiscVCpuBreakpoint.S | GCC=0D RiscV64/RiscVCpuPause.S | GCC=0D RiscV64/RiscVInterrupt.S | GCC=0D - RiscV64/FlushCache.S | GCC=0D + RiscV64/RiscVCacheMgmt.S | GCC=0D RiscV64/CpuScratch.S | GCC=0D RiscV64/ReadTimer.S | GCC=0D RiscV64/RiscVMmu.S | GCC=0D diff --git a/MdePkg/Include/Register/RiscV64/RiscVEncoding.h b/MdePkg/Inclu= de/Register/RiscV64/RiscVEncoding.h index 5c2989b797bf..ea1493578bd5 100644 --- a/MdePkg/Include/Register/RiscV64/RiscVEncoding.h +++ b/MdePkg/Include/Register/RiscV64/RiscVEncoding.h @@ -85,6 +85,10 @@ /* Supervisor Configuration */=0D #define CSR_SENVCFG 0x10a=0D =0D +/* Defined CBO bits*/=0D +#define SENVCFG_CBCFE 0x40UL=0D +#define SENVCFG_CBIE 0x30UL=0D +=0D /* Supervisor Trap Handling */=0D #define CSR_SSCRATCH 0x140=0D #define CSR_SEPC 0x141=0D diff --git a/MdePkg/Library/BaseCacheMaintenanceLib/RiscVCache.c b/MdePkg/L= ibrary/BaseCacheMaintenanceLib/RiscVCache.c index d08fb9f193ca..8b853e5b69fa 100644 --- a/MdePkg/Library/BaseCacheMaintenanceLib/RiscVCache.c +++ b/MdePkg/Library/BaseCacheMaintenanceLib/RiscVCache.c @@ -1,7 +1,8 @@ /** @file=0D - RISC-V specific functionality for cache.=0D + Implement Risc-V Cache Management Operations=0D =0D Copyright (c) 2020, Hewlett Packard Enterprise Development LP. All right= s reserved.
=0D + Copyright (c) 2023, Rivos Inc. All rights reserved.
=0D =0D SPDX-License-Identifier: BSD-2-Clause-Patent=0D **/=0D @@ -10,13 +11,21 @@ #include =0D #include =0D =0D +#define RV64_CACHE_BLOCK_SIZE 64=0D +=0D +typedef enum {=0D + Clean,=0D + Flush,=0D + Invld,=0D +} CACHE_OP;=0D +=0D /**=0D RISC-V invalidate instruction cache.=0D =0D **/=0D VOID=0D EFIAPI=0D -RiscVInvalidateInstCacheAsm (=0D +RiscVInvalidateInstCacheAsm_Fence (=0D VOID=0D );=0D =0D @@ -26,13 +35,144 @@ RiscVInvalidateInstCacheAsm ( **/=0D VOID=0D EFIAPI=0D -RiscVInvalidateDataCacheAsm (=0D +RiscVInvalidateDataCacheAsm_Fence (=0D VOID=0D );=0D =0D +/**=0D + RISC-V flush cache block. Atomically perform a clean operation=0D + followed by an invalidate operation=0D +=0D +**/=0D +VOID=0D +EFIAPI=0D +RiscVCpuCacheFlushAsm_Cbo (=0D + UINTN=0D + );=0D +=0D +/**=0D +Perform a write transfer to another cache or to memory if the=0D +data in the copy of the cache block have been modified by a store=0D +operation=0D +=0D +**/=0D +VOID=0D +EFIAPI=0D +RiscVCpuCacheCleanAsm_Cbo (=0D + UINTN=0D + );=0D +=0D +/**=0D +Deallocate the copy of the cache block=0D +=0D +**/=0D +VOID=0D +EFIAPI=0D +RiscVCpuCacheInvalAsm_Cbo (=0D + UINTN=0D + );=0D +=0D +/**=0D +Verify CBOs are supported by this HW=0D +CBCFE =3D=3D Cache Block Clean and Flush instruction Enable=0D +CBIE =3D=3D Cache Block Invalidate instruction Enable=0D +=0D +**/=0D +UINTN=0D +RiscvIsCbcfeEnabledAsm (=0D + VOID=0D + );=0D +=0D +UINTN=0D +RiscvIsCbiEnabledAsm (=0D + VOID=0D + );=0D +=0D +/**=0D + Performs required opeartion on cache lines in the cache coherency domain= =0D + of the calling CPU. If Address is not aligned on a cache line boundary,= =0D + then entire cache line containing Address is operated. If Address + Leng= th=0D + is not aligned on a cache line boundary, then the entire cache line=0D + containing Address + Length -1 is operated.=0D +=0D + If Length is greater than (MAX_ADDRESS - Address + 1), then ASSERT().=0D +=0D + @param Address The base address of the cache lines to=0D + invalidate. If the CPU is in a physical addressing mode,=0D + then Address is a physical address. If the CPU is in a virtual=0D + addressing mode, then Address is a virtual address.=0D +=0D + @param Length The number of bytes to invalidate from the instruction=0D + cache.=0D +=0D + @param Op Type of CMO operation to be performed=0D +=0D + @return Address.=0D +=0D +**/=0D +VOID *=0D +EFIAPI=0D +CacheOpCacheRange (=0D + IN VOID *Address,=0D + IN UINTN Length,=0D + IN CACHE_OP Op=0D + )=0D +{=0D + UINTN CacheLineSize;=0D + UINTN Start;=0D + UINTN End;=0D +=0D + if (Length =3D=3D 0) {=0D + return Address;=0D + }=0D +=0D + ASSERT ((Length - 1) <=3D (MAX_ADDRESS - (UINTN)Address));=0D +=0D + //=0D + // Cache line size is 8 * Bits 15-08 of EBX returned from CPUID 01H=0D + //=0D + CacheLineSize =3D RV64_CACHE_BLOCK_SIZE;=0D +=0D + Start =3D (UINTN)Address;=0D + //=0D + // Calculate the cache line alignment=0D + //=0D + End =3D (Start + Length + (CacheLineSize - 1)) & ~(CacheLineSize - 1)= ;=0D + Start &=3D ~((UINTN)CacheLineSize - 1);=0D +=0D + DEBUG (=0D + (DEBUG_INFO,=0D + "%a Performing Cache Management Operation %d \n", __func__, Op)=0D + );=0D +=0D + do {=0D + switch (Op) {=0D + case Invld:=0D + RiscVCpuCacheInvalAsm_Cbo (Start);=0D + break;=0D + case Flush:=0D + RiscVCpuCacheFlushAsm_Cbo (Start);=0D + break;=0D + case Clean:=0D + RiscVCpuCacheCleanAsm_Cbo (Start);=0D + break;=0D + default:=0D + DEBUG ((DEBUG_ERROR, "%a:RISC-V unsupported operation\n"));=0D + break;=0D + }=0D +=0D + Start =3D Start + CacheLineSize;=0D + } while (Start !=3D End);=0D +=0D + return Address;=0D +}=0D +=0D /**=0D Invalidates the entire instruction cache in cache coherency domain of th= e=0D - calling CPU.=0D + calling CPU. Risc-V does not have currently an CBO implementation which = can=0D + invalidate entire I-cache. Hence using Fence instruction for now. P.S. F= ence=0D + instruction may or may not implement full I-cache invd functionality on = all=0D + implementations.=0D =0D **/=0D VOID=0D @@ -41,7 +181,7 @@ InvalidateInstructionCache ( VOID=0D )=0D {=0D - RiscVInvalidateInstCacheAsm ();=0D + RiscVInvalidateInstCacheAsm_Fence ();=0D }=0D =0D /**=0D @@ -76,12 +216,17 @@ InvalidateInstructionCacheRange ( IN UINTN Length=0D )=0D {=0D - DEBUG (=0D - (DEBUG_WARN,=0D - "%a:RISC-V unsupported function.\n"=0D - "Invalidating the whole instruction cache instead.\n", __func__)=0D - );=0D - InvalidateInstructionCache ();=0D + if (RiscvIsCbiEnabledAsm () =3D=3D RETURN_SUCCESS) {=0D + CacheOpCacheRange (Address, Length, Invld);=0D + } else {=0D + DEBUG (=0D + (DEBUG_WARN,=0D + "%a:RISC-V unsupported function.\n"=0D + "Invalidating the whole instruction cache instead.\n", __func__)=0D + );=0D + InvalidateInstructionCache ();=0D + }=0D +=0D return Address;=0D }=0D =0D @@ -137,7 +282,12 @@ WriteBackInvalidateDataCacheRange ( IN UINTN Length=0D )=0D {=0D - DEBUG ((DEBUG_ERROR, "%a:RISC-V unsupported function.\n", __func__));=0D + if (RiscvIsCbcfeEnabledAsm () =3D=3D RETURN_SUCCESS) {=0D + CacheOpCacheRange (Address, Length, Flush);=0D + } else {=0D + DEBUG ((DEBUG_ERROR, "%a:RISC-V unsupported function.\n", __func__));= =0D + }=0D +=0D return Address;=0D }=0D =0D @@ -192,7 +342,12 @@ WriteBackDataCacheRange ( IN UINTN Length=0D )=0D {=0D - DEBUG ((DEBUG_ERROR, "%a:RISC-V unsupported function.\n", __func__));=0D + if (RiscvIsCbcfeEnabledAsm () =3D=3D RETURN_SUCCESS) {=0D + CacheOpCacheRange (Address, Length, Clean);=0D + } else {=0D + DEBUG ((DEBUG_ERROR, "%a:RISC-V unsupported function.\n", __func__));= =0D + }=0D +=0D return Address;=0D }=0D =0D @@ -213,7 +368,12 @@ InvalidateDataCache ( VOID=0D )=0D {=0D - RiscVInvalidateDataCacheAsm ();=0D + DEBUG (=0D + (DEBUG_WARN,=0D + "%a:RISC-V unsupported function.\n"=0D + "Invalidating the whole instruction cache instead.\n", __func__)=0D + );=0D + RiscVInvalidateDataCacheAsm_Fence ();=0D }=0D =0D /**=0D @@ -250,6 +410,16 @@ InvalidateDataCacheRange ( IN UINTN Length=0D )=0D {=0D - DEBUG ((DEBUG_ERROR, "%a:RISC-V unsupported function.\n", __func__));=0D + if (RiscvIsCbiEnabledAsm () =3D=3D RETURN_SUCCESS) {=0D + CacheOpCacheRange (Address, Length, Invld);=0D + } else {=0D + DEBUG (=0D + (DEBUG_WARN,=0D + "%a:RISC-V unsupported function.\n"=0D + "Invalidating the whole instruction cache instead.\n", __func__)=0D + );=0D + InvalidateDataCache ();=0D + }=0D +=0D return Address;=0D }=0D diff --git a/MdePkg/Library/BaseLib/RiscV64/FlushCache.S b/MdePkg/Library/B= aseLib/RiscV64/FlushCache.S deleted file mode 100644 index 7c10fdd268af..000000000000 --- a/MdePkg/Library/BaseLib/RiscV64/FlushCache.S +++ /dev/null @@ -1,21 +0,0 @@ -//------------------------------------------------------------------------= ------=0D -//=0D -// RISC-V cache operation.=0D -//=0D -// Copyright (c) 2020, Hewlett Packard Enterprise Development LP. All righ= ts reserved.
=0D -//=0D -// SPDX-License-Identifier: BSD-2-Clause-Patent=0D -//=0D -//------------------------------------------------------------------------= ------=0D -=0D -.align 3=0D -ASM_GLOBAL ASM_PFX(RiscVInvalidateInstCacheAsm)=0D -ASM_GLOBAL ASM_PFX(RiscVInvalidateDataCacheAsm)=0D -=0D -ASM_PFX(RiscVInvalidateInstCacheAsm):=0D - fence.i=0D - ret=0D -=0D -ASM_PFX(RiscVInvalidateDataCacheAsm):=0D - fence=0D - ret=0D diff --git a/MdePkg/Library/BaseLib/RiscV64/RiscVCacheMgmt.S b/MdePkg/Libra= ry/BaseLib/RiscV64/RiscVCacheMgmt.S new file mode 100644 index 000000000000..ecf391632221 --- /dev/null +++ b/MdePkg/Library/BaseLib/RiscV64/RiscVCacheMgmt.S @@ -0,0 +1,64 @@ +//------------------------------------------------------------------------= ------=0D +//=0D +// RISC-V cache operation.=0D +//=0D +// Copyright (c) 2020, Hewlett Packard Enterprise Development LP. All righ= ts reserved.
=0D +// Copyright (c) 2022, Rivos Inc. All rights reserved.
=0D +//=0D +// SPDX-License-Identifier: BSD-2-Clause-Patent=0D +//=0D +//------------------------------------------------------------------------= ------=0D +#include =0D +=0D +.align 3=0D +ASM_GLOBAL ASM_PFX(RiscVInvalidateInstCacheAsm_Fence)=0D +ASM_GLOBAL ASM_PFX(RiscVInvalidateDataCacheAsm_Fence)=0D +=0D +ASM_PFX(RiscVInvalidateInstCacheAsm_Fence):=0D + fence.i=0D + ret=0D +=0D +ASM_PFX(RiscVInvalidateDataCacheAsm_Fence):=0D + fence=0D + ret=0D +=0D +ASM_GLOBAL ASM_PFX (RiscVCpuCacheFlushAsm_Cbo)=0D +ASM_PFX (RiscVCpuCacheFlushAsm_Cbo):=0D +=0D + .long 0x0025200f=0D + ret=0D +=0D +ASM_GLOBAL ASM_PFX (RiscVCpuCacheCleanAsm_Cbo)=0D +ASM_PFX (RiscVCpuCacheCleanAsm_Cbo):=0D + .long 0x0015200f=0D + ret=0D +=0D +ASM_GLOBAL ASM_PFX (RiscVCpuCacheInvalAsm_Cbo)=0D +ASM_PFX (RiscVCpuCacheInvalAsm_Cbo):=0D + .long 0x0005200f=0D + ret=0D +=0D +ASM_GLOBAL ASM_PFX(RiscvIsCbiEnabledAsm)=0D +ASM_PFX(RiscvIsCbiEnabledAsm):=0D + li a0, 3=0D + csrr a1, CSR_SENVCFG=0D + and a1, a1, SENVCFG_CBIE=0D + beqz a1, skip=0D +=0D + and a1, a1, 0x10=0D + beqz a1, skip=0D +=0D + mv a0, x0=0D +skip:=0D + ret=0D +=0D +ASM_GLOBAL ASM_PFX(RiscvIsCbcfeEnabledAsm)=0D +ASM_PFX(RiscvIsCbcfeEnabledAsm):=0D + li a0, 3=0D + csrr a1, CSR_SENVCFG=0D + and a1, a1, SENVCFG_CBCFE=0D + beqz a1, next=0D +=0D + mv a0, x0=0D +next:=0D + ret=0D --=20 2.34.1