From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail02.groups.io (mail02.groups.io [66.175.222.108]) by spool.mail.gandi.net (Postfix) with ESMTPS id 0D309941866 for ; Mon, 8 Jan 2024 13:21:32 +0000 (UTC) DKIM-Signature: a=rsa-sha256; bh=K+ZTBwN6+HzfELjcd95Hs7IUlkXBLasOHUd+5BEFF/U=; c=relaxed/simple; d=groups.io; h=Message-ID:Date:MIME-Version:Subject:To:Cc:References:From:In-Reply-To:Precedence:List-Subscribe:List-Help:Sender:List-Id:Mailing-List:Delivered-To:Reply-To:List-Unsubscribe-Post:List-Unsubscribe:Content-Language:Content-Type:Content-Transfer-Encoding; s=20140610; t=1704720091; v=1; b=tmkL+eq/Zf25JxXusz4uuYwgme94cK/PBzQFJ3s6av3SIs5BDLE109xBJbXBlaQDokk2DVn+ UNXDenN4TTzBwGJ+VcJfA9PIak61Xhdozxez04DXUhxNJkKiaZFGp0caNGVAYUEhrdA01UgYQM1 zTdocWwqcUu2jLURryP0DNeo= X-Received: by 127.0.0.2 with SMTP id Dp0rYY7687511xc4G4yJvP7b; Mon, 08 Jan 2024 05:21:31 -0800 X-Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by mx.groups.io with SMTP id smtpd.web10.6383.1704720090689189550 for ; Mon, 08 Jan 2024 05:21:30 -0800 X-Received: from mimecast-mx02.redhat.com (mx-ext.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-90-W-uhDEGHMhyb-9zjCFI-TQ-1; Mon, 08 Jan 2024 08:21:26 -0500 X-MC-Unique: W-uhDEGHMhyb-9zjCFI-TQ-1 X-Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 0BF101C05132; Mon, 8 Jan 2024 13:21:26 +0000 (UTC) X-Received: from [10.39.192.221] (unknown [10.39.192.221]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 50C392026D66; Mon, 8 Jan 2024 13:21:24 +0000 (UTC) Message-ID: <71b9c7de-7686-a3f2-0deb-63d9cc66f4c4@redhat.com> Date: Mon, 8 Jan 2024 14:21:23 +0100 MIME-Version: 1.0 Subject: Re: [edk2-devel] [PATCH v10 4/5] MdePkg: Utilize Cache Management Operations Implementation For RISC-V To: devel@edk2.groups.io, dhaval@rivosinc.com Cc: Michael D Kinney , Liming Gao , Zhiguang Liu , Pedro Falcato , yangcheng.work@foxmail.com References: <20231213145931.28307-1-dhaval@rivosinc.com> <20231213145931.28307-5-dhaval@rivosinc.com> From: "Laszlo Ersek" In-Reply-To: <20231213145931.28307-5-dhaval@rivosinc.com> X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.4 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Precedence: Bulk List-Subscribe: List-Help: Sender: devel@edk2.groups.io List-Id: Mailing-List: list devel@edk2.groups.io; contact devel+owner@edk2.groups.io Reply-To: devel@edk2.groups.io,lersek@redhat.com List-Unsubscribe-Post: List-Unsubscribe=One-Click List-Unsubscribe: X-Gm-Message-State: 66MQiIOsXYLjbD4dnPO77lIex7686176AA= Content-Language: en-US Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-GND-Status: LEGIT Authentication-Results: spool.mail.gandi.net; dkim=pass header.d=groups.io header.s=20140610 header.b="tmkL+eq/"; dmarc=fail reason="SPF not aligned (relaxed), DKIM not aligned (relaxed)" header.from=redhat.com (policy=none); spf=pass (spool.mail.gandi.net: domain of bounce@groups.io designates 66.175.222.108 as permitted sender) smtp.mailfrom=bounce@groups.io Hi Dhaval, On 12/13/23 15:59, Dhaval Sharma wrote: > Use newly defined cache management operations for RISC-V where possible > It builds up on the support added for RISC-V cache management > instructions in BaseLib. > Cc: Michael D Kinney > Cc: Liming Gao > Cc: Zhiguang Liu > Cc: Laszlo Ersek > Cc: Pedro Falcato >=20 > Signed-off-by: Dhaval Sharma > Acked-by: Laszlo Ersek > Reviewed-by: Pedro Falcato > --- >=20 > Notes: > V10: > - Fix formatting to keep comments within 80 > - Replace RV with RISC-V > - Fix an issue with multi line comments > - Added assert to an unsupported function > - Minor case modification in str in .uni >=20 > V9: > - Fixed an issue with Instruction cache invalidation. Use fence.i > instruction as CMO does not support i-cache operations. > V8: > - Added note to convert PCD into RISC-V feature bitmap pointer > - Modified function names to be more explicit about cache ops > - Added RB tag > V7: > - Added PcdLib > - Restructure DEBUG message based on feedback on V6 > - Make naming consistent to CMO, remove all CBO references > - Add ASSERT for not supported functions instead of plain debug messa= ge > - Added RB tag > V6: > - Utilize cache management instructions if HW supports it > This patch is part of restructuring on top of v5 >=20 > MdePkg/MdePkg.dec | 8= + > MdePkg/Library/BaseCacheMaintenanceLib/BaseCacheMaintenanceLib.inf | 5= + > MdePkg/Library/BaseCacheMaintenanceLib/RiscVCache.c | 177= ++++++++++++++++---- > MdePkg/MdePkg.uni | 4= + > 4 files changed, 166 insertions(+), 28 deletions(-) >=20 > diff --git a/MdePkg/MdePkg.dec b/MdePkg/MdePkg.dec > index ac54338089e8..fa92673ff633 100644 > --- a/MdePkg/MdePkg.dec > +++ b/MdePkg/MdePkg.dec > @@ -2399,6 +2399,14 @@ [PcdsFixedAtBuild.AARCH64, PcdsPatchableInModule.A= ARCH64] > # @Prompt CPU Rng algorithm's GUID. > gEfiMdePkgTokenSpaceGuid.PcdCpuRngSupportedAlgorithm|{0x00,0x00,0x00,0= x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00}|VOID*|0x00= 000037 > =20 > +[PcdsFixedAtBuild.RISCV64, PcdsPatchableInModule.RISCV64] > + # > + # Configurability to override RISC-V CPU Features > + # BIT 0 =3D Cache Management Operations. This bit is relevant only if > + # previous stage has feature enabled and user wants to disable it. > + # > + gEfiMdePkgTokenSpaceGuid.PcdRiscVFeatureOverride|0xFFFFFFFFFFFFFFFF|UI= NT64|0x69 > + > [PcdsFixedAtBuild, PcdsPatchableInModule, PcdsDynamic, PcdsDynamicEx] > ## This value is used to set the base address of PCI express hierarchy= . > # @Prompt PCI Express Base Address. > diff --git a/MdePkg/Library/BaseCacheMaintenanceLib/BaseCacheMaintenanceL= ib.inf b/MdePkg/Library/BaseCacheMaintenanceLib/BaseCacheMaintenanceLib.inf > index 6fd9cbe5f6c9..601a38d6c109 100644 > --- a/MdePkg/Library/BaseCacheMaintenanceLib/BaseCacheMaintenanceLib.inf > +++ b/MdePkg/Library/BaseCacheMaintenanceLib/BaseCacheMaintenanceLib.inf > @@ -56,3 +56,8 @@ [LibraryClasses] > BaseLib > DebugLib > =20 > +[LibraryClasses.RISCV64] > + PcdLib > + > +[Pcd.RISCV64] > + gEfiMdePkgTokenSpaceGuid.PcdRiscVFeatureOverride ## CONSUMES > diff --git a/MdePkg/Library/BaseCacheMaintenanceLib/RiscVCache.c b/MdePkg= /Library/BaseCacheMaintenanceLib/RiscVCache.c > index ac2a3c23a249..7c53a17abbb5 100644 > --- a/MdePkg/Library/BaseCacheMaintenanceLib/RiscVCache.c > +++ b/MdePkg/Library/BaseCacheMaintenanceLib/RiscVCache.c > @@ -2,6 +2,7 @@ > RISC-V specific functionality for cache. > =20 > Copyright (c) 2020, Hewlett Packard Enterprise Development LP. All rig= hts reserved.
> + Copyright (c) 2023, Rivos Inc. All rights reserved.
> =20 > SPDX-License-Identifier: BSD-2-Clause-Patent > **/ > @@ -9,10 +10,116 @@ > #include > #include > #include > +#include > + > +// > +// TODO: Grab cache block size and make Cache Management Operation > +// enabling decision based on RISC-V CPU HOB in > +// future when it is available and convert PcdRiscVFeatureOverride > +// PCD to a pointer that contains pointer to bitmap structure > +// which can be operated more elegantly. > +// > +#define RISCV_CACHE_BLOCK_SIZE 64 > +#define RISCV_CPU_FEATURE_CMO_BITMASK 0x1 > + > +typedef enum { > + CacheOpClean, > + CacheOpFlush, > + CacheOpInvld, > +} CACHE_OP; > + > +/** > +Verify CBOs are supported by this HW > +TODO: Use RISC-V CPU HOB once available. > + > +**/ > +STATIC > +BOOLEAN > +RiscVIsCMOEnabled ( > + VOID > + ) > +{ > + // If CMO is disabled in HW, skip Override check > + // Otherwise this PCD can override settings > + return ((PcdGet64 (PcdRiscVFeatureOverride) & RISCV_CPU_FEATURE_CMO_BI= TMASK) !=3D 0); > +} > + > +/** > + Performs required opeartion on cache lines in the cache coherency doma= in > + of the calling CPU. If Address is not aligned on a cache line boundary= , > + then entire cache line containing Address is operated. If Address + Le= ngth > + is not aligned on a cache line boundary, then the entire cache line > + containing Address + Length -1 is operated. > + If Length is greater than (MAX_ADDRESS - Address + 1), then ASSERT(). > + @param Address The base address of the cache lines to > + invalidate. > + @param Length The number of bytes to invalidate from the instruction > + cache. > + @param Op Type of CMO operation to be performed > + @return Address. > + > +**/ > +STATIC > +VOID > +CacheOpCacheRange ( > + IN VOID *Address, > + IN UINTN Length, > + IN CACHE_OP Op > + ) > +{ > + UINTN CacheLineSize; > + UINTN Start; > + UINTN End; > + > + if (Length =3D=3D 0) { > + return; > + } > + > + if ((Op !=3D CacheOpInvld) && (Op !=3D CacheOpFlush) && (Op !=3D Cache= OpClean)) { > + return; > + } > + > + ASSERT ((Length - 1) <=3D (MAX_ADDRESS - (UINTN)Address)); > + > + CacheLineSize =3D RISCV_CACHE_BLOCK_SIZE; > + > + Start =3D (UINTN)Address; > + // > + // Calculate the cache line alignment > + // > + End =3D (Start + Length + (CacheLineSize - 1)) & ~(CacheLineSize - = 1); > + Start &=3D ~((UINTN)CacheLineSize - 1); > + > + DEBUG ( > + (DEBUG_VERBOSE, > + "CacheOpCacheRange: Performing Cache Management Operation %d \n", O= p) > + ); > + > + do { > + switch (Op) { > + case CacheOpInvld: > + RiscVCpuCacheInvalCmoAsm (Start); > + break; > + case CacheOpFlush: > + RiscVCpuCacheFlushCmoAsm (Start); > + break; > + case CacheOpClean: > + RiscVCpuCacheCleanCmoAsm (Start); > + break; > + default: > + break; > + } > + > + Start =3D Start + CacheLineSize; > + } while (Start !=3D End); > +} > =20 > /** > Invalidates the entire instruction cache in cache coherency domain of = the > - calling CPU. > + calling CPU. Risc-V does not have currently an CBO implementation whic= h can > + invalidate the entire I-cache. Hence using Fence instruction for now. = P.S. > + Fence instruction may or may not implement full I-cache invd functiona= lity > + on all implementations. > =20 > **/ > VOID > @@ -28,17 +135,11 @@ InvalidateInstructionCache ( > Invalidates a range of instruction cache lines in the cache coherency = domain > of the calling CPU. > =20 > - Invalidates the instruction cache lines specified by Address and Lengt= h. If > - Address is not aligned on a cache line boundary, then entire instructi= on > - cache line containing Address is invalidated. If Address + Length is n= ot > - aligned on a cache line boundary, then the entire instruction cache li= ne > - containing Address + Length -1 is invalidated. This function may choos= e to > - invalidate the entire instruction cache if that is more efficient than > - invalidating the specified range. If Length is 0, then no instruction = cache > - lines are invalidated. Address is returned. > - > - If Length is greater than (MAX_ADDRESS - Address + 1), then ASSERT(). > - > + An operation from a CMO instruction is defined to operate only on the = copies > + of a cache block that are cached in the caches accessible by the expli= cit > + memory accesses performed by the set of coherent agents.In other words= CMO > + operations are not applicable to instruction cache. Use fence.i instru= ction > + instead to achieve the same purpose. > @param Address The base address of the instruction cache lines to > invalidate. If the CPU is in a physical addressing mod= e, then > Address is a physical address. If the CPU is in a virt= ual > @@ -57,9 +158,10 @@ InvalidateInstructionCacheRange ( > ) > { > DEBUG ( > - (DEBUG_WARN, > - "%a:RISC-V unsupported function.\n" > - "Invalidating the whole instruction cache instead.\n", __func__) > + (DEBUG_VERBOSE, > + "InvalidateInstructionCacheRange: RISC-V unsupported function.\n" > + "Invalidating the whole instruction cache instead.\n" > + ) > ); > InvalidateInstructionCache (); > return Address; > @@ -81,7 +183,12 @@ WriteBackInvalidateDataCache ( > VOID > ) > { > - DEBUG ((DEBUG_ERROR, "%a:RISC-V unsupported function.\n", __func__)); > + ASSERT (FALSE); > + DEBUG (( > + DEBUG_ERROR, > + "WriteBackInvalidateDataCache:" \ > + "RISC-V unsupported function.\n" > + )); > } > =20 > /** > @@ -117,7 +224,12 @@ WriteBackInvalidateDataCacheRange ( > IN UINTN Length > ) > { > - DEBUG ((DEBUG_ERROR, "%a:RISC-V unsupported function.\n", __func__)); > + if (RiscVIsCMOEnabled ()) { > + CacheOpCacheRange (Address, Length, CacheOpFlush); > + } else { > + ASSERT (FALSE); > + } > + > return Address; > } This change (replacing the DEBUG with an ASSERT()) seems to be causing a failure for some physical platforms: https://bugzilla.tianocore.org/show_bug.cgi?id=3D4637 Can you please check that ticket? (I'd have CC'd you on the ticket, but the CC search field returned no hits for "Dhaval" -- I think you may not have an account in Bugzilla yet.) If a platform has no support for CMO (and the platform DSC sets PcdRiscVFeatureOverride accordingly), then WriteBackInvalidateDataCacheRange() is effectively uncallable on that platform. Is that intentional? I think there are two possibilities (when RiscVIsCMOEnabled() returns FALSE): - the platform has "no need" for writing back and invalidating a range of the data cache -- perhaps because it has no data cache in the CPUs at all. In that case, a no-op is better (and still safe) than a failed ASSERT(). - or else, the platform needs this kind of flushing and invalidation, but the CPU just doesn't support the particular CMO / fine-grained instruction. In that case, can we do something heavier-weight, but functionally *sufficient*? A good example is in this same patch's InvalidateDataCacheRange() function, which falls back to InvalidateDataCache() if CMO is missing. Now, I can see that WriteBackDataCache() *itself* is not supported (regardless of CMO). Why is that? Does it make no sense on RISC-V platforms? If that's the case, should it be a no-op instead? Yangcheng: can you provide a backtrace? What outer call path led you to the ASSERT() in WriteBackInvalidateDataCacheRange()? Thanks! Laszlo > =20 > @@ -137,7 +249,7 @@ WriteBackDataCache ( > VOID > ) > { > - DEBUG ((DEBUG_ERROR, "%a:RISC-V unsupported function.\n", __func__)); > + ASSERT (FALSE); > } > =20 > /** > @@ -156,10 +268,7 @@ WriteBackDataCache ( > =20 > If Length is greater than (MAX_ADDRESS - Address + 1), then ASSERT(). > =20 > - @param Address The base address of the data cache lines to write back= . If > - the CPU is in a physical addressing mode, then Address= is a > - physical address. If the CPU is in a virtual addressin= g > - mode, then Address is a virtual address. > + @param Address The base address of the data cache lines to write back= . > @param Length The number of bytes to write back from the data cache. > =20 > @return Address of cache written in main memory. > @@ -172,7 +281,12 @@ WriteBackDataCacheRange ( > IN UINTN Length > ) > { > - DEBUG ((DEBUG_ERROR, "%a:RISC-V unsupported function.\n", __func__)); > + if (RiscVIsCMOEnabled ()) { > + CacheOpCacheRange (Address, Length, CacheOpClean); > + } else { > + ASSERT (FALSE); > + } > + > return Address; > } > =20 > @@ -214,10 +328,7 @@ InvalidateDataCache ( > =20 > If Length is greater than (MAX_ADDRESS - Address + 1), then ASSERT(). > =20 > - @param Address The base address of the data cache lines to invalidate= . If > - the CPU is in a physical addressing mode, then Address= is a > - physical address. If the CPU is in a virtual addressin= g mode, > - then Address is a virtual address. > + @param Address The base address of the data cache lines to invalidate= . > @param Length The number of bytes to invalidate from the data cache. > =20 > @return Address. > @@ -230,6 +341,16 @@ InvalidateDataCacheRange ( > IN UINTN Length > ) > { > - DEBUG ((DEBUG_ERROR, "%a:RISC-V unsupported function.\n", __func__)); > + if (RiscVIsCMOEnabled ()) { > + CacheOpCacheRange (Address, Length, CacheOpInvld); > + } else { > + DEBUG ( > + (DEBUG_VERBOSE, > + "InvalidateDataCacheRange: Zicbom not supported.\n" \ > + "Invalidating the whole Data cache instead.\n") \ > + ); > + InvalidateDataCache (); > + } > + > return Address; > } > diff --git a/MdePkg/MdePkg.uni b/MdePkg/MdePkg.uni > index 5c1fa24065c7..73b5dd8f32cc 100644 > --- a/MdePkg/MdePkg.uni > +++ b/MdePkg/MdePkg.uni > @@ -287,6 +287,10 @@ > =20 > #string STR_gEfiMdePkgTokenSpaceGuid_PcdGuidedExtractHandlerTableAddress= _HELP #language en-US "This value is used to set the available memory addr= ess to store Guided Extract Handlers. The required memory space is decided = by the value of PcdMaximumGuidedExtractHandler." > =20 > +#string STR_gEfiMdePkgTokenSpaceGuid_PcdRiscVFeatureOverride_PROMPT #la= nguage en-US "RISC-V Feature Override" > + > +#string STR_gEfiMdePkgTokenSpaceGuid_PcdRiscVFeatureOverride_HELP #lang= uage en-US "This value is used to override any RISC-V specific features sup= ported by this PCD" > + > #string STR_gEfiMdePkgTokenSpaceGuid_PcdPciExpressBaseAddress_PROMPT #l= anguage en-US "PCI Express Base Address" > =20 > #string STR_gEfiMdePkgTokenSpaceGuid_PcdPciExpressBaseAddress_HELP #lan= guage en-US "This value is used to set the base address of PCI express hier= archy." -=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D- Groups.io Links: You receive all messages sent to this group. View/Reply Online (#113390): https://edk2.groups.io/g/devel/message/113390 Mute This Topic: https://groups.io/mt/103150435/7686176 Group Owner: devel+owner@edk2.groups.io Unsubscribe: https://edk2.groups.io/g/devel/leave/12367111/7686176/19134562= 12/xyzzy [rebecca@openfw.io] -=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-