From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-wm1-f68.google.com (mail-wm1-f68.google.com [209.85.128.68]) by mx.groups.io with SMTP id smtpd.web12.15303.1583344023648500938 for ; Wed, 04 Mar 2020 09:47:03 -0800 Authentication-Results: mx.groups.io; dkim=pass header.i=@linaro.org header.s=google header.b=VmSgbEqd; spf=pass (domain: linaro.org, ip: 209.85.128.68, mailfrom: ard.biesheuvel@linaro.org) Received: by mail-wm1-f68.google.com with SMTP id m3so2794588wmi.0 for ; Wed, 04 Mar 2020 09:47:03 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=pk12X0hwLwaecgwwwzXhSjme5i247pnALYcCcNL2+jI=; b=VmSgbEqdFbpnSEL1xbx8OZKLW4XkgHaeTev+wXSOiRhv1qJs6bIiwmlLoXVgPnZg1b 6FahBl3fLq1t1WU6PXeRkwRAqGlse8Y43IELKExDGRKdBSRdxANFXJijTrkv9u1qzuYx LY41e+93843RLIiHVeGitNYoIVpHN1rgnlhEOLSjQ8J25r97k2Y6jFM2N6JBR/GA/djA 8/bpgF3pHMIE2x9bOSgfkCyjy8n0I/IUXYeBNHqvy66Qw46XueVyd/tW9VC8JekRBcim /5HO74cYQpeaCcrO4ZFI1/wQZs7vBXPrb2JsJv/8j7p53ciatUhfd7aP8YRU2hnHQo9A +Zhw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=pk12X0hwLwaecgwwwzXhSjme5i247pnALYcCcNL2+jI=; b=iIMqY1JnWBbhvrTJrshFKtVwVb2rcCnJwEOaLwJ1z+aAstqMaFvyI3YnHzgK5Kp34k xWsXvXM3C+4uwWa9pRPOUCwOOTUa1Kd0QP3vXAy4Mrjm+XFEdQipqYAcQqDeiZ7+TgzA HUdjvkuSmZ01dhX1XP0yz+//tT65mg7NyOmBaBbTb0FLD+5JSaEUshRPUiXKu4ZozRMr fzNLphUsuv6GAeai/zL8siuSw/M/1hTDP7/BemwkkICYbGgpUcuU3N7riFn6vU3PzOFN LqQuiVneD2UgHQsCeigVQTKfgdUgdH5H9BBMBOh0XhbjDvK17CkNBE4R2sBgDLgqB3VD LkMg== X-Gm-Message-State: ANhLgQ156CYlLvwzaD1uPtdWHvSQ5aKbn1/rsMz0ycTDC0s2UxGGoY4l Sbv6EMOU1A4IioUK68j9TyYXG62nqAEaQyIWa8BmWqbW3HVf7g== X-Google-Smtp-Source: ADFU+vvNuQoP2BWOCpXFxAMz8wxUkBnS+td3FIioT+DV4VgdimkFtOZlO1NqGugKV2iqGwzxHLZ8VqGCXQU1KMvdEV0= X-Received: by 2002:a7b:cb93:: with SMTP id m19mr4848654wmi.133.1583344021956; Wed, 04 Mar 2020 09:47:01 -0800 (PST) MIME-Version: 1.0 References: <20200221110714.17966-1-ard.biesheuvel@arm.com> In-Reply-To: From: "Ard Biesheuvel" Date: Wed, 4 Mar 2020 18:46:51 +0100 Message-ID: Subject: Re: [edk2-devel] [PATCH 1/1] ArmPlatformPkg/PrePeiCore: replace set/way cache ops with by-VA ones To: edk2-devel-groups-io , Laszlo Ersek Cc: Ard Biesheuvel , Leif Lindholm Content-Type: text/plain; charset="UTF-8" On Fri, 21 Feb 2020 at 16:54, Laszlo Ersek wrote: > > On 02/21/20 12:07, Ard Biesheuvel wrote: > > Cache maintenance operations by set/way are only intended to be used > > in the context of on/offlining a core, while it has been taken out of > > the coherency domain. Any use intended to ensure that the contents of > > the cache have made it to main memory is unreliable, since cacheline > > migration and non-architected system caches may cause these contents > > to linger elsewhere, without being visible in main memory once the > > MMU and caches are disabled. > > > > In KVM on Linux, there are horrid hacks in place to ensure that such > > set/way operations are trapped, and replaced with a single by-VA > > clean/invalidate of the entire guest VA space once the MMU state > > changes, which can be costly, and is unnecessary if we manage the > > caches a bit more carefully, and perform maintenance by virtual > > address only. > > > > So let's get rid of the call to ArmInvalidateDataCache () in the > > PrePeiCore startup code, and instead, invalidate the temporary RAM > > region by virtual address, which is the only memory region we will > > be touching with the caches and MMU both disabled and enabled. > > (This will lead to data corruption if data written with the MMU off > > is shadowed by clean, stale cachelines that stick around when the > > MMU is enabled again.) > > > > Signed-off-by: Ard Biesheuvel > > --- > > Tested on bare metal (SynQuacer 32-bit) and KVM (mach-virt 64-bit) > > > > ArmPlatformPkg/PrePeiCore/PrePeiCore.c | 6 ++++-- > > ArmPlatformPkg/PrePeiCore/PrePeiCoreMPCore.inf | 1 + > > ArmPlatformPkg/PrePeiCore/PrePeiCoreUniCore.inf | 1 + > > 3 files changed, 6 insertions(+), 2 deletions(-) > > > > diff --git a/ArmPlatformPkg/PrePeiCore/PrePeiCore.c b/ArmPlatformPkg/PrePeiCore/PrePeiCore.c > > index 4911f67577a2..a7a61fe9ddb5 100644 > > --- a/ArmPlatformPkg/PrePeiCore/PrePeiCore.c > > +++ b/ArmPlatformPkg/PrePeiCore/PrePeiCore.c > > @@ -8,6 +8,7 @@ > > **/ > > > > #include > > +#include > > #include > > #include > > > > @@ -59,13 +60,14 @@ CEntryPoint ( > > { > > // Data Cache enabled on Primary core when MMU is enabled. > > ArmDisableDataCache (); > > - // Invalidate Data cache > > - ArmInvalidateDataCache (); > > // Invalidate instruction cache > > ArmInvalidateInstructionCache (); > > // Enable Instruction Caches on all cores. > > ArmEnableInstructionCache (); > > > > + InvalidateDataCacheRange ((VOID *)(UINTN)PcdGet64 (PcdCPUCoresStackBase), > > + PcdGet32 (PcdCPUCorePrimaryStackSize)); > > + > > // > > // Note: Doesn't have to Enable CPU interface in non-secure world, > > // as Non-secure interface is already enabled in Secure world. > > diff --git a/ArmPlatformPkg/PrePeiCore/PrePeiCoreMPCore.inf b/ArmPlatformPkg/PrePeiCore/PrePeiCoreMPCore.inf > > index f2ac45d171bc..9d90d46dcfc5 100644 > > --- a/ArmPlatformPkg/PrePeiCore/PrePeiCoreMPCore.inf > > +++ b/ArmPlatformPkg/PrePeiCore/PrePeiCoreMPCore.inf > > @@ -44,6 +44,7 @@ [Packages] > > [LibraryClasses] > > ArmLib > > ArmPlatformLib > > + CacheMaintenanceLib > > BaseLib > > DebugLib > > DebugAgentLib > > diff --git a/ArmPlatformPkg/PrePeiCore/PrePeiCoreUniCore.inf b/ArmPlatformPkg/PrePeiCore/PrePeiCoreUniCore.inf > > index 84c319c3679b..0749a6d575cf 100644 > > --- a/ArmPlatformPkg/PrePeiCore/PrePeiCoreUniCore.inf > > +++ b/ArmPlatformPkg/PrePeiCore/PrePeiCoreUniCore.inf > > @@ -44,6 +44,7 @@ [Packages] > > [LibraryClasses] > > ArmLib > > ArmPlatformLib > > + CacheMaintenanceLib > > BaseLib > > DebugLib > > DebugAgentLib > > > > Acked-by: Laszlo Ersek > Pushed as 6c9a3d4233d78a04db5f25aeed254396740f4cae Thanks all