From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from ams.source.kernel.org (ams.source.kernel.org [145.40.68.75]) by mx.groups.io with SMTP id smtpd.web10.198300.1673967240698538038 for ; Tue, 17 Jan 2023 06:54:01 -0800 Authentication-Results: mx.groups.io; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=fdiDwIGP; spf=pass (domain: kernel.org, ip: 145.40.68.75, mailfrom: ardb@kernel.org) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id 0A6F6B812A2 for ; Tue, 17 Jan 2023 14:53:58 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id A50C4C433EF for ; Tue, 17 Jan 2023 14:53:56 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1673967236; bh=3EAVqblbZ5BbSsAozGcucg+eBjWRhaJXUP8g87AKXtg=; h=References:In-Reply-To:From:Date:Subject:To:Cc:From; b=fdiDwIGPLxH76Hg4oYgoPuNZ8WVg1L/Alao80o+7p8RRIoH6qxZBy2xJAzPU2T5P3 nOJ/shMvDEUkmThNYZQFEt5x/JD3R3SebdctsQYaQSM2Fd+VPsNgmcHuMkc625gBEy bFZ7aeGycFeBpMhnvARN2T6es94MCFWGxqFANlJ5JlHBsj1AcvL+bbDXaKx7sIoTm8 9ya0GmPEv8hZOgQ/0QYyAdnEhjFG/MjOs/QTNZsskOXhh/Wc6ZwPrjiZIzT43yeCh4 5aynBsmJWI5AN+/PCPj60fiQmcNGHmiDiTlbsK4d5CYgezNn/szyOFovqgQtHw/LZx xoCJk+o83yF9w== Received: by mail-lf1-f42.google.com with SMTP id b3so47446006lfv.2 for ; Tue, 17 Jan 2023 06:53:56 -0800 (PST) X-Gm-Message-State: AFqh2ko0yxJoXO+Ax7UrgvoQ39dQWJ66A1EILrvemAIL9XKvRXpwl9GR nVEI9hh7JTRL1CjKF4JLHVlyWQUQxPitViDvu9U= X-Google-Smtp-Source: AMrXdXvj5rV0xJe46P0SIghZ9fpaGaK+KZk1Kj2TYBZQdokrdex/x90IEzbIwWopzjQiZelRYGOC8Xb9hjVbjC3aAAc= X-Received: by 2002:ac2:4ade:0:b0:4d0:7b7:65dc with SMTP id m30-20020ac24ade000000b004d007b765dcmr144632lfp.122.1673967234571; Tue, 17 Jan 2023 06:53:54 -0800 (PST) MIME-Version: 1.0 References: <20230105162528.1430368-1-ardb@kernel.org> <20230105162528.1430368-2-ardb@kernel.org> In-Reply-To: From: "Ard Biesheuvel" Date: Tue, 17 Jan 2023 15:53:43 +0100 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: [edk2-devel] [PATCH v2 2/2] ArmVirtPkg/ArmVirtQemu: Avoid early ID map on ThunderX To: devel@edk2.groups.io, osteffen@redhat.com Cc: dann.frazier@canonical.com Content-Type: text/plain; charset="UTF-8" On Tue, 17 Jan 2023 at 13:48, Oliver Steffen wrote: > > Hi Ard, Hi everyone, > > Thanks for the work! > > But somehow this patch (as it was merged into master branch) does not > work for me on the ThunderX box we have. > > Any idea what could be wrong? I'm not sure I understand the question. The patch targets ThunderX, and you are using a ThunderX2. What were you expecting to happen, and what is happening instead? > I enabled the erratum during build ;-) > > CPU Info: > # lscpu > Architecture: aarch64 > CPU op-mode(s): 64-bit > Byte Order: Little Endian > CPU(s): 224 > On-line CPU(s) list: 0-223 > Vendor ID: Cavium > BIOS Vendor ID: Cavium Inc. > Model name: ThunderX2 99xx > BIOS Model name: Cavium ThunderX2(R) CPU CN9975 v2.2 @ 2.0GHz > Model: 2 > Thread(s) per core: 4 > Core(s) per socket: 28 > Socket(s): 2 > Stepping: 0x1 > BogoMIPS: 400.00 > Flags: fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics cpuid asimdrdm > Caches (sum of all): > L1d: 1.8 MiB (56 instances) > L1i: 1.8 MiB (56 instances) > L2: 14 MiB (56 instances) > L3: 64 MiB (2 instances) > [...] > > Thanks a lot! > - Oliver > > > On Tue, Jan 10, 2023 at 1:08 AM dann frazier wrote: >> >> On Thu, Jan 05, 2023 at 05:25:28PM +0100, Ard Biesheuvel wrote: >> > The early ID map used by ArmVirtQemu uses ASID scoped non-global >> > mappings, as this allows us to switch to the permanent ID map seamlessly >> > without the need for explicit TLB maintenance. >> > >> > However, this triggers a known erratum on ThunderX, which does not >> > tolerate non-global mappings that are executable at EL1, as this appears >> > to result in I-cache corruption. (Linux disables the KPTI based Meltdown >> > mitigation on ThunderX for the same reason) >> > >> > So work around this, by detecting the CPU implementor and part number, >> > and proceeding without the early ID map if a ThunderX CPU is detected. >> > >> > Note that this requires the C code to be built with strict alignment >> > again, as we may end up executing it with the MMU and caches off. >> > >> > Signed-off-by: Ard Biesheuvel >> > --- >> > ArmVirtPkg/ArmVirtQemu.dsc | 5 +++++ >> > ArmVirtPkg/Library/ArmPlatformLibQemu/AArch64/ArmPlatformHelper.S | 15 +++++++++++++++ >> > 2 files changed, 20 insertions(+) >> >> FTR, this v2 series also worked for me. >> >> -dann >> >> > diff --git a/ArmVirtPkg/ArmVirtQemu.dsc b/ArmVirtPkg/ArmVirtQemu.dsc >> > index f77443229e8e..5dd8b6104cca 100644 >> > --- a/ArmVirtPkg/ArmVirtQemu.dsc >> > +++ b/ArmVirtPkg/ArmVirtQemu.dsc >> > @@ -31,6 +31,7 @@ [Defines] >> > DEFINE SECURE_BOOT_ENABLE = FALSE >> > DEFINE TPM2_ENABLE = FALSE >> > DEFINE TPM2_CONFIG_ENABLE = FALSE >> > + DEFINE CAVIUM_ERRATUM_27456 = FALSE >> > >> > # >> > # Network definition >> > @@ -117,7 +118,11 @@ [LibraryClasses.common.UEFI_DRIVER] >> > UefiScsiLib|MdePkg/Library/UefiScsiLib/UefiScsiLib.inf >> > >> > [BuildOptions] >> > +!if $(CAVIUM_ERRATUM_27456) == TRUE >> > + GCC:*_*_AARCH64_PP_FLAGS = -DCAVIUM_ERRATUM_27456 >> > +!else >> > GCC:*_*_AARCH64_CC_XIPFLAGS == >> > +!endif >> > >> > !include NetworkPkg/NetworkBuildOptions.dsc.inc >> > >> > diff --git a/ArmVirtPkg/Library/ArmPlatformLibQemu/AArch64/ArmPlatformHelper.S b/ArmVirtPkg/Library/ArmPlatformLibQemu/AArch64/ArmPlatformHelper.S >> > index 1787d52fbf51..5ac7c732f6ec 100644 >> > --- a/ArmVirtPkg/Library/ArmPlatformLibQemu/AArch64/ArmPlatformHelper.S >> > +++ b/ArmVirtPkg/Library/ArmPlatformLibQemu/AArch64/ArmPlatformHelper.S >> > @@ -42,6 +42,21 @@ >> > >> > >> > ASM_FUNC(ArmPlatformPeiBootAction) >> > +#ifdef CAVIUM_ERRATUM_27456 >> > + /* >> > + * On Cavium ThunderX, using non-global mappings that are executable at EL1 >> > + * results in I-cache corruption. So just avoid the early ID mapping there. >> > + * >> > + * MIDR implementor 0x43 >> > + * MIDR part numbers 0xA1 0xA2 (but not 0xAF) >> > + */ >> > + mrs x0, midr_el1 // read the MIDR into X0 >> > + ubfx x1, x0, #24, #8 // grab implementor id >> > + ubfx x0, x0, #7, #9 // grab part number bits [11:3] >> > + cmp x1, #0x43 // compare implementor id >> > + ccmp x0, #0xA0 >> 3, #0, eq // compare part# bits [11:3] >> > + b.eq 0f >> > +#endif >> > mrs x0, CurrentEL // check current exception level >> > tbnz x0, #3, 0f // omit early ID map if above EL1 >> > >> >> >> >> >> >