On Tue, Jan 17, 2023 at 3:57 PM Ard Biesheuvel wrote: > On Tue, 17 Jan 2023 at 13:48, Oliver Steffen wrote: > > > > Hi Ard, Hi everyone, > > > > Thanks for the work! > > > > But somehow this patch (as it was merged into master branch) does not > > work for me on the ThunderX box we have. > > > > Any idea what could be wrong? > > I'm not sure I understand the question. The patch targets ThunderX, > and you are using a ThunderX2. > > What were you expecting to happen, and what is happening instead? Firmware does not start at all when using KVM. Please excuse my limited knowledge of Arm processor variants. I assumed that ThunderX and ThunderX2 are very similar and hoped the fix would also work for this case. The issue was introduced by the same commit that Dann reported (07be1d34d95460a238fcd0f6693efb747c28b329): "ArmVirtPkg/ArmVirtQemu: enable initial ID map at early boot". > I enabled the erratum during build ;-) > > > > CPU Info: > > # lscpu > > Architecture: aarch64 > > CPU op-mode(s): 64-bit > > Byte Order: Little Endian > > CPU(s): 224 > > On-line CPU(s) list: 0-223 > > Vendor ID: Cavium > > BIOS Vendor ID: Cavium Inc. > > Model name: ThunderX2 99xx > > BIOS Model name: Cavium ThunderX2(R) CPU CN9975 v2.2 @ 2.0GHz > > Model: 2 > > Thread(s) per core: 4 > > Core(s) per socket: 28 > > Socket(s): 2 > > Stepping: 0x1 > > BogoMIPS: 400.00 > > Flags: fp asimd evtstrm aes pmull sha1 sha2 crc32 > atomics cpuid asimdrdm > > Caches (sum of all): > > L1d: 1.8 MiB (56 instances) > > L1i: 1.8 MiB (56 instances) > > L2: 14 MiB (56 instances) > > L3: 64 MiB (2 instances) > > [...] > > > > Thanks a lot! > > - Oliver > > > > > > On Tue, Jan 10, 2023 at 1:08 AM dann frazier > wrote: > >> > >> On Thu, Jan 05, 2023 at 05:25:28PM +0100, Ard Biesheuvel wrote: > >> > The early ID map used by ArmVirtQemu uses ASID scoped non-global > >> > mappings, as this allows us to switch to the permanent ID map > seamlessly > >> > without the need for explicit TLB maintenance. > >> > > >> > However, this triggers a known erratum on ThunderX, which does not > >> > tolerate non-global mappings that are executable at EL1, as this > appears > >> > to result in I-cache corruption. (Linux disables the KPTI based > Meltdown > >> > mitigation on ThunderX for the same reason) > >> > > >> > So work around this, by detecting the CPU implementor and part number, > >> > and proceeding without the early ID map if a ThunderX CPU is detected. > >> > > >> > Note that this requires the C code to be built with strict alignment > >> > again, as we may end up executing it with the MMU and caches off. > >> > > >> > Signed-off-by: Ard Biesheuvel > >> > --- > >> > ArmVirtPkg/ArmVirtQemu.dsc | > 5 +++++ > >> > ArmVirtPkg/Library/ArmPlatformLibQemu/AArch64/ArmPlatformHelper.S | > 15 +++++++++++++++ > >> > 2 files changed, 20 insertions(+) > >> > >> FTR, this v2 series also worked for me. > >> > >> -dann > >> > >> > diff --git a/ArmVirtPkg/ArmVirtQemu.dsc b/ArmVirtPkg/ArmVirtQemu.dsc > >> > index f77443229e8e..5dd8b6104cca 100644 > >> > --- a/ArmVirtPkg/ArmVirtQemu.dsc > >> > +++ b/ArmVirtPkg/ArmVirtQemu.dsc > >> > @@ -31,6 +31,7 @@ [Defines] > >> > DEFINE SECURE_BOOT_ENABLE = FALSE > >> > DEFINE TPM2_ENABLE = FALSE > >> > DEFINE TPM2_CONFIG_ENABLE = FALSE > >> > + DEFINE CAVIUM_ERRATUM_27456 = FALSE > >> > > >> > # > >> > # Network definition > >> > @@ -117,7 +118,11 @@ [LibraryClasses.common.UEFI_DRIVER] > >> > UefiScsiLib|MdePkg/Library/UefiScsiLib/UefiScsiLib.inf > >> > > >> > [BuildOptions] > >> > +!if $(CAVIUM_ERRATUM_27456) == TRUE > >> > + GCC:*_*_AARCH64_PP_FLAGS = -DCAVIUM_ERRATUM_27456 > >> > +!else > >> > GCC:*_*_AARCH64_CC_XIPFLAGS == > >> > +!endif > >> > > >> > !include NetworkPkg/NetworkBuildOptions.dsc.inc > >> > > >> > diff --git > a/ArmVirtPkg/Library/ArmPlatformLibQemu/AArch64/ArmPlatformHelper.S > b/ArmVirtPkg/Library/ArmPlatformLibQemu/AArch64/ArmPlatformHelper.S > >> > index 1787d52fbf51..5ac7c732f6ec 100644 > >> > --- > a/ArmVirtPkg/Library/ArmPlatformLibQemu/AArch64/ArmPlatformHelper.S > >> > +++ > b/ArmVirtPkg/Library/ArmPlatformLibQemu/AArch64/ArmPlatformHelper.S > >> > @@ -42,6 +42,21 @@ > >> > > >> > > >> > ASM_FUNC(ArmPlatformPeiBootAction) > >> > +#ifdef CAVIUM_ERRATUM_27456 > >> > + /* > >> > + * On Cavium ThunderX, using non-global mappings that are > executable at EL1 > >> > + * results in I-cache corruption. So just avoid the early ID > mapping there. > >> > + * > >> > + * MIDR implementor 0x43 > >> > + * MIDR part numbers 0xA1 0xA2 (but not 0xAF) > >> > + */ > >> > + mrs x0, midr_el1 // read the MIDR into X0 > >> > + ubfx x1, x0, #24, #8 // grab implementor id > >> > + ubfx x0, x0, #7, #9 // grab part number bits [11:3] > >> > + cmp x1, #0x43 // compare implementor id > >> > + ccmp x0, #0xA0 >> 3, #0, eq // compare part# bits [11:3] > >> > + b.eq 0f > >> > +#endif > >> > mrs x0, CurrentEL // check current exception level > >> > tbnz x0, #3, 0f // omit early ID map if above EL1 > >> > > >