Re: [edk2-devel] [PATCH v2 2/2] ArmVirtPkg/ArmVirtQemu: Avoid early ID map on ThunderX

["Oliver Steffen" <osteffen@redhat.com>]

[-- Attachment #1: Type: text/plain, Size: 5508 bytes --]

On Tue, Jan 17, 2023 at 3:57 PM Ard Biesheuvel <ardb@kernel.org> wrote:

> On Tue, 17 Jan 2023 at 13:48, Oliver Steffen <osteffen@redhat.com> wrote:
> >
> > Hi Ard, Hi everyone,
> >
> > Thanks for the work!
> >
> > But somehow this patch (as it was merged into master branch) does not
> > work for me on the ThunderX box we have.
> >
> > Any idea what could be wrong?
>
> I'm not sure I understand the question. The patch targets ThunderX,
> and you are using a ThunderX2.
>
> What were you expecting to happen, and what is happening instead?


Firmware does not start at all when using KVM.

Please excuse my limited knowledge of Arm processor variants.
I assumed that ThunderX and ThunderX2 are very similar and hoped
the fix would also work for this case.

The issue was introduced by the same commit that Dann
reported (07be1d34d95460a238fcd0f6693efb747c28b329):
"ArmVirtPkg/ArmVirtQemu: enable initial ID map at early boot".

> I enabled the erratum during build ;-)
> >
> > CPU Info:
> > # lscpu
> > Architecture:           aarch64
> >   CPU op-mode(s):       64-bit
> >   Byte Order:           Little Endian
> > CPU(s):                 224
> >   On-line CPU(s) list:  0-223
> > Vendor ID:              Cavium
> >   BIOS Vendor ID:       Cavium Inc.
> >   Model name:           ThunderX2 99xx
> >     BIOS Model name:    Cavium ThunderX2(R) CPU CN9975 v2.2 @ 2.0GHz
> >     Model:              2
> >     Thread(s) per core: 4
> >     Core(s) per socket: 28
> >     Socket(s):          2
> >     Stepping:           0x1
> >     BogoMIPS:           400.00
> >     Flags:              fp asimd evtstrm aes pmull sha1 sha2 crc32
> atomics cpuid asimdrdm
> > Caches (sum of all):
> >   L1d:                  1.8 MiB (56 instances)
> >   L1i:                  1.8 MiB (56 instances)
> >   L2:                   14 MiB (56 instances)
> >   L3:                   64 MiB (2 instances)
> > [...]
> >
> > Thanks a lot!
> > - Oliver
> >
> >
> > On Tue, Jan 10, 2023 at 1:08 AM dann frazier <dann.frazier@canonical.com>
> wrote:
> >>
> >> On Thu, Jan 05, 2023 at 05:25:28PM +0100, Ard Biesheuvel wrote:
> >> > The early ID map used by ArmVirtQemu uses ASID scoped non-global
> >> > mappings, as this allows us to switch to the permanent ID map
> seamlessly
> >> > without the need for explicit TLB maintenance.
> >> >
> >> > However, this triggers a known erratum on ThunderX, which does not
> >> > tolerate non-global mappings that are executable at EL1, as this
> appears
> >> > to result in I-cache corruption. (Linux disables the KPTI based
> Meltdown
> >> > mitigation on ThunderX for the same reason)
> >> >
> >> > So work around this, by detecting the CPU implementor and part number,
> >> > and proceeding without the early ID map if a ThunderX CPU is detected.
> >> >
> >> > Note that this requires the C code to be built with strict alignment
> >> > again, as we may end up executing it with the MMU and caches off.
> >> >
> >> > Signed-off-by: Ard Biesheuvel <ardb@kernel.org>
> >> > ---
> >> >  ArmVirtPkg/ArmVirtQemu.dsc                                        |
> 5 +++++
> >> >  ArmVirtPkg/Library/ArmPlatformLibQemu/AArch64/ArmPlatformHelper.S |
> 15 +++++++++++++++
> >> >  2 files changed, 20 insertions(+)
> >>
> >> FTR, this v2 series also worked for me.
> >>
> >>   -dann
> >>
> >> > diff --git a/ArmVirtPkg/ArmVirtQemu.dsc b/ArmVirtPkg/ArmVirtQemu.dsc
> >> > index f77443229e8e..5dd8b6104cca 100644
> >> > --- a/ArmVirtPkg/ArmVirtQemu.dsc
> >> > +++ b/ArmVirtPkg/ArmVirtQemu.dsc
> >> > @@ -31,6 +31,7 @@ [Defines]
> >> >    DEFINE SECURE_BOOT_ENABLE      = FALSE
> >> >    DEFINE TPM2_ENABLE             = FALSE
> >> >    DEFINE TPM2_CONFIG_ENABLE      = FALSE
> >> > +  DEFINE CAVIUM_ERRATUM_27456    = FALSE
> >> >
> >> >    #
> >> >    # Network definition
> >> > @@ -117,7 +118,11 @@ [LibraryClasses.common.UEFI_DRIVER]
> >> >    UefiScsiLib|MdePkg/Library/UefiScsiLib/UefiScsiLib.inf
> >> >
> >> >  [BuildOptions]
> >> > +!if $(CAVIUM_ERRATUM_27456) == TRUE
> >> > +  GCC:*_*_AARCH64_PP_FLAGS = -DCAVIUM_ERRATUM_27456
> >> > +!else
> >> >    GCC:*_*_AARCH64_CC_XIPFLAGS ==
> >> > +!endif
> >> >
> >> >  !include NetworkPkg/NetworkBuildOptions.dsc.inc
> >> >
> >> > diff --git
> a/ArmVirtPkg/Library/ArmPlatformLibQemu/AArch64/ArmPlatformHelper.S
> b/ArmVirtPkg/Library/ArmPlatformLibQemu/AArch64/ArmPlatformHelper.S
> >> > index 1787d52fbf51..5ac7c732f6ec 100644
> >> > ---
> a/ArmVirtPkg/Library/ArmPlatformLibQemu/AArch64/ArmPlatformHelper.S
> >> > +++
> b/ArmVirtPkg/Library/ArmPlatformLibQemu/AArch64/ArmPlatformHelper.S
> >> > @@ -42,6 +42,21 @@
> >> >
> >> >
> >> >  ASM_FUNC(ArmPlatformPeiBootAction)
> >> > +#ifdef CAVIUM_ERRATUM_27456
> >> > +  /*
> >> > +   * On Cavium ThunderX, using non-global mappings that are
> executable at EL1
> >> > +   * results in I-cache corruption. So just avoid the early ID
> mapping there.
> >> > +   *
> >> > +   * MIDR implementor   0x43
> >> > +   * MIDR part numbers  0xA1 0xA2 (but not 0xAF)
> >> > +   */
> >> > +  mrs    x0, midr_el1            // read the MIDR into X0
> >> > +  ubfx   x1, x0, #24, #8         // grab implementor id
> >> > +  ubfx   x0, x0, #7, #9          // grab part number bits [11:3]
> >> > +  cmp    x1, #0x43               // compare implementor id
> >> > +  ccmp   x0, #0xA0 >> 3, #0, eq  // compare part# bits [11:3]
> >> > +  b.eq   0f
> >> > +#endif
> >> >    mrs    x0, CurrentEL           // check current exception level
> >> >    tbnz   x0, #3, 0f              // omit early ID map if above EL1
> >> >
>
>

[-- Attachment #2: Type: text/html, Size: 8163 bytes --]