* [PATCH] ArmVirtPkg/ArmVirtQemu: Avoid early ID map on ThunderX
@ 2023-01-04 17:22 Ard Biesheuvel
2023-01-05 11:46 ` Ard Biesheuvel
0 siblings, 1 reply; 4+ messages in thread
From: Ard Biesheuvel @ 2023-01-04 17:22 UTC (permalink / raw)
To: devel; +Cc: dann.frazier, Ard Biesheuvel
The early ID map used by ArmVirtQemu uses ASID scoped non-global
mappings, as this allows us to switch to the permanent ID map seamlessly
without the need for explicit TLB maintenance.
However, this triggers a known erratum on ThunderX, which does not
tolerate non-global mappings that are executable at EL1, as this appears
to result in I-cache corruption. (Linux disables the KPTI based Meltdown
mitigation on ThunderX for the same reason)
So work around this, by detecting the CPU implementor and part number,
and proceeding without the early ID map if a ThunderX CPU is detected.
Note that this requires the C code to be built with strict alignment
again, as we may end up executing it with the MMU and caches off.
Signed-off-by: Ard Biesheuvel <ardb@kernel.org>
---
ArmVirtPkg/ArmVirtQemu.dsc | 6 ++++++
ArmVirtPkg/Library/ArmPlatformLibQemu/AArch64/ArmPlatformHelper.S | 18 ++++++++++++++++++
2 files changed, 24 insertions(+)
diff --git a/ArmVirtPkg/ArmVirtQemu.dsc b/ArmVirtPkg/ArmVirtQemu.dsc
index f77443229e8e..340b36f69c2c 100644
--- a/ArmVirtPkg/ArmVirtQemu.dsc
+++ b/ArmVirtPkg/ArmVirtQemu.dsc
@@ -31,6 +31,7 @@ [Defines]
DEFINE SECURE_BOOT_ENABLE = FALSE
DEFINE TPM2_ENABLE = FALSE
DEFINE TPM2_CONFIG_ENABLE = FALSE
+ DEFINE CAVIUM_ERRATUM_27456 = FALSE
#
# Network definition
@@ -117,7 +118,12 @@ [LibraryClasses.common.UEFI_DRIVER]
UefiScsiLib|MdePkg/Library/UefiScsiLib/UefiScsiLib.inf
[BuildOptions]
+!if $(CAVIUM_ERRATUM_27456) == TRUE
+ GCC:*_*_AARCH64_CC_XIPFLAGS = -mno-strict-align
+ GCC:*_*_AARCH64_PP_FLAGS = -DCAVIUM_ERRATUM_27456
+!else
GCC:*_*_AARCH64_CC_XIPFLAGS ==
+!endif
!include NetworkPkg/NetworkBuildOptions.dsc.inc
diff --git a/ArmVirtPkg/Library/ArmPlatformLibQemu/AArch64/ArmPlatformHelper.S b/ArmVirtPkg/Library/ArmPlatformLibQemu/AArch64/ArmPlatformHelper.S
index 05ccc7f9f043..962f1ba3a4d7 100644
--- a/ArmVirtPkg/Library/ArmPlatformLibQemu/AArch64/ArmPlatformHelper.S
+++ b/ArmVirtPkg/Library/ArmPlatformLibQemu/AArch64/ArmPlatformHelper.S
@@ -44,8 +44,26 @@
ASM_FUNC(ArmPlatformPeiBootAction)
+#ifdef CAVIUM_ERRATUM_27456
+ /*
+ * On Cavium ThunderX, using non-global mappings that are executable at EL1
+ * results in I-cache corruption. So just avoid the early ID mapping there.
+ *
+ * MIDR implementor 0x43
+ * MIDR part numbers 0xA1 0xA2
+ */
+ mrs x0, midr_el1 // read the MIDR into X0
+ ubfx x1, x0, #6, #10 // grab part number bits [11:2]
+ ubfx x0, x0, #24, #8 // grab implementor id
+ mov x2, #0xA0 >> 2
+ cmp x0, #0x43 // compare implementor id
+ ccmp x1, x2, #0, eq // compare part# bits [11:2]
+ b.eq .Lreturn
+#endif
+
mrs x0, CurrentEL // check current exception level
tbz x0, #3, 0f // bail if above EL1
+.Lreturn:
ret
0:mov_i x0, mairval
--
2.39.0
^ permalink raw reply related [flat|nested] 4+ messages in thread
* Re: [PATCH] ArmVirtPkg/ArmVirtQemu: Avoid early ID map on ThunderX
2023-01-04 17:22 [PATCH] ArmVirtPkg/ArmVirtQemu: Avoid early ID map on ThunderX Ard Biesheuvel
@ 2023-01-05 11:46 ` Ard Biesheuvel
2023-01-05 18:25 ` dann frazier
0 siblings, 1 reply; 4+ messages in thread
From: Ard Biesheuvel @ 2023-01-05 11:46 UTC (permalink / raw)
To: devel; +Cc: dann.frazier
On Wed, 4 Jan 2023 at 18:23, Ard Biesheuvel <ardb@kernel.org> wrote:
>
> The early ID map used by ArmVirtQemu uses ASID scoped non-global
> mappings, as this allows us to switch to the permanent ID map seamlessly
> without the need for explicit TLB maintenance.
>
> However, this triggers a known erratum on ThunderX, which does not
> tolerate non-global mappings that are executable at EL1, as this appears
> to result in I-cache corruption. (Linux disables the KPTI based Meltdown
> mitigation on ThunderX for the same reason)
>
> So work around this, by detecting the CPU implementor and part number,
> and proceeding without the early ID map if a ThunderX CPU is detected.
>
> Note that this requires the C code to be built with strict alignment
> again, as we may end up executing it with the MMU and caches off.
>
> Signed-off-by: Ard Biesheuvel <ardb@kernel.org>
> ---
> ArmVirtPkg/ArmVirtQemu.dsc | 6 ++++++
> ArmVirtPkg/Library/ArmPlatformLibQemu/AArch64/ArmPlatformHelper.S | 18 ++++++++++++++++++
> 2 files changed, 24 insertions(+)
>
> diff --git a/ArmVirtPkg/ArmVirtQemu.dsc b/ArmVirtPkg/ArmVirtQemu.dsc
> index f77443229e8e..340b36f69c2c 100644
> --- a/ArmVirtPkg/ArmVirtQemu.dsc
> +++ b/ArmVirtPkg/ArmVirtQemu.dsc
> @@ -31,6 +31,7 @@ [Defines]
> DEFINE SECURE_BOOT_ENABLE = FALSE
> DEFINE TPM2_ENABLE = FALSE
> DEFINE TPM2_CONFIG_ENABLE = FALSE
> + DEFINE CAVIUM_ERRATUM_27456 = FALSE
>
> #
> # Network definition
> @@ -117,7 +118,12 @@ [LibraryClasses.common.UEFI_DRIVER]
> UefiScsiLib|MdePkg/Library/UefiScsiLib/UefiScsiLib.inf
>
> [BuildOptions]
> +!if $(CAVIUM_ERRATUM_27456) == TRUE
> + GCC:*_*_AARCH64_CC_XIPFLAGS = -mno-strict-align
This is wrong - this should be '-mstrict-align'
> + GCC:*_*_AARCH64_PP_FLAGS = -DCAVIUM_ERRATUM_27456
> +!else
> GCC:*_*_AARCH64_CC_XIPFLAGS ==
> +!endif
>
> !include NetworkPkg/NetworkBuildOptions.dsc.inc
>
> diff --git a/ArmVirtPkg/Library/ArmPlatformLibQemu/AArch64/ArmPlatformHelper.S b/ArmVirtPkg/Library/ArmPlatformLibQemu/AArch64/ArmPlatformHelper.S
> index 05ccc7f9f043..962f1ba3a4d7 100644
> --- a/ArmVirtPkg/Library/ArmPlatformLibQemu/AArch64/ArmPlatformHelper.S
> +++ b/ArmVirtPkg/Library/ArmPlatformLibQemu/AArch64/ArmPlatformHelper.S
> @@ -44,8 +44,26 @@
>
>
> ASM_FUNC(ArmPlatformPeiBootAction)
> +#ifdef CAVIUM_ERRATUM_27456
> + /*
> + * On Cavium ThunderX, using non-global mappings that are executable at EL1
> + * results in I-cache corruption. So just avoid the early ID mapping there.
> + *
> + * MIDR implementor 0x43
> + * MIDR part numbers 0xA1 0xA2
> + */
> + mrs x0, midr_el1 // read the MIDR into X0
> + ubfx x1, x0, #6, #10 // grab part number bits [11:2]
> + ubfx x0, x0, #24, #8 // grab implementor id
> + mov x2, #0xA0 >> 2
> + cmp x0, #0x43 // compare implementor id
> + ccmp x1, x2, #0, eq // compare part# bits [11:2]
> + b.eq .Lreturn
> +#endif
> +
> mrs x0, CurrentEL // check current exception level
> tbz x0, #3, 0f // bail if above EL1
> +.Lreturn:
> ret
>
> 0:mov_i x0, mairval
> --
> 2.39.0
>
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: [PATCH] ArmVirtPkg/ArmVirtQemu: Avoid early ID map on ThunderX
2023-01-05 11:46 ` Ard Biesheuvel
@ 2023-01-05 18:25 ` dann frazier
2023-01-06 6:51 ` [edk2-devel] " Laszlo Ersek
0 siblings, 1 reply; 4+ messages in thread
From: dann frazier @ 2023-01-05 18:25 UTC (permalink / raw)
To: Ard Biesheuvel; +Cc: devel
On Thu, Jan 05, 2023 at 12:46:19PM +0100, Ard Biesheuvel wrote:
> On Wed, 4 Jan 2023 at 18:23, Ard Biesheuvel <ardb@kernel.org> wrote:
> >
> > The early ID map used by ArmVirtQemu uses ASID scoped non-global
> > mappings, as this allows us to switch to the permanent ID map seamlessly
> > without the need for explicit TLB maintenance.
> >
> > However, this triggers a known erratum on ThunderX, which does not
> > tolerate non-global mappings that are executable at EL1, as this appears
> > to result in I-cache corruption. (Linux disables the KPTI based Meltdown
> > mitigation on ThunderX for the same reason)
> >
> > So work around this, by detecting the CPU implementor and part number,
> > and proceeding without the early ID map if a ThunderX CPU is detected.
> >
> > Note that this requires the C code to be built with strict alignment
> > again, as we may end up executing it with the MMU and caches off.
> >
> > Signed-off-by: Ard Biesheuvel <ardb@kernel.org>
> > ---
> > ArmVirtPkg/ArmVirtQemu.dsc | 6 ++++++
> > ArmVirtPkg/Library/ArmPlatformLibQemu/AArch64/ArmPlatformHelper.S | 18 ++++++++++++++++++
> > 2 files changed, 24 insertions(+)
> >
> > diff --git a/ArmVirtPkg/ArmVirtQemu.dsc b/ArmVirtPkg/ArmVirtQemu.dsc
> > index f77443229e8e..340b36f69c2c 100644
> > --- a/ArmVirtPkg/ArmVirtQemu.dsc
> > +++ b/ArmVirtPkg/ArmVirtQemu.dsc
> > @@ -31,6 +31,7 @@ [Defines]
> > DEFINE SECURE_BOOT_ENABLE = FALSE
> > DEFINE TPM2_ENABLE = FALSE
> > DEFINE TPM2_CONFIG_ENABLE = FALSE
> > + DEFINE CAVIUM_ERRATUM_27456 = FALSE
> >
> > #
> > # Network definition
> > @@ -117,7 +118,12 @@ [LibraryClasses.common.UEFI_DRIVER]
> > UefiScsiLib|MdePkg/Library/UefiScsiLib/UefiScsiLib.inf
> >
> > [BuildOptions]
> > +!if $(CAVIUM_ERRATUM_27456) == TRUE
> > + GCC:*_*_AARCH64_CC_XIPFLAGS = -mno-strict-align
>
> This is wrong - this should be '-mstrict-align'
Ah, I wondered :) With that adjustment, the patch works for me on a
Cavium ThunderX. Thanks Ard!
-dann
> > + GCC:*_*_AARCH64_PP_FLAGS = -DCAVIUM_ERRATUM_27456
> > +!else
> > GCC:*_*_AARCH64_CC_XIPFLAGS ==
> > +!endif
> >
> > !include NetworkPkg/NetworkBuildOptions.dsc.inc
> >
> > diff --git a/ArmVirtPkg/Library/ArmPlatformLibQemu/AArch64/ArmPlatformHelper.S b/ArmVirtPkg/Library/ArmPlatformLibQemu/AArch64/ArmPlatformHelper.S
> > index 05ccc7f9f043..962f1ba3a4d7 100644
> > --- a/ArmVirtPkg/Library/ArmPlatformLibQemu/AArch64/ArmPlatformHelper.S
> > +++ b/ArmVirtPkg/Library/ArmPlatformLibQemu/AArch64/ArmPlatformHelper.S
> > @@ -44,8 +44,26 @@
> >
> >
> > ASM_FUNC(ArmPlatformPeiBootAction)
> > +#ifdef CAVIUM_ERRATUM_27456
> > + /*
> > + * On Cavium ThunderX, using non-global mappings that are executable at EL1
> > + * results in I-cache corruption. So just avoid the early ID mapping there.
> > + *
> > + * MIDR implementor 0x43
> > + * MIDR part numbers 0xA1 0xA2
> > + */
> > + mrs x0, midr_el1 // read the MIDR into X0
> > + ubfx x1, x0, #6, #10 // grab part number bits [11:2]
> > + ubfx x0, x0, #24, #8 // grab implementor id
> > + mov x2, #0xA0 >> 2
> > + cmp x0, #0x43 // compare implementor id
> > + ccmp x1, x2, #0, eq // compare part# bits [11:2]
> > + b.eq .Lreturn
> > +#endif
> > +
> > mrs x0, CurrentEL // check current exception level
> > tbz x0, #3, 0f // bail if above EL1
> > +.Lreturn:
> > ret
> >
> > 0:mov_i x0, mairval
> >
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: [edk2-devel] [PATCH] ArmVirtPkg/ArmVirtQemu: Avoid early ID map on ThunderX
2023-01-05 18:25 ` dann frazier
@ 2023-01-06 6:51 ` Laszlo Ersek
0 siblings, 0 replies; 4+ messages in thread
From: Laszlo Ersek @ 2023-01-06 6:51 UTC (permalink / raw)
To: devel, dann.frazier, Ard Biesheuvel
On 1/5/23 19:25, dann frazier wrote:
> On Thu, Jan 05, 2023 at 12:46:19PM +0100, Ard Biesheuvel wrote:
>> On Wed, 4 Jan 2023 at 18:23, Ard Biesheuvel <ardb@kernel.org> wrote:
>>>
>>> The early ID map used by ArmVirtQemu uses ASID scoped non-global
>>> mappings, as this allows us to switch to the permanent ID map seamlessly
>>> without the need for explicit TLB maintenance.
>>>
>>> However, this triggers a known erratum on ThunderX, which does not
>>> tolerate non-global mappings that are executable at EL1, as this appears
>>> to result in I-cache corruption. (Linux disables the KPTI based Meltdown
>>> mitigation on ThunderX for the same reason)
>>>
>>> So work around this, by detecting the CPU implementor and part number,
>>> and proceeding without the early ID map if a ThunderX CPU is detected.
>>>
>>> Note that this requires the C code to be built with strict alignment
>>> again, as we may end up executing it with the MMU and caches off.
>>>
>>> Signed-off-by: Ard Biesheuvel <ardb@kernel.org>
>>> ---
>>> ArmVirtPkg/ArmVirtQemu.dsc | 6 ++++++
>>> ArmVirtPkg/Library/ArmPlatformLibQemu/AArch64/ArmPlatformHelper.S | 18 ++++++++++++++++++
>>> 2 files changed, 24 insertions(+)
>>>
>>> diff --git a/ArmVirtPkg/ArmVirtQemu.dsc b/ArmVirtPkg/ArmVirtQemu.dsc
>>> index f77443229e8e..340b36f69c2c 100644
>>> --- a/ArmVirtPkg/ArmVirtQemu.dsc
>>> +++ b/ArmVirtPkg/ArmVirtQemu.dsc
>>> @@ -31,6 +31,7 @@ [Defines]
>>> DEFINE SECURE_BOOT_ENABLE = FALSE
>>> DEFINE TPM2_ENABLE = FALSE
>>> DEFINE TPM2_CONFIG_ENABLE = FALSE
>>> + DEFINE CAVIUM_ERRATUM_27456 = FALSE
>>>
>>> #
>>> # Network definition
>>> @@ -117,7 +118,12 @@ [LibraryClasses.common.UEFI_DRIVER]
>>> UefiScsiLib|MdePkg/Library/UefiScsiLib/UefiScsiLib.inf
>>>
>>> [BuildOptions]
>>> +!if $(CAVIUM_ERRATUM_27456) == TRUE
>>> + GCC:*_*_AARCH64_CC_XIPFLAGS = -mno-strict-align
>>
>> This is wrong - this should be '-mstrict-align'
>
> Ah, I wondered :) With that adjustment, the patch works for me on a
> Cavium ThunderX. Thanks Ard!
>
With the typo fixed:
Acked-by: Laszlo Ersek <lersek@redhat.com>
> -dann
>
>>> + GCC:*_*_AARCH64_PP_FLAGS = -DCAVIUM_ERRATUM_27456
>>> +!else
>>> GCC:*_*_AARCH64_CC_XIPFLAGS ==
>>> +!endif
>>>
>>> !include NetworkPkg/NetworkBuildOptions.dsc.inc
>>>
>>> diff --git a/ArmVirtPkg/Library/ArmPlatformLibQemu/AArch64/ArmPlatformHelper.S b/ArmVirtPkg/Library/ArmPlatformLibQemu/AArch64/ArmPlatformHelper.S
>>> index 05ccc7f9f043..962f1ba3a4d7 100644
>>> --- a/ArmVirtPkg/Library/ArmPlatformLibQemu/AArch64/ArmPlatformHelper.S
>>> +++ b/ArmVirtPkg/Library/ArmPlatformLibQemu/AArch64/ArmPlatformHelper.S
>>> @@ -44,8 +44,26 @@
>>>
>>>
>>> ASM_FUNC(ArmPlatformPeiBootAction)
>>> +#ifdef CAVIUM_ERRATUM_27456
>>> + /*
>>> + * On Cavium ThunderX, using non-global mappings that are executable at EL1
>>> + * results in I-cache corruption. So just avoid the early ID mapping there.
>>> + *
>>> + * MIDR implementor 0x43
>>> + * MIDR part numbers 0xA1 0xA2
>>> + */
>>> + mrs x0, midr_el1 // read the MIDR into X0
>>> + ubfx x1, x0, #6, #10 // grab part number bits [11:2]
>>> + ubfx x0, x0, #24, #8 // grab implementor id
>>> + mov x2, #0xA0 >> 2
>>> + cmp x0, #0x43 // compare implementor id
>>> + ccmp x1, x2, #0, eq // compare part# bits [11:2]
>>> + b.eq .Lreturn
>>> +#endif
>>> +
>>> mrs x0, CurrentEL // check current exception level
>>> tbz x0, #3, 0f // bail if above EL1
>>> +.Lreturn:
>>> ret
>>>
>>> 0:mov_i x0, mairval
>>>
>
>
>
>
>
^ permalink raw reply [flat|nested] 4+ messages in thread
end of thread, other threads:[~2023-01-06 6:51 UTC | newest]
Thread overview: 4+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2023-01-04 17:22 [PATCH] ArmVirtPkg/ArmVirtQemu: Avoid early ID map on ThunderX Ard Biesheuvel
2023-01-05 11:46 ` Ard Biesheuvel
2023-01-05 18:25 ` dann frazier
2023-01-06 6:51 ` [edk2-devel] " Laszlo Ersek
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox