From mboxrd@z Thu Jan 1 00:00:00 1970
Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217])
by mx.groups.io with SMTP id smtpd.web10.11265.1674033746977749589
for <devel@edk2.groups.io>;
Wed, 18 Jan 2023 01:22:27 -0800
Authentication-Results: mx.groups.io;
dkim=pass header.i=@kernel.org header.s=k20201202 header.b=EKeSkBFD;
spf=pass (domain: kernel.org, ip: 139.178.84.217, mailfrom: ardb@kernel.org)
Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140])
(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
(No client certificate requested)
by dfw.source.kernel.org (Postfix) with ESMTPS id 4B0C861739
for <devel@edk2.groups.io>; Wed, 18 Jan 2023 09:22:26 +0000 (UTC)
Received: by smtp.kernel.org (Postfix) with ESMTPSA id AB435C433EF
for <devel@edk2.groups.io>; Wed, 18 Jan 2023 09:22:25 +0000 (UTC)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
s=k20201202; t=1674033745;
bh=/CgJbNE8J4JdX7/fZcvPkrDcDONDlqYhtSWi3ydOGbg=;
h=References:In-Reply-To:From:Date:Subject:To:Cc:From;
b=EKeSkBFDCo9miDMos3chHQzSUnnf2lOE7AXB2X7GawQ0tXdJPZu5KE/RdDSsfMfTv
47ajYaYbJcSdWtoKZZ2J00pI7S65FyE6wem98xpacu/FRYgCckb8DOu3TBLRyNJW+O
NjQ4UDM/vgZL9o6EK7fyN0F9/sNZ0psQfjkwHUsVTd/oEXzOoygrjrzTwg8ZqBQtzC
pS4vVetEotlO9Evp6Ztj4t32oOZeFvPFrzhsja+J0obF3+VtrJVH1aMCRDqYLHnM6s
aBsA6i+cZk566tv0qqXaqQm5b/N6xPZELucmuCMgCCXbXLY06tYuiPu2piktRYzle7
KVPDiKifXoeQQ==
Received: by mail-lf1-f45.google.com with SMTP id b3so50825203lfv.2
for <devel@edk2.groups.io>; Wed, 18 Jan 2023 01:22:25 -0800 (PST)
X-Gm-Message-State: AFqh2koye97g/Egu8eQYGmrpYThUonn21Ki36hW1/dwRsvhaNH6kWj8z
M7TYiT1S65gWR/8r2TBOsM9Ai4utc7NQMnT/+1E=
X-Google-Smtp-Source: AMrXdXtnrYNscguDoGimoOdACx2+xHwn5nF4QEtZwcF3PB+Rf2lmpNFjaXwkESfjmB4OWA72uhkP7z0KQVTVFdqW8/4=
X-Received: by 2002:a19:c501:0:b0:4b8:9001:a694 with SMTP id
w1-20020a19c501000000b004b89001a694mr318959lfe.426.1674033743646; Wed, 18 Jan
2023 01:22:23 -0800 (PST)
MIME-Version: 1.0
References: <20230105162528.1430368-1-ardb@kernel.org> <20230105162528.1430368-2-ardb@kernel.org>
<Y7yshC6ZQNorwAwS@xps13.dannf> <CA+bRGFo7reFa0M3fFj4juUJ1dYfVrg6iDz=Q5oRhm9VaryV8Jw@mail.gmail.com>
<CAMj1kXGhaAW1h-2kbaC2jpd+a4SzamT2f39LLVnoZjFg40Z5kw@mail.gmail.com>
<CA+bRGFqTTUX9M9c-k91v1LNWJHqwGcHq_03WrKba_2rSDvmU3Q@mail.gmail.com>
<CAMj1kXEyqRpKCum67O07AXq5FHD4OZCPy9Ph8TH7PZvywN_xhQ@mail.gmail.com>
<CA+bRGFpQ_GPTHoCHA-E80B2tiuE3J7RojFwXWLryh+Q9wrtBtQ@mail.gmail.com> <CAMj1kXEDL2ZKYdTMruUwaAo-+YkYEOZXvxCAprB7JaB0Ms6wMg@mail.gmail.com>
In-Reply-To: <CAMj1kXEDL2ZKYdTMruUwaAo-+YkYEOZXvxCAprB7JaB0Ms6wMg@mail.gmail.com>
From: "Ard Biesheuvel" <ardb@kernel.org>
Date: Wed, 18 Jan 2023 10:22:12 +0100
X-Gmail-Original-Message-ID: <CAMj1kXHiBAxvNZORVNK9kezTGBCMdKY6hQFks1wiWYPy8kWPGA@mail.gmail.com>
Message-ID: <CAMj1kXHiBAxvNZORVNK9kezTGBCMdKY6hQFks1wiWYPy8kWPGA@mail.gmail.com>
Subject: Re: [edk2-devel] [PATCH v2 2/2] ArmVirtPkg/ArmVirtQemu: Avoid early ID map on ThunderX
To: Oliver Steffen <osteffen@redhat.com>
Cc: devel@edk2.groups.io, dann.frazier@canonical.com
Content-Type: text/plain; charset="UTF-8"
On Wed, 18 Jan 2023 at 09:48, Ard Biesheuvel <ardb@kernel.org> wrote:
>
> On Wed, 18 Jan 2023 at 09:28, Oliver Steffen <osteffen@redhat.com> wrote:
> >
> > Quoting Ard Biesheuvel (2023-01-18 08:34:32)
> > > On Wed, 18 Jan 2023 at 07:37, Oliver Steffen <osteffen@redhat.com> wrote:
> > > >
> > > > On Tue, Jan 17, 2023 at 3:57 PM Ard Biesheuvel <ardb@kernel.org> wrote:
> > > >>
> > > >> On Tue, 17 Jan 2023 at 13:48, Oliver Steffen <osteffen@redhat.com> wrote:
> > > >> >
> > > >> > Hi Ard, Hi everyone,
> > > >> >
> > > >> > Thanks for the work!
> > > >> >
> > > >> > But somehow this patch (as it was merged into master branch) does not
> > > >> > work for me on the ThunderX box we have.
> > > >> >
> > > >> > Any idea what could be wrong?
> > > >>
> > > >> I'm not sure I understand the question. The patch targets ThunderX,
> > > >> and you are using a ThunderX2.
> > > >>
> > > >> What were you expecting to happen, and what is happening instead?
> > > >
> > > >
> > > > Firmware does not start at all when using KVM.
> > > >
> > > > Please excuse my limited knowledge of Arm processor variants.
> > > > I assumed that ThunderX and ThunderX2 are very similar and hoped
> > > > the fix would also work for this case.
> > > >
> > > > The issue was introduced by the same commit that Dann
> > > > reported (07be1d34d95460a238fcd0f6693efb747c28b329):
> > > > "ArmVirtPkg/ArmVirtQemu: enable initial ID map at early boot".
> > > >
> > >
> > > Can you share the QEMU command line that you are using? I use a
> > > ThunderX2 basically 24/7 to do all my Linux and EDK2 development, so
> > > this change was developed on ThunderX2 and so I'm surprised you are
> > > seeing this issue.
> > >
> > > Did you try the DEBUG build as well?
> > Yes, debug is on.
> >
> > Here is what I have, trying with the master branch from just now
> > (998ebe5ca0ae5c449e83ede533bee872f97d63af):
> >
> > # make -C BaseTools && \
> > . ./edksetup.sh && \
> > build -t GCC5 -a AARCH64 \
> > -p ArmVirtPkg/ArmVirtQemu.dsc \
> > -DCAVIUM_ERRATUM_27456 \
> > -b DEBUG
> >
> > # /usr/libexec/qemu-kvm \
> > -machine accel=kvm -m 1G -boot menu=on \
> > -blockdev node-name=code,driver=file,filename="${FW_CODE_RESIZED}",read-only=on
> > \
> > -blockdev node-name=vars,driver=file,filename="${FW_VARS}" \
> > -machine pflash0=code \
> > -machine pflash1=vars \
> > -cpu max \
> > -net none \
> > -serial stdio
> >
>
> My distro does not have qemu-kvm, and using the command line above
> results in the following if i try it with qemu-system-aarch64
>
> """
> qemu-system-aarch64: No machine specified, and there is no default
> Use -machine help to list supported machines
> """
>
> unless i change it to
>
> qemu-system-aarch64 -machine virt,accel=kvm -m 1G -boot menu=on \
> -blockdev node-name=code,driver=file,filename=$HOME/bin/flash0.img,read-only=on
> \
> -blockdev node-name=vars,driver=file,filename=$HOME/bin/flash1.img \
> -machine pflash0=code \
> -machine pflash1=vars \
> -cpu max \
> -net none \
> -nographic
>
> and that works fine with my firmware build.
>
>
> > # /usr/libexec/qemu-kvm --version
> > QEMU emulator version 7.2.0 (qemu-kvm-7.2.0-3.el9)
> >
> > # uname -r
> > 5.14.0-234.el9.aarch64
> >
>
> Yeah, that is quite old. One potential issue that comes to mind here
> is the one address by the patch below
>
>
> >
> >
> > Since you have the same CPU... Might this be a bug in KVM?
> >
>
> Indeed. Could you try applying this patch?
>
> commit 406504c7b0405d74d74c15a667cd4c4620c3e7a9
> Author: Marc Zyngier <maz@kernel.org>
> Date: Tue Dec 20 14:03:52 2022 +0000
>
> KVM: arm64: Fix S1PTW handling on RO memslots
>
> Or check whether this is generally reproducible with newer kernels?
Another thing you might try:
- build the firmware with the following hunk applied
"""
diff --git a/ArmVirtPkg/Library/ArmPlatformLibQemu/AArch64/ArmPlatformHelper.S
b/ArmVirtPkg/Library/ArmPlatformLibQemu/AArch64/ArmPlatformHelper.S
index 5ac7c732f6ec..f4e1285beefc 100644
--- a/ArmVirtPkg/Library/ArmPlatformLibQemu/AArch64/ArmPlatformHelper.S
+++ b/ArmVirtPkg/Library/ArmPlatformLibQemu/AArch64/ArmPlatformHelper.S
@@ -40,6 +40,12 @@
.set sctlrval, SCTLR_ELx_M | SCTLR_ELx_C | SCTLR_ELx_SA |
SCTLR_EL1_ITD | SCTLR_EL1_SED
.set sctlrval, sctlrval | SCTLR_ELx_I | SCTLR_EL1_SPAN | SCTLR_EL1_RES1
+ .align 11
+.Lvectors:
+ .rept 16
+ .align 7
+ b .
+ .endr
ASM_FUNC(ArmPlatformPeiBootAction)
#ifdef CAVIUM_ERRATUM_27456
@@ -90,6 +96,8 @@ ASM_FUNC(ArmPlatformPeiBootAction)
msr mair_el1, x0 // set up the 1:1 mapping
msr tcr_el1, x1
msr ttbr0_el1, x2
+ adr x0, .Lvectors
+ msr vbar_el1, x0
isb
tlbi vmalle1 // invalidate any cached translations
"""
- run qemu with the -s option and let it crash
- connect with gdb and dump the exception context
target remote:1234
set radix 16
p $FAR_EL1
p $ESR_EL1
p $ELR_EL1
That should at least tell us why the crash is occurring.