From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by mx.groups.io with SMTP id smtpd.web10.11265.1674033746977749589 for ; Wed, 18 Jan 2023 01:22:27 -0800 Authentication-Results: mx.groups.io; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=EKeSkBFD; spf=pass (domain: kernel.org, ip: 139.178.84.217, mailfrom: ardb@kernel.org) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 4B0C861739 for ; Wed, 18 Jan 2023 09:22:26 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id AB435C433EF for ; Wed, 18 Jan 2023 09:22:25 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1674033745; bh=/CgJbNE8J4JdX7/fZcvPkrDcDONDlqYhtSWi3ydOGbg=; h=References:In-Reply-To:From:Date:Subject:To:Cc:From; b=EKeSkBFDCo9miDMos3chHQzSUnnf2lOE7AXB2X7GawQ0tXdJPZu5KE/RdDSsfMfTv 47ajYaYbJcSdWtoKZZ2J00pI7S65FyE6wem98xpacu/FRYgCckb8DOu3TBLRyNJW+O NjQ4UDM/vgZL9o6EK7fyN0F9/sNZ0psQfjkwHUsVTd/oEXzOoygrjrzTwg8ZqBQtzC pS4vVetEotlO9Evp6Ztj4t32oOZeFvPFrzhsja+J0obF3+VtrJVH1aMCRDqYLHnM6s aBsA6i+cZk566tv0qqXaqQm5b/N6xPZELucmuCMgCCXbXLY06tYuiPu2piktRYzle7 KVPDiKifXoeQQ== Received: by mail-lf1-f45.google.com with SMTP id b3so50825203lfv.2 for ; Wed, 18 Jan 2023 01:22:25 -0800 (PST) X-Gm-Message-State: AFqh2koye97g/Egu8eQYGmrpYThUonn21Ki36hW1/dwRsvhaNH6kWj8z M7TYiT1S65gWR/8r2TBOsM9Ai4utc7NQMnT/+1E= X-Google-Smtp-Source: AMrXdXtnrYNscguDoGimoOdACx2+xHwn5nF4QEtZwcF3PB+Rf2lmpNFjaXwkESfjmB4OWA72uhkP7z0KQVTVFdqW8/4= X-Received: by 2002:a19:c501:0:b0:4b8:9001:a694 with SMTP id w1-20020a19c501000000b004b89001a694mr318959lfe.426.1674033743646; Wed, 18 Jan 2023 01:22:23 -0800 (PST) MIME-Version: 1.0 References: <20230105162528.1430368-1-ardb@kernel.org> <20230105162528.1430368-2-ardb@kernel.org> In-Reply-To: From: "Ard Biesheuvel" Date: Wed, 18 Jan 2023 10:22:12 +0100 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: [edk2-devel] [PATCH v2 2/2] ArmVirtPkg/ArmVirtQemu: Avoid early ID map on ThunderX To: Oliver Steffen Cc: devel@edk2.groups.io, dann.frazier@canonical.com Content-Type: text/plain; charset="UTF-8" On Wed, 18 Jan 2023 at 09:48, Ard Biesheuvel wrote: > > On Wed, 18 Jan 2023 at 09:28, Oliver Steffen wrote: > > > > Quoting Ard Biesheuvel (2023-01-18 08:34:32) > > > On Wed, 18 Jan 2023 at 07:37, Oliver Steffen wrote: > > > > > > > > On Tue, Jan 17, 2023 at 3:57 PM Ard Biesheuvel wrote: > > > >> > > > >> On Tue, 17 Jan 2023 at 13:48, Oliver Steffen wrote: > > > >> > > > > >> > Hi Ard, Hi everyone, > > > >> > > > > >> > Thanks for the work! > > > >> > > > > >> > But somehow this patch (as it was merged into master branch) does not > > > >> > work for me on the ThunderX box we have. > > > >> > > > > >> > Any idea what could be wrong? > > > >> > > > >> I'm not sure I understand the question. The patch targets ThunderX, > > > >> and you are using a ThunderX2. > > > >> > > > >> What were you expecting to happen, and what is happening instead? > > > > > > > > > > > > Firmware does not start at all when using KVM. > > > > > > > > Please excuse my limited knowledge of Arm processor variants. > > > > I assumed that ThunderX and ThunderX2 are very similar and hoped > > > > the fix would also work for this case. > > > > > > > > The issue was introduced by the same commit that Dann > > > > reported (07be1d34d95460a238fcd0f6693efb747c28b329): > > > > "ArmVirtPkg/ArmVirtQemu: enable initial ID map at early boot". > > > > > > > > > > Can you share the QEMU command line that you are using? I use a > > > ThunderX2 basically 24/7 to do all my Linux and EDK2 development, so > > > this change was developed on ThunderX2 and so I'm surprised you are > > > seeing this issue. > > > > > > Did you try the DEBUG build as well? > > Yes, debug is on. > > > > Here is what I have, trying with the master branch from just now > > (998ebe5ca0ae5c449e83ede533bee872f97d63af): > > > > # make -C BaseTools && \ > > . ./edksetup.sh && \ > > build -t GCC5 -a AARCH64 \ > > -p ArmVirtPkg/ArmVirtQemu.dsc \ > > -DCAVIUM_ERRATUM_27456 \ > > -b DEBUG > > > > # /usr/libexec/qemu-kvm \ > > -machine accel=kvm -m 1G -boot menu=on \ > > -blockdev node-name=code,driver=file,filename="${FW_CODE_RESIZED}",read-only=on > > \ > > -blockdev node-name=vars,driver=file,filename="${FW_VARS}" \ > > -machine pflash0=code \ > > -machine pflash1=vars \ > > -cpu max \ > > -net none \ > > -serial stdio > > > > My distro does not have qemu-kvm, and using the command line above > results in the following if i try it with qemu-system-aarch64 > > """ > qemu-system-aarch64: No machine specified, and there is no default > Use -machine help to list supported machines > """ > > unless i change it to > > qemu-system-aarch64 -machine virt,accel=kvm -m 1G -boot menu=on \ > -blockdev node-name=code,driver=file,filename=$HOME/bin/flash0.img,read-only=on > \ > -blockdev node-name=vars,driver=file,filename=$HOME/bin/flash1.img \ > -machine pflash0=code \ > -machine pflash1=vars \ > -cpu max \ > -net none \ > -nographic > > and that works fine with my firmware build. > > > > # /usr/libexec/qemu-kvm --version > > QEMU emulator version 7.2.0 (qemu-kvm-7.2.0-3.el9) > > > > # uname -r > > 5.14.0-234.el9.aarch64 > > > > Yeah, that is quite old. One potential issue that comes to mind here > is the one address by the patch below > > > > > > > > Since you have the same CPU... Might this be a bug in KVM? > > > > Indeed. Could you try applying this patch? > > commit 406504c7b0405d74d74c15a667cd4c4620c3e7a9 > Author: Marc Zyngier > Date: Tue Dec 20 14:03:52 2022 +0000 > > KVM: arm64: Fix S1PTW handling on RO memslots > > Or check whether this is generally reproducible with newer kernels? Another thing you might try: - build the firmware with the following hunk applied """ diff --git a/ArmVirtPkg/Library/ArmPlatformLibQemu/AArch64/ArmPlatformHelper.S b/ArmVirtPkg/Library/ArmPlatformLibQemu/AArch64/ArmPlatformHelper.S index 5ac7c732f6ec..f4e1285beefc 100644 --- a/ArmVirtPkg/Library/ArmPlatformLibQemu/AArch64/ArmPlatformHelper.S +++ b/ArmVirtPkg/Library/ArmPlatformLibQemu/AArch64/ArmPlatformHelper.S @@ -40,6 +40,12 @@ .set sctlrval, SCTLR_ELx_M | SCTLR_ELx_C | SCTLR_ELx_SA | SCTLR_EL1_ITD | SCTLR_EL1_SED .set sctlrval, sctlrval | SCTLR_ELx_I | SCTLR_EL1_SPAN | SCTLR_EL1_RES1 + .align 11 +.Lvectors: + .rept 16 + .align 7 + b . + .endr ASM_FUNC(ArmPlatformPeiBootAction) #ifdef CAVIUM_ERRATUM_27456 @@ -90,6 +96,8 @@ ASM_FUNC(ArmPlatformPeiBootAction) msr mair_el1, x0 // set up the 1:1 mapping msr tcr_el1, x1 msr ttbr0_el1, x2 + adr x0, .Lvectors + msr vbar_el1, x0 isb tlbi vmalle1 // invalidate any cached translations """ - run qemu with the -s option and let it crash - connect with gdb and dump the exception context target remote:1234 set radix 16 p $FAR_EL1 p $ESR_EL1 p $ELR_EL1 That should at least tell us why the crash is occurring.