From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from smtp-relay-internal-0.canonical.com (smtp-relay-internal-0.canonical.com [185.125.188.122]) by mx.groups.io with SMTP id smtpd.web10.3349.1672774714959480475 for ; Tue, 03 Jan 2023 11:38:35 -0800 Authentication-Results: mx.groups.io; dkim=pass header.i=@canonical.com header.s=20210705 header.b=UznDwwMr; spf=pass (domain: canonical.com, ip: 185.125.188.122, mailfrom: dann.frazier@canonical.com) Received: from mail-io1-f70.google.com (mail-io1-f70.google.com [209.85.166.70]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by smtp-relay-internal-0.canonical.com (Postfix) with ESMTPS id 3A98B445A2 for ; Tue, 3 Jan 2023 19:38:32 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=canonical.com; s=20210705; t=1672774712; bh=sX7I9KWZZvow3CWBa5VsDXek8n/Gj/ShXezfv7L3Xpo=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:In-Reply-To; b=UznDwwMrNrLkoqi7os0dhu9NODFpQFEz0Bsl1uoom74rta7WizHjISm4s3OYbtfMB aBnwdIIzdpnD0l1jod8pxzY8U+LAdqqBNaPHIhJHhUESpm/OyC0ZiSTu0ob3FVa2f6 0n5hd+Lx0eiG2BxzXz1VDfjN0XN0vQLbhPAtquZdEP0Ma3S6H/P8SPg98pw8eJgEC0 hg1E/f8/FKVt8GzCuPAWr+dFngFH6LUuMf2KmlXUR/p4OpOx5ExbuRw/HeQqrmzEGx HGgp+dJ63uAZ1btPX5sSf3cq/S6Zt7VHGlevBu5M517ag9NacoCwvFBXWr96NtaYIg o9HmHRDqHPz0A== Received: by mail-io1-f70.google.com with SMTP id m5-20020a6b7f45000000b006fc1dded1b9so1613307ioq.18 for ; Tue, 03 Jan 2023 11:38:32 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=sX7I9KWZZvow3CWBa5VsDXek8n/Gj/ShXezfv7L3Xpo=; b=o3Y534DQtvp3+C/TpMjD3W+7sJx5/Ga7IKJuTHIqF7QwTr9t8aL2HxmFR40Z04LY40 FHg2VzyHGP/UZE07OWTeSwrO1molLvNblnkK46MrhSpNzjidj0DqS6rc5J+FuOUxmARn +xviEM1FnNHWBay8yqHUX84+kzjUIJXLTzcFYQwR3bL9yBIuMQUzk6SBZJCyF3HPYxii jopAUcXc92/4dz+mHArcjebUJ4Dw6DLLMuNIFWrOQpwZQTX9EWEuAAE7spoMsinLEnvY Q6RbaS0Mz+Fe8h/cSvyhf/MTtBr8Vf86dfyqDSnoFYFYQfcy/N0iQYLh+1X4r45EM+e2 ozug== X-Gm-Message-State: AFqh2kqEHhoIUm6X+yqI9INAp/umhP3Vc1IWHwieghRORRkan5UfF5Au BPBaYE+PZvVxaQFrsb8ZePQ7SF5Hj0EzX0pp9yGIrCBqA4Fg/RB/lwimIlW1mgGDsPV6hSB3i62 fKGAHvsjr2nDu7RrjMG3bkdnhVxwWew4= X-Received: by 2002:a92:dd09:0:b0:30c:267e:dba6 with SMTP id n9-20020a92dd09000000b0030c267edba6mr12197776ilm.27.1672774711051; Tue, 03 Jan 2023 11:38:31 -0800 (PST) X-Google-Smtp-Source: AMrXdXvD3yjZwYyV3vKLf0wgJjfXgIMtBt3reM1WxZMsWa+q+nOUClItPrzP9LA9gS7omeOGZfGUBg== X-Received: by 2002:a92:dd09:0:b0:30c:267e:dba6 with SMTP id n9-20020a92dd09000000b0030c267edba6mr12197770ilm.27.1672774710698; Tue, 03 Jan 2023 11:38:30 -0800 (PST) Received: from xps13.dannf ([38.15.56.166]) by smtp.gmail.com with ESMTPSA id e1-20020a92d741000000b00302a7165d9bsm9989003ilq.53.2023.01.03.11.38.30 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 03 Jan 2023 11:38:30 -0800 (PST) Date: Tue, 3 Jan 2023 12:38:28 -0700 From: "dann frazier" To: Ard Biesheuvel Cc: devel@edk2.groups.io, Leif Lindholm , Alexander Graf Subject: Re: [edk2-devel] [PATCH v3 12/16] ArmVirtPkg/ArmVirtQemu: enable initial ID map at early boot Message-ID: References: <20220926082511.2110797-1-ardb@kernel.org> <20220926082511.2110797-13-ardb@kernel.org> MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=us-ascii Content-Disposition: inline On Tue, Jan 03, 2023 at 10:02:27AM +0100, Ard Biesheuvel wrote: > On Thu, 29 Dec 2022 at 22:10, dann frazier wrote: > > > > On Mon, Sep 26, 2022 at 10:25:07AM +0200, Ard Biesheuvel wrote: > > > Now that we have all the pieces in place, switch the AArch64 version of > > > ArmVirtQemu to a mode where the first thing it does out of reset is > > > enable a preliminary ID map that covers the NOR flash and sufficient > > > DRAM to create the UEFI page tables as usual. > > > > > > The advantage of this is that no manipulation of memory occurs any > > > longer before the MMU is enabled, which removes the need for explicit > > > coherency management, which is cumbersome and bad for performance. > > > > > > It also means we no longer need to build all components that may execute > > > with the MMU off (including BASE libraries) with strict alignment. > > > > After this switch, I'm seeing a Synchronous Exception when launching a > > VM, though only on old Cavium ThunderX (CN88XX) systems. I used print > > debugging to narrow it down to ArmSetTTBR0(). Initially I thought it > > might be related to Cavium Erratum 27456, but that doesn't seem to > > make sense because the instruction cache isn't enabled until > > later. I tried implementing the same workaround as Linux does anyway > > (flush caches after the setting ttbr0) without any luck. > > > > Any idea what is going on there? > > > > I suspect it is in fact the same erratum - the I-cache does get > enabled almost immediately after reset, and the use of ASIDs for EL1 > mappings looks suspiciously like failure mode that required us to > disable KPTI for ThunderX on Linux. > > It is a bit disappointing to have to add workarounds for obsolete > platforms in new code, so if I fix this, it will be gated by a -D > build option - is that acceptable to you? It is. fwiw, here's what I tried w/o luck: diff --git a/ArmPkg/Library/ArmLib/AArch64/ArmLibSupport.S b/ArmPkg/Library/ArmLib/AArch64/ArmLibSupport.S index ba0ec5682b..1a94a9782c 100644 --- a/ArmPkg/Library/ArmLib/AArch64/ArmLibSupport.S +++ b/ArmPkg/Library/ArmLib/AArch64/ArmLibSupport.S @@ -56,10 +56,30 @@ ASM_FUNC(ArmReadAuxCr) ASM_FUNC(ArmSetTTBR0) EL1_OR_EL2_OR_EL3(x1) 1:msr ttbr0_el1, x0 // Translation Table Base Reg 0 (TTBR0) + isb + nop + nop + nop + ic iallu + dsb nsh + isb b 4f 2:msr ttbr0_el2, x0 // Translation Table Base Reg 0 (TTBR0) + isb + nop + nop + nop + ic iallu + dsb nsh + isb b 4f 3:msr ttbr0_el3, x0 // Translation Table Base Reg 0 (TTBR0) + isb + nop + nop + nop + ic iallu + dsb nsh 4:isb ret