From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-wr1-f65.google.com (mail-wr1-f65.google.com [209.85.221.65]) by mx.groups.io with SMTP id smtpd.web12.5891.1582711448503960332 for ; Wed, 26 Feb 2020 02:04:08 -0800 Authentication-Results: mx.groups.io; dkim=pass header.i=@linaro.org header.s=google header.b=g9SmQ3/8; spf=pass (domain: linaro.org, ip: 209.85.221.65, mailfrom: ard.biesheuvel@linaro.org) Received: by mail-wr1-f65.google.com with SMTP id j7so2129504wrp.13 for ; Wed, 26 Feb 2020 02:04:08 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=76Ip4wsNqEsXmDAe/sMKuaqziD2hQ2DrzQ895hO8qB4=; b=g9SmQ3/8nSUNhkAbXOK6POQS/AFSyygDCGzZf4pCTg5/vbIBpz1GcfozpBC2NwvtiY phhXa+zXCuDNTdMpBUmewYBccoerKE5V76hKJe85lbFQdYGwngOSUovHt7UGe3XmRVy5 SJNclHN/5kGKbfcTpIx62wbIMrzGz1Vf1dchy8rhfUO19xfdzHyogP4FQGF1kB8QLSAx Sgfw2ENDzn2ppSHHURzi4TnWboFE8GHsBZG6OmVHZ4RQLF8N0ea7YFx9niLXiWi6gf3F iyXqpOAqNlMINHM7IY+TVBMKy3Jhld4hqrXpRFH+GE63UZL3jZzqLWiA3p71i03G63Fe vKbg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=76Ip4wsNqEsXmDAe/sMKuaqziD2hQ2DrzQ895hO8qB4=; b=s3+u36CF8QwaO5S2cfA8/fl4a7mwKXxZelyzkML4UcIiRZr0hf0OC/9arZoWO2pnSe 6+wEmyyHkPQWXXusSSpIhM/9nnpourSa/sp9X5UL+Zi4Hm7Nt7evY/tffR20bENnaa7e ri94p5cmzBPjzNSRycPu2eLd/C03euPwZAhudbd0OXLJPnaAmjYvpwfD1NGZzeJCbO8C bOvvVRv4VT9vl36x2cRKg+z1eBmO1ljOfiEUxvQVJJL7N75LojIkleL49FcBCNZRGdFy iNgvgSqfE+P2wNoNV/iyunT+sazPM+wGNC0h7QjQ7qfHYjbLyYU/TVJnfPpgN3w1F1AR BT1g== X-Gm-Message-State: APjAAAXBHfaCrc8Nvm3BzBrr+wO2vA4VHsW7kK2NbsCmdkxnI6TUguPL K0lqdiCNRKnfqniCIEM64ZjpmrrFJooLzg== X-Google-Smtp-Source: APXvYqzqnkveVQCGExq5jKvxsLuh2LEWagFhfqdzZnJSqVkR7VcPlX+Fv+Huvnxxb/NnVzd1Kv+yFA== X-Received: by 2002:a05:6000:192:: with SMTP id p18mr4560335wrx.218.1582711446622; Wed, 26 Feb 2020 02:04:06 -0800 (PST) Return-Path: Received: from e123331-lin.home ([2a01:cb1d:112:6f00:816e:ff0d:fb69:f613]) by smtp.gmail.com with ESMTPSA id b10sm2559159wrw.61.2020.02.26.02.04.05 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 26 Feb 2020 02:04:05 -0800 (PST) From: "Ard Biesheuvel" To: devel@edk2.groups.io Cc: leif@nuviainc.com, lersek@redhat.com, sami.mujawar@arm.com, Ard Biesheuvel Subject: [PATCH 4/6] ArmPkg/ArmMmuLib AARCH64: cache-invalidate initial page table entries Date: Wed, 26 Feb 2020 11:03:51 +0100 Message-Id: <20200226100353.31962-5-ard.biesheuvel@linaro.org> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20200226100353.31962-1-ard.biesheuvel@linaro.org> References: <20200226100353.31962-1-ard.biesheuvel@linaro.org> In the AARCH64 version of ArmMmuLib, we are currently relying on set/way invalidation to ensure that the caches are in a consistent state with respect to main memory once we turn the MMU on. Even if set/way operations were the appropriate method to achieve this, doing an invalidate-all first and then populating the page table entries creates a window where page table entries could be loaded speculatively into the caches before we modify them, and shadow the new values that we write there. So let's get rid of the blanket clean/invalidate operations, and instead, update ArmUpdateTranslationTableEntry () to invalidate each page table entry *after* it is written if the MMU is still disabled at this point. On ARMv8, it is guaranteed that memory accesses done by the page table walker are cache coherent, and so we can ignore the case where the MMU is on. Since the MMU and D-cache are already off when we reach this point, we can drop the MMU and D-cache disables as well. Maintenance of the I-cache is unnecessary, since we are not modifying any code, and the installed mapping is guaranteed to be 1:1. This means we can also leave it enabled while the page table population code is running. Signed-off-by: Ard Biesheuvel --- ArmPkg/Library/ArmLib/AArch64/ArmLibSupport.S | 9 ++++++++- ArmPkg/Library/ArmMmuLib/AArch64/ArmMmuLibCore.c | 9 --------- 2 files changed, 8 insertions(+), 10 deletions(-) diff --git a/ArmPkg/Library/ArmLib/AArch64/ArmLibSupport.S b/ArmPkg/Library/ArmLib/AArch64/ArmLibSupport.S index 1adf960377a2..f744cd6738b9 100644 --- a/ArmPkg/Library/ArmLib/AArch64/ArmLibSupport.S +++ b/ArmPkg/Library/ArmLib/AArch64/ArmLibSupport.S @@ -13,6 +13,8 @@ .set DAIF_RD_FIQ_BIT, (1 << 6) .set DAIF_RD_IRQ_BIT, (1 << 7) +.set SCTLR_ELx_M_BIT_POS, (0) + ASM_FUNC(ArmReadMidr) mrs x0, midr_el1 // Read from Main ID Register (MIDR) ret @@ -122,11 +124,16 @@ ASM_FUNC(ArmUpdateTranslationTableEntry) lsr x1, x1, #12 EL1_OR_EL2_OR_EL3(x0) 1: tlbi vaae1, x1 // TLB Invalidate VA , EL1 + mrs x2, sctlr_el1 b 4f 2: tlbi vae2, x1 // TLB Invalidate VA , EL2 + mrs x2, sctlr_el2 b 4f 3: tlbi vae3, x1 // TLB Invalidate VA , EL3 -4: dsb nsh + mrs x2, sctlr_el3 +4: tbnz x2, SCTLR_ELx_M_BIT_POS, 5f + dc ivac, x0 // invalidate in Dcache if MMU is still off +5: dsb nsh isb ret diff --git a/ArmPkg/Library/ArmMmuLib/AArch64/ArmMmuLibCore.c b/ArmPkg/Library/ArmMmuLib/AArch64/ArmMmuLibCore.c index e8f5c69e3136..204e33c75f95 100644 --- a/ArmPkg/Library/ArmMmuLib/AArch64/ArmMmuLibCore.c +++ b/ArmPkg/Library/ArmMmuLib/AArch64/ArmMmuLibCore.c @@ -699,15 +699,6 @@ ArmConfigureMmu ( ZeroMem (TranslationTable, RootTableEntryCount * sizeof(UINT64)); - // Disable MMU and caches. ArmDisableMmu() also invalidates the TLBs - ArmDisableMmu (); - ArmDisableDataCache (); - ArmDisableInstructionCache (); - - // Make sure nothing sneaked into the cache - ArmCleanInvalidateDataCache (); - ArmInvalidateInstructionCache (); - TranslationTableAttribute = TT_ATTR_INDX_INVALID; while (MemoryTable->Length != 0) { -- 2.17.1