From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-wr1-f67.google.com (mail-wr1-f67.google.com [209.85.221.67]) by mx.groups.io with SMTP id smtpd.web11.5827.1582711446972002162 for ; Wed, 26 Feb 2020 02:04:07 -0800 Authentication-Results: mx.groups.io; dkim=pass header.i=@linaro.org header.s=google header.b=l/BOFKa5; spf=pass (domain: linaro.org, ip: 209.85.221.67, mailfrom: ard.biesheuvel@linaro.org) Received: by mail-wr1-f67.google.com with SMTP id z15so2215285wrl.1 for ; Wed, 26 Feb 2020 02:04:06 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=TQ0S9v8ZYWYZYsIJzJhFPhO3kjpPp2Tgufyqwfmwnrw=; b=l/BOFKa5pwGknJpYICLtHi8gbCMw5GygVtH9qFToik2PNpWplUT6G/A/iByUKFjTHh adQq4PSjGzN4faVQ4TnjTKypnRddsB8rMh3Gms3LCVTPeKJHyP7E7cPT6tngPmst6xD7 Oxq3xGpVTMfbHwSccx8MO7Oci0Eo3c1rVxc8Eerm7FExxAKn3KKoBdZtjjIj+2JxVqq4 XEy1DofJ0HzNfqcqVVJl2Onwiaa9l67cGTxvn8bVVjef22Yl4yNhinCPYRZ9aTMSbu+j bB6WGqCMhjVFJZQRAW01mRaCQ7oIaUO1ng0BHXkKCEtEhgdKKO4lpRGMjXCJJEGh868c ehMw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=TQ0S9v8ZYWYZYsIJzJhFPhO3kjpPp2Tgufyqwfmwnrw=; b=InLmUFHu3mlNppRbw0cYZdF+wW8mUZsfm6xgacHkuOA9/g6qiTXOzMG12JPS5wohjj 68xFmNt76Wl6ubHJQGFsFOOSxMhaEED0UdStBcQ+Wnd5tGfWi6dV3DUjIzNprvWP/Crr Pj8OvZ2ZtIZGjcur2I6OVBASkTJxIBVI2y7cCSmKgHCylOzLNyGMI6ib3zxZxD8MkTxu czowA6smhSKTXZbMQSuU6PL9CjGD1y7Eoi9iTiMfpQvBIfj3lOQ+L20+XTYwPWKjF+qn vknt88/obBLM5BkyY30WQ0n9LhsorrVH79QA3sZddqqx2sT+OV1nAZVGgbyxobP5ivnU qodg== X-Gm-Message-State: APjAAAUPDOZDRpyFyeaUOOfc31++mOFybT+dbQGULVwP3U8AcMvvGY0X WGjtacSPqOZXDAGe9YKXmgKgtmbSDe4VUQ== X-Google-Smtp-Source: APXvYqwLTCGG1uM4PWXI+S69KaqdPTCiMsjA5+EQSMe4914PWY6Jfb/63RJMpja58eMHFxP7CWPKCg== X-Received: by 2002:adf:e481:: with SMTP id i1mr4957181wrm.158.1582711445248; Wed, 26 Feb 2020 02:04:05 -0800 (PST) Return-Path: Received: from e123331-lin.home ([2a01:cb1d:112:6f00:816e:ff0d:fb69:f613]) by smtp.gmail.com with ESMTPSA id b10sm2559159wrw.61.2020.02.26.02.04.03 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 26 Feb 2020 02:04:04 -0800 (PST) From: "Ard Biesheuvel" To: devel@edk2.groups.io Cc: leif@nuviainc.com, lersek@redhat.com, sami.mujawar@arm.com, Ard Biesheuvel Subject: [PATCH 3/6] ArmPkg/ArmMmuLib ARM: cache-invalidate initial page table entries Date: Wed, 26 Feb 2020 11:03:50 +0100 Message-Id: <20200226100353.31962-4-ard.biesheuvel@linaro.org> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20200226100353.31962-1-ard.biesheuvel@linaro.org> References: <20200226100353.31962-1-ard.biesheuvel@linaro.org> In the ARM version of ArmMmuLib, we are currently relying on set/way invalidation to ensure that the caches are in a consistent state with respect to main memory once we turn the MMU on. Even if set/way operations were the appropriate method to achieve this, doing an invalidate-all first and then populating the page table entries creates a window where page table entries could be loaded speculatively into the caches before we modify them, and shadow the new values that we write there. So let's get rid of the blanket clean/invalidate operations, and instead, update ArmUpdateTranslationTableEntry () to invalidate each page table entry *after* it is written if the MMU is still disabled at this point. On ARMv7, cache maintenance may be required also when the MMU is enabled, in case the page table walker is not cache coherent. However, the code being updated here is guaranteed to run only when the MMU is still off, and so we can disregard the case when the MMU and caches are on. Since the MMU and D-cache are already off when we reach this point, we can drop the MMU and D-cache disables as well. Maintenance of the I-cache is unnecessary, since we are not modifying any code, and the installed mapping is guaranteed to be 1:1. This means we can also leave it enabled while the page table population code is running. Signed-off-by: Ard Biesheuvel --- ArmPkg/Library/ArmMmuLib/Arm/ArmMmuLibCore.c | 25 +++++++++----------- 1 file changed, 11 insertions(+), 14 deletions(-) diff --git a/ArmPkg/Library/ArmMmuLib/Arm/ArmMmuLibCore.c b/ArmPkg/Library/ArmMmuLib/Arm/ArmMmuLibCore.c index aca7a37facac..c5906b4310cc 100644 --- a/ArmPkg/Library/ArmMmuLib/Arm/ArmMmuLibCore.c +++ b/ArmPkg/Library/ArmMmuLib/Arm/ArmMmuLibCore.c @@ -183,6 +183,8 @@ PopulateLevel2PageTable ( PhysicalBase += TT_DESCRIPTOR_PAGE_SIZE; } + InvalidateDataCacheRange ((UINT32 *)TranslationTable + FirstPageOffset, + RemainLength / TT_DESCRIPTOR_PAGE_SIZE * sizeof (*PageEntry)); } STATIC @@ -257,7 +259,11 @@ FillTranslationTable ( RemainLength >= TT_DESCRIPTOR_SECTION_SIZE) { // Case: Physical address aligned on the Section Size (1MB) && the length // is greater than the Section Size - *SectionEntry++ = TT_DESCRIPTOR_SECTION_BASE_ADDRESS(PhysicalBase) | Attributes; + *SectionEntry = TT_DESCRIPTOR_SECTION_BASE_ADDRESS(PhysicalBase) | Attributes; + + ArmDataSynchronizationBarrier (); + ArmInvalidateDataCacheEntryByMVA ((UINTN)SectionEntry++); + PhysicalBase += TT_DESCRIPTOR_SECTION_SIZE; RemainLength -= TT_DESCRIPTOR_SECTION_SIZE; } else { @@ -267,9 +273,12 @@ FillTranslationTable ( // Case: Physical address aligned on the Section Size (1MB) && the length // does not fill a section // Case: Physical address NOT aligned on the Section Size (1MB) - PopulateLevel2PageTable (SectionEntry++, PhysicalBase, PageMapLength, + PopulateLevel2PageTable (SectionEntry, PhysicalBase, PageMapLength, MemoryRegion->Attributes); + ArmDataSynchronizationBarrier (); + ArmInvalidateDataCacheEntryByMVA ((UINTN)SectionEntry++); + // If it is the last entry if (RemainLength < TT_DESCRIPTOR_SECTION_SIZE) { break; @@ -349,18 +358,6 @@ ArmConfigureMmu ( } } - ArmCleanInvalidateDataCache (); - ArmInvalidateInstructionCache (); - - ArmDisableDataCache (); - ArmDisableInstructionCache(); - // TLBs are also invalidated when calling ArmDisableMmu() - ArmDisableMmu (); - - // Make sure nothing sneaked into the cache - ArmCleanInvalidateDataCache (); - ArmInvalidateInstructionCache (); - ArmSetTTBR0 ((VOID *)(UINTN)(((UINTN)TranslationTable & ~TRANSLATION_TABLE_SECTION_ALIGNMENT_MASK) | (TTBRAttributes & 0x7F))); // -- 2.17.1