From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received-SPF: Permerror (SPF Permanent Error: More than 10 MX records returned) identity=mailfrom; client-ip=134.134.136.126; helo=mga18.intel.com; envelope-from=star.zeng@intel.com; receiver=edk2-devel@lists.01.org Received: from mga18.intel.com (mga18.intel.com [134.134.136.126]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ml01.01.org (Postfix) with ESMTPS id 309BB21B0284B for ; Thu, 7 Dec 2017 18:59:48 -0800 (PST) Received: from fmsmga005.fm.intel.com ([10.253.24.32]) by orsmga106.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 07 Dec 2017 19:04:21 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.45,376,1508828400"; d="scan'208";a="184738581" Received: from fmsmsx108.amr.corp.intel.com ([10.18.124.206]) by fmsmga005.fm.intel.com with ESMTP; 07 Dec 2017 19:04:21 -0800 Received: from fmsmsx155.amr.corp.intel.com (10.18.116.71) by FMSMSX108.amr.corp.intel.com (10.18.124.206) with Microsoft SMTP Server (TLS) id 14.3.319.2; Thu, 7 Dec 2017 19:04:20 -0800 Received: from shsmsx104.ccr.corp.intel.com (10.239.4.70) by FMSMSX155.amr.corp.intel.com (10.18.116.71) with Microsoft SMTP Server (TLS) id 14.3.319.2; Thu, 7 Dec 2017 19:04:20 -0800 Received: from shsmsx102.ccr.corp.intel.com ([169.254.2.175]) by SHSMSX104.ccr.corp.intel.com ([169.254.5.152]) with mapi id 14.03.0319.002; Fri, 8 Dec 2017 11:04:18 +0800 From: "Zeng, Star" To: "Wang, Jian J" , "edk2-devel@lists.01.org" CC: "Yao, Jiewen" , "Dong, Eric" , "Ni, Ruiyu" , "Zeng, Star" Thread-Topic: [PATCH v4 1/2] MdeModulePkg/DxeIpl: Mark page table as read-only Thread-Index: AQHTb08ReHZY6w2/yE+B7L5IUXNdkqM4wb6w Date: Fri, 8 Dec 2017 03:04:17 +0000 Message-ID: <0C09AFA07DD0434D9E2A0C6AEB0483103B9C12AF@shsmsx102.ccr.corp.intel.com> References: <20171207113215.14724-1-jian.j.wang@intel.com> <20171207113215.14724-2-jian.j.wang@intel.com> In-Reply-To: <20171207113215.14724-2-jian.j.wang@intel.com> Accept-Language: zh-CN, en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [10.239.127.40] MIME-Version: 1.0 Subject: Re: [PATCH v4 1/2] MdeModulePkg/DxeIpl: Mark page table as read-only X-BeenThere: edk2-devel@lists.01.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: EDK II Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 08 Dec 2017 02:59:48 -0000 Content-Language: en-US Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Could EnablePageTableProtection and AllocatePageTableMemory just be declare= d in VirtualMemory.h but not DxeIpl.h? As their implementation are in VirtualMemory.c that is only used by X64 and= IA32, but not other archs, DxeIpl.h is used by all archs. Also have same comments to ClearFirst4KPage and IsNullDetectionEnabled and = just sent a patch for them at https://lists.01.org/pipermail/edk2-devel/201= 7-December/018854.html. Thanks, Star -----Original Message----- From: Wang, Jian J=20 Sent: Thursday, December 7, 2017 7:32 PM To: edk2-devel@lists.01.org Cc: Yao, Jiewen ; Zeng, Star ; D= ong, Eric ; Ni, Ruiyu Subject: [PATCH v4 1/2] MdeModulePkg/DxeIpl: Mark page table as read-only > v4: > a. Fix a calculation error in pool page number during initialization=20 > b. Change comments to remove constants in description > v3: > Remove the public definition of PAGE_TABLE_POOL_HEADER but keep simila= r > concept locally. CpuDxe has its own page table pool. > v2: > Introduce page table pool to ease the page table memory allocation and > protection, which replaces the direct calling of AllocatePages(). This patch will set the memory pages used for page table as read-only memor= y after the paging is setup. CR0.WP must set to let it take into effect. A simple page table memory management mechanism, page table pool concept, i= s introduced to simplify the page table memory allocation and protection. It will also help to reduce the potential recursive "split" action during u= pdating memory paging attributes. The basic idea is to allocate a bunch of continuous pages of memory in adva= nce as one or more page table pools, and all future page tables consumption= will happen in those pool instead of system memory. If the page pool is re= served at the boundary of 2MB page and with same size of 2MB page, there's = no page granularity "split" operation will be needed, because the memory of= new page tables (if needed) will be usually in the same page as target pag= e table you're working on. And since we have centralized page tables (a few 2MB pages), it's easier to= protect them by changing their attributes to be read-only once and for all= . There's no need to apply the protection for new page tables any more as l= ong as the pool has free pages available. Once current page table pool has been used up, one can allocate another 2MB= memory pool and just set this new 2MB memory block to be read-only instead= of setting the new page tables one page by one page. Two new PCDs PcdPageTablePoolUnitSize and PcdPageTablePoolAlignment are use= d to specify the size and alignment for page table pool. For IA32 processor 0x200000 (2MB) is the only choice for both of them to meet the requirement = of page table pool. Cc: Jiewen Yao Cc: Star Zeng Cc: Eric Dong Cc: Ruiyu Ni Contributed-under: TianoCore Contribution Agreement 1.1 Signed-off-by: Jian J Wang --- MdeModulePkg/Core/DxeIplPeim/DxeIpl.h | 34 +++ MdeModulePkg/Core/DxeIplPeim/Ia32/DxeLoadFunc.c | 8 +- MdeModulePkg/Core/DxeIplPeim/X64/VirtualMemory.c | 299 +++++++++++++++++++= +++- MdeModulePkg/Core/DxeIplPeim/X64/VirtualMemory.h | 26 ++ 4 files changed, 363 insertions(+), 4 deletions(-) diff --git a/MdeModulePkg/Core/DxeIplPeim/DxeIpl.h b/MdeModulePkg/Core/DxeI= plPeim/DxeIpl.h index f3aabdb7e0..9dc80b1508 100644 --- a/MdeModulePkg/Core/DxeIplPeim/DxeIpl.h +++ b/MdeModulePkg/Core/DxeIplPeim/DxeIpl.h @@ -265,4 +265,38 @@ IsNullDetectionEnabled ( VOID ); =20 +/** + Prevent the memory pages used for page table from been overwritten. + + @param[in] PageTableBase Base address of page table (CR3). + +**/ +VOID +EnablePageTableProtection ( + IN UINTN PageTableBase, + IN BOOLEAN Level4Paging + ); + +/** + This API provides a way to allocate memory for page table. + + This API can be called more than once to allocate memory for page tables= . + + Allocates the number of 4KB pages and returns a pointer to the=20 + allocated buffer. The buffer returned is aligned on a 4KB boundary. + + If Pages is 0, then NULL is returned. + If there is not enough memory remaining to satisfy the request, then=20 + NULL is returned. + + @param Pages The number of 4 KB pages to allocate. + + @return A pointer to the allocated buffer or NULL if allocation fails. + +**/ +VOID * +AllocatePageTableMemory ( + IN UINTN Pages + ); + #endif diff --git a/MdeModulePkg/Core/DxeIplPeim/Ia32/DxeLoadFunc.c b/MdeModulePkg= /Core/DxeIplPeim/Ia32/DxeLoadFunc.c index 5649265367..13fff28e93 100644 --- a/MdeModulePkg/Core/DxeIplPeim/Ia32/DxeLoadFunc.c +++ b/MdeModulePkg/Core/DxeIplPeim/Ia32/DxeLoadFunc.c @@ -99,7 +99,7 @@ Create4GPageTablesIa32Pae ( NumberOfPdpEntriesNeeded =3D (UINT32) LShiftU64 (1, (PhysicalAddressBits= - 30)); =20 TotalPagesNum =3D NumberOfPdpEntriesNeeded + 1; - PageAddress =3D (UINTN) AllocatePages (TotalPagesNum); + PageAddress =3D (UINTN) AllocatePageTableMemory (TotalPagesNum); ASSERT (PageAddress !=3D 0); =20 PageMap =3D (VOID *) PageAddress; @@ -149,6 +149,12 @@ Create4GPageTablesIa32Pae ( ); } =20 + // + // Protect the page table by marking the memory used for page table=20 + to be // read-only. + // + EnablePageTableProtection ((UINTN)PageMap, FALSE); + return (UINTN) PageMap; } =20 diff --git a/MdeModulePkg/Core/DxeIplPeim/X64/VirtualMemory.c b/MdeModulePk= g/Core/DxeIplPeim/X64/VirtualMemory.c index 29b6205e88..038aa0d127 100644 --- a/MdeModulePkg/Core/DxeIplPeim/X64/VirtualMemory.c +++ b/MdeModulePkg/Core/DxeIplPeim/X64/VirtualMemory.c @@ -31,6 +31,11 @@ WITHOUT WARRANTIES OR REPRESENTATIONS OF ANY KIND, EITHE= R EXPRESS OR IMPLIED. #include "DxeIpl.h" #include "VirtualMemory.h" =20 +// +// Global variable to keep track current available memory used as page tab= le. +// +PAGE_TABLE_POOL *mPageTablePool =3D NULL; + /** Clear legacy memory located at the first 4K-page, if available. =20 @@ -117,6 +122,110 @@ EnableExecuteDisableBit ( AsmWriteMsr64 (0xC0000080, MsrRegisters); } =20 +/** + Initialize a buffer pool for page table use only. + + To reduce the potential split operation on page table, the pages=20 + reserved for page table should be allocated in the times of=20 + PAGE_TABLE_POOL_UNIT_PAGES and at the boundary of=20 + PAGE_TABLE_POOL_ALIGNMENT. So the page pool is always initialized with n= umber of pages greater than or equal to the given PoolPages. + + Once the pages in the pool are used up, this method should be called=20 + again to reserve at least another PAGE_TABLE_POOL_UNIT_PAGES. But=20 + usually this won't happen in practice. + + @param PoolPages The least page number of the pool to be created. + + @retval TRUE The pool is initialized successfully. + @retval FALSE The memory is out of resource. +**/ +BOOLEAN +InitializePageTablePool ( + IN UINTN PoolPages + ) +{ + VOID *Buffer; + + // + // Always reserve at least PAGE_TABLE_POOL_UNIT_PAGES, including one=20 + page for // header. + // + PoolPages +=3D 1; // Add one page for header. + PoolPages =3D ((PoolPages - 1) / PAGE_TABLE_POOL_UNIT_PAGES + 1) * + PAGE_TABLE_POOL_UNIT_PAGES; Buffer =3D=20 + AllocateAlignedPages (PoolPages, PAGE_TABLE_POOL_ALIGNMENT); if=20 + (Buffer =3D=3D NULL) { + DEBUG ((DEBUG_ERROR, "ERROR: Out of aligned pages\r\n")); + return FALSE; + } + + // + // Link all pools into a list for easier track later. + // + if (mPageTablePool =3D=3D NULL) { + mPageTablePool =3D Buffer; + mPageTablePool->NextPool =3D mPageTablePool; } else { + ((PAGE_TABLE_POOL *)Buffer)->NextPool =3D mPageTablePool->NextPool; + mPageTablePool->NextPool =3D Buffer; + mPageTablePool =3D Buffer; + } + + // + // Reserve one page for pool header. + // + mPageTablePool->FreePages =3D PoolPages - 1; mPageTablePool->Offset = =3D=20 + EFI_PAGES_TO_SIZE (1); + + return TRUE; +} + +/** + This API provides a way to allocate memory for page table. + + This API can be called more than once to allocate memory for page tables= . + + Allocates the number of 4KB pages and returns a pointer to the=20 + allocated buffer. The buffer returned is aligned on a 4KB boundary. + + If Pages is 0, then NULL is returned. + If there is not enough memory remaining to satisfy the request, then=20 + NULL is returned. + + @param Pages The number of 4 KB pages to allocate. + + @return A pointer to the allocated buffer or NULL if allocation fails. + +**/ +VOID * +AllocatePageTableMemory ( + IN UINTN Pages + ) +{ + VOID *Buffer; + + if (Pages =3D=3D 0) { + return NULL; + } + + // + // Renew the pool if necessary. + // + if (mPageTablePool =3D=3D NULL || + Pages > mPageTablePool->FreePages) { + if (!InitializePageTablePool (Pages)) { + return NULL; + } + } + + Buffer =3D (UINT8 *)mPageTablePool + mPageTablePool->Offset; + + mPageTablePool->Offset +=3D EFI_PAGES_TO_SIZE (Pages); + mPageTablePool->FreePages -=3D Pages; + + return Buffer; +} + /** Split 2M page to 4K. =20 @@ -144,7 +253,7 @@ Split2MPageTo4K ( // AddressEncMask =3D PcdGet64 (PcdPteMemoryEncryptionAddressOrMask) & PAGI= NG_1G_ADDRESS_MASK_64; =20 - PageTableEntry =3D AllocatePages (1); + PageTableEntry =3D AllocatePageTableMemory (1); ASSERT (PageTableEntry !=3D NULL); =20 // @@ -204,7 +313,7 @@ Split1GPageTo2M ( // AddressEncMask =3D PcdGet64 (PcdPteMemoryEncryptionAddressOrMask) & PAGI= NG_1G_ADDRESS_MASK_64; =20 - PageDirectoryEntry =3D AllocatePages (1); + PageDirectoryEntry =3D AllocatePageTableMemory (1); ASSERT (PageDirectoryEntry !=3D NULL); =20 // @@ -234,6 +343,184 @@ Split1GPageTo2M ( } } =20 +/** + Set one page of page table pool memory to be read-only. + + @param[in] PageTableBase Base address of page table (CR3). + @param[in] Address Start address of a page to be set as read-on= ly. + @param[in] Level4Paging Level 4 paging flag. + +**/ +VOID +SetPageTablePoolReadOnly ( + IN UINTN PageTableBase, + IN EFI_PHYSICAL_ADDRESS Address, + IN BOOLEAN Level4Paging + ) +{ + UINTN Index; + UINTN EntryIndex; + UINT64 AddressEncMask; + EFI_PHYSICAL_ADDRESS PhysicalAddress; + UINT64 *PageTable; + UINT64 *NewPageTable; + UINT64 PageAttr; + UINT64 LevelSize[5]; + UINT64 LevelMask[5]; + UINTN LevelShift[5]; + UINTN Level; + UINT64 PoolUnitSize; + + ASSERT (PageTableBase !=3D 0); + + // + // Since the page table is always from page table pool, which is=20 + always // located at the boundary of PcdPageTablePoolAlignment, we=20 + just need to // set the whole pool unit to be read-only. + // + Address =3D Address & PAGE_TABLE_POOL_ALIGN_MASK; + + LevelShift[1] =3D PAGING_L1_ADDRESS_SHIFT; LevelShift[2] =3D=20 + PAGING_L2_ADDRESS_SHIFT; LevelShift[3] =3D PAGING_L3_ADDRESS_SHIFT; =20 + LevelShift[4] =3D PAGING_L4_ADDRESS_SHIFT; + + LevelMask[1] =3D PAGING_4K_ADDRESS_MASK_64; LevelMask[2] =3D=20 + PAGING_2M_ADDRESS_MASK_64; LevelMask[3] =3D PAGING_1G_ADDRESS_MASK_64; = =20 + LevelMask[4] =3D PAGING_1G_ADDRESS_MASK_64; + + LevelSize[1] =3D SIZE_4KB; + LevelSize[2] =3D SIZE_2MB; + LevelSize[3] =3D SIZE_1GB; + LevelSize[4] =3D SIZE_512GB; + + AddressEncMask =3D PcdGet64 (PcdPteMemoryEncryptionAddressOrMask) & + PAGING_1G_ADDRESS_MASK_64; + PageTable =3D (UINT64 *)(UINTN)PageTableBase; + PoolUnitSize =3D PAGE_TABLE_POOL_UNIT_SIZE; + + for (Level =3D (Level4Paging) ? 4 : 3; Level > 0; --Level) { + Index =3D ((UINTN)RShiftU64 (Address, LevelShift[Level])); + Index &=3D PAGING_PAE_INDEX_MASK; + + PageAttr =3D PageTable[Index]; + if ((PageAttr & IA32_PG_PS) =3D=3D 0) { + // + // Go to next level of table. + // + PageTable =3D (UINT64 *)(UINTN)(PageAttr & ~AddressEncMask & + PAGING_4K_ADDRESS_MASK_64); + continue; + } + + if (PoolUnitSize >=3D LevelSize[Level]) { + // + // Clear R/W bit if current page granularity is not larger than pool= unit + // size. + // + if ((PageAttr & IA32_PG_RW) !=3D 0) { + while (PoolUnitSize > 0) { + // + // PAGE_TABLE_POOL_UNIT_SIZE and PAGE_TABLE_POOL_ALIGNMENT are f= it in + // one page (2MB). Then we don't need to update attributes for p= ages + // crossing page directory. ASSERT below is for that purpose. + // + ASSERT (Index < EFI_PAGE_SIZE/sizeof (UINT64)); + + PageTable[Index] &=3D ~(UINT64)IA32_PG_RW; + PoolUnitSize -=3D LevelSize[Level]; + + ++Index; + } + } + + break; + + } else { + // + // The smaller granularity of page must be needed. + // + NewPageTable =3D AllocatePageTableMemory (1); + ASSERT (NewPageTable !=3D NULL); + + PhysicalAddress =3D PageAttr & LevelMask[Level]; + for (EntryIndex =3D 0; + EntryIndex < EFI_PAGE_SIZE/sizeof (UINT64); + ++EntryIndex) { + NewPageTable[EntryIndex] =3D PhysicalAddress | AddressEncMask | + IA32_PG_P | IA32_PG_RW; + if (Level > 1) { + NewPageTable[EntryIndex] |=3D IA32_PG_PS; + } + PhysicalAddress +=3D LevelSize[Level]; + } + + PageTable[Index] =3D (UINT64)(UINTN)NewPageTable | AddressEncMask | + IA32_PG_P | IA32_PG_RW; + PageTable =3D NewPageTable; + } + } +} + +/** + Prevent the memory pages used for page table from been overwritten. + + @param[in] PageTableBase Base address of page table (CR3). + @param[in] Level4Paging Level 4 paging flag. + +**/ +VOID +EnablePageTableProtection ( + IN UINTN PageTableBase, + IN BOOLEAN Level4Paging + ) +{ + PAGE_TABLE_POOL *HeadPool; + PAGE_TABLE_POOL *Pool; + UINT64 PoolSize; + EFI_PHYSICAL_ADDRESS Address; + + if (mPageTablePool =3D=3D NULL) { + return; + } + + // + // Disable write protection, because we need to mark page table to be=20 + write // protected. + // + AsmWriteCr0 (AsmReadCr0() & ~CR0_WP); + + // + // SetPageTablePoolReadOnly might update mPageTablePool. It's safer=20 + to // remember original one in advance. + // + HeadPool =3D mPageTablePool; + Pool =3D HeadPool; + do { + Address =3D (EFI_PHYSICAL_ADDRESS)(UINTN)Pool; + PoolSize =3D Pool->Offset + EFI_PAGES_TO_SIZE (Pool->FreePages); + + // + // The size of one pool must be multiple of PAGE_TABLE_POOL_UNIT_SIZE,= which + // is one of page size of the processor (2MB by default). Let's apply = the + // protection to them one by one. + // + while (PoolSize > 0) { + SetPageTablePoolReadOnly(PageTableBase, Address, Level4Paging); + Address +=3D PAGE_TABLE_POOL_UNIT_SIZE; + PoolSize -=3D PAGE_TABLE_POOL_UNIT_SIZE; + } + + Pool =3D Pool->NextPool; + } while (Pool !=3D HeadPool); + + // + // Enable write protection, after page table attribute updated. + // + AsmWriteCr0 (AsmReadCr0() | CR0_WP); +} + /** Allocates and fills in the Page Directory and Page Table Entries to establish a 1:1 Virtual to Physical mapping. @@ -329,7 +616,7 @@ CreateIdentityMappingPageTables ( } else { TotalPagesNum =3D NumberOfPml4EntriesNeeded + 1; } - BigPageAddress =3D (UINTN) AllocatePages (TotalPagesNum); + BigPageAddress =3D (UINTN) AllocatePageTableMemory (TotalPagesNum); ASSERT (BigPageAddress !=3D 0); =20 // @@ -430,6 +717,12 @@ CreateIdentityMappingPageTables ( ); } =20 + // + // Protect the page table by marking the memory used for page table=20 + to be // read-only. + // + EnablePageTableProtection ((UINTN)PageMap, TRUE); + if (PcdGetBool (PcdSetNxForStack)) { EnableExecuteDisableBit (); } diff --git a/MdeModulePkg/Core/DxeIplPeim/X64/VirtualMemory.h b/MdeModulePk= g/Core/DxeIplPeim/X64/VirtualMemory.h index 7c9bb49e3e..b8cf43104e 100644 --- a/MdeModulePkg/Core/DxeIplPeim/X64/VirtualMemory.h +++ b/MdeModulePkg/Core/DxeIplPeim/X64/VirtualMemory.h @@ -148,11 +148,37 @@ typedef union { =20 #pragma pack() =20 +#define CR0_WP BIT16 + #define IA32_PG_P BIT0 #define IA32_PG_RW BIT1 +#define IA32_PG_PS BIT7 + +#define PAGING_PAE_INDEX_MASK 0x1FF =20 +#define PAGING_4K_ADDRESS_MASK_64 0x000FFFFFFFFFF000ull +#define PAGING_2M_ADDRESS_MASK_64 0x000FFFFFFFE00000ull #define PAGING_1G_ADDRESS_MASK_64 0x000FFFFFC0000000ull =20 +#define PAGING_L1_ADDRESS_SHIFT 12 +#define PAGING_L2_ADDRESS_SHIFT 21 +#define PAGING_L3_ADDRESS_SHIFT 30 +#define PAGING_L4_ADDRESS_SHIFT 39 + +#define PAGING_PML4E_NUMBER 4 + +#define PAGE_TABLE_POOL_ALIGNMENT BASE_2MB +#define PAGE_TABLE_POOL_UNIT_SIZE SIZE_2MB +#define PAGE_TABLE_POOL_UNIT_PAGES EFI_SIZE_TO_PAGES=20 +(PAGE_TABLE_POOL_UNIT_SIZE) #define PAGE_TABLE_POOL_ALIGN_MASK \ + (~(EFI_PHYSICAL_ADDRESS)(PAGE_TABLE_POOL_ALIGNMENT - 1)) + +typedef struct { + VOID *NextPool; + UINTN Offset; + UINTN FreePages; +} PAGE_TABLE_POOL; + /** Enable Execute Disable Bit. =20 -- 2.15.1.windows.2