[PATCH 2/2] UefiCpuPkg: Support 5 level page table in ResetVector

[PATCH 2/2] UefiCpuPkg: Support 5 level page table in ResetVector

[Zhiguang Liu]

Use a macro USE_5_LEVEL_PAGE_TABLE to determine whether to create
5 level page table. Whether creating it or not, the highest level
page table address is fixed.

Cc: Eric Dong <eric.dong@intel.com>
Cc: Ray Ni <ray.ni@intel.com>
Cc: Rahul Kumar <rahul1.kumar@intel.com>
Cc: Gerd Hoffmann <kraxel@redhat.com>
Cc: Debkumar De <debkumar.de@intel.com>
Cc: Catharine West <catharine.west@intel.com>
Signed-off-by: Zhiguang Liu <zhiguang.liu@intel.com>
---
 .../ResetVector/Vtf0/Ia32/Flat32ToFlat64.asm  |  5 ++++-
 UefiCpuPkg/ResetVector/Vtf0/Vtf0.nasmb        |  3 +++
 .../ResetVector/Vtf0/X64/PageTables5L.asm     | 19 +++++++++++++++++++
 3 files changed, 26 insertions(+), 1 deletion(-)
 create mode 100644 UefiCpuPkg/ResetVector/Vtf0/X64/PageTables5L.asm

diff --git a/UefiCpuPkg/ResetVector/Vtf0/Ia32/Flat32ToFlat64.asm b/UefiCpuPkg/ResetVector/Vtf0/Ia32/Flat32ToFlat64.asm
index 6891397c2a..b6c245e697 100644
--- a/UefiCpuPkg/ResetVector/Vtf0/Ia32/Flat32ToFlat64.asm
+++ b/UefiCpuPkg/ResetVector/Vtf0/Ia32/Flat32ToFlat64.asm
@@ -2,7 +2,7 @@
 ; @file
 ; Transition from 32 bit flat protected mode into 64 bit flat protected mode
 ;
-; Copyright (c) 2008 - 2018, Intel Corporation. All rights reserved.<BR>
+; Copyright (c) 2008 - 2023, Intel Corporation. All rights reserved.<BR>
 ; SPDX-License-Identifier: BSD-2-Clause-Patent
 ;
 ;------------------------------------------------------------------------------
@@ -18,6 +18,9 @@ Transition32FlatTo64Flat:
 
     mov     eax, cr4
     bts     eax, 5                      ; enable PAE
+%ifdef USE_5_LEVEL_PAGE_TABLE
+    bts     eax, 12                     ; Set LA57=1.
+%endif
     mov     cr4, eax
 
     mov     ecx, 0xc0000080
diff --git a/UefiCpuPkg/ResetVector/Vtf0/Vtf0.nasmb b/UefiCpuPkg/ResetVector/Vtf0/Vtf0.nasmb
index 62887c4e8e..670d6a9053 100644
--- a/UefiCpuPkg/ResetVector/Vtf0/Vtf0.nasmb
+++ b/UefiCpuPkg/ResetVector/Vtf0/Vtf0.nasmb
@@ -46,6 +46,9 @@ StartOfPageTables:
     %include "X64/PageTables2M.asm"
   %endif
 %endif
+%ifdef USE_5_LEVEL_PAGE_TABLE
+  %include "X64/PageTables5L.asm"
+%endif
 EndOfPageTables:
 
 %ifdef DEBUG_PORT80
diff --git a/UefiCpuPkg/ResetVector/Vtf0/X64/PageTables5L.asm b/UefiCpuPkg/ResetVector/Vtf0/X64/PageTables5L.asm
new file mode 100644
index 0000000000..e60e756422
--- /dev/null
+++ b/UefiCpuPkg/ResetVector/Vtf0/X64/PageTables5L.asm
@@ -0,0 +1,19 @@
+;------------------------------------------------------------------------------
+; @file
+; PML5 page table creation.
+;
+; Copyright (c) 2023, Intel Corporation. All rights reserved.<BR>
+; SPDX-License-Identifier: BSD-2-Clause-Patent
+;
+;------------------------------------------------------------------------------
+
+    ;
+    ; PML5 table Pointers
+    ; Assume page table is create from bottom to top, and only one PML4 table there.
+    ;
+    DQ      (ADDR_OF($) - 0x1000 + PAGE_PDP_ATTR)
+
+    ;
+    ; Only first PML5 entry(first 8 bytes) pointting to a PML4 table. Others are zero
+    ;
+    TIMES   (0x1000 - 0x8) DB 0
-- 
2.31.1.windows.1

Re: [PATCH 2/2] UefiCpuPkg: Support 5 level page table in ResetVector

[Gerd Hoffmann]

On Mon, Apr 03, 2023 at 05:28:14PM +0800, Zhiguang Liu wrote:
> Use a macro USE_5_LEVEL_PAGE_TABLE to determine whether to create
> 5 level page table. Whether creating it or not, the highest level
> page table address is fixed.

Can we create the 5level page table unconditionally?

We should have the PML5 table @ fixed at 4G - 12k and the PML4 table
fixed at 4G - 16k then.  Then ideally check processor capabilities and
decide at runtime whenever to use 4level or 5level paging.

thanks,
  Gerd

Re: [edk2-devel] [PATCH 2/2] UefiCpuPkg: Support 5 level page table in ResetVector

[Zhiguang Liu]

Hi Gerd
Sorry for responding late.
Using 5level paging when the below two conditions are both true:
  1. CPU support 5level paging
  2. Platform choose to use 5level paging. (by specifing macro USE_5_LEVEL_PAGE_TABLE)

There is an assumption that if platform choose to use 5level paging, then 5level paging should be supported by CPU. (Platform should know its CPU capability)
So condition#1 is actually depends on condition#2.

The condition#2 is decided at build time.
If Platform choose to use 4level paging at build time, we can save 4K space by not creating the 5 level page.
We can check the Cr4. LA57 to know if 4G - 12k is 4level or 5level paging when debugging after reset vector.

In a word, current way can save 4K space in flash when using 4level paging.
Please let me know if you have any other concern or consideration about this. Thanks.

Thanks
Zhiguang

> -----Original Message-----
> From: devel@edk2.groups.io <devel@edk2.groups.io> On Behalf Of Gerd
> Hoffmann
> Sent: Monday, April 3, 2023 7:56 PM
> To: Liu, Zhiguang <zhiguang.liu@intel.com>
> Cc: devel@edk2.groups.io; Dong, Eric <eric.dong@intel.com>; Ni, Ray
> <ray.ni@intel.com>; Kumar, Rahul R <rahul.r.kumar@intel.com>; De,
> Debkumar <debkumar.de@intel.com>; West, Catharine
> <catharine.west@intel.com>
> Subject: Re: [edk2-devel] [PATCH 2/2] UefiCpuPkg: Support 5 level page
> table in ResetVector
> 
> On Mon, Apr 03, 2023 at 05:28:14PM +0800, Zhiguang Liu wrote:
> > Use a macro USE_5_LEVEL_PAGE_TABLE to determine whether to create
> > 5 level page table. Whether creating it or not, the highest level page
> > table address is fixed.
> 
> Can we create the 5level page table unconditionally?
> 
> We should have the PML5 table @ fixed at 4G - 12k and the PML4 table fixed
> at 4G - 16k then.  Then ideally check processor capabilities and decide at
> runtime whenever to use 4level or 5level paging.
> 
> thanks,
>   Gerd
> 
> 
> 
> 
> 

Re: [edk2-devel] [PATCH 2/2] UefiCpuPkg: Support 5 level page table in ResetVector

[Gerd Hoffmann]

> Using 5level paging when the below two conditions are both true:
>   1. CPU support 5level paging
>   2. Platform choose to use 5level paging. (by specifing macro USE_5_LEVEL_PAGE_TABLE)
> 
> There is an assumption that if platform choose to use 5level paging,
> then 5level paging should be supported by CPU. (Platform should know
> its CPU capability)

That assumption does not hold for virtual machines.  OVMF builds with
5-level paging support enabled should continue to work on CPUs without
5-level paging support.

> If Platform choose to use 4level paging at build time, we can save 4K space by not creating the 5 level page.

I'm fine with that.

But with 5-level paging enabled the reset vector should be able to
fallback to 4-level paging in case the CPU does not support 5-level
paging.

take care,
  Gerd

Re: [edk2-devel] [PATCH 2/2] UefiCpuPkg: Support 5 level page table in ResetVector

[Ni, Ray]

> 
> But with 5-level paging enabled the reset vector should be able to
> fallback to 4-level paging in case the CPU does not support 5-level
> paging.

The fallback makes sense. Otherwise, forcing to use 5L in an incapable CPU
would be a silent failure because IDT and debug log are not enabled at this
early stage.

Later CpuPei/Dxe module could dump the paging status to tell platform owner
that what level is used in this boot.

With that, that means a 4L reset vector can save 4K space by excluding the PML5 page.
But a 5L reset vector could set CR3 to PML4 page if CPU is incapable.

Thanks,
Ray