Re: [PATCH V4 3/3] OvmfPkg/ResetVector: Enable Intel TDX in ResetVector of Ovmf

["Brijesh Singh" <brijesh.singh@amd.com>]

On 8/2/21 8:18 PM, Min Xu wrote:
> RFC: https://bugzilla.tianocore.org/show_bug.cgi?id=3429
> 
> Intel's Trust Domain Extensions (Intel TDX) refers to an Intel technology
> that extends Virtual Machines Extensions (VMX) and Multi-Key Total Memory
> Encryption (MKTME) with a new kind of virutal machines guest called a
> Trust Domain (TD). A TD is desinged to run in a CPU mode that protects the
> confidentiality of TD memory contents and the TD's CPU state from other
> software, including the hosting Virtual-Machine Monitor (VMM), unless
> explicitly shared by the TD itself.
> 
> Note: Intel TDX only is available on X64, so the Tdx related changes are
> in X64 path. In IA32 path, there may be null stub to make the build
> success.
> 
> This patch includes below major changes.
> 
> 1. CC_WORK_AREA
> Ovmf is able to run on different types of VM guest with ONE image. These
> VM guest includes the Legacy guest, SEV guest and TDX guest. In
> ResetVector CC_WORK_AREA is such a memory region to record the VM guest
> information. Below is the definition of the work area.
> 
>   typedef struct {
>     UINT8   Type;     // 0 legacy, 1 SEV, 2 TDX
>     UINT8   SubType;  // Depends on Type
>     UINT8   Rsvd[2];  // Reserved
>     union VM_GUEST {
>       TDX_WORK_AREA Tdx;
>       SEV_WORK_AREA Sev;
>     } Guest;
>   } CC_WORK_AREA_HEAD;
> 
>   typedef struct {
>     ... ...
>   } TDX_WORK_AREA
> 
>   typedef struct {
>     ... ...
>   } SEV_WORK_AREA

This patch need to be broken into multiple sub patches for the easy to 
review and keeping the code bisectable. The first thing we need to do is 
introduce the new CCWorkArea concept and update the existing Sev support 
to use the new CCWorkArea.

> 
> 2. X64/IntelTdxMetadata.asm
> IntelTdxMetadata describes the information about the image for VMM use.
> For example, the base address and length of the TdHob, TdMailbox, etc.
> Its offset is put in a GUID-ed structure which is appended in the GUID-ed
> chain from a fixed GPA (0xffffffd0). Below are the items in TdxMetadata:
>   _Bfv:
>      Boot Firmware Volume
>   _Cfv:
>      Configuration Firmware Volume
>   _Stack:
>      Initial stack
>   _Heap:
>      Initial heap
>   _MailBox:
>      TDVF reserves the memory region so each AP can receive the message
>      sent by the guest OS.
>   _CcWorkarea:
>      Compute Confidential work area which is consumed by CC technologies,
>      such as SEV, TDX.
>   _TdHob:
>      VMM pass the resource information in TdHob to TDVF.
>   _TdxPageTable:
>      If 5-level page table is supported (GPAW is 52), a top level page
>      directory pointers (1 * 256TB entry) is generated in this page.
>   _OvmfPageTable:
>      Initial page table for standard Ovmf.
> 
> TDVF indicate above chunk of temporary initialized memory region (_Stack/
> _Heap/_MailBox/_CcWorkarea/_TdHob/_TdxPageTables/OvmfPageTable) to support
> TDVF code finishing the memory initialization. Because the other
> unaccepted memory cannot be accessed until they're accepted.
> 
> Since AMD SEV has already defined some SEV specific memory region in
> MEMFD. TDX re-use the memory regions defined by SEV.
>   - MailBox      : PcdOvmfSecGhcbBackupBase|PcdOvmfSecGhcbBackupSize
>   - TdHob        : PcdOvmfSecGhcbBase|PcdOvmfSecGhcbSize
>   - TdxPageTable : PcdOvmfSecGhcbPageTableBase|PcdOvmfSecGhcbPageTableSize
>   - CcWorkArea   : PcdSevEsWorkAreaBase|PcdSevEsWorkAreaSize
> 
> 3. Ia32/IntelTdx.asm
> IntelTdx.asm includes below routines used in ResetVector
>   - IsTdx
>     Check if the running system is Tdx guest.
> 
>   - InitTdxWorkarea
>     It initialize the CC_WORK_AREA. Because it is called by both BSP and
>     APs and to avoid the race condition, only BSP can initialize the
>     WORK_AREA. AP will wait until the field of TDX_WORK_AREA_PGTBL_READY
>     is set.
> 
>   - ReloadFlat32
>     After reset all CPUs in TDX are initialized to 32-bit protected mode.
>     But GDT register is not set. So this routine loads the GDT and set the
>     CR0, then jump to Flat 32 protected mode. After that CR4 and other
>     registers are set.
> 
>   - InitTdx
>     This routine wrap above 3 routines together to do Tdx initialization
>     in ResetVector phase.
> 
>   - PostSetCr3PageTables64Tdx
>     It is called after SetCr3PageTables64 in Tdx guest to set CR0/CR4.
>     If GPAW is 52, then CR3 is adjusted as well.
> 
>   - IsTdxEnabled
>     It is a OneTimeCall to probe if TDX is enabled by checking the
>     CC_WORK_AREA.
> 
> 4. Ia32/AmdSev.asm
> AmdSev.asm includes the SEV routines used in ResetVector. This patch
> remove the code of clearing SEV_ES_WORK_AREA in CheckSevFeatures. The
> clearing of SEV_ES_WORK_AREA is called at Main16 in Main.asm.
> 
Yep, this part need to be done in a separate patch. We can do this 
refactoring and addition of the CCWorkArea outside of the TDX and SNP 
series.

> 5. Main.asm
> Previously OvmfPkg/ResetVector use the Main.asm in UefiCpuPkg. There is
> only Main16 entry point. Main32 entry point is needed in Main.asm because
> of Intel TDX. To reduce the complexity of Main.asm in UefiCpuPkg, OvmfPkg
> create its own Main.asm to meet the requirement of Intel TDX. There are
> below changes in this Main.asm:
>   - A new entry point (Main32) is added. TDX guest will jump to Main32
>     from ResetVecotr.
>   - In Main16 entry point, after TransitionFromReal16To32BitFlat,
>     CC_WORK_AREA is cleared to 0.
> 
> 6. Ia32/PageTables64.asm
> GPAW of TDX can be 48 or 52, which determines the level of page table.
> If Level-5(GPAW 52) paging is supported, then an extra page is needed
> to hold the top level Page Directory Pointers (1 * 256TB entry).
> 
> 7. Ia16/ResetVectorVtf0.asm
> In Tdx all CPUs "reset" to run on 32-bit protected mode with flat
> descriptor (paging disabled). But in Non-Td guest the initial state of
> CPUs is 16-bit real mode. To resolve this conflict, BITS 16/32 is used
> in the ResetVectorVtf0.asm. It checks the 32-bit protected mode or 16-bit
> real mode, then jump to the corresponding entry point.
> 
> 8. ResetVector.nasmb
> TDX related macros and files are added in ResetVecotr.nasmb.
> 
> Cc: Ard Biesheuvel <ardb+tianocore@kernel.org>
> Cc: Jordan Justen <jordan.l.justen@intel.com>
> Cc: Brijesh Singh <brijesh.singh@amd.com>
> Cc: Erdem Aktas <erdemaktas@google.com>
> Cc: James Bottomley <jejb@linux.ibm.com>
> Cc: Jiewen Yao <jiewen.yao@intel.com>
> Cc: Tom Lendacky <thomas.lendacky@amd.com>
> Signed-off-by: Min Xu <min.m.xu@intel.com>
> ---
>   OvmfPkg/ResetVector/Ia16/ResetVectorVtf0.asm |  39 +++
>   OvmfPkg/ResetVector/Ia32/AmdSev.asm          |   7 -
>   OvmfPkg/ResetVector/Ia32/Flat32ToFlat64.asm  |   9 +
>   OvmfPkg/ResetVector/Ia32/IntelTdx.asm        | 265 +++++++++++++++++++
>   OvmfPkg/ResetVector/Ia32/PageTables64.asm    | 113 +++++---
>   OvmfPkg/ResetVector/Main.asm                 | 121 +++++++++
>   OvmfPkg/ResetVector/ResetVector.inf          |  12 +-
>   OvmfPkg/ResetVector/ResetVector.nasmb        |  48 +++-
>   OvmfPkg/ResetVector/X64/IntelTdxMetadata.asm | 110 ++++++++
>   9 files changed, 679 insertions(+), 45 deletions(-)
>   create mode 100644 OvmfPkg/ResetVector/Ia32/IntelTdx.asm
>   create mode 100644 OvmfPkg/ResetVector/Main.asm
>   create mode 100644 OvmfPkg/ResetVector/X64/IntelTdxMetadata.asm
> 
> diff --git a/OvmfPkg/ResetVector/Ia16/ResetVectorVtf0.asm b/OvmfPkg/ResetVector/Ia16/ResetVectorVtf0.asm
> index 7ec3c6e980c3..62feeacbee7b 100644
> --- a/OvmfPkg/ResetVector/Ia16/ResetVectorVtf0.asm
> +++ b/OvmfPkg/ResetVector/Ia16/ResetVectorVtf0.asm
> @@ -47,6 +47,24 @@ TIMES (15 - ((guidedStructureEnd - guidedStructureStart + 15) % 16)) DB 0
>   ;
>   guidedStructureStart:
>   
> +%ifdef ARCH_X64
> +;
> +; TDX Metadata offset block
> +;
> +; If TdxMetadata.asm is included then we need below block which describes
> +; the offset of TdxMetadata block in Ovmf image
> +;
> +; GUID : e47a6535-984a-4798-865e-4685a7bf8ec2
> +;
> +tdxMetadataOffsetStart:
> +    DD      (OVMF_IMAGE_SIZE_IN_KB * 1024 - (fourGigabytes - TdxMetadataGuid - 16))
> +    DD      tdxMetadataOffsetEnd - tdxMetadataOffsetStart
> +    DB      0x35, 0x65, 0x7a, 0xe4, 0x4a, 0x98, 0x98, 0x47
> +    DB      0x86, 0x5e, 0x46, 0x85, 0xa7, 0xbf, 0x8e, 0xc2
> +tdxMetadataOffsetEnd:
> +
> +%endif
> +
>   ; SEV Hash Table Block
>   ;
>   ; This describes the guest ram area where the hypervisor should
> @@ -158,10 +176,31 @@ resetVector:
>   ;
>   ; This is where the processor will begin execution
>   ;
> +; In IA32 we follow the standard reset vector flow. While in X64, Td guest
> +; may be supported. Td guest requires the startup mode to be 32-bit
> +; protected mode but the legacy VM startup mode is 16-bit real mode.
> +; To make NASM generate such shared entry code that behaves correctly in
> +; both 16-bit and 32-bit mode, more BITS directives are added.
> +;
> +%ifdef ARCH_IA32
> +
>       nop
>       nop
>       jmp     EarlyBspInitReal16
>   
> +%else
> +
> +    smsw    ax
> +    test    al, 1
> +    jz      .Real
> +BITS 32
> +    jmp     Main32
> +BITS 16
> +.Real:
> +    jmp     EarlyBspInitReal16
> +
> +%endif
> +
>   ALIGN   16
>   
>   fourGigabytes:
> diff --git a/OvmfPkg/ResetVector/Ia32/AmdSev.asm b/OvmfPkg/ResetVector/Ia32/AmdSev.asm
> index aa95d06eaddb..d0d9890d5c4b 100644
> --- a/OvmfPkg/ResetVector/Ia32/AmdSev.asm
> +++ b/OvmfPkg/ResetVector/Ia32/AmdSev.asm
> @@ -128,13 +128,6 @@ SevEsUnexpectedRespTerminate:
>   ; If SEV is disabled then EAX will be zero.
>   ;
>   CheckSevFeatures:
> -    ; Set the first byte of the workarea to zero to communicate to the SEC
> -    ; phase that SEV-ES is not enabled. If SEV-ES is enabled, the CPUID
> -    ; instruction will trigger a #VC exception where the first byte of the
> -    ; workarea will be set to one or, if CPUID is not being intercepted,
> -    ; the MSR check below will set the first byte of the workarea to one.
> -    mov     byte[SEV_ES_WORK_AREA], 0
> -
>       ;
>       ; Set up exception handlers to check for SEV-ES
>       ;   Load temporary RAM stack based on PCDs (see SevEsIdtVmmComm for
> diff --git a/OvmfPkg/ResetVector/Ia32/Flat32ToFlat64.asm b/OvmfPkg/ResetVector/Ia32/Flat32ToFlat64.asm
> index c6d0d898bcd1..ea8723efd417 100644
> --- a/OvmfPkg/ResetVector/Ia32/Flat32ToFlat64.asm
> +++ b/OvmfPkg/ResetVector/Ia32/Flat32ToFlat64.asm
> @@ -17,6 +17,14 @@ Transition32FlatTo64Flat:
>   
>       OneTimeCall SetCr3ForPageTables64
>   
> +    OneTimeCall PostSetCr3PageTables64Tdx
> +
> +    ;
> +    ; If it is TDX, we're done and jump to enable paging
> +    ;
> +    cmp     byte[CC_WORK_AREA], VM_GUEST_TDX
> +    je      EnablePaging
> +
>       mov     eax, cr4
>       bts     eax, 5                      ; enable PAE
>       mov     cr4, eax
> @@ -71,6 +79,7 @@ jumpTo64BitAndLandHere:
>   
>       ;
>       ; Check if the second step of the SEV-ES mitigation is to be performed.
> +    ; If it is Tdx, ebx is cleared in PostSetCr3PageTables64Tdx.
>       ;
>       test    ebx, ebx
>       jz      InsnCompare
> diff --git a/OvmfPkg/ResetVector/Ia32/IntelTdx.asm b/OvmfPkg/ResetVector/Ia32/IntelTdx.asm
> new file mode 100644
> index 000000000000..f40331a5cbce
> --- /dev/null
> +++ b/OvmfPkg/ResetVector/Ia32/IntelTdx.asm
> @@ -0,0 +1,265 @@
> +;------------------------------------------------------------------------------
> +; @file
> +;   Intel TDX routines
> +;
> +; Copyright (c) 2021, Intel Corporation. All rights reserved.<BR>
> +; SPDX-License-Identifier: BSD-2-Clause-Patent
> +;
> +;------------------------------------------------------------------------------
> +
> +%define SEC_DEFAULT_CR0  0x00000023
> +%define SEC_DEFAULT_CR4  0x640
> +
> +%define VM_GUEST_TDX          2
> +
> +BITS 32
> +
> +;
> +; Check if it is Intel Tdx
> +;
> +; Modified: EAX, EBX, ECX, EDX
> +;
> +; If it is Intel Tdx, EAX is zero
> +; If it is not Intel Tdx, EAX is non-zero
> +;
> +IsTdx:
> +    ;
> +    ; CPUID (0)
> +    ;
> +    mov     eax, 0
> +    cpuid
> +    cmp     ebx, 0x756e6547  ; "Genu"
> +    jne     IsNotTdx
> +    cmp     edx, 0x49656e69  ; "ineI"
> +    jne     IsNotTdx
> +    cmp     ecx, 0x6c65746e  ; "ntel"
> +    jne     IsNotTdx
> +
> +    ;
> +    ; CPUID (1)
> +    ;
> +    mov     eax, 1
> +    cpuid
> +    test    ecx, 0x80000000
> +    jz      IsNotTdx
> +
> +    ;
> +    ; CPUID[0].EAX >= 0x21?
> +    ;
> +    mov     eax, 0
> +    cpuid
> +    cmp     eax, 0x21
> +    jl      IsNotTdx
> +
> +    ;
> +    ; CPUID (0x21,0)
> +    ;
> +    mov     eax, 0x21
> +    mov     ecx, 0
> +    cpuid
> +
> +    cmp     ebx, 0x65746E49   ; "Inte"
> +    jne     IsNotTdx
> +    cmp     edx, 0x5844546C   ; "lTDX"
> +    jne     IsNotTdx
> +    cmp     ecx, 0x20202020   ; "    "
> +    jne     IsNotTdx
> +
> +    mov     eax, 0
> +    jmp     ExitIsTdx
> +
> +IsNotTdx:
> +    mov     eax, 1
> +
> +ExitIsTdx:
> +
> +  OneTimeCallRet IsTdx
> +
> +;
> +; Initialize Tdx work area (CC_WORK_AREA) if it is Tdx guest.
> +; BSP and APs all go here. Only BSP can initialize the WORK_AREA
> +;
> +; typedef struct {
> +;   UINT8   Type;     // 0 legacy, 1 SEV, 2 TDX
> +;   UINT8   SubType;  // Depends on Type
> +;   UINT8   Rsvd[2];  // Reserved
> +;   union VM_GUEST {
> +;     TDX_WORK_AREA Tdx;
> +;     SEV_WORK_AREA Sev;
> +;   } Guest;
> +; } CC_WORK_AREA_HEAD;
> +;
> +; typedef struct {
> +;   UINT8    IsPageLevel5;
> +;   UINT8    IsPageTableReady;
> +;   UINT8    Rsvd[2];
> +;   UINT32   Gpaw;
> +; } TDX_WORK_AREA
> +;
> +; Param[in] EBP[6:0]    CPU Supported GPAW (48 or 52)
> +; Param[in] ESI[31:0]   vCPU ID (BSP is 0, others are AP)
> +;
> +; Modified:  EBP
> +;
> +InitTdxWorkarea:
> +
> +    ;
> +    ; First check if it is Tdx
> +    ;
> +    OneTimeCall IsTdx
> +
> +    test    eax, eax
> +    jnz     ExitInitTdxWorkarea
> +
> +    ;
> +    ; In Td guest, BSP/AP shares the same entry point
> +    ; BSP builds up the page table, while APs shouldn't do the same task.
> +    ; Instead, APs just leverage the page table which is built by BSP.
> +    ; APs will wait until the page table is ready.
> +    ;
> +    cmp     esi, 0
> +    je      TdxBspEntry
> +
> +TdxApWait:
> +    cmp     byte[TDX_WORK_AREA_PGTBL_READY], 0
> +    je      TdxApWait
> +    jmp     ExitInitTdxWorkarea
> +
> +TdxBspEntry:
> +    ;
> +    ; Set Type and SubType of CC_WORK_AREA so that the
> +    ; following code can use these information.
> +    ;
> +    mov     byte[CC_WORK_AREA], VM_GUEST_TDX
> +    mov     byte[CC_WORK_AREA_SUBTYPE], 0
> +
> +    ;
> +    ; EBP[6:0] CPU supported GPA width
> +    ;
> +    and     ebp, 0x3f
> +    cmp     ebp, 52
> +    jl      NotPageLevel5
> +    mov     byte[TDX_WORK_AREA_PAGELEVEL5], 1
> +
> +NotPageLevel5:
> +    mov     DWORD[TDX_WORK_AREA_GPAW], ebp
> +
> +ExitInitTdxWorkarea:
> +    OneTimeCallRet InitTdxWorkarea
> +
> +;
> +; Load the GDT and set the CR0, then jump to Flat 32 protected mode.
> +;
> +; Modified:  EAX, EBX, CR0, CR4, DS, ES, FS, GS, SS
> +;
> +ReloadFlat32:
> +
> +    cli
> +    mov     ebx, ADDR_OF(gdtr)
> +    lgdt    [ebx]
> +
> +    mov     eax, SEC_DEFAULT_CR0
> +    mov     cr0, eax
> +
> +    jmp     LINEAR_CODE_SEL:dword ADDR_OF(jumpToFlat32BitAndLandHere)
> +
> +jumpToFlat32BitAndLandHere:
> +
> +    mov     eax, SEC_DEFAULT_CR4
> +    mov     cr4, eax
> +
> +    debugShowPostCode POSTCODE_32BIT_MODE
> +
> +    mov     ax, LINEAR_SEL
> +    mov     ds, ax
> +    mov     es, ax
> +    mov     fs, ax
> +    mov     gs, ax
> +    mov     ss, ax
> +
> +    OneTimeCallRet ReloadFlat32
> +
> +;
> +; Tdx initialization after entering into ResetVector
> +;
> +; Modified:  EAX, EBX, ECX, EDX, EBP, EDI, ESP
> +;
> +InitTdx:
> +    ;
> +    ; Save EBX in EBP because EBX will be changed in ReloadFlat32
> +    ;
> +    mov     ebp, ebx
> +
> +    ;
> +    ; First load the GDT and jump to Flat32 mode
> +    ;
> +    OneTimeCall ReloadFlat32
> +
> +    ;
> +    ; Initialization of Tdx work area
> +    ;
> +    OneTimeCall  InitTdxWorkarea
> +
> +    OneTimeCallRet InitTdx
> +
> +;
> +; Called after SetCr3PageTables64 in Tdx guest to set CR0/CR4.
> +; If GPAW is 52, then CR3 is adjusted as well.
> +;
> +; Modified: EAX, EBX, CR0, CR3, CR4
> +;
> +PostSetCr3PageTables64Tdx:
> +    ;
> +    ; TDX_WORK_AREA was set in InitTdx if it is Tdx guest
> +    ;
> +    cmp     byte[CC_WORK_AREA], VM_GUEST_TDX
> +    jne     ExitPostSetCr3PageTables64Tdx
> +
> +    mov     eax, cr4
> +    bts     eax, 5                      ; enable PAE
> +
> +    ;
> +    ; byte[TDX_WORK_AREA_PAGELEVEL5] holds the indicator whether 52bit is
> +    ; supported. if it is the case, need to set LA57 and use 5-level paging
> +    ;
> +    cmp     byte[TDX_WORK_AREA_PAGELEVEL5], 0
> +    jz      TdxSetCr4
> +    bts     eax, 12
> +
> +TdxSetCr4:
> +    mov     cr4, eax
> +    mov     ebx, cr3
> +
> +    ;
> +    ; if la57 is not set, we are ok
> +    ; if using 5-level paging, adjust top-level page directory
> +    ;
> +    bt      eax, 12
> +    jnc     TdxSetCr3
> +    mov     ebx, TDX_PT_ADDR (0)
> +
> +TdxSetCr3:
> +    mov     cr3, ebx
> +
> +    xor     ebx, ebx
> +
> +ExitPostSetCr3PageTables64Tdx:
> +    OneTimeCallRet PostSetCr3PageTables64Tdx
> +
> +;
> +; Check if TDX is enabled
> +;
> +; Modified:  EAX
> +;
> +; If TDX is enabled then EAX will be 1
> +; If TDX is disabled then EAX will be 0.
> +;
> +IsTdxEnabled:
> +    xor     eax, eax
> +    cmp     byte[CC_WORK_AREA], VM_GUEST_TDX
> +    jne     NotTdx
> +    mov     eax, 1
> +
> +NotTdx:
> +    OneTimeCallRet IsTdxEnabled
> +
> diff --git a/OvmfPkg/ResetVector/Ia32/PageTables64.asm b/OvmfPkg/ResetVector/Ia32/PageTables64.asm
> index eacdb69ddb9f..1f4c0c50ac2e 100644
> --- a/OvmfPkg/ResetVector/Ia32/PageTables64.asm
> +++ b/OvmfPkg/ResetVector/Ia32/PageTables64.asm
> @@ -37,11 +37,81 @@ BITS    32
>                          PAGE_READ_WRITE + \
>                          PAGE_PRESENT)
>   
> +TdxBuildExtraPageTables:
> +    ;
> +    ; Extra page tables built by Tdx guests
> +    ;
> +    xor     eax, eax
> +    mov     ecx, 0x400
> +tdClearTdxPageTablesMemoryLoop:
> +    mov     dword [ecx * 4 + TDX_PT_ADDR(0) - 4], eax
> +    loop    tdClearTdxPageTablesMemoryLoop
> +
> +    ;
> +    ; Top level Page Directory Pointers (1 * 256TB entry)
> +    ;
> +    mov     dword[TDX_PT_ADDR (0)], PT_ADDR(0) + PAGE_PDP_ATTR
> +
> +    ;
> +    ; Set TDX_WORK_AREA_PGTBL_READY to notify APs to go
> +    ;
> +    mov     byte[TDX_WORK_AREA_PGTBL_READY], 1
> +
> +    OneTimeCallRet TdxBuildExtraPageTables
> +
> +SevBuildGhcbPageTables:
> +    ;
> +    ; The initial GHCB will live at GHCB_BASE and needs to be un-encrypted.
> +    ; This requires the 2MB page for this range be broken down into 512 4KB
> +    ; pages.  All will be marked encrypted, except for the GHCB.
> +    ;
> +    mov     ecx, (GHCB_BASE >> 21)
> +    mov     eax, GHCB_PT_ADDR + PAGE_PDP_ATTR
> +    mov     [ecx * 8 + PT_ADDR (0x2000)], eax
> +
> +    ;
> +    ; Page Table Entries (512 * 4KB entries => 2MB)
> +    ;
> +    mov     ecx, 512
> +pageTableEntries4kLoop:
> +    mov     eax, ecx
> +    dec     eax
> +    shl     eax, 12
> +    add     eax, GHCB_BASE & 0xFFE0_0000
> +    add     eax, PAGE_4K_PDE_ATTR
> +    mov     [ecx * 8 + GHCB_PT_ADDR - 8], eax
> +    mov     [(ecx * 8 + GHCB_PT_ADDR - 8) + 4], edx
> +    loop    pageTableEntries4kLoop
> +
> +    ;
> +    ; Clear the encryption bit from the GHCB entry
> +    ;
> +    mov     ecx, (GHCB_BASE & 0x1F_FFFF) >> 12
> +    mov     [ecx * 8 + GHCB_PT_ADDR + 4], strict dword 0
> +
> +    mov     ecx, GHCB_SIZE / 4
> +    xor     eax, eax
> +clearGhcbMemoryLoop:
> +    mov     dword[ecx * 4 + GHCB_BASE - 4], eax
> +    loop    clearGhcbMemoryLoop
> +
> +    OneTimeCallRet  SevBuildGhcbPageTables
> +
>   ;
>   ; Modified:  EAX, EBX, ECX, EDX
>   ;
>   SetCr3ForPageTables64:
>   
> +    cmp     byte[CC_WORK_AREA], VM_GUEST_TDX
> +    jne     CheckSev
> +
> +    cmp     byte[TDX_WORK_AREA_PGTBL_READY], 1
> +    je      SetCr3
> +
> +    xor     edx, edx
> +    jmp     SevNotActive
> +
> +CheckSev:
>       OneTimeCall   CheckSevFeatures
>       xor     edx, edx
>       test    eax, eax
> @@ -101,44 +171,19 @@ pageTableEntriesLoop:
>       mov     [(ecx * 8 + PT_ADDR (0x2000 - 8)) + 4], edx
>       loop    pageTableEntriesLoop
>   
> +    OneTimeCall IsTdxEnabled
> +    test    eax, eax
> +    jz      IsSevEs
> +
> +    OneTimeCall TdxBuildExtraPageTables
> +    jmp     SetCr3
> +
> +IsSevEs:
>       OneTimeCall   IsSevEsEnabled
>       test    eax, eax
>       jz      SetCr3
>   
> -    ;
> -    ; The initial GHCB will live at GHCB_BASE and needs to be un-encrypted.
> -    ; This requires the 2MB page for this range be broken down into 512 4KB
> -    ; pages.  All will be marked encrypted, except for the GHCB.
> -    ;
> -    mov     ecx, (GHCB_BASE >> 21)
> -    mov     eax, GHCB_PT_ADDR + PAGE_PDP_ATTR
> -    mov     [ecx * 8 + PT_ADDR (0x2000)], eax
> -
> -    ;
> -    ; Page Table Entries (512 * 4KB entries => 2MB)
> -    ;
> -    mov     ecx, 512
> -pageTableEntries4kLoop:
> -    mov     eax, ecx
> -    dec     eax
> -    shl     eax, 12
> -    add     eax, GHCB_BASE & 0xFFE0_0000
> -    add     eax, PAGE_4K_PDE_ATTR
> -    mov     [ecx * 8 + GHCB_PT_ADDR - 8], eax
> -    mov     [(ecx * 8 + GHCB_PT_ADDR - 8) + 4], edx
> -    loop    pageTableEntries4kLoop
> -
> -    ;
> -    ; Clear the encryption bit from the GHCB entry
> -    ;
> -    mov     ecx, (GHCB_BASE & 0x1F_FFFF) >> 12
> -    mov     [ecx * 8 + GHCB_PT_ADDR + 4], strict dword 0
> -
> -    mov     ecx, GHCB_SIZE / 4
> -    xor     eax, eax
> -clearGhcbMemoryLoop:
> -    mov     dword[ecx * 4 + GHCB_BASE - 4], eax
> -    loop    clearGhcbMemoryLoop
> +    OneTimeCall   SevBuildGhcbPageTables
>   
>   SetCr3:
>       ;
> diff --git a/OvmfPkg/ResetVector/Main.asm b/OvmfPkg/ResetVector/Main.asm
> new file mode 100644
> index 000000000000..fb1731728af5
> --- /dev/null
> +++ b/OvmfPkg/ResetVector/Main.asm
> @@ -0,0 +1,121 @@
> +;------------------------------------------------------------------------------
> +; @file
> +; Main routine of the pre-SEC code up through the jump into SEC
> +;
> +; Copyright (c) 2008 - 2009, Intel Corporation. All rights reserved.<BR>
> +; SPDX-License-Identifier: BSD-2-Clause-Patent
> +;
> +;------------------------------------------------------------------------------
> +
> +
> +BITS    16
> +
> +;
> +; Modified:  EBX, ECX, EDX, EBP
> +;
> +; @param[in,out]  RAX/EAX  Initial value of the EAX register
> +;                          (BIST: Built-in Self Test)
> +; @param[in,out]  DI       'BP': boot-strap processor, or
> +;                          'AP': application processor
> +; @param[out]     RBP/EBP  Address of Boot Firmware Volume (BFV)
> +; @param[out]     DS       Selector allowing flat access to all addresses
> +; @param[out]     ES       Selector allowing flat access to all addresses
> +; @param[out]     FS       Selector allowing flat access to all addresses
> +; @param[out]     GS       Selector allowing flat access to all addresses
> +; @param[out]     SS       Selector allowing flat access to all addresses
> +;
> +; @return         None  This routine jumps to SEC and does not return
> +;
> +Main16:
> +    OneTimeCall EarlyInit16
> +
> +    ;
> +    ; Transition the processor from 16-bit real mode to 32-bit flat mode
> +    ;
> +    OneTimeCall TransitionFromReal16To32BitFlat
> +
> +BITS    32
> +%ifdef ARCH_X64
> +    ;
> +    ; Clear the first 2 bytes of CC_WORK_AREA.
> +    ;
> +    mov word[CC_WORK_AREA], 0
> +
> +    jmp SearchBfv
> +
> +;
> +; Entry point of Main32
> +;
> +Main32:
> +    OneTimeCall InitTdx
> +
> +SearchBfv:
> +
> +%endif
> +    ;
> +    ; Search for the Boot Firmware Volume (BFV)
> +    ;
> +    OneTimeCall Flat32SearchForBfvBase
> +
> +    ;
> +    ; EBP - Start of BFV
> +    ;
> +
> +    ;
> +    ; Search for the SEC entry point
> +    ;
> +    OneTimeCall Flat32SearchForSecEntryPoint
> +
> +    ;
> +    ; ESI - SEC Core entry point
> +    ; EBP - Start of BFV
> +    ;
> +
> +%ifdef ARCH_IA32
> +
> +    ;
> +    ; Restore initial EAX value into the EAX register
> +    ;
> +    mov     eax, esp
> +
> +    ;
> +    ; Jump to the 32-bit SEC entry point
> +    ;
> +    jmp     esi
> +
> +%else
> +
> +    ;
> +    ; Transition the processor from 32-bit flat mode to 64-bit flat mode
> +    ;
> +    OneTimeCall Transition32FlatTo64Flat
> +
> +BITS    64
> +
> +    ;
> +    ; Some values were calculated in 32-bit mode.  Make sure the upper
> +    ; 32-bits of 64-bit registers are zero for these values.
> +    ;
> +    mov     rax, 0x00000000ffffffff
> +    and     rsi, rax
> +    and     rbp, rax
> +    and     rsp, rax
> +
> +    ;
> +    ; RSI - SEC Core entry point
> +    ; RBP - Start of BFV
> +    ;
> +
> +    ;
> +    ; Restore initial EAX value into the RAX register
> +    ;
> +    mov     rax, rsp
> +
> +    ;
> +    ; Jump to the 64-bit SEC entry point
> +    ;
> +    jmp     rsi
> +
> +%endif
> +
> +
> diff --git a/OvmfPkg/ResetVector/ResetVector.inf b/OvmfPkg/ResetVector/ResetVector.inf
> index d028c92d8cfa..0cf3a8036b97 100644
> --- a/OvmfPkg/ResetVector/ResetVector.inf
> +++ b/OvmfPkg/ResetVector/ResetVector.inf
> @@ -1,7 +1,7 @@
>   ## @file
>   #  Reset Vector
>   #
> -#  Copyright (c) 2006 - 2014, Intel Corporation. All rights reserved.<BR>
> +#  Copyright (c) 2006 - 2021, Intel Corporation. All rights reserved.<BR>
>   #
>   #  SPDX-License-Identifier: BSD-2-Clause-Patent
>   #
> @@ -35,6 +35,7 @@
>   
>   [Pcd]
>     gUefiCpuPkgTokenSpaceGuid.PcdSevEsWorkAreaBase
> +  gUefiCpuPkgTokenSpaceGuid.PcdSevEsWorkAreaSize
>     gUefiOvmfPkgTokenSpaceGuid.PcdOvmfSecGhcbBase
>     gUefiOvmfPkgTokenSpaceGuid.PcdOvmfSecGhcbSize
>     gUefiOvmfPkgTokenSpaceGuid.PcdOvmfSecGhcbPageTableBase
> @@ -43,6 +44,15 @@
>     gUefiOvmfPkgTokenSpaceGuid.PcdOvmfSecPageTablesSize
>     gUefiOvmfPkgTokenSpaceGuid.PcdOvmfSecPeiTempRamBase
>     gUefiOvmfPkgTokenSpaceGuid.PcdOvmfSecPeiTempRamSize
> +  gUefiOvmfPkgTokenSpaceGuid.PcdOvmfSecGhcbBackupBase
> +  gUefiOvmfPkgTokenSpaceGuid.PcdOvmfSecGhcbBackupSize
> +  gUefiOvmfPkgTokenSpaceGuid.PcdOvmfImageSizeInKb
> +  gUefiOvmfPkgTokenSpaceGuid.PcdCfvBase
> +  gUefiOvmfPkgTokenSpaceGuid.PcdCfvRawDataOffset
> +  gUefiOvmfPkgTokenSpaceGuid.PcdCfvRawDataSize
> +  gUefiOvmfPkgTokenSpaceGuid.PcdBfvBase
> +  gUefiOvmfPkgTokenSpaceGuid.PcdBfvRawDataOffset
> +  gUefiOvmfPkgTokenSpaceGuid.PcdBfvRawDataSize
>   
>   [FixedPcd]
>     gUefiOvmfPkgTokenSpaceGuid.PcdSevLaunchSecretBase
> diff --git a/OvmfPkg/ResetVector/ResetVector.nasmb b/OvmfPkg/ResetVector/ResetVector.nasmb
> index acec46a32450..528a6603a20e 100644
> --- a/OvmfPkg/ResetVector/ResetVector.nasmb
> +++ b/OvmfPkg/ResetVector/ResetVector.nasmb
> @@ -67,6 +67,44 @@
>       %error "This implementation inherently depends on PcdOvmfSecGhcbBase not straddling a 2MB boundary"
>     %endif
>   
> +  %define TDX_BFV_RAW_DATA_OFFSET   FixedPcdGet32 (PcdBfvRawDataOffset)
> +  %define TDX_BFV_RAW_DATA_SIZE     FixedPcdGet32 (PcdBfvRawDataSize)
> +  %define TDX_BFV_MEMORY_BASE       FixedPcdGet32 (PcdBfvBase)
> +  %define TDX_BFV_MEMORY_SIZE       FixedPcdGet32 (PcdBfvRawDataSize)
> +
> +  %define TDX_CFV_RAW_DATA_OFFSET   FixedPcdGet32 (PcdCfvRawDataOffset)
> +  %define TDX_CFV_RAW_DATA_SIZE     FixedPcdGet32 (PcdCfvRawDataSize)
> +  %define TDX_CFV_MEMORY_BASE       FixedPcdGet32 (PcdCfvBase),
> +  %define TDX_CFV_MEMORY_SIZE       FixedPcdGet32 (PcdCfvRawDataSize),
> +
> +  %define TDX_HEAP_MEMORY_BASE      FixedPcdGet32 (PcdOvmfSecPeiTempRamBase)
> +  %define TDX_HEAP_MEMORY_SIZE      FixedPcdGet32 (PcdOvmfSecPeiTempRamSize) / 2
> +
> +  %define TDX_STACK_MEMORY_BASE     (TDX_HEAP_MEMORY_BASE + TDX_HEAP_MEMORY_SIZE)
> +  %define TDX_STACK_MEMORY_SIZE     FixedPcdGet32 (PcdOvmfSecPeiTempRamSize) / 2
> +
> +  %define TDX_HOB_MEMORY_BASE       FixedPcdGet32 (PcdOvmfSecGhcbBase)
> +  %define TDX_HOB_MEMORY_SIZE       FixedPcdGet32 (PcdOvmfSecGhcbSize)
> +
> +  %define TDX_MAILBOX_MEMORY_BASE   FixedPcdGet32 (PcdOvmfSecGhcbBackupBase)
> +  %define TDX_MAILBOX_MEMORY_SIZE   FixedPcdGet32 (PcdOvmfSecGhcbBackupSize)
> +
> +  %define OVMF_PAGE_TABLE_BASE      FixedPcdGet32 (PcdOvmfSecPageTablesBase)
> +  %define OVMF_PAGE_TABLE_SIZE      FixedPcdGet32 (PcdOvmfSecPageTablesSize)
> +
> +  %define TDX_EXTRA_PAGE_TABLE_BASE FixedPcdGet32 (PcdOvmfSecGhcbPageTableBase)
> +  %define TDX_EXTRA_PAGE_TABLE_SIZE FixedPcdGet32 (PcdOvmfSecGhcbPageTableSize)
> +
> +  %define CC_WORK_AREA_MEMORY_BASE  FixedPcdGet32 (PcdSevEsWorkAreaBase)
> +  %define CC_WORK_AREA_MEMORY_SIZE  FixedPcdGet32 (PcdSevEsWorkAreaSize)
> +  %define CC_WORK_AREA              (CC_WORK_AREA_MEMORY_BASE)
> +  %define CC_WORK_AREA_SUBTYPE      (CC_WORK_AREA + 1)
> +  %define TDX_WORK_AREA_PAGELEVEL5  (CC_WORK_AREA + 4)
> +  %define TDX_WORK_AREA_PGTBL_READY (CC_WORK_AREA + 5)
> +  %define TDX_WORK_AREA_GPAW        (CC_WORK_AREA + 8)
> +
> +  %define TDX_PT_ADDR(Offset)       (TDX_EXTRA_PAGE_TABLE_BASE + (Offset))
> +
>     %define PT_ADDR(Offset) (FixedPcdGet32 (PcdOvmfSecPageTablesBase) + (Offset))
>   
>     %define GHCB_PT_ADDR (FixedPcdGet32 (PcdOvmfSecGhcbPageTableBase))
> @@ -76,9 +114,12 @@
>     %define SEV_ES_WORK_AREA_RDRAND (FixedPcdGet32 (PcdSevEsWorkAreaBase) + 8)
>     %define SEV_ES_WORK_AREA_ENC_MASK (FixedPcdGet32 (PcdSevEsWorkAreaBase) + 16)
>     %define SEV_ES_VC_TOP_OF_STACK (FixedPcdGet32 (PcdOvmfSecPeiTempRamBase) + FixedPcdGet32 (PcdOvmfSecPeiTempRamSize))
> -%include "Ia32/Flat32ToFlat64.asm"
> -%include "Ia32/AmdSev.asm"
> -%include "Ia32/PageTables64.asm"
> +
> +  %include "X64/IntelTdxMetadata.asm"
> +  %include "Ia32/IntelTdx.asm"
> +  %include "Ia32/Flat32ToFlat64.asm"
> +  %include "Ia32/AmdSev.asm"
> +  %include "Ia32/PageTables64.asm"
>   %endif
>   
>   %include "Ia16/Real16ToFlat32.asm"
> @@ -91,5 +132,6 @@
>     %define SEV_LAUNCH_SECRET_SIZE  FixedPcdGet32 (PcdSevLaunchSecretSize)
>     %define SEV_FW_HASH_BLOCK_BASE  FixedPcdGet32 (PcdQemuHashTableBase)
>     %define SEV_FW_HASH_BLOCK_SIZE  FixedPcdGet32 (PcdQemuHashTableSize)
> +  %define OVMF_IMAGE_SIZE_IN_KB   FixedPcdGet32 (PcdOvmfImageSizeInKb)
>   %include "Ia16/ResetVectorVtf0.asm"
>   
> diff --git a/OvmfPkg/ResetVector/X64/IntelTdxMetadata.asm b/OvmfPkg/ResetVector/X64/IntelTdxMetadata.asm
> new file mode 100644
> index 000000000000..97b1e915d899
> --- /dev/null
> +++ b/OvmfPkg/ResetVector/X64/IntelTdxMetadata.asm
> @@ -0,0 +1,110 @@
> +;------------------------------------------------------------------------------
> +; @file
> +; Tdx Virtual Firmware metadata
> +;
> +; Copyright (c) 2021, Intel Corporation. All rights reserved.<BR>
> +; SPDX-License-Identifier: BSD-2-Clause-Patent
> +;
> +;------------------------------------------------------------------------------
> +
> +BITS    64
> +
> +%define TDX_METADATA_SECTION_TYPE_BFV       0
> +%define TDX_METADATA_SECTION_TYPE_CFV       1
> +%define TDX_METADATA_SECTION_TYPE_TD_HOB    2
> +%define TDX_METADATA_SECTION_TYPE_TEMP_MEM  3
> +%define TDX_METADATA_VERSION                1
> +%define TDX_METADATA_ATTRIBUTES_EXTENDMR    0x00000001
> +
> +ALIGN   16
> +TIMES (15 - ((TdxGuidedStructureEnd - TdxGuidedStructureStart + 15) % 16)) DB 0
> +
> +TdxGuidedStructureStart:
> +
> +;
> +; TDVF meta data
> +;
> +TdxMetadataGuid:
> +  DB  0xf3, 0xf9, 0xea, 0xe9, 0x8e, 0x16, 0xd5, 0x44
> +  DB  0xa8, 0xeb, 0x7f, 0x4d, 0x87, 0x38, 0xf6, 0xae
> +
> +_Descriptor:
> +  DB 'T','D','V','F'                                  ; Signature
> +  DD TdxGuidedStructureEnd - _Descriptor              ; Length
> +  DD TDX_METADATA_VERSION                             ; Version
> +  DD (TdxGuidedStructureEnd - _Descriptor - 16)/32    ; Number of sections
> +
> +_Bfv:
> +  DD TDX_BFV_RAW_DATA_OFFSET
> +  DD TDX_BFV_RAW_DATA_SIZE
> +  DQ TDX_BFV_MEMORY_BASE
> +  DQ TDX_BFV_MEMORY_SIZE
> +  DD TDX_METADATA_SECTION_TYPE_BFV
> +  DD TDX_METADATA_ATTRIBUTES_EXTENDMR
> +
> +_Cfv:
> +  DD TDX_CFV_RAW_DATA_OFFSET
> +  DD TDX_CFV_RAW_DATA_SIZE
> +  DQ TDX_CFV_MEMORY_BASE
> +  DQ TDX_CFV_MEMORY_SIZE
> +  DD TDX_METADATA_SECTION_TYPE_CFV
> +  DD 0
> +
> +_Stack:
> +  DD 0
> +  DD 0
> +  DQ TDX_STACK_MEMORY_BASE
> +  DQ TDX_STACK_MEMORY_SIZE
> +  DD TDX_METADATA_SECTION_TYPE_TEMP_MEM
> +  DD 0
> +
> +_Heap:
> +  DD 0
> +  DD 0
> +  DQ TDX_HEAP_MEMORY_BASE
> +  DQ TDX_HEAP_MEMORY_SIZE
> +  DD TDX_METADATA_SECTION_TYPE_TEMP_MEM
> +  DD 0
> +
> +_MailBox:
> +  DD 0
> +  DD 0
> +  DQ TDX_MAILBOX_MEMORY_BASE
> +  DQ TDX_MAILBOX_MEMORY_SIZE
> +  DD TDX_METADATA_SECTION_TYPE_TEMP_MEM
> +  DD 0
> +
> +_CcWorkarea:
> +  DD 0
> +  DD 0
> +  DQ CC_WORK_AREA_MEMORY_BASE
> +  DQ CC_WORK_AREA_MEMORY_SIZE
> +  DD TDX_METADATA_SECTION_TYPE_TEMP_MEM
> +  DD 0
> +
> +_TdHob:
> +  DD 0
> +  DD 0
> +  DQ TDX_HOB_MEMORY_BASE
> +  DQ TDX_HOB_MEMORY_SIZE
> +  DD TDX_METADATA_SECTION_TYPE_TD_HOB
> +  DD 0
> +
> +_TdxPageTable:
> +  DD 0
> +  DD 0
> +  DQ TDX_EXTRA_PAGE_TABLE_BASE
> +  DQ TDX_EXTRA_PAGE_TABLE_SIZE
> +  DD TDX_METADATA_SECTION_TYPE_TEMP_MEM
> +  DD 0
> +
> +_OvmfPageTable:
> +  DD 0
> +  DD 0
> +  DQ OVMF_PAGE_TABLE_BASE
> +  DQ OVMF_PAGE_TABLE_SIZE
> +  DD TDX_METADATA_SECTION_TYPE_TEMP_MEM
> +  DD 0
> +
> +TdxGuidedStructureEnd:
> +ALIGN   16
>