Re: [PATCH V4 3/3] OvmfPkg/ResetVector: Enable Intel TDX in ResetVector of Ovmf

["Erdem Aktas" <erdemaktas@google.com>]

I agree with Brijesh on that this patch should be divided into smaller ones.

On Mon, Aug 2, 2021 at 6:18 PM Min Xu <min.m.xu@intel.com> wrote:
>
> +    ;
> +    ; EBP[6:0] CPU supported GPA width
> +    ;
> +    and     ebp, 0x3f

Based on https://software.intel.com/content/dam/develop/external/us/en/documents/tdx-module-1eas-v0.85.039.pdf
 sec 8.1.2
Is not this EBP[5:0] for GPA width ? it should be "and ebp, 0x1f"

-Erdem

On Mon, Aug 2, 2021 at 6:18 PM Min Xu <min.m.xu@intel.com> wrote:
>
> RFC: https://bugzilla.tianocore.org/show_bug.cgi?id=3429
>
> Intel's Trust Domain Extensions (Intel TDX) refers to an Intel technology
> that extends Virtual Machines Extensions (VMX) and Multi-Key Total Memory
> Encryption (MKTME) with a new kind of virutal machines guest called a
> Trust Domain (TD). A TD is desinged to run in a CPU mode that protects the
> confidentiality of TD memory contents and the TD's CPU state from other
> software, including the hosting Virtual-Machine Monitor (VMM), unless
> explicitly shared by the TD itself.
>
> Note: Intel TDX only is available on X64, so the Tdx related changes are
> in X64 path. In IA32 path, there may be null stub to make the build
> success.
>
> This patch includes below major changes.
>
> 1. CC_WORK_AREA
> Ovmf is able to run on different types of VM guest with ONE image. These
> VM guest includes the Legacy guest, SEV guest and TDX guest. In
> ResetVector CC_WORK_AREA is such a memory region to record the VM guest
> information. Below is the definition of the work area.
>
>  typedef struct {
>    UINT8   Type;     // 0 legacy, 1 SEV, 2 TDX
>    UINT8   SubType;  // Depends on Type
>    UINT8   Rsvd[2];  // Reserved
>    union VM_GUEST {
>      TDX_WORK_AREA Tdx;
>      SEV_WORK_AREA Sev;
>    } Guest;
>  } CC_WORK_AREA_HEAD;
>
>  typedef struct {
>    ... ...
>  } TDX_WORK_AREA
>
>  typedef struct {
>    ... ...
>  } SEV_WORK_AREA
>
> 2. X64/IntelTdxMetadata.asm
> IntelTdxMetadata describes the information about the image for VMM use.
> For example, the base address and length of the TdHob, TdMailbox, etc.
> Its offset is put in a GUID-ed structure which is appended in the GUID-ed
> chain from a fixed GPA (0xffffffd0). Below are the items in TdxMetadata:
>  _Bfv:
>     Boot Firmware Volume
>  _Cfv:
>     Configuration Firmware Volume
>  _Stack:
>     Initial stack
>  _Heap:
>     Initial heap
>  _MailBox:
>     TDVF reserves the memory region so each AP can receive the message
>     sent by the guest OS.
>  _CcWorkarea:
>     Compute Confidential work area which is consumed by CC technologies,
>     such as SEV, TDX.
>  _TdHob:
>     VMM pass the resource information in TdHob to TDVF.
>  _TdxPageTable:
>     If 5-level page table is supported (GPAW is 52), a top level page
>     directory pointers (1 * 256TB entry) is generated in this page.
>  _OvmfPageTable:
>     Initial page table for standard Ovmf.
>
> TDVF indicate above chunk of temporary initialized memory region (_Stack/
> _Heap/_MailBox/_CcWorkarea/_TdHob/_TdxPageTables/OvmfPageTable) to support
> TDVF code finishing the memory initialization. Because the other
> unaccepted memory cannot be accessed until they're accepted.
>
> Since AMD SEV has already defined some SEV specific memory region in
> MEMFD. TDX re-use the memory regions defined by SEV.
>  - MailBox      : PcdOvmfSecGhcbBackupBase|PcdOvmfSecGhcbBackupSize
>  - TdHob        : PcdOvmfSecGhcbBase|PcdOvmfSecGhcbSize
>  - TdxPageTable : PcdOvmfSecGhcbPageTableBase|PcdOvmfSecGhcbPageTableSize
>  - CcWorkArea   : PcdSevEsWorkAreaBase|PcdSevEsWorkAreaSize
>
> 3. Ia32/IntelTdx.asm
> IntelTdx.asm includes below routines used in ResetVector
>  - IsTdx
>    Check if the running system is Tdx guest.
>
>  - InitTdxWorkarea
>    It initialize the CC_WORK_AREA. Because it is called by both BSP and
>    APs and to avoid the race condition, only BSP can initialize the
>    WORK_AREA. AP will wait until the field of TDX_WORK_AREA_PGTBL_READY
>    is set.
>
>  - ReloadFlat32
>    After reset all CPUs in TDX are initialized to 32-bit protected mode.
>    But GDT register is not set. So this routine loads the GDT and set the
>    CR0, then jump to Flat 32 protected mode. After that CR4 and other
>    registers are set.
>
>  - InitTdx
>    This routine wrap above 3 routines together to do Tdx initialization
>    in ResetVector phase.
>
>  - PostSetCr3PageTables64Tdx
>    It is called after SetCr3PageTables64 in Tdx guest to set CR0/CR4.
>    If GPAW is 52, then CR3 is adjusted as well.
>
>  - IsTdxEnabled
>    It is a OneTimeCall to probe if TDX is enabled by checking the
>    CC_WORK_AREA.
>
> 4. Ia32/AmdSev.asm
> AmdSev.asm includes the SEV routines used in ResetVector. This patch
> remove the code of clearing SEV_ES_WORK_AREA in CheckSevFeatures. The
> clearing of SEV_ES_WORK_AREA is called at Main16 in Main.asm.
>
> 5. Main.asm
> Previously OvmfPkg/ResetVector use the Main.asm in UefiCpuPkg. There is
> only Main16 entry point. Main32 entry point is needed in Main.asm because
> of Intel TDX. To reduce the complexity of Main.asm in UefiCpuPkg, OvmfPkg
> create its own Main.asm to meet the requirement of Intel TDX. There are
> below changes in this Main.asm:
>  - A new entry point (Main32) is added. TDX guest will jump to Main32
>    from ResetVecotr.
>  - In Main16 entry point, after TransitionFromReal16To32BitFlat,
>    CC_WORK_AREA is cleared to 0.
>
> 6. Ia32/PageTables64.asm
> GPAW of TDX can be 48 or 52, which determines the level of page table.
> If Level-5(GPAW 52) paging is supported, then an extra page is needed
> to hold the top level Page Directory Pointers (1 * 256TB entry).
>
> 7. Ia16/ResetVectorVtf0.asm
> In Tdx all CPUs "reset" to run on 32-bit protected mode with flat
> descriptor (paging disabled). But in Non-Td guest the initial state of
> CPUs is 16-bit real mode. To resolve this conflict, BITS 16/32 is used
> in the ResetVectorVtf0.asm. It checks the 32-bit protected mode or 16-bit
> real mode, then jump to the corresponding entry point.
>
> 8. ResetVector.nasmb
> TDX related macros and files are added in ResetVecotr.nasmb.
>
> Cc: Ard Biesheuvel <ardb+tianocore@kernel.org>
> Cc: Jordan Justen <jordan.l.justen@intel.com>
> Cc: Brijesh Singh <brijesh.singh@amd.com>
> Cc: Erdem Aktas <erdemaktas@google.com>
> Cc: James Bottomley <jejb@linux.ibm.com>
> Cc: Jiewen Yao <jiewen.yao@intel.com>
> Cc: Tom Lendacky <thomas.lendacky@amd.com>
> Signed-off-by: Min Xu <min.m.xu@intel.com>
> ---
>  OvmfPkg/ResetVector/Ia16/ResetVectorVtf0.asm |  39 +++
>  OvmfPkg/ResetVector/Ia32/AmdSev.asm          |   7 -
>  OvmfPkg/ResetVector/Ia32/Flat32ToFlat64.asm  |   9 +
>  OvmfPkg/ResetVector/Ia32/IntelTdx.asm        | 265 +++++++++++++++++++
>  OvmfPkg/ResetVector/Ia32/PageTables64.asm    | 113 +++++---
>  OvmfPkg/ResetVector/Main.asm                 | 121 +++++++++
>  OvmfPkg/ResetVector/ResetVector.inf          |  12 +-
>  OvmfPkg/ResetVector/ResetVector.nasmb        |  48 +++-
>  OvmfPkg/ResetVector/X64/IntelTdxMetadata.asm | 110 ++++++++
>  9 files changed, 679 insertions(+), 45 deletions(-)
>  create mode 100644 OvmfPkg/ResetVector/Ia32/IntelTdx.asm
>  create mode 100644 OvmfPkg/ResetVector/Main.asm
>  create mode 100644 OvmfPkg/ResetVector/X64/IntelTdxMetadata.asm
>
> diff --git a/OvmfPkg/ResetVector/Ia16/ResetVectorVtf0.asm b/OvmfPkg/ResetVector/Ia16/ResetVectorVtf0.asm
> index 7ec3c6e980c3..62feeacbee7b 100644
> --- a/OvmfPkg/ResetVector/Ia16/ResetVectorVtf0.asm
> +++ b/OvmfPkg/ResetVector/Ia16/ResetVectorVtf0.asm
> @@ -47,6 +47,24 @@ TIMES (15 - ((guidedStructureEnd - guidedStructureStart + 15) % 16)) DB 0
>  ;
>  guidedStructureStart:
>
> +%ifdef ARCH_X64
> +;
> +; TDX Metadata offset block
> +;
> +; If TdxMetadata.asm is included then we need below block which describes
> +; the offset of TdxMetadata block in Ovmf image
> +;
> +; GUID : e47a6535-984a-4798-865e-4685a7bf8ec2
> +;
> +tdxMetadataOffsetStart:
> +    DD      (OVMF_IMAGE_SIZE_IN_KB * 1024 - (fourGigabytes - TdxMetadataGuid - 16))
> +    DD      tdxMetadataOffsetEnd - tdxMetadataOffsetStart
> +    DB      0x35, 0x65, 0x7a, 0xe4, 0x4a, 0x98, 0x98, 0x47
> +    DB      0x86, 0x5e, 0x46, 0x85, 0xa7, 0xbf, 0x8e, 0xc2
> +tdxMetadataOffsetEnd:
> +
> +%endif
> +
>  ; SEV Hash Table Block
>  ;
>  ; This describes the guest ram area where the hypervisor should
> @@ -158,10 +176,31 @@ resetVector:
>  ;
>  ; This is where the processor will begin execution
>  ;
> +; In IA32 we follow the standard reset vector flow. While in X64, Td guest
> +; may be supported. Td guest requires the startup mode to be 32-bit
> +; protected mode but the legacy VM startup mode is 16-bit real mode.
> +; To make NASM generate such shared entry code that behaves correctly in
> +; both 16-bit and 32-bit mode, more BITS directives are added.
> +;
> +%ifdef ARCH_IA32
> +
>      nop
>      nop
>      jmp     EarlyBspInitReal16
>
> +%else
> +
> +    smsw    ax
> +    test    al, 1
> +    jz      .Real
> +BITS 32
> +    jmp     Main32
> +BITS 16
> +.Real:
> +    jmp     EarlyBspInitReal16
> +
> +%endif
> +
>  ALIGN   16
>
>  fourGigabytes:
> diff --git a/OvmfPkg/ResetVector/Ia32/AmdSev.asm b/OvmfPkg/ResetVector/Ia32/AmdSev.asm
> index aa95d06eaddb..d0d9890d5c4b 100644
> --- a/OvmfPkg/ResetVector/Ia32/AmdSev.asm
> +++ b/OvmfPkg/ResetVector/Ia32/AmdSev.asm
> @@ -128,13 +128,6 @@ SevEsUnexpectedRespTerminate:
>  ; If SEV is disabled then EAX will be zero.
>  ;
>  CheckSevFeatures:
> -    ; Set the first byte of the workarea to zero to communicate to the SEC
> -    ; phase that SEV-ES is not enabled. If SEV-ES is enabled, the CPUID
> -    ; instruction will trigger a #VC exception where the first byte of the
> -    ; workarea will be set to one or, if CPUID is not being intercepted,
> -    ; the MSR check below will set the first byte of the workarea to one.
> -    mov     byte[SEV_ES_WORK_AREA], 0
> -
>      ;
>      ; Set up exception handlers to check for SEV-ES
>      ;   Load temporary RAM stack based on PCDs (see SevEsIdtVmmComm for
> diff --git a/OvmfPkg/ResetVector/Ia32/Flat32ToFlat64.asm b/OvmfPkg/ResetVector/Ia32/Flat32ToFlat64.asm
> index c6d0d898bcd1..ea8723efd417 100644
> --- a/OvmfPkg/ResetVector/Ia32/Flat32ToFlat64.asm
> +++ b/OvmfPkg/ResetVector/Ia32/Flat32ToFlat64.asm
> @@ -17,6 +17,14 @@ Transition32FlatTo64Flat:
>
>      OneTimeCall SetCr3ForPageTables64
>
> +    OneTimeCall PostSetCr3PageTables64Tdx
> +
> +    ;
> +    ; If it is TDX, we're done and jump to enable paging
> +    ;
> +    cmp     byte[CC_WORK_AREA], VM_GUEST_TDX
> +    je      EnablePaging
> +
>      mov     eax, cr4
>      bts     eax, 5                      ; enable PAE
>      mov     cr4, eax
> @@ -71,6 +79,7 @@ jumpTo64BitAndLandHere:
>
>      ;
>      ; Check if the second step of the SEV-ES mitigation is to be performed.
> +    ; If it is Tdx, ebx is cleared in PostSetCr3PageTables64Tdx.
>      ;
>      test    ebx, ebx
>      jz      InsnCompare
> diff --git a/OvmfPkg/ResetVector/Ia32/IntelTdx.asm b/OvmfPkg/ResetVector/Ia32/IntelTdx.asm
> new file mode 100644
> index 000000000000..f40331a5cbce
> --- /dev/null
> +++ b/OvmfPkg/ResetVector/Ia32/IntelTdx.asm
> @@ -0,0 +1,265 @@
> +;------------------------------------------------------------------------------
> +; @file
> +;   Intel TDX routines
> +;
> +; Copyright (c) 2021, Intel Corporation. All rights reserved.<BR>
> +; SPDX-License-Identifier: BSD-2-Clause-Patent
> +;
> +;------------------------------------------------------------------------------
> +
> +%define SEC_DEFAULT_CR0  0x00000023
> +%define SEC_DEFAULT_CR4  0x640
> +
> +%define VM_GUEST_TDX          2
> +
> +BITS 32
> +
> +;
> +; Check if it is Intel Tdx
> +;
> +; Modified: EAX, EBX, ECX, EDX
> +;
> +; If it is Intel Tdx, EAX is zero
> +; If it is not Intel Tdx, EAX is non-zero
> +;
> +IsTdx:
> +    ;
> +    ; CPUID (0)
> +    ;
> +    mov     eax, 0
> +    cpuid
> +    cmp     ebx, 0x756e6547  ; "Genu"
> +    jne     IsNotTdx
> +    cmp     edx, 0x49656e69  ; "ineI"
> +    jne     IsNotTdx
> +    cmp     ecx, 0x6c65746e  ; "ntel"
> +    jne     IsNotTdx
> +
> +    ;
> +    ; CPUID (1)
> +    ;
> +    mov     eax, 1
> +    cpuid
> +    test    ecx, 0x80000000
> +    jz      IsNotTdx
> +
> +    ;
> +    ; CPUID[0].EAX >= 0x21?
> +    ;
> +    mov     eax, 0
> +    cpuid
> +    cmp     eax, 0x21
> +    jl      IsNotTdx
> +
> +    ;
> +    ; CPUID (0x21,0)
> +    ;
> +    mov     eax, 0x21
> +    mov     ecx, 0
> +    cpuid
> +
> +    cmp     ebx, 0x65746E49   ; "Inte"
> +    jne     IsNotTdx
> +    cmp     edx, 0x5844546C   ; "lTDX"
> +    jne     IsNotTdx
> +    cmp     ecx, 0x20202020   ; "    "
> +    jne     IsNotTdx
> +
> +    mov     eax, 0
> +    jmp     ExitIsTdx
> +
> +IsNotTdx:
> +    mov     eax, 1
> +
> +ExitIsTdx:
> +
> +  OneTimeCallRet IsTdx
> +
> +;
> +; Initialize Tdx work area (CC_WORK_AREA) if it is Tdx guest.
> +; BSP and APs all go here. Only BSP can initialize the WORK_AREA
> +;
> +; typedef struct {
> +;   UINT8   Type;     // 0 legacy, 1 SEV, 2 TDX
> +;   UINT8   SubType;  // Depends on Type
> +;   UINT8   Rsvd[2];  // Reserved
> +;   union VM_GUEST {
> +;     TDX_WORK_AREA Tdx;
> +;     SEV_WORK_AREA Sev;
> +;   } Guest;
> +; } CC_WORK_AREA_HEAD;
> +;
> +; typedef struct {
> +;   UINT8    IsPageLevel5;
> +;   UINT8    IsPageTableReady;
> +;   UINT8    Rsvd[2];
> +;   UINT32   Gpaw;
> +; } TDX_WORK_AREA
> +;
> +; Param[in] EBP[6:0]    CPU Supported GPAW (48 or 52)
> +; Param[in] ESI[31:0]   vCPU ID (BSP is 0, others are AP)
> +;
> +; Modified:  EBP
> +;
> +InitTdxWorkarea:
> +
> +    ;
> +    ; First check if it is Tdx
> +    ;
> +    OneTimeCall IsTdx
> +
> +    test    eax, eax
> +    jnz     ExitInitTdxWorkarea
> +
> +    ;
> +    ; In Td guest, BSP/AP shares the same entry point
> +    ; BSP builds up the page table, while APs shouldn't do the same task.
> +    ; Instead, APs just leverage the page table which is built by BSP.
> +    ; APs will wait until the page table is ready.
> +    ;
> +    cmp     esi, 0
> +    je      TdxBspEntry
> +
> +TdxApWait:
> +    cmp     byte[TDX_WORK_AREA_PGTBL_READY], 0
> +    je      TdxApWait
> +    jmp     ExitInitTdxWorkarea
> +
> +TdxBspEntry:
> +    ;
> +    ; Set Type and SubType of CC_WORK_AREA so that the
> +    ; following code can use these information.
> +    ;
> +    mov     byte[CC_WORK_AREA], VM_GUEST_TDX
> +    mov     byte[CC_WORK_AREA_SUBTYPE], 0
> +
> +    ;
> +    ; EBP[6:0] CPU supported GPA width
> +    ;
> +    and     ebp, 0x3f
> +    cmp     ebp, 52
> +    jl      NotPageLevel5
> +    mov     byte[TDX_WORK_AREA_PAGELEVEL5], 1
> +
> +NotPageLevel5:
> +    mov     DWORD[TDX_WORK_AREA_GPAW], ebp
> +
> +ExitInitTdxWorkarea:
> +    OneTimeCallRet InitTdxWorkarea
> +
> +;
> +; Load the GDT and set the CR0, then jump to Flat 32 protected mode.
> +;
> +; Modified:  EAX, EBX, CR0, CR4, DS, ES, FS, GS, SS
> +;
> +ReloadFlat32:
> +
> +    cli
> +    mov     ebx, ADDR_OF(gdtr)
> +    lgdt    [ebx]
> +
> +    mov     eax, SEC_DEFAULT_CR0
> +    mov     cr0, eax
> +
> +    jmp     LINEAR_CODE_SEL:dword ADDR_OF(jumpToFlat32BitAndLandHere)
> +
> +jumpToFlat32BitAndLandHere:
> +
> +    mov     eax, SEC_DEFAULT_CR4
> +    mov     cr4, eax
> +
> +    debugShowPostCode POSTCODE_32BIT_MODE
> +
> +    mov     ax, LINEAR_SEL
> +    mov     ds, ax
> +    mov     es, ax
> +    mov     fs, ax
> +    mov     gs, ax
> +    mov     ss, ax
> +
> +    OneTimeCallRet ReloadFlat32
> +
> +;
> +; Tdx initialization after entering into ResetVector
> +;
> +; Modified:  EAX, EBX, ECX, EDX, EBP, EDI, ESP
> +;
> +InitTdx:
> +    ;
> +    ; Save EBX in EBP because EBX will be changed in ReloadFlat32
> +    ;
> +    mov     ebp, ebx
> +
> +    ;
> +    ; First load the GDT and jump to Flat32 mode
> +    ;
> +    OneTimeCall ReloadFlat32
> +
> +    ;
> +    ; Initialization of Tdx work area
> +    ;
> +    OneTimeCall  InitTdxWorkarea
> +
> +    OneTimeCallRet InitTdx
> +
> +;
> +; Called after SetCr3PageTables64 in Tdx guest to set CR0/CR4.
> +; If GPAW is 52, then CR3 is adjusted as well.
> +;
> +; Modified: EAX, EBX, CR0, CR3, CR4
> +;
> +PostSetCr3PageTables64Tdx:
> +    ;
> +    ; TDX_WORK_AREA was set in InitTdx if it is Tdx guest
> +    ;
> +    cmp     byte[CC_WORK_AREA], VM_GUEST_TDX
> +    jne     ExitPostSetCr3PageTables64Tdx
> +
> +    mov     eax, cr4
> +    bts     eax, 5                      ; enable PAE
> +
> +    ;
> +    ; byte[TDX_WORK_AREA_PAGELEVEL5] holds the indicator whether 52bit is
> +    ; supported. if it is the case, need to set LA57 and use 5-level paging
> +    ;
> +    cmp     byte[TDX_WORK_AREA_PAGELEVEL5], 0
> +    jz      TdxSetCr4
> +    bts     eax, 12
> +
> +TdxSetCr4:
> +    mov     cr4, eax
> +    mov     ebx, cr3
> +
> +    ;
> +    ; if la57 is not set, we are ok
> +    ; if using 5-level paging, adjust top-level page directory
> +    ;
> +    bt      eax, 12
> +    jnc     TdxSetCr3
> +    mov     ebx, TDX_PT_ADDR (0)
> +
> +TdxSetCr3:
> +    mov     cr3, ebx
> +
> +    xor     ebx, ebx
> +
> +ExitPostSetCr3PageTables64Tdx:
> +    OneTimeCallRet PostSetCr3PageTables64Tdx
> +
> +;
> +; Check if TDX is enabled
> +;
> +; Modified:  EAX
> +;
> +; If TDX is enabled then EAX will be 1
> +; If TDX is disabled then EAX will be 0.
> +;
> +IsTdxEnabled:
> +    xor     eax, eax
> +    cmp     byte[CC_WORK_AREA], VM_GUEST_TDX
> +    jne     NotTdx
> +    mov     eax, 1
> +
> +NotTdx:
> +    OneTimeCallRet IsTdxEnabled
> +
> diff --git a/OvmfPkg/ResetVector/Ia32/PageTables64.asm b/OvmfPkg/ResetVector/Ia32/PageTables64.asm
> index eacdb69ddb9f..1f4c0c50ac2e 100644
> --- a/OvmfPkg/ResetVector/Ia32/PageTables64.asm
> +++ b/OvmfPkg/ResetVector/Ia32/PageTables64.asm
> @@ -37,11 +37,81 @@ BITS    32
>                         PAGE_READ_WRITE + \
>                         PAGE_PRESENT)
>
> +TdxBuildExtraPageTables:
> +    ;
> +    ; Extra page tables built by Tdx guests
> +    ;
> +    xor     eax, eax
> +    mov     ecx, 0x400
> +tdClearTdxPageTablesMemoryLoop:
> +    mov     dword [ecx * 4 + TDX_PT_ADDR(0) - 4], eax
> +    loop    tdClearTdxPageTablesMemoryLoop
> +
> +    ;
> +    ; Top level Page Directory Pointers (1 * 256TB entry)
> +    ;
> +    mov     dword[TDX_PT_ADDR (0)], PT_ADDR(0) + PAGE_PDP_ATTR
> +
> +    ;
> +    ; Set TDX_WORK_AREA_PGTBL_READY to notify APs to go
> +    ;
> +    mov     byte[TDX_WORK_AREA_PGTBL_READY], 1
> +
> +    OneTimeCallRet TdxBuildExtraPageTables
> +
> +SevBuildGhcbPageTables:
> +    ;
> +    ; The initial GHCB will live at GHCB_BASE and needs to be un-encrypted.
> +    ; This requires the 2MB page for this range be broken down into 512 4KB
> +    ; pages.  All will be marked encrypted, except for the GHCB.
> +    ;
> +    mov     ecx, (GHCB_BASE >> 21)
> +    mov     eax, GHCB_PT_ADDR + PAGE_PDP_ATTR
> +    mov     [ecx * 8 + PT_ADDR (0x2000)], eax
> +
> +    ;
> +    ; Page Table Entries (512 * 4KB entries => 2MB)
> +    ;
> +    mov     ecx, 512
> +pageTableEntries4kLoop:
> +    mov     eax, ecx
> +    dec     eax
> +    shl     eax, 12
> +    add     eax, GHCB_BASE & 0xFFE0_0000
> +    add     eax, PAGE_4K_PDE_ATTR
> +    mov     [ecx * 8 + GHCB_PT_ADDR - 8], eax
> +    mov     [(ecx * 8 + GHCB_PT_ADDR - 8) + 4], edx
> +    loop    pageTableEntries4kLoop
> +
> +    ;
> +    ; Clear the encryption bit from the GHCB entry
> +    ;
> +    mov     ecx, (GHCB_BASE & 0x1F_FFFF) >> 12
> +    mov     [ecx * 8 + GHCB_PT_ADDR + 4], strict dword 0
> +
> +    mov     ecx, GHCB_SIZE / 4
> +    xor     eax, eax
> +clearGhcbMemoryLoop:
> +    mov     dword[ecx * 4 + GHCB_BASE - 4], eax
> +    loop    clearGhcbMemoryLoop
> +
> +    OneTimeCallRet  SevBuildGhcbPageTables
> +
>  ;
>  ; Modified:  EAX, EBX, ECX, EDX
>  ;
>  SetCr3ForPageTables64:
>
> +    cmp     byte[CC_WORK_AREA], VM_GUEST_TDX
> +    jne     CheckSev
> +
> +    cmp     byte[TDX_WORK_AREA_PGTBL_READY], 1
> +    je      SetCr3
> +
> +    xor     edx, edx
> +    jmp     SevNotActive
> +
> +CheckSev:
>      OneTimeCall   CheckSevFeatures
>      xor     edx, edx
>      test    eax, eax
> @@ -101,44 +171,19 @@ pageTableEntriesLoop:
>      mov     [(ecx * 8 + PT_ADDR (0x2000 - 8)) + 4], edx
>      loop    pageTableEntriesLoop
>
> +    OneTimeCall IsTdxEnabled
> +    test    eax, eax
> +    jz      IsSevEs
> +
> +    OneTimeCall TdxBuildExtraPageTables
> +    jmp     SetCr3
> +
> +IsSevEs:
>      OneTimeCall   IsSevEsEnabled
>      test    eax, eax
>      jz      SetCr3
>
> -    ;
> -    ; The initial GHCB will live at GHCB_BASE and needs to be un-encrypted.
> -    ; This requires the 2MB page for this range be broken down into 512 4KB
> -    ; pages.  All will be marked encrypted, except for the GHCB.
> -    ;
> -    mov     ecx, (GHCB_BASE >> 21)
> -    mov     eax, GHCB_PT_ADDR + PAGE_PDP_ATTR
> -    mov     [ecx * 8 + PT_ADDR (0x2000)], eax
> -
> -    ;
> -    ; Page Table Entries (512 * 4KB entries => 2MB)
> -    ;
> -    mov     ecx, 512
> -pageTableEntries4kLoop:
> -    mov     eax, ecx
> -    dec     eax
> -    shl     eax, 12
> -    add     eax, GHCB_BASE & 0xFFE0_0000
> -    add     eax, PAGE_4K_PDE_ATTR
> -    mov     [ecx * 8 + GHCB_PT_ADDR - 8], eax
> -    mov     [(ecx * 8 + GHCB_PT_ADDR - 8) + 4], edx
> -    loop    pageTableEntries4kLoop
> -
> -    ;
> -    ; Clear the encryption bit from the GHCB entry
> -    ;
> -    mov     ecx, (GHCB_BASE & 0x1F_FFFF) >> 12
> -    mov     [ecx * 8 + GHCB_PT_ADDR + 4], strict dword 0
> -
> -    mov     ecx, GHCB_SIZE / 4
> -    xor     eax, eax
> -clearGhcbMemoryLoop:
> -    mov     dword[ecx * 4 + GHCB_BASE - 4], eax
> -    loop    clearGhcbMemoryLoop
> +    OneTimeCall   SevBuildGhcbPageTables
>
>  SetCr3:
>      ;
> diff --git a/OvmfPkg/ResetVector/Main.asm b/OvmfPkg/ResetVector/Main.asm
> new file mode 100644
> index 000000000000..fb1731728af5
> --- /dev/null
> +++ b/OvmfPkg/ResetVector/Main.asm
> @@ -0,0 +1,121 @@
> +;------------------------------------------------------------------------------
> +; @file
> +; Main routine of the pre-SEC code up through the jump into SEC
> +;
> +; Copyright (c) 2008 - 2009, Intel Corporation. All rights reserved.<BR>
> +; SPDX-License-Identifier: BSD-2-Clause-Patent
> +;
> +;------------------------------------------------------------------------------
> +
> +
> +BITS    16
> +
> +;
> +; Modified:  EBX, ECX, EDX, EBP
> +;
> +; @param[in,out]  RAX/EAX  Initial value of the EAX register
> +;                          (BIST: Built-in Self Test)
> +; @param[in,out]  DI       'BP': boot-strap processor, or
> +;                          'AP': application processor
> +; @param[out]     RBP/EBP  Address of Boot Firmware Volume (BFV)
> +; @param[out]     DS       Selector allowing flat access to all addresses
> +; @param[out]     ES       Selector allowing flat access to all addresses
> +; @param[out]     FS       Selector allowing flat access to all addresses
> +; @param[out]     GS       Selector allowing flat access to all addresses
> +; @param[out]     SS       Selector allowing flat access to all addresses
> +;
> +; @return         None  This routine jumps to SEC and does not return
> +;
> +Main16:
> +    OneTimeCall EarlyInit16
> +
> +    ;
> +    ; Transition the processor from 16-bit real mode to 32-bit flat mode
> +    ;
> +    OneTimeCall TransitionFromReal16To32BitFlat
> +
> +BITS    32
> +%ifdef ARCH_X64
> +    ;
> +    ; Clear the first 2 bytes of CC_WORK_AREA.
> +    ;
> +    mov word[CC_WORK_AREA], 0
> +
> +    jmp SearchBfv
> +
> +;
> +; Entry point of Main32
> +;
> +Main32:
> +    OneTimeCall InitTdx
> +
> +SearchBfv:
> +
> +%endif
> +    ;
> +    ; Search for the Boot Firmware Volume (BFV)
> +    ;
> +    OneTimeCall Flat32SearchForBfvBase
> +
> +    ;
> +    ; EBP - Start of BFV
> +    ;
> +
> +    ;
> +    ; Search for the SEC entry point
> +    ;
> +    OneTimeCall Flat32SearchForSecEntryPoint
> +
> +    ;
> +    ; ESI - SEC Core entry point
> +    ; EBP - Start of BFV
> +    ;
> +
> +%ifdef ARCH_IA32
> +
> +    ;
> +    ; Restore initial EAX value into the EAX register
> +    ;
> +    mov     eax, esp
> +
> +    ;
> +    ; Jump to the 32-bit SEC entry point
> +    ;
> +    jmp     esi
> +
> +%else
> +
> +    ;
> +    ; Transition the processor from 32-bit flat mode to 64-bit flat mode
> +    ;
> +    OneTimeCall Transition32FlatTo64Flat
> +
> +BITS    64
> +
> +    ;
> +    ; Some values were calculated in 32-bit mode.  Make sure the upper
> +    ; 32-bits of 64-bit registers are zero for these values.
> +    ;
> +    mov     rax, 0x00000000ffffffff
> +    and     rsi, rax
> +    and     rbp, rax
> +    and     rsp, rax
> +
> +    ;
> +    ; RSI - SEC Core entry point
> +    ; RBP - Start of BFV
> +    ;
> +
> +    ;
> +    ; Restore initial EAX value into the RAX register
> +    ;
> +    mov     rax, rsp
> +
> +    ;
> +    ; Jump to the 64-bit SEC entry point
> +    ;
> +    jmp     rsi
> +
> +%endif
> +
> +
> diff --git a/OvmfPkg/ResetVector/ResetVector.inf b/OvmfPkg/ResetVector/ResetVector.inf
> index d028c92d8cfa..0cf3a8036b97 100644
> --- a/OvmfPkg/ResetVector/ResetVector.inf
> +++ b/OvmfPkg/ResetVector/ResetVector.inf
> @@ -1,7 +1,7 @@
>  ## @file
>  #  Reset Vector
>  #
> -#  Copyright (c) 2006 - 2014, Intel Corporation. All rights reserved.<BR>
> +#  Copyright (c) 2006 - 2021, Intel Corporation. All rights reserved.<BR>
>  #
>  #  SPDX-License-Identifier: BSD-2-Clause-Patent
>  #
> @@ -35,6 +35,7 @@
>
>  [Pcd]
>    gUefiCpuPkgTokenSpaceGuid.PcdSevEsWorkAreaBase
> +  gUefiCpuPkgTokenSpaceGuid.PcdSevEsWorkAreaSize
>    gUefiOvmfPkgTokenSpaceGuid.PcdOvmfSecGhcbBase
>    gUefiOvmfPkgTokenSpaceGuid.PcdOvmfSecGhcbSize
>    gUefiOvmfPkgTokenSpaceGuid.PcdOvmfSecGhcbPageTableBase
> @@ -43,6 +44,15 @@
>    gUefiOvmfPkgTokenSpaceGuid.PcdOvmfSecPageTablesSize
>    gUefiOvmfPkgTokenSpaceGuid.PcdOvmfSecPeiTempRamBase
>    gUefiOvmfPkgTokenSpaceGuid.PcdOvmfSecPeiTempRamSize
> +  gUefiOvmfPkgTokenSpaceGuid.PcdOvmfSecGhcbBackupBase
> +  gUefiOvmfPkgTokenSpaceGuid.PcdOvmfSecGhcbBackupSize
> +  gUefiOvmfPkgTokenSpaceGuid.PcdOvmfImageSizeInKb
> +  gUefiOvmfPkgTokenSpaceGuid.PcdCfvBase
> +  gUefiOvmfPkgTokenSpaceGuid.PcdCfvRawDataOffset
> +  gUefiOvmfPkgTokenSpaceGuid.PcdCfvRawDataSize
> +  gUefiOvmfPkgTokenSpaceGuid.PcdBfvBase
> +  gUefiOvmfPkgTokenSpaceGuid.PcdBfvRawDataOffset
> +  gUefiOvmfPkgTokenSpaceGuid.PcdBfvRawDataSize
>
>  [FixedPcd]
>    gUefiOvmfPkgTokenSpaceGuid.PcdSevLaunchSecretBase
> diff --git a/OvmfPkg/ResetVector/ResetVector.nasmb b/OvmfPkg/ResetVector/ResetVector.nasmb
> index acec46a32450..528a6603a20e 100644
> --- a/OvmfPkg/ResetVector/ResetVector.nasmb
> +++ b/OvmfPkg/ResetVector/ResetVector.nasmb
> @@ -67,6 +67,44 @@
>      %error "This implementation inherently depends on PcdOvmfSecGhcbBase not straddling a 2MB boundary"
>    %endif
>
> +  %define TDX_BFV_RAW_DATA_OFFSET   FixedPcdGet32 (PcdBfvRawDataOffset)
> +  %define TDX_BFV_RAW_DATA_SIZE     FixedPcdGet32 (PcdBfvRawDataSize)
> +  %define TDX_BFV_MEMORY_BASE       FixedPcdGet32 (PcdBfvBase)
> +  %define TDX_BFV_MEMORY_SIZE       FixedPcdGet32 (PcdBfvRawDataSize)
> +
> +  %define TDX_CFV_RAW_DATA_OFFSET   FixedPcdGet32 (PcdCfvRawDataOffset)
> +  %define TDX_CFV_RAW_DATA_SIZE     FixedPcdGet32 (PcdCfvRawDataSize)
> +  %define TDX_CFV_MEMORY_BASE       FixedPcdGet32 (PcdCfvBase),
> +  %define TDX_CFV_MEMORY_SIZE       FixedPcdGet32 (PcdCfvRawDataSize),
> +
> +  %define TDX_HEAP_MEMORY_BASE      FixedPcdGet32 (PcdOvmfSecPeiTempRamBase)
> +  %define TDX_HEAP_MEMORY_SIZE      FixedPcdGet32 (PcdOvmfSecPeiTempRamSize) / 2
> +
> +  %define TDX_STACK_MEMORY_BASE     (TDX_HEAP_MEMORY_BASE + TDX_HEAP_MEMORY_SIZE)
> +  %define TDX_STACK_MEMORY_SIZE     FixedPcdGet32 (PcdOvmfSecPeiTempRamSize) / 2
> +
> +  %define TDX_HOB_MEMORY_BASE       FixedPcdGet32 (PcdOvmfSecGhcbBase)
> +  %define TDX_HOB_MEMORY_SIZE       FixedPcdGet32 (PcdOvmfSecGhcbSize)
> +
> +  %define TDX_MAILBOX_MEMORY_BASE   FixedPcdGet32 (PcdOvmfSecGhcbBackupBase)
> +  %define TDX_MAILBOX_MEMORY_SIZE   FixedPcdGet32 (PcdOvmfSecGhcbBackupSize)
> +
> +  %define OVMF_PAGE_TABLE_BASE      FixedPcdGet32 (PcdOvmfSecPageTablesBase)
> +  %define OVMF_PAGE_TABLE_SIZE      FixedPcdGet32 (PcdOvmfSecPageTablesSize)
> +
> +  %define TDX_EXTRA_PAGE_TABLE_BASE FixedPcdGet32 (PcdOvmfSecGhcbPageTableBase)
> +  %define TDX_EXTRA_PAGE_TABLE_SIZE FixedPcdGet32 (PcdOvmfSecGhcbPageTableSize)
> +
> +  %define CC_WORK_AREA_MEMORY_BASE  FixedPcdGet32 (PcdSevEsWorkAreaBase)
> +  %define CC_WORK_AREA_MEMORY_SIZE  FixedPcdGet32 (PcdSevEsWorkAreaSize)
> +  %define CC_WORK_AREA              (CC_WORK_AREA_MEMORY_BASE)
> +  %define CC_WORK_AREA_SUBTYPE      (CC_WORK_AREA + 1)
> +  %define TDX_WORK_AREA_PAGELEVEL5  (CC_WORK_AREA + 4)
> +  %define TDX_WORK_AREA_PGTBL_READY (CC_WORK_AREA + 5)
> +  %define TDX_WORK_AREA_GPAW        (CC_WORK_AREA + 8)
> +
> +  %define TDX_PT_ADDR(Offset)       (TDX_EXTRA_PAGE_TABLE_BASE + (Offset))
> +
>    %define PT_ADDR(Offset) (FixedPcdGet32 (PcdOvmfSecPageTablesBase) + (Offset))
>
>    %define GHCB_PT_ADDR (FixedPcdGet32 (PcdOvmfSecGhcbPageTableBase))
> @@ -76,9 +114,12 @@
>    %define SEV_ES_WORK_AREA_RDRAND (FixedPcdGet32 (PcdSevEsWorkAreaBase) + 8)
>    %define SEV_ES_WORK_AREA_ENC_MASK (FixedPcdGet32 (PcdSevEsWorkAreaBase) + 16)
>    %define SEV_ES_VC_TOP_OF_STACK (FixedPcdGet32 (PcdOvmfSecPeiTempRamBase) + FixedPcdGet32 (PcdOvmfSecPeiTempRamSize))
> -%include "Ia32/Flat32ToFlat64.asm"
> -%include "Ia32/AmdSev.asm"
> -%include "Ia32/PageTables64.asm"
> +
> +  %include "X64/IntelTdxMetadata.asm"
> +  %include "Ia32/IntelTdx.asm"
> +  %include "Ia32/Flat32ToFlat64.asm"
> +  %include "Ia32/AmdSev.asm"
> +  %include "Ia32/PageTables64.asm"
>  %endif
>
>  %include "Ia16/Real16ToFlat32.asm"
> @@ -91,5 +132,6 @@
>    %define SEV_LAUNCH_SECRET_SIZE  FixedPcdGet32 (PcdSevLaunchSecretSize)
>    %define SEV_FW_HASH_BLOCK_BASE  FixedPcdGet32 (PcdQemuHashTableBase)
>    %define SEV_FW_HASH_BLOCK_SIZE  FixedPcdGet32 (PcdQemuHashTableSize)
> +  %define OVMF_IMAGE_SIZE_IN_KB   FixedPcdGet32 (PcdOvmfImageSizeInKb)
>  %include "Ia16/ResetVectorVtf0.asm"
>
> diff --git a/OvmfPkg/ResetVector/X64/IntelTdxMetadata.asm b/OvmfPkg/ResetVector/X64/IntelTdxMetadata.asm
> new file mode 100644
> index 000000000000..97b1e915d899
> --- /dev/null
> +++ b/OvmfPkg/ResetVector/X64/IntelTdxMetadata.asm
> @@ -0,0 +1,110 @@
> +;------------------------------------------------------------------------------
> +; @file
> +; Tdx Virtual Firmware metadata
> +;
> +; Copyright (c) 2021, Intel Corporation. All rights reserved.<BR>
> +; SPDX-License-Identifier: BSD-2-Clause-Patent
> +;
> +;------------------------------------------------------------------------------
> +
> +BITS    64
> +
> +%define TDX_METADATA_SECTION_TYPE_BFV       0
> +%define TDX_METADATA_SECTION_TYPE_CFV       1
> +%define TDX_METADATA_SECTION_TYPE_TD_HOB    2
> +%define TDX_METADATA_SECTION_TYPE_TEMP_MEM  3
> +%define TDX_METADATA_VERSION                1
> +%define TDX_METADATA_ATTRIBUTES_EXTENDMR    0x00000001
> +
> +ALIGN   16
> +TIMES (15 - ((TdxGuidedStructureEnd - TdxGuidedStructureStart + 15) % 16)) DB 0
> +
> +TdxGuidedStructureStart:
> +
> +;
> +; TDVF meta data
> +;
> +TdxMetadataGuid:
> +  DB  0xf3, 0xf9, 0xea, 0xe9, 0x8e, 0x16, 0xd5, 0x44
> +  DB  0xa8, 0xeb, 0x7f, 0x4d, 0x87, 0x38, 0xf6, 0xae
> +
> +_Descriptor:
> +  DB 'T','D','V','F'                                  ; Signature
> +  DD TdxGuidedStructureEnd - _Descriptor              ; Length
> +  DD TDX_METADATA_VERSION                             ; Version
> +  DD (TdxGuidedStructureEnd - _Descriptor - 16)/32    ; Number of sections
> +
> +_Bfv:
> +  DD TDX_BFV_RAW_DATA_OFFSET
> +  DD TDX_BFV_RAW_DATA_SIZE
> +  DQ TDX_BFV_MEMORY_BASE
> +  DQ TDX_BFV_MEMORY_SIZE
> +  DD TDX_METADATA_SECTION_TYPE_BFV
> +  DD TDX_METADATA_ATTRIBUTES_EXTENDMR
> +
> +_Cfv:
> +  DD TDX_CFV_RAW_DATA_OFFSET
> +  DD TDX_CFV_RAW_DATA_SIZE
> +  DQ TDX_CFV_MEMORY_BASE
> +  DQ TDX_CFV_MEMORY_SIZE
> +  DD TDX_METADATA_SECTION_TYPE_CFV
> +  DD 0
> +
> +_Stack:
> +  DD 0
> +  DD 0
> +  DQ TDX_STACK_MEMORY_BASE
> +  DQ TDX_STACK_MEMORY_SIZE
> +  DD TDX_METADATA_SECTION_TYPE_TEMP_MEM
> +  DD 0
> +
> +_Heap:
> +  DD 0
> +  DD 0
> +  DQ TDX_HEAP_MEMORY_BASE
> +  DQ TDX_HEAP_MEMORY_SIZE
> +  DD TDX_METADATA_SECTION_TYPE_TEMP_MEM
> +  DD 0
> +
> +_MailBox:
> +  DD 0
> +  DD 0
> +  DQ TDX_MAILBOX_MEMORY_BASE
> +  DQ TDX_MAILBOX_MEMORY_SIZE
> +  DD TDX_METADATA_SECTION_TYPE_TEMP_MEM
> +  DD 0
> +
> +_CcWorkarea:
> +  DD 0
> +  DD 0
> +  DQ CC_WORK_AREA_MEMORY_BASE
> +  DQ CC_WORK_AREA_MEMORY_SIZE
> +  DD TDX_METADATA_SECTION_TYPE_TEMP_MEM
> +  DD 0
> +
> +_TdHob:
> +  DD 0
> +  DD 0
> +  DQ TDX_HOB_MEMORY_BASE
> +  DQ TDX_HOB_MEMORY_SIZE
> +  DD TDX_METADATA_SECTION_TYPE_TD_HOB
> +  DD 0
> +
> +_TdxPageTable:
> +  DD 0
> +  DD 0
> +  DQ TDX_EXTRA_PAGE_TABLE_BASE
> +  DQ TDX_EXTRA_PAGE_TABLE_SIZE
> +  DD TDX_METADATA_SECTION_TYPE_TEMP_MEM
> +  DD 0
> +
> +_OvmfPageTable:
> +  DD 0
> +  DD 0
> +  DQ OVMF_PAGE_TABLE_BASE
> +  DQ OVMF_PAGE_TABLE_SIZE
> +  DD TDX_METADATA_SECTION_TYPE_TEMP_MEM
> +  DD 0
> +
> +TdxGuidedStructureEnd:
> +ALIGN   16
> --
> 2.29.2.windows.2
>