[PATCH V9 0/4] Add Intel TDX support in OvmfPkg/ResetVector

[PATCH V9 0/4] Add Intel TDX support in OvmfPkg/ResetVector

[Min Xu]

REF: https://bugzilla.tianocore.org/show_bug.cgi?id=3429

Intel's Trust Domain Extensions (Intel TDX) refers to an Intel technology
that extends Virtual Machines Extensions (VMX) and Multi-Key Total Memory
Encryption (MKTME) with a new kind of virutal machines guest called a
Trust Domain (TD). A TD is desinged to run in a CPU mode that protects the
confidentiality of TD memory contents and the TD's CPU state from other
software, including the hosting Virtual-Machine Monitor (VMM), unless
explicitly shared by the TD itself.

The patch-sets to support Intel TDX in OvmfPkg is split into several
waves. This is wave-1 which adds Intel TDX support in OvmfPkg/ResetVector.
Note: TDX only works in X64.

Patch #1: Ovmf uses its own Main.asm to reduce the complexity of Main.asm
in UefiCpuPkg

Patch #2: WORK_AREA_GUEST_TYPE is cleared in Main.asm instead of in
WORK_AREA_GUEST_TYPE.

Patch #3: Introduce IntelTdxMetadata.asm which describes the information
about the image for VMM use.

Patch #4: Enable TDX in OvmfPkg/ResetVector for ARCH_X64.

[TDX]: https://software.intel.com/content/dam/develop/external/us/en/
documents/tdx-whitepaper-final9-17.pdf

[TDVF]: https://software.intel.com/content/dam/develop/external/us/en/
documents/tdx-virtual-firmware-design-guide-rev-1.pdf

Code is at https://github.com/mxu9/edk2/tree/tdvf_wave1.v9

v9 changes:
 - Introduce IntelTdxMetadata.asm in a separate commit.
 - Use absolute offset for the start of TdxMetadata so that VMM can
   easily reach to the start of the metadata.

v8 changes:
 - Create a separate commit for Main.asm.
 - Create a separate commit for the clearance of WORK_AREA_GUEST_TYPE.
 - Fix some inaccurate comments.

v7 changes:
 - Refine the offset of TdxMetadata and remove the definition of
   PcdOvmfImageSizeInKB
 - Use MOV CR* instead of smsw in ResetVector
 - Remove the new field (SubType) in
   CONFIDENTIAL_COMPUTING_WORK_AREA_HEADER.

v6 changes:
 - Remove the 5-level paging support. 5-level paging enabling is *NOT*
   super critical for TDX enabling at this moment. It will be enabled
   later in a separate patch.
 - Add a new field (SubType) in CONFIDENTIAL_COMPUTING_WORK_AREA_HEADER
   to record the VM Guest SubType.
 - In Main16 entry point, after TransitionFromReal16To32BitFlat,
   WORK_AREA_GUEST_TYPE is cleared to 0. WORK_AREA_GUEST_TYPE was
   previously cleared in SetCr3ForPageTables64 (see commit ab77b60).
   This doesn't work after TDX is introduced in Ovmf. It is because all
   TDX CPUs (BSP and APs) start to run from 0xfffffff0. In previous code
   WORK_AREA_GUEST_TYPE will be cleared multi-times in TDX guest. So for
   SEV and Legacy guest it is moved to Main16 entry point (after
   TransitionFromReal16To32BitFlat). For TDX guest WORK_AREA_GUEST_TYPE
   is cleared and set in InitTdxWorkarea.
 - Make the return result of IsTdx be consistent with IsTdxEnabled.
 - Fix some typo in the code comments.

v5 changes:
 - Remove the changes of OVMF_WORK_AREA because Commit ab77b60 covers
   those changes.
 - Refine the TDX related changes in PageTables64.asm and
   Flat32ToFlat64.asm.
 - Add CheckTdxFeaturesBeforeBuildPagetables to check Non-Tdx, Tdx-BSP or
   Tdx-APs. This routine is called before building page tables.

v4 changes:
 - Refine the PageTables64.asm and Flat32ToFlat64.asm to enable TDX.
 - Refine SEV_ES_WORK_AREA so that SEV/TDX/Legach guest all can use this
   memory region. https://edk2.groups.io/g/devel/message/78345 is the
   discussion.
 - AmdSev.asm is removed because Brijesh Singh has done it in
   https://edk2.groups.io/g/devel/message/78241.

v3 changes:
 - Refine PageTables64.asm and Flat32ToFlat64.asm based on the review
   comments in [ReviewComment-1] and [ReviewComment-2].
 - SEV codes are in AmdSev.asm
 - TDX codes are in IntelTdx.asm
 - Main.asm is created in OvmfPkg/ResetVector. The one in
   UefiCpuPkg/ResetVector/Vtf0 is not used.
 - Init32.asm/ReloadFlat32.asm in UefiCpuPkg/ResetVector/Vtf0/Ia32 are
   deleted. They're moved to OvmfPkg/ResetVector/Ia32.
 - InitTdx.asm is renamed to InteTdx.asm

v2 changes:
 - Move InitTdx.asm and ReloadFlat32.asm from UefiCpuPkg/ResetVector/Vtf0
   to OvmfPkg/ResetVector. Init32.asm is created which is a null stub of
   32-bit initialization. In Main32 just simply call Init32. It makes
   the Main.asm in UefiCpuPkg/ResetVector clean and clear.
 - Init32.asm/InitTdx.asm/ReloadFlat32.asm are created under
   OvmfPkg/ResetVector/Ia32.
 - Update some descriptions of the patch-sets.
 - Update the REF link in cover letter.
 - Add Ard Biesheuvel in Cc list.

v1: https://edk2.groups.io/g/devel/message/77675

Cc: Ard Biesheuvel <ardb+tianocore@kernel.org>
Cc: Gerd Hoffmann <kraxel@redhat.com>
Cc: Jordan Justen <jordan.l.justen@intel.com>
Cc: Brijesh Singh <brijesh.singh@amd.com>
Cc: Erdem Aktas <erdemaktas@google.com>
Cc: James Bottomley <jejb@linux.ibm.com>
Cc: Jiewen Yao <jiewen.yao@intel.com>
Cc: Tom Lendacky <thomas.lendacky@amd.com>
Signed-off-by: Min Xu <min.m.xu@intel.com>

Min Xu (4):
  OvmfPkg: Copy Main.asm from UefiCpuPkg to OvmfPkg's ResetVector
  OvmfPkg: Clear WORK_AREA_GUEST_TYPE in Main.asm
  OvmfPkg: Add IntelTdxMetadata.asm
  OvmfPkg: Enable TDX in ResetVector

 OvmfPkg/OvmfPkg.dec                          |   9 +
 OvmfPkg/OvmfPkgDefines.fdf.inc               |   9 +
 OvmfPkg/ResetVector/Ia16/ResetVectorVtf0.asm |  39 +++
 OvmfPkg/ResetVector/Ia32/Flat32ToFlat64.asm  |  11 +
 OvmfPkg/ResetVector/Ia32/IntelTdx.asm        | 235 +++++++++++++++++++
 OvmfPkg/ResetVector/Ia32/PageTables64.asm    |  22 +-
 OvmfPkg/ResetVector/Main.asm                 | 121 ++++++++++
 OvmfPkg/ResetVector/ResetVector.inf          |   9 +
 OvmfPkg/ResetVector/ResetVector.nasmb        |  28 +++
 OvmfPkg/ResetVector/X64/IntelTdxMetadata.asm | 115 +++++++++
 10 files changed, 594 insertions(+), 4 deletions(-)
 create mode 100644 OvmfPkg/ResetVector/Ia32/IntelTdx.asm
 create mode 100644 OvmfPkg/ResetVector/Main.asm
 create mode 100644 OvmfPkg/ResetVector/X64/IntelTdxMetadata.asm

-- 
2.29.2.windows.2

[PATCH V9 1/4] OvmfPkg: Copy Main.asm from UefiCpuPkg to OvmfPkg's ResetVector

[Min Xu]

RFC： https://bugzilla.tianocore.org/show_bug.cgi?id=3429

Previously OvmfPkg/ResetVector uses the Main.asm in
UefiCpuPkg/ReseteVector/Vtf0. In this Main.asm there is only Main16
entry point.

This patch-set is to introduce Intel TDX into Ovmf. Main32 entry point
is needed in Main.asm by Intel TDX. To reduce the complexity of Main.asm
in UefiCpuPkg, OvmfPkg create its own Main.asm to meet the requirement
of Intel TDX.

UefiCpuPkg/ResetVector/Vtf0/main.asm -> OvmfPkg/ResetVector/Main.asm

Cc: Ard Biesheuvel <ardb+tianocore@kernel.org>
Cc: Jordan Justen <jordan.l.justen@intel.com>
Cc: Gerd Hoffmann <kraxel@redhat.com>
Cc: Brijesh Singh <brijesh.singh@amd.com>
Cc: Erdem Aktas <erdemaktas@google.com>
Cc: James Bottomley <jejb@linux.ibm.com>
Cc: Jiewen Yao <jiewen.yao@intel.com>
Cc: Tom Lendacky <thomas.lendacky@amd.com>
Signed-off-by: Min Xu <min.m.xu@intel.com>
---
 OvmfPkg/ResetVector/Main.asm | 103 +++++++++++++++++++++++++++++++++++
 1 file changed, 103 insertions(+)
 create mode 100644 OvmfPkg/ResetVector/Main.asm

diff --git a/OvmfPkg/ResetVector/Main.asm b/OvmfPkg/ResetVector/Main.asm
new file mode 100644
index 000000000000..ae90a148fce7
--- /dev/null
+++ b/OvmfPkg/ResetVector/Main.asm
@@ -0,0 +1,103 @@
+;------------------------------------------------------------------------------
+; @file
+; Main routine of the pre-SEC code up through the jump into SEC
+;
+; Copyright (c) 2008 - 2009, Intel Corporation. All rights reserved.<BR>
+; SPDX-License-Identifier: BSD-2-Clause-Patent
+;
+;------------------------------------------------------------------------------
+
+
+BITS    16
+
+;
+; Modified:  EBX, ECX, EDX, EBP
+;
+; @param[in,out]  RAX/EAX  Initial value of the EAX register
+;                          (BIST: Built-in Self Test)
+; @param[in,out]  DI       'BP': boot-strap processor, or
+;                          'AP': application processor
+; @param[out]     RBP/EBP  Address of Boot Firmware Volume (BFV)
+; @param[out]     DS       Selector allowing flat access to all addresses
+; @param[out]     ES       Selector allowing flat access to all addresses
+; @param[out]     FS       Selector allowing flat access to all addresses
+; @param[out]     GS       Selector allowing flat access to all addresses
+; @param[out]     SS       Selector allowing flat access to all addresses
+;
+; @return         None  This routine jumps to SEC and does not return
+;
+Main16:
+    OneTimeCall EarlyInit16
+
+    ;
+    ; Transition the processor from 16-bit real mode to 32-bit flat mode
+    ;
+    OneTimeCall TransitionFromReal16To32BitFlat
+
+BITS    32
+
+    ;
+    ; Search for the Boot Firmware Volume (BFV)
+    ;
+    OneTimeCall Flat32SearchForBfvBase
+
+    ;
+    ; EBP - Start of BFV
+    ;
+
+    ;
+    ; Search for the SEC entry point
+    ;
+    OneTimeCall Flat32SearchForSecEntryPoint
+
+    ;
+    ; ESI - SEC Core entry point
+    ; EBP - Start of BFV
+    ;
+
+%ifdef ARCH_IA32
+
+    ;
+    ; Restore initial EAX value into the EAX register
+    ;
+    mov     eax, esp
+
+    ;
+    ; Jump to the 32-bit SEC entry point
+    ;
+    jmp     esi
+
+%else
+
+    ;
+    ; Transition the processor from 32-bit flat mode to 64-bit flat mode
+    ;
+    OneTimeCall Transition32FlatTo64Flat
+
+BITS    64
+
+    ;
+    ; Some values were calculated in 32-bit mode.  Make sure the upper
+    ; 32-bits of 64-bit registers are zero for these values.
+    ;
+    mov     rax, 0x00000000ffffffff
+    and     rsi, rax
+    and     rbp, rax
+    and     rsp, rax
+
+    ;
+    ; RSI - SEC Core entry point
+    ; RBP - Start of BFV
+    ;
+
+    ;
+    ; Restore initial EAX value into the RAX register
+    ;
+    mov     rax, rsp
+
+    ;
+    ; Jump to the 64-bit SEC entry point
+    ;
+    jmp     rsi
+
+%endif
-- 
2.29.2.windows.2

[PATCH V9 2/4] OvmfPkg: Clear WORK_AREA_GUEST_TYPE in Main.asm

[Min Xu]

RFC： https://bugzilla.tianocore.org/show_bug.cgi?id=3429

Previously WORK_AREA_GUEST_TYPE was cleared in SetCr3ForPageTables64.
This is workable for Legacy guest and SEV guest. But it doesn't work
after Intel TDX is introduced. It is because all TDX CPUs (BSP and APs)
start to run from 0xfffffff0, thus WORK_AREA_GUEST_TYPE will be cleared
multi-times if it is TDX guest. So the clearance of WORK_AREA_GUEST_TYPE
is moved to Main16 entry point in Main.asm.
Note: WORK_AREA_GUEST_TYPE is only defined for ARCH_X64.

For Intel TDX, its corresponding entry point is Main32 (which will be
introduced in next commit in this patch-set). WORK_AREA_GUEST_TYPE will
be cleared there.

Cc: Ard Biesheuvel <ardb+tianocore@kernel.org>
Cc: Jordan Justen <jordan.l.justen@intel.com>
Cc: Gerd Hoffmann <kraxel@redhat.com>
Cc: Brijesh Singh <brijesh.singh@amd.com>
Cc: Erdem Aktas <erdemaktas@google.com>
Cc: James Bottomley <jejb@linux.ibm.com>
Cc: Jiewen Yao <jiewen.yao@intel.com>
Cc: Tom Lendacky <thomas.lendacky@amd.com>
Signed-off-by: Min Xu <min.m.xu@intel.com>
---
 OvmfPkg/ResetVector/Ia32/PageTables64.asm | 4 ----
 OvmfPkg/ResetVector/Main.asm              | 8 ++++++++
 2 files changed, 8 insertions(+), 4 deletions(-)

diff --git a/OvmfPkg/ResetVector/Ia32/PageTables64.asm b/OvmfPkg/ResetVector/Ia32/PageTables64.asm
index 07b6ca070909..02528221e560 100644
--- a/OvmfPkg/ResetVector/Ia32/PageTables64.asm
+++ b/OvmfPkg/ResetVector/Ia32/PageTables64.asm
@@ -42,10 +42,6 @@ BITS    32
 ;
 SetCr3ForPageTables64:
 
-    ; Clear the WorkArea header. The SEV probe routines will populate the
-    ; work area when detected.
-    mov     byte[WORK_AREA_GUEST_TYPE], 0
-
     ; Check whether the SEV is active and populate the SevEsWorkArea
     OneTimeCall   CheckSevFeatures
 
diff --git a/OvmfPkg/ResetVector/Main.asm b/OvmfPkg/ResetVector/Main.asm
index ae90a148fce7..a501fbe880f2 100644
--- a/OvmfPkg/ResetVector/Main.asm
+++ b/OvmfPkg/ResetVector/Main.asm
@@ -36,6 +36,14 @@ Main16:
 
 BITS    32
 
+%ifdef ARCH_X64
+
+    ; Clear the WorkArea header. The SEV probe routines will populate the
+    ; work area when detected.
+    mov     byte[WORK_AREA_GUEST_TYPE], 0
+
+%endif
+
     ;
     ; Search for the Boot Firmware Volume (BFV)
     ;
-- 
2.29.2.windows.2

[PATCH V9 3/4] OvmfPkg: Add IntelTdxMetadata.asm

[Min Xu]

RFC： https://bugzilla.tianocore.org/show_bug.cgi?id=3429

In TDX when host VMM creates a new guest TD, some initial set of
TD-private pages are added using the TDH.MEM.PAGE.ADD function. These
pages typically contain Virtual BIOS code and data along with some clear
pages for stacks and heap. In the meanwhile, some configuration data
need be measured by host VMM. Tdx Metadata is designed for this purpose
to indicate host VMM how to do the above tasks.

More detailed information of Metadata is in [TDVF] Section 11.

Tdx Metadata describes the information about the image for VMM use.
For example, the base address and length of the TdHob, Bfv, Cfv, etc.
The offset of the Metadata is stored in a GUID-ed structure which is
appended in the GUID-ed chain from a fixed GPA (0xffffffd0).

In this commit there are 2 new definitions of BFV & CFV.
Tdx Virtual Firmware (TDVF) includes one Firmware Volume (FV) known
as the Boot Firmware Volume (BFV). The FV format is defined in the
UEFI Platform Initialization (PI) spec. BFV includes all TDVF
components required during boot.

TDVF also include a configuration firmware volume (CFV) that is
separated from the BFV. The reason is because the CFV is measured in
RTMR, while the BFV is measured in MRTD.

In practice BFV is the code part of Ovmf image (OVMF_CODE.fd). CFV is
the vars part of Ovmf image (OVMF_VARS.fd).

Since AMD SEV has already defined some SEV specific memory region in
MEMFD. TDX re-uses some of the memory regions defined by SEV.
 - MailBox : PcdOvmfSecGhcbBackupBase|PcdOvmfSecGhcbBackupSize
 - TdHob   : PcdOvmfSecGhcbBase|PcdOvmfSecGhcbSize

[TDVF] https://software.intel.com/content/dam/develop/external/us/en/
documents/tdx-virtual-firmware-design-guide-rev-1.pdf

Cc: Ard Biesheuvel <ardb+tianocore@kernel.org>
Cc: Jordan Justen <jordan.l.justen@intel.com>
Cc: Gerd Hoffmann <kraxel@redhat.com>
Cc: Brijesh Singh <brijesh.singh@amd.com>
Cc: Erdem Aktas <erdemaktas@google.com>
Cc: James Bottomley <jejb@linux.ibm.com>
Cc: Jiewen Yao <jiewen.yao@intel.com>
Cc: Tom Lendacky <thomas.lendacky@amd.com>
Signed-off-by: Min Xu <min.m.xu@intel.com>
---
 OvmfPkg/OvmfPkg.dec                          |   9 ++
 OvmfPkg/OvmfPkgDefines.fdf.inc               |   9 ++
 OvmfPkg/ResetVector/Ia16/ResetVectorVtf0.asm |  19 +++
 OvmfPkg/ResetVector/ResetVector.inf          |   9 ++
 OvmfPkg/ResetVector/ResetVector.nasmb        |  27 +++++
 OvmfPkg/ResetVector/X64/IntelTdxMetadata.asm | 115 +++++++++++++++++++
 6 files changed, 188 insertions(+)
 create mode 100644 OvmfPkg/ResetVector/X64/IntelTdxMetadata.asm

diff --git a/OvmfPkg/OvmfPkg.dec b/OvmfPkg/OvmfPkg.dec
index 1be8d5dccbc7..340d83f794d0 100644
--- a/OvmfPkg/OvmfPkg.dec
+++ b/OvmfPkg/OvmfPkg.dec
@@ -340,6 +340,15 @@
   # header definition.
   gUefiOvmfPkgTokenSpaceGuid.PcdOvmfConfidentialComputingWorkAreaHeader|4|UINT32|0x51
 
+  ## The base address and size of the TDX Cfv base and size.
+  gUefiOvmfPkgTokenSpaceGuid.PcdCfvBase|0|UINT32|0x52
+  gUefiOvmfPkgTokenSpaceGuid.PcdCfvRawDataOffset|0|UINT32|0x53
+  gUefiOvmfPkgTokenSpaceGuid.PcdCfvRawDataSize|0|UINT32|0x54
+
+  ## The base address and size of the TDX Bfv base and size.
+  gUefiOvmfPkgTokenSpaceGuid.PcdBfvBase|0|UINT32|0x55
+  gUefiOvmfPkgTokenSpaceGuid.PcdBfvRawDataOffset|0|UINT32|0x56
+  gUefiOvmfPkgTokenSpaceGuid.PcdBfvRawDataSize|0|UINT32|0x57
 
 [PcdsDynamic, PcdsDynamicEx]
   gUefiOvmfPkgTokenSpaceGuid.PcdEmuVariableEvent|0|UINT64|2
diff --git a/OvmfPkg/OvmfPkgDefines.fdf.inc b/OvmfPkg/OvmfPkgDefines.fdf.inc
index 3b5e45253916..6170c5993ce5 100644
--- a/OvmfPkg/OvmfPkgDefines.fdf.inc
+++ b/OvmfPkg/OvmfPkgDefines.fdf.inc
@@ -9,6 +9,7 @@
 ##
 
 DEFINE BLOCK_SIZE        = 0x1000
+DEFINE VARS_OFFSET       = 0
 
 #
 # A firmware binary built with FD_SIZE_IN_KB=1024, and a firmware binary built
@@ -88,6 +89,14 @@ SET gEfiMdeModulePkgTokenSpaceGuid.PcdFlashNvStorageFtwSpareSize = $(VARS_SPARE_
 # Computing Work Area header defined in the Include/WorkArea.h
 SET gUefiOvmfPkgTokenSpaceGuid.PcdOvmfConfidentialComputingWorkAreaHeader  = 4
 
+SET gUefiOvmfPkgTokenSpaceGuid.PcdCfvBase           = $(FW_BASE_ADDRESS)
+SET gUefiOvmfPkgTokenSpaceGuid.PcdCfvRawDataOffset  = $(VARS_OFFSET)
+SET gUefiOvmfPkgTokenSpaceGuid.PcdCfvRawDataSize    = $(VARS_SIZE)
+
+SET gUefiOvmfPkgTokenSpaceGuid.PcdBfvBase           = $(CODE_BASE_ADDRESS)
+SET gUefiOvmfPkgTokenSpaceGuid.PcdBfvRawDataOffset  = $(VARS_SIZE)
+SET gUefiOvmfPkgTokenSpaceGuid.PcdBfvRawDataSize    = $(CODE_SIZE)
+
 !if $(SMM_REQUIRE) == TRUE
 SET gEfiMdeModulePkgTokenSpaceGuid.PcdFlashNvStorageVariableBase64 = gUefiOvmfPkgTokenSpaceGuid.PcdOvmfFlashNvStorageVariableBase
 SET gEfiMdeModulePkgTokenSpaceGuid.PcdFlashNvStorageFtwWorkingBase = gUefiOvmfPkgTokenSpaceGuid.PcdOvmfFlashNvStorageFtwWorkingBase
diff --git a/OvmfPkg/ResetVector/Ia16/ResetVectorVtf0.asm b/OvmfPkg/ResetVector/Ia16/ResetVectorVtf0.asm
index 7ec3c6e980c3..7be43fb44a69 100644
--- a/OvmfPkg/ResetVector/Ia16/ResetVectorVtf0.asm
+++ b/OvmfPkg/ResetVector/Ia16/ResetVectorVtf0.asm
@@ -47,6 +47,25 @@ TIMES (15 - ((guidedStructureEnd - guidedStructureStart + 15) % 16)) DB 0
 ;
 guidedStructureStart:
 
+%ifdef ARCH_X64
+;
+; TDX Metadata offset block
+;
+; TdxMetadata.asm is included in ARCH_X64 because Inte TDX is only
+; available in ARCH_X64. Below block describes the offset of
+; TdxMetadata block in Ovmf image
+;
+; GUID : e47a6535-984a-4798-865e-4685a7bf8ec2
+;
+tdxMetadataOffsetStart:
+    DD      fourGigabytes - TdxMetadataGuid - 16
+    DW      tdxMetadataOffsetEnd - tdxMetadataOffsetStart
+    DB      0x35, 0x65, 0x7a, 0xe4, 0x4a, 0x98, 0x98, 0x47
+    DB      0x86, 0x5e, 0x46, 0x85, 0xa7, 0xbf, 0x8e, 0xc2
+tdxMetadataOffsetEnd:
+
+%endif
+
 ; SEV Hash Table Block
 ;
 ; This describes the guest ram area where the hypervisor should
diff --git a/OvmfPkg/ResetVector/ResetVector.inf b/OvmfPkg/ResetVector/ResetVector.inf
index a2520dde5508..320e5f2c6527 100644
--- a/OvmfPkg/ResetVector/ResetVector.inf
+++ b/OvmfPkg/ResetVector/ResetVector.inf
@@ -44,6 +44,15 @@
   gUefiOvmfPkgTokenSpaceGuid.PcdOvmfSecPeiTempRamBase
   gUefiOvmfPkgTokenSpaceGuid.PcdOvmfSecPeiTempRamSize
   gUefiOvmfPkgTokenSpaceGuid.PcdOvmfWorkAreaBase
+  gUefiOvmfPkgTokenSpaceGuid.PcdOvmfWorkAreaSize
+  gUefiOvmfPkgTokenSpaceGuid.PcdOvmfSecGhcbBackupBase
+  gUefiOvmfPkgTokenSpaceGuid.PcdOvmfSecGhcbBackupSize
+  gUefiOvmfPkgTokenSpaceGuid.PcdCfvBase
+  gUefiOvmfPkgTokenSpaceGuid.PcdCfvRawDataOffset
+  gUefiOvmfPkgTokenSpaceGuid.PcdCfvRawDataSize
+  gUefiOvmfPkgTokenSpaceGuid.PcdBfvBase
+  gUefiOvmfPkgTokenSpaceGuid.PcdBfvRawDataOffset
+  gUefiOvmfPkgTokenSpaceGuid.PcdBfvRawDataSize
 
 [FixedPcd]
   gUefiOvmfPkgTokenSpaceGuid.PcdSevLaunchSecretBase
diff --git a/OvmfPkg/ResetVector/ResetVector.nasmb b/OvmfPkg/ResetVector/ResetVector.nasmb
index d1d800c56745..711e0dac4cdf 100644
--- a/OvmfPkg/ResetVector/ResetVector.nasmb
+++ b/OvmfPkg/ResetVector/ResetVector.nasmb
@@ -67,6 +67,31 @@
     %error "This implementation inherently depends on PcdOvmfSecGhcbBase not straddling a 2MB boundary"
   %endif
 
+  %define TDX_BFV_RAW_DATA_OFFSET   FixedPcdGet32 (PcdBfvRawDataOffset)
+  %define TDX_BFV_RAW_DATA_SIZE     FixedPcdGet32 (PcdBfvRawDataSize)
+  %define TDX_BFV_MEMORY_BASE       FixedPcdGet32 (PcdBfvBase)
+  %define TDX_BFV_MEMORY_SIZE       FixedPcdGet32 (PcdBfvRawDataSize)
+
+  %define TDX_CFV_RAW_DATA_OFFSET   FixedPcdGet32 (PcdCfvRawDataOffset)
+  %define TDX_CFV_RAW_DATA_SIZE     FixedPcdGet32 (PcdCfvRawDataSize)
+  %define TDX_CFV_MEMORY_BASE       FixedPcdGet32 (PcdCfvBase),
+  %define TDX_CFV_MEMORY_SIZE       FixedPcdGet32 (PcdCfvRawDataSize),
+
+  %define TDX_HEAP_STACK_BASE       FixedPcdGet32 (PcdOvmfSecPeiTempRamBase)
+  %define TDX_HEAP_STACK_SIZE       FixedPcdGet32 (PcdOvmfSecPeiTempRamSize)
+
+  %define TDX_HOB_MEMORY_BASE       FixedPcdGet32 (PcdOvmfSecGhcbBase)
+  %define TDX_HOB_MEMORY_SIZE       FixedPcdGet32 (PcdOvmfSecGhcbSize)
+
+  %define TDX_INIT_MEMORY_BASE      FixedPcdGet32 (PcdOvmfWorkAreaBase)
+  %define TDX_INIT_MEMORY_SIZE      (FixedPcdGet32 (PcdOvmfWorkAreaSize) + FixedPcdGet32 (PcdOvmfSecGhcbBackupSize))
+
+  %define OVMF_PAGE_TABLE_BASE      FixedPcdGet32 (PcdOvmfSecPageTablesBase)
+  %define OVMF_PAGE_TABLE_SIZE      FixedPcdGet32 (PcdOvmfSecPageTablesSize)
+
+  %define TDX_WORK_AREA_PGTBL_READY (FixedPcdGet32 (PcdOvmfWorkAreaBase) + 4)
+  %define TDX_WORK_AREA_GPAW        (FixedPcdGet32 (PcdOvmfWorkAreaBase) + 8)
+
   %define PT_ADDR(Offset) (FixedPcdGet32 (PcdOvmfSecPageTablesBase) + (Offset))
 
   %define GHCB_PT_ADDR (FixedPcdGet32 (PcdOvmfSecGhcbPageTableBase))
@@ -77,6 +102,8 @@
   %define SEV_ES_WORK_AREA_RDRAND (FixedPcdGet32 (PcdSevEsWorkAreaBase) + 8)
   %define SEV_ES_WORK_AREA_ENC_MASK (FixedPcdGet32 (PcdSevEsWorkAreaBase) + 16)
   %define SEV_ES_VC_TOP_OF_STACK (FixedPcdGet32 (PcdOvmfSecPeiTempRamBase) + FixedPcdGet32 (PcdOvmfSecPeiTempRamSize))
+
+%include "X64/IntelTdxMetadata.asm"
 %include "Ia32/Flat32ToFlat64.asm"
 %include "Ia32/AmdSev.asm"
 %include "Ia32/PageTables64.asm"
diff --git a/OvmfPkg/ResetVector/X64/IntelTdxMetadata.asm b/OvmfPkg/ResetVector/X64/IntelTdxMetadata.asm
new file mode 100644
index 000000000000..07f89ef4931f
--- /dev/null
+++ b/OvmfPkg/ResetVector/X64/IntelTdxMetadata.asm
@@ -0,0 +1,115 @@
+;------------------------------------------------------------------------------
+; @file
+; Tdx Virtual Firmware metadata
+;
+; When host VMM creates a new guest TD, some initial set of TD-private pages
+; are added using the TDH.MEM.PAGE.ADD function. These pages typically contain
+; Virtual BIOS code and data along with some clear pages for stacks and heap.
+; In the meanwhile, some configuration data need be measured by host VMM.
+; Tdx Metadata is designed for this purpose to indicate host VMM how to do the
+; above tasks.
+;
+; Tdx Metadata consists of a DESCRIPTOR as the header followed by several
+; SECTIONs. Host VMM sets up the memory for TDVF according to these sections.
+;
+; _Bfv is the example (Bfv refers to the Virtual BIOS code).
+; - By DataOffset/RawDataSize host VMM knows about the position of the code
+;   in the binary image.
+; - MemoryAddress/MemoryDataSize indicates the guest physical address/size of
+;   the Bfv to be loaded.
+; - Type field means this section is of BFV. This field is designed for the
+;   purpose that in some case host VMM may do some additional processing based
+;   upon the section type. TdHob section is an example. Host VMM pass the
+;   physical memory information to the guest firmware by writing the data in
+;   the memory region designated by TdHob section.
+; - By design code part of the binary image (Bfv) should be measured by host
+;   VMM. This is indicated by the Attributes field.
+;
+; So put all these information together, when a new guest is being created,
+; the initial TD-private pages for BFV is added by TDH.MEM.PAGE.ADD function,
+; and Bfv is loaded at the guest physical address indicated by MemoryAddress.
+; Since the Attributes is TDX_METADATA_ATTRIBUTES_EXTENDMR, Bfv is measured by
+; host VMM.
+;
+; Copyright (c) 2021, Intel Corporation. All rights reserved.<BR>
+; SPDX-License-Identifier: BSD-2-Clause-Patent
+;
+;------------------------------------------------------------------------------
+
+BITS    64
+
+%define TDX_METADATA_SECTION_TYPE_BFV       0
+%define TDX_METADATA_SECTION_TYPE_CFV       1
+%define TDX_METADATA_SECTION_TYPE_TD_HOB    2
+%define TDX_METADATA_SECTION_TYPE_TEMP_MEM  3
+%define TDX_METADATA_VERSION                1
+%define TDX_METADATA_ATTRIBUTES_EXTENDMR    0x00000001
+
+ALIGN   16
+TIMES (15 - ((TdxGuidedStructureEnd - TdxGuidedStructureStart + 15) % 16)) DB 0
+
+TdxGuidedStructureStart:
+
+;
+; TDVF meta data
+;
+TdxMetadataGuid:
+  DB  0xf3, 0xf9, 0xea, 0xe9, 0x8e, 0x16, 0xd5, 0x44
+  DB  0xa8, 0xeb, 0x7f, 0x4d, 0x87, 0x38, 0xf6, 0xae
+
+_Descriptor:
+  DB 'T','D','V','F'                                  ; Signature
+  DD TdxGuidedStructureEnd - _Descriptor              ; Length
+  DD TDX_METADATA_VERSION                             ; Version
+  DD (TdxGuidedStructureEnd - _Descriptor - 16)/32    ; Number of sections
+
+_Bfv:
+  DD TDX_BFV_RAW_DATA_OFFSET
+  DD TDX_BFV_RAW_DATA_SIZE
+  DQ TDX_BFV_MEMORY_BASE
+  DQ TDX_BFV_MEMORY_SIZE
+  DD TDX_METADATA_SECTION_TYPE_BFV
+  DD TDX_METADATA_ATTRIBUTES_EXTENDMR
+
+_Cfv:
+  DD TDX_CFV_RAW_DATA_OFFSET
+  DD TDX_CFV_RAW_DATA_SIZE
+  DQ TDX_CFV_MEMORY_BASE
+  DQ TDX_CFV_MEMORY_SIZE
+  DD TDX_METADATA_SECTION_TYPE_CFV
+  DD 0
+
+_TdxHeapStack:
+  DD 0
+  DD 0
+  DQ TDX_HEAP_STACK_BASE
+  DQ TDX_HEAP_STACK_SIZE
+  DD TDX_METADATA_SECTION_TYPE_TEMP_MEM
+  DD 0
+
+_TdxInitMem:
+  DD 0
+  DD 0
+  DQ TDX_INIT_MEMORY_BASE
+  DQ TDX_INIT_MEMORY_SIZE
+  DD TDX_METADATA_SECTION_TYPE_TEMP_MEM
+  DD 0
+
+_TdHob:
+  DD 0
+  DD 0
+  DQ TDX_HOB_MEMORY_BASE
+  DQ TDX_HOB_MEMORY_SIZE
+  DD TDX_METADATA_SECTION_TYPE_TD_HOB
+  DD 0
+
+_OvmfPageTable:
+  DD 0
+  DD 0
+  DQ OVMF_PAGE_TABLE_BASE
+  DQ OVMF_PAGE_TABLE_SIZE
+  DD TDX_METADATA_SECTION_TYPE_TEMP_MEM
+  DD 0
+
+TdxGuidedStructureEnd:
+ALIGN   16
-- 
2.29.2.windows.2

[PATCH V9 4/4] OvmfPkg: Enable TDX in ResetVector

[Min Xu]

RFC： https://bugzilla.tianocore.org/show_bug.cgi?id=3429

Intel's Trust Domain Extensions (Intel TDX) refers to an Intel technology
that extends Virtual Machines Extensions (VMX) and Multi-Key Total Memory
Encryption (MKTME) with a new kind of virutal machines guest called a
Trust Domain (TD). A TD is desinged to run in a CPU mode that protects the
confidentiality of TD memory contents and the TD's CPU state from other
software, including the hosting Virtual-Machine Monitor (VMM), unless
explicitly shared by the TD itself.

Note: Intel TDX is only available on X64, so the Tdx related changes are
in X64 path. In IA32 path, there may be null stub to make the build
success.

This patch includes below major changes.

1. Ia32/IntelTdx.asm
IntelTdx.asm includes below routines used in ResetVector
 - IsTdx
   Check if the running system is Tdx guest.

 - InitTdxWorkarea
   It initialize the TDX_WORK_AREA. Because it is called by both BSP and
   APs and to avoid the race condition, only BSP can initialize the
   WORK_AREA. AP will wait until the field of TDX_WORK_AREA_PGTBL_READY
   is set.

 - ReloadFlat32
   After reset all CPUs in TDX are initialized to 32-bit protected mode.
   But GDT register is not set. So this routine loads the GDT and set the
   CR0, then jump to Flat 32 protected mode again. After that CR4 and
   other registers are set.

 - InitTdx
   This routine wrap above 3 routines together to do Tdx initialization
   in ResetVector phase.

 - IsTdxEnabled
   It is a OneTimeCall to probe if TDX is enabled by checking the
   CC_WORK_AREA.

 - CheckTdxFeaturesBeforeBuildPagetables
   This routine is called to check if it is Non-TDX guest, TDX-Bsp or
   TDX-APs. Because in TDX guest all the initialization is done by BSP
   (including the page tables). APs should not build the tables.

 - TdxPostBuildPageTables
   It is called after Page Tables are built by BSP.
   byte[TDX_WORK_AREA_PGTBL_READY] is set by BSP to indicate APs can
   leave spin and go.

2. Ia32/PageTables64.asm
As described above only the TDX BSP build the page tables. So
PageTables64.asm is updated to make sure only TDX BSP build the
PageTables. TDX APs will skip the page table building and set Cr3
directly.

3. Ia16/ResetVectorVtf0.asm
In Tdx all CPUs "reset" to run on 32-bit protected mode with flat
descriptor (paging disabled). But in Non-Td guest the initial state of
CPUs is 16-bit real mode. To resolve this conflict, BITS 16/32 is used
in the ResetVectorVtf0.asm. It checks the 32-bit protected mode or 16-bit
real mode, then jump to the corresponding entry point.

Cc: Ard Biesheuvel <ardb+tianocore@kernel.org>
Cc: Jordan Justen <jordan.l.justen@intel.com>
Cc: Gerd Hoffmann <kraxel@redhat.com>
Cc: Brijesh Singh <brijesh.singh@amd.com>
Cc: Erdem Aktas <erdemaktas@google.com>
Cc: James Bottomley <jejb@linux.ibm.com>
Cc: Jiewen Yao <jiewen.yao@intel.com>
Cc: Tom Lendacky <thomas.lendacky@amd.com>
Signed-off-by: Min Xu <min.m.xu@intel.com>
---
 OvmfPkg/ResetVector/Ia16/ResetVectorVtf0.asm |  20 ++
 OvmfPkg/ResetVector/Ia32/Flat32ToFlat64.asm  |  11 +
 OvmfPkg/ResetVector/Ia32/IntelTdx.asm        | 235 +++++++++++++++++++
 OvmfPkg/ResetVector/Ia32/PageTables64.asm    |  18 ++
 OvmfPkg/ResetVector/Main.asm                 |  10 +
 OvmfPkg/ResetVector/ResetVector.nasmb        |   1 +
 6 files changed, 295 insertions(+)
 create mode 100644 OvmfPkg/ResetVector/Ia32/IntelTdx.asm

diff --git a/OvmfPkg/ResetVector/Ia16/ResetVectorVtf0.asm b/OvmfPkg/ResetVector/Ia16/ResetVectorVtf0.asm
index 7be43fb44a69..dee2e3f9de31 100644
--- a/OvmfPkg/ResetVector/Ia16/ResetVectorVtf0.asm
+++ b/OvmfPkg/ResetVector/Ia16/ResetVectorVtf0.asm
@@ -177,10 +177,30 @@ resetVector:
 ;
 ; This is where the processor will begin execution
 ;
+; In IA32 we follow the standard reset vector flow. While in X64, Td guest
+; may be supported. Td guest requires the startup mode to be 32-bit
+; protected mode but the legacy VM startup mode is 16-bit real mode.
+; To make NASM generate such shared entry code that behaves correctly in
+; both 16-bit and 32-bit mode, more BITS directives are added.
+;
+%ifdef ARCH_IA32
     nop
     nop
     jmp     EarlyBspInitReal16
 
+%else
+
+    mov     eax, cr0
+    test    al, 1
+    jz      .Real
+BITS 32
+    jmp     Main32
+BITS 16
+.Real:
+    jmp     EarlyBspInitReal16
+
+%endif
+
 ALIGN   16
 
 fourGigabytes:
diff --git a/OvmfPkg/ResetVector/Ia32/Flat32ToFlat64.asm b/OvmfPkg/ResetVector/Ia32/Flat32ToFlat64.asm
index c6d0d898bcd1..eb3546668ef8 100644
--- a/OvmfPkg/ResetVector/Ia32/Flat32ToFlat64.asm
+++ b/OvmfPkg/ResetVector/Ia32/Flat32ToFlat64.asm
@@ -21,6 +21,17 @@ Transition32FlatTo64Flat:
     bts     eax, 5                      ; enable PAE
     mov     cr4, eax
 
+    ;
+    ; In TDX LME has already been set. So we're done and jump to enable
+    ; paging directly if Tdx is enabled.
+    ; EBX is cleared because in the later it will be used to check if
+    ; the second step of the SEV-ES mitigation is to be performed.
+    ;
+    xor     ebx, ebx
+    OneTimeCall IsTdxEnabled
+    test    eax, eax
+    jnz     EnablePaging
+
     mov     ecx, 0xc0000080
     rdmsr
     bts     eax, 8                      ; set LME
diff --git a/OvmfPkg/ResetVector/Ia32/IntelTdx.asm b/OvmfPkg/ResetVector/Ia32/IntelTdx.asm
new file mode 100644
index 000000000000..f67b1bcf0b2e
--- /dev/null
+++ b/OvmfPkg/ResetVector/Ia32/IntelTdx.asm
@@ -0,0 +1,235 @@
+;------------------------------------------------------------------------------
+; @file
+;   Intel TDX routines
+;
+; Copyright (c) 2021, Intel Corporation. All rights reserved.<BR>
+; SPDX-License-Identifier: BSD-2-Clause-Patent
+;
+;------------------------------------------------------------------------------
+
+%define SEC_DEFAULT_CR0  0x00000023
+%define SEC_DEFAULT_CR4  0x640
+%define VM_GUEST_TDX     2
+
+BITS 32
+
+;
+; Check if it is Intel Tdx
+;
+; Modified: EAX, EBX, ECX, EDX
+;
+; If it is Intel Tdx, EAX is 1
+; If it is not Intel Tdx, EAX is 0
+;
+IsTdx:
+    ;
+    ; CPUID (0)
+    ;
+    mov     eax, 0
+    cpuid
+    cmp     ebx, 0x756e6547  ; "Genu"
+    jne     IsNotTdx
+    cmp     edx, 0x49656e69  ; "ineI"
+    jne     IsNotTdx
+    cmp     ecx, 0x6c65746e  ; "ntel"
+    jne     IsNotTdx
+
+    ;
+    ; CPUID (1)
+    ;
+    mov     eax, 1
+    cpuid
+    test    ecx, 0x80000000
+    jz      IsNotTdx
+
+    ;
+    ; CPUID[0].EAX >= 0x21?
+    ;
+    mov     eax, 0
+    cpuid
+    cmp     eax, 0x21
+    jl      IsNotTdx
+
+    ;
+    ; CPUID (0x21,0)
+    ;
+    mov     eax, 0x21
+    mov     ecx, 0
+    cpuid
+
+    cmp     ebx, 0x65746E49   ; "Inte"
+    jne     IsNotTdx
+    cmp     edx, 0x5844546C   ; "lTDX"
+    jne     IsNotTdx
+    cmp     ecx, 0x20202020   ; "    "
+    jne     IsNotTdx
+
+    mov     eax, 1
+    jmp     ExitIsTdx
+
+IsNotTdx:
+    xor     eax, eax
+
+ExitIsTdx:
+
+  OneTimeCallRet IsTdx
+
+;
+; Initialize work area if it is Tdx guest. Detailed definition is in
+; OvmfPkg/Include/WorkArea.h.
+; BSP and APs all go here. Only BSP initialize this work area.
+;
+; Param[in] EBP[5:0]    CPU Supported GPAW (48 or 52)
+; Param[in] ESI[31:0]   vCPU ID (BSP is 0, others are AP)
+;
+; Modified:  EBP
+;
+InitTdxWorkarea:
+
+    ;
+    ; First check if it is Tdx
+    ;
+    OneTimeCall IsTdx
+
+    test    eax, eax
+    jz      ExitInitTdxWorkarea
+
+    cmp     esi, 0
+    je      TdxBspEntry
+
+    ;
+    ; In Td guest, BSP/AP shares the same entry point
+    ; BSP builds up the page table, while APs shouldn't do the same task.
+    ; Instead, APs just leverage the page table which is built by BSP.
+    ; APs will wait until the page table is ready.
+    ;
+TdxApWait:
+    cmp     byte[TDX_WORK_AREA_PGTBL_READY], 0
+    je      TdxApWait
+    jmp     ExitInitTdxWorkarea
+
+TdxBspEntry:
+    ;
+    ; Set Type of WORK_AREA_GUEST_TYPE so that the following code can use
+    ; these information.
+    ;
+    mov     byte[WORK_AREA_GUEST_TYPE], VM_GUEST_TDX
+
+    ;
+    ; EBP[5:0] CPU supported GPA width
+    ;
+    and     ebp, 0x3f
+    mov     DWORD[TDX_WORK_AREA_GPAW], ebp
+
+ExitInitTdxWorkarea:
+    OneTimeCallRet InitTdxWorkarea
+
+;
+; Load the GDT and set the CR0.
+;
+; Modified:  EAX, EBX, CR0, CR4, DS, ES, FS, GS, SS, CS
+;
+ReloadFlat32:
+
+    cli
+    mov     ebx, ADDR_OF(gdtr)
+    lgdt    [ebx]
+
+    mov     eax, SEC_DEFAULT_CR0
+    mov     cr0, eax
+
+    jmp     LINEAR_CODE_SEL:dword ADDR_OF(jumpToFlat32BitAndLandHere)
+
+jumpToFlat32BitAndLandHere:
+
+    mov     eax, SEC_DEFAULT_CR4
+    mov     cr4, eax
+
+    debugShowPostCode POSTCODE_32BIT_MODE
+
+    mov     ax, LINEAR_SEL
+    mov     ds, ax
+    mov     es, ax
+    mov     fs, ax
+    mov     gs, ax
+    mov     ss, ax
+
+    OneTimeCallRet ReloadFlat32
+
+;
+; Tdx initialization after entering into ResetVector
+;
+; Modified:  EAX, EBX, ECX, EDX, EBP, EDI, ESP
+;
+InitTdx:
+    ;
+    ; Save EBX in EBP because EBX will be changed in ReloadFlat32
+    ;
+    mov     ebp, ebx
+
+    ;
+    ; First load the GDT and jump to Flat32 mode
+    ;
+    OneTimeCall ReloadFlat32
+
+    ;
+    ; Initialization of Tdx work area
+    ;
+    OneTimeCall  InitTdxWorkarea
+
+    OneTimeCallRet InitTdx
+
+;
+; Check TDX features, TDX or TDX-BSP or TDX-APs?
+;
+; By design TDX BSP is reponsible for initializing the PageTables.
+; After PageTables are ready, byte[TDX_WORK_AREA_PGTBL_READY] is set to 1.
+; APs will spin when byte[TDX_WORK_AREA_PGTBL_READY] is 0 until it is set to 1.
+;
+; When this routine is run on TDX BSP, byte[TDX_WORK_AREA_PGTBL_READY] should be 0.
+; When this routine is run on TDX APs, byte[TDX_WORK_AREA_PGTBL_READY] should be 1.
+;
+;
+; Modified:  EAX, EDX
+;
+; 0-NonTdx, 1-TdxBsp, 2-TdxAps
+;
+CheckTdxFeaturesBeforeBuildPagetables:
+    xor     eax, eax
+    cmp     byte[WORK_AREA_GUEST_TYPE], VM_GUEST_TDX
+    jne     NotTdx
+
+    xor     edx, edx
+    mov     al, byte[TDX_WORK_AREA_PGTBL_READY]
+    inc     eax
+
+NotTdx:
+    OneTimeCallRet CheckTdxFeaturesBeforeBuildPagetables
+
+;
+; Set byte[TDX_WORK_AREA_PGTBL_READY] to 1
+;
+TdxPostBuildPageTables:
+    cmp     byte[WORK_AREA_GUEST_TYPE], VM_GUEST_TDX
+    jne     ExitTdxPostBuildPageTables
+    mov     byte[TDX_WORK_AREA_PGTBL_READY], 1
+
+ExitTdxPostBuildPageTables:
+    OneTimeCallRet TdxPostBuildPageTables
+
+;
+; Check if TDX is enabled
+;
+; Modified:  EAX
+;
+; If TDX is enabled then EAX will be 1
+; If TDX is disabled then EAX will be 0.
+;
+IsTdxEnabled:
+    xor     eax, eax
+    cmp     byte[WORK_AREA_GUEST_TYPE], VM_GUEST_TDX
+    jne     TdxNotEnabled
+    mov     eax, 1
+
+TdxNotEnabled:
+    OneTimeCallRet IsTdxEnabled
diff --git a/OvmfPkg/ResetVector/Ia32/PageTables64.asm b/OvmfPkg/ResetVector/Ia32/PageTables64.asm
index 02528221e560..317cad430f29 100644
--- a/OvmfPkg/ResetVector/Ia32/PageTables64.asm
+++ b/OvmfPkg/ResetVector/Ia32/PageTables64.asm
@@ -37,10 +37,23 @@ BITS    32
                        PAGE_READ_WRITE + \
                        PAGE_PRESENT)
 
+%define TDX_BSP         1
+%define TDX_AP          2
+
 ;
 ; Modified:  EAX, EBX, ECX, EDX
 ;
 SetCr3ForPageTables64:
+    ; Check the TDX features.
+    ; If it is TDX APs, then jump to SetCr3 directly.
+    ; In TD guest the initialization is done by BSP, including building
+    ; the page tables. APs will spin on until byte[TDX_WORK_AREA_PGTBL_READY]
+    ; is set.
+    OneTimeCall   CheckTdxFeaturesBeforeBuildPagetables
+    cmp       eax, TDX_BSP
+    je        ClearOvmfPageTables
+    cmp       eax, TDX_AP
+    je        SetCr3
 
     ; Check whether the SEV is active and populate the SevEsWorkArea
     OneTimeCall   CheckSevFeatures
@@ -50,6 +63,7 @@ SetCr3ForPageTables64:
     ; the page table build below.
     OneTimeCall   GetSevCBitMaskAbove31
 
+ClearOvmfPageTables:
     ;
     ; For OVMF, build some initial page tables at
     ; PcdOvmfSecPageTablesBase - (PcdOvmfSecPageTablesBase + 0x6000).
@@ -101,6 +115,10 @@ pageTableEntriesLoop:
     ; Clear the C-bit from the GHCB page if the SEV-ES is enabled.
     OneTimeCall   SevClearPageEncMaskForGhcbPage
 
+    ; TDX will do some PostBuildPages task, such as setting
+    ; byte[TDX_WORK_AREA_PGTBL_READY].
+    OneTimeCall   TdxPostBuildPageTables
+
 SetCr3:
     ;
     ; Set CR3 now that the paging structures are available
diff --git a/OvmfPkg/ResetVector/Main.asm b/OvmfPkg/ResetVector/Main.asm
index a501fbe880f2..07033e9470b5 100644
--- a/OvmfPkg/ResetVector/Main.asm
+++ b/OvmfPkg/ResetVector/Main.asm
@@ -42,6 +42,16 @@ BITS    32
     ; work area when detected.
     mov     byte[WORK_AREA_GUEST_TYPE], 0
 
+    jmp SearchBfv
+
+;
+; Entry point of Main32
+;
+Main32:
+    OneTimeCall InitTdx
+
+SearchBfv:
+
 %endif
 
     ;
diff --git a/OvmfPkg/ResetVector/ResetVector.nasmb b/OvmfPkg/ResetVector/ResetVector.nasmb
index 711e0dac4cdf..75ec800af089 100644
--- a/OvmfPkg/ResetVector/ResetVector.nasmb
+++ b/OvmfPkg/ResetVector/ResetVector.nasmb
@@ -107,6 +107,7 @@
 %include "Ia32/Flat32ToFlat64.asm"
 %include "Ia32/AmdSev.asm"
 %include "Ia32/PageTables64.asm"
+%include "Ia32/IntelTdx.asm"
 %endif
 
 %include "Ia16/Real16ToFlat32.asm"
-- 
2.29.2.windows.2

Re: [PATCH V9 1/4] OvmfPkg: Copy Main.asm from UefiCpuPkg to OvmfPkg's ResetVector

[Gerd Hoffmann]

On Tue, Oct 12, 2021 at 10:37:47AM +0800, Min Xu wrote:
> RFC： https://bugzilla.tianocore.org/show_bug.cgi?id=3429
> 
> Previously OvmfPkg/ResetVector uses the Main.asm in
> UefiCpuPkg/ReseteVector/Vtf0. In this Main.asm there is only Main16
> entry point.
> 
> This patch-set is to introduce Intel TDX into Ovmf. Main32 entry point
> is needed in Main.asm by Intel TDX. To reduce the complexity of Main.asm
> in UefiCpuPkg, OvmfPkg create its own Main.asm to meet the requirement
> of Intel TDX.
> 
> UefiCpuPkg/ResetVector/Vtf0/main.asm -> OvmfPkg/ResetVector/Main.asm

You should mention that this is an unmodified copy (so no functional
change) and the actual changes for tdx come as incremental patches.

With that:
Acked-by: Gerd Hoffmann <kraxel@redhat.com>

take care,
  Gerd

Re: [PATCH V9 2/4] OvmfPkg: Clear WORK_AREA_GUEST_TYPE in Main.asm

[Gerd Hoffmann]

On Tue, Oct 12, 2021 at 10:37:48AM +0800, Min Xu wrote:
> RFC： https://bugzilla.tianocore.org/show_bug.cgi?id=3429
> 
> Previously WORK_AREA_GUEST_TYPE was cleared in SetCr3ForPageTables64.
> This is workable for Legacy guest and SEV guest. But it doesn't work
> after Intel TDX is introduced. It is because all TDX CPUs (BSP and APs)
> start to run from 0xfffffff0, thus WORK_AREA_GUEST_TYPE will be cleared
> multi-times if it is TDX guest. So the clearance of WORK_AREA_GUEST_TYPE
> is moved to Main16 entry point in Main.asm.
> Note: WORK_AREA_GUEST_TYPE is only defined for ARCH_X64.
> 
> For Intel TDX, its corresponding entry point is Main32 (which will be
> introduced in next commit in this patch-set). WORK_AREA_GUEST_TYPE will
> be cleared there.

Acked-by: Gerd Hoffmann <kraxel@redhat.com>

take care,
  Gerd

Re: [PATCH V9 3/4] OvmfPkg: Add IntelTdxMetadata.asm

[Gerd Hoffmann]

  Hi,

> +; - Type field means this section is of BFV. This field is designed for the
> +;   purpose that in some case host VMM may do some additional processing based
> +;   upon the section type. TdHob section is an example. Host VMM pass the
> +;   physical memory information to the guest firmware by writing the data in
> +;   the memory region designated by TdHob section.

Brijesh?  What are your plans?  Do you want use this too?

If so, then the section types need some more structure.  I think best
would be reserve blocks, something like 0x00 -> 0xff common types (bfv,
cfv, temp_mem), 0x100 -> 0x1ff tdx (td_hob), 0x200 -> 0x2ff sev (cpuid,
secrets, ...).

take care,
  Gerd

Re: [PATCH V9 4/4] OvmfPkg: Enable TDX in ResetVector

[Gerd Hoffmann]

  Hi,

> +; Load the GDT and set the CR0.
> +;
> +; Modified:  EAX, EBX, CR0, CR4, DS, ES, FS, GS, SS, CS
> +;
> +ReloadFlat32:
> +
> +    cli
> +    mov     ebx, ADDR_OF(gdtr)
> +    lgdt    [ebx]

No need to modify ebx here, eax should do fine.

> +    mov     eax, SEC_DEFAULT_CR0
> +    mov     cr0, eax
> +
> +    jmp     LINEAR_CODE_SEL:dword ADDR_OF(jumpToFlat32BitAndLandHere)
> +
> +jumpToFlat32BitAndLandHere:

Strictly speaking this is not correct, you are already in Flat32 mode,
so this only loads cs.

> +InitTdx:
> +    ;
> +    ; Save EBX in EBP because EBX will be changed in ReloadFlat32
> +    ;
> +    mov     ebp, ebx

See above, there is no need to modify ebx in ReloadFlat32.
Also: seems ebx is never restored ...

take care,
  Gerd

Re: [PATCH V9 2/4] OvmfPkg: Clear WORK_AREA_GUEST_TYPE in Main.asm

[Lendacky, Thomas]

On 10/11/21 9:37 PM, Min Xu wrote:
> RFC： https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fbugzilla.tianocore.org%2Fshow_bug.cgi%3Fid%3D3429&data=04%7C01%7Cthomas.lendacky%40amd.com%7Cc4c4ac9654a940ada92308d98d2994d0%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637696032012206979%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=1SVRKXztfFcaVVer1AYOhLIhs6sVW%2BwtYQNxuuHHbTE%3D&reserved=0
> 
> Previously WORK_AREA_GUEST_TYPE was cleared in SetCr3ForPageTables64.
> This is workable for Legacy guest and SEV guest. But it doesn't work
> after Intel TDX is introduced. It is because all TDX CPUs (BSP and APs)
> start to run from 0xfffffff0, thus WORK_AREA_GUEST_TYPE will be cleared
> multi-times if it is TDX guest. So the clearance of WORK_AREA_GUEST_TYPE
> is moved to Main16 entry point in Main.asm.
> Note: WORK_AREA_GUEST_TYPE is only defined for ARCH_X64.
> 
> For Intel TDX, its corresponding entry point is Main32 (which will be
> introduced in next commit in this patch-set). WORK_AREA_GUEST_TYPE will
> be cleared there.
> 
> Cc: Ard Biesheuvel <ardb+tianocore@kernel.org>
> Cc: Jordan Justen <jordan.l.justen@intel.com>
> Cc: Gerd Hoffmann <kraxel@redhat.com>
> Cc: Brijesh Singh <brijesh.singh@amd.com>
> Cc: Erdem Aktas <erdemaktas@google.com>
> Cc: James Bottomley <jejb@linux.ibm.com>
> Cc: Jiewen Yao <jiewen.yao@intel.com>
> Cc: Tom Lendacky <thomas.lendacky@amd.com>
> Signed-off-by: Min Xu <min.m.xu@intel.com>
> ---
>   OvmfPkg/ResetVector/Ia32/PageTables64.asm | 4 ----
>   OvmfPkg/ResetVector/Main.asm              | 8 ++++++++
>   2 files changed, 8 insertions(+), 4 deletions(-)
> 
> diff --git a/OvmfPkg/ResetVector/Ia32/PageTables64.asm b/OvmfPkg/ResetVector/Ia32/PageTables64.asm
> index 07b6ca070909..02528221e560 100644
> --- a/OvmfPkg/ResetVector/Ia32/PageTables64.asm
> +++ b/OvmfPkg/ResetVector/Ia32/PageTables64.asm
> @@ -42,10 +42,6 @@ BITS    32
>   ;
>   SetCr3ForPageTables64:
>   
> -    ; Clear the WorkArea header. The SEV probe routines will populate the
> -    ; work area when detected.
> -    mov     byte[WORK_AREA_GUEST_TYPE], 0
> -
>       ; Check whether the SEV is active and populate the SevEsWorkArea
>       OneTimeCall   CheckSevFeatures
>   
> diff --git a/OvmfPkg/ResetVector/Main.asm b/OvmfPkg/ResetVector/Main.asm
> index ae90a148fce7..a501fbe880f2 100644
> --- a/OvmfPkg/ResetVector/Main.asm
> +++ b/OvmfPkg/ResetVector/Main.asm
> @@ -36,6 +36,14 @@ Main16:
>   
>   BITS    32
>   
> +%ifdef ARCH_X64

A regular SEV guest can be built in the hybrid IA32 and X64 configuration, 
so this will break existing SEV firmwares built in that manner. Only 
SEV-ES and SEV-SNP require the full X64 confguration.

Thanks,
Tom

> +
> +    ; Clear the WorkArea header. The SEV probe routines will populate the
> +    ; work area when detected.
> +    mov     byte[WORK_AREA_GUEST_TYPE], 0
> +
> +%endif
> +
>       ;
>       ; Search for the Boot Firmware Volume (BFV)
>       ;
> 

Re: [PATCH V9 1/4] OvmfPkg: Copy Main.asm from UefiCpuPkg to OvmfPkg's ResetVector

[Min Xu]

On October 12, 2021 3:02 PM, Gerd Hoffmann wrote:
> On Tue, Oct 12, 2021 at 10:37:47AM +0800, Min Xu wrote:
> > RFC： https://bugzilla.tianocore.org/show_bug.cgi?id=3429
> >
> > Previously OvmfPkg/ResetVector uses the Main.asm in
> > UefiCpuPkg/ReseteVector/Vtf0. In this Main.asm there is only Main16
> > entry point.
> >
> > This patch-set is to introduce Intel TDX into Ovmf. Main32 entry point
> > is needed in Main.asm by Intel TDX. To reduce the complexity of
> > Main.asm in UefiCpuPkg, OvmfPkg create its own Main.asm to meet the
> > requirement of Intel TDX.
> >
> > UefiCpuPkg/ResetVector/Vtf0/main.asm ->
> OvmfPkg/ResetVector/Main.asm
> 
> You should mention that this is an unmodified copy (so no functional
> change) and the actual changes for tdx come as incremental patches.
>
Thanks for reminder. It will be updated in next version.
> 
> With that:
> Acked-by: Gerd Hoffmann <kraxel@redhat.com>
> 
Thanks!
Min

Re: [edk2-devel] [PATCH V9 2/4] OvmfPkg: Clear WORK_AREA_GUEST_TYPE in Main.asm

[Min Xu]

On October 12, 2021 9:23 PM, Lendacky Thomas wrote:
> On 10/11/21 9:37 PM, Min Xu wrote:
> > diff --git a/OvmfPkg/ResetVector/Main.asm
> > b/OvmfPkg/ResetVector/Main.asm index ae90a148fce7..a501fbe880f2
> 100644
> > --- a/OvmfPkg/ResetVector/Main.asm
> > +++ b/OvmfPkg/ResetVector/Main.asm
> > @@ -36,6 +36,14 @@ Main16:
> >
> >   BITS    32
> >
> > +%ifdef ARCH_X64
> 
> A regular SEV guest can be built in the hybrid IA32 and X64 configuration, so this
> will break existing SEV firmwares built in that manner. Only SEV-ES and SEV-SNP
> require the full X64 confguration.
> 
WORK_AREA_GUEST_TYPE is defined in https://github.com/tianocore/edk2/blob/master/OvmfPkg/ResetVector/ResetVector.nasmb#L75 and previously it was cleared in https://github.com/tianocore/edk2/blob/master/OvmfPkg/ResetVector/Ia32/PageTables64.asm#L47. These 2 lines are both surrounded by ARCH_X64.
So in this commit, the clearance of WORK_AREA_GUEST_TYPE in Main.asm is surrounded by ARCH_X64 too.
Brijesh, what's your comments on this change?

Thanks!
Min

Re: [edk2-devel] [PATCH V9 4/4] OvmfPkg: Enable TDX in ResetVector

[Min Xu]

On October 12, 2021 3:43 PM, Gerd Hoffmann wrote:
>   Hi,
> 
> > +; Load the GDT and set the CR0.
> > +;
> > +; Modified:  EAX, EBX, CR0, CR4, DS, ES, FS, GS, SS, CS ;
> > +ReloadFlat32:
> > +
> > +    cli
> > +    mov     ebx, ADDR_OF(gdtr)
> > +    lgdt    [ebx]
> 
> No need to modify ebx here, eax should do fine.
You're right. It will be updated in next version.
> 
> > +    mov     eax, SEC_DEFAULT_CR0
> > +    mov     cr0, eax
> > +
> > +    jmp     LINEAR_CODE_SEL:dword ADDR_OF(jumpToFlat32BitAndLandHere)
> > +
> > +jumpToFlat32BitAndLandHere:
> 
> Strictly speaking this is not correct, you are already in Flat32 mode, so this only
> loads cs.
TDX: https://software.intel.com/content/dam/develop/external/us/en/documents/tdx-module-1.0-public-spec-v0.931.pdf
In [TDX] Section 10.1.3 CR0 and CR4 are initialized when Tdx guest is created. So initialization of CR0 and CR4 are not needed.
It will be updated in next version.
> 
> > +InitTdx:
> > +    ;
> > +    ; Save EBX in EBP because EBX will be changed in ReloadFlat32
> > +    ;
> > +    mov     ebp, ebx
> 
> See above, there is no need to modify ebx in ReloadFlat32.
> Also: seems ebx is never restored ...
In [TDX] Section 10.1.2 EBX[5:0] contains the GPAW. Since EBX is not changed in ReloadFlat32, *mov ebp, ebx* is not needed.
It will be removed in next version.
> 

Thanks!
Min

Re: [edk2-devel] [PATCH V9 3/4] OvmfPkg: Add IntelTdxMetadata.asm

[Brijesh Singh]

On 10/12/21 12:22 AM, Gerd Hoffmann via groups.io wrote:
>   Hi,
>
>> +; - Type field means this section is of BFV. This field is designed for the
>> +;   purpose that in some case host VMM may do some additional processing based
>> +;   upon the section type. TdHob section is an example. Host VMM pass the
>> +;   physical memory information to the guest firmware by writing the data in
>> +;   the memory region designated by TdHob section.
> Brijesh?  What are your plans?  Do you want use this too?

Based on the previous comment by Jiewen it seems TDX folks want to have
a separate metadata for flexibility. In the v9, I am creating metadata
just for the SEV with a different GUID. Basically same as version 8 but
now the number of descriptors will be smaller (since we focus on SEV-SNP).

thanks 

> If so, then the section types need some more structure.  I think best
> would be reserve blocks, something like 0x00 -> 0xff common types (bfv,
> cfv, temp_mem), 0x100 -> 0x1ff tdx (td_hob), 0x200 -> 0x2ff sev (cpuid,
> secrets, ...).
>
> take care,
>   Gerd
>
>
>
> 
>
>

Re: [edk2-devel] [PATCH V9 3/4] OvmfPkg: Add IntelTdxMetadata.asm

[Gerd Hoffmann]

On Wed, Oct 13, 2021 at 06:44:28AM -0500, Brijesh Singh wrote:
> 
> On 10/12/21 12:22 AM, Gerd Hoffmann via groups.io wrote:
> >   Hi,
> >
> >> +; - Type field means this section is of BFV. This field is designed for the
> >> +;   purpose that in some case host VMM may do some additional processing based
> >> +;   upon the section type. TdHob section is an example. Host VMM pass the
> >> +;   physical memory information to the guest firmware by writing the data in
> >> +;   the memory region designated by TdHob section.
> > Brijesh?  What are your plans?  Do you want use this too?
> 
> Based on the previous comment by Jiewen it seems TDX folks want to have
> a separate metadata for flexibility. In the v9, I am creating metadata
> just for the SEV with a different GUID. Basically same as version 8 but
> now the number of descriptors will be smaller (since we focus on SEV-SNP).

Ok.  We can leave IntelTdxMetadata.asm as-is then.  So

Acked-by: Gerd Hoffmann <kraxel@redhat.com>

take care,
  Gerd

Re: [edk2-devel] [PATCH V9 2/4] OvmfPkg: Clear WORK_AREA_GUEST_TYPE in Main.asm

[Brijesh Singh]

On 10/12/21 5:58 PM, Xu, Min M wrote:
> On October 12, 2021 9:23 PM, Lendacky Thomas wrote:
>> On 10/11/21 9:37 PM, Min Xu wrote:
>>> diff --git a/OvmfPkg/ResetVector/Main.asm
>>> b/OvmfPkg/ResetVector/Main.asm index ae90a148fce7..a501fbe880f2
>> 100644
>>> --- a/OvmfPkg/ResetVector/Main.asm
>>> +++ b/OvmfPkg/ResetVector/Main.asm
>>> @@ -36,6 +36,14 @@ Main16:
>>>
>>>   BITS    32
>>>
>>> +%ifdef ARCH_X64
>> A regular SEV guest can be built in the hybrid IA32 and X64 configuration, so this
>> will break existing SEV firmwares built in that manner. Only SEV-ES and SEV-SNP
>> require the full X64 confguration.
>>
> WORK_AREA_GUEST_TYPE is defined in https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2%2Fblob%2Fmaster%2FOvmfPkg%2FResetVector%2FResetVector.nasmb%23L75&data=04%7C01%7Cbrijesh.singh%40amd.com%7C10fd025cc9324fe76a0808d98de49924%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637696835234415070%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=y7cujCXYfhvORki5aNbLF9Eq3BBrcfHykvlWFcYAciI%3D&reserved=0 and previously it was cleared in https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2%2Fblob%2Fmaster%2FOvmfPkg%2FResetVector%2FIa32%2FPageTables64.asm%23L47&data=04%7C01%7Cbrijesh.singh%40amd.com%7C10fd025cc9324fe76a0808d98de49924%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637696835234425063%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=AQLw4AJ%2FFxoGjfT6arKFSTvkCshRMz1v5RLZOnA8p4A%3D&reserved=0. These 2 lines are both surrounded by ARCH_X64.
> So in this commit, the clearance of WORK_AREA_GUEST_TYPE in Main.asm is surrounded by ARCH_X64 too.
> Brijesh, what's your comments on this change?

Good point Tom. The WORK_AREA_GUEST_TYPE define should be moved outside
the ARCH_X86. I missed it mainly because we renamed the ESWorkArea to
Generic workarea but EsWorkArea was defined in ARCH_X86 only.

Min,

I can send a patch to move define outside the ARCH_X64 and then you can
align the TDX code accordingly.

> Thanks!
> Min

Re: [edk2-devel] [PATCH V9 2/4] OvmfPkg: Clear WORK_AREA_GUEST_TYPE in Main.asm

[Min Xu]

On October 13, 2021 11:46 PM, Brijesh Singh wrote:
> On 10/12/21 5:58 PM, Xu, Min M wrote:
> > On October 12, 2021 9:23 PM, Lendacky Thomas wrote:
> Good point Tom. The WORK_AREA_GUEST_TYPE define should be moved
> outside the ARCH_X86. I missed it mainly because we renamed the
> ESWorkArea to Generic workarea but EsWorkArea was defined in ARCH_X86
> only.
> 
> Min,
> 
> I can send a patch to move define outside the ARCH_X64 and then you can
> align the TDX code accordingly.
> 
Ok. I will update the TDX code based on your new patch.

Thanks.
Min