[PATCH v6 00/29] Add AMD Secure Nested Paging (SEV-SNP) support

[PATCH v6 00/29] Add AMD Secure Nested Paging (SEV-SNP) support

[Brijesh Singh]

BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=3275

SEV-SNP builds upon existing SEV and SEV-ES functionality while adding
new hardware-based memory protections. SEV-SNP adds strong memory integrity
protection to help prevent malicious hypervisor-based attacks like data
replay, memory re-mapping and more in order to create an isolated memory
encryption environment.
 
This series provides the basic building blocks to support booting the SEV-SNP
VMs, it does not cover all the security enhancement introduced by the SEV-SNP
such as interrupt protection.

Many of the integrity guarantees of SEV-SNP are enforced through a new
structure called the Reverse Map Table (RMP). Adding a new page to SEV-SNP
VM requires a 2-step process. First, the hypervisor assigns a page to the
guest using the new RMPUPDATE instruction. This transitions the page to
guest-invalid. Second, the guest validates the page using the new PVALIDATE
instruction. The SEV-SNP VMs can use the new "Page State Change Request NAE"
defined in the GHCB specification to ask hypervisor to add or remove page
from the RMP table.

Each page assigned to the SEV-SNP VM can either be validated or unvalidated,
as indicated by the Validated flag in the page's RMP entry. There are two
approaches that can be taken for the page validation: Pre-validation and
Lazy Validation.

Under pre-validation, the pages are validated prior to first use. And under
lazy validation, pages are validated when first accessed. An access to a
unvalidated page results in a #VC exception, at which time the exception
handler may validate the page. Lazy validation requires careful tracking of
the validated pages to avoid validating the same GPA more than once. The
recently introduced "Unaccepted" memory type can be used to communicate the
unvalidated memory ranges to the Guest OS.

At this time we only support the pre-validation. OVMF detects all the available
system RAM in the PEI phase. When SEV-SNP is enabled, the memory is validated
before it is made available to the EDK2 core.

Now that series contains all the basic support required to launch SEV-SNP
guest. We are still missing the Interrupt security feature provided by the
SNP. The feature will be added after the base support is accepted.

Additional resources
---------------------
SEV-SNP whitepaper
https://www.amd.com/system/files/TechDocs/SEV-SNP-strengthening-vm-isolation-with-integrity-protection-and-more.pdf

APM 2: https://www.amd.com/system/files/TechDocs/24593.pdf (section 15.36)

The complete source is available at
https://github.com/AMDESE/ovmf/tree/sev-snp-rfc-5

GHCB spec:
https://developer.amd.com/wp-content/resources/56421.pdf

SEV-SNP firmware specification:
https://www.amd.com/system/files/TechDocs/56860.pdf

Change since v5:
 * When possible use the CPUID value from CPUID page
 * Move the SEV specific functions from SecMain.c in AmdSev.c
 * Rebase to the latest code
 * Add the review feedback from Yao.

Change since v4:
 * Use the correct MSR for the SEV_STATUS
 * Add VMPL-0 check

Change since v3:
 * ResetVector: move all SEV specific code in AmdSev.asm and add macros to keep
   the code readable.
 * Drop extending the EsWorkArea to contain SNP specific state.
 * Drop the GhcbGpa library and call the VmgExit directly to register GHCB GPA.
 * Install the CC blob config table from AmdSevDxe instead of extending the
   AmdSev/SecretsDxe for it.
 * Add the separate PCDs for the SNP Secrets.

Changes since v2:
 * Add support for the AP creation.
 * Use the module-scoping override to make AmdSevDxe use the IO port for PCI reads.
 * Use the reserved memory type for CPUID and Secrets page.
 * 
Changes since v1:
 * Drop the interval tree support to detect the pre-validated overlap region.
 * Use an array to keep track of pre-validated regions.
 * Add support to query the Hypervisor feature and verify that SNP feature is supported.
 * Introduce MemEncryptSevClearMmioPageEncMask() to clear the C-bit from MMIO ranges.
 * Pull the SevSecretDxe and SevSecretPei into OVMF package build.
 * Extend the SevSecretDxe to expose confidential computing blob location through
   EFI configuration table.

Brijesh Singh (25):
  OvmfPkg: reserve SNP secrets page
  OvmfPkg: reserve CPUID page for SEV-SNP
  OvmfPkg/ResetVector: introduce SEV-SNP boot block GUID
  OvmfPkg/ResetVector: invalidate the GHCB page
  OvmfPkg/ResetVector: check the vmpl level
  OvmfPkg/ResetVector: pre-validate the data pages used in SEC phase
  UefiCpuPkg: Define the SEV-SNP specific dynamic PCDs
  OvmfPkg/MemEncryptSevLib: add MemEncryptSevSnpEnabled()
  OvmfPkg/SecMain: move SEV specific routines in AmdSev.c
  OvmfPkg/SecMain: register GHCB gpa for the SEV-SNP guest
  OvmfPkg/PlatformPei: register GHCB gpa for the SEV-SNP guest
  OvmfPkg/AmdSevDxe: do not use extended PCI config space
  OvmfPkg/MemEncryptSevLib: add support to validate system RAM
  OvmfPkg/BaseMemEncryptSevLib: skip the pre-validated system RAM
  OvmfPkg/MemEncryptSevLib: add support to validate > 4GB memory in PEI
    phase
  OvmfPkg/SecMain: pre-validate the memory used for decompressing Fv
  OvmfPkg/PlatformPei: validate the system RAM when SNP is active
  OvmfPkg/PlatformPei: set the SEV-SNP enabled PCD
  OvmfPkg/PlatformPei: set the Hypervisor Features PCD
  MdePkg/GHCB: increase the GHCB protocol max version
  UefiCpuPkg/MpLib: add support to register GHCB GPA when SEV-SNP is
    enabled
  OvmfPkg/MemEncryptSevLib: change the page state in the RMP table
  OvmfPkg/MemEncryptSevLib: skip page state change for Mmio address
  OvmfPkg/PlatformPei: mark cpuid and secrets memory reserved in EFI map
  OvmfPkg/AmdSev: expose the SNP reserved pages through configuration
    table

Michael Roth (3):
  OvmfPkg/ResetVector: use SEV-SNP-validated CPUID values
  OvmfPkg/VmgExitLib: use SEV-SNP-validated CPUID values
  UefiCpuPkg/MpInitLib: use BSP to do extended topology check

Tom Lendacky (1):
  UefiCpuPkg/MpInitLib: Use SEV-SNP AP Creation NAE event to launch APs

 OvmfPkg/OvmfPkg.dec                           |  23 +
 UefiCpuPkg/UefiCpuPkg.dec                     |  11 +
 OvmfPkg/AmdSev/AmdSevX64.dsc                  |   5 +-
 OvmfPkg/Bhyve/BhyveX64.dsc                    |   5 +-
 OvmfPkg/OvmfPkgIa32.dsc                       |   1 +
 OvmfPkg/OvmfPkgIa32X64.dsc                    |   6 +-
 OvmfPkg/OvmfPkgX64.dsc                        |   5 +-
 OvmfPkg/OvmfXen.dsc                           |   5 +-
 OvmfPkg/OvmfPkgX64.fdf                        |  12 +-
 OvmfPkg/AmdSevDxe/AmdSevDxe.inf               |   7 +
 .../DxeMemEncryptSevLib.inf                   |   3 +
 .../PeiMemEncryptSevLib.inf                   |   7 +
 .../SecMemEncryptSevLib.inf                   |   3 +
 OvmfPkg/Library/VmgExitLib/SecVmgExitLib.inf  |   2 +
 OvmfPkg/Library/VmgExitLib/VmgExitLib.inf     |   3 +
 OvmfPkg/PlatformPei/PlatformPei.inf           |  10 +
 OvmfPkg/ResetVector/ResetVector.inf           |   6 +
 OvmfPkg/Sec/SecMain.inf                       |   4 +
 UefiCpuPkg/Library/MpInitLib/DxeMpInitLib.inf |   4 +
 UefiCpuPkg/Library/MpInitLib/PeiMpInitLib.inf |   4 +
 MdePkg/Include/Register/Amd/Ghcb.h            |   2 +-
 .../Guid/ConfidentialComputingSecret.h        |  18 +
 OvmfPkg/Include/Library/MemEncryptSevLib.h    |  26 +
 .../X64/SnpPageStateChange.h                  |  31 ++
 .../BaseMemEncryptSevLib/X64/VirtualMemory.h  |  19 +
 OvmfPkg/Sec/AmdSev.h                          |  95 ++++
 UefiCpuPkg/Library/MpInitLib/MpLib.h          |  20 +
 OvmfPkg/AmdSevDxe/AmdSevDxe.c                 |  23 +
 .../DxeMemEncryptSevLibInternal.c             |  27 ++
 .../Ia32/MemEncryptSevLib.c                   |  17 +
 .../PeiMemEncryptSevLibInternal.c             |  27 ++
 .../SecMemEncryptSevLibInternal.c             |  19 +
 .../X64/DxeSnpSystemRamValidate.c             |  40 ++
 .../X64/PeiDxeVirtualMemory.c                 | 167 ++++++-
 .../X64/PeiSnpSystemRamValidate.c             | 126 +++++
 .../X64/SecSnpSystemRamValidate.c             |  36 ++
 .../X64/SnpPageStateChangeInternal.c          | 295 ++++++++++++
 OvmfPkg/Library/VmgExitLib/VmgExitVcHandler.c | 444 ++++++++++++++++--
 OvmfPkg/PlatformPei/AmdSev.c                  | 192 ++++++++
 OvmfPkg/PlatformPei/MemDetect.c               |  21 +
 OvmfPkg/Sec/AmdSev.c                          | 267 +++++++++++
 OvmfPkg/Sec/SecMain.c                         | 160 +------
 UefiCpuPkg/Library/MpInitLib/DxeMpLib.c       |  11 +-
 .../MpInitLib/Ia32/SevSnpRmpAdjustInternal.c  |  31 ++
 UefiCpuPkg/Library/MpInitLib/MpLib.c          | 286 ++++++++++-
 .../MpInitLib/X64/SevSnpRmpAdjustInternal.c   |  44 ++
 OvmfPkg/FvmainCompactScratchEnd.fdf.inc       |   5 +
 OvmfPkg/ResetVector/Ia16/ResetVectorVtf0.asm  |  28 ++
 OvmfPkg/ResetVector/Ia32/AmdSev.asm           | 307 +++++++++++-
 OvmfPkg/ResetVector/ResetVector.nasmb         |   6 +
 UefiCpuPkg/Library/MpInitLib/MpEqu.inc        |   2 +
 UefiCpuPkg/Library/MpInitLib/X64/MpFuncs.nasm |  78 +++
 52 files changed, 2771 insertions(+), 225 deletions(-)
 create mode 100644 OvmfPkg/Library/BaseMemEncryptSevLib/X64/SnpPageStateChange.h
 create mode 100644 OvmfPkg/Sec/AmdSev.h
 create mode 100644 OvmfPkg/Library/BaseMemEncryptSevLib/X64/DxeSnpSystemRamValidate.c
 create mode 100644 OvmfPkg/Library/BaseMemEncryptSevLib/X64/PeiSnpSystemRamValidate.c
 create mode 100644 OvmfPkg/Library/BaseMemEncryptSevLib/X64/SecSnpSystemRamValidate.c
 create mode 100644 OvmfPkg/Library/BaseMemEncryptSevLib/X64/SnpPageStateChangeInternal.c
 create mode 100644 OvmfPkg/Sec/AmdSev.c
 create mode 100644 UefiCpuPkg/Library/MpInitLib/Ia32/SevSnpRmpAdjustInternal.c
 create mode 100644 UefiCpuPkg/Library/MpInitLib/X64/SevSnpRmpAdjustInternal.c

-- 
2.17.1

[PATCH v6 01/29] OvmfPkg: reserve SNP secrets page

[Brijesh Singh]

BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=3275

During the SNP guest launch sequence, a special secrets page needs to be
inserted by the VMM. The PSP will populate the page; it will contain the
VM Platform Communication Key (VMPCKs) used by the guest to send and
receive secure messages to the PSP.

The purpose of the secrets page in the SEV-SNP is different from the one
used in SEV guests. In SEV, the secrets page contains the guest owner's
private data after the remote attestation.

Cc: Michael Roth <michael.roth@amd.com>
Cc: James Bottomley <jejb@linux.ibm.com>
Cc: Min Xu <min.m.xu@intel.com>
Cc: Jiewen Yao <jiewen.yao@intel.com>
Cc: Tom Lendacky <thomas.lendacky@amd.com>
Cc: Jordan Justen <jordan.l.justen@intel.com>
Cc: Ard Biesheuvel <ardb+tianocore@kernel.org>
Cc: Erdem Aktas <erdemaktas@google.com>
Signed-off-by: Brijesh Singh <brijesh.singh@amd.com>
---
 OvmfPkg/OvmfPkg.dec    | 6 ++++++
 OvmfPkg/OvmfPkgX64.fdf | 3 +++
 2 files changed, 9 insertions(+)

diff --git a/OvmfPkg/OvmfPkg.dec b/OvmfPkg/OvmfPkg.dec
index c37dafad49bb..6266fdef6054 100644
--- a/OvmfPkg/OvmfPkg.dec
+++ b/OvmfPkg/OvmfPkg.dec
@@ -340,6 +340,12 @@ [PcdsFixedAtBuild]
   # header definition.
   gUefiOvmfPkgTokenSpaceGuid.PcdOvmfConfidentialComputingWorkAreaHeader|0|UINT32|0x51
 
+  ## The base address and size of the SEV-SNP Secrets Area that contains
+  #  the VM platform communication key used to send and recieve the
+  #  messages to the PSP. If this is set in the .fdf, the platform
+  #  is responsible to reserve this area from DXE phase overwrites.
+  gUefiOvmfPkgTokenSpaceGuid.PcdOvmfSnpSecretsBase|0|UINT32|0x52
+  gUefiOvmfPkgTokenSpaceGuid.PcdOvmfSnpSecretsSize|0|UINT32|0x53
 
 [PcdsDynamic, PcdsDynamicEx]
   gUefiOvmfPkgTokenSpaceGuid.PcdEmuVariableEvent|0|UINT64|2
diff --git a/OvmfPkg/OvmfPkgX64.fdf b/OvmfPkg/OvmfPkgX64.fdf
index 23936242e74a..5b871db20ab2 100644
--- a/OvmfPkg/OvmfPkgX64.fdf
+++ b/OvmfPkg/OvmfPkgX64.fdf
@@ -88,6 +88,9 @@ [FD.MEMFD]
 0x00C000|0x001000
 gUefiOvmfPkgTokenSpaceGuid.PcdOvmfSecGhcbBackupBase|gUefiOvmfPkgTokenSpaceGuid.PcdOvmfSecGhcbBackupSize
 
+0x00D000|0x001000
+gUefiOvmfPkgTokenSpaceGuid.PcdOvmfSnpSecretsBase|gUefiOvmfPkgTokenSpaceGuid.PcdOvmfSnpSecretsSize
+
 0x010000|0x010000
 gUefiOvmfPkgTokenSpaceGuid.PcdOvmfSecPeiTempRamBase|gUefiOvmfPkgTokenSpaceGuid.PcdOvmfSecPeiTempRamSize
 
-- 
2.17.1

[PATCH v6 02/29] OvmfPkg: reserve CPUID page for SEV-SNP

[Brijesh Singh]

BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=3275

Platform features and capabilities are traditionally discovered via the
CPUID instruction. Hypervisors typically trap and emulate the CPUID
instruction for a variety of reasons. There are some cases where incorrect
CPUID information can potentially lead to a security issue. The SEV-SNP
firmware provides a feature to filter the CPUID results through the PSP.
The filtered CPUID values are saved on a special page for the guest to
consume. Reserve a page in MEMFD that will contain the results of
filtered CPUID values.

Cc: Michael Roth <michael.roth@amd.com>
Cc: James Bottomley <jejb@linux.ibm.com>
Cc: Min Xu <min.m.xu@intel.com>
Cc: Jiewen Yao <jiewen.yao@intel.com>
Cc: Tom Lendacky <thomas.lendacky@amd.com>
Cc: Jordan Justen <jordan.l.justen@intel.com>
Cc: Ard Biesheuvel <ardb+tianocore@kernel.org>
Cc: Erdem Aktas <erdemaktas@google.com>
Signed-off-by: Brijesh Singh <brijesh.singh@amd.com>
---
 OvmfPkg/OvmfPkg.dec    | 6 ++++++
 OvmfPkg/OvmfPkgX64.fdf | 3 +++
 2 files changed, 9 insertions(+)

diff --git a/OvmfPkg/OvmfPkg.dec b/OvmfPkg/OvmfPkg.dec
index 6266fdef6054..afe9b7135560 100644
--- a/OvmfPkg/OvmfPkg.dec
+++ b/OvmfPkg/OvmfPkg.dec
@@ -347,6 +347,12 @@ [PcdsFixedAtBuild]
   gUefiOvmfPkgTokenSpaceGuid.PcdOvmfSnpSecretsBase|0|UINT32|0x52
   gUefiOvmfPkgTokenSpaceGuid.PcdOvmfSnpSecretsSize|0|UINT32|0x53
 
+  ## The base address and size of the SEV-SNP CPUID Area that contains
+  #  the PSP filtered CPUID results. If this is set in the .fdf, the
+  #  platform is responsible to reserve this area from DXE phase overwrites.
+  gUefiOvmfPkgTokenSpaceGuid.PcdOvmfSnpCpuidBase|0|UINT32|0x54
+  gUefiOvmfPkgTokenSpaceGuid.PcdOvmfSnpCpuidSize|0|UINT32|0x55
+
 [PcdsDynamic, PcdsDynamicEx]
   gUefiOvmfPkgTokenSpaceGuid.PcdEmuVariableEvent|0|UINT64|2
   gUefiOvmfPkgTokenSpaceGuid.PcdOvmfFlashVariablesEnable|FALSE|BOOLEAN|0x10
diff --git a/OvmfPkg/OvmfPkgX64.fdf b/OvmfPkg/OvmfPkgX64.fdf
index 5b871db20ab2..1e292d11ace3 100644
--- a/OvmfPkg/OvmfPkgX64.fdf
+++ b/OvmfPkg/OvmfPkgX64.fdf
@@ -91,6 +91,9 @@ [FD.MEMFD]
 0x00D000|0x001000
 gUefiOvmfPkgTokenSpaceGuid.PcdOvmfSnpSecretsBase|gUefiOvmfPkgTokenSpaceGuid.PcdOvmfSnpSecretsSize
 
+0x00E000|0x001000
+gUefiOvmfPkgTokenSpaceGuid.PcdOvmfSnpCpuidBase|gUefiOvmfPkgTokenSpaceGuid.PcdOvmfSnpCpuidSize
+
 0x010000|0x010000
 gUefiOvmfPkgTokenSpaceGuid.PcdOvmfSecPeiTempRamBase|gUefiOvmfPkgTokenSpaceGuid.PcdOvmfSecPeiTempRamSize
 
-- 
2.17.1

[PATCH v6 03/29] OvmfPkg/ResetVector: introduce SEV-SNP boot block GUID

[Brijesh Singh]

BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=3275

Introduce a new SEV-SNP boot-specific GUID block. The block is used to
communicate the secrets and cpuid memory area reserved by the guest BIOS.
When SEV-SNP is enabled, the hypervisor will locate the SEV-SNP boot
block to get the location of the Secrets and CPUID page and call the
PSP firmware command to populate those memory areas.

Cc: Michael Roth <michael.roth@amd.com>
Cc: James Bottomley <jejb@linux.ibm.com>
Cc: Min Xu <min.m.xu@intel.com>
Cc: Jiewen Yao <jiewen.yao@intel.com>
Cc: Tom Lendacky <thomas.lendacky@amd.com>
Cc: Jordan Justen <jordan.l.justen@intel.com>
Cc: Ard Biesheuvel <ardb+tianocore@kernel.org>
Cc: Erdem Aktas <erdemaktas@google.com>
Signed-off-by: Brijesh Singh <brijesh.singh@amd.com>
---
 OvmfPkg/ResetVector/ResetVector.inf          |  4 ++++
 OvmfPkg/ResetVector/Ia16/ResetVectorVtf0.asm | 23 ++++++++++++++++++++
 OvmfPkg/ResetVector/ResetVector.nasmb        |  4 ++++
 3 files changed, 31 insertions(+)

diff --git a/OvmfPkg/ResetVector/ResetVector.inf b/OvmfPkg/ResetVector/ResetVector.inf
index a2520dde5508..34d843de62c4 100644
--- a/OvmfPkg/ResetVector/ResetVector.inf
+++ b/OvmfPkg/ResetVector/ResetVector.inf
@@ -50,3 +50,7 @@ [FixedPcd]
   gUefiOvmfPkgTokenSpaceGuid.PcdSevLaunchSecretSize
   gUefiOvmfPkgTokenSpaceGuid.PcdQemuHashTableBase
   gUefiOvmfPkgTokenSpaceGuid.PcdQemuHashTableSize
+  gUefiOvmfPkgTokenSpaceGuid.PcdOvmfSnpSecretsBase
+  gUefiOvmfPkgTokenSpaceGuid.PcdOvmfSnpSecretsSize
+  gUefiOvmfPkgTokenSpaceGuid.PcdOvmfSnpCpuidBase
+  gUefiOvmfPkgTokenSpaceGuid.PcdOvmfSnpCpuidSize
diff --git a/OvmfPkg/ResetVector/Ia16/ResetVectorVtf0.asm b/OvmfPkg/ResetVector/Ia16/ResetVectorVtf0.asm
index 7ec3c6e980c3..71e1484cf4e4 100644
--- a/OvmfPkg/ResetVector/Ia16/ResetVectorVtf0.asm
+++ b/OvmfPkg/ResetVector/Ia16/ResetVectorVtf0.asm
@@ -47,6 +47,29 @@ TIMES (15 - ((guidedStructureEnd - guidedStructureStart + 15) % 16)) DB 0
 ;
 guidedStructureStart:
 
+%ifdef ARCH_X64
+; SEV-SNP boot support
+;
+; sevSnpBlock:
+;   For the initial boot of SEV-SNP guest, a CPUID and Secrets page must
+;   be reserved by the BIOS at a RAM area defined by SNP_CPUID_BASE and
+;   SNP_SECRETS_BASE. A hypervisor will locate this information using the
+;   SEV-SNP boot block GUID and provide the GPA to the PSP to populate
+;   the memory area with the required information..
+;
+; GUID (SEV-SNP boot block): bd39c0c2-2f8e-4243-83e8-1b74cebcb7d9
+;
+sevSnpBootBlockStart:
+    DD      SNP_SECRETS_BASE
+    DD      SNP_SECRETS_SIZE
+    DD      SNP_CPUID_BASE
+    DD      SNP_CPUID_SIZE
+    DW      sevSnpBootBlockEnd - sevSnpBootBlockStart
+    DB      0xC2, 0xC0, 0x39, 0xBD, 0x8e, 0x2F, 0x43, 0x42
+    DB      0x83, 0xE8, 0x1B, 0x74, 0xCE, 0xBC, 0xB7, 0xD9
+sevSnpBootBlockEnd:
+%endif
+
 ; SEV Hash Table Block
 ;
 ; This describes the guest ram area where the hypervisor should
diff --git a/OvmfPkg/ResetVector/ResetVector.nasmb b/OvmfPkg/ResetVector/ResetVector.nasmb
index d1d800c56745..9be963206989 100644
--- a/OvmfPkg/ResetVector/ResetVector.nasmb
+++ b/OvmfPkg/ResetVector/ResetVector.nasmb
@@ -77,6 +77,10 @@
   %define SEV_ES_WORK_AREA_RDRAND (FixedPcdGet32 (PcdSevEsWorkAreaBase) + 8)
   %define SEV_ES_WORK_AREA_ENC_MASK (FixedPcdGet32 (PcdSevEsWorkAreaBase) + 16)
   %define SEV_ES_VC_TOP_OF_STACK (FixedPcdGet32 (PcdOvmfSecPeiTempRamBase) + FixedPcdGet32 (PcdOvmfSecPeiTempRamSize))
+  %define SNP_CPUID_BASE  FixedPcdGet32 (PcdOvmfSnpCpuidBase)
+  %define SNP_CPUID_SIZE  FixedPcdGet32 (PcdOvmfSnpCpuidSize)
+  %define SNP_SECRETS_BASE  FixedPcdGet32 (PcdOvmfSnpSecretsBase)
+  %define SNP_SECRETS_SIZE  FixedPcdGet32 (PcdOvmfSnpSecretsSize)
 %include "Ia32/Flat32ToFlat64.asm"
 %include "Ia32/AmdSev.asm"
 %include "Ia32/PageTables64.asm"
-- 
2.17.1

[PATCH v6 04/29] OvmfPkg/ResetVector: invalidate the GHCB page

[Brijesh Singh]

BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=3275

The GHCB page is part of a pre-validated memory range specified through
the SnpBootBlock GUID. When SEV-SNP is active, the GHCB page is
pre-validated by the hyperivosr during the SNP guest creation. On boot,
the reset vector maps the GHCB page as un-encrypted in the initial page
table. Just clearing the encryption attribute from the page table is not
enough. To maintain the security guarantees, the page must be invalidated.

The page invalidation consists of two steps:

1. Use the PVALIDATE instruction to clear Validated Bit from the RMP table.
2. Use the Page State Change VMGEXIT to ask hypervisor to change the page
   state to shared in the RMP table.

Cc: Michael Roth <michael.roth@amd.com>
Cc: James Bottomley <jejb@linux.ibm.com>
Cc: Min Xu <min.m.xu@intel.com>
Cc: Jiewen Yao <jiewen.yao@intel.com>
Cc: Tom Lendacky <thomas.lendacky@amd.com>
Cc: Jordan Justen <jordan.l.justen@intel.com>
Cc: Ard Biesheuvel <ardb+tianocore@kernel.org>
Cc: Erdem Aktas <erdemaktas@google.com>
Signed-off-by: Brijesh Singh <brijesh.singh@amd.com>
---
 OvmfPkg/ResetVector/Ia32/AmdSev.asm | 137 ++++++++++++++++++++++++++++
 1 file changed, 137 insertions(+)

diff --git a/OvmfPkg/ResetVector/Ia32/AmdSev.asm b/OvmfPkg/ResetVector/Ia32/AmdSev.asm
index 250ac8d8b180..0ac78c73c370 100644
--- a/OvmfPkg/ResetVector/Ia32/AmdSev.asm
+++ b/OvmfPkg/ResetVector/Ia32/AmdSev.asm
@@ -7,6 +7,8 @@
 ;
 ;------------------------------------------------------------------------------
 
+%include "Nasm.inc"
+
 BITS    32
 
 ;
@@ -65,6 +67,25 @@ BITS    32
                        PAGE_READ_WRITE + \
                        PAGE_PRESENT)
 
+; SNP page state change failure
+%define TERM_PAGE_STATE_CHANAGE     3
+
+; Hypervisor does not support SEV-SNP feature
+%define TERM_HV_UNSUPPORTED_FEATURE 4
+
+; GHCB SEV Information MSR protocol
+%define GHCB_SEV_INFORMATION_REQUEST        2
+%define GHCB_SEV_INFORMATION_RESPONSE       1
+
+; GHCB Page Invalidate request and response protocol values
+;
+%define GHCB_PAGE_STATE_CHANGE_REQUEST      20
+%define GHCB_PAGE_STATE_CHANGE_RESPONSE     21
+%define GHCB_PAGE_STATE_SHARED              2
+
+; GHCB Hypervisor features MSR protocol
+%define GHCB_HYPERVISOR_FEATURES_REQUEST    128
+%define GHCB_HYPERVISOR_FEATURES_RESPONSE   129
 
 ; Macro is used to issue the MSR protocol based VMGEXIT. The caller is
 ; responsible to populate values in the EDX:EAX registers. After the vmmcall
@@ -183,6 +204,19 @@ clearGhcbMemoryLoop:
     mov     dword[ecx * 4 + GHCB_BASE - 4], eax
     loop    clearGhcbMemoryLoop
 
+    ;
+    ; The page table built above cleared the memory encryption mask from the
+    ; GHCB_BASE (aka made it shared). When SEV-SNP is enabled, to maintain
+    ; the security guarantees, the page state transition from private to
+    ; shared must go through the page invalidation steps. Invalidate the
+    ; memory range before loading the page table below.
+    ;
+    ; NOTE: the invalidation must happen after zeroing the GHCB memory. This
+    ;       is because, in the 32-bit mode all the access are considered private.
+    ;       The invalidation before the zero'ing will cause a #VC.
+    ;
+    OneTimeCall  InvalidateGHCBPage
+
 SevClearPageEncMaskForGhcbPageExit:
     OneTimeCallRet SevClearPageEncMaskForGhcbPage
 
@@ -334,6 +368,109 @@ SevExit:
 
     OneTimeCallRet CheckSevFeatures
 
+; The version 2 of GHCB specification added the support to query the hypervisor
+; features. If the GHCB version is >=2 then read the hypervisor features and
+; verify that SEV-SNP feature is supported.
+;
+CheckSnpHypervisorFeatures:
+    ; Get the SEV Information
+    xor     eax, eax
+    xor     edx, edx
+
+    VmgExit GHCB_SEV_INFORMATION_REQUEST, GHCB_SEV_INFORMATION_RESPONSE
+
+    ;
+    ; SEV Information Response GHCB MSR
+    ;   GHCB_MSR[63:48] = Maximum protocol version
+    ;   GHCB_MSR[47:32] = Minimum protocol version
+    ;
+    shr     edx, 16
+    cmp     edx, 2
+    jl      SevSnpUnsupportedFeature
+
+    ; Get the hypervisor features
+    xor     eax, eax
+    xor     edx, edx
+
+    VmgExit GHCB_HYPERVISOR_FEATURES_REQUEST, GHCB_HYPERVISOR_FEATURES_RESPONSE
+
+    ;
+    ; Hypervisor features reponse
+    ;   GHCB_MSR[63:12] = Features bitmap
+    ;       BIT0        = SEV-SNP Supported
+    ;
+    shr     eax, 12
+    bt      eax, 0
+    jnc     SevSnpUnsupportedFeature
+
+CheckSnpHypervisorFeaturesDone:
+    OneTimeCallRet CheckSnpHypervisorFeatures
+
+; If its an SEV-SNP guest then use the page state change VMGEXIT to invalidate
+; the GHCB page.
+;
+; Modified:  EAX, EBX, ECX, EDX
+;
+InvalidateGHCBPage:
+    ; Check if SEV-SNP is enabled
+    ;  MSR_0xC0010131 - Bit 2 (SEV-SNP enabled)
+    mov       ecx, SEV_STATUS_MSR
+    rdmsr
+    bt        eax, 2
+    jnc       InvalidateGHCBPageDone
+
+    ; Verify that SEV-SNP feature is supported by the hypervisor.
+    OneTimeCall   CheckSnpHypervisorFeatures
+
+    ; Use PVALIDATE instruction to invalidate the page
+    mov     eax, GHCB_BASE
+    mov     ecx, 0
+    mov     edx, 0
+    PVALIDATE
+
+    ; Save the carry flag to be use later.
+    setc    dl
+
+    ; If PVALIDATE fail then abort the launch.
+    cmp     eax, 0
+    jne     SevSnpPageStateFailureTerminate
+
+    ; Check the carry flag to determine if RMP entry was updated.
+    cmp     dl, 0
+    jne     SevSnpPageStateFailureTerminate
+
+    ; Ask hypervisor to change the page state to shared using the
+    ; Page State Change VMGEXIT.
+    ;
+    ; Setup GHCB MSR
+    ;   GHCB_MSR[55:52] = Page Operation
+    ;   GHCB_MSR[51:12] = Guest Physical Frame Number
+    ;
+    mov     eax, (GHCB_BASE >> 12)
+    shl     eax, 12
+    mov     edx, (GHCB_PAGE_STATE_SHARED << 20)
+
+    VmgExit  GHCB_PAGE_STATE_CHANGE_REQUEST, GHCB_PAGE_STATE_CHANGE_RESPONSE
+
+    ;
+    ; Response GHCB MSR
+    ;   GHCB_MSR[63:12] = Error code
+    ;
+    cmp     edx, 0
+    jnz     SevSnpPageStateFailureTerminate
+
+InvalidateGHCBPageDone:
+    OneTimeCallRet InvalidateGHCBPage
+
+; Terminate the SEV-SNP guest due to the page state change failure
+SevSnpPageStateFailureTerminate:
+    TerminateVmgExit   TERM_PAGE_STATE_CHANAGE
+
+; Terminate the SEV-SNP guest because hypervisor does not support
+; the SEV-SNP feature
+SevSnpUnsupportedFeature:
+    TerminateVmgExit   TERM_HV_UNSUPPORTED_FEATURE
+
 ; Start of #VC exception handling routines
 ;
 
-- 
2.17.1

[PATCH v6 05/29] OvmfPkg/ResetVector: check the vmpl level

[Brijesh Singh]

BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=3275

Virtual Machine Privilege Level (VMPL) is an optional feature in the
SEV-SNP architecture, which allows a guest VM to divide its address space
into four levels. The level can be used to provide the hardware isolated
abstraction layers with a VM. The VMPL0 is the highest privilege, and
VMPL3 is the least privilege. Certain operations must be done by the VMPL0
software, such as:

* Validate or invalidate memory range (PVALIDATE instruction)
* Allocate VMSA page (RMPADJUST instruction when VMSA=1)

The initial SEV-SNP support assumes that it's running on VMPL0. Let's add
a check to make sure that we are running at VMPL0 before continuing the
boot. There is no easy method to query the current VMPL level. One simple
approach is to call PVALIDATE instruction and if the instruction causes
a #GP then its SEV-SNP guest is not booted under VMPL0. See the AMD APL
volume 3 (PVALIDATE) for additional information on the PVALIDATE.

Cc: Michael Roth <michael.roth@amd.com>
Cc: James Bottomley <jejb@linux.ibm.com>
Cc: Min Xu <min.m.xu@intel.com>
Cc: Jiewen Yao <jiewen.yao@intel.com>
Cc: Tom Lendacky <thomas.lendacky@amd.com>
Cc: Jordan Justen <jordan.l.justen@intel.com>
Cc: Ard Biesheuvel <ardb+tianocore@kernel.org>
Cc: Erdem Aktas <erdemaktas@google.com>
Signed-off-by: Brijesh Singh <brijesh.singh@amd.com>
---
 OvmfPkg/ResetVector/Ia32/AmdSev.asm | 90 ++++++++++++++++++++++++++++-
 1 file changed, 88 insertions(+), 2 deletions(-)

diff --git a/OvmfPkg/ResetVector/Ia32/AmdSev.asm b/OvmfPkg/ResetVector/Ia32/AmdSev.asm
index 0ac78c73c370..2386b15c0ce0 100644
--- a/OvmfPkg/ResetVector/Ia32/AmdSev.asm
+++ b/OvmfPkg/ResetVector/Ia32/AmdSev.asm
@@ -73,6 +73,12 @@ BITS    32
 ; Hypervisor does not support SEV-SNP feature
 %define TERM_HV_UNSUPPORTED_FEATURE 4
 
+; SEV-SNP guest is not launched at VMPL-0
+%define TERM_SNP_NOT_VMPL0          5
+
+; The #VC is not for PVALIDATE
+%define TERM_VC_NOT_PVALIDATE       6
+
 ; GHCB SEV Information MSR protocol
 %define GHCB_SEV_INFORMATION_REQUEST        2
 %define GHCB_SEV_INFORMATION_RESPONSE       1
@@ -236,6 +242,25 @@ GetSevCBitMaskAbove31:
 GetSevCBitMaskAbove31Exit:
     OneTimeCallRet GetSevCBitMaskAbove31
 
+; Check whether we're booted under the VMPL-0.
+;
+; There is no straightforward way to query the current VMPL level. The simplest
+; method is to use the PVALIDATE instruction to change the page state. If its
+; not a VMPL-0 guest then PVALIDATE will cause #GP.
+;
+CheckSnpVmpl0:
+    ; This routine is part of the ROM, and should have been validated by the SNP
+    ; guest launch sequence. So its safe to re-validate the page containing
+    ; this routine.
+    mov     eax, ADDR_OF(CheckSnpVmpl0)
+    mov     ecx, 0
+    mov     edx, 1
+    PVALIDATE
+
+    ; We will reach here only if we are running at VMPL-0.
+
+    OneTimeCallRet    CheckSnpVmpl0
+
 ; Check if Secure Encrypted Virtualization (SEV) features are enabled.
 ;
 ; Register usage is tight in this routine, so multiple calls for the
@@ -293,6 +318,17 @@ CheckSevFeatures:
     ; Set the work area header to indicate that the SEV is enabled
     mov     byte[WORK_AREA_GUEST_TYPE], 1
 
+    ; Check if we're SEV-SNP guest and booted under VMPL-0.
+    ;
+    ; This check should happen here because the PVALIDATE instruction
+    ; used in the check will cause an exception. The IDT is active
+    ; during the CheckSevFeatures only.
+    ;
+    bt        eax, 2
+    jnc       SkipCheckSnpVmpl0
+    OneTimeCall     CheckSnpVmpl0
+
+SkipCheckSnpVmpl0:
     ; Check for SEV-ES memory encryption feature:
     ; CPUID  Fn8000_001F[EAX] - Bit 3
     ;   CPUID raises a #VC exception if running as an SEV-ES guest
@@ -471,6 +507,37 @@ SevSnpPageStateFailureTerminate:
 SevSnpUnsupportedFeature:
     TerminateVmgExit   TERM_HV_UNSUPPORTED_FEATURE
 
+
+; Start handling of #GP exception handling routines
+;
+SevEsIdtNotPvalidate:
+    TerminateVmgExit TERM_VC_NOT_PVALIDATE
+    iret
+
+SevSnpGpException:
+    ;
+    ; If we're here, then its an SEV-SNP guest and it was due to
+    ; PVALIDATE instruction.
+    ;
+    ; Verify that its an PVALIDATE instruction
+    ; The exception stack looks like this:
+    ;     +---------+
+    ;     | ....    |
+    ;     | eip     |
+    ;     | err code|
+    ;     +---------+
+    pop     ebx
+    pop     ebx
+    mov     ecx, [ebx]
+    cmp     ecx, 0xff010ff2       ; Compare EIP with PVALIDATE menomics
+    jne     SevEsIdtNotPvalidate
+
+    ; The #GP was triggered by the PVALIDATE instruction, this will happen
+    ; only when we're not running at VMPL-0
+    ;
+    TerminateVmgExit TERM_SNP_NOT_VMPL0
+    iret
+
 ; Start of #VC exception handling routines
 ;
 
@@ -600,15 +667,34 @@ ALIGN   16
 ;
 IDT_BASE:
 ;
-; Vectors 0 - 28 (No handlers)
+; Vectors 0 - 12 (No handlers)
 ;
-%rep 29
+%rep 13
     dw      0                                    ; Offset low bits 15..0
     dw      0x10                                 ; Selector
     db      0                                    ; Reserved
     db      0x8E                                 ; Gate Type (IA32_IDT_GATE_TYPE_INTERRUPT_32)
     dw      0                                    ; Offset high bits 31..16
 %endrep
+;
+; Vector 13 (GP Exception)
+;
+    dw      (ADDR_OF(SevSnpGpException) & 0xffff)  ; Offset low bits 15..0
+    dw      0x10                                 ; Selector
+    db      0                                    ; Reserved
+    db      0x8E                                 ; Gate Type (IA32_IDT_GATE_TYPE_INTERRUPT_32)
+    dw      (ADDR_OF(SevSnpGpException) >> 16)   ; Offset high bits 31..16
+;
+; Vectors 14 - 28 (No handlers)
+;
+%rep 15
+    dw      0                                    ; Offset low bits 15..0
+    dw      0x10                                 ; Selector
+    db      0                                    ; Reserved
+    db      0x8E                                 ; Gate Type (IA32_IDT_GATE_TYPE_INTERRUPT_32)
+    dw      0                                    ; Offset high bits 31..16
+%endrep
+
 ;
 ; Vector 29 (VMM Communication Exception)
 ;
-- 
2.17.1

[PATCH v6 06/29] OvmfPkg/ResetVector: pre-validate the data pages used in SEC phase

[Brijesh Singh]

BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=3275

An SEV-SNP guest requires that private memory (aka pages mapped encrypted)
must be validated before being accessed.

The validation process consist of the following sequence:

1) Set the memory encryption attribute in the page table (aka C-bit).
   Note: If the processor is in non-PAE mode, then all the memory accesses
   are considered private.
2) Add the memory range as private in the RMP table. This can be performed
   using the Page State Change VMGEXIT defined in the GHCB specification.
3) Use the PVALIDATE instruction to set the Validated Bit in the RMP table.

During the guest creation time, the VMM encrypts the OVMF_CODE.fd using
the SEV-SNP firmware provided LAUNCH_UPDATE_DATA command. In addition to
encrypting the content, the command also validates the memory region.
This allows us to execute the code without going through the validation
sequence.

During execution, the reset vector need to access some data pages
(such as page tables, SevESWorkarea, Sec stack). The data pages are
accessed as private memory. The data pages are not part of the
OVMF_CODE.fd, so they were not validated during the guest creation.

There are two approaches we can take to validate the data pages before
the access:

a) Enhance the OVMF reset vector code to validate the pages as described
   above (go through step 2 - 3).
OR
b) Validate the pages during the guest creation time. The SEV firmware
   provides a command which can be used by the VMM to validate the pages
   without affecting the measurement of the launch.

Approach #b seems much simpler; it does not require any changes to the
OVMF reset vector code.

Cc: Michael Roth <michael.roth@amd.com>
Cc: James Bottomley <jejb@linux.ibm.com>
Cc: Min Xu <min.m.xu@intel.com>
Cc: Jiewen Yao <jiewen.yao@intel.com>
Cc: Tom Lendacky <thomas.lendacky@amd.com>
Cc: Jordan Justen <jordan.l.justen@intel.com>
Cc: Ard Biesheuvel <ardb+tianocore@kernel.org>
Cc: Erdem Aktas <erdemaktas@google.com>
Signed-off-by: Brijesh Singh <brijesh.singh@amd.com>
---
 OvmfPkg/OvmfPkg.dec                          | 5 +++++
 OvmfPkg/OvmfPkgX64.fdf                       | 6 ++++--
 OvmfPkg/ResetVector/ResetVector.inf          | 2 ++
 OvmfPkg/ResetVector/Ia16/ResetVectorVtf0.asm | 5 +++++
 OvmfPkg/ResetVector/ResetVector.nasmb        | 2 ++
 5 files changed, 18 insertions(+), 2 deletions(-)

diff --git a/OvmfPkg/OvmfPkg.dec b/OvmfPkg/OvmfPkg.dec
index afe9b7135560..41fffd6d3bd9 100644
--- a/OvmfPkg/OvmfPkg.dec
+++ b/OvmfPkg/OvmfPkg.dec
@@ -353,6 +353,11 @@ [PcdsFixedAtBuild]
   gUefiOvmfPkgTokenSpaceGuid.PcdOvmfSnpCpuidBase|0|UINT32|0x54
   gUefiOvmfPkgTokenSpaceGuid.PcdOvmfSnpCpuidSize|0|UINT32|0x55
 
+  ## The start and end of pre-validated memory region by the hypervisor
+  #  through the SEV-SNP firmware.
+  gUefiOvmfPkgTokenSpaceGuid.PcdOvmfSnpHypervisorPreValidatedStart|0x0|UINT32|0x56
+  gUefiOvmfPkgTokenSpaceGuid.PcdOvmfSnpHypervisorPreValidatedEnd|0x0|UINT32|0x57
+
 [PcdsDynamic, PcdsDynamicEx]
   gUefiOvmfPkgTokenSpaceGuid.PcdEmuVariableEvent|0|UINT64|2
   gUefiOvmfPkgTokenSpaceGuid.PcdOvmfFlashVariablesEnable|FALSE|BOOLEAN|0x10
diff --git a/OvmfPkg/OvmfPkgX64.fdf b/OvmfPkg/OvmfPkgX64.fdf
index 1e292d11ace3..10e8ba758a29 100644
--- a/OvmfPkg/OvmfPkgX64.fdf
+++ b/OvmfPkg/OvmfPkgX64.fdf
@@ -110,9 +110,11 @@ [FD.MEMFD]
 #
 SET gUefiCpuPkgTokenSpaceGuid.PcdSevEsWorkAreaBase = $(MEMFD_BASE_ADDRESS) +  gUefiOvmfPkgTokenSpaceGuid.PcdOvmfWorkAreaBase + gUefiOvmfPkgTokenSpaceGuid.PcdOvmfConfidentialComputingWorkAreaHeader
 SET gUefiCpuPkgTokenSpaceGuid.PcdSevEsWorkAreaSize = gUefiOvmfPkgTokenSpaceGuid.PcdOvmfWorkAreaSize - gUefiOvmfPkgTokenSpaceGuid.PcdOvmfConfidentialComputingWorkAreaHeader
-##########################################################################################
 
-################################################################################
+# The range of the pages pre-validated through the SEV-SNP firmware.
+SET gUefiOvmfPkgTokenSpaceGuid.PcdOvmfSnpHypervisorPreValidatedStart = $(MEMFD_BASE_ADDRESS) + gUefiOvmfPkgTokenSpaceGuid.PcdOvmfSecPageTablesBase
+SET gUefiOvmfPkgTokenSpaceGuid.PcdOvmfSnpHypervisorPreValidatedEnd = $(MEMFD_BASE_ADDRESS) + gUefiOvmfPkgTokenSpaceGuid.PcdOvmfPeiMemFvBase
+##########################################################################################
 
 [FV.SECFV]
 FvNameGuid         = 763BED0D-DE9F-48F5-81F1-3E90E1B1A015
diff --git a/OvmfPkg/ResetVector/ResetVector.inf b/OvmfPkg/ResetVector/ResetVector.inf
index 34d843de62c4..307699d8bf22 100644
--- a/OvmfPkg/ResetVector/ResetVector.inf
+++ b/OvmfPkg/ResetVector/ResetVector.inf
@@ -54,3 +54,5 @@ [FixedPcd]
   gUefiOvmfPkgTokenSpaceGuid.PcdOvmfSnpSecretsSize
   gUefiOvmfPkgTokenSpaceGuid.PcdOvmfSnpCpuidBase
   gUefiOvmfPkgTokenSpaceGuid.PcdOvmfSnpCpuidSize
+  gUefiOvmfPkgTokenSpaceGuid.PcdOvmfSnpHypervisorPreValidatedStart
+  gUefiOvmfPkgTokenSpaceGuid.PcdOvmfSnpHypervisorPreValidatedEnd
diff --git a/OvmfPkg/ResetVector/Ia16/ResetVectorVtf0.asm b/OvmfPkg/ResetVector/Ia16/ResetVectorVtf0.asm
index 71e1484cf4e4..a45e828a04a5 100644
--- a/OvmfPkg/ResetVector/Ia16/ResetVectorVtf0.asm
+++ b/OvmfPkg/ResetVector/Ia16/ResetVectorVtf0.asm
@@ -57,9 +57,14 @@ guidedStructureStart:
 ;   SEV-SNP boot block GUID and provide the GPA to the PSP to populate
 ;   the memory area with the required information..
 ;
+;   In order to boot the SEV-SNP guest the hypervisor must pre-validated the
+;   memory range from SNP_HV_VALIDATED_START to SNP_HV_VALIDATED_END.
+;
 ; GUID (SEV-SNP boot block): bd39c0c2-2f8e-4243-83e8-1b74cebcb7d9
 ;
 sevSnpBootBlockStart:
+    DD      SNP_HV_VALIDATED_START
+    DD      SNP_HV_VALIDATED_END
     DD      SNP_SECRETS_BASE
     DD      SNP_SECRETS_SIZE
     DD      SNP_CPUID_BASE
diff --git a/OvmfPkg/ResetVector/ResetVector.nasmb b/OvmfPkg/ResetVector/ResetVector.nasmb
index 9be963206989..75a38697d5d8 100644
--- a/OvmfPkg/ResetVector/ResetVector.nasmb
+++ b/OvmfPkg/ResetVector/ResetVector.nasmb
@@ -81,6 +81,8 @@
   %define SNP_CPUID_SIZE  FixedPcdGet32 (PcdOvmfSnpCpuidSize)
   %define SNP_SECRETS_BASE  FixedPcdGet32 (PcdOvmfSnpSecretsBase)
   %define SNP_SECRETS_SIZE  FixedPcdGet32 (PcdOvmfSnpSecretsSize)
+  %define SNP_HV_VALIDATED_START FixedPcdGet32 (PcdOvmfSnpHypervisorPreValidatedStart)
+  %define SNP_HV_VALIDATED_END FixedPcdGet32 (PcdOvmfSnpHypervisorPreValidatedEnd)
 %include "Ia32/Flat32ToFlat64.asm"
 %include "Ia32/AmdSev.asm"
 %include "Ia32/PageTables64.asm"
-- 
2.17.1

[PATCH v6 07/29] OvmfPkg/ResetVector: use SEV-SNP-validated CPUID values

[Brijesh Singh]

From: Michael Roth <michael.roth@amd.com>

CPUID instructions are issued during early boot to do things like probe
for SEV support. Currently these are handled by a minimal #VC handler
that uses the MSR-based GHCB protocol to fetch the CPUID values from
the hypervisor. When SEV-SNP is enabled, use the firmware-validated
CPUID values from the CPUID page instead [1].

[1]: SEV SNP Firmware ABI Specification, Rev. 0.8, 8.13.2.6

Cc: Michael Roth <michael.roth@amd.com>
Cc: James Bottomley <jejb@linux.ibm.com>
Cc: Min Xu <min.m.xu@intel.com>
Cc: Jiewen Yao <jiewen.yao@intel.com>
Cc: Tom Lendacky <thomas.lendacky@amd.com>
Cc: Jordan Justen <jordan.l.justen@intel.com>
Cc: Ard Biesheuvel <ardb+tianocore@kernel.org>
Cc: Erdem Aktas <erdemaktas@google.com>
Signed-off-by: Michael Roth <michael.roth@amd.com>
Signed-off-by: Brijesh Singh <brijesh.singh@amd.com>
---
 OvmfPkg/ResetVector/Ia32/AmdSev.asm | 80 +++++++++++++++++++++++++++--
 1 file changed, 75 insertions(+), 5 deletions(-)

diff --git a/OvmfPkg/ResetVector/Ia32/AmdSev.asm b/OvmfPkg/ResetVector/Ia32/AmdSev.asm
index 2386b15c0ce0..3ce3dd2785cb 100644
--- a/OvmfPkg/ResetVector/Ia32/AmdSev.asm
+++ b/OvmfPkg/ResetVector/Ia32/AmdSev.asm
@@ -36,6 +36,18 @@ BITS    32
 %define GHCB_CPUID_REGISTER_SHIFT  30
 %define CPUID_INSN_LEN              2
 
+; #VC handler offsets/sizes for accessing SNP CPUID page
+;
+%define SNP_CPUID_ENTRY_SZ         48
+%define SNP_CPUID_COUNT             0
+%define SNP_CPUID_ENTRY            16
+%define SNP_CPUID_ENTRY_EAX_IN      0
+%define SNP_CPUID_ENTRY_ECX_IN      4
+%define SNP_CPUID_ENTRY_EAX        24
+%define SNP_CPUID_ENTRY_EBX        28
+%define SNP_CPUID_ENTRY_ECX        32
+%define SNP_CPUID_ENTRY_EDX        36
+
 
 %define SEV_GHCB_MSR                0xc0010130
 %define SEV_STATUS_MSR              0xc0010131
@@ -545,11 +557,61 @@ SevEsIdtNotCpuid:
     TerminateVmgExit TERM_VC_NOT_CPUID
     iret
 
-    ;
-    ; Total stack usage for the #VC handler is 44 bytes:
-    ;   - 12 bytes for the exception IRET (after popping error code)
-    ;   - 32 bytes for the local variables.
-    ;
+; Use the SNP CPUID page to handle the cpuid lookup
+;
+;  Modified: EAX, EBX, ECX, EDX
+;
+;  Relies on the stack setup/usage in #VC handler:
+;
+;    On entry,
+;      [esp + VC_CPUID_FUNCTION] contains EAX input to cpuid instruction
+;
+;    On return, stores corresponding results of CPUID lookup in:
+;      [esp + VC_CPUID_RESULT_EAX]
+;      [esp + VC_CPUID_RESULT_EBX]
+;      [esp + VC_CPUID_RESULT_ECX]
+;      [esp + VC_CPUID_RESULT_EDX]
+;
+SnpCpuidLookup:
+    mov     eax, [esp + VC_CPUID_FUNCTION]
+    mov     ebx, [SNP_CPUID_BASE + SNP_CPUID_COUNT]
+    mov     ecx, SNP_CPUID_BASE + SNP_CPUID_ENTRY
+    ; Zero these out now so we can simply return if lookup fails
+    mov     dword[esp + VC_CPUID_RESULT_EAX], 0
+    mov     dword[esp + VC_CPUID_RESULT_EBX], 0
+    mov     dword[esp + VC_CPUID_RESULT_ECX], 0
+    mov     dword[esp + VC_CPUID_RESULT_EDX], 0
+
+SnpCpuidCheckEntry:
+    cmp     ebx, 0
+    je      VmmDoneSnpCpuid
+    cmp     dword[ecx + SNP_CPUID_ENTRY_EAX_IN], eax
+    jne     SnpCpuidCheckEntryNext
+    ; As with SEV-ES handler we assume requested CPUID sub-leaf/index is 0
+    cmp     dword[ecx + SNP_CPUID_ENTRY_ECX_IN], 0
+    je      SnpCpuidEntryFound
+
+SnpCpuidCheckEntryNext:
+    dec     ebx
+    add     ecx, SNP_CPUID_ENTRY_SZ
+    jmp     SnpCpuidCheckEntry
+
+SnpCpuidEntryFound:
+    mov     eax, [ecx + SNP_CPUID_ENTRY_EAX]
+    mov     [esp + VC_CPUID_RESULT_EAX], eax
+    mov     eax, [ecx + SNP_CPUID_ENTRY_EBX]
+    mov     [esp + VC_CPUID_RESULT_EBX], eax
+    mov     eax, [ecx + SNP_CPUID_ENTRY_EDX]
+    mov     [esp + VC_CPUID_RESULT_ECX], eax
+    mov     eax, [ecx + SNP_CPUID_ENTRY_ECX]
+    mov     [esp + VC_CPUID_RESULT_EDX], eax
+    jmp     VmmDoneSnpCpuid
+
+;
+; Total stack usage for the #VC handler is 44 bytes:
+;   - 12 bytes for the exception IRET (after popping error code)
+;   - 32 bytes for the local variables.
+;
 SevEsIdtVmmComm:
     ;
     ; If we're here, then we are an SEV-ES guest and this
@@ -577,6 +639,13 @@ SevEsIdtVmmComm:
     ; Save the CPUID function being requested
     mov     [esp + VC_CPUID_FUNCTION], eax
 
+    ; If SEV-SNP is enabled, use the CPUID page to handle the CPUID
+    ; instruction.
+    mov     ecx, SEV_STATUS_MSR
+    rdmsr
+    bt      eax, 2
+    jc      SnpCpuidLookup
+
     ; The GHCB CPUID protocol uses the following mapping to request
     ; a specific register:
     ;   0 => EAX, 1 => EBX, 2 => ECX, 3 => EDX
@@ -634,6 +703,7 @@ VmmDone:
     mov     ecx, SEV_GHCB_MSR
     wrmsr
 
+VmmDoneSnpCpuid:
     mov     eax, [esp + VC_CPUID_RESULT_EAX]
     mov     ebx, [esp + VC_CPUID_RESULT_EBX]
     mov     ecx, [esp + VC_CPUID_RESULT_ECX]
-- 
2.17.1

[PATCH v6 08/29] UefiCpuPkg: Define the SEV-SNP specific dynamic PCDs

[Brijesh Singh]

BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=3275

Define the PCDs used by the MpInitLib while creating the AP when SEV-SNP
is active in the guest VM.

Cc: Michael Roth <michael.roth@amd.com>
Cc: Ray Ni <ray.ni@intel.com>
Cc: Rahul Kumar <rahul1.kumar@intel.com>
Cc: Eric Dong <eric.dong@intel.com>
Cc: James Bottomley <jejb@linux.ibm.com>
Cc: Min Xu <min.m.xu@intel.com>
Cc: Jiewen Yao <jiewen.yao@intel.com>
Cc: Tom Lendacky <thomas.lendacky@amd.com>
Cc: Jordan Justen <jordan.l.justen@intel.com>
Cc: Ard Biesheuvel <ardb+tianocore@kernel.org>
Cc: Erdem Aktas <erdemaktas@google.com>
Signed-off-by: Brijesh Singh <brijesh.singh@amd.com>
---
 UefiCpuPkg/UefiCpuPkg.dec | 11 +++++++++++
 1 file changed, 11 insertions(+)

diff --git a/UefiCpuPkg/UefiCpuPkg.dec b/UefiCpuPkg/UefiCpuPkg.dec
index 62acb291f309..0ec25871a50f 100644
--- a/UefiCpuPkg/UefiCpuPkg.dec
+++ b/UefiCpuPkg/UefiCpuPkg.dec
@@ -396,5 +396,16 @@ [PcdsDynamic, PcdsDynamicEx]
   # @Prompt SEV-ES Status
   gUefiCpuPkgTokenSpaceGuid.PcdSevEsIsEnabled|FALSE|BOOLEAN|0x60000016
 
+  ## This dynamic PCD indicates whether SEV-SNP is enabled
+  #   TRUE  - SEV-SNP is enabled
+  #   FALSE - SEV-SNP is not enabled
+  # @Prompt SEV-SNP Status
+  gUefiCpuPkgTokenSpaceGuid.PcdSevSnpIsEnabled|FALSE|BOOLEAN|0x60000017
+
+  ## This dynamic PCD contains the hypervisor features value obtained through the GHCB HYPERVISOR
+  #  features VMGEXIT defined in the version 2 of GHCB spec.
+  # @Prompt GHCB Hypervisor Features
+  gUefiCpuPkgTokenSpaceGuid.PcdGhcbHypervisorFeatures|0x0|UINT64|0x60000018
+
 [UserExtensions.TianoCore."ExtraFiles"]
   UefiCpuPkgExtra.uni
-- 
2.17.1

[PATCH v6 09/29] OvmfPkg/MemEncryptSevLib: add MemEncryptSevSnpEnabled()

[Brijesh Singh]

BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=3275

Create a function that can be used to determine if VM is running as an
SEV-SNP guest.

Cc: Michael Roth <michael.roth@amd.com>
Cc: James Bottomley <jejb@linux.ibm.com>
Cc: Min Xu <min.m.xu@intel.com>
Cc: Jiewen Yao <jiewen.yao@intel.com>
Cc: Tom Lendacky <thomas.lendacky@amd.com>
Cc: Jordan Justen <jordan.l.justen@intel.com>
Cc: Ard Biesheuvel <ardb+tianocore@kernel.org>
Cc: Erdem Aktas <erdemaktas@google.com>
Signed-off-by: Brijesh Singh <brijesh.singh@amd.com>
---
 OvmfPkg/Include/Library/MemEncryptSevLib.h    | 12 +++++++++
 .../DxeMemEncryptSevLibInternal.c             | 27 +++++++++++++++++++
 .../PeiMemEncryptSevLibInternal.c             | 27 +++++++++++++++++++
 .../SecMemEncryptSevLibInternal.c             | 19 +++++++++++++
 4 files changed, 85 insertions(+)

diff --git a/OvmfPkg/Include/Library/MemEncryptSevLib.h b/OvmfPkg/Include/Library/MemEncryptSevLib.h
index adc490e466ec..796de62ec2f8 100644
--- a/OvmfPkg/Include/Library/MemEncryptSevLib.h
+++ b/OvmfPkg/Include/Library/MemEncryptSevLib.h
@@ -47,6 +47,18 @@ typedef enum {
   MemEncryptSevAddressRangeError,
 } MEM_ENCRYPT_SEV_ADDRESS_RANGE_STATE;
 
+/**
+  Returns a boolean to indicate whether SEV-SNP is enabled
+
+  @retval TRUE           SEV-SNP is enabled
+  @retval FALSE          SEV-SNP is not enabled
+**/
+BOOLEAN
+EFIAPI
+MemEncryptSevSnpIsEnabled (
+  VOID
+  );
+
 /**
   Returns a boolean to indicate whether SEV-ES is enabled.
 
diff --git a/OvmfPkg/Library/BaseMemEncryptSevLib/DxeMemEncryptSevLibInternal.c b/OvmfPkg/Library/BaseMemEncryptSevLib/DxeMemEncryptSevLibInternal.c
index 2816f859a0c4..057129723824 100644
--- a/OvmfPkg/Library/BaseMemEncryptSevLib/DxeMemEncryptSevLibInternal.c
+++ b/OvmfPkg/Library/BaseMemEncryptSevLib/DxeMemEncryptSevLibInternal.c
@@ -19,6 +19,7 @@
 
 STATIC BOOLEAN mSevStatus = FALSE;
 STATIC BOOLEAN mSevEsStatus = FALSE;
+STATIC BOOLEAN mSevSnpStatus = FALSE;
 STATIC BOOLEAN mSevStatusChecked = FALSE;
 
 STATIC UINT64  mSevEncryptionMask = 0;
@@ -82,11 +83,37 @@ InternalMemEncryptSevStatus (
     if (Msr.Bits.SevEsBit) {
       mSevEsStatus = TRUE;
     }
+
+    //
+    // Check MSR_0xC0010131 Bit 2 (Sev-Snp Enabled)
+    //
+    if (Msr.Bits.SevSnpBit) {
+      mSevSnpStatus = TRUE;
+    }
   }
 
   mSevStatusChecked = TRUE;
 }
 
+/**
+  Returns a boolean to indicate whether SEV-SNP is enabled.
+
+  @retval TRUE           SEV-SNP is enabled
+  @retval FALSE          SEV-SNP is not enabled
+**/
+BOOLEAN
+EFIAPI
+MemEncryptSevSnpIsEnabled (
+  VOID
+  )
+{
+  if (!mSevStatusChecked) {
+    InternalMemEncryptSevStatus ();
+  }
+
+  return mSevSnpStatus;
+}
+
 /**
   Returns a boolean to indicate whether SEV-ES is enabled.
 
diff --git a/OvmfPkg/Library/BaseMemEncryptSevLib/PeiMemEncryptSevLibInternal.c b/OvmfPkg/Library/BaseMemEncryptSevLib/PeiMemEncryptSevLibInternal.c
index e2fd109d120f..b561f211f577 100644
--- a/OvmfPkg/Library/BaseMemEncryptSevLib/PeiMemEncryptSevLibInternal.c
+++ b/OvmfPkg/Library/BaseMemEncryptSevLib/PeiMemEncryptSevLibInternal.c
@@ -19,6 +19,7 @@
 
 STATIC BOOLEAN mSevStatus = FALSE;
 STATIC BOOLEAN mSevEsStatus = FALSE;
+STATIC BOOLEAN mSevSnpStatus = FALSE;
 STATIC BOOLEAN mSevStatusChecked = FALSE;
 
 STATIC UINT64  mSevEncryptionMask = 0;
@@ -82,11 +83,37 @@ InternalMemEncryptSevStatus (
     if (Msr.Bits.SevEsBit) {
       mSevEsStatus = TRUE;
     }
+
+    //
+    // Check MSR_0xC0010131 Bit 2 (Sev-Snp Enabled)
+    //
+    if (Msr.Bits.SevSnpBit) {
+      mSevSnpStatus = TRUE;
+    }
   }
 
   mSevStatusChecked = TRUE;
 }
 
+/**
+  Returns a boolean to indicate whether SEV-SNP is enabled.
+
+  @retval TRUE           SEV-SNP is enabled
+  @retval FALSE          SEV-SNP is not enabled
+**/
+BOOLEAN
+EFIAPI
+MemEncryptSevSnpIsEnabled (
+  VOID
+  )
+{
+  if (!mSevStatusChecked) {
+    InternalMemEncryptSevStatus ();
+  }
+
+  return mSevSnpStatus;
+}
+
 /**
   Returns a boolean to indicate whether SEV-ES is enabled.
 
diff --git a/OvmfPkg/Library/BaseMemEncryptSevLib/SecMemEncryptSevLibInternal.c b/OvmfPkg/Library/BaseMemEncryptSevLib/SecMemEncryptSevLibInternal.c
index 56d8f3f3183f..69852779e2ff 100644
--- a/OvmfPkg/Library/BaseMemEncryptSevLib/SecMemEncryptSevLibInternal.c
+++ b/OvmfPkg/Library/BaseMemEncryptSevLib/SecMemEncryptSevLibInternal.c
@@ -62,6 +62,25 @@ InternalMemEncryptSevStatus (
   return ReadSevMsr ? AsmReadMsr32 (MSR_SEV_STATUS) : 0;
 }
 
+/**
+  Returns a boolean to indicate whether SEV-SNP is enabled.
+
+  @retval TRUE           SEV-SNP is enabled
+  @retval FALSE          SEV-SNP is not enabled
+**/
+BOOLEAN
+EFIAPI
+MemEncryptSevSnpIsEnabled (
+  VOID
+  )
+{
+  MSR_SEV_STATUS_REGISTER           Msr;
+
+  Msr.Uint32 = InternalMemEncryptSevStatus ();
+
+  return Msr.Bits.SevSnpBit ? TRUE : FALSE;
+}
+
 /**
   Returns a boolean to indicate whether SEV-ES is enabled.
 
-- 
2.17.1

[PATCH v6 10/29] OvmfPkg/SecMain: move SEV specific routines in AmdSev.c

[Brijesh Singh]

Move all the SEV specific function in AmdSev.c.

No functional change intended.

Cc: Michael Roth <michael.roth@amd.com>
Cc: James Bottomley <jejb@linux.ibm.com>
Cc: Min Xu <min.m.xu@intel.com>
Cc: Jiewen Yao <jiewen.yao@intel.com>
Cc: Tom Lendacky <thomas.lendacky@amd.com>
Cc: Jordan Justen <jordan.l.justen@intel.com>
Cc: Ard Biesheuvel <ardb+tianocore@kernel.org>
Cc: Erdem Aktas <erdemaktas@google.com>
Signed-off-by: Brijesh Singh <brijesh.singh@amd.com>
---
 OvmfPkg/Sec/SecMain.inf |   1 +
 OvmfPkg/Sec/AmdSev.h    |  72 ++++++++++++++++++
 OvmfPkg/Sec/AmdSev.c    | 161 ++++++++++++++++++++++++++++++++++++++++
 OvmfPkg/Sec/SecMain.c   | 153 +-------------------------------------
 4 files changed, 236 insertions(+), 151 deletions(-)
 create mode 100644 OvmfPkg/Sec/AmdSev.h
 create mode 100644 OvmfPkg/Sec/AmdSev.c

diff --git a/OvmfPkg/Sec/SecMain.inf b/OvmfPkg/Sec/SecMain.inf
index ea4b9611f52d..9523a8ea6c8f 100644
--- a/OvmfPkg/Sec/SecMain.inf
+++ b/OvmfPkg/Sec/SecMain.inf
@@ -23,6 +23,7 @@ [Defines]
 
 [Sources]
   SecMain.c
+  AmdSev.c
 
 [Sources.IA32]
   Ia32/SecEntry.nasm
diff --git a/OvmfPkg/Sec/AmdSev.h b/OvmfPkg/Sec/AmdSev.h
new file mode 100644
index 000000000000..adad96d23189
--- /dev/null
+++ b/OvmfPkg/Sec/AmdSev.h
@@ -0,0 +1,72 @@
+/** @file
+  File defines the Sec routines for the AMD SEV
+
+  Copyright (c) 2021, Advanced Micro Devices, Inc. All rights reserved.<BR>
+
+  SPDX-License-Identifier: BSD-2-Clause-Patent
+
+**/
+
+#ifndef _AMD_SEV_SEC_INTERNAL_H__
+#define _AMD_SEV_SEC_INTERNAL_H__
+
+/**
+  Handle an SEV-ES/GHCB protocol check failure.
+
+  Notify the hypervisor using the VMGEXIT instruction that the SEV-ES guest
+  wishes to be terminated.
+
+  @param[in] ReasonCode  Reason code to provide to the hypervisor for the
+                         termination request.
+
+**/
+VOID
+SevEsProtocolFailure (
+  IN UINT8  ReasonCode
+  );
+
+
+/**
+  Validate the SEV-ES/GHCB protocol level.
+
+  Verify that the level of SEV-ES/GHCB protocol supported by the hypervisor
+  and the guest intersect. If they don't intersect, request termination.
+
+**/
+VOID
+SevEsProtocolCheck (
+  VOID
+  );
+
+/**
+ Determine if the SEV is active.
+
+ During the early booting, GuestType is set in the work area. Verify that it
+ is an SEV guest.
+
+ @retval TRUE   SEV is enabled
+ @retval FALSE  SEV is not enabled
+
+**/
+BOOLEAN
+IsSevGuest (
+  VOID
+  );
+
+/**
+  Determine if SEV-ES is active.
+
+  During early booting, SEV-ES support code will set a flag to indicate that
+  SEV-ES is enabled. Return the value of this flag as an indicator that SEV-ES
+  is enabled.
+
+  @retval TRUE   SEV-ES is enabled
+  @retval FALSE  SEV-ES is not enabled
+
+**/
+BOOLEAN
+SevEsIsEnabled (
+  VOID
+  );
+
+#endif
diff --git a/OvmfPkg/Sec/AmdSev.c b/OvmfPkg/Sec/AmdSev.c
new file mode 100644
index 000000000000..3b4adaae32c7
--- /dev/null
+++ b/OvmfPkg/Sec/AmdSev.c
@@ -0,0 +1,161 @@
+/** @file
+  File defines the Sec routines for the AMD SEV
+
+  Copyright (c) 2021, Advanced Micro Devices, Inc. All rights reserved.<BR>
+
+  SPDX-License-Identifier: BSD-2-Clause-Patent
+
+**/
+
+#include <Library/BaseLib.h>
+#include <Library/DebugLib.h>
+#include <Library/MemEncryptSevLib.h>
+#include <Library/BaseMemoryLib.h>
+#include <Register/Amd/Ghcb.h>
+#include <Register/Amd/Msr.h>
+
+#include "AmdSev.h"
+
+/**
+  Handle an SEV-ES/GHCB protocol check failure.
+
+  Notify the hypervisor using the VMGEXIT instruction that the SEV-ES guest
+  wishes to be terminated.
+
+  @param[in] ReasonCode  Reason code to provide to the hypervisor for the
+                         termination request.
+
+**/
+VOID
+SevEsProtocolFailure (
+  IN UINT8  ReasonCode
+  )
+{
+  MSR_SEV_ES_GHCB_REGISTER  Msr;
+
+  //
+  // Use the GHCB MSR Protocol to request termination by the hypervisor
+  //
+  Msr.GhcbPhysicalAddress = 0;
+  Msr.GhcbTerminate.Function = GHCB_INFO_TERMINATE_REQUEST;
+  Msr.GhcbTerminate.ReasonCodeSet = GHCB_TERMINATE_GHCB;
+  Msr.GhcbTerminate.ReasonCode = ReasonCode;
+  AsmWriteMsr64 (MSR_SEV_ES_GHCB, Msr.GhcbPhysicalAddress);
+
+  AsmVmgExit ();
+
+  ASSERT (FALSE);
+  CpuDeadLoop ();
+}
+
+/**
+  Validate the SEV-ES/GHCB protocol level.
+
+  Verify that the level of SEV-ES/GHCB protocol supported by the hypervisor
+  and the guest intersect. If they don't intersect, request termination.
+
+**/
+VOID
+SevEsProtocolCheck (
+  VOID
+  )
+{
+  MSR_SEV_ES_GHCB_REGISTER  Msr;
+  GHCB                      *Ghcb;
+
+  //
+  // Use the GHCB MSR Protocol to obtain the GHCB SEV-ES Information for
+  // protocol checking
+  //
+  Msr.GhcbPhysicalAddress = 0;
+  Msr.GhcbInfo.Function = GHCB_INFO_SEV_INFO_GET;
+  AsmWriteMsr64 (MSR_SEV_ES_GHCB, Msr.GhcbPhysicalAddress);
+
+  AsmVmgExit ();
+
+  Msr.GhcbPhysicalAddress = AsmReadMsr64 (MSR_SEV_ES_GHCB);
+
+  if (Msr.GhcbInfo.Function != GHCB_INFO_SEV_INFO) {
+    SevEsProtocolFailure (GHCB_TERMINATE_GHCB_GENERAL);
+  }
+
+  if (Msr.GhcbProtocol.SevEsProtocolMin > Msr.GhcbProtocol.SevEsProtocolMax) {
+    SevEsProtocolFailure (GHCB_TERMINATE_GHCB_PROTOCOL);
+  }
+
+  if ((Msr.GhcbProtocol.SevEsProtocolMin > GHCB_VERSION_MAX) ||
+      (Msr.GhcbProtocol.SevEsProtocolMax < GHCB_VERSION_MIN)) {
+    SevEsProtocolFailure (GHCB_TERMINATE_GHCB_PROTOCOL);
+  }
+
+  //
+  // SEV-ES protocol checking succeeded, set the initial GHCB address
+  //
+  Msr.GhcbPhysicalAddress = FixedPcdGet32 (PcdOvmfSecGhcbBase);
+  AsmWriteMsr64 (MSR_SEV_ES_GHCB, Msr.GhcbPhysicalAddress);
+
+  Ghcb = Msr.Ghcb;
+  SetMem (Ghcb, sizeof (*Ghcb), 0);
+
+  //
+  // Set the version to the maximum that can be supported
+  //
+  Ghcb->ProtocolVersion = MIN (Msr.GhcbProtocol.SevEsProtocolMax, GHCB_VERSION_MAX);
+  Ghcb->GhcbUsage = GHCB_STANDARD_USAGE;
+}
+
+/**
+ Determine if the SEV is active.
+
+ During the early booting, GuestType is set in the work area. Verify that it
+ is an SEV guest.
+
+ @retval TRUE   SEV is enabled
+ @retval FALSE  SEV is not enabled
+
+**/
+BOOLEAN
+IsSevGuest (
+  VOID
+  )
+{
+  OVMF_WORK_AREA             *WorkArea;
+
+  //
+  // Ensure that the size of the Confidential Computing work area header
+  // is same as what is provided through a fixed PCD.
+  //
+  ASSERT ((UINTN) FixedPcdGet32 (PcdOvmfConfidentialComputingWorkAreaHeader) ==
+          sizeof(CONFIDENTIAL_COMPUTING_WORK_AREA_HEADER));
+
+  WorkArea = (OVMF_WORK_AREA *) FixedPcdGet32 (PcdOvmfWorkAreaBase);
+
+  return ((WorkArea != NULL) && (WorkArea->Header.GuestType == GUEST_TYPE_AMD_SEV));
+}
+
+/**
+  Determine if SEV-ES is active.
+
+  During early booting, SEV-ES support code will set a flag to indicate that
+  SEV-ES is enabled. Return the value of this flag as an indicator that SEV-ES
+  is enabled.
+
+  @retval TRUE   SEV-ES is enabled
+  @retval FALSE  SEV-ES is not enabled
+
+**/
+BOOLEAN
+SevEsIsEnabled (
+  VOID
+  )
+{
+  SEC_SEV_ES_WORK_AREA  *SevEsWorkArea;
+
+  if (!IsSevGuest()) {
+    return FALSE;
+  }
+
+  SevEsWorkArea = (SEC_SEV_ES_WORK_AREA *) FixedPcdGet32 (PcdSevEsWorkAreaBase);
+
+  return (SevEsWorkArea->SevEsEnabled != 0);
+}
diff --git a/OvmfPkg/Sec/SecMain.c b/OvmfPkg/Sec/SecMain.c
index 707b0d4bbff4..406e3a25d0cd 100644
--- a/OvmfPkg/Sec/SecMain.c
+++ b/OvmfPkg/Sec/SecMain.c
@@ -26,12 +26,11 @@
 #include <Library/ExtractGuidedSectionLib.h>
 #include <Library/LocalApicLib.h>
 #include <Library/CpuExceptionHandlerLib.h>
-#include <Library/MemEncryptSevLib.h>
-#include <Register/Amd/Ghcb.h>
-#include <Register/Amd/Msr.h>
 
 #include <Ppi/TemporaryRamSupport.h>
 
+#include "AmdSev.h"
+
 #define SEC_IDT_ENTRY_COUNT  34
 
 typedef struct _SEC_IDT_TABLE {
@@ -717,154 +716,6 @@ FindAndReportEntryPoints (
   return;
 }
 
-/**
-  Handle an SEV-ES/GHCB protocol check failure.
-
-  Notify the hypervisor using the VMGEXIT instruction that the SEV-ES guest
-  wishes to be terminated.
-
-  @param[in] ReasonCode  Reason code to provide to the hypervisor for the
-                         termination request.
-
-**/
-STATIC
-VOID
-SevEsProtocolFailure (
-  IN UINT8  ReasonCode
-  )
-{
-  MSR_SEV_ES_GHCB_REGISTER  Msr;
-
-  //
-  // Use the GHCB MSR Protocol to request termination by the hypervisor
-  //
-  Msr.GhcbPhysicalAddress = 0;
-  Msr.GhcbTerminate.Function = GHCB_INFO_TERMINATE_REQUEST;
-  Msr.GhcbTerminate.ReasonCodeSet = GHCB_TERMINATE_GHCB;
-  Msr.GhcbTerminate.ReasonCode = ReasonCode;
-  AsmWriteMsr64 (MSR_SEV_ES_GHCB, Msr.GhcbPhysicalAddress);
-
-  AsmVmgExit ();
-
-  ASSERT (FALSE);
-  CpuDeadLoop ();
-}
-
-/**
-  Validate the SEV-ES/GHCB protocol level.
-
-  Verify that the level of SEV-ES/GHCB protocol supported by the hypervisor
-  and the guest intersect. If they don't intersect, request termination.
-
-**/
-STATIC
-VOID
-SevEsProtocolCheck (
-  VOID
-  )
-{
-  MSR_SEV_ES_GHCB_REGISTER  Msr;
-  GHCB                      *Ghcb;
-
-  //
-  // Use the GHCB MSR Protocol to obtain the GHCB SEV-ES Information for
-  // protocol checking
-  //
-  Msr.GhcbPhysicalAddress = 0;
-  Msr.GhcbInfo.Function = GHCB_INFO_SEV_INFO_GET;
-  AsmWriteMsr64 (MSR_SEV_ES_GHCB, Msr.GhcbPhysicalAddress);
-
-  AsmVmgExit ();
-
-  Msr.GhcbPhysicalAddress = AsmReadMsr64 (MSR_SEV_ES_GHCB);
-
-  if (Msr.GhcbInfo.Function != GHCB_INFO_SEV_INFO) {
-    SevEsProtocolFailure (GHCB_TERMINATE_GHCB_GENERAL);
-  }
-
-  if (Msr.GhcbProtocol.SevEsProtocolMin > Msr.GhcbProtocol.SevEsProtocolMax) {
-    SevEsProtocolFailure (GHCB_TERMINATE_GHCB_PROTOCOL);
-  }
-
-  if ((Msr.GhcbProtocol.SevEsProtocolMin > GHCB_VERSION_MAX) ||
-      (Msr.GhcbProtocol.SevEsProtocolMax < GHCB_VERSION_MIN)) {
-    SevEsProtocolFailure (GHCB_TERMINATE_GHCB_PROTOCOL);
-  }
-
-  //
-  // SEV-ES protocol checking succeeded, set the initial GHCB address
-  //
-  Msr.GhcbPhysicalAddress = FixedPcdGet32 (PcdOvmfSecGhcbBase);
-  AsmWriteMsr64 (MSR_SEV_ES_GHCB, Msr.GhcbPhysicalAddress);
-
-  Ghcb = Msr.Ghcb;
-  SetMem (Ghcb, sizeof (*Ghcb), 0);
-
-  //
-  // Set the version to the maximum that can be supported
-  //
-  Ghcb->ProtocolVersion = MIN (Msr.GhcbProtocol.SevEsProtocolMax, GHCB_VERSION_MAX);
-  Ghcb->GhcbUsage = GHCB_STANDARD_USAGE;
-}
-
-/**
- Determine if the SEV is active.
-
- During the early booting, GuestType is set in the work area. Verify that it
- is an SEV guest.
-
- @retval TRUE   SEV is enabled
- @retval FALSE  SEV is not enabled
-
-**/
-STATIC
-BOOLEAN
-IsSevGuest (
-  VOID
-  )
-{
-  OVMF_WORK_AREA             *WorkArea;
-
-  //
-  // Ensure that the size of the Confidential Computing work area header
-  // is same as what is provided through a fixed PCD.
-  //
-  ASSERT ((UINTN) FixedPcdGet32 (PcdOvmfConfidentialComputingWorkAreaHeader) ==
-          sizeof(CONFIDENTIAL_COMPUTING_WORK_AREA_HEADER));
-
-  WorkArea = (OVMF_WORK_AREA *) FixedPcdGet32 (PcdOvmfWorkAreaBase);
-
-  return ((WorkArea != NULL) && (WorkArea->Header.GuestType == GUEST_TYPE_AMD_SEV));
-}
-
-/**
-  Determine if SEV-ES is active.
-
-  During early booting, SEV-ES support code will set a flag to indicate that
-  SEV-ES is enabled. Return the value of this flag as an indicator that SEV-ES
-  is enabled.
-
-  @retval TRUE   SEV-ES is enabled
-  @retval FALSE  SEV-ES is not enabled
-
-**/
-STATIC
-BOOLEAN
-SevEsIsEnabled (
-  VOID
-  )
-{
-  SEC_SEV_ES_WORK_AREA  *SevEsWorkArea;
-
-  if (!IsSevGuest()) {
-    return FALSE;
-  }
-
-  SevEsWorkArea = (SEC_SEV_ES_WORK_AREA *) FixedPcdGet32 (PcdSevEsWorkAreaBase);
-
-  return (SevEsWorkArea->SevEsEnabled != 0);
-}
-
 VOID
 EFIAPI
 SecCoreStartupWithStack (
-- 
2.17.1

[PATCH v6 11/29] OvmfPkg/SecMain: register GHCB gpa for the SEV-SNP guest

[Brijesh Singh]

BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=3275

The SEV-SNP guest requires that GHCB GPA must be registered before using.
See the GHCB specification section 2.3.2 for more details.

Cc: Michael Roth <michael.roth@amd.com>
Cc: James Bottomley <jejb@linux.ibm.com>
Cc: Min Xu <min.m.xu@intel.com>
Cc: Jiewen Yao <jiewen.yao@intel.com>
Cc: Tom Lendacky <thomas.lendacky@amd.com>
Cc: Jordan Justen <jordan.l.justen@intel.com>
Cc: Ard Biesheuvel <ardb+tianocore@kernel.org>
Cc: Erdem Aktas <erdemaktas@google.com>
Signed-off-by: Brijesh Singh <brijesh.singh@amd.com>
---
 OvmfPkg/Sec/AmdSev.c | 88 ++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 88 insertions(+)

diff --git a/OvmfPkg/Sec/AmdSev.c b/OvmfPkg/Sec/AmdSev.c
index 3b4adaae32c7..054f19216f1e 100644
--- a/OvmfPkg/Sec/AmdSev.c
+++ b/OvmfPkg/Sec/AmdSev.c
@@ -48,6 +48,83 @@ SevEsProtocolFailure (
   CpuDeadLoop ();
 }
 
+/**
+  Determine if SEV-SNP is active.
+
+  @retval TRUE   SEV-SNP is enabled
+  @retval FALSE  SEV-SNP is not enabled
+
+**/
+STATIC
+BOOLEAN
+SevSnpIsEnabled (
+  VOID
+  )
+{
+  MSR_SEV_STATUS_REGISTER           Msr;
+
+  //
+  // Read the SEV_STATUS MSR to determine whether SEV-SNP is active.
+  //
+  Msr.Uint32 = AsmReadMsr32 (MSR_SEV_STATUS);
+
+  //
+  // Check MSR_0xC0010131 Bit 2 (Sev-Snp Enabled)
+  //
+  if (Msr.Bits.SevSnpBit) {
+    return TRUE;
+  }
+
+  return FALSE;
+}
+
+/**
+ Register the GHCB GPA
+
+*/
+STATIC
+VOID
+SevSnpGhcbRegister (
+  UINTN   Address
+  )
+{
+  MSR_SEV_ES_GHCB_REGISTER  Msr;
+  MSR_SEV_ES_GHCB_REGISTER  CurrentMsr;
+  EFI_PHYSICAL_ADDRESS      GuestFrameNumber;
+
+  GuestFrameNumber = Address >> EFI_PAGE_SHIFT;
+
+  //
+  // Save the current MSR Value
+  //
+  CurrentMsr.GhcbPhysicalAddress = AsmReadMsr64 (MSR_SEV_ES_GHCB);
+
+  //
+  // Use the GHCB MSR Protocol to request to register the GPA.
+  //
+  Msr.GhcbPhysicalAddress = 0;
+  Msr.GhcbGpaRegister.Function = GHCB_INFO_GHCB_GPA_REGISTER_REQUEST;
+  Msr.GhcbGpaRegister.GuestFrameNumber = GuestFrameNumber;
+  AsmWriteMsr64 (MSR_SEV_ES_GHCB, Msr.GhcbPhysicalAddress);
+
+  AsmVmgExit ();
+
+  Msr.GhcbPhysicalAddress = AsmReadMsr64 (MSR_SEV_ES_GHCB);
+
+  //
+  // If hypervisor responded with a different GPA than requested then fail.
+  //
+  if ((Msr.GhcbGpaRegister.Function != GHCB_INFO_GHCB_GPA_REGISTER_RESPONSE) ||
+      (Msr.GhcbGpaRegister.GuestFrameNumber != GuestFrameNumber)) {
+    SevEsProtocolFailure (GHCB_TERMINATE_GHCB_GENERAL);
+  }
+
+  //
+  // Restore the MSR
+  //
+  AsmWriteMsr64 (MSR_SEV_ES_GHCB, CurrentMsr.GhcbPhysicalAddress);
+}
+
 /**
   Validate the SEV-ES/GHCB protocol level.
 
@@ -88,6 +165,17 @@ SevEsProtocolCheck (
     SevEsProtocolFailure (GHCB_TERMINATE_GHCB_PROTOCOL);
   }
 
+  //
+  // We cannot use the MemEncryptSevSnpIsEnabled () because the
+  // ProcessLibraryConstructorList () is not called yet.
+  //
+  if (SevSnpIsEnabled ()) {
+    //
+    // SEV-SNP guest requires that GHCB GPA must be registered before using it.
+    //
+    SevSnpGhcbRegister (FixedPcdGet32 (PcdOvmfSecGhcbBase));
+  }
+
   //
   // SEV-ES protocol checking succeeded, set the initial GHCB address
   //
-- 
2.17.1

[PATCH v6 12/29] OvmfPkg/VmgExitLib: use SEV-SNP-validated CPUID values

[Brijesh Singh]

From: Michael Roth <michael.roth@amd.com>

SEV-SNP firmware allows a special guest page to be populated with
guest CPUID values so that they can be validated against supported
host features before being loaded into encrypted guest memory to be
used instead of hypervisor-provided values [1].

Add handling for this in the CPUID #VC handler and use it whenever
SEV-SNP is enabled. To do so, existing CPUID handling via VmgExit is
moved to a helper, GetCpuidHyp(), and a new helper that uses the CPUID
page to do the lookup, GetCpuidFw(), is used instead when SNP is
enabled. For cases where SNP CPUID lookups still rely on fetching
specific CPUID fields from hypervisor, GetCpuidHyp() is used there as
well.

[1]: SEV SNP Firmware ABI Specification, Rev. 0.8, 8.13.2.6

Cc: James Bottomley <jejb@linux.ibm.com>
Cc: Min Xu <min.m.xu@intel.com>
Cc: Jiewen Yao <jiewen.yao@intel.com>
Cc: Tom Lendacky <thomas.lendacky@amd.com>
Cc: Jordan Justen <jordan.l.justen@intel.com>
Cc: Ard Biesheuvel <ardb+tianocore@kernel.org>
Cc: Erdem Aktas <erdemaktas@google.com>
Signed-off-by: Michael Roth <michael.roth@amd.com>
Signed-off-by: Brijesh Singh <brijesh.singh@amd.com>
---
 OvmfPkg/Library/VmgExitLib/SecVmgExitLib.inf  |   2 +
 OvmfPkg/Library/VmgExitLib/VmgExitLib.inf     |   3 +
 OvmfPkg/Library/VmgExitLib/VmgExitVcHandler.c | 444 ++++++++++++++++--
 3 files changed, 419 insertions(+), 30 deletions(-)

diff --git a/OvmfPkg/Library/VmgExitLib/SecVmgExitLib.inf b/OvmfPkg/Library/VmgExitLib/SecVmgExitLib.inf
index e6f6ea7972fd..67cbe15afafe 100644
--- a/OvmfPkg/Library/VmgExitLib/SecVmgExitLib.inf
+++ b/OvmfPkg/Library/VmgExitLib/SecVmgExitLib.inf
@@ -42,4 +42,6 @@ [LibraryClasses]
 [FixedPcd]
   gUefiOvmfPkgTokenSpaceGuid.PcdOvmfSecGhcbBackupBase
   gUefiOvmfPkgTokenSpaceGuid.PcdOvmfSecGhcbBackupSize
+  gUefiOvmfPkgTokenSpaceGuid.PcdOvmfSnpCpuidBase
+  gUefiOvmfPkgTokenSpaceGuid.PcdOvmfSnpCpuidSize
 
diff --git a/OvmfPkg/Library/VmgExitLib/VmgExitLib.inf b/OvmfPkg/Library/VmgExitLib/VmgExitLib.inf
index c66c68726cdb..a81fb034ad2d 100644
--- a/OvmfPkg/Library/VmgExitLib/VmgExitLib.inf
+++ b/OvmfPkg/Library/VmgExitLib/VmgExitLib.inf
@@ -38,3 +38,6 @@ [LibraryClasses]
   LocalApicLib
   MemEncryptSevLib
 
+[Pcd]
+  gUefiOvmfPkgTokenSpaceGuid.PcdOvmfSnpCpuidBase
+  gUefiOvmfPkgTokenSpaceGuid.PcdOvmfSnpCpuidSize
diff --git a/OvmfPkg/Library/VmgExitLib/VmgExitVcHandler.c b/OvmfPkg/Library/VmgExitLib/VmgExitVcHandler.c
index 41b0c8cc5312..152d38c42cbe 100644
--- a/OvmfPkg/Library/VmgExitLib/VmgExitVcHandler.c
+++ b/OvmfPkg/Library/VmgExitLib/VmgExitVcHandler.c
@@ -17,6 +17,7 @@
 #include <IndustryStandard/InstructionParsing.h>
 
 #include "VmgExitVcHandler.h"
+//#include <Library/MemEncryptSevLib.h>
 
 //
 // Instruction execution mode definition
@@ -130,6 +131,32 @@ UINT64
   SEV_ES_INSTRUCTION_DATA  *InstructionData
   );
 
+//
+// SEV-SNP Cpuid table entry/function
+//
+typedef PACKED struct {
+  UINT32   EaxIn;
+  UINT32   EcxIn;
+  UINT64   Unused;
+  UINT64   Unused2;
+  UINT32   Eax;
+  UINT32   Ebx;
+  UINT32   Ecx;
+  UINT32   Edx;
+  UINT64   Reserved;
+} SEV_SNP_CPUID_FUNCTION;
+
+//
+// SEV-SNP Cpuid page format
+//
+typedef PACKED struct {
+  UINT32   Count;
+  UINT32   Reserved1;
+  UINT64   Reserved2;
+  SEV_SNP_CPUID_FUNCTION function[0];
+} SEV_SNP_CPUID_INFO;
+
+
 /**
   Return a pointer to the contents of the specified register.
 
@@ -1496,58 +1523,415 @@ InvdExit (
 }
 
 /**
-  Handle a CPUID event.
+  Fetch CPUID leaf/function via hypervisor/VMGEXIT.
 
-  Use the VMGEXIT instruction to handle a CPUID event.
+  @param[in, out] Ghcb         Pointer to the Guest-Hypervisor Communication
+                               Block
+  @param[in]      EaxIn        EAX input for cpuid instruction
+  @param[in]      EcxIn        ECX input for cpuid instruction
+  @param[in]      Xcr0In       XCR0 at time of cpuid instruction
+  @param[in, out] Eax          Pointer to store leaf's EAX value
+  @param[in, out] Ebx          Pointer to store leaf's EBX value
+  @param[in, out] Ecx          Pointer to store leaf's ECX value
+  @param[in, out] Edx          Pointer to store leaf's EDX value
+  @param[in, out] Status       Pointer to store status from VMGEXIT (always 0
+                               unless return value indicates failure)
+  @param[in, out] Unsupported  Pointer to store indication of unsupported
+                               VMGEXIT (always false unless return value
+                               indicates failure)
 
-  @param[in, out] Ghcb             Pointer to the Guest-Hypervisor Communication
-                                   Block
-  @param[in, out] Regs             x64 processor context
-  @param[in]      InstructionData  Instruction parsing context
-
-  @retval 0                        Event handled successfully
-  @return                          New exception value to propagate
+  @retval TRUE                 CPUID leaf fetch successfully.
+  @retval FALSE                Error occurred while fetching CPUID leaf. Callers
+                               should Status and Unsupported and handle
+                               accordingly if they indicate a more precise
+                               error condition.
 
 **/
 STATIC
-UINT64
-CpuidExit (
+BOOLEAN
+GetCpuidHyp (
   IN OUT GHCB                     *Ghcb,
-  IN OUT EFI_SYSTEM_CONTEXT_X64   *Regs,
-  IN     SEV_ES_INSTRUCTION_DATA  *InstructionData
+  IN     UINT32                   EaxIn,
+  IN     UINT32                   EcxIn,
+  IN     UINT64                   XCr0,
+  IN OUT UINT32                   *Eax,
+  IN OUT UINT32                   *Ebx,
+  IN OUT UINT32                   *Ecx,
+  IN OUT UINT32                   *Edx,
+  IN OUT UINT64                   *Status,
+  IN OUT BOOLEAN                  *UnsupportedExit
   )
 {
-  UINT64  Status;
-
-  Ghcb->SaveArea.Rax = Regs->Rax;
+  *UnsupportedExit = FALSE;
+  Ghcb->SaveArea.Rax = EaxIn;
   VmgSetOffsetValid (Ghcb, GhcbRax);
-  Ghcb->SaveArea.Rcx = Regs->Rcx;
+  Ghcb->SaveArea.Rcx = EcxIn;
   VmgSetOffsetValid (Ghcb, GhcbRcx);
-  if (Regs->Rax == CPUID_EXTENDED_STATE) {
-    IA32_CR4  Cr4;
-
-    Cr4.UintN = AsmReadCr4 ();
-    Ghcb->SaveArea.XCr0 = (Cr4.Bits.OSXSAVE == 1) ? AsmXGetBv (0) : 1;
+  if (EaxIn == CPUID_EXTENDED_STATE) {
+    Ghcb->SaveArea.XCr0 = XCr0;
     VmgSetOffsetValid (Ghcb, GhcbXCr0);
   }
 
-  Status = VmgExit (Ghcb, SVM_EXIT_CPUID, 0, 0);
-  if (Status != 0) {
-    return Status;
+  *Status = VmgExit (Ghcb, SVM_EXIT_CPUID, 0, 0);
+  if (*Status != 0) {
+    return FALSE;
   }
 
   if (!VmgIsOffsetValid (Ghcb, GhcbRax) ||
       !VmgIsOffsetValid (Ghcb, GhcbRbx) ||
       !VmgIsOffsetValid (Ghcb, GhcbRcx) ||
       !VmgIsOffsetValid (Ghcb, GhcbRdx)) {
-    return UnsupportedExit (Ghcb, Regs, InstructionData);
+    *UnsupportedExit = TRUE;
+    return FALSE;
   }
-  Regs->Rax = Ghcb->SaveArea.Rax;
-  Regs->Rbx = Ghcb->SaveArea.Rbx;
-  Regs->Rcx = Ghcb->SaveArea.Rcx;
-  Regs->Rdx = Ghcb->SaveArea.Rdx;
+
+  if (Eax) {
+    *Eax = Ghcb->SaveArea.Rax;
+  }
+  if (Ebx) {
+    *Ebx = Ghcb->SaveArea.Rbx;
+  }
+  if (Ecx) {
+    *Ecx = Ghcb->SaveArea.Rcx;
+  }
+  if (Edx) {
+    *Edx = Ghcb->SaveArea.Rdx;
+  }
+
+  return TRUE;
+}
+
+/**
+  Check if SEV-SNP enabled.
+
+  @retval TRUE      SEV-SNP is enabled.
+  @retval FALSE     SEV-SNP is disabled.
+
+**/
+STATIC
+BOOLEAN
+SnpEnabled (VOID)
+{
+  MSR_SEV_STATUS_REGISTER  Msr;
+
+  Msr.Uint32 = AsmReadMsr32(MSR_SEV_STATUS);
+
+  return !!Msr.Bits.SevSnpBit;
+}
+
+/**
+  Calculate the total XSAVE area size for enabled XSAVE areas
+
+  @param[in]      XFeaturesEnabled  Bit-mask of enabled XSAVE features/areas as
+                                    indicated by XCR0/MSR_IA32_XSS bits
+  @param[in]      XSaveBaseSize     Base/legacy XSAVE area size (e.g. when
+                                    XCR0 is 1)
+  @param[in, out] XSaveSize         Pointer to storage for calculated XSAVE area
+                                    size
+  @param[in]      Compacted         Whether or not the calculation is for the
+                                    normal XSAVE area size (leaf 0xD,0x0,EBX) or
+                                    compacted XSAVE area size (leaf 0xD,0x1,EBX)
+
+
+  @retval TRUE                      XSAVE size calculation was successful.
+  @retval FALSE                     XSAVE size calculation was unsuccessful.
+**/
+STATIC
+BOOLEAN
+GetCpuidXSaveSize (
+  IN     UINT64   XFeaturesEnabled,
+  IN     UINT32   XSaveBaseSize,
+  IN OUT UINT32   *XSaveSize,
+  IN     BOOLEAN  Compacted
+  )
+{
+  SEV_SNP_CPUID_INFO  *CpuidInfo;
+  UINT64              XFeaturesFound = 0;
+  UINT32              Idx;
+
+  *XSaveSize = XSaveBaseSize;
+  CpuidInfo = (SEV_SNP_CPUID_INFO *)(UINT64)PcdGet32(PcdOvmfSnpCpuidBase);
+
+  for (Idx = 0; Idx < CpuidInfo->Count; Idx++) {
+    SEV_SNP_CPUID_FUNCTION *CpuidFn = &CpuidInfo->function[Idx];
+
+    if (!(CpuidFn->EaxIn == 0xD &&
+          (CpuidFn->EcxIn == 0 || CpuidFn->EcxIn == 1))) {
+      continue;
+    }
+
+    if (XFeaturesFound & (1UL << CpuidFn->EcxIn) ||
+        !(XFeaturesEnabled & (1UL << CpuidFn->EcxIn))) {
+        continue;
+    }
+
+    XFeaturesFound |= (1UL << CpuidFn->EcxIn);
+    if (Compacted) {
+        *XSaveSize += CpuidFn->Eax;
+    } else {
+        *XSaveSize = MAX(*XSaveSize, CpuidFn->Eax + CpuidFn->Ebx);
+    }
+  }
+
+  /*
+   * Either the guest set unsupported XCR0/XSS bits, or the corresponding
+   * entries in the CPUID table were not present. This is an invalid state.
+   */
+  if (XFeaturesFound != (XFeaturesEnabled & ~3UL)) {
+      return FALSE;
+  }
+
+  return TRUE;
+}
+
+/**
+  Check if a CPUID leaf/function is indexed via ECX sub-leaf/sub-function
+
+  @param[in]      EaxIn        EAX input for cpuid instruction
+
+  @retval FALSE                cpuid leaf/function is not indexed by ECX input
+  @retval TRUE                 cpuid leaf/function is indexed by ECX input
+
+**/
+STATIC
+BOOLEAN
+IsFunctionIndexed (
+  IN     UINT32  EaxIn
+  )
+{
+    switch (EaxIn) {
+    case CPUID_CACHE_PARAMS:
+    case CPUID_STRUCTURED_EXTENDED_FEATURE_FLAGS:
+    case CPUID_EXTENDED_TOPOLOGY:
+    case CPUID_EXTENDED_STATE:
+    case CPUID_INTEL_RDT_MONITORING:
+    case CPUID_INTEL_RDT_ALLOCATION:
+    case CPUID_INTEL_SGX:
+    case CPUID_INTEL_PROCESSOR_TRACE:
+    case CPUID_DETERMINISTIC_ADDRESS_TRANSLATION_PARAMETERS:
+    case CPUID_V2_EXTENDED_TOPOLOGY:
+    case 0x8000001D: /* Cache Topology Information */
+        return TRUE;
+    }
+    return FALSE;
+}
+
+/**
+  Fetch CPUID leaf/function via SEV-SNP CPUID table.
+
+  @param[in, out] Ghcb         Pointer to the Guest-Hypervisor Communication
+                               Block
+  @param[in]      EaxIn        EAX input for cpuid instruction
+  @param[in]      EcxIn        ECX input for cpuid instruction
+  @param[in]      Xcr0In       XCR0 at time of cpuid instruction
+  @param[in, out] Eax          Pointer to store leaf's EAX value
+  @param[in, out] Ebx          Pointer to store leaf's EBX value
+  @param[in, out] Ecx          Pointer to store leaf's ECX value
+  @param[in, out] Edx          Pointer to store leaf's EDX value
+  @param[in, out] Status       Pointer to store status from VMGEXIT (always 0
+                               unless return value indicates failure)
+  @param[in, out] Unsupported  Pointer to store indication of unsupported
+                               VMGEXIT (always false unless return value
+                               indicates failure)
+
+  @retval TRUE                 CPUID leaf fetch successfully.
+  @retval FALSE                Error occurred while fetching CPUID leaf. Callers
+                               should Status and Unsupported and handle
+                               accordingly if they indicate a more precise
+                               error condition.
+
+**/
+STATIC
+BOOLEAN
+GetCpuidFw (
+  IN OUT GHCB     *Ghcb,
+  IN     UINT32   EaxIn,
+  IN     UINT32   EcxIn,
+  IN     UINT64   XCr0,
+  IN OUT UINT32   *Eax,
+  IN OUT UINT32   *Ebx,
+  IN OUT UINT32   *Ecx,
+  IN OUT UINT32   *Edx,
+  IN OUT UINT64   *Status,
+  IN OUT BOOLEAN  *Unsupported
+  )
+{
+  SEV_SNP_CPUID_INFO  *CpuidInfo;
+  BOOLEAN             Found;
+  UINT32              Idx;
+
+  CpuidInfo = (SEV_SNP_CPUID_INFO *)(UINT64)PcdGet32(PcdOvmfSnpCpuidBase);
+  Found = FALSE;
+
+  for (Idx = 0; Idx < CpuidInfo->Count; Idx++) {
+    SEV_SNP_CPUID_FUNCTION *CpuidFn = &CpuidInfo->function[Idx];
+
+    if (CpuidFn->EaxIn != EaxIn) {
+      continue;
+    }
+
+    if (IsFunctionIndexed(CpuidFn->EaxIn) && CpuidFn->EcxIn != EcxIn) {
+      continue;
+    }
+
+    *Eax = CpuidFn->Eax;
+    *Ebx = CpuidFn->Ebx;
+    *Ecx = CpuidFn->Ecx;
+    *Edx = CpuidFn->Edx;
+
+    Found = TRUE;
+    break;
+  }
+
+  if (!Found) {
+    *Eax = *Ebx = *Ecx = *Edx = 0;
+    goto Out;
+  }
+
+  if (EaxIn == CPUID_VERSION_INFO) {
+    IA32_CR4  Cr4;
+    UINT32    Ebx2;
+    UINT32    Edx2;
+
+    if (!GetCpuidHyp (Ghcb, EaxIn, EcxIn, XCr0, NULL, &Ebx2, NULL, &Edx2,
+                      Status, Unsupported)) {
+      return FALSE;
+    }
+
+    /* initial APIC ID */
+    *Ebx = (*Ebx & 0x00FFFFFF) | (Ebx2 & 0xFF000000);
+    /* APIC enabled bit */
+    *Edx = (*Edx & ~BIT9) | (Edx2 & BIT9);
+    /* OSXSAVE enabled bit */
+    Cr4.UintN = AsmReadCr4 ();
+    *Ecx = (Cr4.Bits.OSXSAVE) ? (*Ecx & ~BIT27) | (*Ecx & BIT27)
+                              : (*Ecx & ~BIT27);
+  } else if (EaxIn == CPUID_STRUCTURED_EXTENDED_FEATURE_FLAGS) {
+    IA32_CR4  Cr4;
+
+    Cr4.UintN = AsmReadCr4 ();
+    /* OSPKE enabled bit */
+    *Ecx = (Cr4.Bits.PKE) ? (*Ecx | BIT4) : (*Ecx & ~BIT4);
+  } else if (EaxIn == CPUID_EXTENDED_TOPOLOGY) {
+    if (!GetCpuidHyp (Ghcb, EaxIn, EcxIn, XCr0, NULL, NULL, NULL, Edx,
+                      Status, Unsupported)) {
+      return FALSE;
+    }
+  } else if (EaxIn == CPUID_EXTENDED_STATE && (EcxIn == 0 || EcxIn == 1)) {
+    MSR_IA32_XSS_REGISTER  XssMsr;
+    BOOLEAN                Compacted;
+    UINT32                 XSaveSize;
+
+    XssMsr.Uint64 = 0;
+    if (EcxIn == 1) {
+        /*
+         * The PPR and APM aren't clear on what size should be encoded in
+         * 0xD:0x1:EBX when compaction is not enabled by either XSAVEC or
+         * XSAVES, as these are generally fixed to 1 on real CPUs. Report
+         * this undefined case as an error.
+         */
+        if (!(*Eax & (BIT3 | BIT1))) { /* (XSAVES | XSAVEC) */
+            return FALSE;
+        }
+
+        Compacted = TRUE;
+        XssMsr.Uint64 = AsmReadMsr64 (MSR_IA32_XSS);
+    }
+
+    if (!GetCpuidXSaveSize (XCr0 | XssMsr.Uint64, *Ebx, &XSaveSize,
+                            Compacted)) {
+        return FALSE;
+    }
+
+    *Ebx = XSaveSize;
+  } else if (EaxIn == 0x8000001E) {
+    UINT32  Ebx2;
+    UINT32  Ecx2;
+
+    /* extended APIC ID */
+    if (!GetCpuidHyp (Ghcb, EaxIn, EcxIn, XCr0, Eax, &Ebx2, &Ecx2, NULL,
+                      Status, Unsupported)) {
+      return FALSE;
+    }
+    /* compute ID */
+    *Ebx = (*Ebx & 0xFFFFFF00) | (Ebx2 & 0x000000FF);
+    /* node ID */
+    *Ecx = (*Ecx & 0xFFFFFF00) | (Ecx2 & 0x000000FF);
+  }
+
+Out:
+  *Status = 0;
+  *Unsupported = FALSE;
+  return TRUE;
+}
+
+/**
+  Handle a CPUID event.
+
+  Use VMGEXIT instruction or CPUID table to handle a CPUID event.
+
+  @param[in, out] Ghcb             Pointer to the Guest-Hypervisor Communication
+                                   Block
+  @param[in, out] Regs             x64 processor context
+  @param[in]      InstructionData  Instruction parsing context
+
+  @retval 0                        Event handled successfully
+  @return                          New exception value to propagate
+
+**/
+STATIC
+UINT64
+CpuidExit (
+  IN OUT GHCB                     *Ghcb,
+  IN OUT EFI_SYSTEM_CONTEXT_X64   *Regs,
+  IN     SEV_ES_INSTRUCTION_DATA  *InstructionData
+  )
+{
+  BOOLEAN             Unsupported;
+  UINT64              Status;
+  UINT32              EaxIn;
+  UINT32              EcxIn;
+  UINT64              XCr0;
+  UINT32              Eax;
+  UINT32              Ebx;
+  UINT32              Ecx;
+  UINT32              Edx;
+
+  EaxIn = Regs->Rax;
+  EcxIn = Regs->Rcx;
+
+  if (EaxIn == CPUID_EXTENDED_STATE) {
+    IA32_CR4  Cr4;
+
+    Cr4.UintN = AsmReadCr4 ();
+    XCr0 = (Cr4.Bits.OSXSAVE == 1) ? AsmXGetBv (0) : 1;
+  }
+
+  if (SnpEnabled ()) {
+      if (!GetCpuidFw (Ghcb, EaxIn, EcxIn, XCr0, &Eax, &Ebx, &Ecx, &Edx,
+                       &Status, &Unsupported)) {
+        goto CpuidFail;
+      }
+  } else {
+      if (!GetCpuidHyp (Ghcb, EaxIn, EcxIn, XCr0, &Eax, &Ebx, &Ecx, &Edx,
+                        &Status, &Unsupported)) {
+        goto CpuidFail;
+      }
+  }
+
+  Regs->Rax = Eax;
+  Regs->Rbx = Ebx;
+  Regs->Rcx = Ecx;
+  Regs->Rdx = Edx;
 
   return 0;
+
+CpuidFail:
+  if (Unsupported) {
+    return UnsupportedExit (Ghcb, Regs, InstructionData);
+  }
+
+  return Status;
 }
 
 /**
-- 
2.17.1

[PATCH v6 13/29] OvmfPkg/PlatformPei: register GHCB gpa for the SEV-SNP guest

[Brijesh Singh]

BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=3275

The SEV-SNP guest requires that GHCB GPA must be registered before using.
See the GHCB specification section 2.3.2 for more details.

Cc: Michael Roth <michael.roth@amd.com>
Cc: James Bottomley <jejb@linux.ibm.com>
Cc: Min Xu <min.m.xu@intel.com>
Cc: Jiewen Yao <jiewen.yao@intel.com>
Cc: Tom Lendacky <thomas.lendacky@amd.com>
Cc: Jordan Justen <jordan.l.justen@intel.com>
Cc: Ard Biesheuvel <ardb+tianocore@kernel.org>
Cc: Erdem Aktas <erdemaktas@google.com>
Signed-off-by: Brijesh Singh <brijesh.singh@amd.com>
---
 OvmfPkg/PlatformPei/AmdSev.c | 91 ++++++++++++++++++++++++++++++++++++
 1 file changed, 91 insertions(+)

diff --git a/OvmfPkg/PlatformPei/AmdSev.c b/OvmfPkg/PlatformPei/AmdSev.c
index a8bf610022ba..de876fdb478e 100644
--- a/OvmfPkg/PlatformPei/AmdSev.c
+++ b/OvmfPkg/PlatformPei/AmdSev.c
@@ -19,9 +19,93 @@
 #include <PiPei.h>
 #include <Register/Amd/Msr.h>
 #include <Register/Intel/SmramSaveStateMap.h>
+#include <Library/VmgExitLib.h>
 
 #include "Platform.h"
 
+/**
+  Handle an SEV-SNP/GHCB protocol check failure.
+
+  Notify the hypervisor using the VMGEXIT instruction that the SEV-SNP guest
+  wishes to be terminated.
+
+  @param[in] ReasonCode  Reason code to provide to the hypervisor for the
+                         termination request.
+
+**/
+STATIC
+VOID
+SevEsProtocolFailure (
+  IN UINT8  ReasonCode
+  )
+{
+  MSR_SEV_ES_GHCB_REGISTER  Msr;
+
+  //
+  // Use the GHCB MSR Protocol to request termination by the hypervisor
+  //
+  Msr.GhcbPhysicalAddress = 0;
+  Msr.GhcbTerminate.Function = GHCB_INFO_TERMINATE_REQUEST;
+  Msr.GhcbTerminate.ReasonCodeSet = GHCB_TERMINATE_GHCB;
+  Msr.GhcbTerminate.ReasonCode = ReasonCode;
+  AsmWriteMsr64 (MSR_SEV_ES_GHCB, Msr.GhcbPhysicalAddress);
+
+  AsmVmgExit ();
+
+  ASSERT (FALSE);
+  CpuDeadLoop ();
+}
+
+/**
+
+  This function can be used to register the GHCB GPA.
+
+  @param[in]  Address           The physical address to be registered.
+
+**/
+STATIC
+VOID
+GhcbRegister (
+  IN  EFI_PHYSICAL_ADDRESS   Address
+  )
+{
+  MSR_SEV_ES_GHCB_REGISTER  Msr;
+  MSR_SEV_ES_GHCB_REGISTER  CurrentMsr;
+  EFI_PHYSICAL_ADDRESS      GuestFrameNumber;
+
+  GuestFrameNumber = Address >> EFI_PAGE_SHIFT;
+
+  //
+  // Save the current MSR Value
+  //
+  CurrentMsr.GhcbPhysicalAddress = AsmReadMsr64 (MSR_SEV_ES_GHCB);
+
+  //
+  // Use the GHCB MSR Protocol to request to register the GPA.
+  //
+  Msr.GhcbPhysicalAddress = 0;
+  Msr.GhcbGpaRegister.Function = GHCB_INFO_GHCB_GPA_REGISTER_REQUEST;
+  Msr.GhcbGpaRegister.GuestFrameNumber = GuestFrameNumber;
+  AsmWriteMsr64 (MSR_SEV_ES_GHCB, Msr.GhcbPhysicalAddress);
+
+  AsmVmgExit ();
+
+  Msr.GhcbPhysicalAddress = AsmReadMsr64 (MSR_SEV_ES_GHCB);
+
+  //
+  // If hypervisor responded with a different GPA than requested then fail.
+  //
+  if ((Msr.GhcbGpaRegister.Function != GHCB_INFO_GHCB_GPA_REGISTER_RESPONSE) ||
+      (Msr.GhcbGpaRegister.GuestFrameNumber != GuestFrameNumber)) {
+    SevEsProtocolFailure (GHCB_TERMINATE_GHCB_GENERAL);
+  }
+
+  //
+  // Restore the MSR
+  //
+  AsmWriteMsr64 (MSR_SEV_ES_GHCB, CurrentMsr.GhcbPhysicalAddress);
+}
+
 /**
 
   Initialize SEV-ES support if running as an SEV-ES guest.
@@ -109,6 +193,13 @@ AmdSevEsInitialize (
     "SEV-ES is enabled, %lu GHCB backup pages allocated starting at 0x%p\n",
     (UINT64)GhcbBackupPageCount, GhcbBackupBase));
 
+  //
+  // SEV-SNP guest requires that GHCB GPA must be registered before using it.
+  //
+  if (MemEncryptSevSnpIsEnabled ()) {
+    GhcbRegister (GhcbBasePa);
+  }
+
   AsmWriteMsr64 (MSR_SEV_ES_GHCB, GhcbBasePa);
 
   //
-- 
2.17.1

[PATCH v6 14/29] OvmfPkg/AmdSevDxe: do not use extended PCI config space

[Brijesh Singh]

BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=3275

Commit 85b8eac59b8c5bd9c7eb9afdb64357ce1aa2e803 added support to ensure
that MMIO is only performed against the un-encrypted memory. If MMIO
is performed against encrypted memory, a #GP is raised.

The AmdSevDxe uses the functions provided by the MemEncryptSevLib to
clear the memory encryption mask from the page table. If the
MemEncryptSevLib is extended to include VmgExitLib then depedency
chain will look like this:

OvmfPkg/AmdSevDxe/AmdSevDxe.inf
-----> MemEncryptSevLib                    class
-----> "OvmfPkg/BaseMemEncryptSevLib/DxeMemEncryptSevLib.inf"   instance
-----> VmgExitLib                          class
-----> "OvmfPkg/VmgExitLib"    instance
-----> LocalApicLib                        class
-----> "UefiCpuPkg/BaseXApicX2ApicLib/BaseXApicX2ApicLib.inf" instance
-----> TimerLib                            class
-----> "OvmfPkg/AcpiTimerLib/DxeAcpiTimerLib.inf"   instance
-----> PciLib                                           class
-----> "OvmfPkg/DxePciLibI440FxQ35/DxePciLibI440FxQ35.inf"    instance
-----> PciExpressLib                                          class
-----> "MdePkg/BasePciExpressLib/BasePciExpressLib.inf"  instance

The LocalApicLib provides a constructor that gets called before the
AmdSevDxe can clear the memory encryption mask from the MMIO regions.

When running under the Q35 machine type, the call chain looks like this:

AcpiTimerLibConstructor ()  [AcpiTimerLib]
  PciRead32 ()              [DxePciLibI440FxQ35]
   PciExpressRead32 ()      [PciExpressLib]

The PciExpressRead32 () reads the MMIO region. The MMIO regions are not
yet mapped un-encrypted, so the check introduced in the commit
85b8eac59b8c5bd9c7eb9afdb64357ce1aa2e803 raises a #GP.

The AmdSevDxe driver does not require the access to the extended PCI
config space. Accessing a normal PCI config space, via IO port should be
sufficent. Use the module-scope override to make the AmdSevDxe use the
BasePciLib instead of BasePciExpressLib so that PciRead32 () uses the
IO ports instead of the extended config space.

Cc: Michael Roth <michael.roth@amd.com>
Cc: James Bottomley <jejb@linux.ibm.com>
Cc: Min Xu <min.m.xu@intel.com>
Cc: Jiewen Yao <jiewen.yao@intel.com>
Cc: Tom Lendacky <thomas.lendacky@amd.com>
Cc: Jordan Justen <jordan.l.justen@intel.com>
Cc: Ard Biesheuvel <ardb+tianocore@kernel.org>
Cc: Erdem Aktas <erdemaktas@google.com>
Suggested-by: Laszlo Ersek <lersek@redhat.com>
Signed-off-by: Brijesh Singh <brijesh.singh@amd.com>
---
 OvmfPkg/AmdSev/AmdSevX64.dsc | 5 ++++-
 OvmfPkg/Bhyve/BhyveX64.dsc   | 5 ++++-
 OvmfPkg/OvmfPkgIa32X64.dsc   | 5 ++++-
 OvmfPkg/OvmfPkgX64.dsc       | 5 ++++-
 OvmfPkg/OvmfXen.dsc          | 5 ++++-
 5 files changed, 20 insertions(+), 5 deletions(-)

diff --git a/OvmfPkg/AmdSev/AmdSevX64.dsc b/OvmfPkg/AmdSev/AmdSevX64.dsc
index e6cd10b75922..a8ea08a10a99 100644
--- a/OvmfPkg/AmdSev/AmdSevX64.dsc
+++ b/OvmfPkg/AmdSev/AmdSevX64.dsc
@@ -812,7 +812,10 @@ [Components]
 !endif
 
   OvmfPkg/PlatformDxe/Platform.inf
-  OvmfPkg/AmdSevDxe/AmdSevDxe.inf
+  OvmfPkg/AmdSevDxe/AmdSevDxe.inf {
+    <LibraryClasses>
+    PciLib|MdePkg/Library/BasePciLibCf8/BasePciLibCf8.inf
+  }
   OvmfPkg/IoMmuDxe/IoMmuDxe.inf
 
   #
diff --git a/OvmfPkg/Bhyve/BhyveX64.dsc b/OvmfPkg/Bhyve/BhyveX64.dsc
index d8fe607d1cf7..f45634996247 100644
--- a/OvmfPkg/Bhyve/BhyveX64.dsc
+++ b/OvmfPkg/Bhyve/BhyveX64.dsc
@@ -790,7 +790,10 @@ [Components]
 !endif
 
   OvmfPkg/PlatformDxe/Platform.inf
-  OvmfPkg/AmdSevDxe/AmdSevDxe.inf
+  OvmfPkg/AmdSevDxe/AmdSevDxe.inf {
+    <LibraryClasses>
+    PciLib|MdePkg/Library/BasePciLibCf8/BasePciLibCf8.inf
+  }
   OvmfPkg/IoMmuDxe/IoMmuDxe.inf
 
 
diff --git a/OvmfPkg/OvmfPkgIa32X64.dsc b/OvmfPkg/OvmfPkgIa32X64.dsc
index a467ab7090fb..f42abc041c0c 100644
--- a/OvmfPkg/OvmfPkgIa32X64.dsc
+++ b/OvmfPkg/OvmfPkgIa32X64.dsc
@@ -965,7 +965,10 @@ [Components.X64]
 !endif
 
   OvmfPkg/PlatformDxe/Platform.inf
-  OvmfPkg/AmdSevDxe/AmdSevDxe.inf
+  OvmfPkg/AmdSevDxe/AmdSevDxe.inf {
+    <LibraryClasses>
+    PciLib|MdePkg/Library/BasePciLibCf8/BasePciLibCf8.inf
+  }
   OvmfPkg/IoMmuDxe/IoMmuDxe.inf
 
 !if $(SMM_REQUIRE) == TRUE
diff --git a/OvmfPkg/OvmfPkgX64.dsc b/OvmfPkg/OvmfPkgX64.dsc
index e56b83d95e09..19cc5d4122e2 100644
--- a/OvmfPkg/OvmfPkgX64.dsc
+++ b/OvmfPkg/OvmfPkgX64.dsc
@@ -963,7 +963,10 @@ [Components]
 !endif
 
   OvmfPkg/PlatformDxe/Platform.inf
-  OvmfPkg/AmdSevDxe/AmdSevDxe.inf
+  OvmfPkg/AmdSevDxe/AmdSevDxe.inf {
+    <LibraryClasses>
+    PciLib|MdePkg/Library/BasePciLibCf8/BasePciLibCf8.inf
+  }
   OvmfPkg/IoMmuDxe/IoMmuDxe.inf
 
 !if $(SMM_REQUIRE) == TRUE
diff --git a/OvmfPkg/OvmfXen.dsc b/OvmfPkg/OvmfXen.dsc
index 1a9c06c164a8..5ac12a36b981 100644
--- a/OvmfPkg/OvmfXen.dsc
+++ b/OvmfPkg/OvmfXen.dsc
@@ -726,7 +726,10 @@ [Components]
   }
 
   OvmfPkg/PlatformDxe/Platform.inf
-  OvmfPkg/AmdSevDxe/AmdSevDxe.inf
+  OvmfPkg/AmdSevDxe/AmdSevDxe.inf {
+    <LibraryClasses>
+    PciLib|MdePkg/Library/BasePciLibCf8/BasePciLibCf8.inf
+  }
   OvmfPkg/IoMmuDxe/IoMmuDxe.inf
 
   #
-- 
2.17.1

[PATCH v6 15/29] OvmfPkg/MemEncryptSevLib: add support to validate system RAM

[Brijesh Singh]

BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=3275

Many of the integrity guarantees of SEV-SNP are enforced through the
Reverse Map Table (RMP). Each RMP entry contains the GPA at which a
particular page of DRAM should be mapped. The guest can request the
hypervisor to add pages in the RMP table via the Page State Change VMGEXIT
defined in the GHCB specification section 2.5.1 and 4.1.6. Inside each RMP
entry is a Validated flag; this flag is automatically cleared to 0 by the
CPU hardware when a new RMP entry is created for a guest. Each VM page
can be either validated or invalidated, as indicated by the Validated
flag in the RMP entry. Memory access to a private page that is not
validated generates a #VC. A VM can use the PVALIDATE instruction to
validate the private page before using it.

During the guest creation, the boot ROM memory is pre-validated by the
AMD-SEV firmware. The MemEncryptSevSnpValidateSystemRam() can be called
during the SEC and PEI phase to validate the detected system RAM.

One of the fields in the Page State Change NAE is the RMP page size. The
page size input parameter indicates that either a 4KB or 2MB page should
be used while adding the RMP entry. During the validation, when possible,
the MemEncryptSevSnpValidateSystemRam() will use the 2MB entry. A
hypervisor backing the memory may choose to use the different page size
in the RMP entry. In those cases, the PVALIDATE instruction should return
SIZEMISMATCH. If a SIZEMISMATCH is detected, then validate all 512-pages
constituting a 2MB region.

Upon completion, the PVALIDATE instruction sets the rFLAGS.CF to 0 if
instruction changed the RMP entry and to 1 if the instruction did not
change the RMP entry. The rFlags.CF will be 1 only when a memory region
is already validated. We should not double validate a memory
as it could lead to a security compromise. If double validation is
detected, terminate the boot.

Cc: Michael Roth <michael.roth@amd.com>
Cc: James Bottomley <jejb@linux.ibm.com>
Cc: Min Xu <min.m.xu@intel.com>
Cc: Jiewen Yao <jiewen.yao@intel.com>
Cc: Tom Lendacky <thomas.lendacky@amd.com>
Cc: Jordan Justen <jordan.l.justen@intel.com>
Cc: Ard Biesheuvel <ardb+tianocore@kernel.org>
Cc: Erdem Aktas <erdemaktas@google.com>
Signed-off-by: Brijesh Singh <brijesh.singh@amd.com>
---
 OvmfPkg/OvmfPkgIa32.dsc                       |   1 +
 OvmfPkg/OvmfPkgIa32X64.dsc                    |   1 +
 .../DxeMemEncryptSevLib.inf                   |   3 +
 .../PeiMemEncryptSevLib.inf                   |   3 +
 .../SecMemEncryptSevLib.inf                   |   3 +
 OvmfPkg/Include/Library/MemEncryptSevLib.h    |  14 +
 .../X64/SnpPageStateChange.h                  |  31 ++
 .../Ia32/MemEncryptSevLib.c                   |  17 +
 .../X64/DxeSnpSystemRamValidate.c             |  40 +++
 .../X64/PeiSnpSystemRamValidate.c             |  36 +++
 .../X64/SecSnpSystemRamValidate.c             |  36 +++
 .../X64/SnpPageStateChangeInternal.c          | 295 ++++++++++++++++++
 12 files changed, 480 insertions(+)
 create mode 100644 OvmfPkg/Library/BaseMemEncryptSevLib/X64/SnpPageStateChange.h
 create mode 100644 OvmfPkg/Library/BaseMemEncryptSevLib/X64/DxeSnpSystemRamValidate.c
 create mode 100644 OvmfPkg/Library/BaseMemEncryptSevLib/X64/PeiSnpSystemRamValidate.c
 create mode 100644 OvmfPkg/Library/BaseMemEncryptSevLib/X64/SecSnpSystemRamValidate.c
 create mode 100644 OvmfPkg/Library/BaseMemEncryptSevLib/X64/SnpPageStateChangeInternal.c

diff --git a/OvmfPkg/OvmfPkgIa32.dsc b/OvmfPkg/OvmfPkgIa32.dsc
index d1d92c97bae3..626dbc15786a 100644
--- a/OvmfPkg/OvmfPkgIa32.dsc
+++ b/OvmfPkg/OvmfPkgIa32.dsc
@@ -266,6 +266,7 @@ [LibraryClasses.common.SEC]
 !else
   CpuExceptionHandlerLib|UefiCpuPkg/Library/CpuExceptionHandlerLib/SecPeiCpuExceptionHandlerLib.inf
 !endif
+  MemEncryptSevLib|OvmfPkg/Library/BaseMemEncryptSevLib/SecMemEncryptSevLib.inf
 
 [LibraryClasses.common.PEI_CORE]
   HobLib|MdePkg/Library/PeiHobLib/PeiHobLib.inf
diff --git a/OvmfPkg/OvmfPkgIa32X64.dsc b/OvmfPkg/OvmfPkgIa32X64.dsc
index f42abc041c0c..58be97eb0605 100644
--- a/OvmfPkg/OvmfPkgIa32X64.dsc
+++ b/OvmfPkg/OvmfPkgIa32X64.dsc
@@ -270,6 +270,7 @@ [LibraryClasses.common.SEC]
 !else
   CpuExceptionHandlerLib|UefiCpuPkg/Library/CpuExceptionHandlerLib/SecPeiCpuExceptionHandlerLib.inf
 !endif
+  MemEncryptSevLib|OvmfPkg/Library/BaseMemEncryptSevLib/SecMemEncryptSevLib.inf
 
 [LibraryClasses.common.PEI_CORE]
   HobLib|MdePkg/Library/PeiHobLib/PeiHobLib.inf
diff --git a/OvmfPkg/Library/BaseMemEncryptSevLib/DxeMemEncryptSevLib.inf b/OvmfPkg/Library/BaseMemEncryptSevLib/DxeMemEncryptSevLib.inf
index f2e162d68076..f613bb314f5f 100644
--- a/OvmfPkg/Library/BaseMemEncryptSevLib/DxeMemEncryptSevLib.inf
+++ b/OvmfPkg/Library/BaseMemEncryptSevLib/DxeMemEncryptSevLib.inf
@@ -34,8 +34,10 @@ [Sources]
   PeiDxeMemEncryptSevLibInternal.c
 
 [Sources.X64]
+  X64/DxeSnpSystemRamValidate.c
   X64/MemEncryptSevLib.c
   X64/PeiDxeVirtualMemory.c
+  X64/SnpPageStateChangeInternal.c
   X64/VirtualMemory.c
   X64/VirtualMemory.h
 
@@ -49,6 +51,7 @@ [LibraryClasses]
   DebugLib
   MemoryAllocationLib
   PcdLib
+  VmgExitLib
 
 [FeaturePcd]
   gUefiOvmfPkgTokenSpaceGuid.PcdSmmSmramRequire
diff --git a/OvmfPkg/Library/BaseMemEncryptSevLib/PeiMemEncryptSevLib.inf b/OvmfPkg/Library/BaseMemEncryptSevLib/PeiMemEncryptSevLib.inf
index 03a78c32df28..0402e49a1028 100644
--- a/OvmfPkg/Library/BaseMemEncryptSevLib/PeiMemEncryptSevLib.inf
+++ b/OvmfPkg/Library/BaseMemEncryptSevLib/PeiMemEncryptSevLib.inf
@@ -36,6 +36,8 @@ [Sources]
 [Sources.X64]
   X64/MemEncryptSevLib.c
   X64/PeiDxeVirtualMemory.c
+  X64/PeiSnpSystemRamValidate.c
+  X64/SnpPageStateChangeInternal.c
   X64/VirtualMemory.c
   X64/VirtualMemory.h
 
@@ -49,6 +51,7 @@ [LibraryClasses]
   DebugLib
   MemoryAllocationLib
   PcdLib
+  VmgExitLib
 
 [FeaturePcd]
   gUefiOvmfPkgTokenSpaceGuid.PcdSmmSmramRequire
diff --git a/OvmfPkg/Library/BaseMemEncryptSevLib/SecMemEncryptSevLib.inf b/OvmfPkg/Library/BaseMemEncryptSevLib/SecMemEncryptSevLib.inf
index 279c38bfbc2c..939af0a91ea4 100644
--- a/OvmfPkg/Library/BaseMemEncryptSevLib/SecMemEncryptSevLib.inf
+++ b/OvmfPkg/Library/BaseMemEncryptSevLib/SecMemEncryptSevLib.inf
@@ -35,6 +35,8 @@ [Sources]
 [Sources.X64]
   X64/MemEncryptSevLib.c
   X64/SecVirtualMemory.c
+  X64/SecSnpSystemRamValidate.c
+  X64/SnpPageStateChangeInternal.c
   X64/VirtualMemory.c
   X64/VirtualMemory.h
 
@@ -46,6 +48,7 @@ [LibraryClasses]
   CpuLib
   DebugLib
   PcdLib
+  VmgExitLib
 
 [FixedPcd]
   gUefiCpuPkgTokenSpaceGuid.PcdSevEsWorkAreaBase
diff --git a/OvmfPkg/Include/Library/MemEncryptSevLib.h b/OvmfPkg/Include/Library/MemEncryptSevLib.h
index 796de62ec2f8..f708a0cdaa72 100644
--- a/OvmfPkg/Include/Library/MemEncryptSevLib.h
+++ b/OvmfPkg/Include/Library/MemEncryptSevLib.h
@@ -215,4 +215,18 @@ MemEncryptSevClearMmioPageEncMask (
   IN UINTN                    NumPages
   );
 
+/**
+  Pre-validate the system RAM when SEV-SNP is enabled in the guest VM.
+
+  @param[in]  BaseAddress             Base address
+  @param[in]  NumPages                Number of pages starting from the base address
+
+**/
+VOID
+EFIAPI
+MemEncryptSevSnpPreValidateSystemRam (
+  IN PHYSICAL_ADDRESS           BaseAddress,
+  IN UINTN                      NumPages
+  );
+
 #endif // _MEM_ENCRYPT_SEV_LIB_H_
diff --git a/OvmfPkg/Library/BaseMemEncryptSevLib/X64/SnpPageStateChange.h b/OvmfPkg/Library/BaseMemEncryptSevLib/X64/SnpPageStateChange.h
new file mode 100644
index 000000000000..8bbdf06468b9
--- /dev/null
+++ b/OvmfPkg/Library/BaseMemEncryptSevLib/X64/SnpPageStateChange.h
@@ -0,0 +1,31 @@
+/** @file
+
+  SEV-SNP Page Validation functions.
+
+  Copyright (c) 2021 AMD Incorporated. All rights reserved.<BR>
+
+  SPDX-License-Identifier: BSD-2-Clause-Patent
+
+**/
+
+#ifndef SNP_PAGE_STATE_INTERNAL_H_
+#define SNP_PAGE_STATE_INTERNAL_H_
+
+//
+// SEV-SNP Page states
+//
+typedef enum {
+  SevSnpPagePrivate,
+  SevSnpPageShared,
+
+} SEV_SNP_PAGE_STATE;
+
+VOID
+InternalSetPageState (
+  IN EFI_PHYSICAL_ADDRESS             BaseAddress,
+  IN UINTN                            NumPages,
+  IN SEV_SNP_PAGE_STATE               State,
+  IN BOOLEAN                          UseLargeEntry
+  );
+
+#endif
diff --git a/OvmfPkg/Library/BaseMemEncryptSevLib/Ia32/MemEncryptSevLib.c b/OvmfPkg/Library/BaseMemEncryptSevLib/Ia32/MemEncryptSevLib.c
index be260e0d1014..df5e4d61513d 100644
--- a/OvmfPkg/Library/BaseMemEncryptSevLib/Ia32/MemEncryptSevLib.c
+++ b/OvmfPkg/Library/BaseMemEncryptSevLib/Ia32/MemEncryptSevLib.c
@@ -136,3 +136,20 @@ MemEncryptSevClearMmioPageEncMask (
   //
   return RETURN_UNSUPPORTED;
 }
+
+/**
+  Pre-validate the system RAM when SEV-SNP is enabled in the guest VM.
+
+  @param[in]  BaseAddress             Base address
+  @param[in]  NumPages                Number of pages starting from the base address
+
+**/
+VOID
+EFIAPI
+MemEncryptSevSnpPreValidateSystemRam (
+  IN PHYSICAL_ADDRESS           BaseAddress,
+  IN UINTN                      NumPages
+  )
+{
+  ASSERT (FALSE);
+}
diff --git a/OvmfPkg/Library/BaseMemEncryptSevLib/X64/DxeSnpSystemRamValidate.c b/OvmfPkg/Library/BaseMemEncryptSevLib/X64/DxeSnpSystemRamValidate.c
new file mode 100644
index 000000000000..ad8d8b388dc8
--- /dev/null
+++ b/OvmfPkg/Library/BaseMemEncryptSevLib/X64/DxeSnpSystemRamValidate.c
@@ -0,0 +1,40 @@
+/** @file
+
+  SEV-SNP Page Validation functions.
+
+  Copyright (c) 2021 AMD Incorporated. All rights reserved.<BR>
+
+  SPDX-License-Identifier: BSD-2-Clause-Patent
+
+**/
+
+#include <Uefi/UefiBaseType.h>
+#include <Library/BaseLib.h>
+#include <Library/DebugLib.h>
+#include <Library/MemEncryptSevLib.h>
+
+#include "SnpPageStateChange.h"
+
+/**
+  Pre-validate the system RAM when SEV-SNP is enabled in the guest VM.
+
+  @param[in]  BaseAddress             Base address
+  @param[in]  NumPages                Number of pages starting from the base address
+
+**/
+VOID
+EFIAPI
+MemEncryptSevSnpPreValidateSystemRam (
+  IN PHYSICAL_ADDRESS                   BaseAddress,
+  IN UINTN                              NumPages
+  )
+{
+  if (!MemEncryptSevSnpIsEnabled ()) {
+    return;
+  }
+
+  //
+  // All the pre-validation must be completed in the PEI phase.
+  //
+  ASSERT (FALSE);
+}
diff --git a/OvmfPkg/Library/BaseMemEncryptSevLib/X64/PeiSnpSystemRamValidate.c b/OvmfPkg/Library/BaseMemEncryptSevLib/X64/PeiSnpSystemRamValidate.c
new file mode 100644
index 000000000000..64aab7f45b6d
--- /dev/null
+++ b/OvmfPkg/Library/BaseMemEncryptSevLib/X64/PeiSnpSystemRamValidate.c
@@ -0,0 +1,36 @@
+/** @file
+
+  SEV-SNP Page Validation functions.
+
+  Copyright (c) 2021 AMD Incorporated. All rights reserved.<BR>
+
+  SPDX-License-Identifier: BSD-2-Clause-Patent
+
+**/
+
+#include <Uefi/UefiBaseType.h>
+#include <Library/BaseLib.h>
+#include <Library/MemEncryptSevLib.h>
+
+#include "SnpPageStateChange.h"
+
+/**
+  Pre-validate the system RAM when SEV-SNP is enabled in the guest VM.
+
+  @param[in]  BaseAddress             Base address
+  @param[in]  NumPages                Number of pages starting from the base address
+
+**/
+VOID
+EFIAPI
+MemEncryptSevSnpPreValidateSystemRam (
+  IN PHYSICAL_ADDRESS                   BaseAddress,
+  IN UINTN                              NumPages
+  )
+{
+  if (!MemEncryptSevSnpIsEnabled ()) {
+    return;
+  }
+
+  InternalSetPageState (BaseAddress, NumPages, SevSnpPagePrivate, TRUE);
+}
diff --git a/OvmfPkg/Library/BaseMemEncryptSevLib/X64/SecSnpSystemRamValidate.c b/OvmfPkg/Library/BaseMemEncryptSevLib/X64/SecSnpSystemRamValidate.c
new file mode 100644
index 000000000000..64aab7f45b6d
--- /dev/null
+++ b/OvmfPkg/Library/BaseMemEncryptSevLib/X64/SecSnpSystemRamValidate.c
@@ -0,0 +1,36 @@
+/** @file
+
+  SEV-SNP Page Validation functions.
+
+  Copyright (c) 2021 AMD Incorporated. All rights reserved.<BR>
+
+  SPDX-License-Identifier: BSD-2-Clause-Patent
+
+**/
+
+#include <Uefi/UefiBaseType.h>
+#include <Library/BaseLib.h>
+#include <Library/MemEncryptSevLib.h>
+
+#include "SnpPageStateChange.h"
+
+/**
+  Pre-validate the system RAM when SEV-SNP is enabled in the guest VM.
+
+  @param[in]  BaseAddress             Base address
+  @param[in]  NumPages                Number of pages starting from the base address
+
+**/
+VOID
+EFIAPI
+MemEncryptSevSnpPreValidateSystemRam (
+  IN PHYSICAL_ADDRESS                   BaseAddress,
+  IN UINTN                              NumPages
+  )
+{
+  if (!MemEncryptSevSnpIsEnabled ()) {
+    return;
+  }
+
+  InternalSetPageState (BaseAddress, NumPages, SevSnpPagePrivate, TRUE);
+}
diff --git a/OvmfPkg/Library/BaseMemEncryptSevLib/X64/SnpPageStateChangeInternal.c b/OvmfPkg/Library/BaseMemEncryptSevLib/X64/SnpPageStateChangeInternal.c
new file mode 100644
index 000000000000..506df12d4e51
--- /dev/null
+++ b/OvmfPkg/Library/BaseMemEncryptSevLib/X64/SnpPageStateChangeInternal.c
@@ -0,0 +1,295 @@
+/** @file
+
+  SEV-SNP Page Validation functions.
+
+  Copyright (c) 2021 AMD Incorporated. All rights reserved.<BR>
+
+  SPDX-License-Identifier: BSD-2-Clause-Patent
+
+**/
+
+#include <Uefi/UefiBaseType.h>
+#include <Library/BaseLib.h>
+#include <Library/BaseMemoryLib.h>
+#include <Library/MemEncryptSevLib.h>
+#include <Library/DebugLib.h>
+#include <Library/VmgExitLib.h>
+
+#include <Register/Amd/Ghcb.h>
+#include <Register/Amd/Msr.h>
+
+#include "SnpPageStateChange.h"
+
+#define IS_ALIGNED(x, y)        ((((x) & (y - 1)) == 0))
+#define PAGES_PER_LARGE_ENTRY   512
+
+STATIC
+UINTN
+MemoryStateToGhcbOp (
+  IN SEV_SNP_PAGE_STATE   State
+  )
+{
+  UINTN Cmd;
+
+  switch (State) {
+    case SevSnpPageShared: Cmd = SNP_PAGE_STATE_SHARED; break;
+    case SevSnpPagePrivate: Cmd = SNP_PAGE_STATE_PRIVATE; break;
+    default: ASSERT(0);
+  }
+
+  return Cmd;
+}
+
+STATIC
+VOID
+SnpPageStateFailureTerminate (
+  VOID
+  )
+{
+  MSR_SEV_ES_GHCB_REGISTER  Msr;
+
+  //
+  // Use the GHCB MSR Protocol to request termination by the hypervisor
+  //
+  Msr.GhcbPhysicalAddress = 0;
+  Msr.GhcbTerminate.Function = GHCB_INFO_TERMINATE_REQUEST;
+  Msr.GhcbTerminate.ReasonCodeSet = GHCB_TERMINATE_GHCB;
+  Msr.GhcbTerminate.ReasonCode = GHCB_TERMINATE_GHCB_GENERAL;
+  AsmWriteMsr64 (MSR_SEV_ES_GHCB, Msr.GhcbPhysicalAddress);
+
+  AsmVmgExit ();
+
+  ASSERT (FALSE);
+  CpuDeadLoop ();
+}
+
+/**
+ This function issues the PVALIDATE instruction to validate or invalidate the memory
+ range specified. If PVALIDATE returns size mismatch then it retry validating with
+ smaller page size.
+
+ */
+STATIC
+VOID
+PvalidateRange (
+  IN  SNP_PAGE_STATE_CHANGE_INFO    *Info,
+  IN  UINTN                         StartIndex,
+  IN  UINTN                         EndIndex,
+  IN  BOOLEAN                       Validate
+  )
+{
+  UINTN         Address, RmpPageSize, Ret, i;
+
+  for (; StartIndex <= EndIndex; StartIndex++) {
+    //
+    // Get the address and the page size from the Info.
+    //
+    Address = Info->Entry[StartIndex].GuestFrameNumber << EFI_PAGE_SHIFT;
+    RmpPageSize = Info->Entry[StartIndex].PageSize;
+
+    Ret = AsmPvalidate (RmpPageSize, Validate, Address);
+
+    //
+    // If we fail to validate due to size mismatch then try with the
+    // smaller page size. This senario will occur if the backing page in
+    // the RMP entry is 4K and we are validating it as a 2MB.
+    //
+    if ((Ret == PVALIDATE_RET_SIZE_MISMATCH) && (RmpPageSize == PvalidatePageSize2MB)) {
+      for (i = 0; i < PAGES_PER_LARGE_ENTRY; i++) {
+        Ret = AsmPvalidate (PvalidatePageSize4K, Validate, Address);
+        if (Ret) {
+          break;
+        }
+
+        Address = Address + EFI_PAGE_SIZE;
+      }
+    }
+
+    //
+    // If validation failed then do not continue.
+    //
+    if (Ret) {
+      DEBUG ((
+        DEBUG_ERROR, "%a:%a: Failed to %a address 0x%Lx Error code %d\n",
+        gEfiCallerBaseName,
+        __FUNCTION__,
+        Validate ? "Validate" : "Invalidate",
+        Address,
+        Ret
+        ));
+      SnpPageStateFailureTerminate ();
+    }
+  }
+}
+
+STATIC
+EFI_PHYSICAL_ADDRESS
+BuildPageStateBuffer (
+  IN EFI_PHYSICAL_ADDRESS           BaseAddress,
+  IN EFI_PHYSICAL_ADDRESS           EndAddress,
+  IN SEV_SNP_PAGE_STATE             State,
+  IN BOOLEAN                        UseLargeEntry,
+  IN SNP_PAGE_STATE_CHANGE_INFO     *Info
+  )
+{
+  EFI_PHYSICAL_ADDRESS      NextAddress;
+  UINTN                     i, RmpPageSize;
+
+  // Clear the page state structure
+  SetMem (Info, sizeof (*Info), 0);
+
+  i = 0;
+  NextAddress = EndAddress;
+
+  //
+  // Populate the page state entry structure
+  //
+  while ((BaseAddress < EndAddress) && (i < SNP_PAGE_STATE_MAX_ENTRY)) {
+    //
+    // Is this a 2MB aligned page? Check if we can use the Large RMP entry.
+    //
+    if (UseLargeEntry && IS_ALIGNED (BaseAddress, SIZE_2MB) &&
+      ((EndAddress - BaseAddress) >= SIZE_2MB)) {
+      RmpPageSize = PvalidatePageSize2MB;
+      NextAddress = BaseAddress + SIZE_2MB;
+    } else {
+      RmpPageSize = PvalidatePageSize4K;
+      NextAddress = BaseAddress + EFI_PAGE_SIZE;
+    }
+
+    Info->Entry[i].GuestFrameNumber = BaseAddress >> EFI_PAGE_SHIFT;
+    Info->Entry[i].PageSize = RmpPageSize;
+    Info->Entry[i].Operation = MemoryStateToGhcbOp (State);
+    Info->Entry[i].CurrentPage = 0;
+    Info->Header.EndEntry = i;
+
+    BaseAddress = NextAddress;
+    i++;
+  }
+
+  return NextAddress;
+}
+
+STATIC
+VOID
+PageStateChangeVmgExit (
+  IN GHCB                           *Ghcb,
+  IN SNP_PAGE_STATE_CHANGE_INFO     *Info
+  )
+{
+  EFI_STATUS                      Status;
+
+  //
+  // As per the GHCB specification, the hypervisor can resume the guest before
+  // processing all the entries. Checks whether all the entries are processed.
+  //
+  // The stragtegy here is to wait for the hypervisor to change the page
+  // state in the RMP table before guest access the memory pages. If the
+  // page state was not successful, then later memory access will result
+  // in the crash.
+  //
+  while (Info->Header.CurrentEntry <= Info->Header.EndEntry) {
+    Ghcb->SaveArea.SwScratch = (UINT64) Ghcb->SharedBuffer;
+    VmgSetOffsetValid (Ghcb, GhcbSwScratch);
+
+    Status = VmgExit (Ghcb, SVM_EXIT_SNP_PAGE_STATE_CHANGE, 0, 0);
+
+    //
+    // The Page State Change VMGEXIT can pass the failure through the
+    // ExitInfo2. Lets check both the return value as well as ExitInfo2.
+    //
+    if ((Status != 0) || (Ghcb->SaveArea.SwExitInfo2)) {
+      SnpPageStateFailureTerminate ();
+    }
+  }
+}
+
+/**
+ The function is used to set the page state when SEV-SNP is active. The page state
+ transition consist of changing the page ownership in the RMP table, and using the
+ PVALIDATE instruction to update the Validated bit in RMP table.
+
+ When the UseLargeEntry is set to TRUE, then function will try to use the large RMP
+ entry (whevever possible).
+ */
+VOID
+InternalSetPageState (
+  IN EFI_PHYSICAL_ADDRESS             BaseAddress,
+  IN UINTN                            NumPages,
+  IN SEV_SNP_PAGE_STATE               State,
+  IN BOOLEAN                          UseLargeEntry
+  )
+{
+  GHCB                            *Ghcb;
+  EFI_PHYSICAL_ADDRESS            NextAddress, EndAddress;
+  MSR_SEV_ES_GHCB_REGISTER        Msr;
+  BOOLEAN                         InterruptState;
+  SNP_PAGE_STATE_CHANGE_INFO      *Info;
+
+  Msr.GhcbPhysicalAddress = AsmReadMsr64 (MSR_SEV_ES_GHCB);
+  Ghcb = Msr.Ghcb;
+
+  EndAddress = BaseAddress + EFI_PAGES_TO_SIZE (NumPages);
+
+  DEBUG ((
+    DEBUG_VERBOSE, "%a:%a Address 0x%Lx - 0x%Lx State = %a LargeEntry = %d\n",
+    gEfiCallerBaseName,
+    __FUNCTION__,
+    BaseAddress,
+    EndAddress,
+    State == SevSnpPageShared ? "Shared" : "Private",
+    UseLargeEntry
+    ));
+
+  while (BaseAddress < EndAddress) {
+    UINTN       CurrentEntry, EndEntry;
+
+    //
+    // Initialize the GHCB
+    //
+    VmgInit (Ghcb, &InterruptState);
+
+    //
+    // Build the page state structure
+    //
+    Info = (SNP_PAGE_STATE_CHANGE_INFO *) Ghcb->SharedBuffer;
+    NextAddress = BuildPageStateBuffer (BaseAddress,
+                                        EndAddress,
+                                        State,
+                                        UseLargeEntry,
+                                        Info
+                                       );
+
+    //
+    // Save the current and end entry from the page state structure. We need
+    // it later.
+    //
+    CurrentEntry = Info->Header.CurrentEntry;
+    EndEntry = Info->Header.EndEntry;
+
+    //
+    // If the caller requested to change the page state to shared then
+    // invalidate the pages before making the page shared in the RMP table.
+    //
+    if (State == SevSnpPageShared) {
+      PvalidateRange (Info, CurrentEntry, EndEntry, FALSE);
+    }
+
+    //
+    // Invoke the page state change VMGEXIT.
+    //
+    PageStateChangeVmgExit (Ghcb, Info);
+
+    //
+    // If the caller requested to change the page state to private then
+    // validate the pages after it has been added in the RMP table.
+    //
+    if (State == SevSnpPagePrivate) {
+      PvalidateRange (Info, CurrentEntry, EndEntry, TRUE);
+    }
+
+    VmgDone (Ghcb, InterruptState);
+
+    BaseAddress = NextAddress;
+  }
+}
-- 
2.17.1

[PATCH v6 16/29] OvmfPkg/BaseMemEncryptSevLib: skip the pre-validated system RAM

[Brijesh Singh]

BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=3275

The MemEncryptSevSnpPreValidateSystemRam() is used for pre-validating the
system RAM. As the boot progress, each phase validates a fixed region of
the RAM. In the PEI phase, the PlatformPei detects all the available RAM
and calls to pre-validate the detected system RAM.

While validating the system RAM in PEI phase, we must skip previously
validated system RAM to avoid the double validation.

Cc: Michael Roth <michael.roth@amd.com>
Cc: James Bottomley <jejb@linux.ibm.com>
Cc: Min Xu <min.m.xu@intel.com>
Cc: Jiewen Yao <jiewen.yao@intel.com>
Cc: Tom Lendacky <thomas.lendacky@amd.com>
Cc: Jordan Justen <jordan.l.justen@intel.com>
Cc: Ard Biesheuvel <ardb+tianocore@kernel.org>
Cc: Erdem Aktas <erdemaktas@google.com>
Signed-off-by: Brijesh Singh <brijesh.singh@amd.com>
---
 .../PeiMemEncryptSevLib.inf                   |  2 +
 .../X64/PeiSnpSystemRamValidate.c             | 65 ++++++++++++++++++-
 2 files changed, 66 insertions(+), 1 deletion(-)

diff --git a/OvmfPkg/Library/BaseMemEncryptSevLib/PeiMemEncryptSevLib.inf b/OvmfPkg/Library/BaseMemEncryptSevLib/PeiMemEncryptSevLib.inf
index 0402e49a1028..f4058911e7b6 100644
--- a/OvmfPkg/Library/BaseMemEncryptSevLib/PeiMemEncryptSevLib.inf
+++ b/OvmfPkg/Library/BaseMemEncryptSevLib/PeiMemEncryptSevLib.inf
@@ -58,3 +58,5 @@ [FeaturePcd]
 
 [FixedPcd]
   gUefiCpuPkgTokenSpaceGuid.PcdSevEsWorkAreaBase
+  gUefiOvmfPkgTokenSpaceGuid.PcdOvmfSnpHypervisorPreValidatedEnd
+  gUefiOvmfPkgTokenSpaceGuid.PcdOvmfSnpHypervisorPreValidatedStart
diff --git a/OvmfPkg/Library/BaseMemEncryptSevLib/X64/PeiSnpSystemRamValidate.c b/OvmfPkg/Library/BaseMemEncryptSevLib/X64/PeiSnpSystemRamValidate.c
index 64aab7f45b6d..3e692a3b869d 100644
--- a/OvmfPkg/Library/BaseMemEncryptSevLib/X64/PeiSnpSystemRamValidate.c
+++ b/OvmfPkg/Library/BaseMemEncryptSevLib/X64/PeiSnpSystemRamValidate.c
@@ -14,6 +14,44 @@
 
 #include "SnpPageStateChange.h"
 
+typedef struct {
+  UINT64    StartAddress;
+  UINT64    EndAddress;
+} SNP_PRE_VALIDATED_RANGE;
+
+STATIC SNP_PRE_VALIDATED_RANGE mPreValidatedRange[] = {
+  // This range is pre-validated by the Hypervisor.
+  {
+    FixedPcdGet32 (PcdOvmfSnpHypervisorPreValidatedStart),
+    FixedPcdGet32 (PcdOvmfSnpHypervisorPreValidatedEnd)
+  }
+};
+
+STATIC
+BOOLEAN
+DetectPreValidatedOverLap (
+  IN    PHYSICAL_ADDRESS            StartAddress,
+  IN    PHYSICAL_ADDRESS            EndAddress,
+  OUT   SNP_PRE_VALIDATED_RANGE     *OverlapRange
+  )
+{
+  UINTN               i;
+
+  //
+  // Check if the specified address range exist in pre-validated array.
+  //
+  for (i = 0; i < ARRAY_SIZE (mPreValidatedRange); i++) {
+    if ((mPreValidatedRange[i].StartAddress < EndAddress) &&
+        (StartAddress < mPreValidatedRange[i].EndAddress)) {
+      OverlapRange->StartAddress = mPreValidatedRange[i].StartAddress;
+      OverlapRange->EndAddress = mPreValidatedRange[i].EndAddress;
+      return TRUE;
+    }
+  }
+
+  return FALSE;
+}
+
 /**
   Pre-validate the system RAM when SEV-SNP is enabled in the guest VM.
 
@@ -28,9 +66,34 @@ MemEncryptSevSnpPreValidateSystemRam (
   IN UINTN                              NumPages
   )
 {
+  PHYSICAL_ADDRESS          EndAddress;
+  SNP_PRE_VALIDATED_RANGE   OverlapRange;
+
   if (!MemEncryptSevSnpIsEnabled ()) {
     return;
   }
 
-  InternalSetPageState (BaseAddress, NumPages, SevSnpPagePrivate, TRUE);
+  EndAddress = BaseAddress + EFI_PAGES_TO_SIZE (NumPages);
+
+  while (BaseAddress < EndAddress) {
+    //
+    // Check if the range overlaps with the pre-validated ranges.
+    //
+    if (DetectPreValidatedOverLap (BaseAddress, EndAddress, &OverlapRange)) {
+      // Validate the non-overlap regions.
+      if (BaseAddress < OverlapRange.StartAddress) {
+        NumPages = EFI_SIZE_TO_PAGES (OverlapRange.StartAddress - BaseAddress);
+
+        InternalSetPageState (BaseAddress, NumPages, SevSnpPagePrivate, TRUE);
+      }
+
+      BaseAddress = OverlapRange.EndAddress;
+      continue;
+    }
+
+    // Validate the remaining pages.
+    NumPages = EFI_SIZE_TO_PAGES (EndAddress - BaseAddress);
+    InternalSetPageState (BaseAddress, NumPages, SevSnpPagePrivate, TRUE);
+    BaseAddress = EndAddress;
+  }
 }
-- 
2.17.1

[PATCH v6 17/29] OvmfPkg/MemEncryptSevLib: add support to validate > 4GB memory in PEI phase

[Brijesh Singh]

BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=3275

The initial page built during the SEC phase is used by the
MemEncryptSevSnpValidateSystemRam() for the system RAM validation. The
page validation process requires using the PVALIDATE instruction;  the
instruction accepts a virtual address of the memory region that needs
to be validated. If hardware encounters a page table walk failure (due
to page-not-present) then it raises #GP.

The initial page table built in SEC phase address up to 4GB. Add an
internal function to extend the page table to cover > 4GB. The function
builds 1GB entries in the page table for access > 4GB. This will provide
the support to call PVALIDATE instruction for the virtual address >
4GB in PEI phase.

Cc: Michael Roth <michael.roth@amd.com>
Cc: James Bottomley <jejb@linux.ibm.com>
Cc: Min Xu <min.m.xu@intel.com>
Cc: Jiewen Yao <jiewen.yao@intel.com>
Cc: Tom Lendacky <thomas.lendacky@amd.com>
Cc: Jordan Justen <jordan.l.justen@intel.com>
Cc: Ard Biesheuvel <ardb+tianocore@kernel.org>
Cc: Erdem Aktas <erdemaktas@google.com>
Signed-off-by: Brijesh Singh <brijesh.singh@amd.com>
---
 .../BaseMemEncryptSevLib/X64/VirtualMemory.h  |  19 +++
 .../X64/PeiDxeVirtualMemory.c                 | 115 ++++++++++++++++++
 .../X64/PeiSnpSystemRamValidate.c             |  22 ++++
 3 files changed, 156 insertions(+)

diff --git a/OvmfPkg/Library/BaseMemEncryptSevLib/X64/VirtualMemory.h b/OvmfPkg/Library/BaseMemEncryptSevLib/X64/VirtualMemory.h
index 21bbbd1c4f9c..aefef68c30c0 100644
--- a/OvmfPkg/Library/BaseMemEncryptSevLib/X64/VirtualMemory.h
+++ b/OvmfPkg/Library/BaseMemEncryptSevLib/X64/VirtualMemory.h
@@ -143,4 +143,23 @@ InternalMemEncryptSevClearMmioPageEncMask (
   IN  PHYSICAL_ADDRESS        PhysicalAddress,
   IN  UINTN                   Length
   );
+
+/**
+  Create 1GB identity mapping for the specified virtual address range.
+
+  @param[in]  Cr3BaseAddress          Cr3 Base Address (if zero then use
+                                      current CR3)
+  @param[in]  VirtualAddress          Virtual address
+  @param[in]  Length                  Length of virtual address range
+
+  @retval RETURN_INVALID_PARAMETER    Number of pages is zero.
+
+**/
+RETURN_STATUS
+EFIAPI
+InternalMemEncryptSevCreateIdentityMap1G (
+  IN    PHYSICAL_ADDRESS      Cr3BaseAddress,
+  IN    PHYSICAL_ADDRESS      PhysicalAddress,
+  IN    UINTN                 Length
+  );
 #endif
diff --git a/OvmfPkg/Library/BaseMemEncryptSevLib/X64/PeiDxeVirtualMemory.c b/OvmfPkg/Library/BaseMemEncryptSevLib/X64/PeiDxeVirtualMemory.c
index c696745f9d26..f146f6d61cc5 100644
--- a/OvmfPkg/Library/BaseMemEncryptSevLib/X64/PeiDxeVirtualMemory.c
+++ b/OvmfPkg/Library/BaseMemEncryptSevLib/X64/PeiDxeVirtualMemory.c
@@ -536,6 +536,121 @@ EnableReadOnlyPageWriteProtect (
   AsmWriteCr0 (AsmReadCr0() | BIT16);
 }
 
+RETURN_STATUS
+EFIAPI
+InternalMemEncryptSevCreateIdentityMap1G (
+  IN    PHYSICAL_ADDRESS      Cr3BaseAddress,
+  IN    PHYSICAL_ADDRESS      PhysicalAddress,
+  IN    UINTN                 Length
+  )
+{
+  PAGE_MAP_AND_DIRECTORY_POINTER *PageMapLevel4Entry;
+  PAGE_TABLE_1G_ENTRY            *PageDirectory1GEntry;
+  UINT64                         PgTableMask;
+  UINT64                         AddressEncMask;
+  BOOLEAN                        IsWpEnabled;
+  RETURN_STATUS                  Status;
+
+  //
+  // Set PageMapLevel4Entry to suppress incorrect compiler/analyzer warnings.
+  //
+  PageMapLevel4Entry = NULL;
+
+  DEBUG ((
+    DEBUG_VERBOSE,
+    "%a:%a: Cr3Base=0x%Lx Physical=0x%Lx Length=0x%Lx\n",
+    gEfiCallerBaseName,
+    __FUNCTION__,
+    Cr3BaseAddress,
+    PhysicalAddress,
+    (UINT64)Length
+    ));
+
+  if (Length == 0) {
+    return RETURN_INVALID_PARAMETER;
+  }
+
+  //
+  // Check if we have a valid memory encryption mask
+  //
+  AddressEncMask = InternalGetMemEncryptionAddressMask ();
+  if (!AddressEncMask) {
+    return RETURN_ACCESS_DENIED;
+  }
+
+  PgTableMask = AddressEncMask | EFI_PAGE_MASK;
+
+
+  //
+  // Make sure that the page table is changeable.
+  //
+  IsWpEnabled = IsReadOnlyPageWriteProtected ();
+  if (IsWpEnabled) {
+    DisableReadOnlyPageWriteProtect ();
+  }
+
+  Status = EFI_SUCCESS;
+
+  while (Length)
+  {
+    //
+    // If Cr3BaseAddress is not specified then read the current CR3
+    //
+    if (Cr3BaseAddress == 0) {
+      Cr3BaseAddress = AsmReadCr3();
+    }
+
+    PageMapLevel4Entry = (VOID*) (Cr3BaseAddress & ~PgTableMask);
+    PageMapLevel4Entry += PML4_OFFSET(PhysicalAddress);
+    if (!PageMapLevel4Entry->Bits.Present) {
+      DEBUG ((
+        DEBUG_ERROR,
+        "%a:%a: bad PML4 for Physical=0x%Lx\n",
+        gEfiCallerBaseName,
+        __FUNCTION__,
+        PhysicalAddress
+        ));
+      Status = RETURN_NO_MAPPING;
+      goto Done;
+    }
+
+    PageDirectory1GEntry = (VOID *)(
+                             (PageMapLevel4Entry->Bits.PageTableBaseAddress <<
+                              12) & ~PgTableMask
+                             );
+    PageDirectory1GEntry += PDP_OFFSET(PhysicalAddress);
+    if (!PageDirectory1GEntry->Bits.Present) {
+      PageDirectory1GEntry->Bits.Present = 1;
+      PageDirectory1GEntry->Bits.MustBe1 = 1;
+      PageDirectory1GEntry->Bits.MustBeZero = 0;
+      PageDirectory1GEntry->Bits.ReadWrite = 1;
+      PageDirectory1GEntry->Uint64 |= (UINT64)PhysicalAddress | AddressEncMask;
+    }
+
+    if (Length <= BIT30) {
+      Length = 0;
+    } else {
+      Length -= BIT30;
+    }
+
+    PhysicalAddress += BIT30;
+  }
+
+  //
+  // Flush TLB
+  //
+  CpuFlushTlb();
+
+Done:
+  //
+  // Restore page table write protection, if any.
+  //
+  if (IsWpEnabled) {
+    EnableReadOnlyPageWriteProtect ();
+  }
+
+  return Status;
+}
 
 /**
   This function either sets or clears memory encryption bit for the memory
diff --git a/OvmfPkg/Library/BaseMemEncryptSevLib/X64/PeiSnpSystemRamValidate.c b/OvmfPkg/Library/BaseMemEncryptSevLib/X64/PeiSnpSystemRamValidate.c
index 3e692a3b869d..69ffb79633c4 100644
--- a/OvmfPkg/Library/BaseMemEncryptSevLib/X64/PeiSnpSystemRamValidate.c
+++ b/OvmfPkg/Library/BaseMemEncryptSevLib/X64/PeiSnpSystemRamValidate.c
@@ -10,9 +10,12 @@
 
 #include <Uefi/UefiBaseType.h>
 #include <Library/BaseLib.h>
+#include <Library/PcdLib.h>
+#include <Library/DebugLib.h>
 #include <Library/MemEncryptSevLib.h>
 
 #include "SnpPageStateChange.h"
+#include "VirtualMemory.h"
 
 typedef struct {
   UINT64    StartAddress;
@@ -68,6 +71,7 @@ MemEncryptSevSnpPreValidateSystemRam (
 {
   PHYSICAL_ADDRESS          EndAddress;
   SNP_PRE_VALIDATED_RANGE   OverlapRange;
+  EFI_STATUS                Status;
 
   if (!MemEncryptSevSnpIsEnabled ()) {
     return;
@@ -75,6 +79,24 @@ MemEncryptSevSnpPreValidateSystemRam (
 
   EndAddress = BaseAddress + EFI_PAGES_TO_SIZE (NumPages);
 
+  //
+  // The page table used in PEI can address up to 4GB memory. If we are asked to
+  // validate a range above the 4GB, then create an identity mapping so that the
+  // PVALIDATE instruction can execute correctly. If the page table entry is not
+  // present then PVALIDATE will #GP.
+  //
+  if (BaseAddress >= SIZE_4GB) {
+    Status = InternalMemEncryptSevCreateIdentityMap1G (
+                0,
+                BaseAddress,
+                EFI_PAGES_TO_SIZE (NumPages)
+                );
+    if (EFI_ERROR (Status)) {
+      ASSERT (FALSE);
+      CpuDeadLoop ();
+    }
+  }
+
   while (BaseAddress < EndAddress) {
     //
     // Check if the range overlaps with the pre-validated ranges.
-- 
2.17.1

[PATCH v6 18/29] OvmfPkg/SecMain: pre-validate the memory used for decompressing Fv

[Brijesh Singh]

BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=3275

The VMM launch sequence should have pre-validated all the data pages used
in the Reset vector. The range does not cover the data pages used during
the SEC phase (mainly PEI and DXE firmware volume decompression memory).

When SEV-SNP is active, the memory must be pre-validated before the access.
Add support to pre-validate the memory range from SnpSecPreValidatedStart
to SnpSecPreValidatedEnd. This should be sufficent to enter into the PEI
phase.

Cc: Michael Roth <michael.roth@amd.com>
Cc: James Bottomley <jejb@linux.ibm.com>
Cc: Min Xu <min.m.xu@intel.com>
Cc: Jiewen Yao <jiewen.yao@intel.com>
Cc: Tom Lendacky <thomas.lendacky@amd.com>
Cc: Jordan Justen <jordan.l.justen@intel.com>
Cc: Ard Biesheuvel <ardb+tianocore@kernel.org>
Cc: Erdem Aktas <erdemaktas@google.com>
Signed-off-by: Brijesh Singh <brijesh.singh@amd.com>
---
 OvmfPkg/OvmfPkg.dec                           |  5 ++++
 .../PeiMemEncryptSevLib.inf                   |  2 ++
 OvmfPkg/Sec/SecMain.inf                       |  3 +++
 OvmfPkg/Sec/AmdSev.h                          | 23 +++++++++++++++++++
 .../X64/PeiSnpSystemRamValidate.c             |  5 ++++
 OvmfPkg/Sec/AmdSev.c                          | 20 +++++++++++++++-
 OvmfPkg/Sec/SecMain.c                         |  7 ++++++
 OvmfPkg/FvmainCompactScratchEnd.fdf.inc       |  5 ++++
 8 files changed, 69 insertions(+), 1 deletion(-)

diff --git a/OvmfPkg/OvmfPkg.dec b/OvmfPkg/OvmfPkg.dec
index 41fffd6d3bd9..d040ee02b94f 100644
--- a/OvmfPkg/OvmfPkg.dec
+++ b/OvmfPkg/OvmfPkg.dec
@@ -358,6 +358,11 @@ [PcdsFixedAtBuild]
   gUefiOvmfPkgTokenSpaceGuid.PcdOvmfSnpHypervisorPreValidatedStart|0x0|UINT32|0x56
   gUefiOvmfPkgTokenSpaceGuid.PcdOvmfSnpHypervisorPreValidatedEnd|0x0|UINT32|0x57
 
+  ## The range of memory that need to be pre-validated in the SEC phase
+  #  when SEV-SNP is active in the guest VM.
+  gUefiOvmfPkgTokenSpaceGuid.PcdOvmfSnpSecPreValidatedStart|0|UINT32|0x58
+  gUefiOvmfPkgTokenSpaceGuid.PcdOvmfSnpSecPreValidatedEnd|0|UINT32|0x59
+
 [PcdsDynamic, PcdsDynamicEx]
   gUefiOvmfPkgTokenSpaceGuid.PcdEmuVariableEvent|0|UINT64|2
   gUefiOvmfPkgTokenSpaceGuid.PcdOvmfFlashVariablesEnable|FALSE|BOOLEAN|0x10
diff --git a/OvmfPkg/Library/BaseMemEncryptSevLib/PeiMemEncryptSevLib.inf b/OvmfPkg/Library/BaseMemEncryptSevLib/PeiMemEncryptSevLib.inf
index f4058911e7b6..2b60920f4b25 100644
--- a/OvmfPkg/Library/BaseMemEncryptSevLib/PeiMemEncryptSevLib.inf
+++ b/OvmfPkg/Library/BaseMemEncryptSevLib/PeiMemEncryptSevLib.inf
@@ -58,5 +58,7 @@ [FeaturePcd]
 
 [FixedPcd]
   gUefiCpuPkgTokenSpaceGuid.PcdSevEsWorkAreaBase
+  gUefiOvmfPkgTokenSpaceGuid.PcdOvmfSnpSecPreValidatedEnd
+  gUefiOvmfPkgTokenSpaceGuid.PcdOvmfSnpSecPreValidatedStart
   gUefiOvmfPkgTokenSpaceGuid.PcdOvmfSnpHypervisorPreValidatedEnd
   gUefiOvmfPkgTokenSpaceGuid.PcdOvmfSnpHypervisorPreValidatedStart
diff --git a/OvmfPkg/Sec/SecMain.inf b/OvmfPkg/Sec/SecMain.inf
index 9523a8ea6c8f..7e2668ff2f66 100644
--- a/OvmfPkg/Sec/SecMain.inf
+++ b/OvmfPkg/Sec/SecMain.inf
@@ -51,6 +51,7 @@ [LibraryClasses]
   PeCoffExtraActionLib
   ExtractGuidedSectionLib
   LocalApicLib
+  MemEncryptSevLib
   CpuExceptionHandlerLib
 
 [Ppis]
@@ -73,6 +74,8 @@ [Pcd]
   gEfiMdeModulePkgTokenSpaceGuid.PcdInitValueInTempStack
   gUefiOvmfPkgTokenSpaceGuid.PcdOvmfConfidentialComputingWorkAreaHeader
   gUefiOvmfPkgTokenSpaceGuid.PcdOvmfWorkAreaBase
+  gUefiOvmfPkgTokenSpaceGuid.PcdOvmfSnpSecPreValidatedStart
+  gUefiOvmfPkgTokenSpaceGuid.PcdOvmfSnpSecPreValidatedEnd
 
 [FeaturePcd]
   gUefiOvmfPkgTokenSpaceGuid.PcdSmmSmramRequire
diff --git a/OvmfPkg/Sec/AmdSev.h b/OvmfPkg/Sec/AmdSev.h
index adad96d23189..33af32b9925b 100644
--- a/OvmfPkg/Sec/AmdSev.h
+++ b/OvmfPkg/Sec/AmdSev.h
@@ -69,4 +69,27 @@ SevEsIsEnabled (
   VOID
   );
 
+/**
+ Pre-validate System RAM used for decompressing the PEI and DXE firmware volumes
+ when SEV-SNP is active. The PCDs SecPreValidatedStart and SecPreValidatedEnd are
+ set in OvmfPkg/FvmainCompactScratchEnd.fdf.inc.
+
+**/
+VOID
+SevSnpSecPreValidateSystemRam (
+  VOID
+  );
+
+/**
+  Determine if SEV-SNP is active.
+
+  @retval TRUE   SEV-SNP is enabled
+  @retval FALSE  SEV-SNP is not enabled
+
+**/
+BOOLEAN
+SevSnpIsEnabled (
+  VOID
+  );
+
 #endif
diff --git a/OvmfPkg/Library/BaseMemEncryptSevLib/X64/PeiSnpSystemRamValidate.c b/OvmfPkg/Library/BaseMemEncryptSevLib/X64/PeiSnpSystemRamValidate.c
index 69ffb79633c4..253d42073907 100644
--- a/OvmfPkg/Library/BaseMemEncryptSevLib/X64/PeiSnpSystemRamValidate.c
+++ b/OvmfPkg/Library/BaseMemEncryptSevLib/X64/PeiSnpSystemRamValidate.c
@@ -27,6 +27,11 @@ STATIC SNP_PRE_VALIDATED_RANGE mPreValidatedRange[] = {
   {
     FixedPcdGet32 (PcdOvmfSnpHypervisorPreValidatedStart),
     FixedPcdGet32 (PcdOvmfSnpHypervisorPreValidatedEnd)
+  },
+  // This range is pre-validated by the Sec/SecMain.c
+  {
+    FixedPcdGet32 (PcdOvmfSnpSecPreValidatedStart),
+    FixedPcdGet32 (PcdOvmfSnpSecPreValidatedEnd)
   }
 };
 
diff --git a/OvmfPkg/Sec/AmdSev.c b/OvmfPkg/Sec/AmdSev.c
index 054f19216f1e..a626dc3943fe 100644
--- a/OvmfPkg/Sec/AmdSev.c
+++ b/OvmfPkg/Sec/AmdSev.c
@@ -55,7 +55,6 @@ SevEsProtocolFailure (
   @retval FALSE  SEV-SNP is not enabled
 
 **/
-STATIC
 BOOLEAN
 SevSnpIsEnabled (
   VOID
@@ -247,3 +246,22 @@ SevEsIsEnabled (
 
   return (SevEsWorkArea->SevEsEnabled != 0);
 }
+
+/**
+ Pre-validate System RAM used for decompressing the PEI and DXE firmware volumes
+ when SEV-SNP is active. The PCDs SecPreValidatedStart and SecPreValidatedEnd are
+ set in OvmfPkg/FvmainCompactScratchEnd.fdf.inc.
+
+**/
+VOID
+SevSnpSecPreValidateSystemRam (
+  VOID
+  )
+{
+  PHYSICAL_ADDRESS        Start, End;
+
+  Start = (EFI_PHYSICAL_ADDRESS) PcdGet32 (PcdOvmfSnpSecPreValidatedStart);
+  End = (EFI_PHYSICAL_ADDRESS) PcdGet32 (PcdOvmfSnpSecPreValidatedEnd);
+
+  MemEncryptSevSnpPreValidateSystemRam (Start, EFI_SIZE_TO_PAGES (End - Start));
+}
diff --git a/OvmfPkg/Sec/SecMain.c b/OvmfPkg/Sec/SecMain.c
index 406e3a25d0cd..0d80494b062c 100644
--- a/OvmfPkg/Sec/SecMain.c
+++ b/OvmfPkg/Sec/SecMain.c
@@ -847,6 +847,13 @@ SecCoreStartupWithStack (
   SecCoreData.BootFirmwareVolumeBase = BootFv;
   SecCoreData.BootFirmwareVolumeSize = (UINTN) BootFv->FvLength;
 
+  if (SevSnpIsEnabled ()) {
+    //
+    // Pre-validate the System RAM used in the SEC Phase
+    //
+    SevSnpSecPreValidateSystemRam ();
+  }
+
   //
   // Make sure the 8259 is masked before initializing the Debug Agent and the debug timer is enabled
   //
diff --git a/OvmfPkg/FvmainCompactScratchEnd.fdf.inc b/OvmfPkg/FvmainCompactScratchEnd.fdf.inc
index 46f52583297c..b560fb0b8e4f 100644
--- a/OvmfPkg/FvmainCompactScratchEnd.fdf.inc
+++ b/OvmfPkg/FvmainCompactScratchEnd.fdf.inc
@@ -63,3 +63,8 @@
 DEFINE DECOMP_SCRATCH_BASE           = (($(DECOMP_SCRATCH_BASE_UNALIGNED) + $(DECOMP_SCRATCH_BASE_ALIGNMENT)) & $(DECOMP_SCRATCH_BASE_MASK))
 
 SET gUefiOvmfPkgTokenSpaceGuid.PcdOvmfDecompressionScratchEnd = $(DECOMP_SCRATCH_BASE) + $(DECOMP_SCRATCH_SIZE)
+
+#
+# The range of pages that should be pre-validated during the SEC phase when SEV-SNP is active in the guest VM.
+SET gUefiOvmfPkgTokenSpaceGuid.PcdOvmfSnpSecPreValidatedStart = $(MEMFD_BASE_ADDRESS) + gUefiOvmfPkgTokenSpaceGuid.PcdOvmfPeiMemFvBase
+SET gUefiOvmfPkgTokenSpaceGuid.PcdOvmfSnpSecPreValidatedEnd = $(DECOMP_SCRATCH_BASE) + $(DECOMP_SCRATCH_SIZE)
-- 
2.17.1

[PATCH v6 19/29] OvmfPkg/PlatformPei: validate the system RAM when SNP is active

[Brijesh Singh]

BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=3275

When SEV-SNP is active, a memory region mapped encrypted in the page
table must be validated before access. There are two approaches that
can be taken to validate the system RAM detected during the PEI phase:

1) Validate on-demand
OR
2) Validate before access

On-demand
=========
If memory is not validated before access, it will cause a #VC
exception with the page-not-validated error code. The VC exception
handler can perform the validation steps.

The pages that have been validated will need to be tracked to avoid
the double validation scenarios. The range of memory that has not
been validated will need to be communicated to the OS through the
recently introduced unaccepted memory type
https://github.com/microsoft/mu_basecore/pull/66, so that OS can
validate those ranges before using them.

Validate before access
======================
Since the PEI phase detects all the available system RAM, use the
MemEncryptSevSnpValidateSystemRam() function to pre-validate the
system RAM in the PEI phase.

For now, choose option 2 due to the dependency and the complexity
of the on-demand validation.

Cc: Michael Roth <michael.roth@amd.com>
Cc: James Bottomley <jejb@linux.ibm.com>
Cc: Min Xu <min.m.xu@intel.com>
Cc: Jiewen Yao <jiewen.yao@intel.com>
Cc: Tom Lendacky <thomas.lendacky@amd.com>
Cc: Jordan Justen <jordan.l.justen@intel.com>
Cc: Ard Biesheuvel <ardb+tianocore@kernel.org>
Cc: Erdem Aktas <erdemaktas@google.com>
Signed-off-by: Brijesh Singh <brijesh.singh@amd.com>
---
 OvmfPkg/PlatformPei/AmdSev.c | 42 ++++++++++++++++++++++++++++++++++++
 1 file changed, 42 insertions(+)

diff --git a/OvmfPkg/PlatformPei/AmdSev.c b/OvmfPkg/PlatformPei/AmdSev.c
index de876fdb478e..391e7bbb7dbd 100644
--- a/OvmfPkg/PlatformPei/AmdSev.c
+++ b/OvmfPkg/PlatformPei/AmdSev.c
@@ -23,6 +23,40 @@
 
 #include "Platform.h"
 
+/**
+  Initialize SEV-SNP support if running as an SEV-SNP guest.
+
+**/
+STATIC
+VOID
+AmdSevSnpInitialize (
+  VOID
+  )
+{
+  EFI_PEI_HOB_POINTERS          Hob;
+  EFI_HOB_RESOURCE_DESCRIPTOR   *ResourceHob;
+
+  if (!MemEncryptSevSnpIsEnabled ()) {
+    return;
+  }
+
+  //
+  // Iterate through the system RAM and validate it.
+  //
+  for (Hob.Raw = GetHobList (); !END_OF_HOB_LIST (Hob); Hob.Raw = GET_NEXT_HOB (Hob)) {
+    if (Hob.Raw != NULL && GET_HOB_TYPE (Hob) == EFI_HOB_TYPE_RESOURCE_DESCRIPTOR) {
+      ResourceHob = Hob.ResourceDescriptor;
+
+      if (ResourceHob->ResourceType == EFI_RESOURCE_SYSTEM_MEMORY) {
+        MemEncryptSevSnpPreValidateSystemRam (
+          ResourceHob->PhysicalStart,
+          EFI_SIZE_TO_PAGES ((UINTN) ResourceHob->ResourceLength)
+          );
+      }
+    }
+  }
+}
+
 /**
   Handle an SEV-SNP/GHCB protocol check failure.
 
@@ -240,6 +274,14 @@ AmdSevInitialize (
     return;
   }
 
+  //
+  // Check and perform SEV-SNP initialization if required. This need to be
+  // done before the GHCB page is made shared in the AmdSevEsInitialize(). This
+  // is because the system RAM must be validated before it is made shared.
+  // The AmdSevSnpInitialize() validates the system RAM.
+  //
+  AmdSevSnpInitialize ();
+
   //
   // Set Memory Encryption Mask PCD
   //
-- 
2.17.1

[PATCH v6 20/29] OvmfPkg/PlatformPei: set the SEV-SNP enabled PCD

[Brijesh Singh]

BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=3275

The MpInitLib uses the PcdSevSnpIsEnabled to determine whether the SEV-SNP
is active. If the SEV-SNP is active, then set the PCD to TRUE.

Cc: Michael Roth <michael.roth@amd.com>
Cc: James Bottomley <jejb@linux.ibm.com>
Cc: Min Xu <min.m.xu@intel.com>
Cc: Jiewen Yao <jiewen.yao@intel.com>
Cc: Tom Lendacky <thomas.lendacky@amd.com>
Cc: Jordan Justen <jordan.l.justen@intel.com>
Cc: Ard Biesheuvel <ardb+tianocore@kernel.org>
Cc: Erdem Aktas <erdemaktas@google.com>
Signed-off-by: Brijesh Singh <brijesh.singh@amd.com>
---
 OvmfPkg/PlatformPei/PlatformPei.inf | 1 +
 OvmfPkg/PlatformPei/AmdSev.c        | 4 ++++
 2 files changed, 5 insertions(+)

diff --git a/OvmfPkg/PlatformPei/PlatformPei.inf b/OvmfPkg/PlatformPei/PlatformPei.inf
index 67eb7aa7166b..c8c89ab648d0 100644
--- a/OvmfPkg/PlatformPei/PlatformPei.inf
+++ b/OvmfPkg/PlatformPei/PlatformPei.inf
@@ -106,6 +106,7 @@ [Pcd]
   gUefiCpuPkgTokenSpaceGuid.PcdCpuBootLogicalProcessorNumber
   gUefiCpuPkgTokenSpaceGuid.PcdCpuApStackSize
   gUefiCpuPkgTokenSpaceGuid.PcdSevEsIsEnabled
+  gUefiCpuPkgTokenSpaceGuid.PcdSevSnpIsEnabled
 
 [FixedPcd]
   gEfiMdePkgTokenSpaceGuid.PcdPciExpressBaseAddress
diff --git a/OvmfPkg/PlatformPei/AmdSev.c b/OvmfPkg/PlatformPei/AmdSev.c
index 391e7bbb7dbd..ec577ef3efcc 100644
--- a/OvmfPkg/PlatformPei/AmdSev.c
+++ b/OvmfPkg/PlatformPei/AmdSev.c
@@ -33,6 +33,7 @@ AmdSevSnpInitialize (
   VOID
   )
 {
+  RETURN_STATUS                 PcdStatus;
   EFI_PEI_HOB_POINTERS          Hob;
   EFI_HOB_RESOURCE_DESCRIPTOR   *ResourceHob;
 
@@ -40,6 +41,9 @@ AmdSevSnpInitialize (
     return;
   }
 
+  PcdStatus = PcdSetBoolS (PcdSevSnpIsEnabled, TRUE);
+  ASSERT_RETURN_ERROR (PcdStatus);
+
   //
   // Iterate through the system RAM and validate it.
   //
-- 
2.17.1

[PATCH v6 21/29] OvmfPkg/PlatformPei: set the Hypervisor Features PCD

[Brijesh Singh]

BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=3275

Version 2 of the GHCB specification added the support to query the
hypervisor feature bitmap. The feature bitmap provide information
such as whether to use the AP create VmgExit or use the AP jump table
approach to create the APs. The MpInitLib will use the
PcdGhcbHypervisorFeatures to determine which method to use for creating
the AP.

Query the hypervisor feature and set the PCD accordingly.

Cc: Michael Roth <michael.roth@amd.com>
Cc: James Bottomley <jejb@linux.ibm.com>
Cc: Min Xu <min.m.xu@intel.com>
Cc: Jiewen Yao <jiewen.yao@intel.com>
Cc: Tom Lendacky <thomas.lendacky@amd.com>
Cc: Jordan Justen <jordan.l.justen@intel.com>
Cc: Ard Biesheuvel <ardb+tianocore@kernel.org>
Cc: Erdem Aktas <erdemaktas@google.com>
Signed-off-by: Brijesh Singh <brijesh.singh@amd.com>
---
 OvmfPkg/PlatformPei/PlatformPei.inf |  3 ++
 OvmfPkg/PlatformPei/AmdSev.c        | 55 +++++++++++++++++++++++++++++
 2 files changed, 58 insertions(+)

diff --git a/OvmfPkg/PlatformPei/PlatformPei.inf b/OvmfPkg/PlatformPei/PlatformPei.inf
index c8c89ab648d0..8c9795095a10 100644
--- a/OvmfPkg/PlatformPei/PlatformPei.inf
+++ b/OvmfPkg/PlatformPei/PlatformPei.inf
@@ -62,6 +62,7 @@ [LibraryClasses]
   MtrrLib
   MemEncryptSevLib
   PcdLib
+  VmgExitLib
 
 [Pcd]
   gUefiOvmfPkgTokenSpaceGuid.PcdOvmfPeiMemFvBase
@@ -107,6 +108,8 @@ [Pcd]
   gUefiCpuPkgTokenSpaceGuid.PcdCpuApStackSize
   gUefiCpuPkgTokenSpaceGuid.PcdSevEsIsEnabled
   gUefiCpuPkgTokenSpaceGuid.PcdSevSnpIsEnabled
+  gUefiCpuPkgTokenSpaceGuid.PcdGhcbHypervisorFeatures
+
 
 [FixedPcd]
   gEfiMdePkgTokenSpaceGuid.PcdPciExpressBaseAddress
diff --git a/OvmfPkg/PlatformPei/AmdSev.c b/OvmfPkg/PlatformPei/AmdSev.c
index ec577ef3efcc..b488cd5aed9b 100644
--- a/OvmfPkg/PlatformPei/AmdSev.c
+++ b/OvmfPkg/PlatformPei/AmdSev.c
@@ -23,6 +23,12 @@
 
 #include "Platform.h"
 
+STATIC
+UINT64
+GetHypervisorFeature (
+  VOID
+  );
+
 /**
   Initialize SEV-SNP support if running as an SEV-SNP guest.
 
@@ -36,6 +42,7 @@ AmdSevSnpInitialize (
   RETURN_STATUS                 PcdStatus;
   EFI_PEI_HOB_POINTERS          Hob;
   EFI_HOB_RESOURCE_DESCRIPTOR   *ResourceHob;
+  UINT64                        HvFeatures;
 
   if (!MemEncryptSevSnpIsEnabled ()) {
     return;
@@ -44,6 +51,15 @@ AmdSevSnpInitialize (
   PcdStatus = PcdSetBoolS (PcdSevSnpIsEnabled, TRUE);
   ASSERT_RETURN_ERROR (PcdStatus);
 
+  //
+  // Query the hypervisor feature using the VmgExit and set the value in the
+  // hypervisor features PCD.
+  //
+  HvFeatures = GetHypervisorFeature ();
+  PcdStatus = PcdSet64S (PcdGhcbHypervisorFeatures, HvFeatures);
+  ASSERT_RETURN_ERROR (PcdStatus);
+
+
   //
   // Iterate through the system RAM and validate it.
   //
@@ -94,6 +110,45 @@ SevEsProtocolFailure (
   CpuDeadLoop ();
 }
 
+/**
+ Get the hypervisor features bitmap
+
+**/
+STATIC
+UINT64
+GetHypervisorFeature (
+  VOID
+  )
+{
+  RETURN_STATUS                   Status;
+  GHCB                            *Ghcb;
+  MSR_SEV_ES_GHCB_REGISTER        Msr;
+  BOOLEAN                         InterruptState;
+  UINT64                          Features;
+
+  Msr.GhcbPhysicalAddress = AsmReadMsr64 (MSR_SEV_ES_GHCB);
+  Ghcb = Msr.Ghcb;
+
+  //
+  // Initialize the GHCB
+  //
+  VmgInit (Ghcb, &InterruptState);
+
+  //
+  // Query the Hypervisor Features.
+  //
+  Status = VmgExit (Ghcb, SVM_EXIT_HYPERVISOR_FEATURES, 0, 0);
+  if ((Status != 0)) {
+    SevEsProtocolFailure (GHCB_TERMINATE_GHCB_GENERAL);
+  }
+
+  Features = Ghcb->SaveArea.SwExitInfo2;
+
+  VmgDone (Ghcb, InterruptState);
+
+  return Features;
+}
+
 /**
 
   This function can be used to register the GHCB GPA.
-- 
2.17.1

[PATCH v6 22/29] MdePkg/GHCB: increase the GHCB protocol max version

[Brijesh Singh]

BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=3275

Now that OvmfPkg supports version 2 of the GHCB specification, bump the
protocol version.

Cc: Michael Roth <michael.roth@amd.com>
Cc: Ray Ni <ray.ni@intel.com>
Cc: Rahul Kumar <rahul1.kumar@intel.com>
Cc: Eric Dong <eric.dong@intel.com>
Cc: James Bottomley <jejb@linux.ibm.com>
Cc: Min Xu <min.m.xu@intel.com>
Cc: Jiewen Yao <jiewen.yao@intel.com>
Cc: Tom Lendacky <thomas.lendacky@amd.com>
Cc: Jordan Justen <jordan.l.justen@intel.com>
Cc: Ard Biesheuvel <ardb+tianocore@kernel.org>
Cc: Erdem Aktas <erdemaktas@google.com>
Signed-off-by: Brijesh Singh <brijesh.singh@amd.com>
---
 MdePkg/Include/Register/Amd/Ghcb.h | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/MdePkg/Include/Register/Amd/Ghcb.h b/MdePkg/Include/Register/Amd/Ghcb.h
index 8c5f46e4bb53..071aae0c9e09 100644
--- a/MdePkg/Include/Register/Amd/Ghcb.h
+++ b/MdePkg/Include/Register/Amd/Ghcb.h
@@ -24,7 +24,7 @@
 #define VC_EXCEPTION 29
 
 #define GHCB_VERSION_MIN     1
-#define GHCB_VERSION_MAX     1
+#define GHCB_VERSION_MAX     2
 
 #define GHCB_STANDARD_USAGE  0
 
-- 
2.17.1

[PATCH v6 23/29] UefiCpuPkg/MpLib: add support to register GHCB GPA when SEV-SNP is enabled

[Brijesh Singh]

BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=3275

An SEV-SNP guest requires that the physical address of the GHCB must
be registered with the hypervisor before using it. See the GHCB
specification section 2.3.2 for more details.

Cc: Michael Roth <michael.roth@amd.com>
Cc: Eric Dong <eric.dong@intel.com>
Cc: Ray Ni <ray.ni@intel.com>
Cc: Rahul Kumar <rahul1.kumar@intel.com>
Cc: James Bottomley <jejb@linux.ibm.com>
Cc: Min Xu <min.m.xu@intel.com>
Cc: Jiewen Yao <jiewen.yao@intel.com>
Cc: Tom Lendacky <thomas.lendacky@amd.com>
Cc: Jordan Justen <jordan.l.justen@intel.com>
Cc: Ard Biesheuvel <ardb+tianocore@kernel.org>
Cc: Erdem Aktas <erdemaktas@google.com>
Signed-off-by: Brijesh Singh <brijesh.singh@amd.com>
---
 UefiCpuPkg/Library/MpInitLib/DxeMpInitLib.inf |  1 +
 UefiCpuPkg/Library/MpInitLib/PeiMpInitLib.inf |  1 +
 UefiCpuPkg/Library/MpInitLib/MpLib.h          |  2 +
 UefiCpuPkg/Library/MpInitLib/MpLib.c          |  2 +
 UefiCpuPkg/Library/MpInitLib/MpEqu.inc        |  1 +
 UefiCpuPkg/Library/MpInitLib/X64/MpFuncs.nasm | 51 +++++++++++++++++++
 6 files changed, 58 insertions(+)

diff --git a/UefiCpuPkg/Library/MpInitLib/DxeMpInitLib.inf b/UefiCpuPkg/Library/MpInitLib/DxeMpInitLib.inf
index d34419c2a524..48d7dfa4450f 100644
--- a/UefiCpuPkg/Library/MpInitLib/DxeMpInitLib.inf
+++ b/UefiCpuPkg/Library/MpInitLib/DxeMpInitLib.inf
@@ -76,3 +76,4 @@ [Pcd]
   gUefiCpuPkgTokenSpaceGuid.PcdSevEsWorkAreaBase                       ## SOMETIMES_CONSUMES
   gEfiMdeModulePkgTokenSpaceGuid.PcdCpuStackGuard                      ## CONSUMES
   gEfiMdeModulePkgTokenSpaceGuid.PcdGhcbBase                           ## CONSUMES
+  gUefiCpuPkgTokenSpaceGuid.PcdSevSnpIsEnabled                         ## CONSUMES
diff --git a/UefiCpuPkg/Library/MpInitLib/PeiMpInitLib.inf b/UefiCpuPkg/Library/MpInitLib/PeiMpInitLib.inf
index 36fcb96b5852..ab8279df596f 100644
--- a/UefiCpuPkg/Library/MpInitLib/PeiMpInitLib.inf
+++ b/UefiCpuPkg/Library/MpInitLib/PeiMpInitLib.inf
@@ -65,6 +65,7 @@ [Pcd]
   gUefiCpuPkgTokenSpaceGuid.PcdSevEsIsEnabled                      ## CONSUMES
   gUefiCpuPkgTokenSpaceGuid.PcdSevEsWorkAreaBase                   ## SOMETIMES_CONSUMES
   gEfiMdeModulePkgTokenSpaceGuid.PcdGhcbBase                       ## CONSUMES
+  gUefiCpuPkgTokenSpaceGuid.PcdSevSnpIsEnabled                     ## CONSUMES
 
 [Ppis]
   gEdkiiPeiShadowMicrocodePpiGuid        ## SOMETIMES_CONSUMES
diff --git a/UefiCpuPkg/Library/MpInitLib/MpLib.h b/UefiCpuPkg/Library/MpInitLib/MpLib.h
index e88a5355c983..4abaa2243d0a 100644
--- a/UefiCpuPkg/Library/MpInitLib/MpLib.h
+++ b/UefiCpuPkg/Library/MpInitLib/MpLib.h
@@ -218,6 +218,7 @@ typedef struct {
   //
   BOOLEAN               Enable5LevelPaging;
   BOOLEAN               SevEsIsEnabled;
+  BOOLEAN               SevSnpIsEnabled;
   UINTN                 GhcbBase;
 } MP_CPU_EXCHANGE_INFO;
 
@@ -287,6 +288,7 @@ struct _CPU_MP_DATA {
   BOOLEAN                        WakeUpByInitSipiSipi;
 
   BOOLEAN                        SevEsIsEnabled;
+  BOOLEAN                        SevSnpIsEnabled;
   UINTN                          SevEsAPBuffer;
   UINTN                          SevEsAPResetStackStart;
   CPU_MP_DATA                    *NewCpuMpData;
diff --git a/UefiCpuPkg/Library/MpInitLib/MpLib.c b/UefiCpuPkg/Library/MpInitLib/MpLib.c
index b9a06747edbf..586cff2f6813 100644
--- a/UefiCpuPkg/Library/MpInitLib/MpLib.c
+++ b/UefiCpuPkg/Library/MpInitLib/MpLib.c
@@ -1040,6 +1040,7 @@ FillExchangeInfoData (
   DEBUG ((DEBUG_INFO, "%a: 5-Level Paging = %d\n", gEfiCallerBaseName, ExchangeInfo->Enable5LevelPaging));
 
   ExchangeInfo->SevEsIsEnabled  = CpuMpData->SevEsIsEnabled;
+  ExchangeInfo->SevSnpIsEnabled = CpuMpData->SevSnpIsEnabled;
   ExchangeInfo->GhcbBase        = (UINTN) CpuMpData->GhcbBase;
 
   //
@@ -2033,6 +2034,7 @@ MpInitLibInitialize (
   CpuMpData->CpuInfoInHob     = (UINT64) (UINTN) (CpuMpData->CpuData + MaxLogicalProcessorNumber);
   InitializeSpinLock(&CpuMpData->MpLock);
   CpuMpData->SevEsIsEnabled = PcdGetBool (PcdSevEsIsEnabled);
+  CpuMpData->SevSnpIsEnabled = PcdGetBool (PcdSevSnpIsEnabled);
   CpuMpData->SevEsAPBuffer  = (UINTN) -1;
   CpuMpData->GhcbBase       = PcdGet64 (PcdGhcbBase);
 
diff --git a/UefiCpuPkg/Library/MpInitLib/MpEqu.inc b/UefiCpuPkg/Library/MpInitLib/MpEqu.inc
index 2e9368a374a4..01668638f245 100644
--- a/UefiCpuPkg/Library/MpInitLib/MpEqu.inc
+++ b/UefiCpuPkg/Library/MpInitLib/MpEqu.inc
@@ -92,6 +92,7 @@ struc MP_CPU_EXCHANGE_INFO
   .ModeHighSegment:              CTYPE_UINT16 1
   .Enable5LevelPaging:           CTYPE_BOOLEAN 1
   .SevEsIsEnabled:               CTYPE_BOOLEAN 1
+  .SevSnpIsEnabled               CTYPE_BOOLEAN 1
   .GhcbBase:                     CTYPE_UINTN 1
 endstruc
 
diff --git a/UefiCpuPkg/Library/MpInitLib/X64/MpFuncs.nasm b/UefiCpuPkg/Library/MpInitLib/X64/MpFuncs.nasm
index 50df802d1fca..19939c093d2e 100644
--- a/UefiCpuPkg/Library/MpInitLib/X64/MpFuncs.nasm
+++ b/UefiCpuPkg/Library/MpInitLib/X64/MpFuncs.nasm
@@ -194,9 +194,60 @@ LongModeStart:
     mov        rdx, rax
     shr        rdx, 32
     mov        rcx, 0xc0010130
+
+    ;
+    ; Register GHCB GPA when SEV-SNP is enabled
+    ;
+    lea        edi, [esi + MP_CPU_EXCHANGE_INFO_FIELD (SevSnpIsEnabled)]
+    cmp        byte [edi], 1        ; SevSnpIsEnabled
+    jne        SetGhcbAddress
+
+    ; Save the rdi and rsi to used for later comparison
+    push       rdi
+    push       rsi
+    mov        edi, eax
+    mov        esi, edx
+    or         eax, 18              ; Ghcb registration request
+    wrmsr
+    rep vmmcall
+    rdmsr
+    mov        r12, rax
+    and        r12, 0fffh
+    cmp        r12, 19              ; Ghcb registration response
+    jne        GhcbGpaRegisterFailure
+
+    ; Verify that GPA is not changed
+    and        eax, 0fffff000h
+    cmp        edi, eax
+    jne        GhcbGpaRegisterFailure
+    cmp        esi, edx
+    jne        GhcbGpaRegisterFailure
+    pop        rsi
+    pop        rdi
+
+    ;
+    ; Program GHCB
+    ;
+SetGhcbAddress:
     wrmsr
     jmp        CProcedureInvoke
 
+    ;
+    ; Request the guest termination
+    ;
+GhcbGpaRegisterFailure:
+    xor        edx, edx
+    mov        eax, 256             ; GHCB terminate
+    wrmsr
+    rep vmmcall
+
+    ; We should not return from the above terminate request, but if we do
+    ; then enter into the hlt loop.
+DoHltLoop:
+    cli
+    hlt
+    jmp        DoHltLoop
+
 GetApicId:
     lea        edi, [esi + MP_CPU_EXCHANGE_INFO_FIELD (SevEsIsEnabled)]
     cmp        byte [edi], 1        ; SevEsIsEnabled
-- 
2.17.1

[PATCH v6 24/29] UefiCpuPkg/MpInitLib: use BSP to do extended topology check

[Brijesh Singh]

From: Michael Roth <michael.roth@amd.com>

During AP bringup, just after switching to long mode, APs will do some
cpuid calls to verify that the extended topology leaf (0xB) is available
so they can fetch their x2 APIC IDs from it. In the case of SEV-ES,
these cpuid instructions must be handled by direct use of the GHCB MSR
protocol to fetch the values from the hypervisor, since a #VC handler
is not yet available due to the AP's stack not being set up yet.

For SEV-SNP, rather than relying on the GHCB MSR protocol, it is
expected that these values would be obtained from the SEV-SNP CPUID
table instead. The actual x2 APIC ID (and 8-bit APIC IDs) would still
be fetched from hypervisor using the GHCB MSR protocol however, so
introducing support for the SEV-SNP CPUID table in that part of the AP
bring-up code would only be to handle the checks/validation of the
extended topology leaf.

Rather than introducing all the added complexity needed to handle these
checks via the CPUID table, instead let the BSP do the check in advance,
since it can make use of the #VC handler to avoid the need to scan the
SNP CPUID table directly, and add a flag in ExchangeInfo to communicate
the result of this check to APs.

Cc: Eric Dong <eric.dong@intel.com>
Cc: Ray Ni <ray.ni@intel.com>
Cc: Rahul Kumar <rahul1.kumar@intel.com>
Cc: James Bottomley <jejb@linux.ibm.com>
Cc: Min Xu <min.m.xu@intel.com>
Cc: Jiewen Yao <jiewen.yao@intel.com>
Cc: Tom Lendacky <thomas.lendacky@amd.com>
Cc: Jordan Justen <jordan.l.justen@intel.com>
Cc: Ard Biesheuvel <ardb+tianocore@kernel.org>
Cc: Erdem Aktas <erdemaktas@google.com>
Suggested-by: Brijesh Singh <brijesh.singh@amd.com>
Signed-off-by: Michael Roth <michael.roth@amd.com>
Signed-off-by: Brijesh Singh <brijesh.singh@amd.com>
---
 UefiCpuPkg/Library/MpInitLib/MpLib.h          |  1 +
 UefiCpuPkg/Library/MpInitLib/MpLib.c          | 11 ++++++++
 UefiCpuPkg/Library/MpInitLib/MpEqu.inc        |  1 +
 UefiCpuPkg/Library/MpInitLib/X64/MpFuncs.nasm | 27 +++++++++++++++++++
 4 files changed, 40 insertions(+)

diff --git a/UefiCpuPkg/Library/MpInitLib/MpLib.h b/UefiCpuPkg/Library/MpInitLib/MpLib.h
index 4abaa2243d0a..19e91bf7d74e 100644
--- a/UefiCpuPkg/Library/MpInitLib/MpLib.h
+++ b/UefiCpuPkg/Library/MpInitLib/MpLib.h
@@ -220,6 +220,7 @@ typedef struct {
   BOOLEAN               SevEsIsEnabled;
   BOOLEAN               SevSnpIsEnabled;
   UINTN                 GhcbBase;
+  BOOLEAN               ExtTopoAvail;
 } MP_CPU_EXCHANGE_INFO;
 
 #pragma pack()
diff --git a/UefiCpuPkg/Library/MpInitLib/MpLib.c b/UefiCpuPkg/Library/MpInitLib/MpLib.c
index 586cff2f6813..c2b00a1f04ff 100644
--- a/UefiCpuPkg/Library/MpInitLib/MpLib.c
+++ b/UefiCpuPkg/Library/MpInitLib/MpLib.c
@@ -1004,6 +1004,7 @@ FillExchangeInfoData (
   UINTN                            Size;
   IA32_SEGMENT_DESCRIPTOR          *Selector;
   IA32_CR4                         Cr4;
+  UINT32                           StdRangeMax;
 
   ExchangeInfo                  = CpuMpData->MpCpuExchangeInfo;
   ExchangeInfo->StackStart      = CpuMpData->Buffer;
@@ -1043,6 +1044,16 @@ FillExchangeInfoData (
   ExchangeInfo->SevSnpIsEnabled = CpuMpData->SevSnpIsEnabled;
   ExchangeInfo->GhcbBase        = (UINTN) CpuMpData->GhcbBase;
 
+  if (ExchangeInfo->SevSnpIsEnabled) {
+    AsmCpuid (CPUID_SIGNATURE, &StdRangeMax, NULL, NULL, NULL);
+    if (StdRangeMax >= CPUID_EXTENDED_TOPOLOGY) {
+      CPUID_EXTENDED_TOPOLOGY_EBX ExtTopoEbx;
+
+      AsmCpuid (CPUID_EXTENDED_TOPOLOGY, NULL, &ExtTopoEbx.Uint32, NULL, NULL);
+      ExchangeInfo->ExtTopoAvail = !!ExtTopoEbx.Bits.LogicalProcessors;
+    }
+  }
+
   //
   // Get the BSP's data of GDT and IDT
   //
diff --git a/UefiCpuPkg/Library/MpInitLib/MpEqu.inc b/UefiCpuPkg/Library/MpInitLib/MpEqu.inc
index 01668638f245..aba53f57201c 100644
--- a/UefiCpuPkg/Library/MpInitLib/MpEqu.inc
+++ b/UefiCpuPkg/Library/MpInitLib/MpEqu.inc
@@ -94,6 +94,7 @@ struc MP_CPU_EXCHANGE_INFO
   .SevEsIsEnabled:               CTYPE_BOOLEAN 1
   .SevSnpIsEnabled               CTYPE_BOOLEAN 1
   .GhcbBase:                     CTYPE_UINTN 1
+  .ExtTopoAvail:                 CTYPE_BOOLEAN 1
 endstruc
 
 MP_CPU_EXCHANGE_INFO_OFFSET equ (SwitchToRealProcEnd - RendezvousFunnelProcStart)
diff --git a/UefiCpuPkg/Library/MpInitLib/X64/MpFuncs.nasm b/UefiCpuPkg/Library/MpInitLib/X64/MpFuncs.nasm
index 19939c093d2e..76d07a275c7b 100644
--- a/UefiCpuPkg/Library/MpInitLib/X64/MpFuncs.nasm
+++ b/UefiCpuPkg/Library/MpInitLib/X64/MpFuncs.nasm
@@ -264,6 +264,32 @@ GetApicId:
     or         rax, rdx
     mov        rdi, rax             ; RDI now holds the original GHCB GPA
 
+    ;
+    ; For SEV-SNP, the recommended handling for getting the x2APIC ID
+    ; would be to use the SNP CPUID table to fetch CPUID.00H:EAX and
+    ; CPUID:0BH:EBX[15:0] instead of the GHCB MSR protocol vmgexits
+    ; below.
+    ;
+    ; To avoid the unecessary ugliness to accomplish that here, the BSP
+    ; has performed these checks in advance (where #VC handler handles
+    ; the CPUID table lookups automatically) and cached them in a flag
+    ; so those checks can be skipped here.
+    ;
+    mov        eax, [esi + MP_CPU_EXCHANGE_INFO_FIELD (SevSnpIsEnabled)]
+    cmp        al, 1
+    jne        CheckExtTopoAvail
+
+    ;
+    ; Even with SEV-SNP, the actual x2APIC ID in CPUID.0BH:EDX
+    ; fetched from the hypervisor the same way SEV-ES does it.
+    ;
+    mov        eax, [esi + MP_CPU_EXCHANGE_INFO_FIELD (ExtTopoAvail)]
+    cmp        al, 1
+    je         GetApicIdSevEs
+    ; The 8-bit APIC ID fallback is also the same as with SEV-ES
+    jmp        NoX2ApicSevEs
+
+CheckExtTopoAvail:
     mov        rdx, 0               ; CPUID function 0
     mov        rax, 0               ; RAX register requested
     or         rax, 4
@@ -282,6 +308,7 @@ GetApicId:
     test       edx, 0ffffh
     jz         NoX2ApicSevEs        ; CPUID.0BH:EBX[15:0] is zero
 
+GetApicIdSevEs:
     mov        rdx, 0bh             ; CPUID function 0x0b
     mov        rax, 0c0000000h      ; RDX register requested
     or         rax, 4
-- 
2.17.1

[PATCH v6 25/29] OvmfPkg/MemEncryptSevLib: change the page state in the RMP table

[Brijesh Singh]

BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=3275

The MemEncryptSev{Set,Clear}PageEncMask() functions are used to set or
clear the memory encryption attribute in the page table. When SEV-SNP
is active, we also need to change the page state in the RMP table so that
it is in sync with the memory encryption attribute change.

Cc: Michael Roth <michael.roth@amd.com>
Cc: James Bottomley <jejb@linux.ibm.com>
Cc: Min Xu <min.m.xu@intel.com>
Cc: Jiewen Yao <jiewen.yao@intel.com>
Cc: Tom Lendacky <thomas.lendacky@amd.com>
Cc: Jordan Justen <jordan.l.justen@intel.com>
Cc: Ard Biesheuvel <ardb+tianocore@kernel.org>
Cc: Erdem Aktas <erdemaktas@google.com>
Signed-off-by: Brijesh Singh <brijesh.singh@amd.com>
---
 .../X64/PeiDxeVirtualMemory.c                 | 34 +++++++++++++++++++
 1 file changed, 34 insertions(+)

diff --git a/OvmfPkg/Library/BaseMemEncryptSevLib/X64/PeiDxeVirtualMemory.c b/OvmfPkg/Library/BaseMemEncryptSevLib/X64/PeiDxeVirtualMemory.c
index f146f6d61cc5..56db1e4b6ecf 100644
--- a/OvmfPkg/Library/BaseMemEncryptSevLib/X64/PeiDxeVirtualMemory.c
+++ b/OvmfPkg/Library/BaseMemEncryptSevLib/X64/PeiDxeVirtualMemory.c
@@ -17,6 +17,7 @@
 #include <Register/Cpuid.h>
 
 #include "VirtualMemory.h"
+#include "SnpPageStateChange.h"
 
 STATIC BOOLEAN mAddressEncMaskChecked = FALSE;
 STATIC UINT64  mAddressEncMask;
@@ -695,10 +696,12 @@ SetMemoryEncDec (
   PAGE_MAP_AND_DIRECTORY_POINTER *PageDirectoryPointerEntry;
   PAGE_TABLE_1G_ENTRY            *PageDirectory1GEntry;
   PAGE_TABLE_ENTRY               *PageDirectory2MEntry;
+  PHYSICAL_ADDRESS               OrigPhysicalAddress;
   PAGE_TABLE_4K_ENTRY            *PageTableEntry;
   UINT64                         PgTableMask;
   UINT64                         AddressEncMask;
   BOOLEAN                        IsWpEnabled;
+  UINTN                          OrigLength;
   RETURN_STATUS                  Status;
 
   //
@@ -751,6 +754,22 @@ SetMemoryEncDec (
 
   Status = EFI_SUCCESS;
 
+  //
+  // To maintain the security gurantees we must set the page to shared in the RMP
+  // table before clearing the memory encryption mask from the current page table.
+  //
+  // The InternalSetPageState() is used for setting the page state in the RMP table.
+  //
+  if ((Mode == ClearCBit) && MemEncryptSevSnpIsEnabled ()) {
+    InternalSetPageState (PhysicalAddress, EFI_SIZE_TO_PAGES (Length), SevSnpPageShared, FALSE);
+  }
+
+  //
+  // Save the specified length and physical address (we need it later).
+  //
+  OrigLength = Length;
+  OrigPhysicalAddress = PhysicalAddress;
+
   while (Length != 0)
   {
     //
@@ -923,6 +942,21 @@ SetMemoryEncDec (
   //
   CpuFlushTlb();
 
+  //
+  // SEV-SNP requires that all the private pages (i.e pages mapped encrypted) must be
+  // added in the RMP table before the access.
+  //
+  // The InternalSetPageState() is used for setting the page state in the RMP table.
+  //
+  if ((Mode == SetCBit) && MemEncryptSevSnpIsEnabled ()) {
+    InternalSetPageState (
+      OrigPhysicalAddress,
+      EFI_SIZE_TO_PAGES (OrigLength),
+      SevSnpPagePrivate,
+      FALSE
+      );
+  }
+
 Done:
   //
   // Restore page table write protection, if any.
-- 
2.17.1

[PATCH v6 26/29] OvmfPkg/MemEncryptSevLib: skip page state change for Mmio address

[Brijesh Singh]

The SetMemoryEncDec() is used by the higher level routines to set or clear
the page encryption mask for system RAM and Mmio address. When SEV-SNP is
active, in addition to set/clear page mask it also updates the RMP table.
The RMP table updates are required for the system RAM address and not
the Mmio address.

Add a new parameter in SetMemoryEncDec() to tell whether the specified
address is Mmio. If its Mmio then skip the page state change in the RMP
table.

Cc: Michael Roth <michael.roth@amd.com>
Cc: James Bottomley <jejb@linux.ibm.com>
Cc: Min Xu <min.m.xu@intel.com>
Cc: Jiewen Yao <jiewen.yao@intel.com>
Cc: Tom Lendacky <thomas.lendacky@amd.com>
Cc: Jordan Justen <jordan.l.justen@intel.com>
Cc: Ard Biesheuvel <ardb+tianocore@kernel.org>
Cc: Erdem Aktas <erdemaktas@google.com>
Signed-off-by: Brijesh Singh <brijesh.singh@amd.com>
---
 .../X64/PeiDxeVirtualMemory.c                 | 20 ++++++++++++-------
 1 file changed, 13 insertions(+), 7 deletions(-)

diff --git a/OvmfPkg/Library/BaseMemEncryptSevLib/X64/PeiDxeVirtualMemory.c b/OvmfPkg/Library/BaseMemEncryptSevLib/X64/PeiDxeVirtualMemory.c
index 56db1e4b6ecf..0bb86d768017 100644
--- a/OvmfPkg/Library/BaseMemEncryptSevLib/X64/PeiDxeVirtualMemory.c
+++ b/OvmfPkg/Library/BaseMemEncryptSevLib/X64/PeiDxeVirtualMemory.c
@@ -673,6 +673,7 @@ InternalMemEncryptSevCreateIdentityMap1G (
   @param[in]  Mode                    Set or Clear mode
   @param[in]  CacheFlush              Flush the caches before applying the
                                       encryption mask
+  @param[in]  Mmio                    The physical address specified is Mmio
 
   @retval RETURN_SUCCESS              The attributes were cleared for the
                                       memory region.
@@ -688,7 +689,8 @@ SetMemoryEncDec (
   IN    PHYSICAL_ADDRESS         PhysicalAddress,
   IN    UINTN                    Length,
   IN    MAP_RANGE_MODE           Mode,
-  IN    BOOLEAN                  CacheFlush
+  IN    BOOLEAN                  CacheFlush,
+  IN    BOOLEAN                  Mmio
   )
 {
   PAGE_MAP_AND_DIRECTORY_POINTER *PageMapLevel4Entry;
@@ -711,14 +713,15 @@ SetMemoryEncDec (
 
   DEBUG ((
     DEBUG_VERBOSE,
-    "%a:%a: Cr3Base=0x%Lx Physical=0x%Lx Length=0x%Lx Mode=%a CacheFlush=%u\n",
+    "%a:%a: Cr3Base=0x%Lx Physical=0x%Lx Length=0x%Lx Mode=%a CacheFlush=%u Mmio=%u\n",
     gEfiCallerBaseName,
     __FUNCTION__,
     Cr3BaseAddress,
     PhysicalAddress,
     (UINT64)Length,
     (Mode == SetCBit) ? "Encrypt" : "Decrypt",
-    (UINT32)CacheFlush
+    (UINT32)CacheFlush,
+    (UINT32)Mmio
     ));
 
   //
@@ -760,7 +763,7 @@ SetMemoryEncDec (
   //
   // The InternalSetPageState() is used for setting the page state in the RMP table.
   //
-  if ((Mode == ClearCBit) && MemEncryptSevSnpIsEnabled ()) {
+  if (!Mmio && (Mode == ClearCBit) && MemEncryptSevSnpIsEnabled ()) {
     InternalSetPageState (PhysicalAddress, EFI_SIZE_TO_PAGES (Length), SevSnpPageShared, FALSE);
   }
 
@@ -998,7 +1001,8 @@ InternalMemEncryptSevSetMemoryDecrypted (
            PhysicalAddress,
            Length,
            ClearCBit,
-           TRUE
+           TRUE,
+           FALSE
            );
 }
 
@@ -1031,7 +1035,8 @@ InternalMemEncryptSevSetMemoryEncrypted (
            PhysicalAddress,
            Length,
            SetCBit,
-           TRUE
+           TRUE,
+           FALSE
            );
 }
 
@@ -1064,6 +1069,7 @@ InternalMemEncryptSevClearMmioPageEncMask (
            PhysicalAddress,
            Length,
            ClearCBit,
-           FALSE
+           FALSE,
+           TRUE
            );
 }
-- 
2.17.1

[PATCH v6 27/29] OvmfPkg/PlatformPei: mark cpuid and secrets memory reserved in EFI map

[Brijesh Singh]

When SEV-SNP is active, the CPUID and Secrets memory range contains the
information that is used during the VM boot. The content need to be persist
across the kexec boot. Mark the memory range as Reserved in the EFI map
so that guest OS or firmware does not use the range as a system RAM.

Cc: Michael Roth <michael.roth@amd.com>
Cc: James Bottomley <jejb@linux.ibm.com>
Cc: Min Xu <min.m.xu@intel.com>
Cc: Jiewen Yao <jiewen.yao@intel.com>
Cc: Tom Lendacky <thomas.lendacky@amd.com>
Cc: Jordan Justen <jordan.l.justen@intel.com>
Cc: Ard Biesheuvel <ardb+tianocore@kernel.org>
Cc: Erdem Aktas <erdemaktas@google.com>
Signed-off-by: Brijesh Singh <brijesh.singh@amd.com>
---
 OvmfPkg/PlatformPei/PlatformPei.inf |  6 ++++++
 OvmfPkg/PlatformPei/MemDetect.c     | 21 +++++++++++++++++++++
 2 files changed, 27 insertions(+)

diff --git a/OvmfPkg/PlatformPei/PlatformPei.inf b/OvmfPkg/PlatformPei/PlatformPei.inf
index 8c9795095a10..82a0ee721da2 100644
--- a/OvmfPkg/PlatformPei/PlatformPei.inf
+++ b/OvmfPkg/PlatformPei/PlatformPei.inf
@@ -122,6 +122,12 @@ [FixedPcd]
   gUefiOvmfPkgTokenSpaceGuid.PcdOvmfSecGhcbBackupSize
   gUefiOvmfPkgTokenSpaceGuid.PcdOvmfWorkAreaBase
   gUefiOvmfPkgTokenSpaceGuid.PcdOvmfWorkAreaSize
+  gUefiCpuPkgTokenSpaceGuid.PcdSevEsWorkAreaBase
+  gUefiCpuPkgTokenSpaceGuid.PcdSevEsWorkAreaSize
+  gUefiOvmfPkgTokenSpaceGuid.PcdOvmfSnpSecretsBase
+  gUefiOvmfPkgTokenSpaceGuid.PcdOvmfSnpSecretsSize
+  gUefiOvmfPkgTokenSpaceGuid.PcdOvmfSnpCpuidBase
+  gUefiOvmfPkgTokenSpaceGuid.PcdOvmfSnpCpuidSize
 
 [FeaturePcd]
   gUefiOvmfPkgTokenSpaceGuid.PcdCsmEnable
diff --git a/OvmfPkg/PlatformPei/MemDetect.c b/OvmfPkg/PlatformPei/MemDetect.c
index 2c2c4641ec8a..564c3dabc487 100644
--- a/OvmfPkg/PlatformPei/MemDetect.c
+++ b/OvmfPkg/PlatformPei/MemDetect.c
@@ -817,6 +817,27 @@ InitializeRamRegions (
 {
   QemuInitializeRam ();
 
+  if (MemEncryptSevSnpIsEnabled ()) {
+    //
+    // If SEV-SNP is enabled, reserve the Secrets and CPUID memory area.
+    //
+    // This memory range is given to the PSP by the hypervisor to populate
+    // the information used during the SNP VM boots, and it need to persist
+    // across the kexec boots. Mark it as EfiReservedMemoryType so that
+    // the guest firmware and OS does not use it as a system memory.
+    //
+    BuildMemoryAllocationHob (
+      (EFI_PHYSICAL_ADDRESS)(UINTN) PcdGet32 (PcdOvmfSnpSecretsBase),
+      (UINT64)(UINTN) PcdGet32 (PcdOvmfSnpSecretsSize),
+      EfiReservedMemoryType
+      );
+    BuildMemoryAllocationHob (
+      (EFI_PHYSICAL_ADDRESS)(UINTN) PcdGet32 (PcdOvmfSnpCpuidBase),
+      (UINT64)(UINTN) PcdGet32 (PcdOvmfSnpCpuidSize),
+      EfiReservedMemoryType
+      );
+  }
+
   if (mS3Supported && mBootMode != BOOT_ON_S3_RESUME) {
     //
     // This is the memory range that will be used for PEI on S3 resume
-- 
2.17.1

[PATCH v6 28/29] OvmfPkg/AmdSev: expose the SNP reserved pages through configuration table

[Brijesh Singh]

BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=3275

Now that both the secrets and cpuid pages are reserved in the HOB,
extract the location details through fixed PCD and make it available
to the guest OS through the configuration table.

Cc: Michael Roth <michael.roth@amd.com>
Cc: James Bottomley <jejb@linux.ibm.com>
Cc: Min Xu <min.m.xu@intel.com>
Cc: Jiewen Yao <jiewen.yao@intel.com>
Cc: Tom Lendacky <thomas.lendacky@amd.com>
Cc: Jordan Justen <jordan.l.justen@intel.com>
Cc: Ard Biesheuvel <ardb+tianocore@kernel.org>
Cc: Erdem Aktas <erdemaktas@google.com>
Signed-off-by: Brijesh Singh <brijesh.singh@amd.com>
---
 OvmfPkg/OvmfPkg.dec                           |  1 +
 OvmfPkg/AmdSevDxe/AmdSevDxe.inf               |  7 ++++++
 .../Guid/ConfidentialComputingSecret.h        | 18 +++++++++++++++
 OvmfPkg/AmdSevDxe/AmdSevDxe.c                 | 23 +++++++++++++++++++
 4 files changed, 49 insertions(+)

diff --git a/OvmfPkg/OvmfPkg.dec b/OvmfPkg/OvmfPkg.dec
index d040ee02b94f..f4a53ab49901 100644
--- a/OvmfPkg/OvmfPkg.dec
+++ b/OvmfPkg/OvmfPkg.dec
@@ -124,6 +124,7 @@ [Guids]
   gQemuKernelLoaderFsMediaGuid          = {0x1428f772, 0xb64a, 0x441e, {0xb8, 0xc3, 0x9e, 0xbd, 0xd7, 0xf8, 0x93, 0xc7}}
   gGrubFileGuid                         = {0xb5ae312c, 0xbc8a, 0x43b1, {0x9c, 0x62, 0xeb, 0xb8, 0x26, 0xdd, 0x5d, 0x07}}
   gConfidentialComputingSecretGuid      = {0xadf956ad, 0xe98c, 0x484c, {0xae, 0x11, 0xb5, 0x1c, 0x7d, 0x33, 0x64, 0x47}}
+  gConfidentialComputingSevBlobGuid     = {0x067b1f5f, 0xcf26, 0x44c5, {0x85, 0x54, 0x93, 0xd7, 0x77, 0x91, 0x2d, 0x42}}
 
 [Ppis]
   # PPI whose presence in the PPI database signals that the TPM base address
diff --git a/OvmfPkg/AmdSevDxe/AmdSevDxe.inf b/OvmfPkg/AmdSevDxe/AmdSevDxe.inf
index 0676fcc5b6a4..74a55c189a09 100644
--- a/OvmfPkg/AmdSevDxe/AmdSevDxe.inf
+++ b/OvmfPkg/AmdSevDxe/AmdSevDxe.inf
@@ -42,6 +42,13 @@ [FeaturePcd]
 
 [FixedPcd]
   gEfiMdePkgTokenSpaceGuid.PcdPciExpressBaseAddress
+  gUefiOvmfPkgTokenSpaceGuid.PcdOvmfSnpCpuidBase
+  gUefiOvmfPkgTokenSpaceGuid.PcdOvmfSnpCpuidSize
+  gUefiOvmfPkgTokenSpaceGuid.PcdOvmfSnpSecretsBase
+  gUefiOvmfPkgTokenSpaceGuid.PcdOvmfSnpSecretsSize
+
+[Guids]
+  gConfidentialComputingSevBlobGuid
 
 [Pcd]
   gUefiOvmfPkgTokenSpaceGuid.PcdOvmfHostBridgePciDevId
diff --git a/OvmfPkg/Include/Guid/ConfidentialComputingSecret.h b/OvmfPkg/Include/Guid/ConfidentialComputingSecret.h
index 7026fc5b089f..769f2831f3e5 100644
--- a/OvmfPkg/Include/Guid/ConfidentialComputingSecret.h
+++ b/OvmfPkg/Include/Guid/ConfidentialComputingSecret.h
@@ -18,11 +18,29 @@
     { 0xae, 0x11, 0xb5, 0x1c, 0x7d, 0x33, 0x64, 0x47 }, \
   }
 
+#define CONFIDENTIAL_COMPUTING_SEV_BLOB_GUID            \
+  { 0x067b1f5f,                                         \
+    0xcf26,                                             \
+    0x44c5,                                             \
+    { 0x85, 0x54, 0x93, 0xd7, 0x77, 0x91, 0x2d, 0x42 }, \
+  }
+
 typedef struct {
   UINT64 Base;
   UINT64 Size;
 } CONFIDENTIAL_COMPUTING_SECRET_LOCATION;
 
+typedef struct {
+  UINT32  Header;
+  UINT16  Version;
+  UINT16  Reserved1;
+  UINT64  SecretsPhysicalAddress;
+  UINT32  SecretsSize;
+  UINT64  CpuidPhysicalAddress;
+  UINT32  CpuidLSize;
+} CONFIDENTIAL_COMPUTING_SEV_BLOB_LOCATION;
+
 extern EFI_GUID gConfidentialComputingSecretGuid;
+extern EFI_GUID gConfidentialComputingSevBlobGuid;
 
 #endif // SEV_LAUNCH_SECRET_H_
diff --git a/OvmfPkg/AmdSevDxe/AmdSevDxe.c b/OvmfPkg/AmdSevDxe/AmdSevDxe.c
index c66c4e9b9272..3af40f4cf29b 100644
--- a/OvmfPkg/AmdSevDxe/AmdSevDxe.c
+++ b/OvmfPkg/AmdSevDxe/AmdSevDxe.c
@@ -17,8 +17,20 @@
 #include <Library/DxeServicesTableLib.h>
 #include <Library/MemEncryptSevLib.h>
 #include <Library/MemoryAllocationLib.h>
+#include <Library/UefiBootServicesTableLib.h>
+#include <Guid/ConfidentialComputingSecret.h>
 #include <Library/PcdLib.h>
 
+STATIC CONFIDENTIAL_COMPUTING_SEV_BLOB_LOCATION mSnpBootDxeTable = {
+  SIGNATURE_32('A','M','D','E'),
+  1,
+  0,
+  (UINT64)(UINTN) FixedPcdGet32 (PcdOvmfSnpSecretsBase),
+  FixedPcdGet32 (PcdOvmfSnpSecretsSize),
+  (UINT64)(UINTN) FixedPcdGet32 (PcdOvmfSnpCpuidBase),
+  FixedPcdGet32 (PcdOvmfSnpCpuidSize),
+};
+
 EFI_STATUS
 EFIAPI
 AmdSevDxeEntryPoint (
@@ -130,5 +142,16 @@ AmdSevDxeEntryPoint (
     }
   }
 
+  //
+  // If its SEV-SNP active guest then install the CONFIDENTIAL_COMPUTING_SEV_BLOB.
+  // It contains the location for both the Secrets and CPUID page.
+  //
+  if (MemEncryptSevSnpIsEnabled ()) {
+    return gBS->InstallConfigurationTable (
+                  &gConfidentialComputingSevBlobGuid,
+                  &mSnpBootDxeTable
+                  );
+  }
+
   return EFI_SUCCESS;
 }
-- 
2.17.1

[PATCH v6 29/29] UefiCpuPkg/MpInitLib: Use SEV-SNP AP Creation NAE event to launch APs

[Brijesh Singh]

From: Tom Lendacky <thomas.lendacky@amd.com>

BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=3275

Use the SEV-SNP AP Creation NAE event to create and launch APs under
SEV-SNP. This capability will be advertised in the SEV Hypervisor
Feature Support PCD (PcdSevEsHypervisorFeatures).

Cc: Michael Roth <michael.roth@amd.com>
Cc: Eric Dong <eric.dong@intel.com>
Cc: Ray Ni <ray.ni@intel.com>
Cc: Rahul Kumar <rahul1.kumar@intel.com>
Cc: James Bottomley <jejb@linux.ibm.com>
Cc: Min Xu <min.m.xu@intel.com>
Cc: Jiewen Yao <jiewen.yao@intel.com>
Cc: Tom Lendacky <thomas.lendacky@amd.com>
Cc: Jordan Justen <jordan.l.justen@intel.com>
Cc: Ard Biesheuvel <ardb+tianocore@kernel.org>
Cc: Erdem Aktas <erdemaktas@google.com>
Signed-off-by: Tom Lendacky <thomas.lendacky@amd.com>
Signed-off-by: Brijesh Singh <brijesh.singh@amd.com>
---
 UefiCpuPkg/Library/MpInitLib/DxeMpInitLib.inf |   3 +
 UefiCpuPkg/Library/MpInitLib/PeiMpInitLib.inf |   5 +-
 UefiCpuPkg/Library/MpInitLib/MpLib.h          |  17 ++
 UefiCpuPkg/Library/MpInitLib/DxeMpLib.c       |  11 +-
 .../MpInitLib/Ia32/SevSnpRmpAdjustInternal.c  |  31 ++
 UefiCpuPkg/Library/MpInitLib/MpLib.c          | 275 ++++++++++++++++--
 .../MpInitLib/X64/SevSnpRmpAdjustInternal.c   |  44 +++
 7 files changed, 361 insertions(+), 25 deletions(-)
 create mode 100644 UefiCpuPkg/Library/MpInitLib/Ia32/SevSnpRmpAdjustInternal.c
 create mode 100644 UefiCpuPkg/Library/MpInitLib/X64/SevSnpRmpAdjustInternal.c

diff --git a/UefiCpuPkg/Library/MpInitLib/DxeMpInitLib.inf b/UefiCpuPkg/Library/MpInitLib/DxeMpInitLib.inf
index 48d7dfa4450f..b9ce05e81b54 100644
--- a/UefiCpuPkg/Library/MpInitLib/DxeMpInitLib.inf
+++ b/UefiCpuPkg/Library/MpInitLib/DxeMpInitLib.inf
@@ -23,9 +23,11 @@ [Defines]
 
 [Sources.IA32]
   Ia32/MpFuncs.nasm
+  Ia32/SevSnpRmpAdjustInternal.c
 
 [Sources.X64]
   X64/MpFuncs.nasm
+  X64/SevSnpRmpAdjustInternal.c
 
 [Sources.common]
   MpEqu.inc
@@ -72,6 +74,7 @@ [Pcd]
   gUefiCpuPkgTokenSpaceGuid.PcdCpuApLoopMode                           ## CONSUMES
   gUefiCpuPkgTokenSpaceGuid.PcdCpuApTargetCstate                       ## SOMETIMES_CONSUMES
   gUefiCpuPkgTokenSpaceGuid.PcdCpuApStatusCheckIntervalInMicroSeconds  ## CONSUMES
+  gUefiCpuPkgTokenSpaceGuid.PcdGhcbHypervisorFeatures                  ## CONSUMES
   gUefiCpuPkgTokenSpaceGuid.PcdSevEsIsEnabled                          ## CONSUMES
   gUefiCpuPkgTokenSpaceGuid.PcdSevEsWorkAreaBase                       ## SOMETIMES_CONSUMES
   gEfiMdeModulePkgTokenSpaceGuid.PcdCpuStackGuard                      ## CONSUMES
diff --git a/UefiCpuPkg/Library/MpInitLib/PeiMpInitLib.inf b/UefiCpuPkg/Library/MpInitLib/PeiMpInitLib.inf
index ab8279df596f..35057ac07cbb 100644
--- a/UefiCpuPkg/Library/MpInitLib/PeiMpInitLib.inf
+++ b/UefiCpuPkg/Library/MpInitLib/PeiMpInitLib.inf
@@ -23,9 +23,11 @@ [Defines]
 
 [Sources.IA32]
   Ia32/MpFuncs.nasm
+  Ia32/SevSnpRmpAdjustInternal.c
 
 [Sources.X64]
   X64/MpFuncs.nasm
+  X64/SevSnpRmpAdjustInternal.c
 
 [Sources.common]
   MpEqu.inc
@@ -62,10 +64,11 @@ [Pcd]
   gUefiCpuPkgTokenSpaceGuid.PcdCpuMicrocodePatchRegionSize         ## CONSUMES
   gUefiCpuPkgTokenSpaceGuid.PcdCpuApLoopMode                       ## CONSUMES
   gUefiCpuPkgTokenSpaceGuid.PcdCpuApTargetCstate                   ## SOMETIMES_CONSUMES
+  gUefiCpuPkgTokenSpaceGuid.PcdGhcbHypervisorFeatures              ## CONSUMES
   gUefiCpuPkgTokenSpaceGuid.PcdSevEsIsEnabled                      ## CONSUMES
   gUefiCpuPkgTokenSpaceGuid.PcdSevEsWorkAreaBase                   ## SOMETIMES_CONSUMES
-  gEfiMdeModulePkgTokenSpaceGuid.PcdGhcbBase                       ## CONSUMES
   gUefiCpuPkgTokenSpaceGuid.PcdSevSnpIsEnabled                     ## CONSUMES
+  gEfiMdeModulePkgTokenSpaceGuid.PcdGhcbBase                       ## CONSUMES
 
 [Ppis]
   gEdkiiPeiShadowMicrocodePpiGuid        ## SOMETIMES_CONSUMES
diff --git a/UefiCpuPkg/Library/MpInitLib/MpLib.h b/UefiCpuPkg/Library/MpInitLib/MpLib.h
index 19e91bf7d74e..a046de8e5c68 100644
--- a/UefiCpuPkg/Library/MpInitLib/MpLib.h
+++ b/UefiCpuPkg/Library/MpInitLib/MpLib.h
@@ -15,6 +15,7 @@
 
 #include <Register/Intel/Cpuid.h>
 #include <Register/Amd/Cpuid.h>
+#include <Register/Amd/Ghcb.h>
 #include <Register/Intel/Msr.h>
 #include <Register/Intel/LocalApic.h>
 #include <Register/Intel/Microcode.h>
@@ -146,6 +147,7 @@ typedef struct {
   UINT8                          PlatformId;
   UINT64                         MicrocodeEntryAddr;
   UINT32                         MicrocodeRevision;
+  SEV_ES_SAVE_AREA               *SevEsSaveArea;
 } CPU_AP_DATA;
 
 //
@@ -290,6 +292,7 @@ struct _CPU_MP_DATA {
 
   BOOLEAN                        SevEsIsEnabled;
   BOOLEAN                        SevSnpIsEnabled;
+  BOOLEAN                        UseSevEsAPMethod;
   UINTN                          SevEsAPBuffer;
   UINTN                          SevEsAPResetStackStart;
   CPU_MP_DATA                    *NewCpuMpData;
@@ -744,5 +747,19 @@ PlatformShadowMicrocode (
   IN OUT CPU_MP_DATA             *CpuMpData
   );
 
+/**
+  Issue RMPADJUST to adjust the VMSA attribute of an SEV-SNP page.
+
+  @param[in]  PageAddress
+  @param[in]  VmsaPage
+
+  @return  RMPADJUST return value
+**/
+UINT32
+SevSnpRmpAdjust (
+  IN  EFI_PHYSICAL_ADDRESS  PageAddress,
+  IN  BOOLEAN               VmsaPage
+  );
+
 #endif
 
diff --git a/UefiCpuPkg/Library/MpInitLib/DxeMpLib.c b/UefiCpuPkg/Library/MpInitLib/DxeMpLib.c
index 93fc63bf93e3..15e797cd0990 100644
--- a/UefiCpuPkg/Library/MpInitLib/DxeMpLib.c
+++ b/UefiCpuPkg/Library/MpInitLib/DxeMpLib.c
@@ -93,7 +93,12 @@ GetWakeupBuffer (
   EFI_PHYSICAL_ADDRESS    StartAddress;
   EFI_MEMORY_TYPE         MemoryType;
 
-  if (PcdGetBool (PcdSevEsIsEnabled)) {
+  //
+  // An SEV-ES-only guest requires the memory to be reserved. SEV-SNP, which
+  // is also considered SEV-ES, uses a different AP startup method, though,
+  // which does not have the same requirement.
+  //
+  if (PcdGetBool (PcdSevEsIsEnabled) && !PcdGetBool (PcdSevSnpIsEnabled)) {
     MemoryType = EfiReservedMemoryType;
   } else {
     MemoryType = EfiBootServicesData;
@@ -373,7 +378,7 @@ RelocateApLoop (
   MpInitLibWhoAmI (&ProcessorNumber);
   CpuMpData    = GetCpuMpData ();
   MwaitSupport = IsMwaitSupport ();
-  if (CpuMpData->SevEsIsEnabled) {
+  if (CpuMpData->UseSevEsAPMethod) {
     StackStart = CpuMpData->SevEsAPResetStackStart;
   } else {
     StackStart = mReservedTopOfApStack;
@@ -422,7 +427,7 @@ MpInitChangeApLoopCallback (
     CpuPause ();
   }
 
-  if (CpuMpData->SevEsIsEnabled && (CpuMpData->WakeupBuffer != (UINTN) -1)) {
+  if (CpuMpData->UseSevEsAPMethod && (CpuMpData->WakeupBuffer != (UINTN) -1)) {
     //
     // There are APs present. Re-use reserved memory area below 1MB from
     // WakeupBuffer as the area to be used for transitioning to 16-bit mode
diff --git a/UefiCpuPkg/Library/MpInitLib/Ia32/SevSnpRmpAdjustInternal.c b/UefiCpuPkg/Library/MpInitLib/Ia32/SevSnpRmpAdjustInternal.c
new file mode 100644
index 000000000000..5c9ee08b2117
--- /dev/null
+++ b/UefiCpuPkg/Library/MpInitLib/Ia32/SevSnpRmpAdjustInternal.c
@@ -0,0 +1,31 @@
+/** @file
+
+  RMPADJUST helper function.
+
+  Copyright (c) 2021, AMD Incorporated. All rights reserved.<BR>
+
+  SPDX-License-Identifier: BSD-2-Clause-Patent
+
+**/
+
+#include "MpLib.h"
+
+/**
+  Issue RMPADJUST to adjust the VMSA attribute of an SEV-SNP page.
+
+  @param[in]  PageAddress
+  @param[in]  VmsaPage
+
+  @return  RMPADJUST return value
+**/
+UINT32
+SevSnpRmpAdjust (
+  IN  EFI_PHYSICAL_ADDRESS  PageAddress,
+  IN  BOOLEAN               VmsaPage
+  )
+{
+  //
+  // RMPADJUST is not supported in 32-bit mode
+  //
+  return RETURN_UNSUPPORTED;
+}
diff --git a/UefiCpuPkg/Library/MpInitLib/MpLib.c b/UefiCpuPkg/Library/MpInitLib/MpLib.c
index c2b00a1f04ff..01f11ab1d9b3 100644
--- a/UefiCpuPkg/Library/MpInitLib/MpLib.c
+++ b/UefiCpuPkg/Library/MpInitLib/MpLib.c
@@ -295,10 +295,10 @@ GetApLoopMode (
       ApLoopMode = ApInHltLoop;
     }
 
-    if (PcdGetBool (PcdSevEsIsEnabled)) {
+    if (PcdGetBool (PcdSevEsIsEnabled) && !PcdGetBool (PcdSevSnpIsEnabled)) {
       //
-      // For SEV-ES, force AP in Hlt-loop mode in order to use the GHCB
-      // protocol for starting APs
+      // For SEV-ES (SEV-SNP is also considered SEV-ES), force AP in Hlt-loop
+      // mode in order to use the GHCB protocol for starting APs
       //
       ApLoopMode = ApInHltLoop;
     }
@@ -869,7 +869,7 @@ ApWakeupFunction (
       // to allow the APs to issue an AP_RESET_HOLD before the BSP possibly
       // performs another INIT-SIPI-SIPI sequence.
       //
-      if (!CpuMpData->SevEsIsEnabled) {
+      if (!CpuMpData->UseSevEsAPMethod) {
         InterlockedDecrement ((UINT32 *) &CpuMpData->MpCpuExchangeInfo->NumApsExecuting);
       }
     }
@@ -883,7 +883,7 @@ ApWakeupFunction (
       //
       while (TRUE) {
         DisableInterrupts ();
-        if (CpuMpData->SevEsIsEnabled) {
+        if (CpuMpData->UseSevEsAPMethod) {
           MSR_SEV_ES_GHCB_REGISTER  Msr;
           GHCB                      *Ghcb;
           UINT64                    Status;
@@ -1207,9 +1207,11 @@ AllocateResetVector (
                                     );
     //
     // The AP reset stack is only used by SEV-ES guests. Do not allocate it
-    // if SEV-ES is not enabled.
+    // if SEV-ES is not enabled. An SEV-SNP guest is also considered
+    // an SEV-ES guest, but uses a different method of AP startup, eliminating
+    // the need for the allocation.
     //
-    if (PcdGetBool (PcdSevEsIsEnabled)) {
+    if (PcdGetBool (PcdSevEsIsEnabled) && !PcdGetBool (PcdSevSnpIsEnabled)) {
       //
       // Stack location is based on ProcessorNumber, so use the total number
       // of processors for calculating the total stack area.
@@ -1259,7 +1261,7 @@ FreeResetVector (
   // perform the restore as this will overwrite memory which has data
   // needed by SEV-ES.
   //
-  if (!CpuMpData->SevEsIsEnabled) {
+  if (!CpuMpData->UseSevEsAPMethod) {
     RestoreWakeupBuffer (CpuMpData);
   }
 }
@@ -1276,7 +1278,7 @@ AllocateSevEsAPMemory (
 {
   if (CpuMpData->SevEsAPBuffer == (UINTN) -1) {
     CpuMpData->SevEsAPBuffer =
-      CpuMpData->SevEsIsEnabled ? GetSevEsAPMemory () : 0;
+      CpuMpData->UseSevEsAPMethod ? GetSevEsAPMemory () : 0;
   }
 }
 
@@ -1329,6 +1331,222 @@ SetSevEsJumpTable (
   JmpFar->Segment = (UINT16) (SipiVector >> 4);
 }
 
+/**
+  Create an SEV-SNP AP save area (VMSA) for use in running the vCPU.
+
+  @param[in]  CpuMpData        Pointer to CPU MP Data
+  @param[in]  CpuData          Pointer to CPU AP Data
+  @param[in]  ApicId           APIC ID of the vCPU
+**/
+STATIC
+VOID
+SevSnpCreateSaveArea (
+  IN CPU_MP_DATA               *CpuMpData,
+  IN CPU_AP_DATA               *CpuData,
+  UINT32                       ApicId
+  )
+{
+  SEV_ES_SAVE_AREA          *SaveArea;
+  IA32_CR0                  ApCr0;
+  IA32_CR0                  ResetCr0;
+  IA32_CR4                  ApCr4;
+  IA32_CR4                  ResetCr4;
+  UINTN                     StartIp;
+  UINT8                     SipiVector;
+  UINT32                    RmpAdjustStatus;
+  UINT64                    VmgExitStatus;
+  MSR_SEV_ES_GHCB_REGISTER  Msr;
+  GHCB                      *Ghcb;
+  BOOLEAN                   InterruptState;
+  UINT64                    ExitInfo1;
+  UINT64                    ExitInfo2;
+
+  //
+  // Allocate a single page for the SEV-ES Save Area and initialize it.
+  //
+  SaveArea = AllocateReservedPages (1);
+  if (!SaveArea) {
+    return;
+  }
+  ZeroMem (SaveArea, EFI_PAGE_SIZE);
+
+  //
+  // Propogate the CR0.NW and CR0.CD setting to the AP
+  //
+  ResetCr0.UintN = 0x00000010;
+  ApCr0.UintN = CpuData->VolatileRegisters.Cr0;
+  if (ApCr0.Bits.NW) {
+    ResetCr0.Bits.NW = 1;
+  }
+  if (ApCr0.Bits.CD) {
+    ResetCr0.Bits.CD = 1;
+  }
+
+  //
+  // Propagate the CR4.MCE setting to the AP
+  //
+  ResetCr4.UintN = 0;
+  ApCr4.UintN = CpuData->VolatileRegisters.Cr4;
+  if (ApCr4.Bits.MCE) {
+    ResetCr4.Bits.MCE = 1;
+  }
+
+  //
+  // Convert the start IP into a SIPI Vector
+  //
+  StartIp = CpuMpData->MpCpuExchangeInfo->BufferStart;
+  SipiVector = (UINT8) (StartIp >> 12);
+
+  //
+  // Set the CS:RIP value based on the start IP
+  //
+  SaveArea->Cs.Base = SipiVector << 12;
+  SaveArea->Cs.Selector = SipiVector << 8;
+  SaveArea->Cs.Limit = 0xFFFF;
+  SaveArea->Cs.Attributes.Bits.Present = 1;
+  SaveArea->Cs.Attributes.Bits.Sbit = 1;
+  SaveArea->Cs.Attributes.Bits.Type = SEV_ES_RESET_CODE_SEGMENT_TYPE;
+  SaveArea->Rip = StartIp & 0xFFF;
+
+  //
+  // Set the remaining values as defined in APM for INIT
+  //
+  SaveArea->Ds.Limit = 0xFFFF;
+  SaveArea->Ds.Attributes.Bits.Present = 1;
+  SaveArea->Ds.Attributes.Bits.Sbit = 1;
+  SaveArea->Ds.Attributes.Bits.Type = SEV_ES_RESET_DATA_SEGMENT_TYPE;
+  SaveArea->Es = SaveArea->Ds;
+  SaveArea->Fs = SaveArea->Ds;
+  SaveArea->Gs = SaveArea->Ds;
+  SaveArea->Ss = SaveArea->Ds;
+
+  SaveArea->Gdtr.Limit = 0xFFFF;
+  SaveArea->Ldtr.Limit = 0xFFFF;
+  SaveArea->Ldtr.Attributes.Bits.Present = 1;
+  SaveArea->Ldtr.Attributes.Bits.Type = SEV_ES_RESET_LDT_TYPE;
+  SaveArea->Idtr.Limit = 0xFFFF;
+  SaveArea->Tr.Limit = 0xFFFF;
+  SaveArea->Ldtr.Attributes.Bits.Present = 1;
+  SaveArea->Ldtr.Attributes.Bits.Type = SEV_ES_RESET_TSS_TYPE;
+
+  SaveArea->Efer   = 0x1000;
+  SaveArea->Cr4    = ResetCr4.UintN;
+  SaveArea->Cr0    = ResetCr0.UintN;
+  SaveArea->Dr7    = 0x0400;
+  SaveArea->Dr6    = 0xFFFF0FF0;
+  SaveArea->Rflags = 0x0002;
+  SaveArea->GPat   = 0x0007040600070406ULL;
+  SaveArea->XCr0   = 0x0001;
+  SaveArea->Mxcsr  = 0x1F80;
+  SaveArea->X87Ftw = 0x5555;
+  SaveArea->X87Fcw = 0x0040;
+
+  //
+  // Set the SEV-SNP specific fields for the save area:
+  //   VMPL - always VMPL0
+  //   SEV_FEATURES - equivalent to the SEV_STATUS MSR right shifted 2 bits
+  //
+  SaveArea->Vmpl        = 0;
+  SaveArea->SevFeatures = AsmReadMsr64 (MSR_SEV_STATUS) >> 2;
+
+  //
+  // To turn the page into a recognized VMSA page, issue RMPADJUST:
+  //   Target VMPL but numerically higher than current VMPL
+  //   Target PermissionMask is not used
+  //
+  RmpAdjustStatus = SevSnpRmpAdjust (
+                      (EFI_PHYSICAL_ADDRESS) (UINTN) SaveArea,
+                      TRUE
+                      );
+  ASSERT (RmpAdjustStatus == 0);
+
+  ExitInfo1 = (UINT64) ApicId << 32;
+  ExitInfo1 |= SVM_VMGEXIT_SNP_AP_CREATE;
+  ExitInfo2 = (UINT64) (UINTN) SaveArea;
+
+  Msr.GhcbPhysicalAddress = AsmReadMsr64 (MSR_SEV_ES_GHCB);
+  Ghcb = Msr.Ghcb;
+
+  VmgInit (Ghcb, &InterruptState);
+  Ghcb->SaveArea.Rax = SaveArea->SevFeatures;
+  VmgSetOffsetValid (Ghcb, GhcbRax);
+  VmgExitStatus = VmgExit (
+                    Ghcb,
+                    SVM_EXIT_SNP_AP_CREATION,
+                    ExitInfo1,
+                    ExitInfo2
+                    );
+  VmgDone (Ghcb, InterruptState);
+
+  ASSERT (VmgExitStatus == 0);
+  if (VmgExitStatus != 0) {
+    RmpAdjustStatus = SevSnpRmpAdjust (
+                        (EFI_PHYSICAL_ADDRESS) (UINTN) SaveArea,
+                        FALSE
+                        );
+    if (RmpAdjustStatus == 0) {
+      FreePages (SaveArea, 1);
+    } else {
+      DEBUG ((DEBUG_INFO, "SEV-SNP: RMPADJUST failed, leaking VMSA page\n"));
+    }
+
+    SaveArea = NULL;
+  }
+
+  if (CpuData->SevEsSaveArea) {
+    RmpAdjustStatus = SevSnpRmpAdjust (
+                        (EFI_PHYSICAL_ADDRESS) (UINTN) CpuData->SevEsSaveArea,
+                        FALSE
+                        );
+    if (RmpAdjustStatus == 0) {
+      FreePages (CpuData->SevEsSaveArea, 1);
+    } else {
+      DEBUG ((DEBUG_INFO, "SEV-SNP: RMPADJUST failed, leaking VMSA page\n"));
+    }
+  }
+
+  CpuData->SevEsSaveArea = SaveArea;
+}
+
+/**
+  Create SEV-SNP APs.
+
+  @param[in]  CpuMpData        Pointer to CPU MP Data
+  @param[in]  ProcessorNumber  The handle number of specified processor
+                               (-1 for all APs)
+**/
+STATIC
+VOID
+SevSnpCreateAP (
+  IN CPU_MP_DATA               *CpuMpData,
+  IN INTN                      ProcessorNumber
+  )
+{
+  CPU_INFO_IN_HOB  *CpuInfoInHob;
+  CPU_AP_DATA      *CpuData;
+  UINTN            Index;
+  UINT32           ApicId;
+
+  ASSERT (CpuMpData->MpCpuExchangeInfo->BufferStart < 0x100000);
+
+  CpuInfoInHob = (CPU_INFO_IN_HOB *) (UINTN) CpuMpData->CpuInfoInHob;
+
+  if (ProcessorNumber < 0) {
+    for (Index = 0; Index < CpuMpData->CpuCount; Index++) {
+      if (Index != CpuMpData->BspNumber) {
+        CpuData = &CpuMpData->CpuData[Index];
+        ApicId = CpuInfoInHob[Index].ApicId,
+        SevSnpCreateSaveArea (CpuMpData, CpuData, ApicId);
+      }
+    }
+  } else {
+    Index = (UINTN) ProcessorNumber;
+    CpuData = &CpuMpData->CpuData[Index];
+    ApicId = CpuInfoInHob[ProcessorNumber].ApicId,
+    SevSnpCreateSaveArea (CpuMpData, CpuData, ApicId);
+  }
+}
+
 /**
   This function will be called by BSP to wakeup AP.
 
@@ -1360,7 +1578,7 @@ WakeUpAP (
   ResetVectorRequired = FALSE;
 
   if (CpuMpData->WakeUpByInitSipiSipi ||
-      CpuMpData->InitFlag   != ApInitDone) {
+      CpuMpData->InitFlag != ApInitDone) {
     ResetVectorRequired = TRUE;
     AllocateResetVector (CpuMpData);
     AllocateSevEsAPMemory (CpuMpData);
@@ -1401,7 +1619,7 @@ WakeUpAP (
     }
     if (ResetVectorRequired) {
       //
-      // For SEV-ES, the initial AP boot address will be defined by
+      // For SEV-ES and SEV-SNP, the initial AP boot address will be defined by
       // PcdSevEsWorkAreaBase. The Segment/Rip must be the jump address
       // from the original INIT-SIPI-SIPI.
       //
@@ -1411,8 +1629,14 @@ WakeUpAP (
 
       //
       // Wakeup all APs
+      //   Must use the INIT-SIPI-SIPI method for initial configuration in
+      //   order to obtain the APIC ID.
       //
-      SendInitSipiSipiAllExcludingSelf ((UINT32) ExchangeInfo->BufferStart);
+      if (CpuMpData->SevSnpIsEnabled && CpuMpData->InitFlag != ApInitConfig) {
+        SevSnpCreateAP (CpuMpData, -1);
+      } else {
+        SendInitSipiSipiAllExcludingSelf ((UINT32) ExchangeInfo->BufferStart);
+      }
     }
     if (CpuMpData->InitFlag == ApInitConfig) {
       if (PcdGet32 (PcdCpuBootLogicalProcessorNumber) > 0) {
@@ -1502,7 +1726,7 @@ WakeUpAP (
       CpuInfoInHob = (CPU_INFO_IN_HOB *) (UINTN) CpuMpData->CpuInfoInHob;
 
       //
-      // For SEV-ES, the initial AP boot address will be defined by
+      // For SEV-ES and SEV-SNP, the initial AP boot address will be defined by
       // PcdSevEsWorkAreaBase. The Segment/Rip must be the jump address
       // from the original INIT-SIPI-SIPI.
       //
@@ -1510,10 +1734,14 @@ WakeUpAP (
         SetSevEsJumpTable (ExchangeInfo->BufferStart);
       }
 
-      SendInitSipiSipi (
-        CpuInfoInHob[ProcessorNumber].ApicId,
-        (UINT32) ExchangeInfo->BufferStart
-        );
+      if (CpuMpData->SevSnpIsEnabled && CpuMpData->InitFlag != ApInitConfig) {
+        SevSnpCreateAP (CpuMpData, (INTN) ProcessorNumber);
+      } else {
+        SendInitSipiSipi (
+          CpuInfoInHob[ProcessorNumber].ApicId,
+          (UINT32) ExchangeInfo->BufferStart
+          );
+      }
     }
     //
     // Wait specified AP waken up
@@ -2044,10 +2272,15 @@ MpInitLibInitialize (
   CpuMpData->CpuData          = (CPU_AP_DATA *) (CpuMpData + 1);
   CpuMpData->CpuInfoInHob     = (UINT64) (UINTN) (CpuMpData->CpuData + MaxLogicalProcessorNumber);
   InitializeSpinLock(&CpuMpData->MpLock);
-  CpuMpData->SevEsIsEnabled = PcdGetBool (PcdSevEsIsEnabled);
-  CpuMpData->SevSnpIsEnabled = PcdGetBool (PcdSevSnpIsEnabled);
-  CpuMpData->SevEsAPBuffer  = (UINTN) -1;
-  CpuMpData->GhcbBase       = PcdGet64 (PcdGhcbBase);
+  CpuMpData->SevEsIsEnabled   = PcdGetBool (PcdSevEsIsEnabled);
+  CpuMpData->SevSnpIsEnabled  = PcdGetBool (PcdSevSnpIsEnabled);
+  CpuMpData->SevEsAPBuffer    = (UINTN) -1;
+  CpuMpData->GhcbBase         = PcdGet64 (PcdGhcbBase);
+  CpuMpData->UseSevEsAPMethod = CpuMpData->SevEsIsEnabled && !CpuMpData->SevSnpIsEnabled;
+
+  if (CpuMpData->SevSnpIsEnabled) {
+    ASSERT ((PcdGet64 (PcdGhcbHypervisorFeatures) & GHCB_HV_FEATURES_SNP_AP_CREATE) == GHCB_HV_FEATURES_SNP_AP_CREATE);
+  }
 
   //
   // Make sure no memory usage outside of the allocated buffer.
diff --git a/UefiCpuPkg/Library/MpInitLib/X64/SevSnpRmpAdjustInternal.c b/UefiCpuPkg/Library/MpInitLib/X64/SevSnpRmpAdjustInternal.c
new file mode 100644
index 000000000000..3048742facb5
--- /dev/null
+++ b/UefiCpuPkg/Library/MpInitLib/X64/SevSnpRmpAdjustInternal.c
@@ -0,0 +1,44 @@
+/** @file
+
+  RMPADJUST helper function.
+
+  Copyright (c) 2021, AMD Incorporated. All rights reserved.<BR>
+
+  SPDX-License-Identifier: BSD-2-Clause-Patent
+
+**/
+
+#include "MpLib.h"
+
+/**
+  Issue RMPADJUST to adjust the VMSA attribute of an SEV-SNP page.
+
+  @param[in]  PageAddress
+  @param[in]  VmsaPage
+
+  @return  RMPADJUST return value
+**/
+UINT32
+SevSnpRmpAdjust (
+  IN  EFI_PHYSICAL_ADDRESS  PageAddress,
+  IN  BOOLEAN               VmsaPage
+  )
+{
+  UINT64  Rdx;
+
+  //
+  // The RMPADJUST instruction is used to set or clear the VMSA bit for a
+  // page. The VMSA change is only made when running at VMPL0 and is ignored
+  // otherwise. If too low a target VMPL is specified, the instruction can
+  // succeed without changing the VMSA bit when not running at VMPL0. Using a
+  // target VMPL level of 1, RMPADJUST will return a FAIL_PERMISSION error if
+  // not running at VMPL0, thus ensuring that the VMSA bit is set appropriately
+  // when no error is returned.
+  //
+  Rdx = 1;
+  if (VmsaPage) {
+    Rdx |= RMPADJUST_VMSA_PAGE_BIT;
+  }
+
+  return AsmRmpAdjust ((UINT64) PageAddress, 0, Rdx);
+}
-- 
2.17.1

Re: [PATCH v6 02/29] OvmfPkg: reserve CPUID page for SEV-SNP

[Gerd Hoffmann]

On Wed, Sep 01, 2021 at 11:16:19AM -0500, Brijesh Singh wrote:
> BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=3275
> 
> Platform features and capabilities are traditionally discovered via the
> CPUID instruction. Hypervisors typically trap and emulate the CPUID
> instruction for a variety of reasons. There are some cases where incorrect
> CPUID information can potentially lead to a security issue. The SEV-SNP
> firmware provides a feature to filter the CPUID results through the PSP.
> The filtered CPUID values are saved on a special page for the guest to
> consume. Reserve a page in MEMFD that will contain the results of
> filtered CPUID values.

Is the format of the page documented somewhere?
Is this snp-specific?  Or could this also be used without snp?

thanks,
  Gerd

Re: [PATCH v6 06/29] OvmfPkg/ResetVector: pre-validate the data pages used in SEC phase

[Gerd Hoffmann]

  Hi,

> During the guest creation time, the VMM encrypts the OVMF_CODE.fd using
> the SEV-SNP firmware provided LAUNCH_UPDATE_DATA command. In addition to
> encrypting the content, the command also validates the memory region.
> This allows us to execute the code without going through the validation
> sequence.

Hmm, tdx must handle this too.

> +  gUefiOvmfPkgTokenSpaceGuid.PcdOvmfSnpHypervisorPreValidatedStart|0x0|UINT32|0x56
> +  gUefiOvmfPkgTokenSpaceGuid.PcdOvmfSnpHypervisorPreValidatedEnd|0x0|UINT32|0x57

So maybe we should drop the "Snp" from the name here ...

>  ; GUID (SEV-SNP boot block): bd39c0c2-2f8e-4243-83e8-1b74cebcb7d9
>  ;
>  sevSnpBootBlockStart:
> +    DD      SNP_HV_VALIDATED_START
> +    DD      SNP_HV_VALIDATED_END

... and store the range which needs validation in another, not snp-specific block?

Jiewen? Min?

take care,
  Gerd

Re: [PATCH v6 15/29] OvmfPkg/MemEncryptSevLib: add support to validate system RAM

[Gerd Hoffmann]

  Hi,

> During the guest creation, the boot ROM memory is pre-validated by the
> AMD-SEV firmware. The MemEncryptSevSnpValidateSystemRam() can be called
> during the SEC and PEI phase to validate the detected system RAM.

[ for this and the following few patches ]

So, sev-snp and tdx have the same fundamental requirement here.  Both
must track what the state of page ranges is.  Both must talk to the vmm
before they can actually use pages.  snp calls this "validate", tdx
"accept", but the underlying concept should be roughly comparable.

The vmm part obviously needs to be different.  I can't see any good
reason why the state tracking and the state hand over from one boot
stage to the next (vmm -> sec -> pei -> dxe -> os) should be different
though.  Having a common workflow makes long-term maintenance and
testing easier.

    So, can you all look at each others patches and find common ground
    here?  It worked out well for the WorkArea, so *please* continue
    that way.

This series seems to side-step most of this by validating all memory in
the pei stage, so there is nothing to hand over to dxe and os.  Only the
range where the compressed code gets uncompressed to must be passed from
sec to pei.  And there is the memory range registered for pre-validation
by the vmm.

Intels (long-term?) plan seems to be to do lazily validate/accept
memory, triggered by memory allocations, to reduce boot time.  Which
implies the dxe memory management core needs to be aware of page state
and invoke an vmm-specific protocol to handle validation/acceptance.
Concept posted to the list earlier this week.  Slides only so far, no
patches yet.

take care,
  Gerd

Re: [PATCH v6 02/29] OvmfPkg: reserve CPUID page for SEV-SNP

[Brijesh Singh]

Hi Gerd,

On 9/2/21 3:04 AM, Gerd Hoffmann wrote:
> On Wed, Sep 01, 2021 at 11:16:19AM -0500, Brijesh Singh wrote:
>> BZ: https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fbugzilla.tianocore.org%2Fshow_bug.cgi%3Fid%3D3275&data=04%7C01%7Cbrijesh.singh%40amd.com%7C13c81a39aa2e4f22430e08d96de85a69%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637661666978547521%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=4b22Sv6xoUGQ3xutPYdsqb4cNh1SS9Z8MOQG7dHiqYU%3D&reserved=0
>>
>> Platform features and capabilities are traditionally discovered via the
>> CPUID instruction. Hypervisors typically trap and emulate the CPUID
>> instruction for a variety of reasons. There are some cases where incorrect
>> CPUID information can potentially lead to a security issue. The SEV-SNP
>> firmware provides a feature to filter the CPUID results through the PSP.
>> The filtered CPUID values are saved on a special page for the guest to
>> consume. Reserve a page in MEMFD that will contain the results of
>> filtered CPUID values.
> Is the format of the page documented somewhere?

Yes, it is documented in the SEV-SNP spec [1] section 7.1 and the checks
performed by the SEV-SNP firmware are documented in the PPR [2] section
2.1.5.3. I will document these link in the commit message.

[1] https://www.amd.com/system/files/TechDocs/56860.pdf

[2]
https://www.amd.com/en/support/tech-docs/processor-programming-reference-ppr-for-amd-family-19h-model-01h-revision-b1


> Is this snp-specific?  Or could this also be used without snp?

This is SNP specific format and cannot be used without SNP.

thanks

Brijesh

Re: [PATCH v6 15/29] OvmfPkg/MemEncryptSevLib: add support to validate system RAM

[Brijesh Singh]

On 9/2/21 4:50 AM, Gerd Hoffmann wrote:
>   Hi,
>
>> During the guest creation, the boot ROM memory is pre-validated by the
>> AMD-SEV firmware. The MemEncryptSevSnpValidateSystemRam() can be called
>> during the SEC and PEI phase to validate the detected system RAM.
> [ for this and the following few patches ]
>
> So, sev-snp and tdx have the same fundamental requirement here.  Both
> must track what the state of page ranges is.  Both must talk to the vmm
> before they can actually use pages.  snp calls this "validate", tdx
> "accept", but the underlying concept should be roughly comparable.

Yes, both the technologies need to accept/validate the pages before
access. Both the hypervisor and guest implementation will be different.
e.g., in SNP, communication to the hypervisor is done through the
VMGEXIT defined in the GHCB spec, whereas TDX will follow a separate
exit on how it reaches the hypervisor calls the TDX shim, etc. All the
platform-specific details on how the validation is executed should live
inside the vendor-specific libraries. That is why I added all the
validation flow in MemEncryptSevLib (which is AMD SEV specification
library that provides routines to change the page state etc). The Intel
TDX patches add a similar flow in the TDX specific library and Ovmf core
calls it. Once both the libraries are in, we can develop a shared
library that OVMF can use.

IIRC, in the TDX proposal, I got the impression that TDX implementation
will skip the PEI phase, and it jumps from SEC->DXE. The SEC phase
somehow will know the guest memory range and validate it. For SEV-SNP,
we are trying to stick to the existing boot flow and validate the memory
as soon as its discovered during the PEI phase.

As explained in the commit, there are multiple options we can take to
validate the guest memory.
1) The OVMF PEI validates the entire RAM while its discovers it.
2) The OVMF memory probe routine marks the discovered system RAM as
"Unaccepted".
3) The OVMF memory probe routine does nothing during the discovery phase.

Each of these approaches have advantages and disadvantage

Approach #1

The main advantage is that EDK2 core and guest OS can accept the memory
without any validation step. It will be slower because validation will
require the hypervisor to exist and also touch the memory. For SNP, a
new VMGEXIT was added in the GHCB spec that can be used to batch
multiple validation requests. e.g. one PSC (Page State Change) can
contain up to 253 entries, and each entry can cover up to 2MB. In other
words, we can validate up to 500MB in one VMGEXIT.

Approach #2

The main advantage of this approach is that it can support lazy
validation, and it can undoubtedly help reduce boot time. But to support
this, the EDK2 core memory management needs to be enhanced to know the
unaccepted memory type and validate it before access. The EDK2 core
should also build the EFI memory map to communicate to the guest OS on
validated so that guest OS does not double validation.

For this to work, the guest OS needs to know how to deal with the
unaccepted EFI memory range.

Approach #3

Validate memory on demand. In SNP hardware, access to the unvalidated
page causes a #VC. The guest BIOS and OS can use the #VC
(page-not-validated) exception and validate the range from exception
handler itself. It looks attractive until we run into a situation where
the guest is doing a memset() of significant memory, and each access is
causing the #VC and thus making the boot or run slow on the first access.

This patch series implements #1. And we will be looking at approach #2
after the base is enabled. Approach #2 builds upon #1. As you
highlighted below that we have not seen the patches for the Lazy
validation here so its hard to comment but I am hopeful that it will
integrated just fine with the SNP.


> The vmm part obviously needs to be different.  I can't see any good
> reason why the state tracking and the state hand over from one boot
> stage to the next (vmm -> sec -> pei -> dxe -> os) should be different
> though.  Having a common workflow makes long-term maintenance and
> testing easier.
>
>     So, can you all look at each others patches and find common ground
>     here?  It worked out well for the WorkArea, so *please* continue
>     that way.
>
> This series seems to side-step most of this by validating all memory in
> the pei stage, so there is nothing to hand over to dxe and os.  Only the
> range where the compressed code gets uncompressed to must be passed from
> sec to pei.  And there is the memory range registered for pre-validation
> by the vmm.

Yes, I am taking phased approach. Making the Lazy validation work will
require changes in both the edk2 and guest OS core memory management.


> Intels (long-term?) plan seems to be to do lazily validate/accept
> memory, triggered by memory allocations, to reduce boot time.  Which
> implies the dxe memory management core needs to be aware of page state
> and invoke an vmm-specific protocol to handle validation/acceptance.
> Concept posted to the list earlier this week.  Slides only so far, no
> patches yet.
> take care,
>   Gerd
>

Re: [PATCH v6 02/29] OvmfPkg: reserve CPUID page for SEV-SNP

[Brijesh Singh]

On 9/2/21 7:28 AM, Brijesh Singh wrote:
> Hi Gerd,
>
> On 9/2/21 3:04 AM, Gerd Hoffmann wrote:
>> On Wed, Sep 01, 2021 at 11:16:19AM -0500, Brijesh Singh wrote:
>>> BZ: https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fbugzilla.tianocore.org%2Fshow_bug.cgi%3Fid%3D3275&data=04%7C01%7Cbrijesh.singh%40amd.com%7C13c81a39aa2e4f22430e08d96de85a69%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637661666978547521%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=4b22Sv6xoUGQ3xutPYdsqb4cNh1SS9Z8MOQG7dHiqYU%3D&reserved=0
>>>
>>> Platform features and capabilities are traditionally discovered via the
>>> CPUID instruction. Hypervisors typically trap and emulate the CPUID
>>> instruction for a variety of reasons. There are some cases where incorrect
>>> CPUID information can potentially lead to a security issue. The SEV-SNP
>>> firmware provides a feature to filter the CPUID results through the PSP.
>>> The filtered CPUID values are saved on a special page for the guest to
>>> consume. Reserve a page in MEMFD that will contain the results of
>>> filtered CPUID values.
>> Is the format of the page documented somewhere?
> Yes, it is documented in the SEV-SNP spec [1] section 7.1 and the checks
> performed by the SEV-SNP firmware are documented in the PPR [2] section
> 2.1.5.3. I will document these link in the commit message.
>
> [1] https://www.amd.com/system/files/TechDocs/56860.pdf
>
> [2]
> https://www.amd.com/en/support/tech-docs/processor-programming-reference-ppr-for-amd-family-19h-model-01h-revision-b1
>
>
>> Is this snp-specific?  Or could this also be used without snp?
> This is SNP specific format and cannot be used without SNP.


I should clarify the statement, the format itself does not contain
anything  SNP specific. However, the CPUID page format is documented in
the SNP specific spec. Are you thinking about using it for non SEV guest
to avoid the VM exit ? If so, it should be very much possible. For that
we should define the format outside of SNP specific spec and make it a
generic so that guest and HV's can implement it consume it in the
non-SNP guest. 

thanks

Re: [PATCH v6 02/29] OvmfPkg: reserve CPUID page for SEV-SNP

[Gerd Hoffmann]

  Hi,

> >> Is this snp-specific?  Or could this also be used without snp?
> > This is SNP specific format and cannot be used without SNP.
> 
> I should clarify the statement, the format itself does not contain
> anything  SNP specific. However, the CPUID page format is documented in
> the SNP specific spec.

Who populates the page?  qemu?  sev-snp firmware?

> Are you thinking about using it for non SEV guest
> to avoid the VM exit ? If so, it should be very much possible.

Yes, that is the background.  Avoiding vmexits would be one advantage.
Being able to test the code without SNP-capable hardware (for example
in CI) would be another one.

> For that
> we should define the format outside of SNP specific spec and make it a
> generic so that guest and HV's can implement it consume it in the
> non-SNP guest. 

I think that would be useful.

take care,
  Gerd

Re: [PATCH v6 15/29] OvmfPkg/MemEncryptSevLib: add support to validate system RAM

[Gerd Hoffmann]

  Hi,

> IIRC, in the TDX proposal, I got the impression that TDX implementation
> will skip the PEI phase, and it jumps from SEC->DXE. The SEC phase
> somehow will know the guest memory range and validate it.

Well, their design review document lists two configurations, one (named
"config-a" in the slides) following the existing boot flow and another
("config-b") which skips PEI.

The motivation for config-b is not clear from the design review
document.  The slides describe what they are doing but there isn't much
information on _why_ things are done that way.  Asked a few days ago,
answer is still outstanding.

I'd prefer to not have two completely different initialization code
paths in ovmf.  Easier for TDX/SNP code sharing, also easier for
long-term maintenance.

> Approach #1
> 
> The main advantage is that EDK2 core and guest OS can accept the memory
> without any validation step. [ ... ]

> Approach #2
> 
> The main advantage of this approach is that it can support lazy
> validation, and it can undoubtedly help reduce boot time. [ ... ]

> This patch series implements #1. And we will be looking at approach #2
> after the base is enabled. Approach #2 builds upon #1. As you
> highlighted below that we have not seen the patches for the Lazy
> validation here so its hard to comment but I am hopeful that it will
> integrated just fine with the SNP.

Good, that plan makes sense to me.

take care,
  Gerd

Re: [edk2-devel] [PATCH v6 02/29] OvmfPkg: reserve CPUID page for SEV-SNP

[Brijesh Singh]

On 9/3/21 1:28 AM, Gerd Hoffmann via groups.io wrote:
>   Hi,
>
>>>> Is this snp-specific?  Or could this also be used without snp?
>>> This is SNP specific format and cannot be used without SNP.
>> I should clarify the statement, the format itself does not contain
>> anything  SNP specific. However, the CPUID page format is documented in
>> the SNP specific spec.
> Who populates the page?  qemu?  sev-snp firmware?

The page is populated by the QEMU and filtered by the SEV-SNP firmware.
At the end of SNP launch flow, a CPUID page will be encrypted, and
measurement of the page metadata is included in the attestation report
so that the guest owner can validate whether the hypervisor used the
CPUID page during the launch.

thanks

Re: [edk2-devel] [PATCH v6 06/29] OvmfPkg/ResetVector: pre-validate the data pages used in SEC phase

[Min Xu]

On September 2, 2021 4:20 PM, Gerd Hoffmann wrote:
>   Hi,
> 
> > During the guest creation time, the VMM encrypts the OVMF_CODE.fd
> > using the SEV-SNP firmware provided LAUNCH_UPDATE_DATA command. In
> > addition to encrypting the content, the command also validates the
> memory region.
> > This allows us to execute the code without going through the
> > validation sequence.
> 
> Hmm, tdx must handle this too.
> 
> > +
> > +
> gUefiOvmfPkgTokenSpaceGuid.PcdOvmfSnpHypervisorPreValidatedStart|0x0
> > + |UINT32|0x56
> > +
> > +
> gUefiOvmfPkgTokenSpaceGuid.PcdOvmfSnpHypervisorPreValidatedEnd|0x0|
> U
> > + INT32|0x57
> 
> So maybe we should drop the "Snp" from the name here ...
> 
> >  ; GUID (SEV-SNP boot block): bd39c0c2-2f8e-4243-83e8-1b74cebcb7d9
> >  ;
> >  sevSnpBootBlockStart:
> > +    DD      SNP_HV_VALIDATED_START
> > +    DD      SNP_HV_VALIDATED_END
> 
> ... and store the range which needs validation in another, not snp-specific
> block?
> 
> Jiewen? Min?
We pack all the Tdx information into  a blob (TdxMetadata). These tdx information
Includes the BFV(i.e. OVMF_CODE.fd), the CFV(i.e. OVMF_VARS.fd), TdMailbox, etc.
The offset to the TdxMetadata is in the GUIDed chain in ResetVectorVtf0.asm.
;
; GUID : e47a6535-984a-4798-865e-4685a7bf8ec2
;
tdxMetadataOffsetStart:
    DD      (OVMF_IMAGE_SIZE_IN_KB * 1024 - (fourGigabytes - TdxMetadataGuid - 16))
    DW      tdxMetadataOffsetEnd - tdxMetadataOffsetStart
    DB      0x35, 0x65, 0x7a, 0xe4, 0x4a, 0x98, 0x98, 0x47
    DB      0x86, 0x5e, 0x46, 0x85, 0xa7, 0xbf, 0x8e, 0xc2
tdxMetadataOffsetEnd:

In the future new metadata can be added into the TdxMetadata without changes
in ResetVectorVtf0.asm.

Thanks!
Min

Re: [edk2-devel] [PATCH v6 06/29] OvmfPkg/ResetVector: pre-validate the data pages used in SEC phase

[Gerd Hoffmann]

  Hi,

> > >  sevSnpBootBlockStart:
> > > +    DD      SNP_HV_VALIDATED_START
> > > +    DD      SNP_HV_VALIDATED_END

> We pack all the Tdx information into  a blob (TdxMetadata). These tdx information
> Includes the BFV(i.e. OVMF_CODE.fd), the CFV(i.e. OVMF_VARS.fd), TdMailbox, etc.
> The offset to the TdxMetadata is in the GUIDed chain in ResetVectorVtf0.asm.

> In the future new metadata can be added into the TdxMetadata without changes
> in ResetVectorVtf0.asm.

[ Looking at https://www.mail-archive.com/devel@edk2.groups.io/msg33605.html ]

So, there isn't much tdx-specific in tdx-metadata.  Most ranges are
TDX_METADATA_SECTION_TYPE_TEMP_MEM which I think basically means these
ranges should be accepted by the hypervisor, which is pretty much the
same issue snp tries to solve with this pre-validation range.  Then
there are the ranges for code (aka bfv), for vars (aka cfv) and td_hob.

td_hob is the only tdx-specific item there, and even that concept (pass
memory ranges as hob list from hypervisor to guest) might be useful
outside tdx.

So, can we settle on one approach for this please?  I think the
tdx-metadata style approach is more flexible and future-proof.  It can
easily be extended without changing data structures, we only need new
section types.  I expect this will work better long-term when it comes
to backward-compatibility.

I'd suggest we generalize the tdx-metadata idea and define both generic
and vmm-specific section types:

enum {
  OVMF_SECTION_TYPE_UNDEFINED = 0;

  /* generic */
  OVMF_SECTION_TYPE_CODE = 0x100,
  OVMF_SECTION_TYPE_VARS
  OVMF_SECTION_TYPE_SEC_MEM  /* vmm should accept/validate this */

  /* sev */
  OVMF_SECTION_TYPE_SEV_SECRETS = 0x200,
  OVMF_SECTION_TYPE_SEV_CPUID /* or move to generic? */

  /* tdx */
  OVMV_SECTION_TYPE_TDX_TD_HOB = 0x300,
};

Comments?

Looking at tdx-metadata I have a few questions:

+_Bfv:
+  DD TDX_BFV_RAW_DATA_OFFSET
+  DD TDX_BFV_RAW_DATA_SIZE

What is this and why is it needed?

+  DQ TDX_BFV_MEMORY_BASE
+  DQ TDX_BFV_MEMORY_SIZE

Why "DQ"?  TDX is defined to start in 32bit mode, so you can hardly have
addresses here which do not fit into "DD", correct?

+  DD TDX_METADATA_SECTION_TYPE_BFV
+  DD TDX_METADATA_ATTRIBUTES_EXTENDMR

What does this attribute mean?

take care,
  Gerd

Re: [edk2-devel] [PATCH v6 06/29] OvmfPkg/ResetVector: pre-validate the data pages used in SEC phase

[Min Xu]

On September 6, 2021 8:17 PM, Gerd Hoffman wrote:
>   Hi,
> 
> > > >  sevSnpBootBlockStart:
> > > > +    DD      SNP_HV_VALIDATED_START
> > > > +    DD      SNP_HV_VALIDATED_END
> 
> > We pack all the Tdx information into  a blob (TdxMetadata). These tdx
> > information Includes the BFV(i.e. OVMF_CODE.fd), the CFV(i.e.
> OVMF_VARS.fd), TdMailbox, etc.
> > The offset to the TdxMetadata is in the GUIDed chain in
> ResetVectorVtf0.asm.
> 
> > In the future new metadata can be added into the TdxMetadata without
> > changes in ResetVectorVtf0.asm.
> 
> [ Looking at https://www.mail-
> archive.com/devel@edk2.groups.io/msg33605.html ]
> 
> So, there isn't much tdx-specific in tdx-metadata.  Most ranges are
> TDX_METADATA_SECTION_TYPE_TEMP_MEM which I think basically means
> these ranges should be accepted by the hypervisor, which is pretty much the
> same issue snp tries to solve with this pre-validation range.  Then there are
> the ranges for code (aka bfv), for vars (aka cfv) and td_hob.
> 
> td_hob is the only tdx-specific item there, and even that concept (pass
> memory ranges as hob list from hypervisor to guest) might be useful outside
> tdx.
Mailbox is tdx-specific too. But Stack/Heap/OvmfWorkarea/OvmfPageTable are
common. BFV/CFV are common too.
> 
> So, can we settle on one approach for this please?  I think the tdx-metadata
> style approach is more flexible and future-proof.  It can easily be extended
> without changing data structures, we only need new section types.  I expect
> this will work better long-term when it comes to backward-compatibility.
> 
> I'd suggest we generalize the tdx-metadata idea and define both generic and
> vmm-specific section types:
> 
> enum {
>   OVMF_SECTION_TYPE_UNDEFINED = 0;
> 
>   /* generic */
>   OVMF_SECTION_TYPE_CODE = 0x100,
>   OVMF_SECTION_TYPE_VARS
>   OVMF_SECTION_TYPE_SEC_MEM  /* vmm should accept/validate this */
> 
>   /* sev */
>   OVMF_SECTION_TYPE_SEV_SECRETS = 0x200,
>   OVMF_SECTION_TYPE_SEV_CPUID /* or move to generic? */
> 
>   /* tdx */
>   OVMV_SECTION_TYPE_TDX_TD_HOB = 0x300,
> };
> 
> Comments?
TDX has similar section type.
But I am not sure if SEV can use this metadata mechanism.
Need SEV's comments.
> 
> Looking at tdx-metadata I have a few questions:
> 
> +_Bfv:
> +  DD TDX_BFV_RAW_DATA_OFFSET
> +  DD TDX_BFV_RAW_DATA_SIZE
> 
> What is this and why is it needed?
Host VMM need to measure the code part (BFV) to MRTD register
(which is similar to TPM PCRs).
> 
> +  DQ TDX_BFV_MEMORY_BASE
> +  DQ TDX_BFV_MEMORY_SIZE
> 
> Why "DQ"?  TDX is defined to start in 32bit mode, so you can hardly have
> addresses here which do not fit into "DD", correct?
Those are the memory. TDX is running in long mode. So it is DQ.
> 
> +  DD TDX_METADATA_SECTION_TYPE_BFV
> +  DD TDX_METADATA_ATTRIBUTES_EXTENDMR
> 
> What does this attribute mean?
It indicates if the host VMM should use TDCALL[TDH.MR.EXTEND] for this section.
https://software.intel.com/content/dam/develop/external/us/en/documents/tdx-virtual-firmware-design-guide-rev-1.pdf
Please refer to Section 11 TDVF Metadata for more detailed information.
Note: this Section will be updated according to the comments from the community.
> 

Thanks!
Min

Re: [PATCH v6 00/29] Add AMD Secure Nested Paging (SEV-SNP) support

[Yao, Jiewen]

Thank you Brijesh
It took me a while to review this series. Here is my feedback.
I am not sure what you prefer, to put all comment together? Or reply 29 email separately?
Let me put them together in this version. If you prefer a different way, please let me know.

My strategy is same as previous. I will focus on common part and review as detail as possible.
For SEV specific thing, I will ACK and let AMD people make decision unless I have big concern on the design.
You can add my A-B and R-B in next version.


0001-OvmfPkg-reserve-SNP-secrets-page
Reviewed-by: Jiewen Yao <Jiewen.yao@intel.com>

0002-OvmfPkg-reserve-CPUID-page-for-SEV-SNP
Reviewed-by: Jiewen Yao <Jiewen.yao@intel.com>

0003-OvmfPkg-ResetVector-introduce-SEV-SNP-boot-block-GUID
I am still thinking if it is possible to move all SEV define GUID blob to a standalone file, and TDX define GUID blob to another file.
Anyway, that can be done later.
Reviewed-by: Jiewen Yao <Jiewen.yao@intel.com>

0004-OvmfPkg-ResetVector-invalidate-the-GHCB-page
Acked-by: Jiewen Yao <Jiewen.yao@intel.com>

0005-OvmfPkg-ResetVector-check-the-vmpl-level
Acked-by: Jiewen Yao <Jiewen.yao@intel.com>

0006-OvmfPkg-ResetVector-pre-validate-the-data-pages-used-in-SEC-phase
Acked-by: Jiewen Yao <Jiewen.yao@intel.com>

0007-OvmfPkg-ResetVector-use-SEV-SNP-validated-CPUID-values
Acked-by: Jiewen Yao <Jiewen.yao@intel.com>

0008-UefiCpuPkg-Define-the-SEV-SNP-specific-dynamic-PCDs
I really don't like the idea to use BOOL PcdSevEsIsEnabled and PcdSevSnpIsEnabled.
Can we define *one* PCD - such as PcdConfidentialComputingCategory?
We can assign range 0x0000~0xFFFF to AMD SEV, 0x10000~0x1FFFF to Intel TDX.
Then SEV=0x0000, SEV-ES=0x0001, SEV-SNP=0x0002, and TDX=0x10000 later.
I really don't want to keep adding PCD endlessly in the future, like PcdSevXXXIsEnabled, PcdSevYYYIsEnabled, PcdTdxIsEnabled, PcdTdx20Enabled, PcdTdx30Enabled, ......


0009-OvmfPkg-MemEncryptSevLib-add-MemEncryptSevSnpEnabled()
I am not sure since we have PCD in 0008, why we need to expose the function - MemEncryptSevSnpIsEnabled() and MemEncryptSevEsIsEnabled()?
Should we always use PCD anywhere else?
Anyway, Acked-by: Jiewen Yao <Jiewen.yao@intel.com>

0010-OvmfPkg-SecMain-move-SEV-specific-routines-in-AmdSev.c
Reviewed-by: Jiewen Yao <Jiewen.yao@intel.com>

0011-OvmfPkg-SecMain-register-GHCB-gpa-for-the-SEV-SNP-guest
Acked-by: Jiewen Yao <Jiewen.yao@intel.com>

0012-OvmfPkg-VmgExitLib-use-SEV-SNP-validated-CPUID-values
Acked-by: Jiewen Yao <Jiewen.yao@intel.com>

0013-OvmfPkg-PlatformPei-register-GHCB-gpa-for-the-SEV-SNP-guest
Acked-by: Jiewen Yao <Jiewen.yao@intel.com>

0014-OvmfPkg-AmdSevDxe-do-not-use-extended-PCI-config-space
Acked-by: Jiewen Yao <Jiewen.yao@intel.com>

0015-OvmfPkg-MemEncryptSevLib-add-support-to-validate-system-RAM
Acked-by: Jiewen Yao <Jiewen.yao@intel.com>

0016-OvmfPkg-BaseMemEncryptSevLib-skip-the-pre-validated-system-RAM
Acked-by: Jiewen Yao <Jiewen.yao@intel.com>

0017-OvmfPkg-MemEncryptSevLib-add-support-to-validate-4GB-memory-in-PEI-phase
Acked-by: Jiewen Yao <Jiewen.yao@intel.com>

0018-OvmfPkg-SecMain-pre-validate-the-memory-used-for-decompressing-Fv
Acked-by: Jiewen Yao <Jiewen.yao@intel.com>

0019-OvmfPkg-PlatformPei-validate-the-system-RAM-when-SNP-is-active
Acked-by: Jiewen Yao <Jiewen.yao@intel.com>

0020-OvmfPkg-PlatformPei-set-the-SEV-SNP-enabled-PCD
See 0008

0021-OvmfPkg-PlatformPei-set-the-Hypervisor-Features-PCD
Acked-by: Jiewen Yao <Jiewen.yao@intel.com>

0022-MdePkg-GHCB-increase-the-GHCB-protocol-max-version
Acked-by: Jiewen Yao <Jiewen.yao@intel.com>

0023-UefiCpuPkg-MpLib-add-support-to-register-GHCB-GPA-when-SEV-SNP-is-enabled
1) See 0008.
2) For MpFuncs.nasm, I recommend to move AmdSev specific initialization to a standalone file, such as Sev.nasm

0024-UefiCpuPkg-MpInitLib-use-BSP-to-do-extended-topology-check
See 0023

0025-OvmfPkg-MemEncryptSevLib-change-the-page-state-in-the-RMP-table
Acked-by: Jiewen Yao <Jiewen.yao@intel.com>

0026-OvmfPkg-MemEncryptSevLib-skip-page-state-change-for-Mmio-address
Acked-by: Jiewen Yao <Jiewen.yao@intel.com>

0027-OvmfPkg-PlatformPei-mark-cpuid-and-secrets-memory-reserved-in-EFI-map
Would you please move SEV specific init to another Sev.c?
Also I found MemEncryptSevEsIsEnabled() and MemEncryptSevSnpIsEnabled() are there.
I suggest just use one API
MemEncryptSevEsIsEnabled() {
    DoSevInitializeRamRegions()
}
Then you can check more in DoSevInitializeRamRegions().
DoSevInitializeRamRegions() {
  MemEncryptSevSnpIsEnabled() {
  }
}

0028-OvmfPkg-AmdSev-expose-the-SNP-reserved-pages-through-configuration-table
I am not convinced to include SEV specific data structure in a generic structure in ConfidentialComputingSecret.h.
I recommend moving it to SEV specific file.

0029-UefiCpuPkg-MpInitLib-Use-SEV-SNP-AP-Creation-NAE-event-to-launch-APs
See 0008, 0023.
I recommend to move SevSnpCreateSaveArea() to Sev.c.

Thank you
Yao Jiewen



> -----Original Message-----
> From: Brijesh Singh <brijesh.singh@amd.com>
> Sent: Thursday, September 2, 2021 12:16 AM
> To: devel@edk2.groups.io
> Cc: James Bottomley <jejb@linux.ibm.com>; Xu, Min M <min.m.xu@intel.com>;
> Yao, Jiewen <jiewen.yao@intel.com>; Tom Lendacky
> <thomas.lendacky@amd.com>; Justen, Jordan L <jordan.l.justen@intel.com>;
> Ard Biesheuvel <ardb+tianocore@kernel.org>; Erdem Aktas
> <erdemaktas@google.com>; Michael Roth <Michael.Roth@amd.com>; Gerd
> Hoffmann <kraxel@redhat.com>; Brijesh Singh <brijesh.singh@amd.com>
> Subject: [PATCH v6 00/29] Add AMD Secure Nested Paging (SEV-SNP) support
> 
> BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=3275
> 
> SEV-SNP builds upon existing SEV and SEV-ES functionality while adding
> new hardware-based memory protections. SEV-SNP adds strong memory
> integrity
> protection to help prevent malicious hypervisor-based attacks like data
> replay, memory re-mapping and more in order to create an isolated memory
> encryption environment.
> 
> This series provides the basic building blocks to support booting the SEV-SNP
> VMs, it does not cover all the security enhancement introduced by the SEV-SNP
> such as interrupt protection.
> 
> Many of the integrity guarantees of SEV-SNP are enforced through a new
> structure called the Reverse Map Table (RMP). Adding a new page to SEV-SNP
> VM requires a 2-step process. First, the hypervisor assigns a page to the
> guest using the new RMPUPDATE instruction. This transitions the page to
> guest-invalid. Second, the guest validates the page using the new PVALIDATE
> instruction. The SEV-SNP VMs can use the new "Page State Change Request
> NAE"
> defined in the GHCB specification to ask hypervisor to add or remove page
> from the RMP table.
> 
> Each page assigned to the SEV-SNP VM can either be validated or unvalidated,
> as indicated by the Validated flag in the page's RMP entry. There are two
> approaches that can be taken for the page validation: Pre-validation and
> Lazy Validation.
> 
> Under pre-validation, the pages are validated prior to first use. And under
> lazy validation, pages are validated when first accessed. An access to a
> unvalidated page results in a #VC exception, at which time the exception
> handler may validate the page. Lazy validation requires careful tracking of
> the validated pages to avoid validating the same GPA more than once. The
> recently introduced "Unaccepted" memory type can be used to communicate
> the
> unvalidated memory ranges to the Guest OS.
> 
> At this time we only support the pre-validation. OVMF detects all the available
> system RAM in the PEI phase. When SEV-SNP is enabled, the memory is validated
> before it is made available to the EDK2 core.
> 
> Now that series contains all the basic support required to launch SEV-SNP
> guest. We are still missing the Interrupt security feature provided by the
> SNP. The feature will be added after the base support is accepted.
> 
> Additional resources
> ---------------------
> SEV-SNP whitepaper
> https://www.amd.com/system/files/TechDocs/SEV-SNP-strengthening-vm-
> isolation-with-integrity-protection-and-more.pdf
> 
> APM 2: https://www.amd.com/system/files/TechDocs/24593.pdf (section 15.36)
> 
> The complete source is available at
> https://github.com/AMDESE/ovmf/tree/sev-snp-rfc-5
> 
> GHCB spec:
> https://developer.amd.com/wp-content/resources/56421.pdf
> 
> SEV-SNP firmware specification:
> https://www.amd.com/system/files/TechDocs/56860.pdf
> 
> Change since v5:
>  * When possible use the CPUID value from CPUID page
>  * Move the SEV specific functions from SecMain.c in AmdSev.c
>  * Rebase to the latest code
>  * Add the review feedback from Yao.
> 
> Change since v4:
>  * Use the correct MSR for the SEV_STATUS
>  * Add VMPL-0 check
> 
> Change since v3:
>  * ResetVector: move all SEV specific code in AmdSev.asm and add macros to
> keep
>    the code readable.
>  * Drop extending the EsWorkArea to contain SNP specific state.
>  * Drop the GhcbGpa library and call the VmgExit directly to register GHCB GPA.
>  * Install the CC blob config table from AmdSevDxe instead of extending the
>    AmdSev/SecretsDxe for it.
>  * Add the separate PCDs for the SNP Secrets.
> 
> Changes since v2:
>  * Add support for the AP creation.
>  * Use the module-scoping override to make AmdSevDxe use the IO port for PCI
> reads.
>  * Use the reserved memory type for CPUID and Secrets page.
>  *
> Changes since v1:
>  * Drop the interval tree support to detect the pre-validated overlap region.
>  * Use an array to keep track of pre-validated regions.
>  * Add support to query the Hypervisor feature and verify that SNP feature is
> supported.
>  * Introduce MemEncryptSevClearMmioPageEncMask() to clear the C-bit from
> MMIO ranges.
>  * Pull the SevSecretDxe and SevSecretPei into OVMF package build.
>  * Extend the SevSecretDxe to expose confidential computing blob location
> through
>    EFI configuration table.
> 
> Brijesh Singh (25):
>   OvmfPkg: reserve SNP secrets page
>   OvmfPkg: reserve CPUID page for SEV-SNP
>   OvmfPkg/ResetVector: introduce SEV-SNP boot block GUID
>   OvmfPkg/ResetVector: invalidate the GHCB page
>   OvmfPkg/ResetVector: check the vmpl level
>   OvmfPkg/ResetVector: pre-validate the data pages used in SEC phase
>   UefiCpuPkg: Define the SEV-SNP specific dynamic PCDs
>   OvmfPkg/MemEncryptSevLib: add MemEncryptSevSnpEnabled()
>   OvmfPkg/SecMain: move SEV specific routines in AmdSev.c
>   OvmfPkg/SecMain: register GHCB gpa for the SEV-SNP guest
>   OvmfPkg/PlatformPei: register GHCB gpa for the SEV-SNP guest
>   OvmfPkg/AmdSevDxe: do not use extended PCI config space
>   OvmfPkg/MemEncryptSevLib: add support to validate system RAM
>   OvmfPkg/BaseMemEncryptSevLib: skip the pre-validated system RAM
>   OvmfPkg/MemEncryptSevLib: add support to validate > 4GB memory in PEI
>     phase
>   OvmfPkg/SecMain: pre-validate the memory used for decompressing Fv
>   OvmfPkg/PlatformPei: validate the system RAM when SNP is active
>   OvmfPkg/PlatformPei: set the SEV-SNP enabled PCD
>   OvmfPkg/PlatformPei: set the Hypervisor Features PCD
>   MdePkg/GHCB: increase the GHCB protocol max version
>   UefiCpuPkg/MpLib: add support to register GHCB GPA when SEV-SNP is
>     enabled
>   OvmfPkg/MemEncryptSevLib: change the page state in the RMP table
>   OvmfPkg/MemEncryptSevLib: skip page state change for Mmio address
>   OvmfPkg/PlatformPei: mark cpuid and secrets memory reserved in EFI map
>   OvmfPkg/AmdSev: expose the SNP reserved pages through configuration
>     table
> 
> Michael Roth (3):
>   OvmfPkg/ResetVector: use SEV-SNP-validated CPUID values
>   OvmfPkg/VmgExitLib: use SEV-SNP-validated CPUID values
>   UefiCpuPkg/MpInitLib: use BSP to do extended topology check
> 
> Tom Lendacky (1):
>   UefiCpuPkg/MpInitLib: Use SEV-SNP AP Creation NAE event to launch APs
> 
>  OvmfPkg/OvmfPkg.dec                           |  23 +
>  UefiCpuPkg/UefiCpuPkg.dec                     |  11 +
>  OvmfPkg/AmdSev/AmdSevX64.dsc                  |   5 +-
>  OvmfPkg/Bhyve/BhyveX64.dsc                    |   5 +-
>  OvmfPkg/OvmfPkgIa32.dsc                       |   1 +
>  OvmfPkg/OvmfPkgIa32X64.dsc                    |   6 +-
>  OvmfPkg/OvmfPkgX64.dsc                        |   5 +-
>  OvmfPkg/OvmfXen.dsc                           |   5 +-
>  OvmfPkg/OvmfPkgX64.fdf                        |  12 +-
>  OvmfPkg/AmdSevDxe/AmdSevDxe.inf               |   7 +
>  .../DxeMemEncryptSevLib.inf                   |   3 +
>  .../PeiMemEncryptSevLib.inf                   |   7 +
>  .../SecMemEncryptSevLib.inf                   |   3 +
>  OvmfPkg/Library/VmgExitLib/SecVmgExitLib.inf  |   2 +
>  OvmfPkg/Library/VmgExitLib/VmgExitLib.inf     |   3 +
>  OvmfPkg/PlatformPei/PlatformPei.inf           |  10 +
>  OvmfPkg/ResetVector/ResetVector.inf           |   6 +
>  OvmfPkg/Sec/SecMain.inf                       |   4 +
>  UefiCpuPkg/Library/MpInitLib/DxeMpInitLib.inf |   4 +
>  UefiCpuPkg/Library/MpInitLib/PeiMpInitLib.inf |   4 +
>  MdePkg/Include/Register/Amd/Ghcb.h            |   2 +-
>  .../Guid/ConfidentialComputingSecret.h        |  18 +
>  OvmfPkg/Include/Library/MemEncryptSevLib.h    |  26 +
>  .../X64/SnpPageStateChange.h                  |  31 ++
>  .../BaseMemEncryptSevLib/X64/VirtualMemory.h  |  19 +
>  OvmfPkg/Sec/AmdSev.h                          |  95 ++++
>  UefiCpuPkg/Library/MpInitLib/MpLib.h          |  20 +
>  OvmfPkg/AmdSevDxe/AmdSevDxe.c                 |  23 +
>  .../DxeMemEncryptSevLibInternal.c             |  27 ++
>  .../Ia32/MemEncryptSevLib.c                   |  17 +
>  .../PeiMemEncryptSevLibInternal.c             |  27 ++
>  .../SecMemEncryptSevLibInternal.c             |  19 +
>  .../X64/DxeSnpSystemRamValidate.c             |  40 ++
>  .../X64/PeiDxeVirtualMemory.c                 | 167 ++++++-
>  .../X64/PeiSnpSystemRamValidate.c             | 126 +++++
>  .../X64/SecSnpSystemRamValidate.c             |  36 ++
>  .../X64/SnpPageStateChangeInternal.c          | 295 ++++++++++++
>  OvmfPkg/Library/VmgExitLib/VmgExitVcHandler.c | 444 ++++++++++++++++--
>  OvmfPkg/PlatformPei/AmdSev.c                  | 192 ++++++++
>  OvmfPkg/PlatformPei/MemDetect.c               |  21 +
>  OvmfPkg/Sec/AmdSev.c                          | 267 +++++++++++
>  OvmfPkg/Sec/SecMain.c                         | 160 +------
>  UefiCpuPkg/Library/MpInitLib/DxeMpLib.c       |  11 +-
>  .../MpInitLib/Ia32/SevSnpRmpAdjustInternal.c  |  31 ++
>  UefiCpuPkg/Library/MpInitLib/MpLib.c          | 286 ++++++++++-
>  .../MpInitLib/X64/SevSnpRmpAdjustInternal.c   |  44 ++
>  OvmfPkg/FvmainCompactScratchEnd.fdf.inc       |   5 +
>  OvmfPkg/ResetVector/Ia16/ResetVectorVtf0.asm  |  28 ++
>  OvmfPkg/ResetVector/Ia32/AmdSev.asm           | 307 +++++++++++-
>  OvmfPkg/ResetVector/ResetVector.nasmb         |   6 +
>  UefiCpuPkg/Library/MpInitLib/MpEqu.inc        |   2 +
>  UefiCpuPkg/Library/MpInitLib/X64/MpFuncs.nasm |  78 +++
>  52 files changed, 2771 insertions(+), 225 deletions(-)
>  create mode 100644
> OvmfPkg/Library/BaseMemEncryptSevLib/X64/SnpPageStateChange.h
>  create mode 100644 OvmfPkg/Sec/AmdSev.h
>  create mode 100644
> OvmfPkg/Library/BaseMemEncryptSevLib/X64/DxeSnpSystemRamValidate.c
>  create mode 100644
> OvmfPkg/Library/BaseMemEncryptSevLib/X64/PeiSnpSystemRamValidate.c
>  create mode 100644
> OvmfPkg/Library/BaseMemEncryptSevLib/X64/SecSnpSystemRamValidate.c
>  create mode 100644
> OvmfPkg/Library/BaseMemEncryptSevLib/X64/SnpPageStateChangeInternal.c
>  create mode 100644 OvmfPkg/Sec/AmdSev.c
>  create mode 100644
> UefiCpuPkg/Library/MpInitLib/Ia32/SevSnpRmpAdjustInternal.c
>  create mode 100644
> UefiCpuPkg/Library/MpInitLib/X64/SevSnpRmpAdjustInternal.c
> 
> --
> 2.17.1

Re: [edk2-devel] [PATCH v6 06/29] OvmfPkg/ResetVector: pre-validate the data pages used in SEC phase

[Gerd Hoffmann]

  Hi,

> > [ Looking at https://www.mail-
> > archive.com/devel@edk2.groups.io/msg33605.html ]
> > 
> > So, there isn't much tdx-specific in tdx-metadata.  Most ranges are
> > TDX_METADATA_SECTION_TYPE_TEMP_MEM which I think basically means
> > these ranges should be accepted by the hypervisor, which is pretty much the
> > same issue snp tries to solve with this pre-validation range.  Then there are
> > the ranges for code (aka bfv), for vars (aka cfv) and td_hob.
> > 
> > td_hob is the only tdx-specific item there, and even that concept (pass
> > memory ranges as hob list from hypervisor to guest) might be useful outside
> > tdx.
> Mailbox is tdx-specific too. But Stack/Heap/OvmfWorkarea/OvmfPageTable are
> common. BFV/CFV are common too.

Mailbox is tagged "TDX_METADATA_SECTION_TYPE_TEMP_MEM", so nothing
special to do when loading the firmware, right?

> > I'd suggest we generalize the tdx-metadata idea and define both generic and
> > vmm-specific section types:
> > 
> > enum {
> >   OVMF_SECTION_TYPE_UNDEFINED = 0;
> > 
> >   /* generic */
> >   OVMF_SECTION_TYPE_CODE = 0x100,
> >   OVMF_SECTION_TYPE_VARS
> >   OVMF_SECTION_TYPE_SEC_MEM  /* vmm should accept/validate this */
> > 
> >   /* sev */
> >   OVMF_SECTION_TYPE_SEV_SECRETS = 0x200,
> >   OVMF_SECTION_TYPE_SEV_CPUID /* or move to generic? */
> > 
> >   /* tdx */
> >   OVMV_SECTION_TYPE_TDX_TD_HOB = 0x300,
> > };
> > 
> > Comments?
> TDX has similar section type.

Yes.  Both TDX and SNP have simliar requirements, they want store memory
ranges in the firmware binary in a way that allows qemu finding them and
using them when initializing the guest.

SNP stores the ranges directly in the GUID-chained block in the reset
vector.  The range types are implicit (first is pre-validate area,
second is cpuid page, ...).

TDX stores a pointer to tdx-metadata in the GUID-chained block, then the
tdx-metadata has a list of ranges.  The ranges are explicitly typed
(section type field).

The indirection used by TDX keeps the reset vector small.  Also the
explicit typing of the ranges makes it easier to extend later on if
needed.

IMHO SEV should at minimum add explicit types to the memory ranges in
the boot block, but I'd very much prefer it if SEV and TDX can agree
on a way to store the memory ranges.

> But I am not sure if SEV can use this metadata mechanism.
> Need SEV's comments.

Brijesh?

> > Looking at tdx-metadata I have a few questions:
> > 
> > +_Bfv:
> > +  DD TDX_BFV_RAW_DATA_OFFSET
> > +  DD TDX_BFV_RAW_DATA_SIZE
> > 
> > What is this and why is it needed?
> Host VMM need to measure the code part (BFV) to MRTD register
> (which is similar to TPM PCRs).

Sure, but why you can't use TDX_BFV_MEMORY_BASE + TDX_BFV_MEMORY_SIZE
for that?  The tdvf design guide says it is the file offset.  The
firmware must be mapped right below 4G, which implies
RAW_DATA_OFFSET + (4G - FIRMWARE_SIZE) == MEMORY_BASE.
So having both RAW_DATA_OFFSET and MEMORY_BASE looks redundant to me.

> > +  DQ TDX_BFV_MEMORY_BASE
> > +  DQ TDX_BFV_MEMORY_SIZE
> > 
> > Why "DQ"?  TDX is defined to start in 32bit mode, so you can hardly have
> > addresses here which do not fit into "DD", correct?
> Those are the memory. TDX is running in long mode. So it is DQ.

Hmm?  TDX entry vector is defined to be 32bit.  Which pretty much
implies you can't map the firmware above 4G, even if one of the first
actions of the reset vector code is the switch to long mode.

take care,
  Gerd

Re: [edk2-devel] [PATCH v6 06/29] OvmfPkg/ResetVector: pre-validate the data pages used in SEC phase

[Brijesh Singh]

On 9/7/21 2:07 AM, kraxel@redhat.com wrote:
>    Hi,
> 
>>> [ Looking at https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.mail-%2F&data=04%7C01%7Cbrijesh.singh%40amd.com%7Cb2dfcc7e0f934cacdce408d971ce2f5a%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637665952633333113%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=WsSaU8kFvw1NWES%2BYOw7xENZr9cpPgQvBjsXWkc8nkg%3D&reserved=0
>>> archive.com/devel@edk2.groups.io/msg33605.html ]
>>>
>>> So, there isn't much tdx-specific in tdx-metadata.  Most ranges are
>>> TDX_METADATA_SECTION_TYPE_TEMP_MEM which I think basically means
>>> these ranges should be accepted by the hypervisor, which is pretty much the
>>> same issue snp tries to solve with this pre-validation range.  Then there are
>>> the ranges for code (aka bfv), for vars (aka cfv) and td_hob.
>>>
>>> td_hob is the only tdx-specific item there, and even that concept (pass
>>> memory ranges as hob list from hypervisor to guest) might be useful outside
>>> tdx.
>> Mailbox is tdx-specific too. But Stack/Heap/OvmfWorkarea/OvmfPageTable are
>> common. BFV/CFV are common too.
> 
> Mailbox is tagged "TDX_METADATA_SECTION_TYPE_TEMP_MEM", so nothing
> special to do when loading the firmware, right?
> 
>>> I'd suggest we generalize the tdx-metadata idea and define both generic and
>>> vmm-specific section types:
>>>
>>> enum {
>>>    OVMF_SECTION_TYPE_UNDEFINED = 0;
>>>
>>>    /* generic */
>>>    OVMF_SECTION_TYPE_CODE = 0x100,
>>>    OVMF_SECTION_TYPE_VARS
>>>    OVMF_SECTION_TYPE_SEC_MEM  /* vmm should accept/validate this */
>>>
>>>    /* sev */
>>>    OVMF_SECTION_TYPE_SEV_SECRETS = 0x200,
>>>    OVMF_SECTION_TYPE_SEV_CPUID /* or move to generic? */
>>>
>>>    /* tdx */
>>>    OVMV_SECTION_TYPE_TDX_TD_HOB = 0x300,
>>> };
>>>
>>> Comments?
>> TDX has similar section type.
> 
> Yes.  Both TDX and SNP have simliar requirements, they want store memory
> ranges in the firmware binary in a way that allows qemu finding them and
> using them when initializing the guest.
> 
> SNP stores the ranges directly in the GUID-chained block in the reset
> vector.  The range types are implicit (first is pre-validate area,
> second is cpuid page, ...).
> 
> TDX stores a pointer to tdx-metadata in the GUID-chained block, then the
> tdx-metadata has a list of ranges.  The ranges are explicitly typed
> (section type field).
> 
> The indirection used by TDX keeps the reset vector small.  Also the
> explicit typing of the ranges makes it easier to extend later on if
> needed.
> 
> IMHO SEV should at minimum add explicit types to the memory ranges in
> the boot block, but I'd very much prefer it if SEV and TDX can agree
> on a way to store the memory ranges.
> 
>> But I am not sure if SEV can use this metadata mechanism.
>> Need SEV's comments.
> 
> Brijesh?
> 

We should be able to make use of the metadata approach for the SEV-SNP. 
I will update the SNP patches to use the metadata approach in next rev.

thanks

Re: [PATCH v6 00/29] Add AMD Secure Nested Paging (SEV-SNP) support

[Min Xu]

On September 7, 2021 10:37 AM, Jiewen Yao wrote:
> 
> 0008-UefiCpuPkg-Define-the-SEV-SNP-specific-dynamic-PCDs
> I really don't like the idea to use BOOL PcdSevEsIsEnabled and
> PcdSevSnpIsEnabled.
> Can we define *one* PCD - such as PcdConfidentialComputingCategory?
> We can assign range 0x0000~0xFFFF to AMD SEV, 0x10000~0x1FFFF to Intel TDX.
> Then SEV=0x0000, SEV-ES=0x0001, SEV-SNP=0x0002, and TDX=0x10000 later.
> I really don't want to keep adding PCD endlessly in the future, like
> PcdSevXXXIsEnabled, PcdSevYYYIsEnabled, PcdTdxIsEnabled, PcdTdx20Enabled,
> PcdTdx30Enabled, ......
> 
We have CONFIDENTIAL_COMPUTING_WORK_AREA_HEADER definition in OvmfPkg\Include\WorkArea.h like below:
typedef struct _CONFIDENTIAL_COMPUTING_WORK_AREA_HEADER {
  UINT8                   GuestType;   // 0 - legacy guest, 1 - SEV guest, 2 - tdx guest
  UINT8                   Reserved1[3];
} CONFIDENTIAL_COMPUTING_WORK_AREA_HEADER;

Can we define the PcdConfidentialComputingCategory like below:
  ## This dynamic PCD indicates the Confidential Computing Category
  #  [7:0]     Confidential Computing Category  (0 - Non-Cc, 1 - AmdSev, 2 - IntelTdx)
  #  [15:8]   Sub-Category (defined by each vendor, SEV-ES, SEV-SNP, or TDX-1.0, TDX-2.0, etc)
  #  [31:16] Reserved
  # @Prompt Confidential Computing Category
  gUefiCpuPkgTokenSpaceGuid.PcdConfidentialComputingCategory|0|UINT32|0x60000018

Thanks!
Min

Re: [PATCH v6 00/29] Add AMD Secure Nested Paging (SEV-SNP) support

[Yao, Jiewen]

Yes, that is good idea.

> -----Original Message-----
> From: Xu, Min M <min.m.xu@intel.com>
> Sent: Wednesday, September 8, 2021 10:30 AM
> To: Yao, Jiewen <jiewen.yao@intel.com>; Brijesh Singh
> <brijesh.singh@amd.com>; devel@edk2.groups.io
> Cc: James Bottomley <jejb@linux.ibm.com>; Tom Lendacky
> <thomas.lendacky@amd.com>; Justen, Jordan L <jordan.l.justen@intel.com>;
> Ard Biesheuvel <ardb+tianocore@kernel.org>; Erdem Aktas
> <erdemaktas@google.com>; Michael Roth <Michael.Roth@amd.com>; Gerd
> Hoffmann <kraxel@redhat.com>
> Subject: RE: [PATCH v6 00/29] Add AMD Secure Nested Paging (SEV-SNP)
> support
> 
> On September 7, 2021 10:37 AM, Jiewen Yao wrote:
> >
> > 0008-UefiCpuPkg-Define-the-SEV-SNP-specific-dynamic-PCDs
> > I really don't like the idea to use BOOL PcdSevEsIsEnabled and
> > PcdSevSnpIsEnabled.
> > Can we define *one* PCD - such as PcdConfidentialComputingCategory?
> > We can assign range 0x0000~0xFFFF to AMD SEV, 0x10000~0x1FFFF to Intel
> TDX.
> > Then SEV=0x0000, SEV-ES=0x0001, SEV-SNP=0x0002, and TDX=0x10000 later.
> > I really don't want to keep adding PCD endlessly in the future, like
> > PcdSevXXXIsEnabled, PcdSevYYYIsEnabled, PcdTdxIsEnabled, PcdTdx20Enabled,
> > PcdTdx30Enabled, ......
> >
> We have CONFIDENTIAL_COMPUTING_WORK_AREA_HEADER definition in
> OvmfPkg\Include\WorkArea.h like below:
> typedef struct _CONFIDENTIAL_COMPUTING_WORK_AREA_HEADER {
>   UINT8                   GuestType;   // 0 - legacy guest, 1 - SEV guest, 2 - tdx guest
>   UINT8                   Reserved1[3];
> } CONFIDENTIAL_COMPUTING_WORK_AREA_HEADER;
> 
> Can we define the PcdConfidentialComputingCategory like below:
>   ## This dynamic PCD indicates the Confidential Computing Category
>   #  [7:0]     Confidential Computing Category  (0 - Non-Cc, 1 - AmdSev, 2 -
> IntelTdx)
>   #  [15:8]   Sub-Category (defined by each vendor, SEV-ES, SEV-SNP, or TDX-1.0,
> TDX-2.0, etc)
>   #  [31:16] Reserved
>   # @Prompt Confidential Computing Category
> 
> gUefiCpuPkgTokenSpaceGuid.PcdConfidentialComputingCategory|0|UINT32|0x
> 60000018
> 
> Thanks!
> Min

Re: [edk2-devel] [PATCH v6 06/29] OvmfPkg/ResetVector: pre-validate the data pages used in SEC phase

[Min Xu]

Hi, Brijesh
When I go thru the code I find a potential bug in MemEncryptSevEsIsEnabled(). 

In the current code both SEV and TDX leverage the OvmfWorkArea to record the SEV/TDX information.
Byte [0] record the guest type, 0 for legacy guest. 1 for sev, 2 for tdx.
Byte [3:1] are reserved.
From Byte[4] on its meaning depends on the guest type (byte[0]).
For SEV it is SEC_SEV_ES_WORK_AREA.

In the InternalMemEncryptSevStatus(), it check the SEC_SEV_ES_WORK_AREA directly without checking if it is SEV guest. (byte[0])
  SevEsWorkArea = (SEC_SEV_ES_WORK_AREA *) FixedPcdGet32 (PcdSevEsWorkAreaBase);

I am afraid it is not correct. Would you please double check it?

Thanks!
Min

> -----Original Message-----
> From: Brijesh Singh <brijesh.singh@amd.com>
> Sent: Tuesday, September 7, 2021 9:27 PM
> To: kraxel@redhat.com; Xu, Min M <min.m.xu@intel.com>
> Cc: brijesh.singh@amd.com; devel@edk2.groups.io; James Bottomley
> <jejb@linux.ibm.com>; Yao, Jiewen <jiewen.yao@intel.com>; Tom Lendacky
> <thomas.lendacky@amd.com>; Justen, Jordan L <jordan.l.justen@intel.com>;
> Ard Biesheuvel <ardb+tianocore@kernel.org>; Erdem Aktas
> <erdemaktas@google.com>; Michael Roth <Michael.Roth@amd.com>
> Subject: Re: [edk2-devel] [PATCH v6 06/29] OvmfPkg/ResetVector: pre-validate
> the data pages used in SEC phase
> 
> 
> 
> On 9/7/21 2:07 AM, kraxel@redhat.com wrote:
> >    Hi,
> >
> >>> [ Looking at
> >>> https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fww
> >>> w.mail-
> %2F&amp;data=04%7C01%7Cbrijesh.singh%40amd.com%7Cb2dfcc7e0f93
> >>>
> 4cacdce408d971ce2f5a%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%
> 7C63
> >>>
> 7665952633333113%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiL
> CJQIjo
> >>>
> iV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=WsSaU8kFv
> w1
> >>> NWES%2BYOw7xENZr9cpPgQvBjsXWkc8nkg%3D&amp;reserved=0
> >>> archive.com/devel@edk2.groups.io/msg33605.html ]
> >>>
> >>> So, there isn't much tdx-specific in tdx-metadata.  Most ranges are
> >>> TDX_METADATA_SECTION_TYPE_TEMP_MEM which I think basically means
> >>> these ranges should be accepted by the hypervisor, which is pretty
> >>> much the same issue snp tries to solve with this pre-validation
> >>> range.  Then there are the ranges for code (aka bfv), for vars (aka cfv) and
> td_hob.
> >>>
> >>> td_hob is the only tdx-specific item there, and even that concept
> >>> (pass memory ranges as hob list from hypervisor to guest) might be
> >>> useful outside tdx.
> >> Mailbox is tdx-specific too. But
> >> Stack/Heap/OvmfWorkarea/OvmfPageTable are common. BFV/CFV are
> common too.
> >
> > Mailbox is tagged "TDX_METADATA_SECTION_TYPE_TEMP_MEM", so nothing
> > special to do when loading the firmware, right?
> >
> >>> I'd suggest we generalize the tdx-metadata idea and define both
> >>> generic and vmm-specific section types:
> >>>
> >>> enum {
> >>>    OVMF_SECTION_TYPE_UNDEFINED = 0;
> >>>
> >>>    /* generic */
> >>>    OVMF_SECTION_TYPE_CODE = 0x100,
> >>>    OVMF_SECTION_TYPE_VARS
> >>>    OVMF_SECTION_TYPE_SEC_MEM  /* vmm should accept/validate this */
> >>>
> >>>    /* sev */
> >>>    OVMF_SECTION_TYPE_SEV_SECRETS = 0x200,
> >>>    OVMF_SECTION_TYPE_SEV_CPUID /* or move to generic? */
> >>>
> >>>    /* tdx */
> >>>    OVMV_SECTION_TYPE_TDX_TD_HOB = 0x300, };
> >>>
> >>> Comments?
> >> TDX has similar section type.
> >
> > Yes.  Both TDX and SNP have simliar requirements, they want store
> > memory ranges in the firmware binary in a way that allows qemu finding
> > them and using them when initializing the guest.
> >
> > SNP stores the ranges directly in the GUID-chained block in the reset
> > vector.  The range types are implicit (first is pre-validate area,
> > second is cpuid page, ...).
> >
> > TDX stores a pointer to tdx-metadata in the GUID-chained block, then
> > the tdx-metadata has a list of ranges.  The ranges are explicitly
> > typed (section type field).
> >
> > The indirection used by TDX keeps the reset vector small.  Also the
> > explicit typing of the ranges makes it easier to extend later on if
> > needed.
> >
> > IMHO SEV should at minimum add explicit types to the memory ranges in
> > the boot block, but I'd very much prefer it if SEV and TDX can agree
> > on a way to store the memory ranges.
> >
> >> But I am not sure if SEV can use this metadata mechanism.
> >> Need SEV's comments.
> >
> > Brijesh?
> >
> 
> We should be able to make use of the metadata approach for the SEV-SNP.
> I will update the SNP patches to use the metadata approach in next rev.
> 
> thanks

Re: [PATCH v6 00/29] Add AMD Secure Nested Paging (SEV-SNP) support

[Brijesh Singh]

Thank you so much Yao for reviewing the patches. Based on some comments 
from Gerd I may update code around the reset vector area (mainly use the 
metadata format etc). For your comments regarding the introducing a new 
PcdConfidentialComputingCategory I will look to see what I can come up 
with and in UefiCpuPkg I will try to move all the SEV specific functions 
in new files (where applicable).

thanks
Brijesh

On 9/6/21 9:36 PM, Yao, Jiewen wrote:
> Thank you Brijesh
> It took me a while to review this series. Here is my feedback.
> I am not sure what you prefer, to put all comment together? Or reply 29 email separately?
> Let me put them together in this version. If you prefer a different way, please let me know.
> 
> My strategy is same as previous. I will focus on common part and review as detail as possible.
> For SEV specific thing, I will ACK and let AMD people make decision unless I have big concern on the design.
> You can add my A-B and R-B in next version.
> 
> 
> 0001-OvmfPkg-reserve-SNP-secrets-page
> Reviewed-by: Jiewen Yao <Jiewen.yao@intel.com>
> 
> 0002-OvmfPkg-reserve-CPUID-page-for-SEV-SNP
> Reviewed-by: Jiewen Yao <Jiewen.yao@intel.com>
> 
> 0003-OvmfPkg-ResetVector-introduce-SEV-SNP-boot-block-GUID
> I am still thinking if it is possible to move all SEV define GUID blob to a standalone file, and TDX define GUID blob to another file.
> Anyway, that can be done later.
> Reviewed-by: Jiewen Yao <Jiewen.yao@intel.com>
> 
> 0004-OvmfPkg-ResetVector-invalidate-the-GHCB-page
> Acked-by: Jiewen Yao <Jiewen.yao@intel.com>
> 
> 0005-OvmfPkg-ResetVector-check-the-vmpl-level
> Acked-by: Jiewen Yao <Jiewen.yao@intel.com>
> 
> 0006-OvmfPkg-ResetVector-pre-validate-the-data-pages-used-in-SEC-phase
> Acked-by: Jiewen Yao <Jiewen.yao@intel.com>
> 
> 0007-OvmfPkg-ResetVector-use-SEV-SNP-validated-CPUID-values
> Acked-by: Jiewen Yao <Jiewen.yao@intel.com>
> 
> 0008-UefiCpuPkg-Define-the-SEV-SNP-specific-dynamic-PCDs
> I really don't like the idea to use BOOL PcdSevEsIsEnabled and PcdSevSnpIsEnabled.
> Can we define *one* PCD - such as PcdConfidentialComputingCategory?
> We can assign range 0x0000~0xFFFF to AMD SEV, 0x10000~0x1FFFF to Intel TDX.
> Then SEV=0x0000, SEV-ES=0x0001, SEV-SNP=0x0002, and TDX=0x10000 later.
> I really don't want to keep adding PCD endlessly in the future, like PcdSevXXXIsEnabled, PcdSevYYYIsEnabled, PcdTdxIsEnabled, PcdTdx20Enabled, PcdTdx30Enabled, ......
> 
> 
> 0009-OvmfPkg-MemEncryptSevLib-add-MemEncryptSevSnpEnabled()
> I am not sure since we have PCD in 0008, why we need to expose the function - MemEncryptSevSnpIsEnabled() and MemEncryptSevEsIsEnabled()?
> Should we always use PCD anywhere else?
> Anyway, Acked-by: Jiewen Yao <Jiewen.yao@intel.com>
> 
> 0010-OvmfPkg-SecMain-move-SEV-specific-routines-in-AmdSev.c
> Reviewed-by: Jiewen Yao <Jiewen.yao@intel.com>
> 
> 0011-OvmfPkg-SecMain-register-GHCB-gpa-for-the-SEV-SNP-guest
> Acked-by: Jiewen Yao <Jiewen.yao@intel.com>
> 
> 0012-OvmfPkg-VmgExitLib-use-SEV-SNP-validated-CPUID-values
> Acked-by: Jiewen Yao <Jiewen.yao@intel.com>
> 
> 0013-OvmfPkg-PlatformPei-register-GHCB-gpa-for-the-SEV-SNP-guest
> Acked-by: Jiewen Yao <Jiewen.yao@intel.com>
> 
> 0014-OvmfPkg-AmdSevDxe-do-not-use-extended-PCI-config-space
> Acked-by: Jiewen Yao <Jiewen.yao@intel.com>
> 
> 0015-OvmfPkg-MemEncryptSevLib-add-support-to-validate-system-RAM
> Acked-by: Jiewen Yao <Jiewen.yao@intel.com>
> 
> 0016-OvmfPkg-BaseMemEncryptSevLib-skip-the-pre-validated-system-RAM
> Acked-by: Jiewen Yao <Jiewen.yao@intel.com>
> 
> 0017-OvmfPkg-MemEncryptSevLib-add-support-to-validate-4GB-memory-in-PEI-phase
> Acked-by: Jiewen Yao <Jiewen.yao@intel.com>
> 
> 0018-OvmfPkg-SecMain-pre-validate-the-memory-used-for-decompressing-Fv
> Acked-by: Jiewen Yao <Jiewen.yao@intel.com>
> 
> 0019-OvmfPkg-PlatformPei-validate-the-system-RAM-when-SNP-is-active
> Acked-by: Jiewen Yao <Jiewen.yao@intel.com>
> 
> 0020-OvmfPkg-PlatformPei-set-the-SEV-SNP-enabled-PCD
> See 0008
> 
> 0021-OvmfPkg-PlatformPei-set-the-Hypervisor-Features-PCD
> Acked-by: Jiewen Yao <Jiewen.yao@intel.com>
> 
> 0022-MdePkg-GHCB-increase-the-GHCB-protocol-max-version
> Acked-by: Jiewen Yao <Jiewen.yao@intel.com>
> 
> 0023-UefiCpuPkg-MpLib-add-support-to-register-GHCB-GPA-when-SEV-SNP-is-enabled
> 1) See 0008.
> 2) For MpFuncs.nasm, I recommend to move AmdSev specific initialization to a standalone file, such as Sev.nasm
> 
> 0024-UefiCpuPkg-MpInitLib-use-BSP-to-do-extended-topology-check
> See 0023
> 
> 0025-OvmfPkg-MemEncryptSevLib-change-the-page-state-in-the-RMP-table
> Acked-by: Jiewen Yao <Jiewen.yao@intel.com>
> 
> 0026-OvmfPkg-MemEncryptSevLib-skip-page-state-change-for-Mmio-address
> Acked-by: Jiewen Yao <Jiewen.yao@intel.com>
> 
> 0027-OvmfPkg-PlatformPei-mark-cpuid-and-secrets-memory-reserved-in-EFI-map
> Would you please move SEV specific init to another Sev.c?
> Also I found MemEncryptSevEsIsEnabled() and MemEncryptSevSnpIsEnabled() are there.
> I suggest just use one API
> MemEncryptSevEsIsEnabled() {
>      DoSevInitializeRamRegions()
> }
> Then you can check more in DoSevInitializeRamRegions().
> DoSevInitializeRamRegions() {
>    MemEncryptSevSnpIsEnabled() {
>    }
> }
> 
> 0028-OvmfPkg-AmdSev-expose-the-SNP-reserved-pages-through-configuration-table
> I am not convinced to include SEV specific data structure in a generic structure in ConfidentialComputingSecret.h.
> I recommend moving it to SEV specific file.
> 
> 0029-UefiCpuPkg-MpInitLib-Use-SEV-SNP-AP-Creation-NAE-event-to-launch-APs
> See 0008, 0023.
> I recommend to move SevSnpCreateSaveArea() to Sev.c.
> 
> Thank you
> Yao Jiewen
> 
> 
> 
>> -----Original Message-----
>> From: Brijesh Singh <brijesh.singh@amd.com>
>> Sent: Thursday, September 2, 2021 12:16 AM
>> To: devel@edk2.groups.io
>> Cc: James Bottomley <jejb@linux.ibm.com>; Xu, Min M <min.m.xu@intel.com>;
>> Yao, Jiewen <jiewen.yao@intel.com>; Tom Lendacky
>> <thomas.lendacky@amd.com>; Justen, Jordan L <jordan.l.justen@intel.com>;
>> Ard Biesheuvel <ardb+tianocore@kernel.org>; Erdem Aktas
>> <erdemaktas@google.com>; Michael Roth <Michael.Roth@amd.com>; Gerd
>> Hoffmann <kraxel@redhat.com>; Brijesh Singh <brijesh.singh@amd.com>
>> Subject: [PATCH v6 00/29] Add AMD Secure Nested Paging (SEV-SNP) support
>>
>> BZ: https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fbugzilla.tianocore.org%2Fshow_bug.cgi%3Fid%3D3275&amp;data=04%7C01%7Cbrijesh.singh%40amd.com%7C33df27781053475362e208d971a85cee%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637665791405981353%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=8vfBxVawRoEeDCR0DHJhfhTgPr66704twMGZ8%2BY%2BLGI%3D&amp;reserved=0
>>
>> SEV-SNP builds upon existing SEV and SEV-ES functionality while adding
>> new hardware-based memory protections. SEV-SNP adds strong memory
>> integrity
>> protection to help prevent malicious hypervisor-based attacks like data
>> replay, memory re-mapping and more in order to create an isolated memory
>> encryption environment.
>>
>> This series provides the basic building blocks to support booting the SEV-SNP
>> VMs, it does not cover all the security enhancement introduced by the SEV-SNP
>> such as interrupt protection.
>>
>> Many of the integrity guarantees of SEV-SNP are enforced through a new
>> structure called the Reverse Map Table (RMP). Adding a new page to SEV-SNP
>> VM requires a 2-step process. First, the hypervisor assigns a page to the
>> guest using the new RMPUPDATE instruction. This transitions the page to
>> guest-invalid. Second, the guest validates the page using the new PVALIDATE
>> instruction. The SEV-SNP VMs can use the new "Page State Change Request
>> NAE"
>> defined in the GHCB specification to ask hypervisor to add or remove page
>> from the RMP table.
>>
>> Each page assigned to the SEV-SNP VM can either be validated or unvalidated,
>> as indicated by the Validated flag in the page's RMP entry. There are two
>> approaches that can be taken for the page validation: Pre-validation and
>> Lazy Validation.
>>
>> Under pre-validation, the pages are validated prior to first use. And under
>> lazy validation, pages are validated when first accessed. An access to a
>> unvalidated page results in a #VC exception, at which time the exception
>> handler may validate the page. Lazy validation requires careful tracking of
>> the validated pages to avoid validating the same GPA more than once. The
>> recently introduced "Unaccepted" memory type can be used to communicate
>> the
>> unvalidated memory ranges to the Guest OS.
>>
>> At this time we only support the pre-validation. OVMF detects all the available
>> system RAM in the PEI phase. When SEV-SNP is enabled, the memory is validated
>> before it is made available to the EDK2 core.
>>
>> Now that series contains all the basic support required to launch SEV-SNP
>> guest. We are still missing the Interrupt security feature provided by the
>> SNP. The feature will be added after the base support is accepted.
>>
>> Additional resources
>> ---------------------
>> SEV-SNP whitepaper
>> https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.amd.com%2Fsystem%2Ffiles%2FTechDocs%2FSEV-SNP-strengthening-vm-&amp;data=04%7C01%7Cbrijesh.singh%40amd.com%7C33df27781053475362e208d971a85cee%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637665791405981353%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=tobk2zHk1ziA6nZ9bvwNrohRuIN7bTEh5ZXFNzwTTX0%3D&amp;reserved=0
>> isolation-with-integrity-protection-and-more.pdf
>>
>> APM 2: https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.amd.com%2Fsystem%2Ffiles%2FTechDocs%2F24593.pdf&amp;data=04%7C01%7Cbrijesh.singh%40amd.com%7C33df27781053475362e208d971a85cee%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637665791405981353%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=%2FDTzbh8F6CtvvC263r7xJGX6WAQ8yCAuKLkPM7GwBvQ%3D&amp;reserved=0 (section 15.36)
>>
>> The complete source is available at
>> https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FAMDESE%2Fovmf%2Ftree%2Fsev-snp-rfc-5&amp;data=04%7C01%7Cbrijesh.singh%40amd.com%7C33df27781053475362e208d971a85cee%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637665791405981353%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=6lvmuqOQbvNoXG50qK5QGYG6XEdojJ%2BHlkKrODZRAHY%3D&amp;reserved=0
>>
>> GHCB spec:
>> https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdeveloper.amd.com%2Fwp-content%2Fresources%2F56421.pdf&amp;data=04%7C01%7Cbrijesh.singh%40amd.com%7C33df27781053475362e208d971a85cee%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637665791405981353%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=Q1oa5gB3CthKPkentzFJE3B3LfBpZq%2B4y8EzPTlPzl8%3D&amp;reserved=0
>>
>> SEV-SNP firmware specification:
>> https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.amd.com%2Fsystem%2Ffiles%2FTechDocs%2F56860.pdf&amp;data=04%7C01%7Cbrijesh.singh%40amd.com%7C33df27781053475362e208d971a85cee%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637665791405981353%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=7cLoMR52WAvMe%2Fr4rKGGYx2wadopvXKnSGi%2FghyEdJA%3D&amp;reserved=0
>>
>> Change since v5:
>>   * When possible use the CPUID value from CPUID page
>>   * Move the SEV specific functions from SecMain.c in AmdSev.c
>>   * Rebase to the latest code
>>   * Add the review feedback from Yao.
>>
>> Change since v4:
>>   * Use the correct MSR for the SEV_STATUS
>>   * Add VMPL-0 check
>>
>> Change since v3:
>>   * ResetVector: move all SEV specific code in AmdSev.asm and add macros to
>> keep
>>     the code readable.
>>   * Drop extending the EsWorkArea to contain SNP specific state.
>>   * Drop the GhcbGpa library and call the VmgExit directly to register GHCB GPA.
>>   * Install the CC blob config table from AmdSevDxe instead of extending the
>>     AmdSev/SecretsDxe for it.
>>   * Add the separate PCDs for the SNP Secrets.
>>
>> Changes since v2:
>>   * Add support for the AP creation.
>>   * Use the module-scoping override to make AmdSevDxe use the IO port for PCI
>> reads.
>>   * Use the reserved memory type for CPUID and Secrets page.
>>   *
>> Changes since v1:
>>   * Drop the interval tree support to detect the pre-validated overlap region.
>>   * Use an array to keep track of pre-validated regions.
>>   * Add support to query the Hypervisor feature and verify that SNP feature is
>> supported.
>>   * Introduce MemEncryptSevClearMmioPageEncMask() to clear the C-bit from
>> MMIO ranges.
>>   * Pull the SevSecretDxe and SevSecretPei into OVMF package build.
>>   * Extend the SevSecretDxe to expose confidential computing blob location
>> through
>>     EFI configuration table.
>>
>> Brijesh Singh (25):
>>    OvmfPkg: reserve SNP secrets page
>>    OvmfPkg: reserve CPUID page for SEV-SNP
>>    OvmfPkg/ResetVector: introduce SEV-SNP boot block GUID
>>    OvmfPkg/ResetVector: invalidate the GHCB page
>>    OvmfPkg/ResetVector: check the vmpl level
>>    OvmfPkg/ResetVector: pre-validate the data pages used in SEC phase
>>    UefiCpuPkg: Define the SEV-SNP specific dynamic PCDs
>>    OvmfPkg/MemEncryptSevLib: add MemEncryptSevSnpEnabled()
>>    OvmfPkg/SecMain: move SEV specific routines in AmdSev.c
>>    OvmfPkg/SecMain: register GHCB gpa for the SEV-SNP guest
>>    OvmfPkg/PlatformPei: register GHCB gpa for the SEV-SNP guest
>>    OvmfPkg/AmdSevDxe: do not use extended PCI config space
>>    OvmfPkg/MemEncryptSevLib: add support to validate system RAM
>>    OvmfPkg/BaseMemEncryptSevLib: skip the pre-validated system RAM
>>    OvmfPkg/MemEncryptSevLib: add support to validate > 4GB memory in PEI
>>      phase
>>    OvmfPkg/SecMain: pre-validate the memory used for decompressing Fv
>>    OvmfPkg/PlatformPei: validate the system RAM when SNP is active
>>    OvmfPkg/PlatformPei: set the SEV-SNP enabled PCD
>>    OvmfPkg/PlatformPei: set the Hypervisor Features PCD
>>    MdePkg/GHCB: increase the GHCB protocol max version
>>    UefiCpuPkg/MpLib: add support to register GHCB GPA when SEV-SNP is
>>      enabled
>>    OvmfPkg/MemEncryptSevLib: change the page state in the RMP table
>>    OvmfPkg/MemEncryptSevLib: skip page state change for Mmio address
>>    OvmfPkg/PlatformPei: mark cpuid and secrets memory reserved in EFI map
>>    OvmfPkg/AmdSev: expose the SNP reserved pages through configuration
>>      table
>>
>> Michael Roth (3):
>>    OvmfPkg/ResetVector: use SEV-SNP-validated CPUID values
>>    OvmfPkg/VmgExitLib: use SEV-SNP-validated CPUID values
>>    UefiCpuPkg/MpInitLib: use BSP to do extended topology check
>>
>> Tom Lendacky (1):
>>    UefiCpuPkg/MpInitLib: Use SEV-SNP AP Creation NAE event to launch APs
>>
>>   OvmfPkg/OvmfPkg.dec                           |  23 +
>>   UefiCpuPkg/UefiCpuPkg.dec                     |  11 +
>>   OvmfPkg/AmdSev/AmdSevX64.dsc                  |   5 +-
>>   OvmfPkg/Bhyve/BhyveX64.dsc                    |   5 +-
>>   OvmfPkg/OvmfPkgIa32.dsc                       |   1 +
>>   OvmfPkg/OvmfPkgIa32X64.dsc                    |   6 +-
>>   OvmfPkg/OvmfPkgX64.dsc                        |   5 +-
>>   OvmfPkg/OvmfXen.dsc                           |   5 +-
>>   OvmfPkg/OvmfPkgX64.fdf                        |  12 +-
>>   OvmfPkg/AmdSevDxe/AmdSevDxe.inf               |   7 +
>>   .../DxeMemEncryptSevLib.inf                   |   3 +
>>   .../PeiMemEncryptSevLib.inf                   |   7 +
>>   .../SecMemEncryptSevLib.inf                   |   3 +
>>   OvmfPkg/Library/VmgExitLib/SecVmgExitLib.inf  |   2 +
>>   OvmfPkg/Library/VmgExitLib/VmgExitLib.inf     |   3 +
>>   OvmfPkg/PlatformPei/PlatformPei.inf           |  10 +
>>   OvmfPkg/ResetVector/ResetVector.inf           |   6 +
>>   OvmfPkg/Sec/SecMain.inf                       |   4 +
>>   UefiCpuPkg/Library/MpInitLib/DxeMpInitLib.inf |   4 +
>>   UefiCpuPkg/Library/MpInitLib/PeiMpInitLib.inf |   4 +
>>   MdePkg/Include/Register/Amd/Ghcb.h            |   2 +-
>>   .../Guid/ConfidentialComputingSecret.h        |  18 +
>>   OvmfPkg/Include/Library/MemEncryptSevLib.h    |  26 +
>>   .../X64/SnpPageStateChange.h                  |  31 ++
>>   .../BaseMemEncryptSevLib/X64/VirtualMemory.h  |  19 +
>>   OvmfPkg/Sec/AmdSev.h                          |  95 ++++
>>   UefiCpuPkg/Library/MpInitLib/MpLib.h          |  20 +
>>   OvmfPkg/AmdSevDxe/AmdSevDxe.c                 |  23 +
>>   .../DxeMemEncryptSevLibInternal.c             |  27 ++
>>   .../Ia32/MemEncryptSevLib.c                   |  17 +
>>   .../PeiMemEncryptSevLibInternal.c             |  27 ++
>>   .../SecMemEncryptSevLibInternal.c             |  19 +
>>   .../X64/DxeSnpSystemRamValidate.c             |  40 ++
>>   .../X64/PeiDxeVirtualMemory.c                 | 167 ++++++-
>>   .../X64/PeiSnpSystemRamValidate.c             | 126 +++++
>>   .../X64/SecSnpSystemRamValidate.c             |  36 ++
>>   .../X64/SnpPageStateChangeInternal.c          | 295 ++++++++++++
>>   OvmfPkg/Library/VmgExitLib/VmgExitVcHandler.c | 444 ++++++++++++++++--
>>   OvmfPkg/PlatformPei/AmdSev.c                  | 192 ++++++++
>>   OvmfPkg/PlatformPei/MemDetect.c               |  21 +
>>   OvmfPkg/Sec/AmdSev.c                          | 267 +++++++++++
>>   OvmfPkg/Sec/SecMain.c                         | 160 +------
>>   UefiCpuPkg/Library/MpInitLib/DxeMpLib.c       |  11 +-
>>   .../MpInitLib/Ia32/SevSnpRmpAdjustInternal.c  |  31 ++
>>   UefiCpuPkg/Library/MpInitLib/MpLib.c          | 286 ++++++++++-
>>   .../MpInitLib/X64/SevSnpRmpAdjustInternal.c   |  44 ++
>>   OvmfPkg/FvmainCompactScratchEnd.fdf.inc       |   5 +
>>   OvmfPkg/ResetVector/Ia16/ResetVectorVtf0.asm  |  28 ++
>>   OvmfPkg/ResetVector/Ia32/AmdSev.asm           | 307 +++++++++++-
>>   OvmfPkg/ResetVector/ResetVector.nasmb         |   6 +
>>   UefiCpuPkg/Library/MpInitLib/MpEqu.inc        |   2 +
>>   UefiCpuPkg/Library/MpInitLib/X64/MpFuncs.nasm |  78 +++
>>   52 files changed, 2771 insertions(+), 225 deletions(-)
>>   create mode 100644
>> OvmfPkg/Library/BaseMemEncryptSevLib/X64/SnpPageStateChange.h
>>   create mode 100644 OvmfPkg/Sec/AmdSev.h
>>   create mode 100644
>> OvmfPkg/Library/BaseMemEncryptSevLib/X64/DxeSnpSystemRamValidate.c
>>   create mode 100644
>> OvmfPkg/Library/BaseMemEncryptSevLib/X64/PeiSnpSystemRamValidate.c
>>   create mode 100644
>> OvmfPkg/Library/BaseMemEncryptSevLib/X64/SecSnpSystemRamValidate.c
>>   create mode 100644
>> OvmfPkg/Library/BaseMemEncryptSevLib/X64/SnpPageStateChangeInternal.c
>>   create mode 100644 OvmfPkg/Sec/AmdSev.c
>>   create mode 100644
>> UefiCpuPkg/Library/MpInitLib/Ia32/SevSnpRmpAdjustInternal.c
>>   create mode 100644
>> UefiCpuPkg/Library/MpInitLib/X64/SevSnpRmpAdjustInternal.c
>>
>> --
>> 2.17.1
> 

Re: [PATCH v6 00/29] Add AMD Secure Nested Paging (SEV-SNP) support

[Min Xu]

On September 9, 2021 3:46 AM, Brijesh Singh wrote:
> 
> Thank you so much Yao for reviewing the patches. Based on some comments
> from Gerd I may update code around the reset vector area (mainly use the
> metadata format etc). For your comments regarding the introducing a new
> PcdConfidentialComputingCategory I will look to see what I can come up with
> and in UefiCpuPkg I will try to move all the SEV specific functions in new files
> (where applicable).
> 
Hi, Brijesh
if you are considering to introduce a new PcdConfidentialComputingCategory
as Jiewen suggested below:
> >
> > 0008-UefiCpuPkg-Define-the-SEV-SNP-specific-dynamic-PCDs
> > I really don't like the idea to use BOOL PcdSevEsIsEnabled and
> PcdSevSnpIsEnabled.
> > Can we define *one* PCD - such as PcdConfidentialComputingCategory?
> > We can assign range 0x0000~0xFFFF to AMD SEV, 0x10000~0x1FFFF to Intel
> TDX.
> > Then SEV=0x0000, SEV-ES=0x0001, SEV-SNP=0x0002, and TDX=0x10000
> later.
> > I really don't want to keep adding PCD endlessly in the future, like
> PcdSevXXXIsEnabled, PcdSevYYYIsEnabled, PcdTdxIsEnabled,
> PcdTdx20Enabled, PcdTdx30Enabled, ......
> >
I also have some suggestions.

As we have below definition in WorkArea.h
  typedef struct _CONFIDENTIAL_COMPUTING_WORK_AREA_HEADER {
    UINT8                   GuestType;
    UINT8                   Reserved1[3];
  } CONFIDENTIAL_COMPUTING_WORK_AREA_HEADER;

Can we update above CONFIDENTIAL_COMPUTING_WORK_AREA_HEADER to below:
  typedef struct _CONFIDENTIAL_COMPUTING_WORK_AREA_HEADER {
    UINT8                   GuestType;
    UINT8                   SubType;               // subtype which indicates SEV-ES, SEV-NP, or TDX 1.0, TDX 2.0 etc.
    UINT8                   Reserved1[2];
  } CONFIDENTIAL_COMPUTING_WORK_AREA_HEADER;

The PcdConfidentialComputingCategory can be defined as UINT32, like below:
  ## This dynamic PCD indicates the Confidential Computing Category
  #  [7:0]   Confidential Computing Category  (0 - Non-Cc, 1 - AmdSev, 2 - IntelTdx)
  #  [15:8]  Sub-Category (defined by each vendor, SEV-ES, SEV-SNP, or TDX-1.0, TDX-2.0, etc)
  #  [31:16] Reserved
  # @Prompt Confidential Computing Category
  gUefiCpuPkgTokenSpaceGuid.PcdConfidentialComputingCategory|0|UINT32|0x60000018

So that we simply copy the CONFIDENTIAL_COMPUTING_WORK_AREA_HEADER to PcdConfidentialComputingCategory.
What's your thought?

Thanks!
Min

Re: [PATCH v6 00/29] Add AMD Secure Nested Paging (SEV-SNP) support

[Brijesh Singh]

Hi Min,

On 9/8/21 7:31 PM, Xu, Min M wrote:
> On September 9, 2021 3:46 AM, Brijesh Singh wrote:
>> Thank you so much Yao for reviewing the patches. Based on some comments
>> from Gerd I may update code around the reset vector area (mainly use the
>> metadata format etc). For your comments regarding the introducing a new
>> PcdConfidentialComputingCategory I will look to see what I can come up with
>> and in UefiCpuPkg I will try to move all the SEV specific functions in new files
>> (where applicable).
>>
> Hi, Brijesh
> if you are considering to introduce a new PcdConfidentialComputingCategory
> as Jiewen suggested below:
>>> 0008-UefiCpuPkg-Define-the-SEV-SNP-specific-dynamic-PCDs
>>> I really don't like the idea to use BOOL PcdSevEsIsEnabled and
>> PcdSevSnpIsEnabled.
>>> Can we define *one* PCD - such as PcdConfidentialComputingCategory?
>>> We can assign range 0x0000~0xFFFF to AMD SEV, 0x10000~0x1FFFF to Intel
>> TDX.
>>> Then SEV=0x0000, SEV-ES=0x0001, SEV-SNP=0x0002, and TDX=0x10000
>> later.
>>> I really don't want to keep adding PCD endlessly in the future, like
>> PcdSevXXXIsEnabled, PcdSevYYYIsEnabled, PcdTdxIsEnabled,
>> PcdTdx20Enabled, PcdTdx30Enabled, ......
> I also have some suggestions.
>
> As we have below definition in WorkArea.h
>   typedef struct _CONFIDENTIAL_COMPUTING_WORK_AREA_HEADER {
>     UINT8                   GuestType;
>     UINT8                   Reserved1[3];
>   } CONFIDENTIAL_COMPUTING_WORK_AREA_HEADER;
>
> Can we update above CONFIDENTIAL_COMPUTING_WORK_AREA_HEADER to below:
>   typedef struct _CONFIDENTIAL_COMPUTING_WORK_AREA_HEADER {
>     UINT8                   GuestType;
>     UINT8                   SubType;               // subtype which indicates SEV-ES, SEV-NP, or TDX 1.0, TDX 2.0 etc.
>     UINT8                   Reserved1[2];
>   } CONFIDENTIAL_COMPUTING_WORK_AREA_HEADER;
>
> The PcdConfidentialComputingCategory can be defined as UINT32, like below:
>   ## This dynamic PCD indicates the Confidential Computing Category
>   #  [7:0]   Confidential Computing Category  (0 - Non-Cc, 1 - AmdSev, 2 - IntelTdx)
>   #  [15:8]  Sub-Category (defined by each vendor, SEV-ES, SEV-SNP, or TDX-1.0, TDX-2.0, etc)
>   #  [31:16] Reserved
>   # @Prompt Confidential Computing Category
>   gUefiCpuPkgTokenSpaceGuid.PcdConfidentialComputingCategory|0|UINT32|0x60000018
>
> So that we simply copy the CONFIDENTIAL_COMPUTING_WORK_AREA_HEADER to PcdConfidentialComputingCategory.
> What's your thought?

I am not sure if its a good idea to pack a header like above in a 32-bit
PCD. The caller need to unpack the 32-bit number and perform a bitshit
etc. Additionally we also need to check for reserved bits being set to
zero etc. I am more inclined toward something like this:

enum {

   /* The guest is running with memory encryption disabled. */

    CC_ATTR_UNDEF,

  /* The guest is running with active AMD SEV memory encryption. */

   CC_ATTR_AMD_SEV,

  /* The guest is running with active AMD SEV-ES memory encryption. */

  CC_ATTR_AMD_SEV_ES,

  /* The guest is running with active AMD SEV-SNP memory encryption. */

  CC_ATTR_AMD_SEV_SNP,

 /* The guest is running with active Intel TDX memory encryption. */

 CC_ATTR_INTEL_TDX,

 /* The guest is running with active Intel SGX memory encryption. */

 CC_ATTR_INTEL_SGX,

} ConfidentialComputingAttr;

The PcdConfidentialComputingAttr will be set to zero. The OVMF will set
this dynamic PCD in PEI phase. The UefiCpuPkg will provide a new
function that can be used by anyone to check the CC guest type.

BOOLEAN

CcPlatformHas(enum ConfidentialComputingAttr attr)

{

    UINT32 Val;

    Val = PcdGet32 (PcdConfidentialComputingAttr);

    return val == attr;

}


> Thanks!
> Min

Re: [PATCH v6 00/29] Add AMD Secure Nested Paging (SEV-SNP) support

[Gerd Hoffmann]

  Hi,

> I am not sure if its a good idea to pack a header like above in a 32-bit
> PCD. The caller need to unpack the 32-bit number and perform a bitshit
> etc. Additionally we also need to check for reserved bits being set to
> zero etc. I am more inclined toward something like this:
> 
> enum {

Well, various places probably just need to know whenever they should
call into the sev or the tdx library, so grouping stuff makes sense to
me.  We don't need bitfields for that though, could also be done this
way:

enum {
   NOT_ENCRYPTED = 0,
   AMD_SEV = 0x100,
   AMD_SEV_ES,
   [ ... ]
   INTEL_TDX = 0x200,
   [ ... ]
}

So if you need the exact mode you can compare values as-is, if you want
figure which vendor library should be called you'll just mask out the
least significant 8 bits.

take care,
  Gerd

Re: [PATCH v6 00/29] Add AMD Secure Nested Paging (SEV-SNP) support

[Brijesh Singh]

On 9/9/21 6:22 AM, Gerd Hoffmann wrote:
>   Hi,
>
>> I am not sure if its a good idea to pack a header like above in a 32-bit
>> PCD. The caller need to unpack the 32-bit number and perform a bitshit
>> etc. Additionally we also need to check for reserved bits being set to
>> zero etc. I am more inclined toward something like this:
>>
>> enum {
> Well, various places probably just need to know whenever they should
> call into the sev or the tdx library, so grouping stuff makes sense to
> me.  We don't need bitfields for that though, could also be done this
> way:
>
> enum {
>    NOT_ENCRYPTED = 0,
>    AMD_SEV = 0x100,
>    AMD_SEV_ES,
>    [ ... ]
>    INTEL_TDX = 0x200,
>    [ ... ]
> }
>
> So if you need the exact mode you can compare values as-is, if you want
> figure which vendor library should be called you'll just mask out the
> least significant 8 bits.

Yes, this also works fine.

thanks

Re: [edk2-devel] [PATCH v6 00/29] Add AMD Secure Nested Paging (SEV-SNP) support

[Min Xu]

On September 9, 2021 7:40 PM, Brijesh Singh wrote:
> On 9/9/21 6:22 AM, Gerd Hoffmann wrote:
> >   Hi,
> >
> >> I am not sure if its a good idea to pack a header like above in a
> >> 32-bit PCD. The caller need to unpack the 32-bit number and perform a
> >> bitshit etc. Additionally we also need to check for reserved bits
> >> being set to zero etc. I am more inclined toward something like this:
> >>
> >> enum {
> > Well, various places probably just need to know whenever they should
> > call into the sev or the tdx library, so grouping stuff makes sense to
> > me.  We don't need bitfields for that though, could also be done this
> > way:
> >
> > enum {
> >    NOT_ENCRYPTED = 0,
> >    AMD_SEV = 0x100,
> >    AMD_SEV_ES,
> >    [ ... ]
> >    INTEL_TDX = 0x200,
> >    [ ... ]
> > }
> >
> > So if you need the exact mode you can compare values as-is, if you
> > want figure which vendor library should be called you'll just mask out
> > the least significant 8 bits.
> 
> Yes, this also works fine.
Agree. So let's follow this way.

Thanks!
Min

Re: [edk2-devel] [PATCH v6 00/29] Add AMD Secure Nested Paging (SEV-SNP) support

[Yao, Jiewen]

I do not see any conflict here.

You can use below definition
 enum {
   CC_ATTR_AMD_SEV = 0x0001,
   CC_ATTR_AMD_SEV_ES =  0x0101,
   CC_ATTR_AMD_SEV_SNP =  0x0201,
   CC_ATTR_INTEL_TDX =  0x0002,
 } ConfidentialComputingAttr;

BTW: Please remove SGX, we don’t need it here.

Thank you
Yao Jiewen

> -----Original Message-----
> From: devel@edk2.groups.io <devel@edk2.groups.io> On Behalf Of Brijesh
> Singh via groups.io
> Sent: Thursday, September 9, 2021 6:51 PM
> To: Xu, Min M <min.m.xu@intel.com>; Yao, Jiewen <jiewen.yao@intel.com>;
> devel@edk2.groups.io
> Cc: James Bottomley <jejb@linux.ibm.com>; Tom Lendacky
> <thomas.lendacky@amd.com>; Justen, Jordan L <jordan.l.justen@intel.com>;
> Ard Biesheuvel <ardb+tianocore@kernel.org>; Erdem Aktas
> <erdemaktas@google.com>; Michael Roth <Michael.Roth@amd.com>; Gerd
> Hoffmann <kraxel@redhat.com>
> Subject: Re: [edk2-devel] [PATCH v6 00/29] Add AMD Secure Nested Paging
> (SEV-SNP) support
> 
> Hi Min,
> 
> On 9/8/21 7:31 PM, Xu, Min M wrote:
> > On September 9, 2021 3:46 AM, Brijesh Singh wrote:
> >> Thank you so much Yao for reviewing the patches. Based on some comments
> >> from Gerd I may update code around the reset vector area (mainly use the
> >> metadata format etc). For your comments regarding the introducing a new
> >> PcdConfidentialComputingCategory I will look to see what I can come up
> with
> >> and in UefiCpuPkg I will try to move all the SEV specific functions in new files
> >> (where applicable).
> >>
> > Hi, Brijesh
> > if you are considering to introduce a new PcdConfidentialComputingCategory
> > as Jiewen suggested below:
> >>> 0008-UefiCpuPkg-Define-the-SEV-SNP-specific-dynamic-PCDs
> >>> I really don't like the idea to use BOOL PcdSevEsIsEnabled and
> >> PcdSevSnpIsEnabled.
> >>> Can we define *one* PCD - such as PcdConfidentialComputingCategory?
> >>> We can assign range 0x0000~0xFFFF to AMD SEV, 0x10000~0x1FFFF to Intel
> >> TDX.
> >>> Then SEV=0x0000, SEV-ES=0x0001, SEV-SNP=0x0002, and TDX=0x10000
> >> later.
> >>> I really don't want to keep adding PCD endlessly in the future, like
> >> PcdSevXXXIsEnabled, PcdSevYYYIsEnabled, PcdTdxIsEnabled,
> >> PcdTdx20Enabled, PcdTdx30Enabled, ......
> > I also have some suggestions.
> >
> > As we have below definition in WorkArea.h
> >   typedef struct _CONFIDENTIAL_COMPUTING_WORK_AREA_HEADER {
> >     UINT8                   GuestType;
> >     UINT8                   Reserved1[3];
> >   } CONFIDENTIAL_COMPUTING_WORK_AREA_HEADER;
> >
> > Can we update above CONFIDENTIAL_COMPUTING_WORK_AREA_HEADER to
> below:
> >   typedef struct _CONFIDENTIAL_COMPUTING_WORK_AREA_HEADER {
> >     UINT8                   GuestType;
> >     UINT8                   SubType;               // subtype which indicates SEV-ES, SEV-NP,
> or TDX 1.0, TDX 2.0 etc.
> >     UINT8                   Reserved1[2];
> >   } CONFIDENTIAL_COMPUTING_WORK_AREA_HEADER;
> >
> > The PcdConfidentialComputingCategory can be defined as UINT32, like below:
> >   ## This dynamic PCD indicates the Confidential Computing Category
> >   #  [7:0]   Confidential Computing Category  (0 - Non-Cc, 1 - AmdSev, 2 -
> IntelTdx)
> >   #  [15:8]  Sub-Category (defined by each vendor, SEV-ES, SEV-SNP, or TDX-1.0,
> TDX-2.0, etc)
> >   #  [31:16] Reserved
> >   # @Prompt Confidential Computing Category
> >
> gUefiCpuPkgTokenSpaceGuid.PcdConfidentialComputingCategory|0|UINT32|0x
> 60000018
> >
> > So that we simply copy the
> CONFIDENTIAL_COMPUTING_WORK_AREA_HEADER to
> PcdConfidentialComputingCategory.
> > What's your thought?
> 
> I am not sure if its a good idea to pack a header like above in a 32-bit
> PCD. The caller need to unpack the 32-bit number and perform a bitshit
> etc. Additionally we also need to check for reserved bits being set to
> zero etc. I am more inclined toward something like this:
> 
> enum {
> 
>    /* The guest is running with memory encryption disabled. */
> 
>     CC_ATTR_UNDEF,
> 
>   /* The guest is running with active AMD SEV memory encryption. */
> 
>    CC_ATTR_AMD_SEV,
> 
>   /* The guest is running with active AMD SEV-ES memory encryption. */
> 
>   CC_ATTR_AMD_SEV_ES,
> 
>   /* The guest is running with active AMD SEV-SNP memory encryption. */
> 
>   CC_ATTR_AMD_SEV_SNP,
> 
>  /* The guest is running with active Intel TDX memory encryption. */
> 
>  CC_ATTR_INTEL_TDX,
> 
>  /* The guest is running with active Intel SGX memory encryption. */
> 
>  CC_ATTR_INTEL_SGX,
> 
> } ConfidentialComputingAttr;
> 
> The PcdConfidentialComputingAttr will be set to zero. The OVMF will set
> this dynamic PCD in PEI phase. The UefiCpuPkg will provide a new
> function that can be used by anyone to check the CC guest type.
> 
> BOOLEAN
> 
> CcPlatformHas(enum ConfidentialComputingAttr attr)
> 
> {
> 
>     UINT32 Val;
> 
>     Val = PcdGet32 (PcdConfidentialComputingAttr);
> 
>     return val == attr;
> 
> }
> 
> 
> > Thanks!
> > Min
> 
> 
> 
> 

Re: [PATCH v6 00/29] Add AMD Secure Nested Paging (SEV-SNP) support

[Brijesh Singh]

Hi Yao,

I am going through implementing your feedback. I have covered most of
it. But your comment on moving some of the changes from MpFunc.nasm to
Sev.nasm may make code harder to read. It is mainly because the GPA
registration and Topo check are not self-contained routines. They depend
on some previous register states and need to jump to different paths
based on the condition. All those jump are in generic AP init routines.

As you can see, the changes in MpFunc.nasm are more petite, and if it's
a big issue, we can revisit it later and devise a proposal to move the
big chunk of generic code in a separate file then have TDX and SEV call
into it. At that time, we can add OneTimeCall or OneTimeReturn macros
etc., to jump between different files to make it more readable. Would
you please let me know if that is acceptable?

-Brijesh

On 9/6/21 9:36 PM, Yao, Jiewen wrote:
>
> 0023-UefiCpuPkg-MpLib-add-support-to-register-GHCB-GPA-when-SEV-SNP-is-enabled
> 1) See 0008.
> 2) For MpFuncs.nasm, I recommend to move AmdSev specific initialization to a standalone file, such as Sev.nasm
>
> 0024-UefiCpuPkg-MpInitLib-use-BSP-to-do-extended-topology-check
> See 0023
>

Re: [PATCH v6 00/29] Add AMD Secure Nested Paging (SEV-SNP) support

[Yao, Jiewen]

Hi Brijesh
I think it is OK to leave MpFunc.nasm in this series.
We can revisit later.

Thank you
Yao Jiewen

> -----Original Message-----
> From: Brijesh Singh <brijesh.singh@amd.com>
> Sent: Monday, September 13, 2021 6:56 AM
> To: Yao, Jiewen <jiewen.yao@intel.com>; devel@edk2.groups.io
> Cc: James Bottomley <jejb@linux.ibm.com>; Xu, Min M <min.m.xu@intel.com>;
> Tom Lendacky <thomas.lendacky@amd.com>; Justen, Jordan L
> <jordan.l.justen@intel.com>; Ard Biesheuvel <ardb+tianocore@kernel.org>;
> Erdem Aktas <erdemaktas@google.com>; Michael Roth
> <Michael.Roth@amd.com>; Gerd Hoffmann <kraxel@redhat.com>
> Subject: Re: [PATCH v6 00/29] Add AMD Secure Nested Paging (SEV-SNP)
> support
> 
> Hi Yao,
> 
> I am going through implementing your feedback. I have covered most of
> it. But your comment on moving some of the changes from MpFunc.nasm to
> Sev.nasm may make code harder to read. It is mainly because the GPA
> registration and Topo check are not self-contained routines. They depend
> on some previous register states and need to jump to different paths
> based on the condition. All those jump are in generic AP init routines.
> 
> As you can see, the changes in MpFunc.nasm are more petite, and if it's
> a big issue, we can revisit it later and devise a proposal to move the
> big chunk of generic code in a separate file then have TDX and SEV call
> into it. At that time, we can add OneTimeCall or OneTimeReturn macros
> etc., to jump between different files to make it more readable. Would
> you please let me know if that is acceptable?
> 
> -Brijesh
> 
> On 9/6/21 9:36 PM, Yao, Jiewen wrote:
> >
> > 0023-UefiCpuPkg-MpLib-add-support-to-register-GHCB-GPA-when-SEV-SNP-
> is-enabled
> > 1) See 0008.
> > 2) For MpFuncs.nasm, I recommend to move AmdSev specific initialization to a
> standalone file, such as Sev.nasm
> >
> > 0024-UefiCpuPkg-MpInitLib-use-BSP-to-do-extended-topology-check
> > See 0023
> >

Re: [edk2-devel] [PATCH v6 06/29] OvmfPkg/ResetVector: pre-validate the data pages used in SEC phase

[Yao, Jiewen]

I can explain why we prefer DQ instead of DD.

You are right that current TD entrypoint is 32bit. However, we cannot predict that is always TRUE for the future.

Back to 16 bit machine, we have entrypoint at F000:FFF0. But in 32bit mode we removed 1M limitation and move entrypoint to below 4G.
There is no limitation that we move 64bit entrypoint above 4G for 64bit machine.
We still choose 4G for easy implementation and compatibility consideration.

And we still leave room for extension in the future. For example, the firmware information table (FIT) defines 64bit address, although only 32bit is used today. (https://software.intel.com/content/dam/develop/external/us/en/documents/firmware-interface-table-bios-specification-r1p2p1.pdf). 

Thank you
Yao Jiewen

> -----Original Message-----
> From: devel@edk2.groups.io <devel@edk2.groups.io> On Behalf Of Gerd
> Hoffmann
> Sent: Tuesday, September 7, 2021 3:08 PM
> To: Xu, Min M <min.m.xu@intel.com>
> Cc: devel@edk2.groups.io; Brijesh Singh <brijesh.singh@amd.com>; James
> Bottomley <jejb@linux.ibm.com>; Yao, Jiewen <jiewen.yao@intel.com>; Tom
> Lendacky <thomas.lendacky@amd.com>; Justen, Jordan L
> <jordan.l.justen@intel.com>; Ard Biesheuvel <ardb+tianocore@kernel.org>;
> Erdem Aktas <erdemaktas@google.com>; Michael Roth
> <Michael.Roth@amd.com>
> Subject: Re: [edk2-devel] [PATCH v6 06/29] OvmfPkg/ResetVector: pre-validate
> the data pages used in SEC phase
> 
>   Hi,
> 
> > > [ Looking at https://www.mail-
> > > archive.com/devel@edk2.groups.io/msg33605.html ]
> > >
> > > So, there isn't much tdx-specific in tdx-metadata.  Most ranges are
> > > TDX_METADATA_SECTION_TYPE_TEMP_MEM which I think basically means
> > > these ranges should be accepted by the hypervisor, which is pretty much the
> > > same issue snp tries to solve with this pre-validation range.  Then there are
> > > the ranges for code (aka bfv), for vars (aka cfv) and td_hob.
> > >
> > > td_hob is the only tdx-specific item there, and even that concept (pass
> > > memory ranges as hob list from hypervisor to guest) might be useful outside
> > > tdx.
> > Mailbox is tdx-specific too. But Stack/Heap/OvmfWorkarea/OvmfPageTable
> are
> > common. BFV/CFV are common too.
> 
> Mailbox is tagged "TDX_METADATA_SECTION_TYPE_TEMP_MEM", so nothing
> special to do when loading the firmware, right?
> 
> > > I'd suggest we generalize the tdx-metadata idea and define both generic and
> > > vmm-specific section types:
> > >
> > > enum {
> > >   OVMF_SECTION_TYPE_UNDEFINED = 0;
> > >
> > >   /* generic */
> > >   OVMF_SECTION_TYPE_CODE = 0x100,
> > >   OVMF_SECTION_TYPE_VARS
> > >   OVMF_SECTION_TYPE_SEC_MEM  /* vmm should accept/validate this */
> > >
> > >   /* sev */
> > >   OVMF_SECTION_TYPE_SEV_SECRETS = 0x200,
> > >   OVMF_SECTION_TYPE_SEV_CPUID /* or move to generic? */
> > >
> > >   /* tdx */
> > >   OVMV_SECTION_TYPE_TDX_TD_HOB = 0x300,
> > > };
> > >
> > > Comments?
> > TDX has similar section type.
> 
> Yes.  Both TDX and SNP have simliar requirements, they want store memory
> ranges in the firmware binary in a way that allows qemu finding them and
> using them when initializing the guest.
> 
> SNP stores the ranges directly in the GUID-chained block in the reset
> vector.  The range types are implicit (first is pre-validate area,
> second is cpuid page, ...).
> 
> TDX stores a pointer to tdx-metadata in the GUID-chained block, then the
> tdx-metadata has a list of ranges.  The ranges are explicitly typed
> (section type field).
> 
> The indirection used by TDX keeps the reset vector small.  Also the
> explicit typing of the ranges makes it easier to extend later on if
> needed.
> 
> IMHO SEV should at minimum add explicit types to the memory ranges in
> the boot block, but I'd very much prefer it if SEV and TDX can agree
> on a way to store the memory ranges.
> 
> > But I am not sure if SEV can use this metadata mechanism.
> > Need SEV's comments.
> 
> Brijesh?
> 
> > > Looking at tdx-metadata I have a few questions:
> > >
> > > +_Bfv:
> > > +  DD TDX_BFV_RAW_DATA_OFFSET
> > > +  DD TDX_BFV_RAW_DATA_SIZE
> > >
> > > What is this and why is it needed?
> > Host VMM need to measure the code part (BFV) to MRTD register
> > (which is similar to TPM PCRs).
> 
> Sure, but why you can't use TDX_BFV_MEMORY_BASE +
> TDX_BFV_MEMORY_SIZE
> for that?  The tdvf design guide says it is the file offset.  The
> firmware must be mapped right below 4G, which implies
> RAW_DATA_OFFSET + (4G - FIRMWARE_SIZE) == MEMORY_BASE.
> So having both RAW_DATA_OFFSET and MEMORY_BASE looks redundant to me.
> 
> > > +  DQ TDX_BFV_MEMORY_BASE
> > > +  DQ TDX_BFV_MEMORY_SIZE
> > >
> > > Why "DQ"?  TDX is defined to start in 32bit mode, so you can hardly have
> > > addresses here which do not fit into "DD", correct?
> > Those are the memory. TDX is running in long mode. So it is DQ.
> 
> Hmm?  TDX entry vector is defined to be 32bit.  Which pretty much
> implies you can't map the firmware above 4G, even if one of the first
> actions of the reset vector code is the switch to long mode.
> 
> take care,
>   Gerd
> 
> 
> 
> 
> 

Re: [edk2-devel] [PATCH v6 06/29] OvmfPkg/ResetVector: pre-validate the data pages used in SEC phase

[Gerd Hoffmann]

On Tue, Sep 14, 2021 at 03:49:31AM +0000, Yao, Jiewen wrote:
> I can explain why we prefer DQ instead of DD.
> 
> You are right that current TD entrypoint is 32bit. However, we cannot predict that is always TRUE for the future.

So a "save space in MEMFD" vs. "be future proof" tradeoff.

> Back to 16 bit machine, we have entrypoint at F000:FFF0. But in 32bit mode we removed 1M limitation and move entrypoint to below 4G.
> There is no limitation that we move 64bit entrypoint above 4G for 64bit machine.

Any plans to actually do that?

Defining an 64-bit entry point is a more than just using 64-bit
addresses.  For starters long mode requires paging being enabled.
So a set of initial page tables is needed, and you possibly need
to handle virtual vs. physical addresses.  You also need to find
a place above 4G where you can place the firmware without conflicts
with something else.

So I have my doubts that trying to make the struct future-proof
will actually work out in practice.  Or that we will actually
need that anytime soon.

take care,
  Gerd