[PATCH v7 0/6] SEV Live Migration support for OVMF.

[PATCH v7 0/6] SEV Live Migration support for OVMF.

[Ashish Kalra]

From: Ashish Kalra <ashish.kalra@amd.com>

BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=3467

By default all the SEV guest memory regions are considered encrypted,
if a guest changes the encryption attribute of the page (e.g mark a
page as decrypted) then notify hypervisor. Hypervisor will need to
track the unencrypted pages. The information will be used during
guest live migration, guest page migration and guest debugging.

The patch-set detects if it is running under KVM hypervisor and then
checks for SEV live migration feature support via KVM_FEATURE_CPUID,
if detected setup a new UEFI enviroment variable to indicate OVMF
support for SEV live migration.

This environment variable is created by UEFI but consumed by the
(guest) linux kernel. This is actually part of a 3-way negotiation
of the live migration feature between hypervisor, guest OVMF and
guest kernel. Host indicates support for live migration, which is
detected by OVMF and correspondingly OVMF sets this SetLiveMigrationEnabled
UEFI variable, which is read by the guest kernel and it indicates
to the guest kernel that both host and OVMF support and have
enabled the live migration feature.

A branch containing these patches is available here:
https://github.com/ashkalra/edk2-1/tree/sev_live_migration_v5_11

Changes since v6:
 - Move KvmDetectSevLiveMigrationFeature() function in a new header
   file PeiDxeMemEncryptSevLibInternal.h as this function is
   only used internally.
 - Code Style fixes.
 - More descriptive comments for use of SetLiveMigrationEnabled
   UEFI variable.

Changes since v5:
 - Split first patch into three components, one patch for the
   MemEncryptSevLiveMigrationIsEnabled() API, one patch for the
   SetMemoryEncDecHypercall3() API, one patch to make use of the
   SetMemoryEncDecHypercall3() API. 
 - Fix patch subject, in code and patch comments and 
   additionally add relevant comments.
 - Replace SetMemoryEncDecHypercall3() API's Status argument
   with a boolean IsEncrypted argument and corresponding fixes
   to users of this API call.
 - Fix AsciiStrCmp() usage in KVM hypervisor detection code.

Changes since v4:
 - Remove MemEncryptHypercallLib Library and add support to issue
   hypercall in the BaseMemEncryptSevLib library itself.
 - For SEV-ES, make the VC handler hypercall aware by comparing
   the hypercall number and add the additional register values
   in the GHCB.
 - Fix comments in the hypercall API interface.
 - The encryption bit is set/clear on the smallest page size, hence
   use the 4k page size in MAP_GPA_RANGE hypercall.
 - Make the hypercall expect the guest physical address to be
   page-aligned.
 - Add KVM live migration feature flag check in BaseMemEncryptSevLib
   library similar to how BaseMemEncryptSevLib does for the
   MemEncryptSevIsEnabled() and check it before invoking HC. Also
   export the MemEncryptSevLiveMigrationIsEnabled() function as 
   part of the library.
 - Add error handling on hypercall return, on failure, return error
   code to caller which potentially will cause an assert() and
   terminate the boot.
 
Changes since v3:
 - Fix all DSC files under OvmfPkg except X64 to add support for 
   BaseMemEncryptLib and add NULL instance of BaseMemEncryptLib
   for 32 bit platforms.
 - Add the MemEncryptHypercallLib-related files to Maintainers.txt,
   in section "OvmfPkg: Confidential Computing".
 - Add support for the new KVM_HC_MAP_GPA_RANGE hypercall interface.
 - Add patch for SEV live migration support.

Changes since v2:
 - GHCB_BASE setup during reset-vector as decrypted is marked explicitly
   in the hypervisor page encryption bitmap after setting the 
   PcdSevEsIsEnabled PCD.

Changes since v1:
 - Mark GHCB_BASE setup during reset-vector as decrypted explicitly in
   the hypervisor page encryption bitmap.
 - Resending the series with correct shallow threading.

Ashish Kalra (6):
  OvmfPkg/BaseMemEncryptLib: Detect SEV live migration feature.
  OvmfPkg/BaseMemEncryptLib: Hypercall API for page encryption state
    change
  OvmfPkg/BaseMemEncryptLib: Invoke page encryption state change
    hypercall
  OvmfPkg/VmgExitLib: Encryption state change hypercall support in VC
    handler
  OvmfPkg/PlatformPei: Mark SEC GHCB page as unencrypted via hypercall
  OvmfPkg/AmdSevDxe: Add support for SEV live migration.

 OvmfPkg/AmdSevDxe/AmdSevDxe.c                 | 64 +++++++++++++++++++
 OvmfPkg/AmdSevDxe/AmdSevDxe.inf               |  4 ++
 OvmfPkg/Include/Guid/AmdSevMemEncryptLib.h    | 20 ++++++
 OvmfPkg/Include/Library/MemEncryptSevLib.h    | 64 +++++++++++++++++++
 .../DxeMemEncryptSevLib.inf                   |  1 +
 .../DxeMemEncryptSevLibInternal.c             | 41 ++++++++++++
 .../Ia32/MemEncryptSevLib.c                   | 27 ++++++++
 .../PeiDxeMemEncryptSevLibInternal.c          | 57 +++++++++++++++++
 .../PeiDxeMemEncryptSevLibInternal.h          | 31 +++++++++
 .../PeiMemEncryptSevLib.inf                   |  1 +
 .../PeiMemEncryptSevLibInternal.c             | 41 ++++++++++++
 .../SecMemEncryptSevLibInternal.c             | 38 +++++++++++
 .../X64/AsmHelperStub.nasm                    | 33 ++++++++++
 .../X64/MemEncryptSevLib.c                    | 64 +++++++++++++++++++
 .../X64/PeiDxeVirtualMemory.c                 | 20 ++++++
 OvmfPkg/Library/VmgExitLib/VmgExitVcHandler.c | 13 ++++
 OvmfPkg/OvmfPkg.dec                           |  1 +
 OvmfPkg/PlatformPei/AmdSev.c                  | 11 ++++
 18 files changed, 531 insertions(+)
 create mode 100644 OvmfPkg/Include/Guid/AmdSevMemEncryptLib.h
 create mode 100644 OvmfPkg/Library/BaseMemEncryptSevLib/PeiDxeMemEncryptSevLibInternal.h
 create mode 100644 OvmfPkg/Library/BaseMemEncryptSevLib/X64/AsmHelperStub.nasm

-- 
2.17.1

[PATCH v7 1/6] OvmfPkg/BaseMemEncryptLib: Detect SEV live migration feature.

[Ashish Kalra]

From: Ashish Kalra <ashish.kalra@amd.com>

Add support to check if we are running inside KVM HVM and
KVM HVM supports SEV Live Migration feature.

Cc: Jordan Justen <jordan.l.justen@intel.com>
Cc: Ard Biesheuvel <ard.biesheuvel@arm.com>
Signed-off-by: Ashish Kalra <ashish.kalra@amd.com>
---
 OvmfPkg/Include/Library/MemEncryptSevLib.h                            | 12 +++++
 OvmfPkg/Library/BaseMemEncryptSevLib/DxeMemEncryptSevLibInternal.c    | 41 ++++++++++++++
 OvmfPkg/Library/BaseMemEncryptSevLib/PeiDxeMemEncryptSevLibInternal.c | 57 ++++++++++++++++++++
 OvmfPkg/Library/BaseMemEncryptSevLib/PeiDxeMemEncryptSevLibInternal.h | 31 +++++++++++
 OvmfPkg/Library/BaseMemEncryptSevLib/PeiMemEncryptSevLibInternal.c    | 41 ++++++++++++++
 OvmfPkg/Library/BaseMemEncryptSevLib/SecMemEncryptSevLibInternal.c    | 18 +++++++
 6 files changed, 200 insertions(+)

diff --git a/OvmfPkg/Include/Library/MemEncryptSevLib.h b/OvmfPkg/Include/Library/MemEncryptSevLib.h
index 76d06c206c..b619cadf51 100644
--- a/OvmfPkg/Include/Library/MemEncryptSevLib.h
+++ b/OvmfPkg/Include/Library/MemEncryptSevLib.h
@@ -90,6 +90,18 @@ MemEncryptSevIsEnabled (
   VOID
   );
 
+/**
+  Returns a boolean to indicate whether SEV live migration is enabled.
+
+  @retval TRUE           SEV live migration is enabled
+  @retval FALSE          SEV live migration is not enabled
+**/
+BOOLEAN
+EFIAPI
+MemEncryptSevLiveMigrationIsEnabled (
+  VOID
+  );
+
 /**
   This function clears memory encryption bit for the memory region specified by
   BaseAddress and NumPages from the current page table context.
diff --git a/OvmfPkg/Library/BaseMemEncryptSevLib/DxeMemEncryptSevLibInternal.c b/OvmfPkg/Library/BaseMemEncryptSevLib/DxeMemEncryptSevLibInternal.c
index 2816f859a0..1a99d38866 100644
--- a/OvmfPkg/Library/BaseMemEncryptSevLib/DxeMemEncryptSevLibInternal.c
+++ b/OvmfPkg/Library/BaseMemEncryptSevLib/DxeMemEncryptSevLibInternal.c
@@ -17,9 +17,13 @@
 #include <Register/Cpuid.h>
 #include <Uefi/UefiBaseType.h>
 
+#include "PeiDxeMemEncryptSevLibInternal.h"
+
 STATIC BOOLEAN mSevStatus = FALSE;
 STATIC BOOLEAN mSevEsStatus = FALSE;
 STATIC BOOLEAN mSevStatusChecked = FALSE;
+STATIC BOOLEAN mSevLiveMigrationStatus = FALSE;
+STATIC BOOLEAN mSevLiveMigrationStatusChecked = FALSE;
 
 STATIC UINT64  mSevEncryptionMask = 0;
 STATIC BOOLEAN mSevEncryptionMaskSaved = FALSE;
@@ -87,6 +91,24 @@ InternalMemEncryptSevStatus (
   mSevStatusChecked = TRUE;
 }
 
+/**
+  Figures out if we are running inside KVM HVM and
+  KVM HVM supports SEV Live Migration feature.
+**/
+STATIC
+VOID
+EFIAPI
+InternalDetectSevLiveMigrationFeature (
+  VOID
+  )
+{
+  if (KvmDetectSevLiveMigrationFeature ()) {
+        mSevLiveMigrationStatus = TRUE;
+  }
+
+  mSevLiveMigrationStatusChecked = TRUE;
+}
+
 /**
   Returns a boolean to indicate whether SEV-ES is enabled.
 
@@ -125,6 +147,25 @@ MemEncryptSevIsEnabled (
   return mSevStatus;
 }
 
+/**
+  Returns a boolean to indicate whether SEV live migration is enabled.
+
+  @retval TRUE           SEV live migration is enabled
+  @retval FALSE          SEV live migration is not enabled
+**/
+BOOLEAN
+EFIAPI
+MemEncryptSevLiveMigrationIsEnabled (
+  VOID
+  )
+{
+  if (!mSevLiveMigrationStatusChecked) {
+    InternalDetectSevLiveMigrationFeature ();
+  }
+
+  return mSevLiveMigrationStatus;
+}
+
 /**
   Returns the SEV encryption mask.
 
diff --git a/OvmfPkg/Library/BaseMemEncryptSevLib/PeiDxeMemEncryptSevLibInternal.c b/OvmfPkg/Library/BaseMemEncryptSevLib/PeiDxeMemEncryptSevLibInternal.c
index b4a9f464e2..30f2d90889 100644
--- a/OvmfPkg/Library/BaseMemEncryptSevLib/PeiDxeMemEncryptSevLibInternal.c
+++ b/OvmfPkg/Library/BaseMemEncryptSevLib/PeiDxeMemEncryptSevLibInternal.c
@@ -16,6 +16,8 @@
 #include <Register/SmramSaveStateMap.h>
 #include <Uefi/UefiBaseType.h>
 
+#include "PeiDxeMemEncryptSevLibInternal.h"
+
 /**
   Locate the page range that covers the initial (pre-SMBASE-relocation) SMRAM
   Save State Map.
@@ -61,3 +63,58 @@ MemEncryptSevLocateInitialSmramSaveStateMapPages (
 
   return RETURN_SUCCESS;
 }
+
+/**
+  Figures out if we are running inside KVM HVM and
+  KVM HVM supports SEV Live Migration feature.
+
+  @retval TRUE           SEV live migration is supported.
+  @retval FALSE          SEV live migration is not supported.
+**/
+BOOLEAN
+EFIAPI
+KvmDetectSevLiveMigrationFeature (
+  VOID
+  )
+{
+  CHAR8 Signature[13];
+  UINT32 mKvmLeaf;
+  UINT32 RegEax;
+  UINT32 RegEbx;
+  UINT32 RegEcx;
+  UINT32 RegEdx;
+
+  Signature[12] = '\0';
+  for (mKvmLeaf = 0x40000000; mKvmLeaf < 0x40010000; mKvmLeaf += 0x100) {
+    AsmCpuid (
+      mKvmLeaf,
+      NULL,
+      (UINT32 *) &Signature[0],
+      (UINT32 *) &Signature[4],
+      (UINT32 *) &Signature[8]);
+
+    if (AsciiStrCmp (Signature, "KVMKVMKVM") == 0) {
+      DEBUG ((
+        DEBUG_INFO,
+        "%a: KVM Detected, signature = %a\n",
+        __FUNCTION__,
+        Signature
+        ));
+
+      RegEax = mKvmLeaf + 1;
+      RegEcx = 0;
+      AsmCpuid (mKvmLeaf + 1, &RegEax, &RegEbx, &RegEcx, &RegEdx);
+      if ((RegEax & KVM_FEATURE_MIGRATION_CONTROL) != 0) {
+        DEBUG ((
+          DEBUG_INFO,
+          "%a: SEV Live Migration feature supported\n",
+          __FUNCTION__
+          ));
+
+        return TRUE;
+      }
+    }
+  }
+
+  return FALSE;
+}
diff --git a/OvmfPkg/Library/BaseMemEncryptSevLib/PeiDxeMemEncryptSevLibInternal.h b/OvmfPkg/Library/BaseMemEncryptSevLib/PeiDxeMemEncryptSevLibInternal.h
new file mode 100644
index 0000000000..30cf5de995
--- /dev/null
+++ b/OvmfPkg/Library/BaseMemEncryptSevLib/PeiDxeMemEncryptSevLibInternal.h
@@ -0,0 +1,31 @@
+/** @file
+
+  Secure Encrypted Virtualization (SEV) library helper function
+
+  Copyright (c) 2021, AMD Incorporated. All rights reserved.<BR>
+
+  SPDX-License-Identifier: BSD-2-Clause-Patent
+
+**/
+
+#ifndef PEI_DXE_MEM_ENCRYPT_SEV_LIB_INTERNAL_H_
+#define PEI_DXE_MEM_ENCRYPT_SEV_LIB_INTERNAL_H_
+
+#include <Library/BaseLib.h>
+
+#define KVM_FEATURE_MIGRATION_CONTROL   BIT17
+
+/**
+  Figures out if we are running inside KVM HVM and
+  KVM HVM supports SEV Live Migration feature.
+
+  @retval TRUE           SEV live migration is supported.
+  @retval FALSE          SEV live migration is not supported.
+**/
+BOOLEAN
+EFIAPI
+KvmDetectSevLiveMigrationFeature(
+  VOID
+  );
+
+#endif // PEI_DXE_MEM_ENCRYPT_SEV_LIB_INTERNAL_H_
diff --git a/OvmfPkg/Library/BaseMemEncryptSevLib/PeiMemEncryptSevLibInternal.c b/OvmfPkg/Library/BaseMemEncryptSevLib/PeiMemEncryptSevLibInternal.c
index e2fd109d12..9293b5652d 100644
--- a/OvmfPkg/Library/BaseMemEncryptSevLib/PeiMemEncryptSevLibInternal.c
+++ b/OvmfPkg/Library/BaseMemEncryptSevLib/PeiMemEncryptSevLibInternal.c
@@ -17,9 +17,13 @@
 #include <Register/Cpuid.h>
 #include <Uefi/UefiBaseType.h>
 
+#include "PeiDxeMemEncryptSevLibInternal.h"
+
 STATIC BOOLEAN mSevStatus = FALSE;
 STATIC BOOLEAN mSevEsStatus = FALSE;
 STATIC BOOLEAN mSevStatusChecked = FALSE;
+STATIC BOOLEAN mSevLiveMigrationStatus = FALSE;
+STATIC BOOLEAN mSevLiveMigrationStatusChecked = FALSE;
 
 STATIC UINT64  mSevEncryptionMask = 0;
 STATIC BOOLEAN mSevEncryptionMaskSaved = FALSE;
@@ -87,6 +91,24 @@ InternalMemEncryptSevStatus (
   mSevStatusChecked = TRUE;
 }
 
+/**
+  Figures out if we are running inside KVM HVM and
+  KVM HVM supports SEV Live Migration feature.
+**/
+STATIC
+VOID
+EFIAPI
+InternalDetectSevLiveMigrationFeature (
+  VOID
+  )
+{
+  if (KvmDetectSevLiveMigrationFeature ()) {
+    mSevLiveMigrationStatus = TRUE;
+  }
+
+  mSevLiveMigrationStatusChecked = TRUE;
+}
+
 /**
   Returns a boolean to indicate whether SEV-ES is enabled.
 
@@ -125,6 +147,25 @@ MemEncryptSevIsEnabled (
   return mSevStatus;
 }
 
+/**
+  Returns a boolean to indicate whether SEV live migration is enabled.
+
+  @retval TRUE           SEV live migration is enabled
+  @retval FALSE          SEV live migration is not enabled
+**/
+BOOLEAN
+EFIAPI
+MemEncryptSevLiveMigrationIsEnabled (
+  VOID
+  )
+{
+  if (!mSevLiveMigrationStatusChecked) {
+    InternalDetectSevLiveMigrationFeature ();
+  }
+
+  return mSevLiveMigrationStatus;
+}
+
 /**
   Returns the SEV encryption mask.
 
diff --git a/OvmfPkg/Library/BaseMemEncryptSevLib/SecMemEncryptSevLibInternal.c b/OvmfPkg/Library/BaseMemEncryptSevLib/SecMemEncryptSevLibInternal.c
index 56d8f3f318..d9f7befcd2 100644
--- a/OvmfPkg/Library/BaseMemEncryptSevLib/SecMemEncryptSevLibInternal.c
+++ b/OvmfPkg/Library/BaseMemEncryptSevLib/SecMemEncryptSevLibInternal.c
@@ -100,6 +100,24 @@ MemEncryptSevIsEnabled (
   return Msr.Bits.SevBit ? TRUE : FALSE;
 }
 
+/**
+  Returns a boolean to indicate whether SEV live migration is enabled.
+
+  @retval TRUE           SEV live migration is enabled
+  @retval FALSE          SEV live migration is not enabled
+**/
+BOOLEAN
+EFIAPI
+MemEncryptSevLiveMigrationIsEnabled (
+  VOID
+  )
+{
+  //
+  // Not used in SEC phase.
+  //
+  return FALSE;
+}
+
 /**
   Returns the SEV encryption mask.
 
-- 
2.17.1

[PATCH v7 2/6] OvmfPkg/BaseMemEncryptLib: Hypercall API for page encryption state change

[Ashish Kalra]

From: Ashish Kalra <ashish.kalra@amd.com>

Add API to issue hypercall on page encryption state change.

By default all the SEV guest memory regions are considered encrypted,
if a guest changes the encryption attribute of the page (e.g mark a
page as decrypted) then notify hypervisor. Hypervisor will need to
track the unencrypted pages. The information will be used during
guest live migration, guest page migration and guest debugging.

This hypercall is used to notify hypervisor when the page's
encryption state changes.

Cc: Jordan Justen <jordan.l.justen@intel.com>
Cc: Ard Biesheuvel <ard.biesheuvel@arm.com>
Signed-off-by: Brijesh Singh <brijesh.singh@amd.com>
Signed-off-by: Ashish Kalra <ashish.kalra@amd.com>
---
 OvmfPkg/Include/Library/MemEncryptSevLib.h                         | 52 ++++++++++++++++
 OvmfPkg/Library/BaseMemEncryptSevLib/DxeMemEncryptSevLib.inf       |  1 +
 OvmfPkg/Library/BaseMemEncryptSevLib/Ia32/MemEncryptSevLib.c       | 27 +++++++++
 OvmfPkg/Library/BaseMemEncryptSevLib/PeiMemEncryptSevLib.inf       |  1 +
 OvmfPkg/Library/BaseMemEncryptSevLib/SecMemEncryptSevLibInternal.c | 20 ++++++
 OvmfPkg/Library/BaseMemEncryptSevLib/X64/AsmHelperStub.nasm        | 33 ++++++++++
 OvmfPkg/Library/BaseMemEncryptSevLib/X64/MemEncryptSevLib.c        | 64 ++++++++++++++++++++
 7 files changed, 198 insertions(+)

diff --git a/OvmfPkg/Include/Library/MemEncryptSevLib.h b/OvmfPkg/Include/Library/MemEncryptSevLib.h
index b619cadf51..9d46c5f5d6 100644
--- a/OvmfPkg/Include/Library/MemEncryptSevLib.h
+++ b/OvmfPkg/Include/Library/MemEncryptSevLib.h
@@ -234,4 +234,56 @@ MemEncryptSevClearMmioPageEncMask (
   IN UINTN                    NumPages
   );
 
+/**
+ This hypercall is used to notify hypervisor when the page's encryption
+ state changes.
+
+ @param[in]   PhysicalAddress       The physical address that is the start address
+                                    of a memory region.
+ @param[in]   Pages                 Number of pages in memory region.
+ @param[in]   IsEncrypted           Encrypted or Decrypted.
+
+ @retval RETURN_SUCCESS             Hypercall returned success.
+ @retval RETURN_UNSUPPORTED         Hypercall not supported.
+ @retval RETURN_NO_MAPPING          Hypercall returned error.
+**/
+RETURN_STATUS
+EFIAPI
+SetMemoryEncDecHypercall3 (
+  IN  UINTN     PhysicalAddress,
+  IN  UINTN     Pages,
+  IN  BOOLEAN   IsEncrypted
+  );
+
+#define KVM_HC_MAP_GPA_RANGE   12
+#define KVM_MAP_GPA_RANGE_PAGE_SZ_4K    0
+#define KVM_MAP_GPA_RANGE_PAGE_SZ_2M    BIT0
+#define KVM_MAP_GPA_RANGE_PAGE_SZ_1G    BIT1
+#define KVM_MAP_GPA_RANGE_ENC_STATE(n)  ((n) << 4)
+#define KVM_MAP_GPA_RANGE_ENCRYPTED     KVM_MAP_GPA_RANGE_ENC_STATE(1)
+#define KVM_MAP_GPA_RANGE_DECRYPTED     KVM_MAP_GPA_RANGE_ENC_STATE(0)
+
+/**
+ Interface exposed by the ASM implementation of the core hypercall
+
+ @param[in]   HypercallNum          KVM_HC_MAP_GPA_RANGE hypercall.
+ @param[in]   PhysicalAddress       The physical address that is the start address
+                                    of a memory region.
+ @param[in]   Pages                 Number of pages in memory region.
+ @param[in]   Attributes            Bits 3:0 - preferred page size encoding,
+                                    0 = 4kb, 1 = 2mb, 2 = 1gb, etc...
+                                    Bit  4 - plaintext = 0, encrypted = 1
+                                    Bits 63:5 - reserved (must be zero)
+
+  @retval Hypercall returned status.
+**/
+UINTN
+EFIAPI
+SetMemoryEncDecHypercall3AsmStub (
+  IN  UINTN  HypercallNum,
+  IN  UINTN  PhysicalAddress,
+  IN  UINTN  Pages,
+  IN  UINTN  Attributes
+  );
+
 #endif // _MEM_ENCRYPT_SEV_LIB_H_
diff --git a/OvmfPkg/Library/BaseMemEncryptSevLib/DxeMemEncryptSevLib.inf b/OvmfPkg/Library/BaseMemEncryptSevLib/DxeMemEncryptSevLib.inf
index f2e162d680..0c28afadee 100644
--- a/OvmfPkg/Library/BaseMemEncryptSevLib/DxeMemEncryptSevLib.inf
+++ b/OvmfPkg/Library/BaseMemEncryptSevLib/DxeMemEncryptSevLib.inf
@@ -38,6 +38,7 @@
   X64/PeiDxeVirtualMemory.c
   X64/VirtualMemory.c
   X64/VirtualMemory.h
+  X64/AsmHelperStub.nasm
 
 [Sources.IA32]
   Ia32/MemEncryptSevLib.c
diff --git a/OvmfPkg/Library/BaseMemEncryptSevLib/Ia32/MemEncryptSevLib.c b/OvmfPkg/Library/BaseMemEncryptSevLib/Ia32/MemEncryptSevLib.c
index be260e0d10..516d639489 100644
--- a/OvmfPkg/Library/BaseMemEncryptSevLib/Ia32/MemEncryptSevLib.c
+++ b/OvmfPkg/Library/BaseMemEncryptSevLib/Ia32/MemEncryptSevLib.c
@@ -136,3 +136,30 @@ MemEncryptSevClearMmioPageEncMask (
   //
   return RETURN_UNSUPPORTED;
 }
+
+/**
+ This hyercall is used to notify hypervisor when the page's encryption
+ state changes.
+
+ @param[in]   PhysicalAddress       The physical address that is the start address
+                                    of a memory region.
+ @param[in]   Pages                 Number of Pages in the memory region.
+ @param[in]   IsEncrypted           Encrypted or Decrypted.
+
+ @retval RETURN_SUCCESS             Hypercall returned success.
+ @retval RETURN_UNSUPPORTED         Hypercall not supported.
+ @retval RETURN_NO_MAPPING          Hypercall returned error.
+**/
+RETURN_STATUS
+EFIAPI
+SetMemoryEncDecHypercall3 (
+  IN  UINTN     PhysicalAddress,
+  IN  UINTN     Pages,
+  IN  BOOLEAN   IsEncrypted
+  )
+{
+  //
+  // Memory encryption bit is not accessible in 32-bit mode
+  //
+  return RETURN_UNSUPPORTED;
+}
diff --git a/OvmfPkg/Library/BaseMemEncryptSevLib/PeiMemEncryptSevLib.inf b/OvmfPkg/Library/BaseMemEncryptSevLib/PeiMemEncryptSevLib.inf
index 03a78c32df..3233ca7979 100644
--- a/OvmfPkg/Library/BaseMemEncryptSevLib/PeiMemEncryptSevLib.inf
+++ b/OvmfPkg/Library/BaseMemEncryptSevLib/PeiMemEncryptSevLib.inf
@@ -38,6 +38,7 @@
   X64/PeiDxeVirtualMemory.c
   X64/VirtualMemory.c
   X64/VirtualMemory.h
+  X64/AsmHelperStub.nasm
 
 [Sources.IA32]
   Ia32/MemEncryptSevLib.c
diff --git a/OvmfPkg/Library/BaseMemEncryptSevLib/SecMemEncryptSevLibInternal.c b/OvmfPkg/Library/BaseMemEncryptSevLib/SecMemEncryptSevLibInternal.c
index d9f7befcd2..ebb1c39319 100644
--- a/OvmfPkg/Library/BaseMemEncryptSevLib/SecMemEncryptSevLibInternal.c
+++ b/OvmfPkg/Library/BaseMemEncryptSevLib/SecMemEncryptSevLibInternal.c
@@ -118,6 +118,26 @@ MemEncryptSevLiveMigrationIsEnabled (
   return FALSE;
 }
 
+/**
+  Interface exposed by the ASM implementation of the core hypercall
+
+  @retval Hypercall returned status.
+**/
+UINTN
+EFIAPI
+SetMemoryEncDecHypercall3AsmStub (
+  IN  UINTN  HypercallNum,
+  IN  UINTN  PhysicalAddress,
+  IN  UINTN  Pages,
+  IN  UINTN  Attributes
+  )
+{
+  //
+  // Not used in SEC phase.
+  //
+  return RETURN_UNSUPPORTED;
+}
+
 /**
   Returns the SEV encryption mask.
 
diff --git a/OvmfPkg/Library/BaseMemEncryptSevLib/X64/AsmHelperStub.nasm b/OvmfPkg/Library/BaseMemEncryptSevLib/X64/AsmHelperStub.nasm
new file mode 100644
index 0000000000..0ec35dd9b6
--- /dev/null
+++ b/OvmfPkg/Library/BaseMemEncryptSevLib/X64/AsmHelperStub.nasm
@@ -0,0 +1,33 @@
+/** @file
+
+  ASM helper stub to invoke hypercall
+
+  Copyright (c) 2021, AMD Incorporated. All rights reserved.<BR>
+
+  SPDX-License-Identifier: BSD-2-Clause-Patent
+
+**/
+
+DEFAULT REL
+SECTION .text
+
+; UINTN
+; EFIAPI
+; SetMemoryEncDecHypercall3AsmStub (
+;   IN UINTN HypercallNum,
+;   IN UINTN Arg1,
+;   IN UINTN Arg2,
+;   IN UINTN Arg3
+;   );
+global ASM_PFX(SetMemoryEncDecHypercall3AsmStub)
+ASM_PFX(SetMemoryEncDecHypercall3AsmStub):
+  ; UEFI calling conventions require RBX to
+  ; be nonvolatile/callee-saved.
+  push rbx
+  mov rax, rcx    ; Copy HypercallNumber to rax
+  mov rbx, rdx    ; Copy Arg1 to the register expected by KVM
+  mov rcx, r8     ; Copy Arg2 to register expected by KVM
+  mov rdx, r9     ; Copy Arg3 to register expected by KVM
+  vmmcall         ; Call VMMCALL
+  pop rbx
+  ret
diff --git a/OvmfPkg/Library/BaseMemEncryptSevLib/X64/MemEncryptSevLib.c b/OvmfPkg/Library/BaseMemEncryptSevLib/X64/MemEncryptSevLib.c
index a57e8fd37f..a6246d4312 100644
--- a/OvmfPkg/Library/BaseMemEncryptSevLib/X64/MemEncryptSevLib.c
+++ b/OvmfPkg/Library/BaseMemEncryptSevLib/X64/MemEncryptSevLib.c
@@ -143,3 +143,67 @@ MemEncryptSevClearMmioPageEncMask (
            );
 
 }
+
+/**
+ This hyercall is used to notify hypervisor when the page's encryption
+ state changes.
+
+ @param[in]   PhysicalAddress       The physical address that is the start address
+                                    of a memory region.
+ @param[in]   Pages                 Number of Pages in the memory region.
+ @param[in]   IsEncrypted           Encrypted or Decrypted.
+
+ @retval RETURN_SUCCESS             Hypercall returned success.
+ @retval RETURN_UNSUPPORTED         Hypercall not supported.
+ @retval RETURN_NO_MAPPING          Hypercall returned error.
+**/
+RETURN_STATUS
+EFIAPI
+SetMemoryEncDecHypercall3 (
+  IN  UINTN     PhysicalAddress,
+  IN  UINTN     Pages,
+  IN  BOOLEAN   IsEncrypted
+  )
+{
+  RETURN_STATUS Ret;
+  UINTN Error;
+  UINTN EncryptState;
+
+  Ret = RETURN_UNSUPPORTED;
+
+  if (MemEncryptSevLiveMigrationIsEnabled ()) {
+    Ret = RETURN_SUCCESS;
+    //
+    // The encryption bit is set/clear on the smallest page size, hence
+    // use the 4k page size in MAP_GPA_RANGE hypercall below.
+    //
+    // Also, when the GCD map is being walked and the c-bit being cleared
+    // from MMIO and NonExistent memory spaces, the physical address
+    // range being passed may not be page-aligned and adding an assert
+    // here prevents booting. Hence, rounding it down when calling
+    // SetMemoryEncDecHypercall3AsmStub below.
+    //
+
+    EncryptState = IsEncrypted ? KVM_MAP_GPA_RANGE_ENCRYPTED :
+        KVM_MAP_GPA_RANGE_DECRYPTED;
+
+    Error = SetMemoryEncDecHypercall3AsmStub (
+              KVM_HC_MAP_GPA_RANGE,
+              PhysicalAddress & ~EFI_PAGE_MASK,
+              Pages,
+              KVM_MAP_GPA_RANGE_PAGE_SZ_4K | EncryptState
+              );
+
+    if (Error != 0) {
+      DEBUG ((DEBUG_ERROR,
+        "SetMemoryEncDecHypercall3 failed, Phys = %x, Pages = %d, Err = %Ld\n",
+        PhysicalAddress,
+        Pages,
+        (INT64)Error));
+
+      Ret = RETURN_NO_MAPPING;
+    }
+  }
+
+  return Ret;
+}
-- 
2.17.1

[PATCH v7 3/6] OvmfPkg/BaseMemEncryptLib: Invoke page encryption state change hypercall

[Ashish Kalra]

From: Ashish Kalra <ashish.kalra@amd.com>

Invoke the hypercall API to notify hypervisor when the page's
encryption state changes.

Cc: Jordan Justen <jordan.l.justen@intel.com>
Cc: Ard Biesheuvel <ard.biesheuvel@arm.com>
Signed-off-by: Brijesh Singh <brijesh.singh@amd.com>
Signed-off-by: Ashish Kalra <ashish.kalra@amd.com>
---
 OvmfPkg/Library/BaseMemEncryptSevLib/X64/PeiDxeVirtualMemory.c | 20 ++++++++++++++++++++
 1 file changed, 20 insertions(+)

diff --git a/OvmfPkg/Library/BaseMemEncryptSevLib/X64/PeiDxeVirtualMemory.c b/OvmfPkg/Library/BaseMemEncryptSevLib/X64/PeiDxeVirtualMemory.c
index c696745f9d..f562e16fc2 100644
--- a/OvmfPkg/Library/BaseMemEncryptSevLib/X64/PeiDxeVirtualMemory.c
+++ b/OvmfPkg/Library/BaseMemEncryptSevLib/X64/PeiDxeVirtualMemory.c
@@ -585,6 +585,9 @@ SetMemoryEncDec (
   UINT64                         AddressEncMask;
   BOOLEAN                        IsWpEnabled;
   RETURN_STATUS                  Status;
+  UINTN                          Size;
+  BOOLEAN                        CBitChanged;
+  PHYSICAL_ADDRESS               OrigPhysicalAddress;
 
   //
   // Set PageMapLevel4Entry to suppress incorrect compiler/analyzer warnings.
@@ -636,6 +639,9 @@ SetMemoryEncDec (
 
   Status = EFI_SUCCESS;
 
+  Size = Length;
+  CBitChanged = FALSE;
+  OrigPhysicalAddress = PhysicalAddress;
   while (Length != 0)
   {
     //
@@ -695,6 +701,7 @@ SetMemoryEncDec (
           ));
         PhysicalAddress += BIT30;
         Length -= BIT30;
+        CBitChanged = TRUE;
       } else {
         //
         // We must split the page
@@ -749,6 +756,7 @@ SetMemoryEncDec (
           SetOrClearCBit (&PageDirectory2MEntry->Uint64, Mode);
           PhysicalAddress += BIT21;
           Length -= BIT21;
+          CBitChanged = TRUE;
         } else {
           //
           // We must split up this page into 4K pages
@@ -791,6 +799,7 @@ SetMemoryEncDec (
         SetOrClearCBit (&PageTableEntry->Uint64, Mode);
         PhysicalAddress += EFI_PAGE_SIZE;
         Length -= EFI_PAGE_SIZE;
+        CBitChanged = TRUE;
       }
     }
   }
@@ -808,6 +817,17 @@ SetMemoryEncDec (
   //
   CpuFlushTlb();
 
+  //
+  // Notify Hypervisor on C-bit status
+  //
+  if (CBitChanged) {
+    Status = SetMemoryEncDecHypercall3 (
+               OrigPhysicalAddress,
+               EFI_SIZE_TO_PAGES(Size),
+               (Mode == SetCBit) ? TRUE : FALSE
+               );
+  }
+
 Done:
   //
   // Restore page table write protection, if any.
-- 
2.17.1

[PATCH v7 4/6] OvmfPkg/VmgExitLib: Encryption state change hypercall support in VC handler

[Ashish Kalra]

From: Ashish Kalra <ashish.kalra@amd.com>

Make the #VC handler aware of the page encryption state
change hypercall by adding support to check KVM_HC_MAP_GPA_RANGE
hypercall and add the additional register values used by
hypercall in the GHCB.

Cc: Jordan Justen <jordan.l.justen@intel.com>
Cc: Ard Biesheuvel <ard.biesheuvel@arm.com>
Signed-off-by: Ashish Kalra <ashish.kalra@amd.com>
---
 OvmfPkg/Library/VmgExitLib/VmgExitVcHandler.c | 13 +++++++++++++
 1 file changed, 13 insertions(+)

diff --git a/OvmfPkg/Library/VmgExitLib/VmgExitVcHandler.c b/OvmfPkg/Library/VmgExitLib/VmgExitVcHandler.c
index 41b0c8cc53..2d06343e92 100644
--- a/OvmfPkg/Library/VmgExitLib/VmgExitVcHandler.c
+++ b/OvmfPkg/Library/VmgExitLib/VmgExitVcHandler.c
@@ -1171,6 +1171,19 @@ VmmCallExit (
   Ghcb->SaveArea.Cpl = (UINT8) (Regs->Cs & 0x3);
   VmgSetOffsetValid (Ghcb, GhcbCpl);
 
+  if (Regs->Rax == KVM_HC_MAP_GPA_RANGE) {
+    //
+    // KVM_HC_MAP_GPA_RANGE hypercall requires these
+    // extra registers.
+    //
+    Ghcb->SaveArea.Rbx = Regs->Rbx;
+    VmgSetOffsetValid (Ghcb, GhcbRbx);
+    Ghcb->SaveArea.Rcx = Regs->Rcx;
+    VmgSetOffsetValid (Ghcb, GhcbRcx);
+    Ghcb->SaveArea.Rdx = Regs->Rdx;
+    VmgSetOffsetValid (Ghcb, GhcbRdx);
+  }
+
   Status = VmgExit (Ghcb, SVM_EXIT_VMMCALL, 0, 0);
   if (Status != 0) {
     return Status;
-- 
2.17.1

[PATCH v7 5/6] OvmfPkg/PlatformPei: Mark SEC GHCB page as unencrypted via hypercall

[Ashish Kalra]

From: Ashish Kalra <ashish.kalra@amd.com>

Mark the SEC GHCB page (that is mapped as unencrypted in
ResetVector code) in the hypervisor's guest page encryption
state tracking.

Cc: Jordan Justen <jordan.l.justen@intel.com>
Cc: Ard Biesheuvel <ard.biesheuvel@arm.com>
Signed-off-by: Ashish Kalra <ashish.kalra@amd.com>
---
 OvmfPkg/PlatformPei/AmdSev.c | 11 +++++++++++
 1 file changed, 11 insertions(+)

diff --git a/OvmfPkg/PlatformPei/AmdSev.c b/OvmfPkg/PlatformPei/AmdSev.c
index a8bf610022..1d38056ec0 100644
--- a/OvmfPkg/PlatformPei/AmdSev.c
+++ b/OvmfPkg/PlatformPei/AmdSev.c
@@ -52,6 +52,17 @@ AmdSevEsInitialize (
   PcdStatus = PcdSetBoolS (PcdSevEsIsEnabled, TRUE);
   ASSERT_RETURN_ERROR (PcdStatus);
 
+  //
+  // The SEC Ghcb setup during reset-vector needs to be marked as
+  // decrypted in the hypervisor's guest page encryption state
+  // tracking.
+  //
+  SetMemoryEncDecHypercall3 (
+    FixedPcdGet32 (PcdOvmfSecGhcbBase),
+    EFI_SIZE_TO_PAGES(FixedPcdGet32 (PcdOvmfSecGhcbSize)),
+    FALSE
+    );
+
   //
   // Allocate GHCB and per-CPU variable pages.
   //   Since the pages must survive across the UEFI to OS transition
-- 
2.17.1

[PATCH v7 6/6] OvmfPkg/AmdSevDxe: Add support for SEV live migration.

[Ashish Kalra]

From: Ashish Kalra <ashish.kalra@amd.com>

Check for SEV live migration feature support, if detected
setup a new UEFI enviroment variable to indicate OVMF
support for SEV live migration.

This environment variable is created by UEFI but consumed
by the (guest) linux kernel. This is actually part of a
3-way negotiation of the live migration feature between
hypervisor, guest OVMF and guest kernel. Host indicates
support for live migration, which is detected by OVMF
and correspondingly OVMF sets this SetLiveMigrationEnabled
UEFI variable, which is read by the guest kernel and it
indicates to the guest kernel that both host and OVMF
support and have enabled the live migration feature.

The new runtime UEFI environment variable is set via the
notification function registered for the
EFI_END_OF_DXE_EVENT_GROUP_GUID event in AmdSevDxe driver.

AmdSevDxe module is an apriori driver so it gets loaded between PEI
and DXE phases and the SetVariable call will fail at the driver's
entry point as the Variable DXE module is still not loaded yet.
So we need to wait for an event notification which is signaled
after the Variable DXE module is loaded, hence, using the
EndOfDxe event notification to make this call.

Signed-off-by: Ashish Kalra <ashish.kalra@amd.com>
---
 OvmfPkg/AmdSevDxe/AmdSevDxe.c              | 64 ++++++++++++++++++++
 OvmfPkg/AmdSevDxe/AmdSevDxe.inf            |  4 ++
 OvmfPkg/Include/Guid/AmdSevMemEncryptLib.h | 20 ++++++
 OvmfPkg/OvmfPkg.dec                        |  1 +
 4 files changed, 89 insertions(+)

diff --git a/OvmfPkg/AmdSevDxe/AmdSevDxe.c b/OvmfPkg/AmdSevDxe/AmdSevDxe.c
index c66c4e9b92..c4d28985bf 100644
--- a/OvmfPkg/AmdSevDxe/AmdSevDxe.c
+++ b/OvmfPkg/AmdSevDxe/AmdSevDxe.c
@@ -15,10 +15,47 @@
 #include <Library/BaseMemoryLib.h>
 #include <Library/DebugLib.h>
 #include <Library/DxeServicesTableLib.h>
+#include <Library/UefiRuntimeServicesTableLib.h>
+#include <Library/UefiBootServicesTableLib.h>
 #include <Library/MemEncryptSevLib.h>
 #include <Library/MemoryAllocationLib.h>
+#include <Guid/AmdSevMemEncryptLib.h>
+#include <Guid/EventGroup.h>
 #include <Library/PcdLib.h>
 
+STATIC
+VOID
+EFIAPI
+AmdSevDxeOnEndOfDxe (
+  IN EFI_EVENT Event,
+  IN VOID      *EventToSignal
+  )
+{
+  EFI_STATUS Status;
+  BOOLEAN SevLiveMigrationEnabled;
+
+  SevLiveMigrationEnabled = MemEncryptSevLiveMigrationIsEnabled();
+
+  if (SevLiveMigrationEnabled) {
+    Status = gRT->SetVariable (
+               L"SevLiveMigrationEnabled",
+               &gAmdSevMemEncryptGuid,
+               EFI_VARIABLE_NON_VOLATILE |
+               EFI_VARIABLE_BOOTSERVICE_ACCESS |
+               EFI_VARIABLE_RUNTIME_ACCESS,
+               sizeof SevLiveMigrationEnabled,
+               &SevLiveMigrationEnabled
+               );
+
+    DEBUG ((
+      DEBUG_INFO,
+      "%a: Setting SevLiveMigrationEnabled variable, status = %lx\n",
+      __FUNCTION__,
+      Status
+      ));
+  }
+}
+
 EFI_STATUS
 EFIAPI
 AmdSevDxeEntryPoint (
@@ -30,6 +67,7 @@ AmdSevDxeEntryPoint (
   EFI_GCD_MEMORY_SPACE_DESCRIPTOR  *AllDescMap;
   UINTN                            NumEntries;
   UINTN                            Index;
+  EFI_EVENT                        Event;
 
   //
   // Do nothing when SEV is not enabled
@@ -130,5 +168,31 @@ AmdSevDxeEntryPoint (
     }
   }
 
+  //
+  // AmdSevDxe module is an apriori driver so it gets loaded between PEI
+  // and DXE phases and the SetVariable call will fail at the driver's
+  // entry point as the Variable DXE module is still not loaded yet.
+  // So we need to wait for an event notification which is signaled
+  // after the Variable DXE module is loaded, hence, using the
+  // EndOfDxe event notification to make this call.
+  //
+  // Register EFI_END_OF_DXE_EVENT_GROUP_GUID event.
+  // The notification function sets the runtime variable indicating OVMF
+  // support for SEV live migration.
+  //
+  Status = gBS->CreateEventEx (
+                  EVT_NOTIFY_SIGNAL,
+                  TPL_CALLBACK,
+                  AmdSevDxeOnEndOfDxe,
+                  NULL,
+                  &gEfiEndOfDxeEventGroupGuid,
+                  &Event
+                  );
+
+  if (EFI_ERROR (Status)) {
+      DEBUG ((DEBUG_ERROR, "%a: CreateEventEx(): %r\n",
+        __FUNCTION__, Status));
+  }
+
   return EFI_SUCCESS;
 }
diff --git a/OvmfPkg/AmdSevDxe/AmdSevDxe.inf b/OvmfPkg/AmdSevDxe/AmdSevDxe.inf
index 0676fcc5b6..2ad1fb8632 100644
--- a/OvmfPkg/AmdSevDxe/AmdSevDxe.inf
+++ b/OvmfPkg/AmdSevDxe/AmdSevDxe.inf
@@ -45,3 +45,7 @@
 
 [Pcd]
   gUefiOvmfPkgTokenSpaceGuid.PcdOvmfHostBridgePciDevId
+
+[Guids]
+  gAmdSevMemEncryptGuid
+  gEfiEndOfDxeEventGroupGuid         ## CONSUMES   ## Event
diff --git a/OvmfPkg/Include/Guid/AmdSevMemEncryptLib.h b/OvmfPkg/Include/Guid/AmdSevMemEncryptLib.h
new file mode 100644
index 0000000000..1c948fbcdd
--- /dev/null
+++ b/OvmfPkg/Include/Guid/AmdSevMemEncryptLib.h
@@ -0,0 +1,20 @@
+/** @file
+
+  AMD Memory Encryption GUID, define a new GUID for defining
+  new UEFI environment variables assocaiated with SEV Memory Encryption.
+
+  Copyright (c) 2021, AMD Inc. All rights reserved.<BR>
+
+  SPDX-License-Identifier: BSD-2-Clause-Patent
+
+**/
+
+#ifndef __AMD_SEV_MEMENCRYPT_LIB_H__
+#define __AMD_SEV_MEMENCRYPT_LIB_H__
+
+#define AMD_SEV_MEMENCRYPT_GUID \
+{0x0cf29b71, 0x9e51, 0x433a, {0xa3, 0xb7, 0x81, 0xf3, 0xab, 0x16, 0xb8, 0x75}}
+
+extern EFI_GUID gAmdSevMemEncryptGuid;
+
+#endif
diff --git a/OvmfPkg/OvmfPkg.dec b/OvmfPkg/OvmfPkg.dec
index 8fb6f257e8..35d6362a4d 100644
--- a/OvmfPkg/OvmfPkg.dec
+++ b/OvmfPkg/OvmfPkg.dec
@@ -124,6 +124,7 @@
   gQemuKernelLoaderFsMediaGuid          = {0x1428f772, 0xb64a, 0x441e, {0xb8, 0xc3, 0x9e, 0xbd, 0xd7, 0xf8, 0x93, 0xc7}}
   gGrubFileGuid                         = {0xb5ae312c, 0xbc8a, 0x43b1, {0x9c, 0x62, 0xeb, 0xb8, 0x26, 0xdd, 0x5d, 0x07}}
   gConfidentialComputingSecretGuid      = {0xadf956ad, 0xe98c, 0x484c, {0xae, 0x11, 0xb5, 0x1c, 0x7d, 0x33, 0x64, 0x47}}
+  gAmdSevMemEncryptGuid                 = {0x0cf29b71, 0x9e51, 0x433a, {0xa3, 0xb7, 0x81, 0xf3, 0xab, 0x16, 0xb8, 0x75}}
 
 [Ppis]
   # PPI whose presence in the PPI database signals that the TPM base address
-- 
2.17.1

Re: [edk2-devel] [PATCH v7 5/6] OvmfPkg/PlatformPei: Mark SEC GHCB page as unencrypted via hypercall

[Nikunj A. Dadhania]

Hi Ashish,

On 8/19/2021 7:36 PM, Ashish Kalra via groups.io wrote:
> From: Ashish Kalra <ashish.kalra@amd.com>
> 
> Mark the SEC GHCB page (that is mapped as unencrypted in
> ResetVector code) in the hypervisor's guest page encryption
> state tracking.
> 
> Cc: Jordan Justen <jordan.l.justen@intel.com>
> Cc: Ard Biesheuvel <ard.biesheuvel@arm.com>
> Signed-off-by: Ashish Kalra <ashish.kalra@amd.com>
> ---
>  OvmfPkg/PlatformPei/AmdSev.c | 11 +++++++++++
>  1 file changed, 11 insertions(+)
> 
> diff --git a/OvmfPkg/PlatformPei/AmdSev.c b/OvmfPkg/PlatformPei/AmdSev.c
> index a8bf610022..1d38056ec0 100644
> --- a/OvmfPkg/PlatformPei/AmdSev.c
> +++ b/OvmfPkg/PlatformPei/AmdSev.c
> @@ -52,6 +52,17 @@ AmdSevEsInitialize (
>    PcdStatus = PcdSetBoolS (PcdSevEsIsEnabled, TRUE);
>    ASSERT_RETURN_ERROR (PcdStatus);
>  
> +  //
> +  // The SEC Ghcb setup during reset-vector needs to be marked as
> +  // decrypted in the hypervisor's guest page encryption state
> +  // tracking.
> +  //
> +  SetMemoryEncDecHypercall3 (
> +    FixedPcdGet32 (PcdOvmfSecGhcbBase),
> +    EFI_SIZE_TO_PAGES(FixedPcdGet32 (PcdOvmfSecGhcbSize)),
> +    FALSE
> +    );

PcdOvmfSecGhcbSize is set to 2 pages (8192 bytes). AFAIU, only first page 
needs to be change to shared, second page should be kept private.

>    //
>    // Allocate GHCB and per-CPU variable pages.
>    //   Since the pages must survive across the UEFI to OS transition
> 

Regards
Nikunj

Re: [edk2-devel] [PATCH v7 5/6] OvmfPkg/PlatformPei: Mark SEC GHCB page as unencrypted via hypercall

[Ashish Kalra]

[AMD Official Use Only - General]

Hello Nikunj,

That is true, but there is no SEV live migration happening before GHCB pages are re-setup and the hypercalls are invoked again for GHCB pages from
the guest kernel after it checks the live migration runtime environment flag setup by OVMF.

Thanks,
Ashish

-----Original Message-----
From: Dadhania, Nikunj <NikunjAmritlal.Dadhania@amd.com> 
Sent: Tuesday, May 31, 2022 10:14 AM
To: devel@edk2.groups.io; Kalra, Ashish <Ashish.Kalra@amd.com>
Cc: dovmurik@linux.vnet.ibm.com; brijesh.singh@amd.com; tobin@ibm.com; Lendacky, Thomas <Thomas.Lendacky@amd.com>; jejb@linux.ibm.com; erdemaktas@google.com; jiewen.yao@intel.com; min.m.xu@intel.com; jordan.l.justen@intel.com; ard.biesheuvel@arm.com
Subject: Re: [edk2-devel] [PATCH v7 5/6] OvmfPkg/PlatformPei: Mark SEC GHCB page as unencrypted via hypercall

Hi Ashish,

On 8/19/2021 7:36 PM, Ashish Kalra via groups.io wrote:
> From: Ashish Kalra <ashish.kalra@amd.com>
> 
> Mark the SEC GHCB page (that is mapped as unencrypted in ResetVector 
> code) in the hypervisor's guest page encryption state tracking.
> 
> Cc: Jordan Justen <jordan.l.justen@intel.com>
> Cc: Ard Biesheuvel <ard.biesheuvel@arm.com>
> Signed-off-by: Ashish Kalra <ashish.kalra@amd.com>
> ---
>  OvmfPkg/PlatformPei/AmdSev.c | 11 +++++++++++
>  1 file changed, 11 insertions(+)
> 
> diff --git a/OvmfPkg/PlatformPei/AmdSev.c 
> b/OvmfPkg/PlatformPei/AmdSev.c index a8bf610022..1d38056ec0 100644
> --- a/OvmfPkg/PlatformPei/AmdSev.c
> +++ b/OvmfPkg/PlatformPei/AmdSev.c
> @@ -52,6 +52,17 @@ AmdSevEsInitialize (
>    PcdStatus = PcdSetBoolS (PcdSevEsIsEnabled, TRUE);
>    ASSERT_RETURN_ERROR (PcdStatus);
>  
> +  //
> +  // The SEC Ghcb setup during reset-vector needs to be marked as  // 
> + decrypted in the hypervisor's guest page encryption state  // 
> + tracking.
> +  //
> +  SetMemoryEncDecHypercall3 (
> +    FixedPcdGet32 (PcdOvmfSecGhcbBase),
> +    EFI_SIZE_TO_PAGES(FixedPcdGet32 (PcdOvmfSecGhcbSize)),
> +    FALSE
> +    );

PcdOvmfSecGhcbSize is set to 2 pages (8192 bytes). AFAIU, only first page needs to be change to shared, second page should be kept private.

>    //
>    // Allocate GHCB and per-CPU variable pages.
>    //   Since the pages must survive across the UEFI to OS transition
> 

Regards
Nikunj

Re: [edk2-devel] [PATCH v7 5/6] OvmfPkg/PlatformPei: Mark SEC GHCB page as unencrypted via hypercall

[Ashish Kalra]

[AMD Official Use Only - General]

Hello Nikunj,

But, yes this needs to be fixed.

Thanks,
Ashish

-----Original Message-----
From: Kalra, Ashish 
Sent: Tuesday, May 31, 2022 12:51 PM
To: Dadhania, Nikunj <NikunjAmritlal.Dadhania@amd.com>; devel@edk2.groups.io
Cc: dovmurik@linux.vnet.ibm.com; brijesh.singh@amd.com; tobin@ibm.com; Lendacky, Thomas <Thomas.Lendacky@amd.com>; jejb@linux.ibm.com; erdemaktas@google.com; jiewen.yao@intel.com; min.m.xu@intel.com; jordan.l.justen@intel.com; ard.biesheuvel@arm.com
Subject: RE: [edk2-devel] [PATCH v7 5/6] OvmfPkg/PlatformPei: Mark SEC GHCB page as unencrypted via hypercall

[AMD Official Use Only - General]

Hello Nikunj,

That is true, but there is no SEV live migration happening before GHCB pages are re-setup and the hypercalls are invoked again for GHCB pages from the guest kernel after it checks the live migration runtime environment flag setup by OVMF.

Thanks,
Ashish

-----Original Message-----
From: Dadhania, Nikunj <NikunjAmritlal.Dadhania@amd.com>
Sent: Tuesday, May 31, 2022 10:14 AM
To: devel@edk2.groups.io; Kalra, Ashish <Ashish.Kalra@amd.com>
Cc: dovmurik@linux.vnet.ibm.com; brijesh.singh@amd.com; tobin@ibm.com; Lendacky, Thomas <Thomas.Lendacky@amd.com>; jejb@linux.ibm.com; erdemaktas@google.com; jiewen.yao@intel.com; min.m.xu@intel.com; jordan.l.justen@intel.com; ard.biesheuvel@arm.com
Subject: Re: [edk2-devel] [PATCH v7 5/6] OvmfPkg/PlatformPei: Mark SEC GHCB page as unencrypted via hypercall

Hi Ashish,

On 8/19/2021 7:36 PM, Ashish Kalra via groups.io wrote:
> From: Ashish Kalra <ashish.kalra@amd.com>
> 
> Mark the SEC GHCB page (that is mapped as unencrypted in ResetVector
> code) in the hypervisor's guest page encryption state tracking.
> 
> Cc: Jordan Justen <jordan.l.justen@intel.com>
> Cc: Ard Biesheuvel <ard.biesheuvel@arm.com>
> Signed-off-by: Ashish Kalra <ashish.kalra@amd.com>
> ---
>  OvmfPkg/PlatformPei/AmdSev.c | 11 +++++++++++
>  1 file changed, 11 insertions(+)
> 
> diff --git a/OvmfPkg/PlatformPei/AmdSev.c 
> b/OvmfPkg/PlatformPei/AmdSev.c index a8bf610022..1d38056ec0 100644
> --- a/OvmfPkg/PlatformPei/AmdSev.c
> +++ b/OvmfPkg/PlatformPei/AmdSev.c
> @@ -52,6 +52,17 @@ AmdSevEsInitialize (
>    PcdStatus = PcdSetBoolS (PcdSevEsIsEnabled, TRUE);
>    ASSERT_RETURN_ERROR (PcdStatus);
>  
> +  //
> +  // The SEC Ghcb setup during reset-vector needs to be marked as  // 
> + decrypted in the hypervisor's guest page encryption state  // 
> + tracking.
> +  //
> +  SetMemoryEncDecHypercall3 (
> +    FixedPcdGet32 (PcdOvmfSecGhcbBase),
> +    EFI_SIZE_TO_PAGES(FixedPcdGet32 (PcdOvmfSecGhcbSize)),
> +    FALSE
> +    );

PcdOvmfSecGhcbSize is set to 2 pages (8192 bytes). AFAIU, only first page needs to be change to shared, second page should be kept private.

>    //
>    // Allocate GHCB and per-CPU variable pages.
>    //   Since the pages must survive across the UEFI to OS transition
> 

Regards
Nikunj