Re: [edk2-devel] [RFC PATCH 00/14] Firmware Support for Fast Live Migration for AMD SEV

["Yao, Jiewen" <jiewen.yao@intel.com>]

Hi
We discuss the patch internally. We do see PROs and CONs with this approach.
The advantage is that it is very simple. In-VM migration can save lots of effort on security context restore.
On the other hand, we feel not so comfortable to reserve a dedicate CPU to achieve that. Similar to the feedback in the community.

Using Hot-Plug is not a solution for Intel TDX as well. It is unsupported now.

I like the idea to diverge the migration boot mode v.s. normal boot mode in SEC phase.
We must be very carefully handle this migration boot mode, to avoid any touching on system memory.
Intel TDX Virtual Firmware skips the PEI phase directly. If we choose this approach, SEC-based migration is our preference.

Besides this patch, we would like to understand a full picture.
1) How the key is passed from source VM to destination?
I saw you mentions: "Key sharing is out of scope for this part of the RFC."
"This will probably be implemented via inject-launch-secret in the future"

Does that mean two PSP will sync with each other and negotiate the key, after the Migration Agent (MA) checks the policy?

2) How the attestation is supported?
I read the whitepaper https://www.amd.com/system/files/TechDocs/SEV-SNP-strengthening-vm-isolation-with-integrity-protection-and-more.pdf.
It seems SEV and SEV-ES only support attestation during launch, I don't believe this migration feature will impact the attestation report. Am I right?
SEV-SNP supports more flexible attestation, does it include any information about the new migrated content?


> -----Original Message-----
> From: devel@edk2.groups.io <devel@edk2.groups.io> On Behalf Of Yao, Jiewen
> Sent: Thursday, March 4, 2021 9:49 AM
> To: devel@edk2.groups.io; tobin@linux.ibm.com
> Cc: Dov Murik <dovmurik@linux.vnet.ibm.com>; Tobin Feldman-Fitzthum
> <tobin@ibm.com>; James Bottomley <jejb@linux.ibm.com>; Hubertus Franke
> <frankeh@us.ibm.com>; Brijesh Singh <brijesh.singh@amd.com>; Ashish Kalra
> <ashish.kalra@amd.com>; Jon Grimm <jon.grimm@amd.com>; Tom Lendacky
> <thomas.lendacky@amd.com>; Yao, Jiewen <jiewen.yao@intel.com>
> Subject: Re: [edk2-devel] [RFC PATCH 00/14] Firmware Support for Fast Live
> Migration for AMD SEV
> 
> Hi Tobin
> Thanks for your patch.
> You may that Intel is working on TDX for the same live migration feature.
> 
> Please give me some time (about 1 work week) to digest and evaluate the patch
> and impact.
> Then I will provide feedback.
> 
> Thank you
> Yao Jiewen
> 
> > -----Original Message-----
> > From: devel@edk2.groups.io <devel@edk2.groups.io> On Behalf Of Tobin
> > Feldman-Fitzthum
> > Sent: Wednesday, March 3, 2021 4:48 AM
> > To: devel@edk2.groups.io
> > Cc: Dov Murik <dovmurik@linux.vnet.ibm.com>; Tobin Feldman-Fitzthum
> > <tobin@ibm.com>; Tobin Feldman-Fitzthum <tobin@linux.ibm.com>; James
> > Bottomley <jejb@linux.ibm.com>; Hubertus Franke <frankeh@us.ibm.com>;
> > Brijesh Singh <brijesh.singh@amd.com>; Ashish Kalra
> <ashish.kalra@amd.com>;
> > Jon Grimm <jon.grimm@amd.com>; Tom Lendacky
> > <thomas.lendacky@amd.com>
> > Subject: [edk2-devel] [RFC PATCH 00/14] Firmware Support for Fast Live
> > Migration for AMD SEV
> >
> > This is a demonstration of fast migration for encrypted virtual machines
> > using a Migration Handler that lives in OVMF. This demo uses AMD SEV,
> > but the ideas may generalize to other confidential computing platforms.
> > With AMD SEV, guest memory is encrypted and the hypervisor cannot access
> > or move it. This makes migration tricky. In this demo, we show how the
> > HV can ask a Migration Handler (MH) in the firmware for an encrypted
> > page. The MH encrypts the page with a transport key prior to releasing
> > it to the HV. The target machine also runs an MH that decrypts the page
> > once it is passed in by the target HV. These patches are not ready for
> > production, but the are a full end-to-end solution that facilitates a
> > fast live migration between two SEV VMs.
> >
> > Corresponding patches for QEMU have been posted my colleague Dov Murik
> > on qemu-devel. Our approach needs little kernel support, requiring only
> > one hypercall that the guest can use to mark a page as encrypted or
> > shared. This series includes updated patches from Ashish Kalra and
> > Brijesh Singh that allow OVMF to use this hypercall.
> >
> > The MH runs continuously in the guest, waiting for communication from
> > the HV. The HV starts an additional vCPU for the MH but does not expose
> > it to the guest OS via ACPI. We use the MpService to start the MH. The
> > MpService is only available at runtime and processes that are started by
> > it are usually cleaned up on ExitBootServices. Since we need the MH to
> > run continuously, we had to make some modifications. Ideally a feature
> > could be added to the MpService to allow for the starting of
> > long-running processes. Besides migration, this could support other
> > background processes that need to operate within the encryption
> > boundary. For now, we have included a handful of patches that modify the
> > MpService to allow the MH to keep running after ExitBootServices. These
> > are temporary.
> >
> > Ashish Kalra (2):
> >   OvmfPkg/PlatformPei: Mark SEC GHCB page in the page encrpytion bitmap.
> >   OvmfPkg/PlatformDxe: Add support for SEV live migration.
> >
> > Brijesh Singh (1):
> >   OvmfPkg/BaseMemEncryptLib: Support to issue unencrypted hypercall
> >
> > Dov Murik (1):
> >   OvmfPkg/AmdSev: Build page table for migration handler
> >
> > Tobin Feldman-Fitzthum (10):
> >   OvmfPkg/AmdSev: Base for Confidential Migration Handler
> >   OvmfPkg/PlatfomPei: Set Confidential Migration PCD
> >   OvmfPkg/AmdSev: Setup Migration Handler Mailbox
> >   OvmfPkg/AmdSev: MH support for mailbox protocol
> >   UefiCpuPkg/MpInitLib: temp removal of MpLib cleanup
> >   UefiCpuPkg/MpInitLib: Allocate MP buffer as runtime memory
> >   UefiCpuPkg/CpuExceptionHandlerLib: Exception handling as runtime
> >     memory
> >   OvmfPkg/AmdSev: Don't overwrite mailbox or pagetables
> >   OvmfPkg/AmdSev: Don't overwrite MH stack
> >   OvmfPkg/AmdSev: MH page encryption POC
> >
> >  OvmfPkg/OvmfPkg.dec                           |  11 +
> >  OvmfPkg/AmdSev/AmdSevX64.dsc                  |   2 +
> >  OvmfPkg/AmdSev/AmdSevX64.fdf                  |  13 +-
> >  .../ConfidentialMigrationDxe.inf              |  45 +++
> >  .../ConfidentialMigrationPei.inf              |  35 ++
> >  .../DxeMemEncryptSevLib.inf                   |   1 +
> >  .../PeiMemEncryptSevLib.inf                   |   1 +
> >  OvmfPkg/PlatformDxe/Platform.inf              |   2 +
> >  OvmfPkg/PlatformPei/PlatformPei.inf           |   2 +
> >  UefiCpuPkg/Library/MpInitLib/DxeMpInitLib.inf |   2 +
> >  UefiCpuPkg/Library/MpInitLib/PeiMpInitLib.inf |   2 +
> >  OvmfPkg/AmdSev/ConfidentialMigration/MpLib.h  | 235 +++++++++++++
> >  .../ConfidentialMigration/VirtualMemory.h     | 177 ++++++++++
> >  OvmfPkg/Include/Guid/MemEncryptLib.h          |  16 +
> >  OvmfPkg/PlatformDxe/PlatformConfig.h          |   5 +
> >  .../ConfidentialMigrationDxe.c                | 325 ++++++++++++++++++
> >  .../ConfidentialMigrationPei.c                |  25 ++
> >  .../X64/PeiDxeVirtualMemory.c                 |  18 +
> >  OvmfPkg/PlatformDxe/AmdSev.c                  |  99 ++++++
> >  OvmfPkg/PlatformDxe/Platform.c                |   6 +
> >  OvmfPkg/PlatformPei/AmdSev.c                  |  10 +
> >  OvmfPkg/PlatformPei/Platform.c                |  10 +
> >  .../CpuExceptionHandlerLib/DxeException.c     |   8 +-
> >  UefiCpuPkg/Library/MpInitLib/DxeMpLib.c       |  21 +-
> >  UefiCpuPkg/Library/MpInitLib/MpLib.c          |   7 +-
> >  25 files changed, 1061 insertions(+), 17 deletions(-)
> >  create mode 100644
> > OvmfPkg/AmdSev/ConfidentialMigration/ConfidentialMigrationDxe.inf
> >  create mode 100644
> > OvmfPkg/AmdSev/ConfidentialMigration/ConfidentialMigrationPei.inf
> >  create mode 100644 OvmfPkg/AmdSev/ConfidentialMigration/MpLib.h
> >  create mode 100644
> > OvmfPkg/AmdSev/ConfidentialMigration/VirtualMemory.h
> >  create mode 100644 OvmfPkg/Include/Guid/MemEncryptLib.h
> >  create mode 100644
> > OvmfPkg/AmdSev/ConfidentialMigration/ConfidentialMigrationDxe.c
> >  create mode 100644
> > OvmfPkg/AmdSev/ConfidentialMigration/ConfidentialMigrationPei.c
> >  create mode 100644 OvmfPkg/PlatformDxe/AmdSev.c
> >
> > --
> > 2.20.1
> >
> >
> >
> >
> >
> 
> 
> 
> 
>