Re: [edk2-devel] [RFC PATCH 00/14] Firmware Support for Fast Live Migration for AMD SEV

["Yao, Jiewen" <jiewen.yao@intel.com>]

Thank you very much Tobin and Brijesh.

Yes, I agree that there are multiple ways to pass the transport key from source to destination.
I will wait for your final solution.


> -----Original Message-----
> From: devel@edk2.groups.io <devel@edk2.groups.io> On Behalf Of Tobin
> Feldman-Fitzthum
> Sent: Wednesday, March 17, 2021 1:47 AM
> To: Yao, Jiewen <jiewen.yao@intel.com>; devel@edk2.groups.io
> Cc: Dov Murik <dovmurik@linux.vnet.ibm.com>; Tobin Feldman-Fitzthum
> <tobin@ibm.com>; James Bottomley <jejb@linux.ibm.com>; Hubertus Franke
> <frankeh@us.ibm.com>; Brijesh Singh <brijesh.singh@amd.com>; Ashish Kalra
> <ashish.kalra@amd.com>; Jon Grimm <jon.grimm@amd.com>; Tom Lendacky
> <thomas.lendacky@amd.com>
> Subject: Re: [edk2-devel] [RFC PATCH 00/14] Firmware Support for Fast Live
> Migration for AMD SEV
> 
> On 3/12/21 9:32 PM, Yao, Jiewen wrote:
> 
> > Hi
> > We discuss the patch internally. We do see PROs and CONs with this approach.
> > The advantage is that it is very simple. In-VM migration can save lots of effort
> on security context restore.
> > On the other hand, we feel not so comfortable to reserve a dedicate CPU to
> achieve that. Similar to the feedback in the community.
> >
> > Using Hot-Plug is not a solution for Intel TDX as well. It is unsupported now.
> >
> > I like the idea to diverge the migration boot mode v.s. normal boot mode in
> SEC phase.
> > We must be very carefully handle this migration boot mode, to avoid any
> touching on system memory.
> > Intel TDX Virtual Firmware skips the PEI phase directly. If we choose this
> approach, SEC-based migration is our preference.
> >
> > Besides this patch, we would like to understand a full picture.
> > 1) How the key is passed from source VM to destination?
> > I saw you mentions: "Key sharing is out of scope for this part of the RFC."
> > "This will probably be implemented via inject-launch-secret in the future"
> >
> > Does that mean two PSP will sync with each other and negotiate the key, after
> the Migration Agent (MA) checks the policy?
> 
> The source and destination migration handlers will need to share a key.
> If we only relied on the PSP for migration, we could use the existing
> secure channel between the PSP and the guest owner to transfer the
> pages. Unfortunately the throughput of this approach is far too low.
> Thus, we have some migration handler running on a guest vCPU with a
> transport key shared between the source and the target.
> 
> The main mechanism for getting a key to the migration handler is
> inject-launch-secret. Here the guest owner can provide a secret to the
> PSP via a secure channel and the PSP will inject it at some guest
> physical address. You use inject-launch-secret after the launch
> measurement of the guest has been generated to inject the secret
> conditionally. One approach would be to inject the transport key
> directly in the source and the target. This is pretty simple, but might
> have a few drawbacks. The injection has to happen at boot, meaning that
> the source machine would have to be provisioned with a transport key
> before a migration happens and that all migrations from that machine
> would have to use the same transport key. One way around this would be
> to inject asymmetric keys and use them to derive the transport key.
> 
> Another approach entirely is to use the PSP to migrate just a few pages,
> which might include a secret set by the source MH that the target MH
> could use to decrypt incoming pages. Using the PSP to migrate pages
> requires some extra kernel support.
> 
> For the RFC, we just assume that there is some shared key. We have
> talked some about the various options internally.
> 
> > 2) How the attestation is supported?
> > I read the whitepaper https://www.amd.com/system/files/TechDocs/SEV-
> SNP-strengthening-vm-isolation-with-integrity-protection-and-more.pdf.
> > It seems SEV and SEV-ES only support attestation during launch, I don't believe
> this migration feature will impact the attestation report. Am I right?
> > SEV-SNP supports more flexible attestation, does it include any information
> about the new migrated content?
> 
> Brijesh already addressed most of this. In our approach the MH is baked
> into the firmware, which can be attested prior to injecting the key. In
> other words there aren't any additional steps to attest the MH and it
> does not change the functionality of any existing attestation mechanisms.
> 
> -Tobin
> 
> >
> >> -----Original Message-----
> >> From: devel@edk2.groups.io <devel@edk2.groups.io> On Behalf Of Yao,
> Jiewen
> >> Sent: Thursday, March 4, 2021 9:49 AM
> >> To: devel@edk2.groups.io; tobin@linux.ibm.com
> >> Cc: Dov Murik <dovmurik@linux.vnet.ibm.com>; Tobin Feldman-Fitzthum
> >> <tobin@ibm.com>; James Bottomley <jejb@linux.ibm.com>; Hubertus Franke
> >> <frankeh@us.ibm.com>; Brijesh Singh <brijesh.singh@amd.com>; Ashish
> Kalra
> >> <ashish.kalra@amd.com>; Jon Grimm <jon.grimm@amd.com>; Tom
> Lendacky
> >> <thomas.lendacky@amd.com>; Yao, Jiewen <jiewen.yao@intel.com>
> >> Subject: Re: [edk2-devel] [RFC PATCH 00/14] Firmware Support for Fast Live
> >> Migration for AMD SEV
> >>
> >> Hi Tobin
> >> Thanks for your patch.
> >> You may that Intel is working on TDX for the same live migration feature.
> >>
> >> Please give me some time (about 1 work week) to digest and evaluate the
> patch
> >> and impact.
> >> Then I will provide feedback.
> >>
> >> Thank you
> >> Yao Jiewen
> >>
> >>> -----Original Message-----
> >>> From: devel@edk2.groups.io <devel@edk2.groups.io> On Behalf Of Tobin
> >>> Feldman-Fitzthum
> >>> Sent: Wednesday, March 3, 2021 4:48 AM
> >>> To: devel@edk2.groups.io
> >>> Cc: Dov Murik <dovmurik@linux.vnet.ibm.com>; Tobin Feldman-Fitzthum
> >>> <tobin@ibm.com>; Tobin Feldman-Fitzthum <tobin@linux.ibm.com>; James
> >>> Bottomley <jejb@linux.ibm.com>; Hubertus Franke <frankeh@us.ibm.com>;
> >>> Brijesh Singh <brijesh.singh@amd.com>; Ashish Kalra
> >> <ashish.kalra@amd.com>;
> >>> Jon Grimm <jon.grimm@amd.com>; Tom Lendacky
> >>> <thomas.lendacky@amd.com>
> >>> Subject: [edk2-devel] [RFC PATCH 00/14] Firmware Support for Fast Live
> >>> Migration for AMD SEV
> >>>
> >>> This is a demonstration of fast migration for encrypted virtual machines
> >>> using a Migration Handler that lives in OVMF. This demo uses AMD SEV,
> >>> but the ideas may generalize to other confidential computing platforms.
> >>> With AMD SEV, guest memory is encrypted and the hypervisor cannot
> access
> >>> or move it. This makes migration tricky. In this demo, we show how the
> >>> HV can ask a Migration Handler (MH) in the firmware for an encrypted
> >>> page. The MH encrypts the page with a transport key prior to releasing
> >>> it to the HV. The target machine also runs an MH that decrypts the page
> >>> once it is passed in by the target HV. These patches are not ready for
> >>> production, but the are a full end-to-end solution that facilitates a
> >>> fast live migration between two SEV VMs.
> >>>
> >>> Corresponding patches for QEMU have been posted my colleague Dov
> Murik
> >>> on qemu-devel. Our approach needs little kernel support, requiring only
> >>> one hypercall that the guest can use to mark a page as encrypted or
> >>> shared. This series includes updated patches from Ashish Kalra and
> >>> Brijesh Singh that allow OVMF to use this hypercall.
> >>>
> >>> The MH runs continuously in the guest, waiting for communication from
> >>> the HV. The HV starts an additional vCPU for the MH but does not expose
> >>> it to the guest OS via ACPI. We use the MpService to start the MH. The
> >>> MpService is only available at runtime and processes that are started by
> >>> it are usually cleaned up on ExitBootServices. Since we need the MH to
> >>> run continuously, we had to make some modifications. Ideally a feature
> >>> could be added to the MpService to allow for the starting of
> >>> long-running processes. Besides migration, this could support other
> >>> background processes that need to operate within the encryption
> >>> boundary. For now, we have included a handful of patches that modify the
> >>> MpService to allow the MH to keep running after ExitBootServices. These
> >>> are temporary.
> >>>
> >>> Ashish Kalra (2):
> >>>    OvmfPkg/PlatformPei: Mark SEC GHCB page in the page encrpytion bitmap.
> >>>    OvmfPkg/PlatformDxe: Add support for SEV live migration.
> >>>
> >>> Brijesh Singh (1):
> >>>    OvmfPkg/BaseMemEncryptLib: Support to issue unencrypted hypercall
> >>>
> >>> Dov Murik (1):
> >>>    OvmfPkg/AmdSev: Build page table for migration handler
> >>>
> >>> Tobin Feldman-Fitzthum (10):
> >>>    OvmfPkg/AmdSev: Base for Confidential Migration Handler
> >>>    OvmfPkg/PlatfomPei: Set Confidential Migration PCD
> >>>    OvmfPkg/AmdSev: Setup Migration Handler Mailbox
> >>>    OvmfPkg/AmdSev: MH support for mailbox protocol
> >>>    UefiCpuPkg/MpInitLib: temp removal of MpLib cleanup
> >>>    UefiCpuPkg/MpInitLib: Allocate MP buffer as runtime memory
> >>>    UefiCpuPkg/CpuExceptionHandlerLib: Exception handling as runtime
> >>>      memory
> >>>    OvmfPkg/AmdSev: Don't overwrite mailbox or pagetables
> >>>    OvmfPkg/AmdSev: Don't overwrite MH stack
> >>>    OvmfPkg/AmdSev: MH page encryption POC
> >>>
> >>>   OvmfPkg/OvmfPkg.dec                           |  11 +
> >>>   OvmfPkg/AmdSev/AmdSevX64.dsc                  |   2 +
> >>>   OvmfPkg/AmdSev/AmdSevX64.fdf                  |  13 +-
> >>>   .../ConfidentialMigrationDxe.inf              |  45 +++
> >>>   .../ConfidentialMigrationPei.inf              |  35 ++
> >>>   .../DxeMemEncryptSevLib.inf                   |   1 +
> >>>   .../PeiMemEncryptSevLib.inf                   |   1 +
> >>>   OvmfPkg/PlatformDxe/Platform.inf              |   2 +
> >>>   OvmfPkg/PlatformPei/PlatformPei.inf           |   2 +
> >>>   UefiCpuPkg/Library/MpInitLib/DxeMpInitLib.inf |   2 +
> >>>   UefiCpuPkg/Library/MpInitLib/PeiMpInitLib.inf |   2 +
> >>>   OvmfPkg/AmdSev/ConfidentialMigration/MpLib.h  | 235 +++++++++++++
> >>>   .../ConfidentialMigration/VirtualMemory.h     | 177 ++++++++++
> >>>   OvmfPkg/Include/Guid/MemEncryptLib.h          |  16 +
> >>>   OvmfPkg/PlatformDxe/PlatformConfig.h          |   5 +
> >>>   .../ConfidentialMigrationDxe.c                | 325 ++++++++++++++++++
> >>>   .../ConfidentialMigrationPei.c                |  25 ++
> >>>   .../X64/PeiDxeVirtualMemory.c                 |  18 +
> >>>   OvmfPkg/PlatformDxe/AmdSev.c                  |  99 ++++++
> >>>   OvmfPkg/PlatformDxe/Platform.c                |   6 +
> >>>   OvmfPkg/PlatformPei/AmdSev.c                  |  10 +
> >>>   OvmfPkg/PlatformPei/Platform.c                |  10 +
> >>>   .../CpuExceptionHandlerLib/DxeException.c     |   8 +-
> >>>   UefiCpuPkg/Library/MpInitLib/DxeMpLib.c       |  21 +-
> >>>   UefiCpuPkg/Library/MpInitLib/MpLib.c          |   7 +-
> >>>   25 files changed, 1061 insertions(+), 17 deletions(-)
> >>>   create mode 100644
> >>> OvmfPkg/AmdSev/ConfidentialMigration/ConfidentialMigrationDxe.inf
> >>>   create mode 100644
> >>> OvmfPkg/AmdSev/ConfidentialMigration/ConfidentialMigrationPei.inf
> >>>   create mode 100644 OvmfPkg/AmdSev/ConfidentialMigration/MpLib.h
> >>>   create mode 100644
> >>> OvmfPkg/AmdSev/ConfidentialMigration/VirtualMemory.h
> >>>   create mode 100644 OvmfPkg/Include/Guid/MemEncryptLib.h
> >>>   create mode 100644
> >>> OvmfPkg/AmdSev/ConfidentialMigration/ConfidentialMigrationDxe.c
> >>>   create mode 100644
> >>> OvmfPkg/AmdSev/ConfidentialMigration/ConfidentialMigrationPei.c
> >>>   create mode 100644 OvmfPkg/PlatformDxe/AmdSev.c
> >>>
> >>> --
> >>> 2.20.1
> >>>
> >>>
> >>>
> >>>
> >>>
> >>
> >>
> >>
> >>
> 
> 
> 
>