From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mx0b-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) by mx.groups.io with SMTP id smtpd.web08.95.1615916835458792571 for ; Tue, 16 Mar 2021 10:47:16 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=@ibm.com header.s=pp1 header.b=bsvpxCJI; spf=pass (domain: linux.ibm.com, ip: 148.163.158.5, mailfrom: tobin@linux.ibm.com) Received: from pps.filterd (m0098417.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.43/8.16.0.43) with SMTP id 12GHeeAT162601; Tue, 16 Mar 2021 13:47:13 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=subject : to : cc : references : from : message-id : date : mime-version : in-reply-to : content-type : content-transfer-encoding; s=pp1; bh=3RksA3U+KX2u3dIXOLr2/MIKC67gWazO7mjEbQCG+F4=; b=bsvpxCJImG//cBFSFad5ZSO1ZfZxIPMa7Pbm6zDgebvkQ0SIh4e6EU1SatwL5hwD5SFA k4uWOLUjdOBe6c6cjICdz6BCRtb+MnJZAme4RDopCxQZ6VUblRNtQH0enK2Ie85MbCqf /cACXYN6gx92UZISJfTvY6Sxak5L4mlmvliZuL57ArJSGP7X8wDCta3nDc5a1iLIE312 Q2GgWtIA5pDDztPylVFouPd/KUpd5fhxEOFaNLnfD6CPXvjk2iqtra4087fQn6Z4qoAe RlkTJnl3niBps6eFrzPa0ImSkYop/yc66GqY4/QAlq1vw0VRchlmiKSwfwJjXSYiM2oe aA== Received: from ppma04dal.us.ibm.com (7a.29.35a9.ip4.static.sl-reverse.com [169.53.41.122]) by mx0a-001b2d01.pphosted.com with ESMTP id 37b0krhk18-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 16 Mar 2021 13:47:12 -0400 Received: from pps.filterd (ppma04dal.us.ibm.com [127.0.0.1]) by ppma04dal.us.ibm.com (8.16.0.43/8.16.0.43) with SMTP id 12GHl68P030614; Tue, 16 Mar 2021 17:47:11 GMT Received: from b01cxnp23033.gho.pok.ibm.com (b01cxnp23033.gho.pok.ibm.com [9.57.198.28]) by ppma04dal.us.ibm.com with ESMTP id 37a3gce7x0-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 16 Mar 2021 17:47:11 +0000 Received: from b01ledav006.gho.pok.ibm.com (b01ledav006.gho.pok.ibm.com [9.57.199.111]) by b01cxnp23033.gho.pok.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 12GHlAeY22413576 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 16 Mar 2021 17:47:10 GMT Received: from b01ledav006.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id A563AAC059; Tue, 16 Mar 2021 17:47:10 +0000 (GMT) Received: from b01ledav006.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id CB4BBAC05B; Tue, 16 Mar 2021 17:47:09 +0000 (GMT) Received: from Tobins-MacBook-Pro-2.local (unknown [9.85.143.70]) by b01ledav006.gho.pok.ibm.com (Postfix) with ESMTP; Tue, 16 Mar 2021 17:47:09 +0000 (GMT) Subject: Re: [edk2-devel] [RFC PATCH 00/14] Firmware Support for Fast Live Migration for AMD SEV To: "Yao, Jiewen" , "devel@edk2.groups.io" Cc: Dov Murik , Tobin Feldman-Fitzthum , James Bottomley , Hubertus Franke , Brijesh Singh , Ashish Kalra , Jon Grimm , Tom Lendacky References: <20210302204839.82042-1-tobin@linux.ibm.com> <166900903D364B89.9163@groups.io> From: "Tobin Feldman-Fitzthum" Message-ID: Date: Tue, 16 Mar 2021 13:47:09 -0400 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:78.0) Gecko/20100101 Thunderbird/78.7.0 MIME-Version: 1.0 In-Reply-To: X-TM-AS-GCONF: 00 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.369,18.0.761 definitions=2021-03-16_06:2021-03-16,2021-03-16 signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 mlxscore=0 malwarescore=0 lowpriorityscore=0 bulkscore=0 adultscore=0 phishscore=0 mlxlogscore=999 suspectscore=0 clxscore=1011 impostorscore=0 spamscore=0 priorityscore=1501 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2009150000 definitions=main-2103160111 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit Content-Language: en-US On 3/12/21 9:32 PM, Yao, Jiewen wrote: > Hi > We discuss the patch internally. We do see PROs and CONs with this approach. > The advantage is that it is very simple. In-VM migration can save lots of effort on security context restore. > On the other hand, we feel not so comfortable to reserve a dedicate CPU to achieve that. Similar to the feedback in the community. > > Using Hot-Plug is not a solution for Intel TDX as well. It is unsupported now. > > I like the idea to diverge the migration boot mode v.s. normal boot mode in SEC phase. > We must be very carefully handle this migration boot mode, to avoid any touching on system memory. > Intel TDX Virtual Firmware skips the PEI phase directly. If we choose this approach, SEC-based migration is our preference. > > Besides this patch, we would like to understand a full picture. > 1) How the key is passed from source VM to destination? > I saw you mentions: "Key sharing is out of scope for this part of the RFC." > "This will probably be implemented via inject-launch-secret in the future" > > Does that mean two PSP will sync with each other and negotiate the key, after the Migration Agent (MA) checks the policy? The source and destination migration handlers will need to share a key. If we only relied on the PSP for migration, we could use the existing secure channel between the PSP and the guest owner to transfer the pages. Unfortunately the throughput of this approach is far too low. Thus, we have some migration handler running on a guest vCPU with a transport key shared between the source and the target. The main mechanism for getting a key to the migration handler is inject-launch-secret. Here the guest owner can provide a secret to the PSP via a secure channel and the PSP will inject it at some guest physical address. You use inject-launch-secret after the launch measurement of the guest has been generated to inject the secret conditionally. One approach would be to inject the transport key directly in the source and the target. This is pretty simple, but might have a few drawbacks. The injection has to happen at boot, meaning that the source machine would have to be provisioned with a transport key before a migration happens and that all migrations from that machine would have to use the same transport key. One way around this would be to inject asymmetric keys and use them to derive the transport key. Another approach entirely is to use the PSP to migrate just a few pages, which might include a secret set by the source MH that the target MH could use to decrypt incoming pages. Using the PSP to migrate pages requires some extra kernel support. For the RFC, we just assume that there is some shared key. We have talked some about the various options internally. > 2) How the attestation is supported? > I read the whitepaper https://www.amd.com/system/files/TechDocs/SEV-SNP-strengthening-vm-isolation-with-integrity-protection-and-more.pdf. > It seems SEV and SEV-ES only support attestation during launch, I don't believe this migration feature will impact the attestation report. Am I right? > SEV-SNP supports more flexible attestation, does it include any information about the new migrated content? Brijesh already addressed most of this. In our approach the MH is baked into the firmware, which can be attested prior to injecting the key. In other words there aren't any additional steps to attest the MH and it does not change the functionality of any existing attestation mechanisms. -Tobin > >> -----Original Message----- >> From: devel@edk2.groups.io On Behalf Of Yao, Jiewen >> Sent: Thursday, March 4, 2021 9:49 AM >> To: devel@edk2.groups.io; tobin@linux.ibm.com >> Cc: Dov Murik ; Tobin Feldman-Fitzthum >> ; James Bottomley ; Hubertus Franke >> ; Brijesh Singh ; Ashish Kalra >> ; Jon Grimm ; Tom Lendacky >> ; Yao, Jiewen >> Subject: Re: [edk2-devel] [RFC PATCH 00/14] Firmware Support for Fast Live >> Migration for AMD SEV >> >> Hi Tobin >> Thanks for your patch. >> You may that Intel is working on TDX for the same live migration feature. >> >> Please give me some time (about 1 work week) to digest and evaluate the patch >> and impact. >> Then I will provide feedback. >> >> Thank you >> Yao Jiewen >> >>> -----Original Message----- >>> From: devel@edk2.groups.io On Behalf Of Tobin >>> Feldman-Fitzthum >>> Sent: Wednesday, March 3, 2021 4:48 AM >>> To: devel@edk2.groups.io >>> Cc: Dov Murik ; Tobin Feldman-Fitzthum >>> ; Tobin Feldman-Fitzthum ; James >>> Bottomley ; Hubertus Franke ; >>> Brijesh Singh ; Ashish Kalra >> ; >>> Jon Grimm ; Tom Lendacky >>> >>> Subject: [edk2-devel] [RFC PATCH 00/14] Firmware Support for Fast Live >>> Migration for AMD SEV >>> >>> This is a demonstration of fast migration for encrypted virtual machines >>> using a Migration Handler that lives in OVMF. This demo uses AMD SEV, >>> but the ideas may generalize to other confidential computing platforms. >>> With AMD SEV, guest memory is encrypted and the hypervisor cannot access >>> or move it. This makes migration tricky. In this demo, we show how the >>> HV can ask a Migration Handler (MH) in the firmware for an encrypted >>> page. The MH encrypts the page with a transport key prior to releasing >>> it to the HV. The target machine also runs an MH that decrypts the page >>> once it is passed in by the target HV. These patches are not ready for >>> production, but the are a full end-to-end solution that facilitates a >>> fast live migration between two SEV VMs. >>> >>> Corresponding patches for QEMU have been posted my colleague Dov Murik >>> on qemu-devel. Our approach needs little kernel support, requiring only >>> one hypercall that the guest can use to mark a page as encrypted or >>> shared. This series includes updated patches from Ashish Kalra and >>> Brijesh Singh that allow OVMF to use this hypercall. >>> >>> The MH runs continuously in the guest, waiting for communication from >>> the HV. The HV starts an additional vCPU for the MH but does not expose >>> it to the guest OS via ACPI. We use the MpService to start the MH. The >>> MpService is only available at runtime and processes that are started by >>> it are usually cleaned up on ExitBootServices. Since we need the MH to >>> run continuously, we had to make some modifications. Ideally a feature >>> could be added to the MpService to allow for the starting of >>> long-running processes. Besides migration, this could support other >>> background processes that need to operate within the encryption >>> boundary. For now, we have included a handful of patches that modify the >>> MpService to allow the MH to keep running after ExitBootServices. These >>> are temporary. >>> >>> Ashish Kalra (2): >>> OvmfPkg/PlatformPei: Mark SEC GHCB page in the page encrpytion bitmap. >>> OvmfPkg/PlatformDxe: Add support for SEV live migration. >>> >>> Brijesh Singh (1): >>> OvmfPkg/BaseMemEncryptLib: Support to issue unencrypted hypercall >>> >>> Dov Murik (1): >>> OvmfPkg/AmdSev: Build page table for migration handler >>> >>> Tobin Feldman-Fitzthum (10): >>> OvmfPkg/AmdSev: Base for Confidential Migration Handler >>> OvmfPkg/PlatfomPei: Set Confidential Migration PCD >>> OvmfPkg/AmdSev: Setup Migration Handler Mailbox >>> OvmfPkg/AmdSev: MH support for mailbox protocol >>> UefiCpuPkg/MpInitLib: temp removal of MpLib cleanup >>> UefiCpuPkg/MpInitLib: Allocate MP buffer as runtime memory >>> UefiCpuPkg/CpuExceptionHandlerLib: Exception handling as runtime >>> memory >>> OvmfPkg/AmdSev: Don't overwrite mailbox or pagetables >>> OvmfPkg/AmdSev: Don't overwrite MH stack >>> OvmfPkg/AmdSev: MH page encryption POC >>> >>> OvmfPkg/OvmfPkg.dec | 11 + >>> OvmfPkg/AmdSev/AmdSevX64.dsc | 2 + >>> OvmfPkg/AmdSev/AmdSevX64.fdf | 13 +- >>> .../ConfidentialMigrationDxe.inf | 45 +++ >>> .../ConfidentialMigrationPei.inf | 35 ++ >>> .../DxeMemEncryptSevLib.inf | 1 + >>> .../PeiMemEncryptSevLib.inf | 1 + >>> OvmfPkg/PlatformDxe/Platform.inf | 2 + >>> OvmfPkg/PlatformPei/PlatformPei.inf | 2 + >>> UefiCpuPkg/Library/MpInitLib/DxeMpInitLib.inf | 2 + >>> UefiCpuPkg/Library/MpInitLib/PeiMpInitLib.inf | 2 + >>> OvmfPkg/AmdSev/ConfidentialMigration/MpLib.h | 235 +++++++++++++ >>> .../ConfidentialMigration/VirtualMemory.h | 177 ++++++++++ >>> OvmfPkg/Include/Guid/MemEncryptLib.h | 16 + >>> OvmfPkg/PlatformDxe/PlatformConfig.h | 5 + >>> .../ConfidentialMigrationDxe.c | 325 ++++++++++++++++++ >>> .../ConfidentialMigrationPei.c | 25 ++ >>> .../X64/PeiDxeVirtualMemory.c | 18 + >>> OvmfPkg/PlatformDxe/AmdSev.c | 99 ++++++ >>> OvmfPkg/PlatformDxe/Platform.c | 6 + >>> OvmfPkg/PlatformPei/AmdSev.c | 10 + >>> OvmfPkg/PlatformPei/Platform.c | 10 + >>> .../CpuExceptionHandlerLib/DxeException.c | 8 +- >>> UefiCpuPkg/Library/MpInitLib/DxeMpLib.c | 21 +- >>> UefiCpuPkg/Library/MpInitLib/MpLib.c | 7 +- >>> 25 files changed, 1061 insertions(+), 17 deletions(-) >>> create mode 100644 >>> OvmfPkg/AmdSev/ConfidentialMigration/ConfidentialMigrationDxe.inf >>> create mode 100644 >>> OvmfPkg/AmdSev/ConfidentialMigration/ConfidentialMigrationPei.inf >>> create mode 100644 OvmfPkg/AmdSev/ConfidentialMigration/MpLib.h >>> create mode 100644 >>> OvmfPkg/AmdSev/ConfidentialMigration/VirtualMemory.h >>> create mode 100644 OvmfPkg/Include/Guid/MemEncryptLib.h >>> create mode 100644 >>> OvmfPkg/AmdSev/ConfidentialMigration/ConfidentialMigrationDxe.c >>> create mode 100644 >>> OvmfPkg/AmdSev/ConfidentialMigration/ConfidentialMigrationPei.c >>> create mode 100644 OvmfPkg/PlatformDxe/AmdSev.c >>> >>> -- >>> 2.20.1 >>> >>> >>> >>> >>> >> >> >> >>