From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.158.5]) by mx.groups.io with SMTP id smtpd.web09.442.1614718133586267162 for ; Tue, 02 Mar 2021 12:48:53 -0800 Authentication-Results: mx.groups.io; dkim=pass header.i=@ibm.com header.s=pp1 header.b=dWtYGAIT; spf=pass (domain: linux.ibm.com, ip: 148.163.158.5, mailfrom: tobin@linux.ibm.com) Received: from pps.filterd (m0098416.ppops.net [127.0.0.1]) by mx0b-001b2d01.pphosted.com (8.16.0.43/8.16.0.43) with SMTP id 122KhIwC191181; Tue, 2 Mar 2021 15:48:51 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=from : to : cc : subject : date : message-id : mime-version : content-transfer-encoding; s=pp1; bh=A7pADg0o8nw6ad/T3+x/eVMck5wm90xogUUqpjGE0dI=; b=dWtYGAIT+v6aSjfHGbCJuUwIWOvu+mPI6ybiJ/r0Xusagm/baL6msg6cmvQ9yzF+XoMt Vy01bYJ9wD63pGpxEBBVA4qQ+HvJ7wuZO0ClkvnJzOtSMml1B4MIXKxji+RuiYvQqR7J fnGcvmFHqFHAULeMq6del2nnK2wsXrIsOm4aanA7feFC7gq79pt0coDqkR6JY+hFQwPV 4nis9LWznqlXgIMquZFDvG96lMVTEuBi3mtA/u0N3H9zkqDQmG/QNpg3PHfJMGaxlPBW FJJQWFk+g/EF+fdxxFIq5nteeoy6yY+MOFHgS6Hp9O6b1AlaYAphatayUVtWg/fylnvl Cw== Received: from ppma01dal.us.ibm.com (83.d6.3fa9.ip4.static.sl-reverse.com [169.63.214.131]) by mx0b-001b2d01.pphosted.com with ESMTP id 371vn7r85b-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 02 Mar 2021 15:48:50 -0500 Received: from pps.filterd (ppma01dal.us.ibm.com [127.0.0.1]) by ppma01dal.us.ibm.com (8.16.0.42/8.16.0.42) with SMTP id 122KkqMO028086; Tue, 2 Mar 2021 20:48:49 GMT Received: from b03cxnp07029.gho.boulder.ibm.com (b03cxnp07029.gho.boulder.ibm.com [9.17.130.16]) by ppma01dal.us.ibm.com with ESMTP id 371qmuagwf-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 02 Mar 2021 20:48:49 +0000 Received: from b03ledav006.gho.boulder.ibm.com (b03ledav006.gho.boulder.ibm.com [9.17.130.237]) by b03cxnp07029.gho.boulder.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 122KmkXB50463126 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 2 Mar 2021 20:48:46 GMT Received: from b03ledav006.gho.boulder.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 7B518C6059; Tue, 2 Mar 2021 20:48:46 +0000 (GMT) Received: from b03ledav006.gho.boulder.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id E5DAFC6055; Tue, 2 Mar 2021 20:48:45 +0000 (GMT) Received: from amdrome1.watson.ibm.com (unknown [9.2.130.16]) by b03ledav006.gho.boulder.ibm.com (Postfix) with ESMTP; Tue, 2 Mar 2021 20:48:45 +0000 (GMT) From: "Tobin Feldman-Fitzthum" To: devel@edk2.groups.io Cc: Dov Murik , Tobin Feldman-Fitzthum , Tobin Feldman-Fitzthum , James Bottomley , Hubertus Franke , Brijesh Singh , Ashish Kalra , Jon Grimm , Tom Lendacky Subject: [RFC PATCH 00/14] Firmware Support for Fast Live Migration for AMD SEV Date: Tue, 2 Mar 2021 15:48:25 -0500 Message-Id: <20210302204839.82042-1-tobin@linux.ibm.com> X-Mailer: git-send-email 2.20.1 MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.369,18.0.761 definitions=2021-03-02_08:2021-03-01,2021-03-02 signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 malwarescore=0 mlxlogscore=999 adultscore=0 suspectscore=0 spamscore=0 lowpriorityscore=0 mlxscore=0 phishscore=0 priorityscore=1501 bulkscore=0 impostorscore=0 clxscore=1015 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2009150000 definitions=main-2103020156 Content-Transfer-Encoding: 8bit This is a demonstration of fast migration for encrypted virtual machines using a Migration Handler that lives in OVMF. This demo uses AMD SEV, but the ideas may generalize to other confidential computing platforms. With AMD SEV, guest memory is encrypted and the hypervisor cannot access or move it. This makes migration tricky. In this demo, we show how the HV can ask a Migration Handler (MH) in the firmware for an encrypted page. The MH encrypts the page with a transport key prior to releasing it to the HV. The target machine also runs an MH that decrypts the page once it is passed in by the target HV. These patches are not ready for production, but the are a full end-to-end solution that facilitates a fast live migration between two SEV VMs. Corresponding patches for QEMU have been posted my colleague Dov Murik on qemu-devel. Our approach needs little kernel support, requiring only one hypercall that the guest can use to mark a page as encrypted or shared. This series includes updated patches from Ashish Kalra and Brijesh Singh that allow OVMF to use this hypercall. The MH runs continuously in the guest, waiting for communication from the HV. The HV starts an additional vCPU for the MH but does not expose it to the guest OS via ACPI. We use the MpService to start the MH. The MpService is only available at runtime and processes that are started by it are usually cleaned up on ExitBootServices. Since we need the MH to run continuously, we had to make some modifications. Ideally a feature could be added to the MpService to allow for the starting of long-running processes. Besides migration, this could support other background processes that need to operate within the encryption boundary. For now, we have included a handful of patches that modify the MpService to allow the MH to keep running after ExitBootServices. These are temporary. Ashish Kalra (2): OvmfPkg/PlatformPei: Mark SEC GHCB page in the page encrpytion bitmap. OvmfPkg/PlatformDxe: Add support for SEV live migration. Brijesh Singh (1): OvmfPkg/BaseMemEncryptLib: Support to issue unencrypted hypercall Dov Murik (1): OvmfPkg/AmdSev: Build page table for migration handler Tobin Feldman-Fitzthum (10): OvmfPkg/AmdSev: Base for Confidential Migration Handler OvmfPkg/PlatfomPei: Set Confidential Migration PCD OvmfPkg/AmdSev: Setup Migration Handler Mailbox OvmfPkg/AmdSev: MH support for mailbox protocol UefiCpuPkg/MpInitLib: temp removal of MpLib cleanup UefiCpuPkg/MpInitLib: Allocate MP buffer as runtime memory UefiCpuPkg/CpuExceptionHandlerLib: Exception handling as runtime memory OvmfPkg/AmdSev: Don't overwrite mailbox or pagetables OvmfPkg/AmdSev: Don't overwrite MH stack OvmfPkg/AmdSev: MH page encryption POC OvmfPkg/OvmfPkg.dec | 11 + OvmfPkg/AmdSev/AmdSevX64.dsc | 2 + OvmfPkg/AmdSev/AmdSevX64.fdf | 13 +- .../ConfidentialMigrationDxe.inf | 45 +++ .../ConfidentialMigrationPei.inf | 35 ++ .../DxeMemEncryptSevLib.inf | 1 + .../PeiMemEncryptSevLib.inf | 1 + OvmfPkg/PlatformDxe/Platform.inf | 2 + OvmfPkg/PlatformPei/PlatformPei.inf | 2 + UefiCpuPkg/Library/MpInitLib/DxeMpInitLib.inf | 2 + UefiCpuPkg/Library/MpInitLib/PeiMpInitLib.inf | 2 + OvmfPkg/AmdSev/ConfidentialMigration/MpLib.h | 235 +++++++++++++ .../ConfidentialMigration/VirtualMemory.h | 177 ++++++++++ OvmfPkg/Include/Guid/MemEncryptLib.h | 16 + OvmfPkg/PlatformDxe/PlatformConfig.h | 5 + .../ConfidentialMigrationDxe.c | 325 ++++++++++++++++++ .../ConfidentialMigrationPei.c | 25 ++ .../X64/PeiDxeVirtualMemory.c | 18 + OvmfPkg/PlatformDxe/AmdSev.c | 99 ++++++ OvmfPkg/PlatformDxe/Platform.c | 6 + OvmfPkg/PlatformPei/AmdSev.c | 10 + OvmfPkg/PlatformPei/Platform.c | 10 + .../CpuExceptionHandlerLib/DxeException.c | 8 +- UefiCpuPkg/Library/MpInitLib/DxeMpLib.c | 21 +- UefiCpuPkg/Library/MpInitLib/MpLib.c | 7 +- 25 files changed, 1061 insertions(+), 17 deletions(-) create mode 100644 OvmfPkg/AmdSev/ConfidentialMigration/ConfidentialMigrationDxe.inf create mode 100644 OvmfPkg/AmdSev/ConfidentialMigration/ConfidentialMigrationPei.inf create mode 100644 OvmfPkg/AmdSev/ConfidentialMigration/MpLib.h create mode 100644 OvmfPkg/AmdSev/ConfidentialMigration/VirtualMemory.h create mode 100644 OvmfPkg/Include/Guid/MemEncryptLib.h create mode 100644 OvmfPkg/AmdSev/ConfidentialMigration/ConfidentialMigrationDxe.c create mode 100644 OvmfPkg/AmdSev/ConfidentialMigration/ConfidentialMigrationPei.c create mode 100644 OvmfPkg/PlatformDxe/AmdSev.c -- 2.20.1