From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mga18.intel.com (mga18.intel.com [134.134.136.126]) by mx.groups.io with SMTP id smtpd.web08.1454.1615602752267373211 for ; Fri, 12 Mar 2021 18:32:32 -0800 Authentication-Results: mx.groups.io; dkim=pass header.i=@intel.onmicrosoft.com header.s=selector2-intel-onmicrosoft-com header.b=elCs7yaF; spf=pass (domain: intel.com, ip: 134.134.136.126, mailfrom: jiewen.yao@intel.com) IronPort-SDR: 5F3NmdhtVOGoizT5LF/xlQeUIYr1YzN7kcX60fBXgze8rZcxGgL4nGOAZuzVFXOggq6NRJjyto NPvLPP2g2h0Q== X-IronPort-AV: E=McAfee;i="6000,8403,9921"; a="176503251" X-IronPort-AV: E=Sophos;i="5.81,245,1610438400"; d="scan'208";a="176503251" Received: from fmsmga001.fm.intel.com ([10.253.24.23]) by orsmga106.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 12 Mar 2021 18:32:30 -0800 IronPort-SDR: dVHtkw8akoEHfjjVAoEQE7x8V2xlEeJw/nbCpA5o2LfsMqWiWTQrZ51tKkbAaij3WzzVxOEty+ uy8GfjGuT/4Q== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.81,245,1610438400"; d="scan'208";a="510512134" Received: from orsmsx601.amr.corp.intel.com ([10.22.229.14]) by fmsmga001.fm.intel.com with ESMTP; 12 Mar 2021 18:32:30 -0800 Received: from orsmsx612.amr.corp.intel.com (10.22.229.25) by ORSMSX601.amr.corp.intel.com (10.22.229.14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2106.2; Fri, 12 Mar 2021 18:32:29 -0800 Received: from orsmsx609.amr.corp.intel.com (10.22.229.22) by ORSMSX612.amr.corp.intel.com (10.22.229.25) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2106.2; Fri, 12 Mar 2021 18:32:29 -0800 Received: from ORSEDG602.ED.cps.intel.com (10.7.248.7) by orsmsx609.amr.corp.intel.com (10.22.229.22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2106.2 via Frontend Transport; Fri, 12 Mar 2021 18:32:29 -0800 Received: from NAM10-DM6-obe.outbound.protection.outlook.com (104.47.58.109) by edgegateway.intel.com (134.134.137.103) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.1.2106.2; Fri, 12 Mar 2021 18:32:29 -0800 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=R7uwJ6dibu27yGnr8ki2noBGJBGBFSB7EjW3GEttjkQ7Y4SM5sEkkjX4ez8LCDb3Lg6qGcYN7eM84XCX6HZIUggnrxJ1HoN+4ILy3mH5ZbiwOVXPJ+H0plwnzAgeUjL0Pxxmx7cc37X5+invJ9LpN7mxzCeJ/cc9eSfJr5B7psSjP3yIhTos1HCwS60WZp2N84NvlQKTm76CxnPhBeFfkWffwDNlRfR5Il8bOcOb6hObSU0ycHhtmYA/HTqZMzEP9u69mV1rQFKsL4nPjmqTwSDWL7XqdBJCK3nEv1r0QKy8NI3NefVJ/BPP8CBzCgohSNMCRRfUrM+qGb3XSlM7tg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=LNHP+ZFPqVZjsIDAHV+xsob1HEWNR7sEupHlIJezVZo=; b=czgMc00NOFpa6rAcsRDcRGOp8ooguQh5CJmVCDZ4mhKRZ5QzJXhm7njP+Uy3YX+tqjOyYA1VcvxbWkCjMKK5sN/GVjaThp0XS8R5qunMOgUfcndqdlO3SVz8frWeSAcxylZm3EShfgFRlhODCiQ603/DpW82caVkMhRf2s7UdM8KpsJtRe+f10DlOE3zek0Ez9zxz5FlCWekfRS9/LOGwDA2mlIIAz0XFDOiIl2TmxaWFMp7e+UmuqwuZlF4T7meCZzIjHzjqfb3Phd4KcfpHlLF9IZY4VSR7yhpkFTk5H7W9i2vxcKwA54uJoeBQI8R8uBJigga7fPsEoUuQwIIEg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com; dkim=pass header.d=intel.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=intel.onmicrosoft.com; s=selector2-intel-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=LNHP+ZFPqVZjsIDAHV+xsob1HEWNR7sEupHlIJezVZo=; b=elCs7yaF8JVUv2hD/aY0+nn649eouTxtMxlnGCf9tb3TcPVobOap9shgr34cv/S7XPDF1N2f2K0hRJaB0CAWd90wRH1UKwP7vWtoLpHEM+CTcD489GqpU3YQc4EP6EqCcK5c395CvnXgMSDYmXJW83L17yZ9/7Ej6Q21FeRTtoE= Received: from BY5PR11MB4166.namprd11.prod.outlook.com (2603:10b6:a03:191::25) by BYAPR11MB3239.namprd11.prod.outlook.com (2603:10b6:a03:7b::12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3912.26; Sat, 13 Mar 2021 02:32:26 +0000 Received: from BY5PR11MB4166.namprd11.prod.outlook.com ([fe80::5983:f233:56d6:8132]) by BY5PR11MB4166.namprd11.prod.outlook.com ([fe80::5983:f233:56d6:8132%4]) with mapi id 15.20.3933.031; Sat, 13 Mar 2021 02:32:26 +0000 From: "Yao, Jiewen" To: "devel@edk2.groups.io" , "Yao, Jiewen" , "tobin@linux.ibm.com" CC: Dov Murik , Tobin Feldman-Fitzthum , James Bottomley , Hubertus Franke , Brijesh Singh , Ashish Kalra , Jon Grimm , Tom Lendacky Subject: Re: [edk2-devel] [RFC PATCH 00/14] Firmware Support for Fast Live Migration for AMD SEV Thread-Topic: [edk2-devel] [RFC PATCH 00/14] Firmware Support for Fast Live Migration for AMD SEV Thread-Index: AQHXD6WGO2wl4FzFUEaMI0mL8je3S6pzERzwgA4kddA= Date: Sat, 13 Mar 2021 02:32:25 +0000 Message-ID: References: <20210302204839.82042-1-tobin@linux.ibm.com> <166900903D364B89.9163@groups.io> In-Reply-To: <166900903D364B89.9163@groups.io> Accept-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: dlp-version: 11.5.1.3 dlp-product: dlpe-windows dlp-reaction: no-action authentication-results: edk2.groups.io; dkim=none (message not signed) header.d=none;edk2.groups.io; dmarc=none action=none header.from=intel.com; x-originating-ip: [101.87.139.49] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: fc8c9730-b03a-4d72-3e64-08d8e5c83d87 x-ms-traffictypediagnostic: BYAPR11MB3239: x-ms-exchange-transport-forked: True x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:9508; x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: vieTehhwuplHcVFzBt2PUyptKZtQSiS6kOaS7o3n3twIeFz2n4kASxkv7Mx1RlIgu/yrSEIHRUZWUSHxFkqIgQurjCjBP7wdotItGdhZp6loBAchgACmHN7zvTuSz1DYMR0ZD5iAfs0yjdCoJf4gny9o5L3R4drb0NoD7QHg//yq0POFMrvkV6eCTG0zgFR5Y7bK4xWCrHRMRMegyGDPPgeMdxYrik182dzYwp3FwYq7zfaAFwWO7AM6e4Pm55JQhmDknGGr7MjXUQMYQNkxaTxk5bmASboAyv6dGoh4yINY/pAGIMl3R6pnyQB3EpkxHg99ikk8s8z88KYUwEv7YjTf9vvoDJi3Kun741dUbHujj6/kUsv7Jxc4IgsYvCcqod4iqEnWUxg6ivzhT0vdUxNFhpJ4ghf8PHu/3nRqwDwAoLv434S5k23sfmDUK7uRcmLQ4pGPZjH87Ej8OIziX9+LvIP7cMTLDtfp54tsoJ8gJNkO61Cv0U21sPkt1V90KRc9m5GtvKpKkexwLS0IK0sgtzFOSDejbYQwEp0No7UmgiFLcQMQ6ICKWmny1KBmr5IM1hbp+8ew8UZhFJhLNyfJZzR49o/3yG3grKcSGe8= x-forefront-antispam-report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BY5PR11MB4166.namprd11.prod.outlook.com;PTR:;CAT:NONE;SFS:(136003)(396003)(376002)(39860400002)(346002)(366004)(55016002)(478600001)(53546011)(8936002)(26005)(2906002)(4326008)(33656002)(86362001)(7696005)(54906003)(66556008)(64756008)(66476007)(76116006)(110136005)(66446008)(66946007)(19627235002)(71200400001)(5660300002)(83380400001)(186003)(966005)(6506007)(52536014)(316002)(7416002)(9686003)(8676002);DIR:OUT;SFP:1102; x-ms-exchange-antispam-messagedata: =?us-ascii?Q?ayrl64z4iMKdsJGoSv4ODtLVNvI0V1pC1/RAIrW0/Cedq4X9MhYQIWVDn0dw?= =?us-ascii?Q?Lb5jgn2b4aulTizEhvXprHeClXWxepCouhUT8Zlta62VOfKZFbRd1kwpUPjo?= =?us-ascii?Q?8KS+yrsxCCLoQEs4+cwvF4hudeENQYNbuwp9Pbj7/OsYfvRFcJxe3UWadOrE?= =?us-ascii?Q?9eNoM8ozfgtm5ZbDbRyMa4Yfa633JmjJl8b53cJz7Highe771chfzFwh7w1u?= =?us-ascii?Q?JFKQATLhPOkhJRY3u1YHnOVYEYHcRhfsjPIehuBG+wi/JaarCc5rmYO6uyec?= =?us-ascii?Q?8YoD6x7FbJ3JGctBYgIMQXI2Uin+mYE7DAjZyNjgxwjM+LG/734reOM+sEOh?= =?us-ascii?Q?lYaqmN+H22pEyE3a0wlEaXmVVew9aWbs02u0ML+F9duOZGcOm+1zf+3q3J6E?= =?us-ascii?Q?Cdu+ifvJ8yjaD9Yx1NoBYXEw9q/rDaNfriBjN/CO+TPOGRycBW1ENhGZPOpl?= =?us-ascii?Q?7bcW0Tv+sSAxBNlU1Av1qDnYE0ps4BmCebxJUkJaAYSPmCC9X9Qnp2CgWTfx?= =?us-ascii?Q?VDqW7yM9iqhXsuBcxx/brdScjKmUnPCFoMo6qGSFJnyolB5Xj3tZL2QAAZ72?= =?us-ascii?Q?n5/jDWYqRizu1eE0LT0pHTqSqirwPosEbgyrRIEhwX8hy+JMa8mg/b16i6KF?= =?us-ascii?Q?LmkuzJNSuvlZOFkbZBtGKXxNGsjq3Fyu4T88ratLlYL713+bH1vpdeqcmqCw?= =?us-ascii?Q?SPX/d86b3XoUBVCYW1Jk9bWyvW0FeTj9yssShFQ1LXxXUfjndCilASfGK1yA?= =?us-ascii?Q?jZS+fQjJHntrBM0o4TOs7dF+hr7xKkfEWowwLDdC0qBnulaP2JM9o2IzJxiD?= =?us-ascii?Q?Rp2UuaHD0W4Qpxp6ZCilVZ2OkFnc95W8aY4YWoz3lfDzMdY+Cu3HBYc2Ill8?= =?us-ascii?Q?rRfNd0RQcHGPki3VGb6Vb1wwwRT1112P6opC6baaDxLQqnjPSCIpwuUZGFkw?= =?us-ascii?Q?RFUTUCU6OWjbg9E71l+sPDXMQKxwVkGpv/XeAPAIB4mIIsYV/GRj/pzP8BGj?= =?us-ascii?Q?ZeHZQXoFJ0HL2bT6BQqbUi9/ZWHTt22uTJjsXvvt38VsrBtalWYsxJQYUHU1?= =?us-ascii?Q?z/qKxIcopBDQrPM79k6Y5te35VkGvrWC5E8GJ3hHiwzj5k1IRufjDfV4MJ/t?= =?us-ascii?Q?018eiCCgozj3G2xbLL9WZY1QWezVAihKDQIc3zZ5Rk+l1iyrALXnIVkM+pB1?= =?us-ascii?Q?UhR06cvWpJifzi2jJrJZ27rduKk/3YTWmmS4Qa9Sq5f6lBca2xSAWAH8KitO?= =?us-ascii?Q?huDMpX2URUyAJYnHnjSjWXniNE8gH8347KPmdDpRJKqvm2ifLkyhb8aex6i8?= =?us-ascii?Q?hMHdzzOQJ3BqVWiCNZs4W0kg?= MIME-Version: 1.0 X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: BY5PR11MB4166.namprd11.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: fc8c9730-b03a-4d72-3e64-08d8e5c83d87 X-MS-Exchange-CrossTenant-originalarrivaltime: 13 Mar 2021 02:32:26.0069 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 46c98d88-e344-4ed4-8496-4ed7712e255d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: PaaSNJXsfhn2QoIBw+yeHJ1fZoPpZEuJum9v4c3Vd00sgGrWxM69NIvzX/tZxqGm37BkJ7SIcjJxQI3WZM2Xpw== X-MS-Exchange-Transport-CrossTenantHeadersStamped: BYAPR11MB3239 Return-Path: jiewen.yao@intel.com X-OriginatorOrg: intel.com Content-Language: en-US Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Hi We discuss the patch internally. We do see PROs and CONs with this approac= h. The advantage is that it is very simple. In-VM migration can save lots of = effort on security context restore. On the other hand, we feel not so comfortable to reserve a dedicate CPU to= achieve that. Similar to the feedback in the community. Using Hot-Plug is not a solution for Intel TDX as well. It is unsupported = now. I like the idea to diverge the migration boot mode v.s. normal boot mode i= n SEC phase. We must be very carefully handle this migration boot mode, to avoid any to= uching on system memory. Intel TDX Virtual Firmware skips the PEI phase directly. If we choose this= approach, SEC-based migration is our preference. Besides this patch, we would like to understand a full picture. 1) How the key is passed from source VM to destination? I saw you mentions: "Key sharing is out of scope for this part of the RFC.= " "This will probably be implemented via inject-launch-secret in the future" Does that mean two PSP will sync with each other and negotiate the key, af= ter the Migration Agent (MA) checks the policy? 2) How the attestation is supported? I read the whitepaper https://www.amd.com/system/files/TechDocs/SEV-SNP-st= rengthening-vm-isolation-with-integrity-protection-and-more.pdf. It seems SEV and SEV-ES only support attestation during launch, I don't be= lieve this migration feature will impact the attestation report. Am I right= ? SEV-SNP supports more flexible attestation, does it include any informatio= n about the new migrated content? > -----Original Message----- > From: devel@edk2.groups.io On Behalf Of Yao, Jiew= en > Sent: Thursday, March 4, 2021 9:49 AM > To: devel@edk2.groups.io; tobin@linux.ibm.com > Cc: Dov Murik ; Tobin Feldman-Fitzthum > ; James Bottomley ; Hubertus Franke > ; Brijesh Singh ; Ashish Kalr= a > ; Jon Grimm ; Tom Lendacky > ; Yao, Jiewen > Subject: Re: [edk2-devel] [RFC PATCH 00/14] Firmware Support for Fast Li= ve > Migration for AMD SEV >=20 > Hi Tobin > Thanks for your patch. > You may that Intel is working on TDX for the same live migration feature= . >=20 > Please give me some time (about 1 work week) to digest and evaluate the = patch > and impact. > Then I will provide feedback. >=20 > Thank you > Yao Jiewen >=20 > > -----Original Message----- > > From: devel@edk2.groups.io On Behalf Of Tobin > > Feldman-Fitzthum > > Sent: Wednesday, March 3, 2021 4:48 AM > > To: devel@edk2.groups.io > > Cc: Dov Murik ; Tobin Feldman-Fitzthum > > ; Tobin Feldman-Fitzthum ; James > > Bottomley ; Hubertus Franke ; > > Brijesh Singh ; Ashish Kalra > ; > > Jon Grimm ; Tom Lendacky > > > > Subject: [edk2-devel] [RFC PATCH 00/14] Firmware Support for Fast Live > > Migration for AMD SEV > > > > This is a demonstration of fast migration for encrypted virtual machin= es > > using a Migration Handler that lives in OVMF. This demo uses AMD SEV, > > but the ideas may generalize to other confidential computing platforms= . > > With AMD SEV, guest memory is encrypted and the hypervisor cannot acce= ss > > or move it. This makes migration tricky. In this demo, we show how the > > HV can ask a Migration Handler (MH) in the firmware for an encrypted > > page. The MH encrypts the page with a transport key prior to releasing > > it to the HV. The target machine also runs an MH that decrypts the pag= e > > once it is passed in by the target HV. These patches are not ready for > > production, but the are a full end-to-end solution that facilitates a > > fast live migration between two SEV VMs. > > > > Corresponding patches for QEMU have been posted my colleague Dov Murik > > on qemu-devel. Our approach needs little kernel support, requiring onl= y > > one hypercall that the guest can use to mark a page as encrypted or > > shared. This series includes updated patches from Ashish Kalra and > > Brijesh Singh that allow OVMF to use this hypercall. > > > > The MH runs continuously in the guest, waiting for communication from > > the HV. The HV starts an additional vCPU for the MH but does not expos= e > > it to the guest OS via ACPI. We use the MpService to start the MH. The > > MpService is only available at runtime and processes that are started = by > > it are usually cleaned up on ExitBootServices. Since we need the MH to > > run continuously, we had to make some modifications. Ideally a feature > > could be added to the MpService to allow for the starting of > > long-running processes. Besides migration, this could support other > > background processes that need to operate within the encryption > > boundary. For now, we have included a handful of patches that modify t= he > > MpService to allow the MH to keep running after ExitBootServices. Thes= e > > are temporary. > > > > Ashish Kalra (2): > > OvmfPkg/PlatformPei: Mark SEC GHCB page in the page encrpytion bitma= p. > > OvmfPkg/PlatformDxe: Add support for SEV live migration. > > > > Brijesh Singh (1): > > OvmfPkg/BaseMemEncryptLib: Support to issue unencrypted hypercall > > > > Dov Murik (1): > > OvmfPkg/AmdSev: Build page table for migration handler > > > > Tobin Feldman-Fitzthum (10): > > OvmfPkg/AmdSev: Base for Confidential Migration Handler > > OvmfPkg/PlatfomPei: Set Confidential Migration PCD > > OvmfPkg/AmdSev: Setup Migration Handler Mailbox > > OvmfPkg/AmdSev: MH support for mailbox protocol > > UefiCpuPkg/MpInitLib: temp removal of MpLib cleanup > > UefiCpuPkg/MpInitLib: Allocate MP buffer as runtime memory > > UefiCpuPkg/CpuExceptionHandlerLib: Exception handling as runtime > > memory > > OvmfPkg/AmdSev: Don't overwrite mailbox or pagetables > > OvmfPkg/AmdSev: Don't overwrite MH stack > > OvmfPkg/AmdSev: MH page encryption POC > > > > OvmfPkg/OvmfPkg.dec | 11 + > > OvmfPkg/AmdSev/AmdSevX64.dsc | 2 + > > OvmfPkg/AmdSev/AmdSevX64.fdf | 13 +- > > .../ConfidentialMigrationDxe.inf | 45 +++ > > .../ConfidentialMigrationPei.inf | 35 ++ > > .../DxeMemEncryptSevLib.inf | 1 + > > .../PeiMemEncryptSevLib.inf | 1 + > > OvmfPkg/PlatformDxe/Platform.inf | 2 + > > OvmfPkg/PlatformPei/PlatformPei.inf | 2 + > > UefiCpuPkg/Library/MpInitLib/DxeMpInitLib.inf | 2 + > > UefiCpuPkg/Library/MpInitLib/PeiMpInitLib.inf | 2 + > > OvmfPkg/AmdSev/ConfidentialMigration/MpLib.h | 235 +++++++++++++ > > .../ConfidentialMigration/VirtualMemory.h | 177 ++++++++++ > > OvmfPkg/Include/Guid/MemEncryptLib.h | 16 + > > OvmfPkg/PlatformDxe/PlatformConfig.h | 5 + > > .../ConfidentialMigrationDxe.c | 325 +++++++++++++++++= + > > .../ConfidentialMigrationPei.c | 25 ++ > > .../X64/PeiDxeVirtualMemory.c | 18 + > > OvmfPkg/PlatformDxe/AmdSev.c | 99 ++++++ > > OvmfPkg/PlatformDxe/Platform.c | 6 + > > OvmfPkg/PlatformPei/AmdSev.c | 10 + > > OvmfPkg/PlatformPei/Platform.c | 10 + > > .../CpuExceptionHandlerLib/DxeException.c | 8 +- > > UefiCpuPkg/Library/MpInitLib/DxeMpLib.c | 21 +- > > UefiCpuPkg/Library/MpInitLib/MpLib.c | 7 +- > > 25 files changed, 1061 insertions(+), 17 deletions(-) > > create mode 100644 > > OvmfPkg/AmdSev/ConfidentialMigration/ConfidentialMigrationDxe.inf > > create mode 100644 > > OvmfPkg/AmdSev/ConfidentialMigration/ConfidentialMigrationPei.inf > > create mode 100644 OvmfPkg/AmdSev/ConfidentialMigration/MpLib.h > > create mode 100644 > > OvmfPkg/AmdSev/ConfidentialMigration/VirtualMemory.h > > create mode 100644 OvmfPkg/Include/Guid/MemEncryptLib.h > > create mode 100644 > > OvmfPkg/AmdSev/ConfidentialMigration/ConfidentialMigrationDxe.c > > create mode 100644 > > OvmfPkg/AmdSev/ConfidentialMigration/ConfidentialMigrationPei.c > > create mode 100644 OvmfPkg/PlatformDxe/AmdSev.c > > > > -- > > 2.20.1 > > > > > > > > > > >=20 >=20 >=20 >=20 >=20