From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from NAM12-BN8-obe.outbound.protection.outlook.com (NAM12-BN8-obe.outbound.protection.outlook.com [40.107.237.44]) by mx.groups.io with SMTP id smtpd.web12.39.1624383971107611049 for ; Tue, 22 Jun 2021 10:46:11 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=@amd.com header.s=selector1 header.b=x3CYajU8; spf=permerror, err=parse error for token &{10 18 %{i}._ip.%{h}._ehlo.%{d}._spf.vali.email}: invalid domain name (domain: amd.com, ip: 40.107.237.44, mailfrom: ashish.kalra@amd.com) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=BdDf9rDhdxwHBY2r68gH14Hq8DBSogKNOyt3gSwB5AccKbkL216kDR6JogfWtl57YcWh6ZrGPnsb7aOkYsfxPAwViabu3iDSyVxBWJde556/09nSqhSCRBz1yyrnNhBdN9IZZmNM23k2B1/dShjZd/DcbCkg6gnn7G3xHwIpveWwz/fHO1BcCmNjZwn0fqYh0HJGoh6kpiFHjPa55vBQgx7vNjFE62Csxdw+8r6EeUy0GMk4yD05X+6keHjsIpzTd7N2nH8wLOLq55qvSRcUstQmka8wHmKBkEB3Idi0CKp3FfNbkIITLUo4FWQGPSbqj0Ohc47SD4eWvdoWSRD9sw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=aXRZYh81TjS3bH1YWciPOS1wJ0ERSiBlXOM3zKVhLQU=; b=FKlKPRn7hPIL4dMw/ZrGMByVipWnn3h5AMEAc2tI0WIXbCtF89GE7MF5NjJ9BmdNM8rj7nOy6rbUMbEOaNU4S5W+B0gGvNH4BQFrb2jYPdqHc+fLR+KekWIxJX6n6w2NpXCsOMf1FkBFmxDCom9NBiyAltxZqgvF51TKDDxwdOehzgS2ha/Ndrlgwp5uMVDwYmMzZY85us7d8+xf1RiM6LqfnvP9eiKMJoydkDUeI+82gTfU3Oz10VQpNT1Z4G5XF9sHEfQ768matywkvqgBYI3M/FMdX0OuRnxLn/TILsz2/2V7ZGaqUP6ykRMjylDeY40jxfg3excqbefxOJYBEQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=amd.com; dmarc=pass action=none header.from=amd.com; dkim=pass header.d=amd.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=aXRZYh81TjS3bH1YWciPOS1wJ0ERSiBlXOM3zKVhLQU=; b=x3CYajU8tXKr7+f4aOQZx6dw5oHAXyEz1U5XwQeCLBtMFV12Q+TL1zSVhWeCNs9ORbx8pjwQ55CGa2yyfCjHly3ieZwGx7wtGM6MZoUBYwWOt79kLxaeKwNy60KvpXd0AxD2BujotmEwbM2VBWIOepeQI+rbJUBPlLaf1NBAcu4= Authentication-Results: redhat.com; dkim=none (message not signed) header.d=none;redhat.com; dmarc=none action=none header.from=amd.com; Received: from SN6PR12MB2767.namprd12.prod.outlook.com (2603:10b6:805:75::23) by SN1PR12MB2415.namprd12.prod.outlook.com (2603:10b6:802:26::23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4264.18; Tue, 22 Jun 2021 17:46:09 +0000 Received: from SN6PR12MB2767.namprd12.prod.outlook.com ([fe80::958d:2e44:518c:744c]) by SN6PR12MB2767.namprd12.prod.outlook.com ([fe80::958d:2e44:518c:744c%7]) with mapi id 15.20.4242.023; Tue, 22 Jun 2021 17:46:09 +0000 Date: Tue, 22 Jun 2021 17:46:03 +0000 From: "Ashish Kalra" To: Laszlo Ersek Cc: devel@edk2.groups.io, brijesh.singh@amd.com, Thomas.Lendacky@amd.com, jejb@linux.ibm.com, erdemaktas@google.com, jiewen.yao@intel.com, min.m.xu@intel.com, jordan.l.justen@intel.com, ard.biesheuvel@arm.com, "Dr. David Alan Gilbert" , Paolo Bonzini Subject: Re: [PATCH v4 0/4] SEV Live Migration support for OVMF. Message-ID: <20210622174603.GA6366@ashkalra_ubuntu_server> References: In-Reply-To: User-Agent: Mutt/1.9.4 (2018-02-28) X-Originating-IP: [165.204.77.1] X-ClientProxiedBy: SA9PR11CA0012.namprd11.prod.outlook.com (2603:10b6:806:6e::17) To SN6PR12MB2767.namprd12.prod.outlook.com (2603:10b6:805:75::23) Return-Path: ashish.kalra@amd.com MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from ashkalra_ubuntu_server (165.204.77.1) by SA9PR11CA0012.namprd11.prod.outlook.com (2603:10b6:806:6e::17) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4242.21 via Frontend Transport; Tue, 22 Jun 2021 17:46:08 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 5c7fac3d-19c7-4ab5-f688-08d935a59e18 X-MS-TrafficTypeDiagnostic: SN1PR12MB2415: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:10000; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: tv5r/p4kpVbl1GTVUMhy2P06azFZ39GYXk4eeQPuqwViVe/uU76RHUvrkjL2tVCWVElyZZiCIAj20yxVwVXB6KHF4vVgNnQFLN2UYo4GnL62DuPc1OHkw4jlqEvkGsNdoXSmv8wA4IajAwTa4Rb0OiaORD6mrsUSa/dHWzKepFDcuBFZ1lD5uTKBeGxhImvGmXYmZqtJ2h/khHP94EP9zhyIjhOr0WyT6cmEpezEgt0us81rF00Kfxd6FfWi3RoJEs+q5shGy6I2oRSmFzsYEqn5GGX/C9vUSiX1r/PJqkNS9xJ8L4/hDUt6ewbqBike5dwQkX6d0H6MEugC/TLixgfDnbuwzaLXLrjTeAV/2m3jnFzaEnEXA2YMHMr/2hxUcVk+8GYe0m9sf1qaEsxBRKWpKKc8ZKJbhp4ybcRiWoEJmae3fO/NGB+hDd3Am6lFoNh46QeyNJhUmPjy4lHo7jJBB2TFFxsS8jFp9fEK9ndAS6Tky/hTpkBRKlBnW/4SPPVCzgWSD0l5draTEJbBmyXqiRcw9tM4ygLqRzkF98ugWQgA1KQaK+A9l5ycPuc8pH43sz3NNOYiLR0r3eVObDFzLs7GAEY+XKYKxPDiVs3n7DlRh67+yYI06pQN0qUTbj0Z+hk0KWVUNq9wNg/i8wCT5RTMmfL6EcXIcBVZkPqP++fvL97xW4fxMuGLwdRL9d4mxKRdnGuj+SjgDvMcccRPebzq2smAr9SNV75H25vDmUnc5+eRSP39F2QYN19kKwQj6idJdgQGy78YfauGArPzJxhUvBnG3fPAtbWWXWLwsPwjFV2bqvl3EauOGxEz X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:SN6PR12MB2767.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(39860400002)(346002)(396003)(376002)(136003)(366004)(66476007)(66556008)(66946007)(956004)(44832011)(2906002)(8936002)(45080400002)(8676002)(316002)(54906003)(19627235002)(6666004)(5660300002)(86362001)(186003)(16526019)(33716001)(33656002)(55016002)(9686003)(53546011)(83380400001)(26005)(1076003)(6496006)(52116002)(38350700002)(966005)(38100700002)(4326008)(478600001)(6916009)(7416002);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?iinup90vTqEWIgAtoDsrrnyUCKF4KuYcxYuwKaHdy4hcU+w7IPH2fslwW87/?= =?us-ascii?Q?8NiLEijwi3+VXwiY2oWdD6s6jUzJ/kf2ZZ9/qIbjCaulezrxbIGWmNXjQNnk?= =?us-ascii?Q?WEFVdGsS8DWXhIFGZwvJ/W95acvkPy7wEk6sPjYSBL1jHDrINKXa+wtGbR7D?= =?us-ascii?Q?6wBn8GS6c98eqHhGfFz2bqiuMzxPj56cWrtz2jK2jQRe9NXoEOEav+672/RO?= =?us-ascii?Q?fTrEikWOItoH23JNq2j13Zozi4eY2zqTlLoAQ/nJa/047VwmpkqOQeXVQ0m+?= =?us-ascii?Q?oQxwjtHID4qYk4jYbgNOR+nFFytyh7HPnjCF4C+VZQxEAiBMVUHxixos6IyK?= =?us-ascii?Q?xG79d0gi0lbaGP8mjQy7oG42W3JE4/r5IjjX+62QQL8MmlKX+uhzm0ji/xGC?= =?us-ascii?Q?kfyo6nlM5HGX2iR7vf4lUx1CF2WcfFn8qv241KzJAtK+tKWLaEs+6/1rfclt?= =?us-ascii?Q?ZM+YWATqnRuYVLkdT+l16J+7EHci0QnO4Fzn3fQIXa8OUtXOK4nP3kNe+YH/?= =?us-ascii?Q?OEV9cD+3wdPpRp40z/VCK6oDE2yf8nb90qIPtYyDCYcws8/qVPXIszcOlXxI?= =?us-ascii?Q?Xq9pz7Z2zBB+tC3cTt71G8CoopWcsS7jZAYb4bd9Qf+YUCc1VAErGDZAdWwE?= =?us-ascii?Q?5FOlTBAPEkw/q+lQBqrQ08mZjLu1AviyTZ+/eO9QH+Ap3IYAny+xWPNNkXeN?= =?us-ascii?Q?kzFQVvU6tCD8KbUQ4rJjnkwjpErYw7ED+v4GwNyFsL2+C5WSz33X3SRkVNyO?= =?us-ascii?Q?xijD3TNNd3jXyHSEmJPCj8gQkgUhLwSkfJXGNEK3ZN/05VKK75Ar52B8K8YZ?= =?us-ascii?Q?snCpe1W3MHaZuff+vfkqVARbB8mS6V8CjkMXYtjqAfNpgaZ9M6BuxnkkivoE?= =?us-ascii?Q?Cw5MYgDP4Ap24wX6GumHPSxjifAKt+WUiOriBZ99QPHuf7MyKN0H2jwilA2L?= =?us-ascii?Q?OEnsca+ABYHLEpmJ9zVafchpYWPNRq7FBToon/XUKV4eDBqoWLY149SAX1/j?= =?us-ascii?Q?2LR1lyhvwvM1hOC9QJAwX+ORFJhJW2x33jVWqciDGE8TiQDQPY5KKHBAEuP6?= =?us-ascii?Q?v6p8JsZKWQBmKFWy+wQwx3woTqSoektAuR46SfjGMgMZXsg7Ya8bGIJC34yu?= =?us-ascii?Q?OYf3LtXrm1twuFcy9koKq9kmidyda9AxtOuaB5N88fJIXjB24Yo0/rmN+UrI?= =?us-ascii?Q?bOvO8I2Umkul0SQS+JO9pYKtSkoCSCL7kw2yXx3a4nmiHxh6JhIkgAnWlUOb?= =?us-ascii?Q?qU3+t8R8+Em7Auf7UNyXDD9Qu/ZV79T41ei144NAPTUukyX8Xwa+cF9TqU4H?= =?us-ascii?Q?ZQMitn1es3gUw7ewVjum/s6e?= X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-Network-Message-Id: 5c7fac3d-19c7-4ab5-f688-08d935a59e18 X-MS-Exchange-CrossTenant-AuthSource: SN6PR12MB2767.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 22 Jun 2021 17:46:08.9886 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: +isAZ2QOYJaU94meWOTrLjvu7bjVOdWmzA0Bh6C7YFMp8u2o0QcxB2xe7sPs7Ba69tYHFWbrAtG90j7vyihMVg== X-MS-Exchange-Transport-CrossTenantHeadersStamped: SN1PR12MB2415 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Hello Laszlo, Please see my replies below : On Tue, Jun 22, 2021 at 07:20:53PM +0200, Laszlo Ersek wrote: > Hi Ashish, > > (+Dave, +Paolo) > > On 06/21/21 15:56, Ashish Kalra wrote: > > From: Ashish Kalra > > > > By default all the SEV guest memory regions are considered encrypted, > > if a guest changes the encryption attribute of the page (e.g mark a > > page as decrypted) then notify hypervisor. Hypervisor will need to > > track the unencrypted pages. The information will be used during > > guest live migration, guest page migration and guest debugging. > > > > The patch-set adds a new SEV and SEV-ES hypercall abstraction > > library to support SEV Page encryption/decryption status hypercalls > > for SEV and SEV-ES guests. > > > > BaseMemEncryptSevLib invokes hypercalls via this new hypercall library. > > > > The patch-set detects if it is running under KVM hypervisor and then > > checks for SEV live migration feature support via KVM_FEATURE_CPUID, > > if detected setup a new UEFI enviroment variable to indicate OVMF > > support for SEV live migration. > > > > A branch containing these patches is available here: > > https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fashkalra%2Fedk2%2Ftree%2Fsev_live_migration_v4&data=04%7C01%7Cashish.kalra%40amd.com%7Ca402470f4cba428feaac08d935a21c79%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637599792647899814%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=41Jj%2BGNUyEL43UhZgI19iwGcOJcP%2FWg94D8fTopaZxQ%3D&reserved=0 > > > > Changes since v3: > > - Fix all DSC files under OvmfPkg except X64 to add support for > > BaseMemEncryptLib and add NULL instance of BaseMemEncryptLib > > for 32 bit platforms. > > - Add the MemEncryptHypercallLib-related files to Maintainers.txt, > > in section "OvmfPkg: Confidential Computing". > > - Add support for the new KVM_HC_MAP_GPA_RANGE hypercall interface. > > - Add patch for SEV live migration support. > > I have absolutely zero context in my mind about this work. > > By v1 / v2 / v3, are you referring to the following patch series (from December 2020): > > - [PATCH v1 0/2] SEV Page Encryption Bitmap support for OVMF. > https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Flistman.redhat.com%2Farchives%2Fedk2-devel-archive%2F2020-December%2Fmsg00081.html&data=04%7C01%7Cashish.kalra%40amd.com%7Ca402470f4cba428feaac08d935a21c79%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637599792647899814%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=cVUvWe6VgjR78OAk5LXgBQiKon4Gp%2F62a2hc%2FKwoLw4%3D&reserved=0 > > - [PATCH v2 0/3] SEV Page Encryption Bitmap support for OVMF. > https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Flistman.redhat.com%2Farchives%2Fedk2-devel-archive%2F2020-December%2Fmsg00198.html&data=04%7C01%7Cashish.kalra%40amd.com%7Ca402470f4cba428feaac08d935a21c79%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637599792647899814%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=IXLGA2ttyxdVoC63HeCkPVNuUMH3u5Vd3U1fc6c8LQc%3D&reserved=0 > > - [PATCH v3 0/3] SEV Page Encryption Bitmap support for OVMF. > https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Flistman.redhat.com%2Farchives%2Fedk2-devel-archive%2F2020-December%2Fmsg00202.html&data=04%7C01%7Cashish.kalra%40amd.com%7Ca402470f4cba428feaac08d935a21c79%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637599792647899814%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=cOJBqGyM3a7xMbfhbBAh3IEm7IBJFOGu2ReQMSFJ%2BDw%3D&reserved=0 > Yes, actually the guest kernel API for SEV live migration was not decided at the time of the last patch-set submission, hence, i am now submitting this patch-set as the guest kernel API has been discussed and closed and the corresponding KVM/kernel patches have been queued now. And therefore, this is simply not the SEV Page Encryption Bitmap support anymore, but the SEV live migration support which includes the guest page encryption status tracking. > We certainly need a new TianoCore BZ for tracking this feature; I only found the above patch set versions because I have full text search for my complete email traffic on my laptop. Sending v4 after half a year hiatus is like sending it in the next century. :) > Ok. As i mentioned above it took such a long time to re-submit the patch-set because of the guest kernel API discussions taking some time and getting closed. > Anyway, where I'm particularly lost is that I (very vaguely) recall conflicting approaches from AMD and IBM on migration. Has an agreement been reached there? > Yes, these are the slow SEV live migration patches. The fast migration support is being developed by IBM and that is a separate effort. As this slow live migration support has now been included in KVM, we will need the corresponding OVMF and QEMU support now. > I certainly apologize for missing the context here; had someone asked me if I had seen any version of this patch set before, I would have *sworn* that I hadn't. > > I'm basically incapable of tracking this volume of development around confidential computing; sorry. > > Please find below your reply on v3 of this patch-set : Please include such a patch in v4 -- if Tom and Brijesh agree, I'd like to put the new lib explicitly under their reviewership. Also, I plan to review this series (v4, at this point) only for formalities. I'd like to receive an R-b from Tom or Brijesh [*], and another from Dov or a colleague at IBM, for this series; those together should suffice for merging the library. So, if you are fine with this approach, then probably Tom or Brijesh can take these patches under their reviewership and provide their R-b for this patch-set to be accepted and merged. I believe that they are anyway the maintainers for confidential computing related stuff. Thanks, Ashish > > > > Changes since v2: > > - GHCB_BASE setup during reset-vector as decrypted is marked explicitly > > in the hypervisor page encryption bitmap after setting the > > PcdSevEsIsEnabled PCD. > > > > Changes since v1: > > - Mark GHCB_BASE setup during reset-vector as decrypted explicitly in > > the hypervisor page encryption bitmap. > > - Resending the series with correct shallow threading. > > > > Ashish Kalra (3): > > OvmfPkg/MemEncryptHypercallLib: add library to support SEV hypercalls. > > OvmfPkg/PlatformPei: Mark SEC GHCB page as unencrypted via hypercall > > OvmfPkg/PlatformDxe: Add support for SEV live migration. > > > > Brijesh Singh (1): > > OvmfPkg/BaseMemEncryptLib: Support to issue unencrypted hypercall > > > > Maintainers.txt | 2 + > > OvmfPkg/Include/Guid/MemEncryptLib.h | 20 ++++ > > .../Include/Library/MemEncryptHypercallLib.h | 43 +++++++ > > .../DxeMemEncryptSevLib.inf | 1 + > > .../PeiMemEncryptSevLib.inf | 1 + > > .../X64/PeiDxeVirtualMemory.c | 22 ++++ > > .../Ia32/MemEncryptHypercallLib.c | 37 ++++++ > > .../MemEncryptHypercallLib.inf | 42 +++++++ > > .../X64/AsmHelperStub.nasm | 28 +++++ > > .../X64/MemEncryptHypercallLib.c | 105 +++++++++++++++++ > > OvmfPkg/OvmfPkg.dec | 1 + > > OvmfPkg/OvmfPkgIa32.dsc | 1 + > > OvmfPkg/OvmfPkgIa32X64.dsc | 1 + > > OvmfPkg/OvmfPkgX64.dsc | 1 + > > OvmfPkg/OvmfXen.dsc | 1 + > > OvmfPkg/PlatformDxe/AmdSev.c | 108 ++++++++++++++++++ > > OvmfPkg/PlatformDxe/Platform.c | 5 + > > OvmfPkg/PlatformDxe/Platform.inf | 2 + > > OvmfPkg/PlatformDxe/PlatformConfig.h | 5 + > > OvmfPkg/PlatformPei/AmdSev.c | 10 ++ > > 20 files changed, 436 insertions(+) > > create mode 100644 OvmfPkg/Include/Guid/MemEncryptLib.h > > create mode 100644 OvmfPkg/Include/Library/MemEncryptHypercallLib.h > > create mode 100644 OvmfPkg/Library/MemEncryptHypercallLib/Ia32/MemEncryptHypercallLib.c > > create mode 100644 OvmfPkg/Library/MemEncryptHypercallLib/MemEncryptHypercallLib.inf > > create mode 100644 OvmfPkg/Library/MemEncryptHypercallLib/X64/AsmHelperStub.nasm > > create mode 100644 OvmfPkg/Library/MemEncryptHypercallLib/X64/MemEncryptHypercallLib.c > > create mode 100644 OvmfPkg/PlatformDxe/AmdSev.c > > >