From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from NAM10-DM6-obe.outbound.protection.outlook.com (NAM10-DM6-obe.outbound.protection.outlook.com [40.107.93.75]) by mx.groups.io with SMTP id smtpd.web12.7138.1628159792300127882 for ; Thu, 05 Aug 2021 03:36:32 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=@amd.com header.s=selector1 header.b=zkWyfl1n; spf=permerror, err=parse error for token &{10 18 %{i}._ip.%{h}._ehlo.%{d}._spf.vali.email}: invalid domain name (domain: amd.com, ip: 40.107.93.75, mailfrom: ashish.kalra@amd.com) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=PFaw24bLf5gysozpjVhz81x0PRqBcHlXmaM+VfjFiT4IpaEI2ByKE9jGACeZXJSDyIldf2Q0gECMN8LenShv/5sA4hWKIJDpoSXnG5ZlfAROi9fNFAr6YVojVfrRbk0RDaSTfoYFNejAn3AjRSJN8bHFdWPyAnsNnTBoOlNFJt3k/mBfzS2L1+0QpcmiTn4onIeFdZw8r2TEd5E0HFX1a3Efulvtd1cUPHee8hX9h/QRfKDqSxe40I+CzNBSaWDeB0/KZiyPHKpueku1njV2hdYss2ipU1LW9c6zjPUn2mT5/4XyYbDQJ6a7+L8iWk0RT/NhFaRemqWAcUECX05OvQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Pn995VGd58q7uim7n/Um3mNT/SeVWAPUX/vf8Ep8FoU=; b=OsNXNN++qbrC3tJ8N2Kz5mbDcohrX+yqcc492sxqHFvDVZVOT0Kp49wKDa6LSSrzDvJZHxBU1fOrLSup7bpARhr75XyErNfmcSqtP7laINEYepUBqF98yk8DS/rZCRwBwydfF5037xCxiSbpoQaSevjmsNPcd07FJ0xj+yDOn7anOwpgK7ICASALAqyvFCXAQqeV/hn1j2eCI0FPNEth2a8rMrRXmdIiBGOTp+Gx7uTCfDbGwDci3B3aIUJG32zx9kVdLvJAYWWVMpsBMm/a5BAaxiwqra9gRXpojZ9wb1yD35kQQ+Zumpu47XncnkwArMj2okCu6ZAgLytbSXfbSg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=amd.com; dmarc=pass action=none header.from=amd.com; dkim=pass header.d=amd.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Pn995VGd58q7uim7n/Um3mNT/SeVWAPUX/vf8Ep8FoU=; b=zkWyfl1n6Ag44bOiVHPjCVzH09E0Q1zNVO0+OeWrq5R7W4TSwGcHUYxeR2DTC4d3WxeBFKmm3WCtRIZoXk60Z2P8ts1dn3s+v1KyoEVdjBBgOR+elXKMMj36PfvdIsuHKe+9wSNm+/sgF4eD8b+e/OaYr+rv/pAqU9VVyiNYp5w= Authentication-Results: intel.com; dkim=none (message not signed) header.d=none;intel.com; dmarc=none action=none header.from=amd.com; Received: from SN6PR12MB2767.namprd12.prod.outlook.com (2603:10b6:805:75::23) by SN6PR12MB2831.namprd12.prod.outlook.com (2603:10b6:805:ec::10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4373.26; Thu, 5 Aug 2021 10:36:27 +0000 Received: from SN6PR12MB2767.namprd12.prod.outlook.com ([fe80::491e:2642:bae2:8b73]) by SN6PR12MB2767.namprd12.prod.outlook.com ([fe80::491e:2642:bae2:8b73%7]) with mapi id 15.20.4373.026; Thu, 5 Aug 2021 10:36:27 +0000 Date: Thu, 5 Aug 2021 10:36:25 +0000 From: "Ashish Kalra" To: "Yao, Jiewen" Cc: "devel@edk2.groups.io" , "dovmurik@linux.vnet.ibm.com" , "brijesh.singh@amd.com" , "tobin@ibm.com" , "Thomas.Lendacky@amd.com" , "jejb@linux.ibm.com" , "Justen, Jordan L" , "ard.biesheuvel@arm.com" , "erdemaktas@google.com" , "Xu, Min M" Subject: Re: [edk2-devel] [PATCH v6 0/6] SEV Live Migration support for OVMF. Message-ID: <20210805103624.GB23670@ashkalra_ubuntu_server> References: In-Reply-To: User-Agent: Mutt/1.9.4 (2018-02-28) X-ClientProxiedBy: SA0PR11CA0116.namprd11.prod.outlook.com (2603:10b6:806:d1::31) To SN6PR12MB2767.namprd12.prod.outlook.com (2603:10b6:805:75::23) Return-Path: ashish.kalra@amd.com MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from ashkalra_ubuntu_server (165.204.77.1) by SA0PR11CA0116.namprd11.prod.outlook.com (2603:10b6:806:d1::31) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4394.15 via Frontend Transport; Thu, 5 Aug 2021 10:36:26 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: f9be23c5-565f-456e-5a53-08d957fce113 X-MS-TrafficTypeDiagnostic: SN6PR12MB2831: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:9508; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: gdLA/N8HFIwkfUxhiRRgfHY0ZWyS2KoXyRLhbZy+PYusrYxtDmi2unD1Ho0ZaNr8H6Bh2SY0PCX4L1QyebAxKxv0GBCpLIEZFlY+hBTJf6T8OGV6E6r7pQIqdGs8yYZ/FbHXYjPZHMESedKlZqVGF+SAfEmeYeUHO0iGugd3/2rDWS0vQ9ORWTNE0LrSjfWsgYElz2XN517FbrlHOcnXdIfZdMKs+5mLhvS3A6oKiB+Y9UMOP1Rugw8ZgbUj6+9IbhBlHgFHbmVVlOr69VJ5+Kzj++4i09SgNw2l9rfvDJlN5vzz5yBPj8mKWoMCF+05AKiPjRB5/iz3Ac7PuU+o51mTkfzmK1vSnkX1aq4KigwJpwtUhsYUB3tiaFM12cG/QeVX9W7vibPU+WocKl3DcYheJeK2B5lvYi704QN51RlFw49jPbdNjWNA1TUk7+ioyo4zHVXnX5lv6cgZXImbyq5dphKrFkkqdKvBmkb+2K84XUhpGkrIRSj/K4wO1x9H+ifNWOVp2zkn/8TRa/C+pCJ0o80Uo0G5dbf9v3nxYckuWs2AhNRN/5/3v4qDaHxdMiOgFGQ9W89CUqylI3OtF7ycgldcWSfBEg6lUqwxlQiYGY/LlC8YAl1pRPKiXy0cVtsJfAI6L4vA1MYk1lE3IZED1ulsqXncZLZoNXdvK0IJJd/6lr8Hb0KpVJwg97hYY199jOtIvNz5bCiK5p0d4ylyo4nF/OzTHQ75o5tWpSZW2fuNqjc3KrzuErslLliW769VmIJtUyS7G1FXsgyj3C1diXX1aOlO7XRaOBVOzwiSLKGpkFIgkxn/03Qqa97nofvKKE1fUf2IiIInm4QIJA== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:SN6PR12MB2767.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(136003)(396003)(346002)(366004)(376002)(39860400002)(26005)(33716001)(6916009)(66556008)(66476007)(186003)(66946007)(2906002)(52116002)(6496006)(8936002)(8676002)(86362001)(53546011)(45080400002)(83380400001)(956004)(1076003)(33656002)(38350700002)(966005)(55016002)(19627235002)(9686003)(44832011)(478600001)(316002)(38100700002)(54906003)(4326008)(5660300002);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?h6O8JzjX58Jk27dZ+CSYntUdyBAiF5HlV846Pzr6xs8k0+pxXNYGxmXRRbFQ?= =?us-ascii?Q?X+7hpBEQraW2v7Gh/LJhAYT+wdeXo1miV92/9M4h9T2pzzuJeKyWPLoWesaV?= =?us-ascii?Q?xQ7yN1IXNl9G0pUHrbXJS0rDv+PFbzUf2Pujs4JVpPHj2//FqqPROkGqs0I0?= =?us-ascii?Q?N351D7otH6mi1j0hTYj6KHeDUwGb7faZI8i1pZUbHy8w2bbbO4vzV46RfQUR?= =?us-ascii?Q?8N08Qyo0z2AzXgIzjL66FUEF6bwSKc9kOHUlPfxtk0e+H3ByfBrFw/GJ9QpN?= =?us-ascii?Q?psNAqfPFx7IPC18Vx0BfwDb+1fK4H5Sts9Oh0KZpRDNcrWav/sbRwFnoFTB2?= =?us-ascii?Q?DJeshvU6+gx9jPXFQ/IAbMiICpXsxs3y5kq6r+6G08jfA/ZDf52EDAyJGLbu?= =?us-ascii?Q?P5I/D4UamrtwuUOlj9joBu3MA4YNe/xJ8uYsbXqeD2nUO0H+lWfu8zV/L0B7?= =?us-ascii?Q?sPIyhRvP6U+2XYWsse6MBRF+Bgd07cINtsW7eSPk80FUONRJrVThF6rS1/Tu?= =?us-ascii?Q?6qO4g3if/qxg6Dzag9aywuN9mX3M0WAmqgNP10UmAzIiIy4ggj3fHC4mvxIj?= =?us-ascii?Q?SQbFk8+lNLgd37WhiOy1AxW95SRWBeOtfOLupD/AeikAsY6ESWic51X7wsqW?= =?us-ascii?Q?B8am+nowxx+1YHS+oN0WCDFO9z47ooFvqTvgLT0GOZyFgI4VtlJBsH8oThnd?= =?us-ascii?Q?DkiusPzSEHrmDhd4PJtqOQeUy/4sQYAHbwZ7GBg+LSaHrxO8FWKTfUmWodLE?= =?us-ascii?Q?rS3UqhuBrCFlLv8c5pjAlcIp6623X1rwa2XlTHm1VulzScuZnMhm1Vrz76sf?= =?us-ascii?Q?wbetYj9N/rqtN6cEfwyrzaVdj9GhVPzOjTTf0QAvYnlXfWWumLNY2Rkogv8a?= =?us-ascii?Q?U6R2dalDMtAzieFQbXsuOqpqLPLytzrv1Brwmi2krpu9RorL9/3MzkJ0AuK2?= =?us-ascii?Q?scGj07uS++Jo9sBIVCrjU9r6If9GvXjCxHMvTHKRGQIJnhfDu2s5WoP/8yJF?= =?us-ascii?Q?YlT+VpJv0SPNFvJLDMzqkADfDNpOIZLTc/Fzauk0W4s/sLcDwRf6VwdjRwHm?= =?us-ascii?Q?W4OR2MTFii6GvIXSvLbVu1B2NzkbORNtyp8OLMAdHGmCvSI1/hr26L+DIQJN?= =?us-ascii?Q?+rguIp2ZdPOvetbgy3r2b5q2aclhSDSCROnqfeI4LotYvx4wVlmhiVTFP5dD?= =?us-ascii?Q?amDHS9Q5/kzPVyRjvm6EZQXJ6oFguFQaFboPqYbLts7kTHHpSeM80sHLCu9+?= =?us-ascii?Q?K1+Bxx5PP1xxHAp5HFwrPT0KdKgCJZ/6BkwWHn6YLf/4b9DC4ZLrWeiPmbBy?= =?us-ascii?Q?ii02RLLmiioGl+Vezn8acq4+?= X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-Network-Message-Id: f9be23c5-565f-456e-5a53-08d957fce113 X-MS-Exchange-CrossTenant-AuthSource: SN6PR12MB2767.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 05 Aug 2021 10:36:27.0891 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: L2ejci9QpPjmab6kZa5mLYgRhwHhsETksc7skDmAoN2cW9lopAvZ6u1X97Anlt8pe0VR6waFezY9ed37jLHg1A== X-MS-Exchange-Transport-CrossTenantHeadersStamped: SN6PR12MB2831 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline On Thu, Aug 05, 2021 at 05:17:15AM +0000, Yao, Jiewen wrote: > Hi > I have some questions: > > 1) May I know what is the usage of this UEFI variable - SevLiveMigrationEnabled? > I only see it is created, but I do not see how it is consumed. Yes, it is created by UEFI but consumed by the (guest) linux kernel. This is actually part of a 3-way negotiation of the live migration feature between hypervisor, guest OVMF and guest kernel. Host indicates support for live migration, which is detected by OVMF and correspondingly OVMF sets this SetLiveMigrationEnabled UEFI variable, which is read by the guest kernel and it indicates to the guest kernel that both host and OVMF support and have enabled the live migration feature. The guest flow for detecting live migration feature is captured below (from the corresponding kernel patch): The guest support for detecting and enabling SEV Live migration feature uses the following logic : - kvm_init_plaform() checks if its booted under the EFI - If not EFI, i) if kvm_para_has_feature(KVM_FEATURE_MIGRATION_CONTROL), issue a wrmsrl() to enable the SEV live migration support - If EFI, i) If kvm_para_has_feature(KVM_FEATURE_MIGRATION_CONTROL), read the UEFI variable which indicates OVMF support for live migration (SetLiveMigrationEnabled) ii) the variable indicates live migration is supported, issue a wrmsrl() to enable the SEV live migration support > > 2) Is this a full live migration patch, or is this just a startup and there will be more on the way? > Yes. Actually this is hypervisor based live migration, so it relies on host KVM and qemu support and it mainly requires OVMF support to make hypercalls for any decrypted memory regions it sets up, for example, MMIO and non-existant memory regions, so that hypervisor can keep track of guest firmware's page encryption status. So this is the full patch-set for UEFI for PSP based live migration. Thanks, Ashish > Thank you > Yao Jiewen > > > > -----Original Message----- > > From: devel@edk2.groups.io On Behalf Of Ashish Kalra > > via groups.io > > Sent: Monday, August 2, 2021 8:31 PM > > To: devel@edk2.groups.io > > Cc: dovmurik@linux.vnet.ibm.com; brijesh.singh@amd.com; tobin@ibm.com; > > Thomas.Lendacky@amd.com; jejb@linux.ibm.com; Justen, Jordan L > > ; ard.biesheuvel@arm.com; > > erdemaktas@google.com; Yao, Jiewen ; Xu, Min M > > > > Subject: [edk2-devel] [PATCH v6 0/6] SEV Live Migration support for OVMF. > > > > From: Ashish Kalra > > > > BZ: https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fbugzilla.tianocore.org%2Fshow_bug.cgi%3Fid%3D3467&data=04%7C01%7Cashish.kalra%40amd.com%7Cd4bf7bc0c17840d8a25208d957d04dcd%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637637374441109760%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=238zqkur1zpuKDorzZLrzcbTNhlO6gM8aFoi7Zr%2BUao%3D&reserved=0 > > > > By default all the SEV guest memory regions are considered encrypted, > > if a guest changes the encryption attribute of the page (e.g mark a > > page as decrypted) then notify hypervisor. Hypervisor will need to > > track the unencrypted pages. The information will be used during > > guest live migration, guest page migration and guest debugging. > > > > The patch-set detects if it is running under KVM hypervisor and then > > checks for SEV live migration feature support via KVM_FEATURE_CPUID, > > if detected setup a new UEFI enviroment variable to indicate OVMF > > support for SEV live migration. > > > > A branch containing these patches is available here: > > https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fashkalra%2Fedk2-1%2Ftree%2Fsev_live_migration_v5_10&data=04%7C01%7Cashish.kalra%40amd.com%7Cd4bf7bc0c17840d8a25208d957d04dcd%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637637374441109760%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=PyChejF9mzIVLCvQiCW5olQ8LOFWM3%2FGGO938qL80f8%3D&reserved=0 > > > > Changes since v5: > > - Split first patch into three components, one patch for the > > MemEncryptSevLiveMigrationIsEnabled() API, one patch for the > > SetMemoryEncDecHypercall3() API, one patch to make use of the > > SetMemoryEncDecHypercall3() API. > > - Fix patch subject, in code and patch comments and > > additionally add relevant comments. > > - Replace SetMemoryEncDecHypercall3() API's Status argument > > with a boolean IsEncrypted argument and corresponding fixes > > to users of this API call. > > - Fix AsciiStrCmp() usage in KVM hypervisor detection code. > > > > Changes since v4: > > - Remove MemEncryptHypercallLib Library and add support to issue > > hypercall in the BaseMemEncryptSevLib library itself. > > - For SEV-ES, make the VC handler hypercall aware by comparing > > the hypercall number and add the additional register values > > in the GHCB. > > - Fix comments in the hypercall API interface. > > - The encryption bit is set/clear on the smallest page size, hence > > use the 4k page size in MAP_GPA_RANGE hypercall. > > - Make the hypercall expect the guest physical address to be > > page-aligned. > > - Add KVM live migration feature flag check in BaseMemEncryptSevLib > > library similar to how BaseMemEncryptSevLib does for the > > MemEncryptSevIsEnabled() and check it before invoking HC. Also > > export the MemEncryptSevLiveMigrationIsEnabled() function as > > part of the library. > > - Add error handling on hypercall return, on failure, return error > > code to caller which potentially will cause an assert() and > > terminate the boot. > > > > Changes since v3: > > - Fix all DSC files under OvmfPkg except X64 to add support for > > BaseMemEncryptLib and add NULL instance of BaseMemEncryptLib > > for 32 bit platforms. > > - Add the MemEncryptHypercallLib-related files to Maintainers.txt, > > in section "OvmfPkg: Confidential Computing". > > - Add support for the new KVM_HC_MAP_GPA_RANGE hypercall interface. > > - Add patch for SEV live migration support. > > > > Changes since v2: > > - GHCB_BASE setup during reset-vector as decrypted is marked explicitly > > in the hypervisor page encryption bitmap after setting the > > PcdSevEsIsEnabled PCD. > > > > Changes since v1: > > - Mark GHCB_BASE setup during reset-vector as decrypted explicitly in > > the hypervisor page encryption bitmap. > > - Resending the series with correct shallow threading. > > > > Ashish Kalra (6): > > OvmfPkg/BaseMemEncryptLib: Detect SEV live migration feature. > > OvmfPkg/BaseMemEncryptLib: Hypercall API for page encryption state > > change > > OvmfPkg/BaseMemEncryptLib: Invoke page encryption state change > > hypercall > > OvmfPkg/VmgExitLib: Encryption state change hypercall support in VC > > handler > > OvmfPkg/PlatformPei: Mark SEC GHCB page as unencrypted via hypercall > > OvmfPkg/AmdSevDxe: Add support for SEV live migration. > > > > OvmfPkg/AmdSevDxe/AmdSevDxe.c | 64 +++++++++++++++++ > > OvmfPkg/AmdSevDxe/AmdSevDxe.inf | 4 ++ > > OvmfPkg/Include/Guid/AmdSevMemEncryptLib.h | 20 ++++++ > > OvmfPkg/Include/Library/MemEncryptSevLib.h | 70 +++++++++++++++++++ > > .../DxeMemEncryptSevLib.inf | 1 + > > .../DxeMemEncryptSevLibInternal.c | 39 +++++++++++ > > .../Ia32/MemEncryptSevLib.c | 27 +++++++ > > .../PeiDxeMemEncryptSevLibInternal.c | 52 ++++++++++++++ > > .../PeiMemEncryptSevLib.inf | 1 + > > .../PeiMemEncryptSevLibInternal.c | 39 +++++++++++ > > .../SecMemEncryptSevLibInternal.c | 38 ++++++++++ > > .../X64/AsmHelperStub.nasm | 33 +++++++++ > > .../X64/MemEncryptSevLib.c | 62 ++++++++++++++++ > > .../X64/PeiDxeVirtualMemory.c | 20 ++++++ > > OvmfPkg/Library/VmgExitLib/VmgExitVcHandler.c | 13 ++++ > > OvmfPkg/OvmfPkg.dec | 1 + > > OvmfPkg/PlatformPei/AmdSev.c | 11 +++ > > 17 files changed, 495 insertions(+) > > create mode 100644 OvmfPkg/Include/Guid/AmdSevMemEncryptLib.h > > create mode 100644 > > OvmfPkg/Library/BaseMemEncryptSevLib/X64/AsmHelperStub.nasm > > > > -- > > 2.17.1 > > > > > > > > > > >