From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from NAM02-SN1-obe.outbound.protection.outlook.com (NAM02-SN1-obe.outbound.protection.outlook.com [40.107.96.76]) by mx.groups.io with SMTP id smtpd.web09.29369.1626708614895323710 for ; Mon, 19 Jul 2021 08:30:15 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=@amd.com header.s=selector1 header.b=qRVUHjjR; spf=permerror, err=parse error for token &{10 18 %{i}._ip.%{h}._ehlo.%{d}._spf.vali.email}: invalid domain name (domain: amd.com, ip: 40.107.96.76, mailfrom: ashish.kalra@amd.com) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=K9fX5rEXFaxT+JX6E4zyyn18P8WoZU0b6MGB12EjUf8vDL6euHZxCk6lHwBPphNeDzFSiTARnbJ8uMng1x9E7fb97LO2my0lrJ7zzrXlYA2Oi6qo+Bh3ipYoRRr566/mGDTZ9F6r9D5nyXkTdzTmknkhvSx1/PofdZXoYG1dQXmn1ivx95bIJLf0uT3SdoOSZ0X9jxhoa75CiTv8pSP2ToYWVU+nVFaDLVt5WxW6b09gV8DK9qgt+hRnB7jPKQ9s//QfG3LAjXGhqvKFzXoRrwcC0QSvqDJjg8e/ZXS9dOtEdmgMr6Gr940PIhyA3GOjGnGIetDStPLoTSvdVNGpdg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=ouz1nj4YFQHLrB4jWxTnkWOIYCSei6aaXRSI0crasrk=; b=JCcQXT7beSsVq9zMHj4UvKc94iNXkbc7Ue/NQ8XjXRe2SJU7Wit2Q1qTCzEm6nYf/u1ziKqKwVaMcN0sECV6ajB2sGtKI/3/CDbb6ElwCXr5VtaiZiM/8txfysAgldx/QxN6Cv8OjuXL5M4GSBeXNK5XbZUc3pv59WbU2WIcpq7TL6kpqes6e2y/XOMSBAGWmYna+wpRc+PLo7lz6m6Bg+0C4ibnsdjdUlKYyUP/h/YCRD2w4wNViQ0LJECxHQC5Q8wPWEn8ThdmrT3ilXjqrksaqS+QGKucsyJBOt54GRP0sgRgNlqPYcVgYwswm2ifmqxRRg+D//4tlpsj7L1GrA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=amd.com; dmarc=pass action=none header.from=amd.com; dkim=pass header.d=amd.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=ouz1nj4YFQHLrB4jWxTnkWOIYCSei6aaXRSI0crasrk=; b=qRVUHjjRCXmGt4Kw3XSbvvv2Ou6v0P+tidnsNTblzXPV9RqicSjEISJpUbEqmFlUKc8OFacz0Dk0A/pGJ8gxSnuNqU6rNVJngnPSOk4WEGualKeFsIid6o+Sz0iW5/KOzEoiSQ97j4TvIULzPQP70QkktGPR+U7kzlFuvDwhK08= Authentication-Results: linux.ibm.com; dkim=none (message not signed) header.d=none;linux.ibm.com; dmarc=none action=none header.from=amd.com; Received: from SN6PR12MB2767.namprd12.prod.outlook.com (2603:10b6:805:75::23) by SN6PR12MB2719.namprd12.prod.outlook.com (2603:10b6:805:6c::12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4331.21; Mon, 19 Jul 2021 15:30:12 +0000 Received: from SN6PR12MB2767.namprd12.prod.outlook.com ([fe80::e8b2:38db:240f:b3ec]) by SN6PR12MB2767.namprd12.prod.outlook.com ([fe80::e8b2:38db:240f:b3ec%7]) with mapi id 15.20.4331.033; Mon, 19 Jul 2021 15:30:12 +0000 Date: Mon, 19 Jul 2021 15:30:09 +0000 From: "Ashish Kalra" To: Dov Murik Cc: devel@edk2.groups.io, dovmurik@linux.vnet.ibm.com, brijesh.singh@amd.com, tobin@ibm.com, Thomas.Lendacky@amd.com, jejb@linux.ibm.com, lersek@redhat.com, jordan.l.justen@intel.com, ard.biesheuvel@arm.com, erdemaktas@google.com, jiewen.yao@intel.com, min.m.xu@intel.com Subject: Re: [PATCH v5 1/4] OvmfPkg/BaseMemEncryptLib: Support to issue unencrypted hypercall Message-ID: <20210719153009.GB24210@ashkalra_ubuntu_server> References: <332172b262929880ef753a3bef36228115b7051a.1625687246.git.ashish.kalra@amd.com> <20210716122925.GA17576@ashkalra_ubuntu_server> <11cc351b-c1c6-f33c-5000-335125b098c8@linux.ibm.com> In-Reply-To: <11cc351b-c1c6-f33c-5000-335125b098c8@linux.ibm.com> User-Agent: Mutt/1.9.4 (2018-02-28) X-ClientProxiedBy: SA0PR11CA0112.namprd11.prod.outlook.com (2603:10b6:806:d1::27) To SN6PR12MB2767.namprd12.prod.outlook.com (2603:10b6:805:75::23) Return-Path: ashish.kalra@amd.com MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from ashkalra_ubuntu_server (165.204.77.1) by SA0PR11CA0112.namprd11.prod.outlook.com (2603:10b6:806:d1::27) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4331.21 via Frontend Transport; Mon, 19 Jul 2021 15:30:11 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 9a312971-b33c-4df2-3916-08d94aca19ac X-MS-TrafficTypeDiagnostic: SN6PR12MB2719: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:4502; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: TkJr9rfHll721gQe9f2XUyLf0sy2ldjV4kgcOievAMprxXG5TLsyg/6blJAD7q25fUjUqqXDGfKCn5VgGad+ZqGkvGVg88OsS0a1YF8NLANiBVw/sJhoifSRurHmXBftaeIev5OXfDPc+7Z434jjJY2n/+z6Sf1bULgrcbN4LqnjxMEfKRNxbZJBNOH6CMY4XMlBDvSSdZSNR1kuy9PFTw1D+NenDbA3/hq2Pqk0K9u7i2iugKKtscdBfBQkdya78dtwm/p8Tn+h2/o/+E0GBo8XS1jQ4Q0kPyQM+UjJ25RrlTEkgv2f1T6xMmZvnnCqpi0br4PSJHF+ywYNL7CUAkktO8nzQw2uqVUm+uj6M/bQ2GmVHOr5warA6JcoigmQ27EwXA5HQiPyoxvb9ZshUuNzXCMRaojgUkFY5xlLNaQ1wX6OJm2relHw8AVzQSFaCpYIs0nC2Y64iAJaL8e1iu6u4Vz8AurZ8QbbCD1Boblw0Xs1/DIBFQzkE0Jm+mtdZo6YGPlqUnnDCSGsslb+6BQDIxjH/+Gt5m8V/Xw5v/qW5fmiyeyuV2ELPpzgt9QpmH9Ecf03KSfw7gk+cBPOPTGXVN3YbkRVjZNgHzjJwhjSD4ZEhKSZqf0Y/98soWNGkUJbyUC6sV9bpLOKik+NQIRCiMFBeBo9OAvlj38Tn7SWvCjEgn9ARhye7Y8b5zn2fyd2C2+X8lEiNbw1F8Lsug== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:SN6PR12MB2767.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(396003)(376002)(346002)(366004)(39860400002)(136003)(44832011)(30864003)(956004)(55016002)(7416002)(5660300002)(316002)(19627235002)(66556008)(66946007)(33656002)(66476007)(4326008)(1076003)(8936002)(9686003)(478600001)(38100700002)(6496006)(186003)(26005)(6916009)(2906002)(53546011)(52116002)(86362001)(8676002)(83380400001)(38350700002)(33716001);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?wDpAoOLOfQPoeh7hcAhxC0c1Geskuw0W5I0p/AioFMstFl1+IXrH3n1o33+5?= =?us-ascii?Q?8zKquFzwbzFZVTfpjQDMIpwezD0cqhC8b/csQoIScRQPZ8m/jQZVykHvePun?= =?us-ascii?Q?xfJDXLxSLVnAR03AFA8Wg9XerjrB2hj3acY4lPMH0NLYWCYHFRNQB/Kn7csK?= =?us-ascii?Q?yTSxZTQ8u9qfAE3ITsa4c7bsSllbj1/7rOtLa0Z+o/rtsDpyO0F9Bt3QDHhc?= =?us-ascii?Q?u0Djv6l3STeEd8rNLUD/a3Idm+hTJYPVAuwVDdUNDqOc6rK9ZLhwqbp+tEqW?= =?us-ascii?Q?xp1AG492tPweXjYI0XfqUviRpJXGcYzfW72p3Vw8wEmg0IhPDBIHFDGbl/C0?= =?us-ascii?Q?qfx87t6B5iBa/IXkzn00Fyn97uQrGR74974BXm3XYWgxPO3JSk5eSbP5v6Ai?= =?us-ascii?Q?nbuXKyjlKavQFEJMTkBF1y2sf9SPI1QTZsVegft3FdxaBRKOeiWOHpDBoEhS?= =?us-ascii?Q?bdVfHm6lvrZR/DgZzqUEiXFr+D+V0HUM1KtaRLj1RREGZ6vzozEom7Ve/Ozo?= =?us-ascii?Q?n2Y3EMmQmmceWXP4OJb7hS+IFrP/GBdyvJHNnyMzXWaEfpT/GY6IbFapCDNT?= =?us-ascii?Q?YpjSwA0mTzTvuan80Eyka3c/ZIHVeygUKY0fmQe42hOfnRy2VJUoLYpo1Zw7?= =?us-ascii?Q?RI9cZGoe7GtALn5xLTspCKrnnOPXJDrr6BuTNZBl8fF0B2ZnZifH6KvH1rYz?= =?us-ascii?Q?hYGPCsDoHIcI//ejNyTB//24jNLHv6xPRK/TebrYwXQm7vZimJxXfEwkfNcV?= =?us-ascii?Q?AwfcH1y98KsC5blW/DlrQhuawlF23MHhnwr7KtG5E9gCJFEjuXE89PD/uPp7?= =?us-ascii?Q?2Cxh8KMDAlp3IKc68ozJ2HV/J4LvJXasqzanYAuiP+o509lJc5hYw8+QtDg/?= =?us-ascii?Q?ayDdrGZ6nAPa35gxPqoVn99arjt32xPP9ARdMi0f8xMVYH81GuMF/2+MDTR+?= =?us-ascii?Q?p7eJHP2AZocilwXxY2nHW93ZKJ73wnIkDNyw6gatAF/vLa7Q+FggB5c23FCz?= =?us-ascii?Q?+5XRZYPdTRqefc/UcxgxtuzYMbUv1j/Fh6C72Rjp6JnA830hblDe3K81LzAD?= =?us-ascii?Q?lt7YH7eYjLRuPImk6IEm3QqgdvDszXFP8vQR1SjFef6+frs5ptW99GaQN3UP?= =?us-ascii?Q?2fHX+SnRkuxBpqBaQgOfC8wQoSphjLxOxr6R87KAne+9pGX6/MpQcGyAM9oN?= =?us-ascii?Q?va3jT3jEsmtlM2BsjhEIX4FExnDy+Hc4p1iMu6yGp7BBI9RW59by31o00uj6?= =?us-ascii?Q?nDfH+zp2vXBO4llnTO8fLnuAS/e1mQBQ+46CR/j0/QKvZ1EHutii4jcqHxkq?= =?us-ascii?Q?EgOlPtp421Iy3zEwylr7c9Ik?= X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-Network-Message-Id: 9a312971-b33c-4df2-3916-08d94aca19ac X-MS-Exchange-CrossTenant-AuthSource: SN6PR12MB2767.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 19 Jul 2021 15:30:12.6710 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 3jFRrDona/DYlAXHtzXxUhs6tTtepbvlV9RmadM0/lR5ugfZ5XI0QSOHhoP4jB/cUFYtBU6zCFdh/Ue7m9foWw== X-MS-Exchange-Transport-CrossTenantHeadersStamped: SN6PR12MB2719 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Hello Dov, On Mon, Jul 19, 2021 at 11:04:17AM +0300, Dov Murik wrote: > > > On 16/07/2021 15:29, Ashish Kalra wrote: > > Hello Dov, > > > > On Thu, Jul 15, 2021 at 11:58:17PM +0300, Dov Murik wrote: > >> Hi Ashish, > >> > >> On 08/07/2021 17:07, Ashish Kalra wrote: > >>> From: Ashish Kalra > >>> > >>> By default all the SEV guest memory regions are considered encrypted, > >>> if a guest changes the encryption attribute of the page (e.g mark a > >>> page as decrypted) then notify hypervisor. Hypervisor will need to > >>> track the unencrypted pages. The information will be used during > >>> guest live migration, guest page migration and guest debugging. > >>> > >>> This hypercall is used to notify hypervisor when the page's > >>> encryption state changes. > >>> > >>> Cc: Jordan Justen > >>> Cc: Laszlo Ersek > >>> Cc: Ard Biesheuvel > >>> Signed-off-by: Brijesh Singh > >>> Signed-off-by: Ashish Kalra > >>> --- > >>> OvmfPkg/Include/Library/MemEncryptSevLib.h | 69 ++++++++++++++++++++ > >>> OvmfPkg/Library/BaseMemEncryptSevLib/DxeMemEncryptSevLib.inf | 1 + > >>> OvmfPkg/Library/BaseMemEncryptSevLib/DxeMemEncryptSevLibInternal.c | 39 +++++++++++ > >>> OvmfPkg/Library/BaseMemEncryptSevLib/Ia32/MemEncryptSevLib.c | 27 ++++++++ > >>> OvmfPkg/Library/BaseMemEncryptSevLib/PeiDxeMemEncryptSevLibInternal.c | 51 +++++++++++++++ > >>> OvmfPkg/Library/BaseMemEncryptSevLib/PeiMemEncryptSevLib.inf | 1 + > >>> OvmfPkg/Library/BaseMemEncryptSevLib/PeiMemEncryptSevLibInternal.c | 39 +++++++++++ > >>> OvmfPkg/Library/BaseMemEncryptSevLib/SecMemEncryptSevLibInternal.c | 38 +++++++++++ > >>> OvmfPkg/Library/BaseMemEncryptSevLib/X64/AsmHelperStub.nasm | 33 ++++++++++ > >>> OvmfPkg/Library/BaseMemEncryptSevLib/X64/MemEncryptSevLib.c | 54 +++++++++++++++ > >>> OvmfPkg/Library/BaseMemEncryptSevLib/X64/PeiDxeVirtualMemory.c | 22 ++++++- > >>> 11 files changed, 373 insertions(+), 1 deletion(-) > >>> > >>> diff --git a/OvmfPkg/Include/Library/MemEncryptSevLib.h b/OvmfPkg/Include/Library/MemEncryptSevLib.h > >>> index 76d06c206c..c2b2a99a08 100644 > >>> --- a/OvmfPkg/Include/Library/MemEncryptSevLib.h > >>> +++ b/OvmfPkg/Include/Library/MemEncryptSevLib.h > >>> @@ -90,6 +90,18 @@ MemEncryptSevIsEnabled ( > >>> VOID > >>> ); > >>> > >>> +/** > >>> + Returns a boolean to indicate whether SEV live migration is enabled. > >>> + > >>> + @retval TRUE SEV live migration is enabled > >>> + @retval FALSE SEV live migration is not enabled > >>> +**/ > >>> +BOOLEAN > >>> +EFIAPI > >>> +MemEncryptSevLiveMigrationIsEnabled ( > >>> + VOID > >>> + ); > >>> + > >>> /** > >>> This function clears memory encryption bit for the memory region specified by > >>> BaseAddress and NumPages from the current page table context. > >>> @@ -222,4 +234,61 @@ MemEncryptSevClearMmioPageEncMask ( > >>> IN UINTN NumPages > >>> ); > >>> > >>> +/** > >>> + This hypercall is used to notify hypervisor when the page's encryption > >>> + state changes. > >>> + > >>> + @param[in] PhysicalAddress The physical address that is the start address > >>> + of a memory region. The PhysicalAddress is > >>> + expected to be PAGE_SIZE aligned. > >>> + @param[in] Pages Number of pages in memory region. > >>> + @param[in] Status Encrypted(1) or Decrypted(0). > >>> + > >>> +@retval RETURN_SUCCESS Hypercall returned success. > >>> +**/ > >>> +RETURN_STATUS > >>> +EFIAPI > >>> +SetMemoryEncDecHypercall3 ( > >>> + IN UINTN PhysicalAddress, > >>> + IN UINTN Pages, > >>> + IN UINTN Status > >>> + ); > >>> + > >>> +#define KVM_HC_MAP_GPA_RANGE 12 > >>> +#define KVM_MAP_GPA_RANGE_PAGE_SZ_4K 0 > >>> +#define KVM_MAP_GPA_RANGE_PAGE_SZ_2M BIT0 > >>> +#define KVM_MAP_GPA_RANGE_PAGE_SZ_1G BIT1 > >>> +#define KVM_MAP_GPA_RANGE_ENC_STAT(n) ((n) << 4) > >>> +#define KVM_MAP_GPA_RANGE_ENCRYPTED KVM_MAP_GPA_RANGE_ENC_STAT(1) > >>> +#define KVM_MAP_GPA_RANGE_DECRYPTED KVM_MAP_GPA_RANGE_ENC_STAT(0) > >>> + > >>> +#define KVM_FEATURE_MIGRATION_CONTROL BIT17 > >>> + > >>> +/** > >>> + Figures out if we are running inside KVM HVM and > >>> + KVM HVM supports SEV Live Migration feature. > >>> + > >>> + @retval TRUE SEV live migration is supported. > >>> + @retval FALSE SEV live migration is not supported. > >>> +**/ > >>> +BOOLEAN > >>> +EFIAPI > >>> +KvmDetectSevLiveMigrationFeature( > >>> + VOID > >>> + ); > >>> + > >>> +/** > >>> + Interface exposed by the ASM implementation of the core hypercall > >>> + > >>> + @retval Hypercall returned status. > >>> +**/ > >>> +UINTN > >>> +EFIAPI > >>> +SetMemoryEncDecHypercall3AsmStub ( > >>> + IN UINTN HypercallNum, > >>> + IN UINTN PhysicalAddress, > >>> + IN UINTN Pages, > >>> + IN UINTN Attributes > >>> + ); > >>> + > >>> #endif // _MEM_ENCRYPT_SEV_LIB_H_ > >>> diff --git a/OvmfPkg/Library/BaseMemEncryptSevLib/DxeMemEncryptSevLib.inf b/OvmfPkg/Library/BaseMemEncryptSevLib/DxeMemEncryptSevLib.inf > >>> index f2e162d680..0c28afadee 100644 > >>> --- a/OvmfPkg/Library/BaseMemEncryptSevLib/DxeMemEncryptSevLib.inf > >>> +++ b/OvmfPkg/Library/BaseMemEncryptSevLib/DxeMemEncryptSevLib.inf > >>> @@ -38,6 +38,7 @@ > >>> X64/PeiDxeVirtualMemory.c > >>> X64/VirtualMemory.c > >>> X64/VirtualMemory.h > >>> + X64/AsmHelperStub.nasm > >>> > >>> [Sources.IA32] > >>> Ia32/MemEncryptSevLib.c > >>> diff --git a/OvmfPkg/Library/BaseMemEncryptSevLib/DxeMemEncryptSevLibInternal.c b/OvmfPkg/Library/BaseMemEncryptSevLib/DxeMemEncryptSevLibInternal.c > >>> index 2816f859a0..ead754cd7b 100644 > >>> --- a/OvmfPkg/Library/BaseMemEncryptSevLib/DxeMemEncryptSevLibInternal.c > >>> +++ b/OvmfPkg/Library/BaseMemEncryptSevLib/DxeMemEncryptSevLibInternal.c > >>> @@ -20,6 +20,8 @@ > >>> STATIC BOOLEAN mSevStatus = FALSE; > >>> STATIC BOOLEAN mSevEsStatus = FALSE; > >>> STATIC BOOLEAN mSevStatusChecked = FALSE; > >>> +STATIC BOOLEAN mSevLiveMigrationStatus = FALSE; > >>> +STATIC BOOLEAN mSevLiveMigrationStatusChecked = FALSE; > >>> > >>> STATIC UINT64 mSevEncryptionMask = 0; > >>> STATIC BOOLEAN mSevEncryptionMaskSaved = FALSE; > >>> @@ -87,6 +89,24 @@ InternalMemEncryptSevStatus ( > >>> mSevStatusChecked = TRUE; > >>> } > >>> > >>> +/** > >>> + Figures out if we are running inside KVM HVM and > >>> + KVM HVM supports SEV Live Migration feature. > >>> +**/ > >>> +STATIC > >>> +VOID > >>> +EFIAPI > >>> +InternalDetectSevLiveMigrationFeature( > >>> + VOID > >>> + ) > >>> +{ > >>> + if (KvmDetectSevLiveMigrationFeature()) { > >>> + mSevLiveMigrationStatus = TRUE; > >>> + } > >>> + > >>> + mSevLiveMigrationStatusChecked = TRUE; > >>> +} > >>> + > >>> /** > >>> Returns a boolean to indicate whether SEV-ES is enabled. > >>> > >>> @@ -125,6 +145,25 @@ MemEncryptSevIsEnabled ( > >>> return mSevStatus; > >>> } > >>> > >>> +/** > >>> + Returns a boolean to indicate whether SEV live migration is enabled. > >>> + > >>> + @retval TRUE SEV live migration is enabled > >>> + @retval FALSE SEV live migration is not enabled > >>> +**/ > >>> +BOOLEAN > >>> +EFIAPI > >>> +MemEncryptSevLiveMigrationIsEnabled ( > >>> + VOID > >>> + ) > >>> +{ > >>> + if (!mSevLiveMigrationStatusChecked) { > >>> + InternalDetectSevLiveMigrationFeature (); > >>> + } > >>> + > >>> + return mSevLiveMigrationStatus; > >>> +} > >>> + > >>> /** > >>> Returns the SEV encryption mask. > >>> > >>> diff --git a/OvmfPkg/Library/BaseMemEncryptSevLib/Ia32/MemEncryptSevLib.c b/OvmfPkg/Library/BaseMemEncryptSevLib/Ia32/MemEncryptSevLib.c > >>> index be260e0d10..62392309fe 100644 > >>> --- a/OvmfPkg/Library/BaseMemEncryptSevLib/Ia32/MemEncryptSevLib.c > >>> +++ b/OvmfPkg/Library/BaseMemEncryptSevLib/Ia32/MemEncryptSevLib.c > >>> @@ -136,3 +136,30 @@ MemEncryptSevClearMmioPageEncMask ( > >>> // > >>> return RETURN_UNSUPPORTED; > >>> } > >>> + > >>> +/** > >>> + This hyercall is used to notify hypervisor when the page's encryption > >>> + state changes. > >>> + > >>> + @param[in] PhysicalAddress The physical address that is the start address > >>> + of a memory region. The physical address is > >>> + expected to be PAGE_SIZE aligned. > >>> + @param[in] Pages Number of Pages in the memory region. > >>> + @param[in] Status Encrypted(1) or Decrypted(0). > >>> + > >>> +@retval RETURN_SUCCESS Hypercall returned success. > >>> +**/ > >>> +RETURN_STATUS > >>> +EFIAPI > >>> +SetMemoryEncDecHypercall3 ( > >>> + IN UINTN PhysicalAddress, > >>> + IN UINTN Pages, > >>> + IN UINTN Status > >>> + ) > >>> +{ > >>> + // > >>> + // Memory encryption bit is not accessible in 32-bit mode > >>> + // > >>> + return RETURN_UNSUPPORTED; > >>> +} > >>> + > >>> diff --git a/OvmfPkg/Library/BaseMemEncryptSevLib/PeiDxeMemEncryptSevLibInternal.c b/OvmfPkg/Library/BaseMemEncryptSevLib/PeiDxeMemEncryptSevLibInternal.c > >>> index b4a9f464e2..0c9f7e17ba 100644 > >>> --- a/OvmfPkg/Library/BaseMemEncryptSevLib/PeiDxeMemEncryptSevLibInternal.c > >>> +++ b/OvmfPkg/Library/BaseMemEncryptSevLib/PeiDxeMemEncryptSevLibInternal.c > >>> @@ -61,3 +61,54 @@ MemEncryptSevLocateInitialSmramSaveStateMapPages ( > >>> > >>> return RETURN_SUCCESS; > >>> } > >>> + > >>> +/** > >>> + Figures out if we are running inside KVM HVM and > >>> + KVM HVM supports SEV Live Migration feature. > >>> + > >>> + @retval TRUE SEV live migration is supported. > >>> + @retval FALSE SEV live migration is not supported. > >>> +**/ > >>> +BOOLEAN > >>> +EFIAPI > >>> +KvmDetectSevLiveMigrationFeature( > >>> + VOID > >>> + ) > >>> +{ > >>> + CHAR8 Signature[13]; > >>> + UINT32 mKvmLeaf; > >>> + UINT32 RegEax, RegEbx, RegEcx, RegEdx; > >>> + > >>> + Signature[12] = '\0'; > >>> + for (mKvmLeaf = 0x40000000; mKvmLeaf < 0x40010000; mKvmLeaf += 0x100) { > >>> + AsmCpuid (mKvmLeaf, > > Put the first argument on its own line. > Ok. > > >>> + NULL, > >>> + (UINT32 *) &Signature[0], > >>> + (UINT32 *) &Signature[4], > >>> + (UINT32 *) &Signature[8]); > >>> + > >>> + if (AsciiStrCmp ((CHAR8 *) Signature, "KVMKVMKVM\0\0\0") == 0) { > >> > >> I assume this will also match if Signature is "KVMKVMKVM\0YZ". I don't > >> know if that matters. > >> > > > > I don't understand what do you mean by "KVMKVMKVM\0YZ", this is > > comparing for "KVMKVMKVM\0\0\0" ? > > > > AsciiStrCmp will stop at the first '\0' char. So adding those three > '\0' at the end is pointless (the compiler will add one '\0' at the end > of a literal string). > > > Instead, you can use: > > if (CompareMem (Signature, "KVMKVMKVM\0\0\0", 12) == 0) > > and then you are sure to compare all 12 signature bytes. > > > I'm not sure this matters at all, maybe a simple: > > if (AsciiStrCmp (Signature, "KVMKVMKVM") == 0) > > is good enough. I see similar code to detect Xen in > OvmfPkg/XenPlatformPei/Xen.c . > > Yes, as with the Xen detection code, AsciiStrCmp(Signature, "KVMKVMKVM") should be good. > > > >>> + DEBUG (( > >>> + DEBUG_INFO, > >>> + "%a: KVM Detected, signature = %s\n", > > s/%s/%a/ > > (edk2 format strings are confusing.) > Ok. > >>> + __FUNCTION__, > >>> + Signature > >>> + )); > >>> + > >>> + RegEax = mKvmLeaf + 1; > >>> + RegEcx = 0; > >>> + AsmCpuid (mKvmLeaf + 1, &RegEax, &RegEbx, &RegEcx, &RegEdx); > >>> + if ((RegEax & KVM_FEATURE_MIGRATION_CONTROL) != 0) { > >>> + DEBUG (( > >>> + DEBUG_INFO, > >>> + "%a: Live Migration feature supported\n", > >> > >> I'd write: "%a: SEV Live Migration feature supported\n" > >> > > Ok. > >> > >>> + __FUNCTION__ > >>> + )); > >>> + > >>> + return TRUE; > >>> + } > >>> + } > >>> + } > >>> + > >>> + return FALSE; > >>> +} > >>> diff --git a/OvmfPkg/Library/BaseMemEncryptSevLib/PeiMemEncryptSevLib.inf b/OvmfPkg/Library/BaseMemEncryptSevLib/PeiMemEncryptSevLib.inf > >>> index 03a78c32df..3233ca7979 100644 > >>> --- a/OvmfPkg/Library/BaseMemEncryptSevLib/PeiMemEncryptSevLib.inf > >>> +++ b/OvmfPkg/Library/BaseMemEncryptSevLib/PeiMemEncryptSevLib.inf > >>> @@ -38,6 +38,7 @@ > >>> X64/PeiDxeVirtualMemory.c > >>> X64/VirtualMemory.c > >>> X64/VirtualMemory.h > >>> + X64/AsmHelperStub.nasm > >>> > >>> [Sources.IA32] > >>> Ia32/MemEncryptSevLib.c > >>> diff --git a/OvmfPkg/Library/BaseMemEncryptSevLib/PeiMemEncryptSevLibInternal.c b/OvmfPkg/Library/BaseMemEncryptSevLib/PeiMemEncryptSevLibInternal.c > >>> index e2fd109d12..9db6c2ef71 100644 > >>> --- a/OvmfPkg/Library/BaseMemEncryptSevLib/PeiMemEncryptSevLibInternal.c > >>> +++ b/OvmfPkg/Library/BaseMemEncryptSevLib/PeiMemEncryptSevLibInternal.c > >>> @@ -20,6 +20,8 @@ > >>> STATIC BOOLEAN mSevStatus = FALSE; > >>> STATIC BOOLEAN mSevEsStatus = FALSE; > >>> STATIC BOOLEAN mSevStatusChecked = FALSE; > >>> +STATIC BOOLEAN mSevLiveMigrationStatus = FALSE; > >>> +STATIC BOOLEAN mSevLiveMigrationStatusChecked = FALSE; > >>> > >>> STATIC UINT64 mSevEncryptionMask = 0; > >>> STATIC BOOLEAN mSevEncryptionMaskSaved = FALSE; > >>> @@ -87,6 +89,24 @@ InternalMemEncryptSevStatus ( > >>> mSevStatusChecked = TRUE; > >>> } > >>> > >>> +/** > >>> + Figures out if we are running inside KVM HVM and > >>> + KVM HVM supports SEV Live Migration feature. > >>> +**/ > >>> +STATIC > >>> +VOID > >>> +EFIAPI > >>> +InternalDetectSevLiveMigrationFeature( > >>> + VOID > >>> + ) > >>> +{ > >>> + if (KvmDetectSevLiveMigrationFeature()) { > >>> + mSevLiveMigrationStatus = TRUE; > >>> + } > >>> + > >>> + mSevLiveMigrationStatusChecked = TRUE; > >>> +} > >>> + > >>> /** > >>> Returns a boolean to indicate whether SEV-ES is enabled. > >>> > >>> @@ -125,6 +145,25 @@ MemEncryptSevIsEnabled ( > >>> return mSevStatus; > >>> } > >>> > >>> +/** > >>> + Returns a boolean to indicate whether SEV live migration is enabled. > >>> + > >>> + @retval TRUE SEV live migration is enabled > >>> + @retval FALSE SEV live migration is not enabled > >>> +**/ > >>> +BOOLEAN > >>> +EFIAPI > >>> +MemEncryptSevLiveMigrationIsEnabled ( > >>> + VOID > >>> + ) > >>> +{ > >>> + if (!mSevLiveMigrationStatusChecked) { > >>> + InternalDetectSevLiveMigrationFeature (); > >>> + } > >>> + > >>> + return mSevLiveMigrationStatus; > >>> +} > >>> + > >>> /** > >>> Returns the SEV encryption mask. > >>> > >>> diff --git a/OvmfPkg/Library/BaseMemEncryptSevLib/SecMemEncryptSevLibInternal.c b/OvmfPkg/Library/BaseMemEncryptSevLib/SecMemEncryptSevLibInternal.c > >>> index 56d8f3f318..b926c7b786 100644 > >>> --- a/OvmfPkg/Library/BaseMemEncryptSevLib/SecMemEncryptSevLibInternal.c > >>> +++ b/OvmfPkg/Library/BaseMemEncryptSevLib/SecMemEncryptSevLibInternal.c > >>> @@ -100,6 +100,44 @@ MemEncryptSevIsEnabled ( > >>> return Msr.Bits.SevBit ? TRUE : FALSE; > >>> } > >>> > >>> +/** > >>> + Interface exposed by the ASM implementation of the core hypercall > >>> + > >>> + @retval Hypercall returned status. > >>> +**/ > >>> +UINTN > >>> +EFIAPI > >>> +SetMemoryEncDecHypercall3AsmStub ( > >>> + IN UINTN HypercallNum, > >>> + IN UINTN PhysicalAddress, > >>> + IN UINTN Pages, > >>> + IN UINTN Attributes > >>> + ) > >>> +{ > >>> + // > >>> + // Not used in SEC phase. > >>> + // > >>> + return RETURN_UNSUPPORTED; > >>> +} > >>> + > >>> +/** > >>> + Returns a boolean to indicate whether SEV live migration is enabled. > >>> + > >>> + @retval TRUE SEV live migration is enabled > >>> + @retval FALSE SEV live migration is not enabled > >>> +**/ > >>> +BOOLEAN > >>> +EFIAPI > >>> +MemEncryptSevLiveMigrationIsEnabled ( > >>> + VOID > >>> + ) > >>> +{ > >>> + // > >>> + // Not used in SEC phase. > >>> + // > >>> + return FALSE; > >>> +} > >>> + > >>> /** > >>> Returns the SEV encryption mask. > >>> > >>> diff --git a/OvmfPkg/Library/BaseMemEncryptSevLib/X64/AsmHelperStub.nasm b/OvmfPkg/Library/BaseMemEncryptSevLib/X64/AsmHelperStub.nasm > >>> new file mode 100644 > >>> index 0000000000..c7c11f77f1 > >>> --- /dev/null > >>> +++ b/OvmfPkg/Library/BaseMemEncryptSevLib/X64/AsmHelperStub.nasm > >>> @@ -0,0 +1,33 @@ > >>> +/** @file > >>> + > >>> + ASM helper stub to invoke hypercall > >>> + > >>> + Copyright (c) 2021, AMD Incorporated. All rights reserved.
> >>> + > >>> + SPDX-License-Identifier: BSD-2-Clause-Patent > >>> + > >>> +**/ > >>> + > >>> +DEFAULT REL > >>> +SECTION .text > >>> + > >>> +; UINTN > >>> +; EFIAPI > >>> +; SetMemoryEncDecHypercall3AsmStub ( > >>> +; IN UINTN HypercallNum, > >>> +; IN UINTN Arg1, > >>> +; IN UINTN Arg2, > >>> +; IN UINTN Arg3 > >>> +; ); > >>> +global ASM_PFX(SetMemoryEncDecHypercall3AsmStub) > >>> +ASM_PFX(SetMemoryEncDecHypercall3AsmStub): > >>> + ; UEFI calling conventions require RBX to > >>> + ; be nonvolatile/callee-saved. > >>> + push rbx > >>> + mov rax, rcx ; Copy HypercallNumber to rax > >>> + mov rbx, rdx ; Copy Arg1 to the register expected by KVM > >>> + mov rcx, r8 ; Copy Arg2 to register expected by KVM > >>> + mov rdx, r9 ; Copy Arg2 to register expected by KVM > >> > >> Comment: s/Arg2/Arg3/ > >> > > Yes. > >>> + vmmcall ; Call VMMCALL > >>> + pop rbx > >>> + ret > >>> diff --git a/OvmfPkg/Library/BaseMemEncryptSevLib/X64/MemEncryptSevLib.c b/OvmfPkg/Library/BaseMemEncryptSevLib/X64/MemEncryptSevLib.c > >>> index a57e8fd37f..57447e69dc 100644 > >>> --- a/OvmfPkg/Library/BaseMemEncryptSevLib/X64/MemEncryptSevLib.c > >>> +++ b/OvmfPkg/Library/BaseMemEncryptSevLib/X64/MemEncryptSevLib.c > >>> @@ -143,3 +143,57 @@ MemEncryptSevClearMmioPageEncMask ( > >>> ); > >>> > >>> } > >>> + > >>> +/** > >>> + This hyercall is used to notify hypervisor when the page's encryption > >>> + state changes. > >>> + > >>> + @param[in] PhysicalAddress The physical address that is the start address > >>> + of a memory region. The physical address is > >>> + expected to be PAGE_SIZE aligned. > >>> + @param[in] Pages Number of Pages in the memory region. > >>> + @param[in] Status Encrypted(1) or Decrypted(0). > >>> + > >>> +@retval RETURN_SUCCESS Hypercall returned success. > >> > >> or RETURN_UNSUPPORTED or RETURN_NO_MAPPING. > >> > > > > Ok. > > > >>> +**/ > >>> +RETURN_STATUS > >>> +EFIAPI > >>> +SetMemoryEncDecHypercall3 ( > >>> + IN UINTN PhysicalAddress, > >>> + IN UINTN Pages, > >>> + IN UINTN Status > >> > >> Consider: > >> > >> IN BOOL IsEncrypted > >> > >> or: > >> > >> IN MAP_RANGE_MODE EncMode > >> > >> (it's not a Status in the EFI_STATUS sense that appears all around edk2). > >> > > Ok, i think i will prefer something like a MAP_RANGE_MODE. > >> > >>> + ) > >>> +{ > >>> + RETURN_STATUS Ret; > >>> + INTN Error; > >>> + > >> > >> Add assert for the expected alignment of PhysicalAddress, and then > >> you don't need to round it down when calling SetMemoryEncDecHypercall3AsmStub. > >> > > > > Cannot really use an assert here, as when the GCD map is being walked > > and the c-bit being cleared from MMIO and NonExistent memory spaces, the > > physical address range being passed may not be page-aligned, so adding > > an assert here prevents booting. Hence, rounding it down when calling > > SetMemoryEncDecHypercall3AsmStub below. > > > > OK. So fix the comment above the function (which says: "The physical > address is expected to be PAGE_SIZE aligned.") > Ok. > > >> > >>> + Ret = RETURN_UNSUPPORTED; > >>> + > >>> + if (MemEncryptSevLiveMigrationIsEnabled ()) { > >>> + Ret = EFI_SUCCESS; > >>> + // > >>> + // The encryption bit is set/clear on the smallest page size, hence > >>> + // use the 4k page size in MAP_GPA_RANGE hypercall below. > >>> + // Also, the hypercall expects the guest physical address to be > >>> + // page-aligned. > >>> + // > >>> + Error = SetMemoryEncDecHypercall3AsmStub ( > >>> + KVM_HC_MAP_GPA_RANGE, > >>> + (PhysicalAddress & (~(EFI_PAGE_SIZE-1))), > >> > >> Simpler: > >> > >> PhysicalAddress & ~EFI_PAGE_MASK > >> > >> > > > > Ok. > > > >>> + Pages, > >>> + KVM_MAP_GPA_RANGE_PAGE_SZ_4K | KVM_MAP_GPA_RANGE_ENC_STAT(Status) > >>> + ); > >>> + > >>> + if (Error != 0) { > >>> + DEBUG ((DEBUG_ERROR, > >>> + "SetMemoryEncDecHypercall3 failed, Phys = %Lx, Pages = %Ld, Err = %Ld\n", > >>> + PhysicalAddress, > >>> + Pages, > >>> + (INT64)Error)); > >>> + > >>> + Ret = RETURN_NO_MAPPING; > >>> + } > >>> + } > >>> + > >>> + return Ret; > >>> +} > >>> diff --git a/OvmfPkg/Library/BaseMemEncryptSevLib/X64/PeiDxeVirtualMemory.c b/OvmfPkg/Library/BaseMemEncryptSevLib/X64/PeiDxeVirtualMemory.c > >>> index c696745f9d..0b1588a4c1 100644 > >>> --- a/OvmfPkg/Library/BaseMemEncryptSevLib/X64/PeiDxeVirtualMemory.c > >>> +++ b/OvmfPkg/Library/BaseMemEncryptSevLib/X64/PeiDxeVirtualMemory.c > >>> @@ -536,7 +536,6 @@ EnableReadOnlyPageWriteProtect ( > >>> AsmWriteCr0 (AsmReadCr0() | BIT16); > >>> } > >>> > >>> - > >>> /** > >>> This function either sets or clears memory encryption bit for the memory > >>> region specified by PhysicalAddress and Length from the current page table > >>> @@ -585,6 +584,9 @@ SetMemoryEncDec ( > >>> UINT64 AddressEncMask; > >>> BOOLEAN IsWpEnabled; > >>> RETURN_STATUS Status; > >>> + UINTN Size; > >>> + BOOLEAN CBitChanged; > >>> + PHYSICAL_ADDRESS OrigPhysicalAddress; > >>> > >>> // > >>> // Set PageMapLevel4Entry to suppress incorrect compiler/analyzer warnings. > >>> @@ -636,6 +638,10 @@ SetMemoryEncDec ( > >>> > >>> Status = EFI_SUCCESS; > >>> > >>> + Size = Length; > >>> + CBitChanged = FALSE; > >>> + OrigPhysicalAddress = PhysicalAddress; > >>> + > >>> while (Length != 0) > >>> { > >>> // > >>> @@ -695,6 +701,7 @@ SetMemoryEncDec ( > >>> )); > >>> PhysicalAddress += BIT30; > >>> Length -= BIT30; > >>> + CBitChanged = TRUE; > >>> } else { > >>> // > >>> // We must split the page > >>> @@ -749,6 +756,7 @@ SetMemoryEncDec ( > >>> SetOrClearCBit (&PageDirectory2MEntry->Uint64, Mode); > >>> PhysicalAddress += BIT21; > >>> Length -= BIT21; > >>> + CBitChanged = TRUE; > >>> } else { > >>> // > >>> // We must split up this page into 4K pages > >>> @@ -791,6 +799,7 @@ SetMemoryEncDec ( > >>> SetOrClearCBit (&PageTableEntry->Uint64, Mode); > >>> PhysicalAddress += EFI_PAGE_SIZE; > >>> Length -= EFI_PAGE_SIZE; > >>> + CBitChanged = TRUE; > >>> } > >>> } > >>> } > >>> @@ -808,6 +817,17 @@ SetMemoryEncDec ( > >>> // > >>> CpuFlushTlb(); > >>> > >>> + // > >>> + // Notify Hypervisor on C-bit status > >>> + // > >>> + if (CBitChanged) { > >>> + Status = SetMemoryEncDecHypercall3 ( > >>> + OrigPhysicalAddress, > >>> + EFI_SIZE_TO_PAGES(Size), > >>> + !Mode > >> > >> Here you pass !Mode (which is 0 or 1) as the third argument to > >> SetMemoryEncDecHypercall3 . > >> > >> But on patch 3/4 you pass KVM_MAP_GPA_RANGE_DECRYPTED (which is 0<<4); > >> but that hints that you expect either 0<<4 or 1<<4 as this third argument. > >> If this is the case, then here it should be: > >> > >> (Mode == SetCBit) ? KVM_MAP_GPA_RANGE_ENCRYPTED : KVM_MAP_GPA_RANGE_DECRYPTED > >> > >> If it's the other way around, then patch 3/4 needs to pass a simple 0 as > >> the third argument. > >> > > > > Yes, i need to pass a 0 (decrypted) as the third argument in that patch. > > > > Even clearer than a literal 0 -- pass a ClearCBit as the third argument > (assuming you're changing the argument type to MAP_RANGE_MODE). > Ok. Thanks, Ashish > >> > >> > >>> + ); > >>> + } > >>> + > >>> Done: > >>> // > >>> // Restore page table write protection, if any. > >>>