From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from NAM11-CO1-obe.outbound.protection.outlook.com (NAM11-CO1-obe.outbound.protection.outlook.com [40.107.220.73]) by mx.groups.io with SMTP id smtpd.web12.6863.1626444688333427599 for ; Fri, 16 Jul 2021 07:11:28 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=@amd.com header.s=selector1 header.b=xpvZZC7/; spf=permerror, err=parse error for token &{10 18 %{i}._ip.%{h}._ehlo.%{d}._spf.vali.email}: invalid domain name (domain: amd.com, ip: 40.107.220.73, mailfrom: thomas.lendacky@amd.com) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=PaA0uLAH1BkvJYZZrCcVNSePyDs/rbGq167XnJ3q25dzISRqJcACOAEtHX0hoapIGUSox7KavbqTZd1OtP30otg+hh0uHpL9IZYJI7WgD5ydrEOzMZZWf0cIXs8zgnezmrh4uWJxtQMEEv6/dHuC4M6z7r89oNzc8kAHiAOqedl3DRKgxKjucIE7kKl5lU4VVAfGcaOXJCSZVmdPcDb2UNF8Dn/IVDJyNXhhfpmFXFoO15diHDJ9sK+0dzd2lECNxPLFfOM8H4YqQt9CL4Dr2v4rbSAzd2s9tLGscJ+XEu8q1s/VuvPuknFnl5NWTLP2Vs/vewsaNSoCvHJ/eS+g0Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=f7mYVWJOKZvX/rffUkSkK4j+E+OMaY4MSjuWtagsRgc=; b=KbSi6Fy5e0EWCgLJ0Rrde6ZlFWT71+1EA6EEI1KWeYifqX3rvErDGIldKICSSDnLH5TDpWdpa33PC9IMjXnae3p16UIL6rfBR1VxoDweaA54zoaAQWFOhnxd06VbzZY16wSXr4KtZREAEUJgVCkMqt9TTGP4lV8OyaXZkf+9+/BvBRfCUZ2pae86yk/Q5D+70cfeSiwh6Nt9fDdbmb9SbpM5JHNAim/nZz0VxQXI51CC4IaeW26SAAQ+fQsmZkbAVrxvLPYQmRmtk4uHAkPSTZEf3Iwwz9g+6986vvvZ2e0Sc8vxSNE/9Nnu17MFLSqkVqSQWvggdHjF3gPwPlVn8g== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=amd.com; dmarc=pass action=none header.from=amd.com; dkim=pass header.d=amd.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=f7mYVWJOKZvX/rffUkSkK4j+E+OMaY4MSjuWtagsRgc=; b=xpvZZC7/yPNT0QK93HSTMR6oV23usQuDD5aC6y9Da0PYz/AuFllx3fOEf5/f4b3Itv3JHw+BXiPymAhJTd3SCQ8y5mYDH5kWXyK9wf0Z7MiskqkDVWXGnB8xD9w4lqchPKBOSWwj4h+MZ3aTJKbJh9UNDvBRBN+ZADFs2g5Jt2s= Authentication-Results: intel.com; dkim=none (message not signed) header.d=none;intel.com; dmarc=none action=none header.from=amd.com; Received: from DM4PR12MB5229.namprd12.prod.outlook.com (2603:10b6:5:398::12) by DM4PR12MB5054.namprd12.prod.outlook.com (2603:10b6:5:389::24) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4331.21; Fri, 16 Jul 2021 14:11:26 +0000 Received: from DM4PR12MB5229.namprd12.prod.outlook.com ([fe80::73:2581:970b:3208]) by DM4PR12MB5229.namprd12.prod.outlook.com ([fe80::73:2581:970b:3208%3]) with mapi id 15.20.4331.026; Fri, 16 Jul 2021 14:11:26 +0000 Subject: Re: [PATCH v5 1/4] OvmfPkg/BaseMemEncryptLib: Support to issue unencrypted hypercall To: Ashish Kalra , devel@edk2.groups.io Cc: dovmurik@linux.vnet.ibm.com, brijesh.singh@amd.com, tobin@ibm.com, jejb@linux.ibm.com, lersek@redhat.com, jordan.l.justen@intel.com, ard.biesheuvel@arm.com, erdemaktas@google.com, jiewen.yao@intel.com, min.m.xu@intel.com References: <332172b262929880ef753a3bef36228115b7051a.1625687246.git.ashish.kalra@amd.com> From: "Lendacky, Thomas" Message-ID: <8a307c55-337d-b93f-b77c-a33846b8f965@amd.com> Date: Fri, 16 Jul 2021 09:11:23 -0500 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Thunderbird/78.8.1 In-Reply-To: <332172b262929880ef753a3bef36228115b7051a.1625687246.git.ashish.kalra@amd.com> X-ClientProxiedBy: SN7PR04CA0182.namprd04.prod.outlook.com (2603:10b6:806:126::7) To DM4PR12MB5229.namprd12.prod.outlook.com (2603:10b6:5:398::12) Return-Path: thomas.lendacky@amd.com MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from office-ryzen.texastahm.com (67.79.209.213) by SN7PR04CA0182.namprd04.prod.outlook.com (2603:10b6:806:126::7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4331.21 via Frontend Transport; Fri, 16 Jul 2021 14:11:25 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 0a25a428-193a-40b3-4e5b-08d94863994a X-MS-TrafficTypeDiagnostic: DM4PR12MB5054: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:10000; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: EaR6QbosIX2qKFD1C01w1r5K5e407NQB6UwVCLxWHt1ZeQ5/p5yAF4Q4c54X2MpUDhqFgd+9sJBgAFOHu9g3JTnhCsaX+VThieXk4GOfi9Dip3fJGz+cf32/JfQAuQLp5tlQLI3gaOWI7NEnb9ddRFuDINA0np0zMMu3wChHGlpKzBJeteR+lFw+SgJZcq4Helmg0S+EDmdPtmiJuDvlfzE65QvwmraVXpJez115S5SP2eBpKdCO3ziDhFzqDi6ey7xOPaV3TfYzY+pe37xG3XxvV3TYcaFThEKNbgIvqwDI81rpY3NZ023UpqVcon+9gjYArM/vHirLjBiyvZryDmsd9Ny4KuTBZq10M4iJRxh3pjgtGSSn3bOtR0msGRNKwJmWpWFaEALew0t0iU7EIRuAXYKil0BEve7i2UIEJRPnVHDPD6Owk+wCeSO72s1/EG3Qo5v2X0e26quDjKQPDl6vyERcuQezZa35An+Quq8zNZV4eUbpT4rGu3j1OuaXSiKl9BfMdJLvKYue6iPW3Y1X57lx/EguiItRqs303ERVTUg1Nmt6pMBmi0acbHItUI4hkK+a3clsLmVrIO+5Qg6whLaSFFG6HTD3oel4OERUT1Vx7UbBr/No1ohmf/Fqp7klhe+Ry9X+92MN85CNlFhhhdNgcFnqSlfUMGI60g3wY453ZyeBLSZ5U3fYgDCwo2XHkjHI6Pk06+LRhsgoCZzuJprm6tkUfcV5zScin9E83SEkbiqyMh788MJIeWgi X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DM4PR12MB5229.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(396003)(39860400002)(366004)(376002)(136003)(346002)(8936002)(8676002)(83380400001)(478600001)(26005)(316002)(2616005)(956004)(66556008)(186003)(53546011)(38100700002)(6506007)(66476007)(31696002)(66946007)(2906002)(6486002)(5660300002)(86362001)(7416002)(31686004)(19627235002)(30864003)(4326008)(36756003)(6512007)(45980500001)(43740500002);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?RlIzem44dlI0NHNzVGlUZUVhVzJ1QldER2xhVHhNN0o5YW9BKy9EYk53Z0Jl?= =?utf-8?B?NkNuSEQ1SE5LMXNKMDNDM1dyMUhJNlJJTFhpd2RGU1laMGRobFhNaG9HTC9x?= =?utf-8?B?V3Mxd3B0VFhJZjFGQ3ViQUJ4RVIxU2RQN01wQTVpdis5UzVDckN0SXJ5Y3Fa?= =?utf-8?B?eTlOdWwvUXdqNUJGdTJwRk14Ris4VnlHckJFbEU2YzNVUEIzam1JTjNHSEc5?= =?utf-8?B?Z0RibXJob0R0djVUNFpZTGVORms0SHNsbWVjS1B3T1AvZ1JPbG8vWnNmUHZZ?= =?utf-8?B?RjJ2YlMwbk5MWUJSd2xCdE02eWpMWWJ1NVhyWHpPc1daRzhsQjM3by82YnQ0?= =?utf-8?B?Um1RVXBnK01JYmxJK3ExbFVwMDBBdk9uMXJwbDdBdFNXQ3A2dFRZdWR3cnEx?= =?utf-8?B?VFIwRzRvYVh1U3gzeTNYNXpHdnBHWFBhbUxMaWdScnlJV0ZoamlGL3IwdGU0?= =?utf-8?B?RHBZenJSY0pRMy9Nc091alBvdUE3enVJa3dJODltVkZzRExMU3JZL1Bvc2Zo?= =?utf-8?B?bTY4cHNleHdCYXl2RkdlRHlyMjA3RkZHR3Y4anBockF5REtRKzlzV1lPRzhw?= =?utf-8?B?ZjFUZGl4d2ordDUxVzR3dEllRzc0VXpGcXpXR3owQVFnYlVCcVRwSWx1bHpy?= =?utf-8?B?enViSTM4eGVVenZWUXFlN3FBOVJOSzhrcklVWGx4VjZYVEpBdUxhckUxQlUx?= =?utf-8?B?YVJYbHpObktPbkt2dHpmMGgxWEdRTU1Tc2Zxb0R4VHdGNVVtUXM4RHM0T2tp?= =?utf-8?B?bmxta0xOa29xWVE1dHI2S1FLb0VWaklMNnhnbHBNdjQ2YmJhRmIyZ2s4cE5W?= =?utf-8?B?UjcrazJwdWEyNnRQaDFFYjVWSGFEVXY2MVdyVFpJRmYyVUZ5bW84M2Jtd0xH?= =?utf-8?B?akpzaHAxdWZPaXRuRUZjT1orelJJUUNGK2ZxUmVLZFZtb0dvaisvVlBzMzg2?= =?utf-8?B?bnJZUEgwVkdMZklRb0tJUFMyeFVBUmVpNFZQM1EzUm5vRHh4MGsydTJ6d2lR?= =?utf-8?B?UmY4Mnl3eHc0YXFjUHFGbGRJMi9jeWtPQTVxS3paQjBwWlBpS09wTXBtZUhS?= =?utf-8?B?cmFVaDBTeHQzbjRCcXd4Tk1EZCtFbjVQY1NkNVg4Z0pKd3NTMlVYY05BUkRM?= =?utf-8?B?MHVoaTV2ajkxdXd3em1ub0tlbUNLTlVWbXpVaVh1L2ZLTDNqNHpBajYxeVVh?= =?utf-8?B?N2xUalNVdlhzVWZsQVlSOXNrVGN6a285L2k5NzUwWjJtMlhzRlphRGxISUVk?= =?utf-8?B?ZjV3OHh3MXZyazFscktRYjNyd08vZHRaUkYxaWJSM1l4aTA4WWdDWU1SM0hC?= =?utf-8?B?dnFsNFZIOFNQZFYweHZYUmxEdXpISXVUamFBeVdmQktGV2xxRTVwbDNQbTRt?= =?utf-8?B?TEw0Y080QjdibUR1WHFqVkl6NTdTWHo2VkdnalJCa1FlbFVjTTBNMC84ZkRT?= =?utf-8?B?c0ZBWU5tdVRidHB3TFZlWlB6a0U4MVdqMHJzMFhUYzhsNDV0WTNzZWFoMHRE?= =?utf-8?B?SjBjamlBZXpJei9qcGdCQ1psc2YvNE9uN1pQYklNa0xBM0ZmWUxHeVhsK2RG?= =?utf-8?B?dmUwZ0xPSzMweW1pSGpyRnZ4TjRiN3ppOXBjWFpSQjBtc3Y4MFM3QmJzM1Js?= =?utf-8?B?ODNMY2pVQ2VWZm1LKytFbHNEZDRERENYT3NVbFRxbnF6NnI2N1hOWjJvQzhw?= =?utf-8?B?QXc0NzV2L3RYV2hXZ2FRNnlDUkhCL3d4VTd2cHdmaUNGOVJOUkJ6Yk01UElP?= =?utf-8?Q?T0ZdoxMEQVuHcM5cViXqa9MOdeDdQhb4zzqgNrU?= X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-Network-Message-Id: 0a25a428-193a-40b3-4e5b-08d94863994a X-MS-Exchange-CrossTenant-AuthSource: DM4PR12MB5229.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 16 Jul 2021 14:11:26.2661 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 7Y12UlnmzaHFCdFeQfgeSp8O0GkpJngqtPxirtqQNp7HwEElRGEktJnZQRoHKSP5ynBkXCqSV/4X1/i+UWIf+Q== X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM4PR12MB5054 Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 7bit On 7/8/21 9:07 AM, Ashish Kalra wrote: > From: Ashish Kalra > The patch subject is a bit confusing. Something more like "Add API to issue hypercall on page encryption state change" or similar, since this is issued for changes to shared and private, not just shared. > By default all the SEV guest memory regions are considered encrypted, > if a guest changes the encryption attribute of the page (e.g mark a > page as decrypted) then notify hypervisor. Hypervisor will need to > track the unencrypted pages. The information will be used during > guest live migration, guest page migration and guest debugging. > > This hypercall is used to notify hypervisor when the page's > encryption state changes. This is a large patch. It looks like this should be split into a few patches. - one patch for the MemEncryptSevLiveMigrationIsEnabled() API - one patch for the SetMemoryEncDecHypercall3() API - one patch to make use of the SetMemoryEncDecHypercall3() API. > > Cc: Jordan Justen > Cc: Laszlo Ersek > Cc: Ard Biesheuvel > Signed-off-by: Brijesh Singh > Signed-off-by: Ashish Kalra > --- > OvmfPkg/Include/Library/MemEncryptSevLib.h | 69 ++++++++++++++++++++ > OvmfPkg/Library/BaseMemEncryptSevLib/DxeMemEncryptSevLib.inf | 1 + > OvmfPkg/Library/BaseMemEncryptSevLib/DxeMemEncryptSevLibInternal.c | 39 +++++++++++ > OvmfPkg/Library/BaseMemEncryptSevLib/Ia32/MemEncryptSevLib.c | 27 ++++++++ > OvmfPkg/Library/BaseMemEncryptSevLib/PeiDxeMemEncryptSevLibInternal.c | 51 +++++++++++++++ > OvmfPkg/Library/BaseMemEncryptSevLib/PeiMemEncryptSevLib.inf | 1 + > OvmfPkg/Library/BaseMemEncryptSevLib/PeiMemEncryptSevLibInternal.c | 39 +++++++++++ > OvmfPkg/Library/BaseMemEncryptSevLib/SecMemEncryptSevLibInternal.c | 38 +++++++++++ > OvmfPkg/Library/BaseMemEncryptSevLib/X64/AsmHelperStub.nasm | 33 ++++++++++ > OvmfPkg/Library/BaseMemEncryptSevLib/X64/MemEncryptSevLib.c | 54 +++++++++++++++ > OvmfPkg/Library/BaseMemEncryptSevLib/X64/PeiDxeVirtualMemory.c | 22 ++++++- > 11 files changed, 373 insertions(+), 1 deletion(-) > > diff --git a/OvmfPkg/Include/Library/MemEncryptSevLib.h b/OvmfPkg/Include/Library/MemEncryptSevLib.h > index 76d06c206c..c2b2a99a08 100644 > --- a/OvmfPkg/Include/Library/MemEncryptSevLib.h > +++ b/OvmfPkg/Include/Library/MemEncryptSevLib.h > @@ -90,6 +90,18 @@ MemEncryptSevIsEnabled ( > VOID > ); > > +/** > + Returns a boolean to indicate whether SEV live migration is enabled. > + > + @retval TRUE SEV live migration is enabled > + @retval FALSE SEV live migration is not enabled > +**/ > +BOOLEAN > +EFIAPI > +MemEncryptSevLiveMigrationIsEnabled ( > + VOID > + ); > + > /** > This function clears memory encryption bit for the memory region specified by > BaseAddress and NumPages from the current page table context. > @@ -222,4 +234,61 @@ MemEncryptSevClearMmioPageEncMask ( > IN UINTN NumPages > ); > > +/** > + This hypercall is used to notify hypervisor when the page's encryption > + state changes. > + > + @param[in] PhysicalAddress The physical address that is the start address > + of a memory region. The PhysicalAddress is > + expected to be PAGE_SIZE aligned. > + @param[in] Pages Number of pages in memory region. > + @param[in] Status Encrypted(1) or Decrypted(0). > + > +@retval RETURN_SUCCESS Hypercall returned success. It looks like RETURN_UNSUPPORTED is also possible. > +**/ > +RETURN_STATUS > +EFIAPI > +SetMemoryEncDecHypercall3 ( > + IN UINTN PhysicalAddress, > + IN UINTN Pages, > + IN UINTN Status > + ); > + > +#define KVM_HC_MAP_GPA_RANGE 12 > +#define KVM_MAP_GPA_RANGE_PAGE_SZ_4K 0 > +#define KVM_MAP_GPA_RANGE_PAGE_SZ_2M BIT0 > +#define KVM_MAP_GPA_RANGE_PAGE_SZ_1G BIT1 > +#define KVM_MAP_GPA_RANGE_ENC_STAT(n) ((n) << 4) > +#define KVM_MAP_GPA_RANGE_ENCRYPTED KVM_MAP_GPA_RANGE_ENC_STAT(1) > +#define KVM_MAP_GPA_RANGE_DECRYPTED KVM_MAP_GPA_RANGE_ENC_STAT(0) You define these but don't use them (and you should). > + > +#define KVM_FEATURE_MIGRATION_CONTROL BIT17 > + > +/** > + Figures out if we are running inside KVM HVM and > + KVM HVM supports SEV Live Migration feature. > + > + @retval TRUE SEV live migration is supported. > + @retval FALSE SEV live migration is not supported. > +**/ > +BOOLEAN > +EFIAPI > +KvmDetectSevLiveMigrationFeature( > + VOID > + ); > + > +/** > + Interface exposed by the ASM implementation of the core hypercall > + > + @retval Hypercall returned status. > +**/ > +UINTN > +EFIAPI > +SetMemoryEncDecHypercall3AsmStub ( > + IN UINTN HypercallNum, > + IN UINTN PhysicalAddress, > + IN UINTN Pages, > + IN UINTN Attributes > + ); > + > #endif // _MEM_ENCRYPT_SEV_LIB_H_ > diff --git a/OvmfPkg/Library/BaseMemEncryptSevLib/DxeMemEncryptSevLib.inf b/OvmfPkg/Library/BaseMemEncryptSevLib/DxeMemEncryptSevLib.inf > index f2e162d680..0c28afadee 100644 > --- a/OvmfPkg/Library/BaseMemEncryptSevLib/DxeMemEncryptSevLib.inf > +++ b/OvmfPkg/Library/BaseMemEncryptSevLib/DxeMemEncryptSevLib.inf > @@ -38,6 +38,7 @@ > X64/PeiDxeVirtualMemory.c > X64/VirtualMemory.c > X64/VirtualMemory.h > + X64/AsmHelperStub.nasm > > [Sources.IA32] > Ia32/MemEncryptSevLib.c > diff --git a/OvmfPkg/Library/BaseMemEncryptSevLib/DxeMemEncryptSevLibInternal.c b/OvmfPkg/Library/BaseMemEncryptSevLib/DxeMemEncryptSevLibInternal.c > index 2816f859a0..ead754cd7b 100644 > --- a/OvmfPkg/Library/BaseMemEncryptSevLib/DxeMemEncryptSevLibInternal.c > +++ b/OvmfPkg/Library/BaseMemEncryptSevLib/DxeMemEncryptSevLibInternal.c > @@ -20,6 +20,8 @@ > STATIC BOOLEAN mSevStatus = FALSE; > STATIC BOOLEAN mSevEsStatus = FALSE; > STATIC BOOLEAN mSevStatusChecked = FALSE; > +STATIC BOOLEAN mSevLiveMigrationStatus = FALSE; > +STATIC BOOLEAN mSevLiveMigrationStatusChecked = FALSE; > > STATIC UINT64 mSevEncryptionMask = 0; > STATIC BOOLEAN mSevEncryptionMaskSaved = FALSE; > @@ -87,6 +89,24 @@ InternalMemEncryptSevStatus ( > mSevStatusChecked = TRUE; > } > > +/** > + Figures out if we are running inside KVM HVM and > + KVM HVM supports SEV Live Migration feature. > +**/ > +STATIC > +VOID > +EFIAPI > +InternalDetectSevLiveMigrationFeature( > + VOID > + ) > +{ > + if (KvmDetectSevLiveMigrationFeature()) { > + mSevLiveMigrationStatus = TRUE; > + } > + > + mSevLiveMigrationStatusChecked = TRUE; > +} > + > /** > Returns a boolean to indicate whether SEV-ES is enabled. > > @@ -125,6 +145,25 @@ MemEncryptSevIsEnabled ( > return mSevStatus; > } > > +/** > + Returns a boolean to indicate whether SEV live migration is enabled. > + > + @retval TRUE SEV live migration is enabled > + @retval FALSE SEV live migration is not enabled > +**/ > +BOOLEAN > +EFIAPI > +MemEncryptSevLiveMigrationIsEnabled ( > + VOID > + ) > +{ > + if (!mSevLiveMigrationStatusChecked) { > + InternalDetectSevLiveMigrationFeature (); > + } > + > + return mSevLiveMigrationStatus; > +} > + > /** > Returns the SEV encryption mask. > > diff --git a/OvmfPkg/Library/BaseMemEncryptSevLib/Ia32/MemEncryptSevLib.c b/OvmfPkg/Library/BaseMemEncryptSevLib/Ia32/MemEncryptSevLib.c > index be260e0d10..62392309fe 100644 > --- a/OvmfPkg/Library/BaseMemEncryptSevLib/Ia32/MemEncryptSevLib.c > +++ b/OvmfPkg/Library/BaseMemEncryptSevLib/Ia32/MemEncryptSevLib.c > @@ -136,3 +136,30 @@ MemEncryptSevClearMmioPageEncMask ( > // > return RETURN_UNSUPPORTED; > } > + > +/** > + This hyercall is used to notify hypervisor when the page's encryption > + state changes. > + > + @param[in] PhysicalAddress The physical address that is the start address > + of a memory region. The physical address is > + expected to be PAGE_SIZE aligned. > + @param[in] Pages Number of Pages in the memory region. > + @param[in] Status Encrypted(1) or Decrypted(0). > + > +@retval RETURN_SUCCESS Hypercall returned success. > +**/ > +RETURN_STATUS > +EFIAPI > +SetMemoryEncDecHypercall3 ( > + IN UINTN PhysicalAddress, > + IN UINTN Pages, > + IN UINTN Status > + ) > +{ > + // > + // Memory encryption bit is not accessible in 32-bit mode > + // > + return RETURN_UNSUPPORTED; > +} > + > diff --git a/OvmfPkg/Library/BaseMemEncryptSevLib/PeiDxeMemEncryptSevLibInternal.c b/OvmfPkg/Library/BaseMemEncryptSevLib/PeiDxeMemEncryptSevLibInternal.c > index b4a9f464e2..0c9f7e17ba 100644 > --- a/OvmfPkg/Library/BaseMemEncryptSevLib/PeiDxeMemEncryptSevLibInternal.c > +++ b/OvmfPkg/Library/BaseMemEncryptSevLib/PeiDxeMemEncryptSevLibInternal.c > @@ -61,3 +61,54 @@ MemEncryptSevLocateInitialSmramSaveStateMapPages ( > > return RETURN_SUCCESS; > } > + > +/** > + Figures out if we are running inside KVM HVM and > + KVM HVM supports SEV Live Migration feature. > + > + @retval TRUE SEV live migration is supported. > + @retval FALSE SEV live migration is not supported. > +**/ > +BOOLEAN > +EFIAPI > +KvmDetectSevLiveMigrationFeature( > + VOID > + ) > +{ > + CHAR8 Signature[13]; > + UINT32 mKvmLeaf; > + UINT32 RegEax, RegEbx, RegEcx, RegEdx; > + > + Signature[12] = '\0'; > + for (mKvmLeaf = 0x40000000; mKvmLeaf < 0x40010000; mKvmLeaf += 0x100) { > + AsmCpuid (mKvmLeaf, > + NULL, > + (UINT32 *) &Signature[0], > + (UINT32 *) &Signature[4], > + (UINT32 *) &Signature[8]); > + > + if (AsciiStrCmp ((CHAR8 *) Signature, "KVMKVMKVM\0\0\0") == 0) { > + DEBUG (( > + DEBUG_INFO, > + "%a: KVM Detected, signature = %s\n", > + __FUNCTION__, > + Signature > + )); > + > + RegEax = mKvmLeaf + 1; > + RegEcx = 0; > + AsmCpuid (mKvmLeaf + 1, &RegEax, &RegEbx, &RegEcx, &RegEdx); > + if ((RegEax & KVM_FEATURE_MIGRATION_CONTROL) != 0) { > + DEBUG (( > + DEBUG_INFO, > + "%a: Live Migration feature supported\n", > + __FUNCTION__ > + )); > + > + return TRUE; > + } > + } > + } > + > + return FALSE; > +} > diff --git a/OvmfPkg/Library/BaseMemEncryptSevLib/PeiMemEncryptSevLib.inf b/OvmfPkg/Library/BaseMemEncryptSevLib/PeiMemEncryptSevLib.inf > index 03a78c32df..3233ca7979 100644 > --- a/OvmfPkg/Library/BaseMemEncryptSevLib/PeiMemEncryptSevLib.inf > +++ b/OvmfPkg/Library/BaseMemEncryptSevLib/PeiMemEncryptSevLib.inf > @@ -38,6 +38,7 @@ > X64/PeiDxeVirtualMemory.c > X64/VirtualMemory.c > X64/VirtualMemory.h > + X64/AsmHelperStub.nasm > > [Sources.IA32] > Ia32/MemEncryptSevLib.c > diff --git a/OvmfPkg/Library/BaseMemEncryptSevLib/PeiMemEncryptSevLibInternal.c b/OvmfPkg/Library/BaseMemEncryptSevLib/PeiMemEncryptSevLibInternal.c > index e2fd109d12..9db6c2ef71 100644 > --- a/OvmfPkg/Library/BaseMemEncryptSevLib/PeiMemEncryptSevLibInternal.c > +++ b/OvmfPkg/Library/BaseMemEncryptSevLib/PeiMemEncryptSevLibInternal.c > @@ -20,6 +20,8 @@ > STATIC BOOLEAN mSevStatus = FALSE; > STATIC BOOLEAN mSevEsStatus = FALSE; > STATIC BOOLEAN mSevStatusChecked = FALSE; > +STATIC BOOLEAN mSevLiveMigrationStatus = FALSE; > +STATIC BOOLEAN mSevLiveMigrationStatusChecked = FALSE; > > STATIC UINT64 mSevEncryptionMask = 0; > STATIC BOOLEAN mSevEncryptionMaskSaved = FALSE; > @@ -87,6 +89,24 @@ InternalMemEncryptSevStatus ( > mSevStatusChecked = TRUE; > } > > +/** > + Figures out if we are running inside KVM HVM and > + KVM HVM supports SEV Live Migration feature. > +**/ > +STATIC > +VOID > +EFIAPI > +InternalDetectSevLiveMigrationFeature( > + VOID > + ) > +{ > + if (KvmDetectSevLiveMigrationFeature()) { > + mSevLiveMigrationStatus = TRUE; > + } > + > + mSevLiveMigrationStatusChecked = TRUE; > +} > + > /** > Returns a boolean to indicate whether SEV-ES is enabled. > > @@ -125,6 +145,25 @@ MemEncryptSevIsEnabled ( > return mSevStatus; > } > > +/** > + Returns a boolean to indicate whether SEV live migration is enabled. > + > + @retval TRUE SEV live migration is enabled > + @retval FALSE SEV live migration is not enabled > +**/ > +BOOLEAN > +EFIAPI > +MemEncryptSevLiveMigrationIsEnabled ( > + VOID > + ) > +{ > + if (!mSevLiveMigrationStatusChecked) { > + InternalDetectSevLiveMigrationFeature (); > + } > + > + return mSevLiveMigrationStatus; > +} > + > /** > Returns the SEV encryption mask. > > diff --git a/OvmfPkg/Library/BaseMemEncryptSevLib/SecMemEncryptSevLibInternal.c b/OvmfPkg/Library/BaseMemEncryptSevLib/SecMemEncryptSevLibInternal.c > index 56d8f3f318..b926c7b786 100644 > --- a/OvmfPkg/Library/BaseMemEncryptSevLib/SecMemEncryptSevLibInternal.c > +++ b/OvmfPkg/Library/BaseMemEncryptSevLib/SecMemEncryptSevLibInternal.c > @@ -100,6 +100,44 @@ MemEncryptSevIsEnabled ( > return Msr.Bits.SevBit ? TRUE : FALSE; > } > > +/** > + Interface exposed by the ASM implementation of the core hypercall > + > + @retval Hypercall returned status. > +**/ > +UINTN > +EFIAPI > +SetMemoryEncDecHypercall3AsmStub ( > + IN UINTN HypercallNum, > + IN UINTN PhysicalAddress, > + IN UINTN Pages, > + IN UINTN Attributes > + ) > +{ > + // > + // Not used in SEC phase. > + // > + return RETURN_UNSUPPORTED; > +} > + > +/** > + Returns a boolean to indicate whether SEV live migration is enabled. > + > + @retval TRUE SEV live migration is enabled > + @retval FALSE SEV live migration is not enabled > +**/ > +BOOLEAN > +EFIAPI > +MemEncryptSevLiveMigrationIsEnabled ( > + VOID > + ) > +{ > + // > + // Not used in SEC phase. > + // > + return FALSE; > +} > + > /** > Returns the SEV encryption mask. > > diff --git a/OvmfPkg/Library/BaseMemEncryptSevLib/X64/AsmHelperStub.nasm b/OvmfPkg/Library/BaseMemEncryptSevLib/X64/AsmHelperStub.nasm > new file mode 100644 > index 0000000000..c7c11f77f1 > --- /dev/null > +++ b/OvmfPkg/Library/BaseMemEncryptSevLib/X64/AsmHelperStub.nasm > @@ -0,0 +1,33 @@ > +/** @file > + > + ASM helper stub to invoke hypercall > + > + Copyright (c) 2021, AMD Incorporated. All rights reserved.
> + > + SPDX-License-Identifier: BSD-2-Clause-Patent > + > +**/ > + > +DEFAULT REL > +SECTION .text > + > +; UINTN > +; EFIAPI > +; SetMemoryEncDecHypercall3AsmStub ( > +; IN UINTN HypercallNum, > +; IN UINTN Arg1, > +; IN UINTN Arg2, > +; IN UINTN Arg3 > +; ); > +global ASM_PFX(SetMemoryEncDecHypercall3AsmStub) > +ASM_PFX(SetMemoryEncDecHypercall3AsmStub): > + ; UEFI calling conventions require RBX to > + ; be nonvolatile/callee-saved. > + push rbx > + mov rax, rcx ; Copy HypercallNumber to rax > + mov rbx, rdx ; Copy Arg1 to the register expected by KVM > + mov rcx, r8 ; Copy Arg2 to register expected by KVM > + mov rdx, r9 ; Copy Arg2 to register expected by KVM > + vmmcall ; Call VMMCALL > + pop rbx > + ret > diff --git a/OvmfPkg/Library/BaseMemEncryptSevLib/X64/MemEncryptSevLib.c b/OvmfPkg/Library/BaseMemEncryptSevLib/X64/MemEncryptSevLib.c > index a57e8fd37f..57447e69dc 100644 > --- a/OvmfPkg/Library/BaseMemEncryptSevLib/X64/MemEncryptSevLib.c > +++ b/OvmfPkg/Library/BaseMemEncryptSevLib/X64/MemEncryptSevLib.c > @@ -143,3 +143,57 @@ MemEncryptSevClearMmioPageEncMask ( > ); > > } > + > +/** > + This hyercall is used to notify hypervisor when the page's encryption > + state changes. > + > + @param[in] PhysicalAddress The physical address that is the start address > + of a memory region. The physical address is > + expected to be PAGE_SIZE aligned. > + @param[in] Pages Number of Pages in the memory region. > + @param[in] Status Encrypted(1) or Decrypted(0). > + > +@retval RETURN_SUCCESS Hypercall returned success. I see RETURN_NO_MAPPING also, so you'll need to update the retvals everywhere. > +**/ > +RETURN_STATUS > +EFIAPI > +SetMemoryEncDecHypercall3 ( > + IN UINTN PhysicalAddress, > + IN UINTN Pages, > + IN UINTN Status > + ) > +{ > + RETURN_STATUS Ret; > + INTN Error; Should be UINTN. > + > + Ret = RETURN_UNSUPPORTED; > + > + if (MemEncryptSevLiveMigrationIsEnabled ()) { > + Ret = EFI_SUCCESS; RETURN_SUCCESS since Ret is type RETURN_STATUS. > + // > + // The encryption bit is set/clear on the smallest page size, hence > + // use the 4k page size in MAP_GPA_RANGE hypercall below. > + // Also, the hypercall expects the guest physical address to be > + // page-aligned. > + // > + Error = SetMemoryEncDecHypercall3AsmStub ( > + KVM_HC_MAP_GPA_RANGE, > + (PhysicalAddress & (~(EFI_PAGE_SIZE-1))), > + Pages, > + KVM_MAP_GPA_RANGE_PAGE_SZ_4K | KVM_MAP_GPA_RANGE_ENC_STAT(Status) Status is UINTN, but is passed from an enum variable. If for any reason that enum should change in the future, this may break. So you should fixup your call to explicitly pass 0 or 1 and then you can safely use that value here. Maybe add an "ASSERT (Status == 0 || Status == 1)" to catch bad input values. > + ); > + > + if (Error != 0) { > + DEBUG ((DEBUG_ERROR, > + "SetMemoryEncDecHypercall3 failed, Phys = %Lx, Pages = %Ld, Err = %Ld\n", > + PhysicalAddress, > + Pages, > + (INT64)Error)); > + > + Ret = RETURN_NO_MAPPING; > + } > + } > + > + return Ret; > +} > diff --git a/OvmfPkg/Library/BaseMemEncryptSevLib/X64/PeiDxeVirtualMemory.c b/OvmfPkg/Library/BaseMemEncryptSevLib/X64/PeiDxeVirtualMemory.c > index c696745f9d..0b1588a4c1 100644 > --- a/OvmfPkg/Library/BaseMemEncryptSevLib/X64/PeiDxeVirtualMemory.c > +++ b/OvmfPkg/Library/BaseMemEncryptSevLib/X64/PeiDxeVirtualMemory.c > @@ -536,7 +536,6 @@ EnableReadOnlyPageWriteProtect ( > AsmWriteCr0 (AsmReadCr0() | BIT16); > } > > - > /** > This function either sets or clears memory encryption bit for the memory > region specified by PhysicalAddress and Length from the current page table > @@ -585,6 +584,9 @@ SetMemoryEncDec ( > UINT64 AddressEncMask; > BOOLEAN IsWpEnabled; > RETURN_STATUS Status; > + UINTN Size; > + BOOLEAN CBitChanged; > + PHYSICAL_ADDRESS OrigPhysicalAddress; > > // > // Set PageMapLevel4Entry to suppress incorrect compiler/analyzer warnings. > @@ -636,6 +638,10 @@ SetMemoryEncDec ( > > Status = EFI_SUCCESS; > > + Size = Length; > + CBitChanged = FALSE; > + OrigPhysicalAddress = PhysicalAddress; > + > while (Length != 0) > { > // > @@ -695,6 +701,7 @@ SetMemoryEncDec ( > )); > PhysicalAddress += BIT30; > Length -= BIT30; > + CBitChanged = TRUE; > } else { > // > // We must split the page > @@ -749,6 +756,7 @@ SetMemoryEncDec ( > SetOrClearCBit (&PageDirectory2MEntry->Uint64, Mode); > PhysicalAddress += BIT21; > Length -= BIT21; > + CBitChanged = TRUE; > } else { > // > // We must split up this page into 4K pages > @@ -791,6 +799,7 @@ SetMemoryEncDec ( > SetOrClearCBit (&PageTableEntry->Uint64, Mode); > PhysicalAddress += EFI_PAGE_SIZE; > Length -= EFI_PAGE_SIZE; > + CBitChanged = TRUE; > } > } > } > @@ -808,6 +817,17 @@ SetMemoryEncDec ( > // > CpuFlushTlb(); > > + // > + // Notify Hypervisor on C-bit status > + // > + if (CBitChanged) { > + Status = SetMemoryEncDecHypercall3 ( > + OrigPhysicalAddress, > + EFI_SIZE_TO_PAGES(Size), > + !Mode "Mode" is a MAP_RANGE_MODE enum that is local to this file. So you need to either move this to a common header file so you can use it with SetMemoryEncDecHypercall3() or set a 0 or 1 based on Mode and pass that. Thanks, Tom > + ); > + } > + > Done: > // > // Restore page table write protection, if any. >