From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from NAM10-DM6-obe.outbound.protection.outlook.com (NAM10-DM6-obe.outbound.protection.outlook.com [40.107.93.60]) by mx.groups.io with SMTP id smtpd.web11.3790.1680026987996362731 for ; Tue, 28 Mar 2023 11:09:48 -0700 Authentication-Results: mx.groups.io; dkim=fail reason="body hash did not verify" header.i=@amd.com header.s=selector1 header.b=ttqWTHP1; spf=permerror, err=parse error for token &{10 18 %{i}._ip.%{h}._ehlo.%{d}._spf.vali.email}: invalid domain name (domain: amd.com, ip: 40.107.93.60, mailfrom: thomas.lendacky@amd.com) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=FzWmW04BvZaFdorlPM0VI3kl0iuzNTU6UDTg836ll4M3kQElQd4y2Y3BkR3aOI0qhBOGS8xYXgGItgM2PQUmKMI5xlLNtmmIGn07N4bxzRX7DnSmNFBPfedoEma7LzzdgnJNhlCXk05jL8NM4GpA3bvrz+b03ysObNm0v1OZjh1uWeWVPOzBjpOTVK1kkwD/9HfF8wCUA5hcEBxhUI7RUEoiLye8N4kRXzEidjZ3LAlK76kz6u3fdEsHgnJ63g2+IwF9JicVkvozKwKT5TZFvokQqT0VeV4uVtRTUd3ybAAtNQuV/hpfpKhtKUgsciqftxOhopOZM9OGJvcRweZPqg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=+TsFR+DxnVfWDR6hxl0gElpSvdoXZmXYZJ9+I8aTiA0=; b=U/1WE5sa4+gJfzm3lHxwMZh/kJfh3Zz7jBQvXLKMxRJb4RQnIh86L0qY9wNCC930DEGSF4NJ0Sla3M2A2m7WFePgKGBwGYGc+Zg+ULgWlG4uja7VB9BwKPxxX1jv9p/K15LQIuJPd1ELx/dli3Av73sEScFUQuPOknvtPbbWuU4/rDQnflWXB2+uzvGlDLWroSweMbM0LUs7RGVIU27eFSSohoIfkC68LjMrWh3arbzW2IF9A5DcR58kh7iWUgnsVJe8aZEfWRsLa8oMsj2ZUiimiy80lgR0H4F9DCJfs14KCweTjdP/UW0TeZV+i5QaDu6ZQMu7eQoh8IijGyzUNQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=edk2.groups.io smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=+TsFR+DxnVfWDR6hxl0gElpSvdoXZmXYZJ9+I8aTiA0=; b=ttqWTHP1LYXsCLYheGa80yX4xpLCZ/n5JOvtJNE5mkWESe2sP1Ngzzmx/vjN4ZMilSAupLsk25eLowSvMBuAvxhE/KT4AQKBrrCoczEU97oBvq5+qfsk0vVj/yybPCZ0BSWGR61HTFsR3dJhj+lVRnAJ+JtcpW6Wt/9zlHe9nIU= Received: from MN2PR06CA0001.namprd06.prod.outlook.com (2603:10b6:208:23d::6) by SA1PR12MB6945.namprd12.prod.outlook.com (2603:10b6:806:24c::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6222.33; Tue, 28 Mar 2023 18:09:46 +0000 Received: from BL02EPF0000C405.namprd05.prod.outlook.com (2603:10b6:208:23d:cafe::d4) by MN2PR06CA0001.outlook.office365.com (2603:10b6:208:23d::6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6178.43 via Frontend Transport; Tue, 28 Mar 2023 18:09:45 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by BL02EPF0000C405.mail.protection.outlook.com (10.167.241.7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.6178.30 via Frontend Transport; Tue, 28 Mar 2023 18:09:45 +0000 Received: from tlendack-t1.amdoffice.net (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.34; Tue, 28 Mar 2023 13:09:44 -0500 From: "Lendacky, Thomas" To: CC: Eric Dong , Ray Ni , Rahul Kumar , Gerd Hoffmann , Michael Roth , Ashish Kalra Subject: [PATCH v2 1/2] UefiCpuPkg/MpInitLib: Reuse VMSA allocation to avoid unreserved allocation Date: Tue, 28 Mar 2023 13:09:23 -0500 Message-ID: X-Mailer: git-send-email 2.40.0 In-Reply-To: References: MIME-Version: 1.0 Return-Path: Thomas.Lendacky@amd.com X-Originating-IP: [10.180.168.240] X-ClientProxiedBy: SATLEXMB03.amd.com (10.181.40.144) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BL02EPF0000C405:EE_|SA1PR12MB6945:EE_ X-MS-Office365-Filtering-Correlation-Id: 9377b07a-9005-4408-f06d-08db2fb79c99 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: J+9EE1BLZUZtt1265DFsX6QcAfJ0QRaWJUppxzeMlC35auNG07xPu1ukjxzI1loFx77iFQdov+aUEJ+m32UDFY3SYm4Dz6HwJ2LCLu+AEk+JksBhZCgnGn7w3Q0AQ79jAI4MZ51xf7i4efiKjoe6DWcJS3BumoU+uxlO+Kp8S1n3trerEC3bGac67aTahsH5a5xYhIyNAl7WI7xUZZfSPVwRX/3PCx9bCtaNGZi3o6V4v11tyR43p5+Gmr90PC3vsvwt7O8Zi7Rrjfvy61a1VDnz/IuUJWiWdvhYHHmAWlm+nOBFycjXIV0qsZXwmvsD2Y4JAMlHIvpyDb7YyJ7KkLPs4jJF+URK6kzi1cdHBEw4LGXdo3fadoRTyYIk950mXNiNti6G6/xNzwr5pwv3tzQTee+YpJDMKU8BoIBXGAAGFAIb/56S0v7H+6aq3TGIymnan8MCKw0HuUaNe+X5ikAudwC70SLs2j9wYICOQsoNTZhivKXljnZo5s+QCebMyXb6ZjiwJaLeYWFLqfiWdGu7p/6cgwNAXguq9IqqCpy0Ip6eyuhbxlP3xi3mOzZKaSIzdffeayzIsZ3hwlLlT2TDU4wI5ARm7e4/MiRGMAzHkVuhExbqb/xRIF0JcJGoFgWV7Ro8KYYKTOC0jyFpKRv0hwikJn7OOGUa4VIYwYv7CZ+TktnYhQL5t0d7zWoUv1d4+6rYyzw+aj0ww4AfmUuNMJy8khJQ+PwKevwnCZE= X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230028)(6029001)(4636009)(39860400002)(396003)(346002)(136003)(376002)(451199021)(40470700004)(46966006)(36840700001)(6666004)(26005)(966005)(6916009)(316002)(54906003)(19627235002)(478600001)(186003)(47076005)(36860700001)(16526019)(70586007)(70206006)(426003)(8676002)(336012)(2616005)(83380400001)(41300700001)(8936002)(81166007)(356005)(5660300002)(4326008)(2906002)(82740400003)(40460700003)(40480700001)(86362001)(82310400005)(36756003)(36900700001);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 28 Mar 2023 18:09:45.5443 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 9377b07a-9005-4408-f06d-08db2fb79c99 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: BL02EPF0000C405.namprd05.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA1PR12MB6945 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain https://bugzilla.tianocore.org/show_bug.cgi?id=3D4353 When parking the APs on exiting from UEFI, a new page allocation is made. This allocation, however, does not end up being marked reserved in the memory map supplied to the OS. To avoid this, re-use the VMSA by clearing the VMSA RMP flag, updating the page contents and re-setting the VMSA RMP flag. Fixes: 06544455d0d4 ("UefiCpuPkg/MpInitLib: Use SEV-SNP AP Creation ...") Signed-off-by: Tom Lendacky --- UefiCpuPkg/Library/MpInitLib/X64/AmdSev.c | 204 +++++++++++++--------- 1 file changed, 124 insertions(+), 80 deletions(-) diff --git a/UefiCpuPkg/Library/MpInitLib/X64/AmdSev.c b/UefiCpuPkg/Library= /MpInitLib/X64/AmdSev.c index bfda1e19030d..509be9b41757 100644 --- a/UefiCpuPkg/Library/MpInitLib/X64/AmdSev.c +++ b/UefiCpuPkg/Library/MpInitLib/X64/AmdSev.c @@ -14,40 +14,140 @@ #include =20 /** - Create an SEV-SNP AP save area (VMSA) for use in running the vCPU. + Perform the requested AP Creation action. =20 - @param[in] CpuMpData Pointer to CPU MP Data - @param[in] CpuData Pointer to CPU AP Data + @param[in] SaveArea Pointer to VM save area (VMSA) @param[in] ApicId APIC ID of the vCPU + @param[in] Action AP action to perform + + @retval TRUE Action completed successfully + @retval FALSE Action did not complete successfully **/ -VOID -SevSnpCreateSaveArea ( - IN CPU_MP_DATA *CpuMpData, - IN CPU_AP_DATA *CpuData, - UINT32 ApicId +STATIC +BOOLEAN +SevSnpPerformApAction ( + IN SEV_ES_SAVE_AREA *SaveArea, + IN UINT32 ApicId, + IN UINTN Action ) { - SEV_ES_SAVE_AREA *SaveArea; - IA32_CR0 ApCr0; - IA32_CR0 ResetCr0; - IA32_CR4 ApCr4; - IA32_CR4 ResetCr4; - UINTN StartIp; - UINT8 SipiVector; - UINT32 RmpAdjustStatus; - UINT64 VmgExitStatus; MSR_SEV_ES_GHCB_REGISTER Msr; GHCB *Ghcb; BOOLEAN InterruptState; UINT64 ExitInfo1; UINT64 ExitInfo2; + UINT32 RmpAdjustStatus; + UINT64 VmgExitStatus; =20 - // - // Allocate a single page for the SEV-ES Save Area and initialize it. - // - SaveArea =3D AllocateReservedPages (1); - if (!SaveArea) { - return; + if (Action =3D=3D SVM_VMGEXIT_SNP_AP_CREATE) { + // + // To turn the page into a recognized VMSA page, issue RMPADJUST: + // Target VMPL but numerically higher than current VMPL + // Target PermissionMask is not used + // + RmpAdjustStatus =3D SevSnpRmpAdjust ( + (EFI_PHYSICAL_ADDRESS)(UINTN)SaveArea, + TRUE + ); + if (RmpAdjustStatus !=3D 0) { + DEBUG ((DEBUG_INFO, "SEV-SNP: RMPADJUST failed for VMSA creation\n")= ); + ASSERT (FALSE); + + return FALSE; + } + } + + ExitInfo1 =3D (UINT64)ApicId << 32; + ExitInfo1 |=3D Action; + ExitInfo2 =3D (UINT64)(UINTN)SaveArea; + + Msr.GhcbPhysicalAddress =3D AsmReadMsr64 (MSR_SEV_ES_GHCB); + Ghcb =3D Msr.Ghcb; + + CcExitVmgInit (Ghcb, &InterruptState); + + if (Action =3D=3D SVM_VMGEXIT_SNP_AP_CREATE) { + Ghcb->SaveArea.Rax =3D SaveArea->SevFeatures; + CcExitVmgSetOffsetValid (Ghcb, GhcbRax); + } + + VmgExitStatus =3D CcExitVmgExit ( + Ghcb, + SVM_EXIT_SNP_AP_CREATION, + ExitInfo1, + ExitInfo2 + ); + + CcExitVmgDone (Ghcb, InterruptState); + + if (VmgExitStatus !=3D 0) { + DEBUG ((DEBUG_INFO, "SEV-SNP: AP Destroy failed\n")); + ASSERT (FALSE); + + return FALSE; + } + + if (Action =3D=3D SVM_VMGEXIT_SNP_AP_DESTROY) { + // + // Make the current VMSA not runnable and accessible to be + // reprogrammed. + // + RmpAdjustStatus =3D SevSnpRmpAdjust ( + (EFI_PHYSICAL_ADDRESS)(UINTN)SaveArea, + FALSE + ); + if (RmpAdjustStatus !=3D 0) { + DEBUG ((DEBUG_INFO, "SEV-SNP: RMPADJUST failed for VMSA reset\n")); + ASSERT (FALSE); + + return FALSE; + } + } + + return TRUE; +} + +/** + Create an SEV-SNP AP save area (VMSA) for use in running the vCPU. + + @param[in] CpuMpData Pointer to CPU MP Data + @param[in] CpuData Pointer to CPU AP Data + @param[in] ApicId APIC ID of the vCPU +**/ +VOID +SevSnpCreateSaveArea ( + IN CPU_MP_DATA *CpuMpData, + IN CPU_AP_DATA *CpuData, + UINT32 ApicId + ) +{ + SEV_ES_SAVE_AREA *SaveArea; + IA32_CR0 ApCr0; + IA32_CR0 ResetCr0; + IA32_CR4 ApCr4; + IA32_CR4 ResetCr4; + UINTN StartIp; + UINT8 SipiVector; + + if (CpuData->SevEsSaveArea =3D=3D NULL) { + // + // Allocate a single page for the SEV-ES Save Area and initialize it. + // + SaveArea =3D AllocateReservedPages (1); + if (!SaveArea) { + return; + } + + CpuData->SevEsSaveArea =3D SaveArea; + } else { + SaveArea =3D CpuData->SevEsSaveArea; + + // + // Tell the hypervisor to not use the current VMSA + // + if (!SevSnpPerformApAction (SaveArea, ApicId, SVM_VMGEXIT_SNP_AP_DESTR= OY)) { + return; + } } =20 ZeroMem (SaveArea, EFI_PAGE_SIZE); @@ -132,63 +232,7 @@ SevSnpCreateSaveArea ( SaveArea->Vmpl =3D 0; SaveArea->SevFeatures =3D AsmReadMsr64 (MSR_SEV_STATUS) >> 2; =20 - // - // To turn the page into a recognized VMSA page, issue RMPADJUST: - // Target VMPL but numerically higher than current VMPL - // Target PermissionMask is not used - // - RmpAdjustStatus =3D SevSnpRmpAdjust ( - (EFI_PHYSICAL_ADDRESS)(UINTN)SaveArea, - TRUE - ); - ASSERT (RmpAdjustStatus =3D=3D 0); - - ExitInfo1 =3D (UINT64)ApicId << 32; - ExitInfo1 |=3D SVM_VMGEXIT_SNP_AP_CREATE; - ExitInfo2 =3D (UINT64)(UINTN)SaveArea; - - Msr.GhcbPhysicalAddress =3D AsmReadMsr64 (MSR_SEV_ES_GHCB); - Ghcb =3D Msr.Ghcb; - - CcExitVmgInit (Ghcb, &InterruptState); - Ghcb->SaveArea.Rax =3D SaveArea->SevFeatures; - CcExitVmgSetOffsetValid (Ghcb, GhcbRax); - VmgExitStatus =3D CcExitVmgExit ( - Ghcb, - SVM_EXIT_SNP_AP_CREATION, - ExitInfo1, - ExitInfo2 - ); - CcExitVmgDone (Ghcb, InterruptState); - - ASSERT (VmgExitStatus =3D=3D 0); - if (VmgExitStatus !=3D 0) { - RmpAdjustStatus =3D SevSnpRmpAdjust ( - (EFI_PHYSICAL_ADDRESS)(UINTN)SaveArea, - FALSE - ); - if (RmpAdjustStatus =3D=3D 0) { - FreePages (SaveArea, 1); - } else { - DEBUG ((DEBUG_INFO, "SEV-SNP: RMPADJUST failed, leaking VMSA page\n"= )); - } - - SaveArea =3D NULL; - } - - if (CpuData->SevEsSaveArea) { - RmpAdjustStatus =3D SevSnpRmpAdjust ( - (EFI_PHYSICAL_ADDRESS)(UINTN)CpuData->SevEsSaveAre= a, - FALSE - ); - if (RmpAdjustStatus =3D=3D 0) { - FreePages (CpuData->SevEsSaveArea, 1); - } else { - DEBUG ((DEBUG_INFO, "SEV-SNP: RMPADJUST failed, leaking VMSA page\n"= )); - } - } - - CpuData->SevEsSaveArea =3D SaveArea; + SevSnpPerformApAction (SaveArea, ApicId, SVM_VMGEXIT_SNP_AP_CREATE); } =20 /** --=20 2.40.0