From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from NAM02-CY1-obe.outbound.protection.outlook.com (NAM02-CY1-obe.outbound.protection.outlook.com [40.107.76.57]) by mx.groups.io with SMTP id smtpd.web10.13506.1574280439944511918 for ; Wed, 20 Nov 2019 12:07:20 -0800 Authentication-Results: mx.groups.io; dkim=pass header.i=@amdcloud.onmicrosoft.com header.s=selector2-amdcloud-onmicrosoft-com header.b=tptnCD9u; spf=none, err=SPF record not found (domain: amd.com, ip: 40.107.76.57, mailfrom: thomas.lendacky@amd.com) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=XzYlfQcR3svBoRAFI/V04mqJTUvRntLuTxZ50gxrL1aPGeuYheIcNboWx0iAU8YIg12Fga/woC9nXlpvBcEE+ct4bCcyLjLX6J0vswg9FStjjCo1wXQF2JARtqKiEeVzlPUWjwaC1jhWEGAE7+4zGWZeIP8FE84iS1xXSeaXUS+qMNrNKWRTXXnwlw/+wyjz7aF/44WH4S2Sel49ssaUz6bb52LOYl4+5y1/X4+F7nqupNFNs2v6FQNvh6mOKRncLCRNukb4nGgTpsQ2jGk9IEUUNzTb0GoqEAe9LMe7HYG19eTNAit0OfVAMjbrv3M3IiOlet5YCRuXEwtlI0BxnA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=tmiRHqMa5L1qjOszWZ4qnH6BrRQIJE7Wdk1OILeSjmM=; b=TOmu6V9nN6iwizHVq/1hCEaGe5RLcBASOLz3J43pNbzcpLJ39E7+1GQ/F4ZR+mHuAbjwZWHBt854AsDHza6vd9RmGI8lQemB/RS6W/+tBJlO/KA8ZEu2CDINKAo0ckmOnrRzCHsfb/5lBlXS3CKmk25Cv8zCMBpTwhvjLix11KxVHVtEsKFjgEH1JY42zRPjNSSnhCWi11VrVWHFV6p1yi4f6pd+YP8gWHS6Y84RuCrYCrrHQtuM7R0WmBAqXXz+zb1H+hSooYI74++kSYWAJctWzsEW9K6BK9pwpAM6vy/w+FO2Wr+O0Dk9cCUvGcme6RLlI2VfbhWP4Uwww5vf1Q== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=amd.com; dmarc=pass action=none header.from=amd.com; dkim=pass header.d=amd.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amdcloud.onmicrosoft.com; s=selector2-amdcloud-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=tmiRHqMa5L1qjOszWZ4qnH6BrRQIJE7Wdk1OILeSjmM=; b=tptnCD9unnO1L//ApOQWlw77GDwy9xTruUJMNj9xfFcUoou2Y4rA1lW4Zx89A7n94mJrHLKSqBZINwi544W8wE1R6GhtnMDlNVrgT8IOdSPJojBxpJN8QTzX8AwmEnyaoG7+e9IAyZ49fEKjOVE0HCcDMBjJAFwdpZl83YPB6qU= Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=Thomas.Lendacky@amd.com; Received: from DM6PR12MB3163.namprd12.prod.outlook.com (20.179.71.154) by DM6PR12MB3675.namprd12.prod.outlook.com (10.255.76.80) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2451.28; Wed, 20 Nov 2019 20:07:17 +0000 Received: from DM6PR12MB3163.namprd12.prod.outlook.com ([fe80::dd0c:8e53:4913:8ef4]) by DM6PR12MB3163.namprd12.prod.outlook.com ([fe80::dd0c:8e53:4913:8ef4%5]) with mapi id 15.20.2451.031; Wed, 20 Nov 2019 20:07:17 +0000 From: "Lendacky, Thomas" To: devel@edk2.groups.io Cc: Jordan Justen , Laszlo Ersek , Ard Biesheuvel , Michael D Kinney , Liming Gao , Eric Dong , Ray Ni , Brijesh Singh , Benjamin You , Dandan Bi , Guo Dong , Hao A Wu , Jian J Wang , Maurice Ma Subject: [RFC PATCH v3 00/43] SEV-ES guest support Date: Wed, 20 Nov 2019 14:06:22 -0600 Message-Id: X-Mailer: git-send-email 2.17.1 X-ClientProxiedBy: DM3PR12CA0087.namprd12.prod.outlook.com (2603:10b6:0:57::31) To DM6PR12MB3163.namprd12.prod.outlook.com (2603:10b6:5:15e::26) Return-Path: thomas.lendacky@amd.com MIME-Version: 1.0 X-Mailer: git-send-email 2.17.1 X-Originating-IP: [165.204.77.1] X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-HT: Tenant X-MS-Office365-Filtering-Correlation-Id: 161d65c2-d8d7-41d3-d370-08d76df53dfe X-MS-TrafficTypeDiagnostic: DM6PR12MB3675: X-MS-Exchange-PUrlCount: 3 X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:4303; X-Forefront-PRVS: 02272225C5 X-Forefront-Antispam-Report: SFV:NSPM;SFS:(10009020)(4636009)(376002)(346002)(396003)(39860400002)(366004)(136003)(189003)(199004)(7416002)(2361001)(50226002)(81156014)(81166006)(8676002)(2906002)(6486002)(99286004)(66476007)(486006)(47776003)(66556008)(25786009)(2616005)(476003)(54906003)(8936002)(316002)(2351001)(7736002)(305945005)(66066001)(6916009)(86362001)(16586007)(3846002)(6116002)(48376002)(14454004)(478600001)(966005)(5660300002)(66946007)(30864003)(36756003)(51416003)(6506007)(386003)(50466002)(52116002)(4326008)(6436002)(14444005)(6666004)(6306002)(6512007)(19627235002)(26005)(186003);DIR:OUT;SFP:1101;SCL:1;SRVR:DM6PR12MB3675;H:DM6PR12MB3163.namprd12.prod.outlook.com;FPR:;SPF:None;LANG:en;PTR:InfoNoRecords;MX:1;A:1; Received-SPF: None (protection.outlook.com: amd.com does not designate permitted sender hosts) X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: /yaRKHfNEE7sdK1Rd5zRSndYxaJjfMyfNABgu/naegyQbPpuAJyZ1PD8CRzWvbMxpm0bn4Y+kW1FUYyFFFvUB+z5iW0KduYy8uMSfggu7/5EPxlosutLfK0iLg4tKZAvfFeJG+qZubmi+X61vqpQ9DtHMCsqBUuwOs3YUg+IRIsjyuf1ojV96/cWsbP2etaZcjgT24dCvpGbfjfPuT9+lG4ZYtXd9a1dza6BPM6TF+9Drw+AaW54HrZPGjoT8kCsl83fVSsXEMiNB9TPpZ9IohBxZSk47uWJvEn8HtzAa1naONrDyz5UsgTUiw3ruOOezpZjYNfJosFr3OE5UNKDa61FCaexq5SCsDi7Tf+UodyCIrcg4D1QMqJ4wZLaAg6pDHo29yxyCqj9YCKPZwnm2SMMghGpypfaQ+G0p7qgTiyN1pFlreF/rlOHis7KUI+xzSGK03v+wfgI+gB+Pxq1BCRF7s2NQdx8C+UKXEOhmYU= X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-Network-Message-Id: 161d65c2-d8d7-41d3-d370-08d76df53dfe X-MS-Exchange-CrossTenant-OriginalArrivalTime: 20 Nov 2019 20:07:17.2219 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: jg+ve5OHVt7y5Xa3V+5vqSHGO4hGjYd4ejUIRfsKnk7aHuiMIu1BaJbgghc1iWw5DpWyyvgdZxbx+xq3iyYV2g== X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6PR12MB3675 Content-Type: text/plain This patch series provides support for running EDK2/OVMF under SEV-ES. Secure Encrypted Virtualization - Encrypted State (SEV-ES) expands on the SEV support to protect the guest register state from the hypervisor. See "AMD64 Architecture Programmer's Manual Volume 2: System Programming", section "15.35 Encrypted State (SEV-ES)" [1]. In order to allow a hypervisor to perform functions on behalf of a guest, there is architectural support for notifying a guest's operating system when certain types of VMEXITs are about to occur. This allows the guest to selectively share information with the hypervisor to satisfy the requested function. The notification is performed using a new exception, the VMM Communication exception (#VC). The information is shared through the Guest-Hypervisor Communication Block (GHCB) using the VMGEXIT instruction. The GHCB format and the protocol for using it is documented in "SEV-ES Guest-Hypervisor Communication Block Standardization" [2]. The main areas of the EDK2 code that are updated to support SEV-ES are around the exception handling support and the AP boot support. Exception support is required starting in Sec, continuing through Pei and into Dxe in order to handle #VC exceptions that are generated. Each AP requires it's own GHCB page as well as a page to hold values specific to that AP. AP booting poses some interesting challenges. The INIT-SIPI-SIPI sequence is typically used to boot the APs. However, the hypervisor is not allowed to update the guest registers. The GHCB document [2] talks about how SMP booting under SEV-ES is performed. Since the GHCB page must be a shared (unencrypted) page, the processor must be running in long mode in order for the guest and hypervisor to communicate with each other. As a result, SEV-ES is only supported under the X64 architecture. [1] https://www.amd.com/system/files/TechDocs/24593.pdf [2] https://developer.amd.com/wp-content/resources/56421.pdf --- These patches are based on commit: cc6854506c2b ("Readme.md: remove positional references from submodule description") Proper execution of SEV-ES relies on Bugzilla 2340 being fixed. A version of the tree (with an extra patch to workaround Bugzilla 2340) can be found at: https://github.com/AMDESE/ovmf/tree/sev-es-v9 Cc: Ard Biesheuvel Cc: Benjamin You Cc: Dandan Bi Cc: Eric Dong Cc: Guo Dong Cc: Hao A Wu Cc: Jian J Wang Cc: Jordan Justen Cc: Laszlo Ersek Cc: Liming Gao Cc: Maurice Ma Cc: Michael D Kinney Cc: Ray Ni Changes since v2: - Added a way to locate the SEV-ES fixed AP RIP address for starting AP's to avoid updating the actual flash image (build time location that is identified with a GUID value). - Create a VmgExit library to replace static inline functions. - Move some PCDs to the appropriate packages - Add support for writing to QEMU flash under SEV-ES - Add additional MMIO opcode support - Cleaned up the GHCB MSR CPUID protocol support Changes since v1: - Patches reworked to be more specific to the component/area being updated and order of definition/usage - Created a library for VMGEXIT-related functions to replace use of inline functions - Allocation method for GDT changed from AllocatePool to AllocatePages - Early caching only enabled for SEV-ES guests - Ensure AP loop mode set to halt loop mode for SEV-ES guests - Reserved SEC GHCB-related memory areas when S3 is enabled Tom Lendacky (43): MdePkg: Create PCDs to be used in support of SEV-ES MdePkg: Add the MSR definition for the GHCB register MdePkg: Add a structure definition for the GHCB MdeModulePkg/DxeIplPeim: Support GHCB pages when creating page tables MdePkg/BaseLib: Add support for the XGETBV instruction MdePkg/BaseLib: Add support for the VMGEXIT instruction UefiCpuPkg: Implement library support for VMGEXIT UefiCpuPkg/CpuExceptionHandler: Add base support for the #VC exception UefiCpuPkg/CpuExceptionHandler: Add support for IOIO_PROT NAE events UefiCpuPkg/CpuExceptionHandler: Support string IO for IOIO_PROT NAE events UefiCpuPkg/CpuExceptionHandler: Add support for CPUID NAE events UefiCpuPkg/CpuExceptionHandler: Add support for MSR_PROT NAE events UefiCpuPkg/CpuExceptionHandler: Add support for NPF NAE events (MMIO) UefiCpuPkg/CpuExceptionHandler: Add support for WBINVD NAE events UefiCpuPkg/CpuExceptionHandler: Add support for RDTSC NAE events UefiCpuPkg/CpuExceptionHandler: Add support for RDPMC NAE events UefiCpuPkg/CpuExceptionHandler: Add support for INVD NAE events UefiCpuPkg/CpuExceptionHandler: Add support for VMMCALL NAE events UefiCpuPkg/CpuExceptionHandler: Add support for RDTSCP NAE events UefiCpuPkg/CpuExceptionHandler: Add support for MONITOR/MONITORX NAE events UefiCpuPkg/CpuExceptionHandler: Add support for MWAIT/MWAITX NAE events UefiCpuPkg/CpuExceptionHandler: Add support for DR7 Read/Write NAE events OvmfPkg/MemEncryptSevLib: Add an SEV-ES guest indicator function OvmfPkg: Add support to perform SEV-ES initialization OvmfPkg/ResetVector: Add support for a 32-bit SEV check OvmfPkg: Create a GHCB page for use during Sec phase OvmfPkg/PlatformPei: Reserve GHCB-related areas if S3 is supported OvmfPkg: Create GHCB pages for use during Pei and Dxe phase OvmfPkg/PlatformPei: Move early GDT into ram when SEV-ES is enabled OvmfPkg/Sec: Add #VC exception handling for Sec phase OvmfPkg/Sec: Enable cache early to speed up booting OvmfPkg/QemuFlashFvbServicesRuntimeDxe: Bypass flash detection with SEV-ES is enabled MdeModulePkg: Reserve a 16-bit protected mode code segment descriptor UefiCpuPkg: Add a 16-bit protected mode code segment descriptor UefiCpuPkg/MpInitLib: Add a CPU MP data flag to indicate if SEV-ES is enabled UefiCpuPkg: Allow AP booting under SEV-ES OvmfPkg: Reserve a page in memory for the SEV-ES AP reset vector OvmfPkg: Move the GHCB allocations into reserved memory MdePkg: Add a finalization function to the CPU protocol UefiCpuPkg/MpInitLib: Add MP finalization interface to MpInitLib UefiCpuPkg/MpInitLib: Prepare SEV-ES guest APs for OS use UefiCpuPkg/CpuDxe: Provide an DXE MP finalization routine to support SEV-ES MdeModulePkg/DxeCore: Perform the CPU protocol finalization support MdeModulePkg/MdeModulePkg.dec | 23 + OvmfPkg/OvmfPkg.dec | 5 + UefiCpuPkg/UefiCpuPkg.dec | 8 + OvmfPkg/OvmfPkgIa32.dsc | 11 + OvmfPkg/OvmfPkgIa32X64.dsc | 11 + OvmfPkg/OvmfPkgX64.dsc | 11 + UefiCpuPkg/UefiCpuPkg.dsc | 5 + UefiPayloadPkg/UefiPayloadPkgIa32.dsc | 2 + UefiPayloadPkg/UefiPayloadPkgIa32X64.dsc | 2 + OvmfPkg/OvmfPkgX64.fdf | 9 + MdeModulePkg/Core/DxeIplPeim/DxeIpl.inf | 2 + MdePkg/Library/BaseLib/BaseLib.inf | 4 + OvmfPkg/PlatformPei/PlatformPei.inf | 7 + .../FvbServicesRuntimeDxe.inf | 2 + OvmfPkg/ResetVector/ResetVector.inf | 9 + OvmfPkg/Sec/SecMain.inf | 1 + .../DxeCpuExceptionHandlerLib.inf | 5 + .../PeiCpuExceptionHandlerLib.inf | 5 + .../SecPeiCpuExceptionHandlerLib.inf | 8 + UefiCpuPkg/Library/MpInitLib/DxeMpInitLib.inf | 4 + UefiCpuPkg/Library/MpInitLib/PeiMpInitLib.inf | 4 + UefiCpuPkg/Library/VmgExitLib/VmgExitLib.inf | 33 + .../Core/DxeIplPeim/X64/VirtualMemory.h | 12 +- MdePkg/Include/Library/BaseLib.h | 31 + MdePkg/Include/Protocol/Cpu.h | 18 + MdePkg/Include/Register/Amd/Fam17Msr.h | 28 + MdePkg/Include/Register/Amd/Ghcb.h | 117 ++ OvmfPkg/Include/Library/MemEncryptSevLib.h | 12 + UefiCpuPkg/CpuDxe/CpuDxe.h | 12 + UefiCpuPkg/CpuDxe/CpuGdt.h | 4 +- UefiCpuPkg/Include/Library/MpInitLib.h | 14 + UefiCpuPkg/Include/Library/VmgExitLib.h | 96 ++ .../CpuExceptionHandlerLib/AMDSevVcCommon.h | 26 + .../CpuExceptionCommon.h | 2 + UefiCpuPkg/Library/MpInitLib/MpLib.h | 79 +- MdeModulePkg/Core/Dxe/DxeMain/DxeMain.c | 5 + .../Core/DxeIplPeim/Ia32/DxeLoadFunc.c | 6 +- .../Core/DxeIplPeim/X64/DxeLoadFunc.c | 11 +- .../Core/DxeIplPeim/X64/VirtualMemory.c | 49 +- MdePkg/Library/BaseLib/Ia32/GccInline.c | 45 + MdePkg/Library/BaseLib/X64/GccInline.c | 47 + .../MemEncryptSevLibInternal.c | 77 +- OvmfPkg/PlatformPei/AmdSev.c | 82 ++ OvmfPkg/PlatformPei/MemDetect.c | 23 + .../QemuFlash.c | 38 +- OvmfPkg/Sec/SecMain.c | 74 +- UefiCpuPkg/CpuDxe/CpuDxe.c | 21 +- UefiCpuPkg/CpuDxe/CpuGdt.c | 8 +- .../CpuExceptionCommon.c | 2 +- .../Ia32/AMDSevVcCommon.c | 20 + .../PeiDxeAMDSevVcHandler.c | 29 + .../PeiDxeSmmCpuException.c | 16 + .../SecAMDSevVcHandler.c | 50 + .../SecPeiCpuException.c | 16 + .../X64/AMDSevVcCommon.c | 1230 +++++++++++++++++ UefiCpuPkg/Library/MpInitLib/DxeMpLib.c | 136 +- UefiCpuPkg/Library/MpInitLib/MpLib.c | 280 +++- UefiCpuPkg/Library/MpInitLib/PeiMpLib.c | 35 + UefiCpuPkg/Library/VmgExitLib/VmgExitLib.c | 132 ++ UefiCpuPkg/PiSmmCpuDxeSmm/X64/SmmFuncsArch.c | 2 +- MdePkg/Library/BaseLib/Ia32/VmgExit.nasm | 37 + MdePkg/Library/BaseLib/Ia32/XGetBv.nasm | 31 + MdePkg/Library/BaseLib/X64/VmgExit.nasm | 32 + MdePkg/Library/BaseLib/X64/XGetBv.nasm | 34 + OvmfPkg/ResetVector/Ia16/ResetVectorVtf0.asm | 94 ++ OvmfPkg/ResetVector/Ia32/PageTables64.asm | 337 ++++- OvmfPkg/ResetVector/ResetVector.nasmb | 19 + .../X64/ExceptionHandlerAsm.nasm | 15 + UefiCpuPkg/Library/MpInitLib/Ia32/MpEqu.inc | 2 +- .../Library/MpInitLib/Ia32/MpFuncs.nasm | 15 + UefiCpuPkg/Library/MpInitLib/X64/MpEqu.inc | 4 +- UefiCpuPkg/Library/MpInitLib/X64/MpFuncs.nasm | 370 ++++- UefiCpuPkg/Library/VmgExitLib/VmgExitLib.uni | 15 + .../ResetVector/Vtf0/Ia16/Real16ToFlat32.asm | 9 + 74 files changed, 3969 insertions(+), 101 deletions(-) create mode 100644 UefiCpuPkg/Library/VmgExitLib/VmgExitLib.inf create mode 100644 MdePkg/Include/Register/Amd/Ghcb.h create mode 100644 UefiCpuPkg/Include/Library/VmgExitLib.h create mode 100644 UefiCpuPkg/Library/CpuExceptionHandlerLib/AMDSevVcCommon.h create mode 100644 UefiCpuPkg/Library/CpuExceptionHandlerLib/Ia32/AMDSevVcCommon.c create mode 100644 UefiCpuPkg/Library/CpuExceptionHandlerLib/PeiDxeAMDSevVcHandler.c create mode 100644 UefiCpuPkg/Library/CpuExceptionHandlerLib/SecAMDSevVcHandler.c create mode 100644 UefiCpuPkg/Library/CpuExceptionHandlerLib/X64/AMDSevVcCommon.c create mode 100644 UefiCpuPkg/Library/VmgExitLib/VmgExitLib.c create mode 100644 MdePkg/Library/BaseLib/Ia32/VmgExit.nasm create mode 100644 MdePkg/Library/BaseLib/Ia32/XGetBv.nasm create mode 100644 MdePkg/Library/BaseLib/X64/VmgExit.nasm create mode 100644 MdePkg/Library/BaseLib/X64/XGetBv.nasm create mode 100644 OvmfPkg/ResetVector/Ia16/ResetVectorVtf0.asm create mode 100644 UefiCpuPkg/Library/VmgExitLib/VmgExitLib.uni -- 2.17.1