From mboxrd@z Thu Jan 1 00:00:00 1970
Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1])
by mx.groups.io with SMTP id smtpd.web09.452.1614718157162035214
for <devel@edk2.groups.io>;
Tue, 02 Mar 2021 12:49:17 -0800
Authentication-Results: mx.groups.io;
dkim=pass header.i=@ibm.com header.s=pp1 header.b=Uf89i/LC;
spf=pass (domain: linux.ibm.com, ip: 148.163.156.1, mailfrom: tobin@linux.ibm.com)
Received: from pps.filterd (m0098394.ppops.net [127.0.0.1])
by mx0a-001b2d01.pphosted.com (8.16.0.43/8.16.0.43) with SMTP id 122KiGWn064906;
Tue, 2 Mar 2021 15:49:13 -0500
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=from : to : cc : subject
: date : message-id : in-reply-to : references : mime-version :
content-transfer-encoding; s=pp1;
bh=YqP+JoPPjtMV3KaGwE+V9wSAlNRZ/U1Mxt9LVRWlid4=;
b=Uf89i/LCqe/QSZ7pb0HEqL0cs8vNHaPkoEpPHo3h5qBdHgeRhKo3VIrhx1gj7Q12pHgU
mH1gA50Rkeq/+Mr33xk0BakNZEUnEmPUAV3Z9S/T86Ng/K1i8bX6G9VJR2mtKG+ghdjK
HGWNbdpfC6ZLYXS0pRKGT8XVhMuNgWFMmeAAzoh0lRFkekC3zGomISjKeaX8Cujnh1L2
wX5AbMOLTYC/0GOUC/0KGeJfPnZ2rnjzpiU1hT/z9XJew71JBb6iwpGHOrzu4pkYUBOJ
gcUBi0QMe71ggaZqyJWc5SZ3z8gNmBS7pVfCKB+CSFsL7BdM6KWG5U+R8/IHr/hJ6Aa+ MA==
Received: from ppma01dal.us.ibm.com (83.d6.3fa9.ip4.static.sl-reverse.com [169.63.214.131])
by mx0a-001b2d01.pphosted.com with ESMTP id 371vnf86mq-1
(version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT);
Tue, 02 Mar 2021 15:49:13 -0500
Received: from pps.filterd (ppma01dal.us.ibm.com [127.0.0.1])
by ppma01dal.us.ibm.com (8.16.0.42/8.16.0.42) with SMTP id 122KkqMV028086;
Tue, 2 Mar 2021 20:48:56 GMT
Received: from b03cxnp07029.gho.boulder.ibm.com (b03cxnp07029.gho.boulder.ibm.com [9.17.130.16])
by ppma01dal.us.ibm.com with ESMTP id 371qmuagxv-1
(version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT);
Tue, 02 Mar 2021 20:48:55 +0000
Received: from b03ledav006.gho.boulder.ibm.com (b03ledav006.gho.boulder.ibm.com [9.17.130.237])
by b03cxnp07029.gho.boulder.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 122KmrXx45547812
(version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK);
Tue, 2 Mar 2021 20:48:53 GMT
Received: from b03ledav006.gho.boulder.ibm.com (unknown [127.0.0.1])
by IMSVA (Postfix) with ESMTP id AD6B3C605B;
Tue, 2 Mar 2021 20:48:53 +0000 (GMT)
Received: from b03ledav006.gho.boulder.ibm.com (unknown [127.0.0.1])
by IMSVA (Postfix) with ESMTP id 1B9ABC6055;
Tue, 2 Mar 2021 20:48:53 +0000 (GMT)
Received: from amdrome1.watson.ibm.com (unknown [9.2.130.16])
by b03ledav006.gho.boulder.ibm.com (Postfix) with ESMTP;
Tue, 2 Mar 2021 20:48:53 +0000 (GMT)
From: "Tobin Feldman-Fitzthum" <tobin@linux.ibm.com>
To: devel@edk2.groups.io
Cc: Dov Murik <dovmurik@linux.vnet.ibm.com>,
Tobin Feldman-Fitzthum <tobin@ibm.com>,
Tobin Feldman-Fitzthum <tobin@linux.ibm.com>,
James Bottomley <jejb@linux.ibm.com>,
Hubertus Franke <frankeh@us.ibm.com>,
Brijesh Singh <brijesh.singh@amd.com>,
Ashish Kalra <ashish.kalra@amd.com>, Jon Grimm <jon.grimm@amd.com>,
Tom Lendacky <thomas.lendacky@amd.com>
Subject: [RFC PATCH 11/14] OvmfPkg/AmdSev: Build page table for migration handler
Date: Tue, 2 Mar 2021 15:48:36 -0500
Message-Id: <20210302204839.82042-12-tobin@linux.ibm.com>
X-Mailer: git-send-email 2.20.1
In-Reply-To: <20210302204839.82042-1-tobin@linux.ibm.com>
References: <20210302204839.82042-1-tobin@linux.ibm.com>
MIME-Version: 1.0
X-TM-AS-GCONF: 00
X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.369,18.0.761
definitions=2021-03-02_08:2021-03-01,2021-03-02 signatures=0
X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 suspectscore=0
priorityscore=1501 bulkscore=0 adultscore=0 mlxlogscore=999 clxscore=1015
spamscore=0 impostorscore=0 mlxscore=0 lowpriorityscore=0 phishscore=0
malwarescore=0 classifier=spam adjust=0 reason=mlx scancount=1
engine=8.12.0-2009150000 definitions=main-2103020156
Content-Transfer-Encoding: 8bit
From: Dov Murik <dovmurik@linux.ibm.com>
The migration handler builds its own page tables and switches
to them. The MH pagetables are reserved as runtime memory.
When the hypervisor asks the MH to import/export a page, the HV
writes the guest physical address of the page in question to the
mailbox. The MH uses an identity mapping so that it can read/write
whatever GPA is requested by the HV. The hypervisor only asks the
MH to import/export encrypted pages. Thus, the C-Bit can be set
for every page in the identity map.
The MH also needs to read shared pages, such as the mailbox.
These are mapped at an offset. The offset must be added to
the physical address before it can be resolved.
Signed-off-by: Tobin Feldman-Fitzthum <tobin@linux.ibm.com>
Signed-off-by: Dov Murik <dovmurik@linux.vnet.ibm.com>
---
.../ConfidentialMigrationDxe.inf | 1 +
.../ConfidentialMigration/VirtualMemory.h | 177 ++++++++++++++++++
.../ConfidentialMigrationDxe.c | 88 ++++++++-
3 files changed, 265 insertions(+), 1 deletion(-)
create mode 100644 OvmfPkg/AmdSev/ConfidentialMigration/VirtualMemory.h
diff --git a/OvmfPkg/AmdSev/ConfidentialMigration/ConfidentialMigrationDxe.inf b/OvmfPkg/AmdSev/ConfidentialMigration/ConfidentialMigrationDxe.inf
index 49457d5d17..8dadfd1d13 100644
--- a/OvmfPkg/AmdSev/ConfidentialMigration/ConfidentialMigrationDxe.inf
+++ b/OvmfPkg/AmdSev/ConfidentialMigration/ConfidentialMigrationDxe.inf
@@ -15,6 +15,7 @@
[Sources]
ConfidentialMigrationDxe.c
+ VirtualMemory.h
[Packages]
MdePkg/MdePkg.dec
diff --git a/OvmfPkg/AmdSev/ConfidentialMigration/VirtualMemory.h b/OvmfPkg/AmdSev/ConfidentialMigration/VirtualMemory.h
new file mode 100644
index 0000000000..c50cb64c63
--- /dev/null
+++ b/OvmfPkg/AmdSev/ConfidentialMigration/VirtualMemory.h
@@ -0,0 +1,177 @@
+/** @file
+ Virtual Memory Management Services to set or clear the memory encryption bit
+ Copyright (c) 2006 - 2016, Intel Corporation. All rights reserved.<BR>
+ Copyright (c) 2017, AMD Incorporated. All rights reserved.<BR>
+ SPDX-License-Identifier: BSD-2-Clause-Patent
+ Code is derived from OvmfPkg/Library/BaseMemEncryptSevLib/X64/VirtualMemory.h
+
+**/
+
+#ifndef __VIRTUAL_MEMORY__
+#define __VIRTUAL_MEMORY__
+
+#include <Library/BaseLib.h>
+#include <Library/BaseMemoryLib.h>
+#include <Library/CacheMaintenanceLib.h>
+#include <Library/DebugLib.h>
+#include <Library/MemoryAllocationLib.h>
+#include <Uefi.h>
+
+#define SYS_CODE64_SEL 0x38
+
+#pragma pack(1)
+
+//
+// Page-Map Level-4 Offset (PML4) and
+// Page-Directory-Pointer Offset (PDPE) entries 4K & 2MB
+//
+
+typedef union {
+ struct {
+ UINT64 Present:1; // 0 = Not present in memory,
+ // 1 = Present in memory
+ UINT64 ReadWrite:1; // 0 = Read-Only, 1= Read/Write
+ UINT64 UserSupervisor:1; // 0 = Supervisor, 1=User
+ UINT64 WriteThrough:1; // 0 = Write-Back caching,
+ // 1 = Write-Through caching
+ UINT64 CacheDisabled:1; // 0 = Cached, 1=Non-Cached
+ UINT64 Accessed:1; // 0 = Not accessed,
+ // 1 = Accessed (set by CPU)
+ UINT64 Reserved:1; // Reserved
+ UINT64 MustBeZero:2; // Must Be Zero
+ UINT64 Available:3; // Available for use by system software
+ UINT64 PageTableBaseAddress:40; // Page Table Base Address
+ UINT64 AvabilableHigh:11; // Available for use by system software
+ UINT64 Nx:1; // No Execute bit
+ } Bits;
+ UINT64 Uint64;
+} PAGE_MAP_AND_DIRECTORY_POINTER;
+
+//
+// Page Table Entry 4KB
+//
+typedef union {
+ struct {
+ UINT64 Present:1; // 0 = Not present in memory,
+ // 1 = Present in memory
+ UINT64 ReadWrite:1; // 0 = Read-Only, 1= Read/Write
+ UINT64 UserSupervisor:1; // 0 = Supervisor, 1=User
+ UINT64 WriteThrough:1; // 0 = Write-Back caching,
+ // 1 = Write-Through caching
+ UINT64 CacheDisabled:1; // 0 = Cached, 1=Non-Cached
+ UINT64 Accessed:1; // 0 = Not accessed,
+ // 1 = Accessed (set by CPU)
+ UINT64 Dirty:1; // 0 = Not Dirty, 1 = written by
+ // processor on access to page
+ UINT64 PAT:1; //
+ UINT64 Global:1; // 0 = Not global page, 1 = global page
+ // TLB not cleared on CR3 write
+ UINT64 Available:3; // Available for use by system software
+ UINT64 PageTableBaseAddress:40; // Page Table Base Address
+ UINT64 AvabilableHigh:11; // Available for use by system software
+ UINT64 Nx:1; // 0 = Execute Code,
+ // 1 = No Code Execution
+ } Bits;
+ UINT64 Uint64;
+} PAGE_TABLE_4K_ENTRY;
+
+//
+// Page Table Entry 2MB
+//
+typedef union {
+ struct {
+ UINT64 Present:1; // 0 = Not present in memory,
+ // 1 = Present in memory
+ UINT64 ReadWrite:1; // 0 = Read-Only, 1= Read/Write
+ UINT64 UserSupervisor:1; // 0 = Supervisor, 1=User
+ UINT64 WriteThrough:1; // 0 = Write-Back caching,
+ // 1=Write-Through caching
+ UINT64 CacheDisabled:1; // 0 = Cached, 1=Non-Cached
+ UINT64 Accessed:1; // 0 = Not accessed,
+ // 1 = Accessed (set by CPU)
+ UINT64 Dirty:1; // 0 = Not Dirty, 1 = written by
+ // processor on access to page
+ UINT64 MustBe1:1; // Must be 1
+ UINT64 Global:1; // 0 = Not global page, 1 = global page
+ // TLB not cleared on CR3 write
+ UINT64 Available:3; // Available for use by system software
+ UINT64 PAT:1; //
+ UINT64 MustBeZero:8; // Must be zero;
+ UINT64 PageTableBaseAddress:31; // Page Table Base Address
+ UINT64 AvabilableHigh:11; // Available for use by system software
+ UINT64 Nx:1; // 0 = Execute Code,
+ // 1 = No Code Execution
+ } Bits;
+ UINT64 Uint64;
+} PAGE_TABLE_ENTRY;
+
+//
+// Page Table Entry 1GB
+//
+typedef union {
+ struct {
+ UINT64 Present:1; // 0 = Not present in memory,
+ // 1 = Present in memory
+ UINT64 ReadWrite:1; // 0 = Read-Only, 1= Read/Write
+ UINT64 UserSupervisor:1; // 0 = Supervisor, 1=User
+ UINT64 WriteThrough:1; // 0 = Write-Back caching,
+ // 1 = Write-Through caching
+ UINT64 CacheDisabled:1; // 0 = Cached, 1=Non-Cached
+ UINT64 Accessed:1; // 0 = Not accessed,
+ // 1 = Accessed (set by CPU)
+ UINT64 Dirty:1; // 0 = Not Dirty, 1 = written by
+ // processor on access to page
+ UINT64 MustBe1:1; // Must be 1
+ UINT64 Global:1; // 0 = Not global page, 1 = global page
+ // TLB not cleared on CR3 write
+ UINT64 Available:3; // Available for use by system software
+ UINT64 PAT:1; //
+ UINT64 MustBeZero:17; // Must be zero;
+ UINT64 PageTableBaseAddress:22; // Page Table Base Address
+ UINT64 AvabilableHigh:11; // Available for use by system software
+ UINT64 Nx:1; // 0 = Execute Code,
+ // 1 = No Code Execution
+ } Bits;
+ UINT64 Uint64;
+} PAGE_TABLE_1G_ENTRY;
+
+#pragma pack()
+
+#define IA32_PG_P BIT0
+#define IA32_PG_RW BIT1
+#define IA32_PG_PS BIT7
+
+#define PAGING_PAE_INDEX_MASK 0x1FF
+
+#define PAGING_4K_ADDRESS_MASK_64 0x000FFFFFFFFFF000ull
+#define PAGING_2M_ADDRESS_MASK_64 0x000FFFFFFFE00000ull
+#define PAGING_1G_ADDRESS_MASK_64 0x000FFFFFC0000000ull
+
+#define PAGING_L1_ADDRESS_SHIFT 12
+#define PAGING_L2_ADDRESS_SHIFT 21
+#define PAGING_L3_ADDRESS_SHIFT 30
+#define PAGING_L4_ADDRESS_SHIFT 39
+
+#define PAGING_PML4E_NUMBER 4
+
+#define PAGETABLE_ENTRY_MASK ((1UL << 9) - 1)
+#define PML4_OFFSET(x) ( (x >> 39) & PAGETABLE_ENTRY_MASK)
+#define PDP_OFFSET(x) ( (x >> 30) & PAGETABLE_ENTRY_MASK)
+#define PDE_OFFSET(x) ( (x >> 21) & PAGETABLE_ENTRY_MASK)
+#define PTE_OFFSET(x) ( (x >> 12) & PAGETABLE_ENTRY_MASK)
+#define PAGING_1G_ADDRESS_MASK_64 0x000FFFFFC0000000ull
+
+#define PAGE_TABLE_POOL_ALIGNMENT BASE_2MB
+#define PAGE_TABLE_POOL_UNIT_SIZE SIZE_2MB
+#define PAGE_TABLE_POOL_UNIT_PAGES \
+ EFI_SIZE_TO_PAGES (PAGE_TABLE_POOL_UNIT_SIZE)
+#define PAGE_TABLE_POOL_ALIGN_MASK \
+ (~(EFI_PHYSICAL_ADDRESS)(PAGE_TABLE_POOL_ALIGNMENT - 1))
+
+typedef struct {
+ VOID *NextPool;
+ UINTN Offset;
+ UINTN FreePages;
+} PAGE_TABLE_POOL;
+
+#endif
diff --git a/OvmfPkg/AmdSev/ConfidentialMigration/ConfidentialMigrationDxe.c b/OvmfPkg/AmdSev/ConfidentialMigration/ConfidentialMigrationDxe.c
index 8402fcc4fa..3df3b09732 100644
--- a/OvmfPkg/AmdSev/ConfidentialMigration/ConfidentialMigrationDxe.c
+++ b/OvmfPkg/AmdSev/ConfidentialMigration/ConfidentialMigrationDxe.c
@@ -11,6 +11,7 @@
#include <Protocol/MpService.h>
#include <Library/BaseMemoryLib.h>
+#include "VirtualMemory.h"
//
// Functions implemented by the migration handler
//
@@ -43,6 +44,83 @@ typedef volatile struct {
UINT32 done;
} MH_COMMAND_PARAMETERS;
+//
+// Addresses for MH page table.
+//
+STATIC PAGE_TABLE_POOL *mPageTablePool = NULL;
+STATIC PHYSICAL_ADDRESS mMigrationHelperPageTables = 0;
+
+//
+// Offset for non-cbit mapping.
+//
+#define UNENC_VIRT_ADDR_BASE 0xffffff8000000000ULL
+
+
+/**
+ Allocates and fills in custom page tables for Migration Handler.
+ The MH must be able to write to any encrypted page. Thus, it
+ uses an identity map where the C-bit is set for every page. The
+ HV should never ask the MH to import/export a shared page. The
+ MH must also be able to read some shared pages. The first 1GB
+ of memory is mapped at offset UNENC_VIRT_ADDR_BASE.
+
+**/
+VOID
+PrepareMigrationHandlerPageTables (
+ VOID
+ )
+{
+ UINTN PoolPages;
+ VOID *Buffer;
+ VOID *Start;
+ PAGE_MAP_AND_DIRECTORY_POINTER *PageMapLevel4Entry;
+ PAGE_TABLE_1G_ENTRY *PageDirectory1GEntry;
+ PAGE_TABLE_1G_ENTRY *Unenc1GEntry;
+ UINT64 AddressEncMask;
+
+ PoolPages = 1 + 10;
+ Buffer = AllocateAlignedRuntimePages (PoolPages, PAGE_TABLE_POOL_ALIGNMENT);
+ mPageTablePool = Buffer;
+ mPageTablePool->NextPool = mPageTablePool;
+ mPageTablePool->FreePages = PoolPages - 1;
+ mPageTablePool->Offset = EFI_PAGES_TO_SIZE (1);
+
+ Start = (UINT8 *)mPageTablePool + mPageTablePool->Offset;
+ ZeroMem(Start, mPageTablePool->FreePages * EFI_PAGE_SIZE);
+
+ AddressEncMask = 1ULL << 47;
+
+ PageMapLevel4Entry = Start;
+ PageDirectory1GEntry = (PAGE_TABLE_1G_ENTRY*)((UINT8*)Start + EFI_PAGE_SIZE);
+ Unenc1GEntry = (PAGE_TABLE_1G_ENTRY*)((UINT8*)Start + 2 * EFI_PAGE_SIZE);
+
+ PageMapLevel4Entry = Start;
+ PageMapLevel4Entry += PML4_OFFSET(0x0ULL);
+ PageMapLevel4Entry->Uint64 = (UINT64)PageDirectory1GEntry | AddressEncMask | 0x23;
+
+ PageMapLevel4Entry = Start;
+ PageMapLevel4Entry += PML4_OFFSET(UNENC_VIRT_ADDR_BASE); // should be 511
+ PageMapLevel4Entry->Uint64 = (UINT64)Unenc1GEntry | AddressEncMask | 0x23;
+
+ UINT64 PageAddr = 0;
+ for (int i = 0; i < 512; i++, PageAddr += SIZE_1GB) {
+ PAGE_TABLE_1G_ENTRY *e = PageDirectory1GEntry + i;
+ e->Uint64 = PageAddr | AddressEncMask | 0xe3; // 1GB page
+ }
+
+ UINT64 UnencPageAddr = 0;
+ Unenc1GEntry->Uint64 = UnencPageAddr | 0xe3; // 1GB page unencrypted
+
+ mMigrationHelperPageTables = (UINT64)Start | AddressEncMask;
+}
+
+VOID
+SwitchToMigrationHelperPageTables(VOID)
+{
+ AsmWriteCr3(mMigrationHelperPageTables);
+}
+
+
VOID
EFIAPI
@@ -56,7 +134,12 @@ MigrationHandlerMain (
DebugPrint (DEBUG_INFO,"MIGRATION Handler Started\n");
- params_base = PcdGet32 (PcdConfidentialMigrationMailboxBase);
+ SwitchToMigrationHelperPageTables();
+
+ //
+ // Shared pages must be offset by UNENC_VIRT_ADDR_BASE.
+ //
+ params_base = PcdGet32 (PcdConfidentialMigrationMailboxBase) + UNENC_VIRT_ADDR_BASE;
params = (VOID *)params_base;
page_va = (VOID *)params_base + 0x1000;
@@ -134,6 +217,8 @@ LaunchMigrationHandler (
MigrationHandlerCpuIndex = NumProc - 1;
+ PrepareMigrationHandlerPageTables();
+
EFI_EVENT Event;
MpProto->GetProcessorInfo (MpProto, MigrationHandlerCpuIndex, &Tcb);
if (Tcb.StatusFlag != 7) {
@@ -154,6 +239,7 @@ LaunchMigrationHandler (
if (PcdGetBool(PcdIsConfidentialMigrationTarget)) {
DebugPrint (DEBUG_INFO,"Waiting for incoming confidential migration.\n");
DisableInterrupts ();
+ SwitchToMigrationHelperPageTables();
CpuDeadLoop ();
}
--
2.20.1