From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from NAM11-BN8-obe.outbound.protection.outlook.com (NAM11-BN8-obe.outbound.protection.outlook.com [40.107.236.85]) by mx.groups.io with SMTP id smtpd.web11.4798.1631964624900850299 for ; Sat, 18 Sep 2021 04:30:25 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=@amd.com header.s=selector1 header.b=tYV9B9/b; spf=permerror, err=parse error for token &{10 18 %{i}._ip.%{h}._ehlo.%{d}._spf.vali.email}: invalid domain name (domain: amd.com, ip: 40.107.236.85, mailfrom: brijesh.singh@amd.com) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Ox/ozKtMn/n+7bzxNFzAKE/w6TXA32EWnnjUrszsxVFXe30lqB6kOGxl58MbgT9+FgybkjikXCfDgKHjBHpOG/IOJ2Nxil2YBnqCmE9vM1C/O7s79vFWfX5CIl99qVALQ2uTJeCoxvF49fpqJwKyGP5IUco2pjUQeTegGUiXO8TpSTJAI6Mn2sTSjGwgbhTAoFZJSzaHx5+AOEfbX6d1f19aWDZjtmVFRDpYXXJRN/5aEYWkNpVtJbqQjvdFBnRONBQeemwm5dCT9/egUjcIQ7Rxa0iOStOd15UPtbDr4wyonUOCGOroIUy4bAGTwjRMOzzPsdd5eQ8PNtxCgVRC9A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=uGzgOLlccqZyw/dDrjb6TflEZo07CIdZNGwwe18lFAE=; b=GbhoHQpOOfZJoEbCIc7XdOBe6c6/fNsxpZfaUW73B4SWp/YvhJ6tdHfJFY+dz9yHjNjlcl2W1sl+Khgwxga9vRT98qINyzPcnPKdWcOWOdFvIQ9I+QY1ZEJagkexV5dRydTwqFA52Yc5r1DM0XYpkq7LzOXbItjD8DQJc3cMDWogiW/e4dYVFZ6ZeneMlzKoLyNUNvv8vBZPs2zUxt0NLRg9ZZGbuqjr7hGVtUDjCq7DthabmXEjeHiojAczCyJyFSszDWOiBXTPH3IBQog5kHe4FRqYF1aHEUnDrHQyE+IR8khGUkhlkYg2A38bph4QKVKDrXWiciF7qxF9bfSYgA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=amd.com; dmarc=pass action=none header.from=amd.com; dkim=pass header.d=amd.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=uGzgOLlccqZyw/dDrjb6TflEZo07CIdZNGwwe18lFAE=; b=tYV9B9/bWNyVplZ6/zQYunhrZ6SEyS7LJC+1bBKLW3UR67CV293BRAgE06z4gyuVKM5J0qh4bqumKrYHC/I2K5x7XNq3J4VvGcVL4kCZ/qjrJEpt1+KSTHt6vTJySrwxR6ICCxCvkAr9twSfZUReeu9Lil+/qmeuX/R5Wj6sMR4= Authentication-Results: amd.com; dkim=none (message not signed) header.d=none;amd.com; dmarc=none action=none header.from=amd.com; Received: from SN6PR12MB2718.namprd12.prod.outlook.com (2603:10b6:805:6f::22) by SA0PR12MB4590.namprd12.prod.outlook.com (2603:10b6:806:93::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.16; Sat, 18 Sep 2021 11:30:22 +0000 Received: from SN6PR12MB2718.namprd12.prod.outlook.com ([fe80::78b7:7336:d363:9be3]) by SN6PR12MB2718.namprd12.prod.outlook.com ([fe80::78b7:7336:d363:9be3%6]) with mapi id 15.20.4523.018; Sat, 18 Sep 2021 11:30:22 +0000 Subject: Re: [edk2-devel] [PATCH V6 1/1] OvmfPkg: Enable TDX in ResetVector To: "Xu, Min M" , "devel@edk2.groups.io" , Vishal Annapurve Cc: Ard Biesheuvel , "Justen, Jordan L" , Gerd Hoffmann , Erdem Aktas , James Bottomley , "Yao, Jiewen" , Tom Lendacky References: <2d085336-386b-8492-5f0e-ce9e0c49e8b6@amd.com> <9757562a-d155-be50-45fe-926ce9427e1b@amd.com> From: "Brijesh Singh" Message-ID: <3069705d-bf76-bcb7-e94f-dec9d19b47d6@amd.com> Date: Sat, 18 Sep 2021 06:30:20 -0500 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:78.0) Gecko/20100101 Thunderbird/78.14.0 In-Reply-To: X-ClientProxiedBy: SN6PR16CA0070.namprd16.prod.outlook.com (2603:10b6:805:ca::47) To SN6PR12MB2718.namprd12.prod.outlook.com (2603:10b6:805:6f::22) Return-Path: brijesh.singh@amd.com MIME-Version: 1.0 Received: from Brijeshs-MacBook-Pro.local (70.112.153.56) by SN6PR16CA0070.namprd16.prod.outlook.com (2603:10b6:805:ca::47) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4523.14 via Frontend Transport; Sat, 18 Sep 2021 11:30:21 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: f49e2942-4eb6-42ec-0fea-08d97a97b362 X-MS-TrafficTypeDiagnostic: SA0PR12MB4590: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:9508; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: vw1nCWCA1tzREetpDZs/RjBLc91Rmuog2xNttz3mFVMX6bwQtzNjEgLJ0onPHgaRnCcjt9/xg14y+pCD/0w93/oc56knPBHK++hk1SKJVm4UjfTyAU4wnHm2Xjvl6CaANWiecZnjxkCNpNWa2iHJzOWjGcf9VqnytK3iByEBk9Q1NbHEVsZC12Aq/M/GnTPZ8A1DYvINqpNIZ0PYd8yyPjCSaenhGgBsSKdzwfyqmrm8yTXGnPQpBWBz5ClW4eT2rJC5kfz2iNBYv6cVTsXiz7QlfqXilSXGxT6iic+sbl+wu/hWvNSz+8yfk0swfzRc93XLB/ZlZnvKqHGXfPZTt2cy0NwI+OR4ULuZ0/7htUbSDo3bNTjV7YIX9FUTHfAs5Vwa/sRp3y/D/bjO6u0DeP6nF9gc2U/m8D0BNn/9HpIpyUlnZzU+EH8DqYuMKECUDowHbWNviJstJ2Ncu7c7B1kmHameYL5dOJV870Ee5x6j3ZTBAOi0NRcs27JO+uL3UHTDu27aVXIGYb0kWgDQGGKJKE0fUEZWP6KEz+Edc1yC+y2oq4jiWzRvR6GylzTbRT/vk64XfhaZULzjyC3fmX58waPExzc5gV2Zt6lOy7RrmUCepIjWcoHVmh9fT8CgNfYl9RHNY/csdyJYTaZR49MNvWWfPkiV7n+Bx9bDKt1LdXVbzRRVLg44xBPjQ8iH5TyqEOSkvispH6fpv19yVgQWy0f0TJWAcSBJDD2IdcKJCr8RhsjTKTiH4M+pgmoyFRpnGqrUJeAJNGchuYqsNw== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:SN6PR12MB2718.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(376002)(136003)(39860400002)(346002)(396003)(52116002)(83380400001)(53546011)(110136005)(6506007)(31686004)(6486002)(54906003)(186003)(38350700002)(2906002)(6512007)(956004)(5660300002)(66946007)(2616005)(26005)(38100700002)(36756003)(66476007)(478600001)(66556008)(316002)(8676002)(31696002)(44832011)(4326008)(19627235002)(86362001)(8936002)(45980500001)(43740500002);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?bFl1N3dzeStGSW9MYU5BZy9NOEdzOU5JcmREUkY5Z2ZCQkRTTE5BQ0xQYmdQ?= =?utf-8?B?UkJ5YW85dDZqWDJTV3FMejdWdTEwVi9McnZCWk5HVjNRRTBUVlJuT1JUMXJr?= =?utf-8?B?REJWNWVtdzBKM2R3bWFJdXlrQmRJanVISDlYa09wdGZ0d2xvUFRVQnMwdG1Y?= =?utf-8?B?M0lUYVN2aTVGbXFtbklFTVlBLys5VmJCeWlnQ1YxdTBGbE02MGRPQW1OZmxz?= =?utf-8?B?SnBYMWJWS1VNbEtqNFJNeE5yNnN1Z2JseGE4bFVjWCt4dTR6RlNibFRhWnRR?= =?utf-8?B?T2ZVNWxMWC8vbHk0WUpGcWZUenIzY3pNbnR0SUkvOHJzT0h2QjY0blAvK1ps?= =?utf-8?B?SDVHem8zeU12L0V1SDB3RkJMd05mR3F4YXNjL0J5bUx0dStvOVQrUWhPUnIr?= =?utf-8?B?S0p6a1p4c2JMdHQvc1l0V0Y0RmxyWGorUEFzYjcySkNVYkxuZ1V5dW5wdkhW?= =?utf-8?B?SHdiMXkvMEs0dm5OUmRwVEZIbVd4aklpMG5KVDloSERrUjFrUWJqOFcrMkxE?= =?utf-8?B?WG9NZVRCekZIMUhrZnA3QTJUcjNSd0pvZ2VmaHgvbmdReHJZd0dZZWVnVDRK?= =?utf-8?B?R1FHQXRaaEx5SGNHT2lkeTBrVk00QjlZU0x5NGMrTENHUU9yNFh6SXdWUFF2?= =?utf-8?B?SDN5SlQ5aVRwUE1wVmVLMmUwZmhYYjA4MVVCVW41OFRyU2Y3TzN0M1RWOXNR?= =?utf-8?B?TEE2RGhQSlMySVlCdHZURGZYdVQ2ZGJJcHdnNmh5aFFsc1AyQUhHWkJwVUEw?= =?utf-8?B?aHVuVVRNazF3SHpNWno3Z1owTmNxSUFvalE5VjBBUXVrRm82MURiRWlGTkZT?= =?utf-8?B?eWxtTW9rdDdEOHpkcTR3ZGtaWnBXSzZIK29lNGxSOWdnQzdTdkNnVkVDYTV0?= =?utf-8?B?WkswcCtzTXNhMXJyZDlpUHYxR084bU4yUFBUQjZ5Q2pPYVZybkFpZFdPU0ts?= =?utf-8?B?cEVaZmRzc3oxUUl1ZnluckE2a01iU29lVUVqcWR6VFVicy9xVVVGaHlDRmtV?= =?utf-8?B?RGVFUkRGZDcyaFhocERPbVAzM3BQVHdlV1J2dWJFbHdhM29BNWpOTFRRS0k4?= =?utf-8?B?cmdDNzkxTE5oeGlZejI2RFB5WlRjNEJ4Y3pMcmlIV3A1VmVhRmJybUExRTFF?= =?utf-8?B?QWdTeFhKd2YyYkloeFU3YTF1S3pBNkxWZktvbFowcmtvR2tXSnQyak9sZHNP?= =?utf-8?B?Uk9aUUhOZUU3aUdaZjRUSmpyVEY2K2xBdXFkSFZFb3JVVFI4UjRkbVNpMEVV?= =?utf-8?B?VkV2aitJY0VBeHVmQm1PS3FFVmRhaUEwUVo2WTNFY1BXOHY3SHNSZ3JtYTU2?= =?utf-8?B?cTBZMUxqS2NSS3ppdXhKQ0xQOTZGWVhxM3dBM201UVY5SVlWK1FXMmZDUE1L?= =?utf-8?B?cHV6N0ZvTGh6aXdzVXE3SytBMU1BazNFbUNIVm9aVVlFN2Q5T0ZPS2M1NFJU?= =?utf-8?B?V3JON3EvMjVSYVFFaFIwZmM3WkVCcTh2eVlaNmwvME9abk9aNmZyYklzcFY1?= =?utf-8?B?VkluQ3ZpZ3Z4NWl3Wk1xKzBVTXljTkcvd2hKQ0VJOUF4T3dNbTFFV1lDdDJE?= =?utf-8?B?YnVZdGd2anVZaHlYc1ZHcGcycTkxVnlSN1QrSURvc2UzRlpmVkVuMTluOTlN?= =?utf-8?B?QzBxRElWdkQ0dk8reVpiQjJHMk1xWmRscWxMWURrUnFWTXZHbW1xRjlqQlg5?= =?utf-8?B?M3p1RzFyamlGOXRTNmtvQzBBZjg1ak5xV1JrbjE5UE9DamVCNVlnRWhUc2Z2?= =?utf-8?Q?gqf6JKoXDFvMYRwDobv8lY3qZWkePMlFl+ecc5U?= X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-Network-Message-Id: f49e2942-4eb6-42ec-0fea-08d97a97b362 X-MS-Exchange-CrossTenant-AuthSource: SN6PR12MB2718.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 18 Sep 2021 11:30:22.0482 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: nIZi5iCcyhHIjlMMo5HpiuRWEcADfm5oybJZ0iv2dwj4GUGNuAUW78a2mNh7LIMGeK8/2ZJVdMTNmXJBFT2qfw== X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA0PR12MB4590 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit Content-Language: en-US Hi Min, On 9/18/21 12:16 AM, Xu, Min M wrote: > Hi, Brijesh > > On September 17, 2021 11:52 PM, Brijesh Singh wrote: >> Hi Min, >> >> On 9/17/21 7:55 AM, Xu, Min M wrote: >> ... >> >>> As I mentioned in my last mail, in the beginning I missed the limitation of >> smsw. >>> So I update the code (ResetVectorVtf0.asm) as below using mov CRx. >>> <1> BITS 16 >>> 176 00000800 0F20C0 <1> mov eax, cr0 <-- previously it >> was smsw >>> 177 00000803 A801 <1> test al, 1 >>> 178 00000805 7405 <1> jz .Real >>> 179 <1> BITS 32 >>> 180 00000807 E951FFFFFF <1> jmp Main32 >>> 181 <1> BITS 16 >>> 182 <1> .Real: >>> 183 0000080C E939FF <1> jmp EarlyBspInitReal16 >>> >>> I test the code in a AMD SEV server and try to launch a SEV guest. This time >> it stuck at the *mov eax, cr0*. >>> I am curious if *mov eax, cr0* works in real mode in a SEV guest? >>> I also test the code in a legacy vm guest and td guest, all passed. >>> Did I miss something? >>> >> Hmm, I am not aware of any limitation w.r.t encrypted VMs. I just added the >> below code in my branch and I do not see any issues, my SEV, SEV-ES and >> SEV-SNP all are able to boot fine. And KVM trace confirms that code it read >> >> diff --git a/OvmfPkg/ResetVector/Ia16/ResetVectorVtf0.asm >> b/OvmfPkg/ResetVector/Ia16/ResetVectorVtf0.asm >> index f0e509d0672e..98e34332b04c 100644 >> --- a/OvmfPkg/ResetVector/Ia16/ResetVectorVtf0.asm >> +++ b/OvmfPkg/ResetVector/Ia16/ResetVectorVtf0.asm >> @@ -175,9 +175,21 @@ resetVector: >> ; >> ; This is where the processor will begin execution >> ; >> +%ifdef ARCH_IA32 >> nop >> nop >> jmp EarlyBspInitReal16 >> +%else >> + mov eax, cr0 >> + test al, 1 >> + jz .Real >> +BITS 32 >> + hlt >> + ;jmp Main32 >> +BITS 16 >> +.Real: >> + jmp EarlyBspInitReal16 >> +%endif >> >> ALIGN 16 >> >> >> And KVM trace: >> >> kvm_exit: vcpu 0 reason npf rip 0xfff0 info1 0x0000000500000014 info2 >> 0x00000000fffff000 intr_info 0x00000000 error_code 0x00000000 >> kvm_page_fault: address fffff000 error_code 500000014 >> kvm_entry: vcpu 0, rip 0xfff0 >> kvm_exit: vcpu 0 reason read_cr0 rip 0xfff0 info1 0x8000000000000000 >> info2 0x0000000000000000 intr_info 0x00000000 error_code 0x00000000 >> kvm_cr: cr_read 0 = 0x60000010 >> kvm_entry: vcpu 0, rip 0xfff3 >> >> As we can see from the kvm trace, the first instruction here is the Cr0 read >> and it was successfully intercepted and rip moved to next instruction. >> >> Can you please provide me KVM trace for your failure case ? Also, provide me >> the output of "lscpu" and "dmesg" from the host. > The OVMF image you tested is built with GCC tool chain, right? Yes, we have been using the GCC tool chain only. > I usually do the development in windows and build the OVMF image with VS2019. > If the new feature works, then I cherry-pick the patch-sets to code base in ubuntu > 18.04 and build/test the new feature. > > The weird thing is that, with VS2019, even the OVMF image is built from edk2-master, > such image doesn't work on AMD SEV server either. But if the image is built by Ubuntu 18.04, > it does work on AMD SEV server. This seems very strange that we are failing to execute the hand written assembly code. I am wondering if somehow the VS compiler is generating a wrong byte code and thus causing a trap on KVM that requires emulation. Since the guest memory is encrypted, so KVM emulation code will not be able to decode the instruction bytes and thus leading in repetitive nested fault. Only way I could verify my theory is if I can get a KVM trace or an OVMF binary. If you have have KVM trace or OVMF_CODE.fd handy then please share. > I applied my TDX patch-sets to the code base on my Ubuntu 18.04, and build the image. > This image does work in both AMD SEV server and Intel TDX server. > > Thanks! > Min