From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from NAM12-MW2-obe.outbound.protection.outlook.com (NAM12-MW2-obe.outbound.protection.outlook.com [40.107.244.88]) by mx.groups.io with SMTP id smtpd.web10.39505.1650288981291649916 for ; Mon, 18 Apr 2022 06:36:21 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=@amd.com header.s=selector1 header.b=LPy4/S24; spf=permerror, err=parse error for token &{10 18 %{i}._ip.%{h}._ehlo.%{d}._spf.vali.email}: invalid domain name (domain: amd.com, ip: 40.107.244.88, mailfrom: thomas.lendacky@amd.com) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=GjmTEn/YLuWkPiIYIRRaiEvVXNJKz6zlY+OlvxEGamv0iD43LcD5xlQgkJJ3Gzkx7ZyOqkMA4XR3IescHqArhf1VWPSlj9iEJYSNyTFJX4RlexBZfkp6YyzrIHi4JPsVqzcYp8r1NigKJzB947qE2EYheaSNJAafkRapIuMtWsS40Ur9zc/tCuMG+oxqjJA7G4W0Iq+7rKEpEdRNYXk2hWGOi7Tj4/qEBZtwLzG5IZszLcz0DxCsZR+yQoYV9iSWY01ZFCHogR/yC1DVXK9eVm28bwMSbe9C6BehxUWoAkSUtK6acWa5WjAmH+UwDCMA3+G/4BtisnXYycU31SQWUw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=AiEuY6zOxmsQH7jaIU/hFpixfcgKxAQFO3pFSREgZ3Q=; b=DS+GKnBaH0V1Yqz1DYcf512HD+ekPbKPwyjnEIHrD/q30FetBBdWUcjBwcMXDvR1HlOwF29NrVRc66pK70l+3D2yG/DgfoSVUNyZk4UqYk3VezlsJfK/CNJDEXrkXSwIa/uhlPLZJQ1pi623JBwac87lL04wwYghSnbYnQwgEQXxDanVPpDue05oORQRdH1rcAC3InV+xXGy993diWLJj5j5wXU0r9Y1NReR9rXjhOq/OovZznahBkq02OI1EohYv0zg9hK7mqGJ+dyXrMMfV2D4tWtRxeuegGKJqXGg1ZjrWsET5SfEaqG8OWSgZdHmXqRWLkgQtwnASfnPQh8T7g== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=amd.com; dmarc=pass action=none header.from=amd.com; dkim=pass header.d=amd.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=AiEuY6zOxmsQH7jaIU/hFpixfcgKxAQFO3pFSREgZ3Q=; b=LPy4/S24mT4p2vK/mxfgTTeZhRJMxGpGrVVggZsYXT/GY5SknvztBYRgQ9SCu5yIPVXeMPTXg5agphxeevkGB8u1tSWTC3itPo3WT6vZGfi8Wnn9XMkx0SqkRacdNAcwaxl3YGnyOxthbLwTDw7QF5ZCHySHrQDVhJh45DS5oGA= Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=amd.com; Received: from DM4PR12MB5229.namprd12.prod.outlook.com (2603:10b6:5:398::12) by BYAPR12MB2936.namprd12.prod.outlook.com (2603:10b6:a03:12f::17) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5164.20; Mon, 18 Apr 2022 13:36:17 +0000 Received: from DM4PR12MB5229.namprd12.prod.outlook.com ([fe80::781d:15d6:8f63:a4e7]) by DM4PR12MB5229.namprd12.prod.outlook.com ([fe80::781d:15d6:8f63:a4e7%5]) with mapi id 15.20.5164.025; Mon, 18 Apr 2022 13:36:17 +0000 Message-ID: <69b0aaa5-3cb9-7279-b55f-6904da6e0e6c@amd.com> Date: Mon, 18 Apr 2022 08:36:14 -0500 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101 Thunderbird/91.7.0 Subject: Re: [edk2-devel] [PATCH V12 42/47] OvmfPkg: Add TdxDxe driver To: "Xu, Min M" , "devel@edk2.groups.io" Cc: Ard Biesheuvel , "Justen, Jordan L" , Brijesh Singh , "Aktas, Erdem" , James Bottomley , "Yao, Jiewen" , Gerd Hoffmann References: <4c3aa6915fe7aac06940bea0f9bc5fdd3c539121.1648555175.git.min.m.xu@intel.com> <5832e647-63d5-6bbe-4daa-f2f5591a7324@amd.com> <8310fb45-f3be-73c3-d914-7ae6ae89d0e1@amd.com> From: "Lendacky, Thomas" In-Reply-To: X-ClientProxiedBy: MN2PR12CA0008.namprd12.prod.outlook.com (2603:10b6:208:a8::21) To DM4PR12MB5229.namprd12.prod.outlook.com (2603:10b6:5:398::12) Return-Path: Thomas.Lendacky@amd.com MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 1bc56cc9-7fae-4f9c-c57c-08da21406a3c X-MS-TrafficTypeDiagnostic: BYAPR12MB2936:EE_ X-Microsoft-Antispam-PRVS: X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: ehttniVwnk6M5Dx6akISS68MUcLexmDX9iWXnBhlQX1RqMKmWziXDJjjrEpwXO9YkCaRs3yVnD+VEMgDJQeH0FlNoj0FI5mbtDWP+bFiw+aPZrh7fP5JxgcVKXdOd0pfusnDra4mj6dGj/onWEBB4g0vEWwzzB64G67ChXZ2md5EBVm7+SJwTpfU5UJAycWoI3ZGYHqvSuWIgI0kEPuBXgDtz/ML5gfNqotQwTQIz4xLROJo4kjzDU9kx+kHgjAreofCVnbbtqLl1m3uyOZFHO9inAM6G+2xxP0hGna8XQa+y/mLpMqOOazhAv6s8Jq0DuLXzzn7eTSKMx2FG1QeyFNT51qYMzB6F1uySHsZzH5fW1bLM2/ERfb9B6zUCoyNBU8ZMtYU0HoaL3C1aUAXq7cY90QCzo1ne+XJ9WPAXBsf1ueETynbDemNMT1UUwbimngatUt1xC6JyecbdLfm3PYDlSsYAweAdyd3YC2Z6of6WoLh5NaUaaFSHuwuR+IFnaYKhn1iJrCzQO860dfKUoDdF/5PT9XcaKeLpjE5kgh6/+Xp4zftYKLgBDpSHplw246pe8/36ceIKjxBZjGjKEEps/ig8YpJXxYfn+p2A/GYt/A86fFKjPOioehwxHLqeJrniZ3OSTSzdTcgCa+JzczVmeg9hIclUjITS0Nb3iOHMM5MpIWiIJiG5XRs3qK4y0wJAWmTTbRu2PPU/m6uph4a9159v/uf5li2WefMM6Y= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DM4PR12MB5229.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230001)(4636009)(366004)(84040400005)(316002)(19627235002)(26005)(6512007)(6506007)(31696002)(110136005)(54906003)(5660300002)(8936002)(2616005)(36756003)(6666004)(186003)(53546011)(2906002)(6486002)(31686004)(86362001)(38100700002)(508600001)(4326008)(8676002)(66476007)(66556008)(66946007)(45980500001)(43740500002);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?NVhydnB3YTUwd0dzY1FlTW53T3pkTHgwSVQ2bDBBc2J1UVQ4eTZvUENrbXE2?= =?utf-8?B?a0pkK0hJV3Y4MTdydzBkYkRPL1FSSjFSZ0NwL21kYjh1KzYvK24xN0RxeDEz?= =?utf-8?B?RVV2Zmw1TVppczEyajBZaCtWUERTOFpjWHcyMkpBVlg3SXU1ejNQOE1YZDVv?= =?utf-8?B?ZkwvNVBPYnVjRXFjYVRraE95MnVickJKNHJWSENJWnN6SisxaU9qM21xQ1ZK?= =?utf-8?B?SVpqdWJMSTZxcERySWJzOEdQQURkTkNPbFZYTmJwRU40cXNWb25qc2F3Ym9H?= =?utf-8?B?WitxNWtleWVWL2hoR1l5NjY4THhsS2pZaEVNeXZSV2pYcW8ySkhCdDltTUtX?= =?utf-8?B?LzgvcWorYVlBQmdZOXIwVHpqSGVRWVVtbkJWV1pzaG5iSXoxUVZYdlhiai9P?= =?utf-8?B?UHRvaWt0OTBSbWlGbkZKc2VWczR6UWZpbEwzMWRBY2tVbHVpdEJ4MGlFeDVz?= =?utf-8?B?M0l5aXNvZGpzQzQ2TGRNeUt0Zm1zN2JwQjVFT0FneE54MU9iYTJUZGtKb0Y0?= =?utf-8?B?bzF0Wm0zNThCTEdWRzh0MStncEZFdGVMeWVmMWVvbWxBaUlPRE1BUzdmb2tk?= =?utf-8?B?dWEwK2VvT0tOYlVkb1dxN3BITXpQL3RUTnlUQXlZWVluc1VUZDhSalVxWHoy?= =?utf-8?B?Y3FsRXd2dlRJdnNYRDl5SEJ4L3FKdjlNeHNoWVBkeWNablVpcFF0c0toYzM0?= =?utf-8?B?K1NrR2Vya29qZlpSRzYzbVBSRUQ5R0pVSXFWeUhMQ3RnNXNUd0dwSWxyV1o2?= =?utf-8?B?amxacHdOa3ovcnEySGZCa0xjcWc3dUNtMjJkK3pTS2ZwS3g0VCtMby9Mc3Ja?= =?utf-8?B?TDZXeUlqV2hMRHZ1b3JCT1ZKZXduMmlkRHN6Z0MrcUZMU0lNQXBsR2RKRlNu?= =?utf-8?B?YlIzcGVsN2hmdnRITzlUNXd1a285MzYrWnM4RHFkaHE4dnlhOVNoQnphRzVh?= =?utf-8?B?WGxYZlpWcmd5Tnpnb3BHNXhJZlp2YUEwSnVQWEJGcjNiOXJ0MmhwM081UjRm?= =?utf-8?B?VVFmbzRHek1uOFhkS0hSSVRIa2ZkSGZ6endZanhZNUtDamlzNlpHOGlXTEZS?= =?utf-8?B?T2hHWkJDTDJTdFBSTFd5RFkvK05LTTFyVkJ0UFpCajBWRWZNc0JweXJCc3JW?= =?utf-8?B?UFlVQTMxR21kRnN5UnJKaVNuckpPdmlSaWp1alV4b2R0R1V4UDA1NnR4dVF3?= =?utf-8?B?UW1XY2VTZlFJVmRHeXMzWXZxelg4Y1JxTENOaFdPbTl3elpDQk5LSjJab1p5?= =?utf-8?B?WDAxS0kwQ2llMzBuV3c5M2dPNGVacEM2d1V4RTR3OVVRODF3L3ExSUhETUow?= =?utf-8?B?b3ZyRW1VaGxBYjRqQTNLNzdOaDlHb1BVNFhwUVZ2b1I2SXovcnBqTjRYanA1?= =?utf-8?B?TGpITlE5QXhoREtZdy9JRk1iS3Yrc25KVmF1dWg2M1lYRzE5dDlMa1QzNHMy?= =?utf-8?B?QVVhTUhxaE1YMUhMTTVRdnFBc2t3WisvM1hUeWF1WEZ3NGlaVGV2SFlLa3oy?= =?utf-8?B?ZTkzN1c0MzBhTENoNHhmMkQ3QlliczRXai9yd3JqWXZvL0psN0JBck5VRkZw?= =?utf-8?B?QjZwbWFwTlhodzhsRFdRb3RSbjVHc2hrenRqcTNsL1o1c1c5a0dpaUhZWklP?= =?utf-8?B?UVhlVzFlbTFtQ04rMmNoSXB4TWQwazNJZlVpcUMzLzlPYTdUNzFnWi94VzlC?= =?utf-8?B?YStKVndtam9yZTZBUVgrZmFOaDNTQVp1SzNKZEJKQzFrNFdXTXZ5Mkptc2ZG?= =?utf-8?B?SFp3THA0Q05pWExCcUFFVnhJNDZFaEJZeE9NdGZ2VGpJQStTRWNyN1BiSG1y?= =?utf-8?B?OTBJM2NxeEpjOWZNT1VOaXpKamJmWkVPZEVLeWNEMk5tK1NtTy9CdGlwSEtJ?= =?utf-8?B?SkdxKytoalIyVUF3TFV1SjZ3eFFtVnJmUWpaN2w1czhOQmpYNDlSMHJYSXRK?= =?utf-8?B?dXZpaVZ4QTJONCtZM1FMNTRhOUlLQkt5Z0x2UEpZRGNHNW0wWGpIZXZvYlZj?= =?utf-8?B?WWpWR0l3U3FqNlVEbkJ3NlQ4cW9rWGR1TjFIRTE2MHZ4Y2FUQ1Z1emtxNlJI?= =?utf-8?B?SGlzdGYybnBKdElpMS84MU5xTHdpd2VxVDJVeXF6eCtsMDdNVTNWZHZKclg2?= =?utf-8?B?QnFKWWgwa3hrU215dFRiRHRNdUN0UGJSWWVQeDBxNW05TVRvMy9lek1TK0E3?= =?utf-8?B?QUFCb1lqVUI5UDdweUY5Rmd1Y0ZIZU1ZRTZ5cGdmOHBCSFpsb1FwaG9MbEtN?= =?utf-8?B?S0x4WVhNaW5YUHRrZEJUb3N5RFM2MFRuNmk1aFlRZ3RmcUJUb2c0M1d5K3Vv?= =?utf-8?B?aDc5TnhCbnJHeldJQzA1UXd4R09qRCtTMVV4dW1CNTBzMGZMRjZJdz09?= X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-Network-Message-Id: 1bc56cc9-7fae-4f9c-c57c-08da21406a3c X-MS-Exchange-CrossTenant-AuthSource: DM4PR12MB5229.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 18 Apr 2022 13:36:17.2068 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: kEdjUv9Dojo4mXRyy8gC6KyhxLHgLu7syapgxqE92SiVGocyRkamsTkAq/lKMMMqrw3gG4E4rsAGkFkllHVQMg== X-MS-Exchange-Transport-CrossTenantHeadersStamped: BYAPR12MB2936 Content-Language: en-US Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit On 4/16/22 19:56, Xu, Min M wrote: > On April 16, 2022 11:09 PM, Lendacky, Thomas wrote: >> On 4/15/22 20:57, Xu, Min M wrote: >>> On April 16, 2022 4:52 AM, Lendacky, Thomas wrote: >>>> >>>> Unfortunately, this driver also breaks SEV-ES. I bypassed the TDX >>>> code in the SEC library, but then hit an issue because this driver is >>>> loaded before the AmdSevDxe driver. The AmdSevDxe driver performs a >>>> MemEncryptSevClearMmioPageEncMask() call against the >>>> PcdPciExpressBaseAddress range to mark it shared/unencrypted. >>>> However, the TdxDxe driver is loaded before the AmdSevDxe driver, and >>>> it appears the dependencies result in an MMIO being performed to an >>>> address in the PcdPciExpressBaseAddress range. Since the range has >>>> not been marked shared/unencrypted, the #VC handler terminates the >>>> guest for trying to do MMIO to an encrypted region. >>>> >>> I carefully check the code TdxDxeEntryPoint@TdxDxe.c. >>> If the working guest is NOT td guest, before it returns, it just does below: >>> 1. check if the GuidHob exists >>> 2. Set PcdOvmfHostBridgePciDevId with the information in the GuidHob >>> >>> SetMmioSharedBit() is called if the working guest is Td guest. So if it is sev >> guest, SetMmioSharedBit will not be called. >>> >>> I don't have a SEV-ES in hand. Can you help to add some debug >> information in TdxDxe to see what the last code before the exception is >> triggered? >> >> I don't think it is anything in your code, I think it is another library that is >> being loaded based on dependencies. I put a DEBUG statement at the start >> of TdxDxeEntryPoint() and never see the output before the crash. >> > I check the libraries loaded by TdxDxe and AmdSev and find that they load different PciLib. > TdxDxe load PciLib|OvmfPkg/Library/DxePciLibI440FxQ35/DxePciLibI440FxQ35.inf. > AmdSev load PciLib|MdePkg/Library/BasePciLibCf8/BasePciLibCf8.inf. > > PciLib is consumed by DxeAcpiTimerLib. In the AcpiTimerLibConstructor@DxeAcpiTimerLib there is below code: > mAcpiTimerIoAddr = (PciRead32 (Pmba) & ~PMBA_RTE) + ACPI_TIMER_OFFSET; > > I think this is the root cause of the exception. > > There are 2 options to fix this issue. > 1. Load AmdSev before TdxDxe > 2. Make TdxDxe to import BasePciLibCf8.inf instead of DxePciLibI440FxQ35.inf (just like AmdSev) > > I tried above 2 options in my Tdx guest and both work. > Tom, Can you help to try above 2 options in your SEV guest to see whether they work? > >> >>> >>> BTW, have you tried to load AmdSev.inf before TdxDxe.inf? I tried it in my >> TDX guest and it works fine. >> >> Yes, moving AmdSevDxe.inf ahead of TdxDxe.inf does fix this issue. Do you >> want to submit the patch or do you want me to? >> > If above option 2 works, I prefer this option to fix the issue. Because there is still potential issues in option 1. I will submit the patch. I added the same library class override to TdxDxe and, yes, option 2 worked. Thanks, Tom > > Thanks > Min