From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from NAM12-BN8-obe.outbound.protection.outlook.com (NAM12-BN8-obe.outbound.protection.outlook.com [40.107.237.59]) by mx.groups.io with SMTP id smtpd.web10.23500.1650557571601596872 for ; Thu, 21 Apr 2022 09:12:52 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=@amd.com header.s=selector1 header.b=f9B7CdF4; spf=permerror, err=parse error for token &{10 18 %{i}._ip.%{h}._ehlo.%{d}._spf.vali.email}: invalid domain name (domain: amd.com, ip: 40.107.237.59, mailfrom: thomas.lendacky@amd.com) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=IGxRBVSUYqTM1TXJCtLFuoaVX4+H0KH5Wj0kbSsKEZpy/5/vnYysjjj46heImVRqtzrer1nBtxfzA9lbGU6VfuTVu4hxYHWo+Z3utZHq/A7Bx7RwZhrnjmgcy5pVTheRsgOHfqPdYKTevnwcd6yCYw7ndVVR0R4YFH2Nc2eubxnObZVWFwxxD5cQD/qr5rLrsWY3IauA/x4aC38Zi02TuXEPQ+yu8iyMj2od7T+cX+TJBG9vGRq0ynaXInk/h1zI906oBOOc+lIglMyeBMirCV+fVSJmOoqmzZIV07YSe6hlVMOWN++NY2b3LLKanaNFsXCYO84fdc3ZAs/zRcxcoQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=9esfOU/ocwvNm3lMtNw648M1Wf5Q17Bm4gGnffCducQ=; b=AbEXNNv3Si/RzCLlHKeCw54g1KHPNavWnxnxEUXywhfzmvZ7WgnaQOYmn5x0qspwHJzBXfF5WcdJNnJUywpG0jFnnLpnWWeXsPTAgEXktK502vu65OBIF9IeQWKYXqXsojIEZOQXjSHG74NontZUNRXrR/2lA5/hvtEwVR00h2JK4oauDhiuZXn7eZ1pBIhBcRX+uVqO3LrsTf2JDj2L2zixeD1VXcxFCMQKf408mNohFKUcsavyx4tQP1x/nwv2LpDCdOuQA79BOzzwwaNNDeUhcdWE3exC9yxyz9zWSCwZo/qghMPiFfB7AwCDYv7i/IU2nutpz5vsEVD2x+KJ7g== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=amd.com; dmarc=pass action=none header.from=amd.com; dkim=pass header.d=amd.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=9esfOU/ocwvNm3lMtNw648M1Wf5Q17Bm4gGnffCducQ=; b=f9B7CdF4Pc0GTGUZ6AqPIQU1fGxDVZz0vz4bTkwGYm761DHzA4Tlef1cPXBKf9atZeINF9q0Q5tzW6JYai9WyMaWMVjzbnlzhpsIk/QSCLI18zG0zzyS/uI1CaYkQ7yzyomSsz69uNX/79ptx/8boE1uqOqwLkxMciVMbZxpXUA= Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=amd.com; Received: from BL1PR12MB5221.namprd12.prod.outlook.com (2603:10b6:208:30b::9) by SN1PR12MB2382.namprd12.prod.outlook.com (2603:10b6:802:2e::30) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5186.14; Thu, 21 Apr 2022 16:12:48 +0000 Received: from BL1PR12MB5221.namprd12.prod.outlook.com ([fe80::e4fe:e7c1:736c:d792]) by BL1PR12MB5221.namprd12.prod.outlook.com ([fe80::e4fe:e7c1:736c:d792%7]) with mapi id 15.20.5186.015; Thu, 21 Apr 2022 16:12:48 +0000 Message-ID: <45a27b6c-802c-ad16-24c5-9e8ce3c155b9@amd.com> Date: Thu, 21 Apr 2022 11:12:45 -0500 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101 Thunderbird/91.5.0 Subject: Re: [edk2-devel] [PATCH] OvmfPkg: Set PciLib for TdxDxe driver To: "Yao, Jiewen" , "devel@edk2.groups.io" , "Xu, Min M" Cc: Brijesh Singh , "Aktas, Erdem" , James Bottomley References: <20220419015828.899-1-min.m.xu@intel.com> <16E732CAB3014272.17418@groups.io> From: "Lendacky, Thomas" In-Reply-To: X-ClientProxiedBy: SN7PR04CA0172.namprd04.prod.outlook.com (2603:10b6:806:125::27) To BL1PR12MB5221.namprd12.prod.outlook.com (2603:10b6:208:30b::9) Return-Path: Thomas.Lendacky@amd.com MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 0596276d-9874-40c8-2a85-08da23b1c6f2 X-MS-TrafficTypeDiagnostic: SN1PR12MB2382:EE_ X-Microsoft-Antispam-PRVS: X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: rR8PESXyxqxH5KY183OTTw/vuCgJw9/pZQ54yS9WxQMfFBfeYm6R3yOeUEvl72dnav/Lh9xEb91VJ9cyOprYC1mMIPKWzr8/12AUcP1EplB0qsejM+kY2vC9f4MU4PJsBStmkCcd2EFiYoQ7unJrJpObeg4bratrCeBXWeCAPWLrm6xsDe7NBCVDYEO0Cp8/uksSNch76KUaZT0Wolv0ex2kG63zf1BJ/SOBa1jA+uLcE6s1AU5U1g+iS7aS22gChSN+1V6QX1tnwP+yWlSHWQJ/uI2RRSIkkklLcjolduaw7N8EmajDB64Rm5uEpGus9DJMbvJMQuL2D4TUGSXIOCYe3EwiPoMU09Hf7U3DENTN1Y9JkjP2behwNutyRbLO3I+sH2UPMsvcDCK3BSPrnFMO7MsU0DxKv15s+J27FsgN5pyuZ9+LmeiUJ1Ri+jr//VBeiInvNG++gAv/+t48lj4owxSzqnn3PRMWzxhQM8Z/a7rfaJL2cA7f3Y6b114E1gKyoK79rWj8KJnIFtG7/BZI6noLXn8XYxwOzUKveUFkwT2LKjK1I0DZAM9sefXk0RtDIiLG6W2tEn3JOhqcqWlW08OO4+nbieGB21ofEnqbmYQEsQdA5AFVMndMUWFZH2pAhO7ErH+ee/jetF4Zk0BZp94muer8OKmMJfjQWJJyRvKMVYtp+ASuL9+1sZ5CwdCv9vUlw5XHFUYeSusD/XL/mmSBq2PMQJJQgePVl2A0ML9DtL3idzj5/sGmVOmC/Ev7Z6ex2Osb8Xp6VIDkPG+J6DLi10feD5WWZhyP9yCjMw4+PfZv3MwT1rrnygkh385tUlKuBELB9NlXayFOM/Ps6KxzpoQ3E4t3RBb4s/A= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BL1PR12MB5221.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230001)(4636009)(366004)(6486002)(6506007)(53546011)(966005)(6512007)(8936002)(508600001)(2906002)(38100700002)(5660300002)(6666004)(66476007)(66946007)(66556008)(2616005)(26005)(186003)(83380400001)(86362001)(31696002)(19627235002)(8676002)(54906003)(316002)(4326008)(110136005)(36756003)(31686004)(45980500001)(43740500002);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?emtnZ0E5b2FLUWRQTEFVdEJQWlJTSnc5OURoVTZJWEE2ZVA0TDBuWlEwTlB5?= =?utf-8?B?UkFyeWNJYlhTOGYrQVZFMVYvVkJ1L0VaeWMvcjhGb0swNVM3ejVYRms4OXAy?= =?utf-8?B?V3laVlp4RGtqRTR4RHlDT1VZczBseStOVGtBam9BdDdBdjJnczFCaDhBa3pL?= =?utf-8?B?YjFKbXM5QVRPcmk0ZHhzN1VWWUYvSHR3MTRTOFZrU2pwR1paa0RqaFJnNXdC?= =?utf-8?B?OGxXUTM4ejBCL3kxck5GQm1TTmZ2cm5KVEdJUmpJSFl3ZVZlQXNzUmNjTHdn?= =?utf-8?B?MTVZRWI3aVlCL2s2WWErY1BmZEpZVk1XT2Z6cVpEV2Z2Qm9RUFh6SGFKVmJm?= =?utf-8?B?UzJaOTdKR3duWWswbksvcm5VRFpER1N0TVJuYmNMZm8wUXQwWnlsT0hWa3Bu?= =?utf-8?B?ekxsR1NxU0xQSFU5ZTBXempQeERqZVZVSHQ5YkFBWUpLTGp1MW9PRitVeFBG?= =?utf-8?B?V0YyZTZlaDBoVHBBM3ZzYUF6aFI2MFZQN2N3dEp4QWowdnk1a2RpbDloQ1J4?= =?utf-8?B?T0F5bTZQNUg2bmF5QjJIQ1dCSkZvZ2FiUzVydGVRZ3U0VGxBTHFWNEdJREl2?= =?utf-8?B?S2t6bGwwWXN6T3VTMFZEaUFVSWdTaHo4TG1QUytJdGtPV0MrUzZQd01SZzJl?= =?utf-8?B?bVZFZEpUNXpsVUZ3NWJFcE1pSDhiZTZxbWRWWDRkdW1WYXVlR2pvTTlvTzhr?= =?utf-8?B?NWlDQ1hZUFlObngrUG50MzM5bWlSYWdoOWp3UkVUdU96c0ZBQkhQUTg3WURF?= =?utf-8?B?RHJ5eHQyVnpMbUNLU1M5ajZlb09RQnJ4ZFcyaDlINFgxV0EzZ2RJVXZlVVlu?= =?utf-8?B?TFJVVUoyVys0b0wvUERCQzZFZ2Mzc2Rjd2xQcld4ZDZkY09ySVFKZUlEZWdE?= =?utf-8?B?WDJOcUxEaHdlRlVPS0x0ZG1pSU9RUE9HeklBZW9LVE5VdWZOWG9ESmN2anlu?= =?utf-8?B?QXpTdmtpZm5wSThuUHBHODdvdndYZE42azVmWEd1QUVGV3dLQmRLOVFkb3I5?= =?utf-8?B?aUt3SXpHN0xTV3NaTEovbitiRFlVVHFoZDdic0dmdWxRQXhleGg0UlNucVA1?= =?utf-8?B?aTBOTjQrdkRyM3V5WmY0OG8zL2FIRmdXTXJoNnJDeFVpMXpncmUxMm9KQUtD?= =?utf-8?B?MzdjY0Q0Yy8zTTk1a3NhdnBjTGFNcm80bXZlR0hIUjB0eVhva1JYU1Q0bXRv?= =?utf-8?B?TURCdzdVd3JRakJNa3VKRjlSdUhGTVNuc2NKNmRkUDlrQlhnaUc5c3pXSjVK?= =?utf-8?B?NlM4MnZ3VjA4c3hNM2hYWGg2d3lvOCtkcmwrUklhNGxCMW1MT2lmMmE0b1R3?= =?utf-8?B?a3FDNmJsc0pjY0ZpVC9WUlM0bEpLa09yaGZkWHdxOVJlTWlQeFRURXZhdHIw?= =?utf-8?B?ZGlucERiMVlsNmhTRE15N09oRjEwVVpienM3ZjNIU04rYngyL2FYcG1jRmtu?= =?utf-8?B?d0tRLzNlUU9EKzFxWUNVVmRhbFBWZnJmQ0MzR2NwV3pRMk9ROUdrK3c5elE5?= =?utf-8?B?Sm1pUjRBVXVZcytXYVMyZEhaRkRaR2JqS09wa1lwRG9QR0JxUG9PK21BV0Vt?= =?utf-8?B?RWV4R1ovUS9qSVlqSmxkTGxWd0NmSlNLZ0hpNlRPZzZuL1ZDYm8zeFhUaE93?= =?utf-8?B?OVFGU2NjT2FhOWcvZDJyR3p6NWVTM0Y5eWtHTTl6MWJYZERsUm1waVRlRmxa?= =?utf-8?B?dko1KzdIY2RVMEcvSFBQRGtjRWx2bVJQMi8vQ1Jxa3ZpdG5RV0FYRGh4d3ZM?= =?utf-8?B?bjFKYVl6Mkh2djB0K01jbEgyT2g3Vm00dzFuOEYza1BHbmRzaXo1cVpVQnEw?= =?utf-8?B?Uk1iMFc3c0FSY3g0NzlCMzJHb2x3Y1lEUlh1V1RqbUtYZEFBRzlMdW9qSFRQ?= =?utf-8?B?YjFzMFUvN2R5aXYvQ2tENmlYSjliTzNvaEs1ZVFMOTk1UTV2NFZZWnphQ3hx?= =?utf-8?B?bU5FS2o0citMbTh0Vk9tR0ZhUXZEdkpwYkNtQytZTzkwcE1HckdMMXVycmVp?= =?utf-8?B?c3kzcmdTRmRSWnF6bW5kYmN0clZxSTFCbUVwdlhrSkpHdS9nd1hRcmQ4dm1v?= =?utf-8?B?Z3pQaVFIZXVueXJ1VWpOUWdkUXlhKzZ2MlZQV0wzamdqZFYyUjltZlNNd2M4?= =?utf-8?B?ZzM5OU16QmdPSXlaRVduU0xRL0VRQnhmU0VxdXZOaThiZ1hTZFhCSTFnbm9t?= =?utf-8?B?REZVSEo3ZHNNeGEvN09mMW9WT1NxSG4zM2lhb1kwN3V5QkJ6U1V3Zkh2SWpC?= =?utf-8?B?OHB6VHJNVi92bURhVjF1Rk5WMlVKUm83K2dodDUwcGdpd3U3RUV1V0R3MnVk?= =?utf-8?B?WFM1NXdTOCt0dG5DRi9GUWQ1ZjFCRjJUdEpRNUZxL29XcGd3OFNEUT09?= X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-Network-Message-Id: 0596276d-9874-40c8-2a85-08da23b1c6f2 X-MS-Exchange-CrossTenant-AuthSource: BL1PR12MB5221.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 21 Apr 2022 16:12:48.2164 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: aauSv3WMG/6jDZ8gtq0IjP/ybSiFwzcXrwV7tZYX6o5U1rEzYmxqew5pacq58lNh2hYu9xKFqbsxZkcYkzsGmg== X-MS-Exchange-Transport-CrossTenantHeadersStamped: SN1PR12MB2382 Content-Language: en-US Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit On 4/19/22 00:06, Yao, Jiewen wrote: > OK. Let me describe what I think. > > PCI Express BAR need to be initialized by someone in the platform. > This initialization may require CFG8. That is understandable. > > A good design is that: After the PCIE BAR is initialized, it can be accessed. > Requires additional step (such as clear C-bit) means the PCIE BAR is not fully initialized originally. I don't think it is a good idea. > > So far, the problem is TdxDxe, but what if a PEI driver also wants to use access PCIE space? It may run into same problem. > > I think the best way is to clear C-bit in PciExBarInitialization(), as SEV specific step to finish initialization. https://github.com/tianocore/edk2/blob/master/OvmfPkg/Library/PlatformInitLib/Platform.c#L261 > > As such, no matter how many drivers want to use PCIE, they can. > > > Splitting PCIE bar programming and C bit clearing is a big problem. In this window, no one can actually touch the PCIE bar, although it seems being initialized... I tried this approach and it does not work. It is because new page tables are used in the DXE phase and so the c-bit has to be cleared in the new page tables vs the page tables used in PEI. Thanks, Tom > > > Thank you > Yao Jiewen > >> -----Original Message----- >> From: devel@edk2.groups.io On Behalf Of Yao, Jiewen >> Sent: Tuesday, April 19, 2022 12:47 PM >> To: Xu, Min M ; devel@edk2.groups.io >> Cc: Brijesh Singh ; Aktas, Erdem >> ; James Bottomley ; Tom >> Lendacky >> Subject: Re: [edk2-devel] [PATCH] OvmfPkg: Set PciLib for TdxDxe driver >> >> Can SEV clear the C-bit in SEC phase? >> >> I think that is right way to ensure PCI Express can always be accessed by anyone. >> >> >>> -----Original Message----- >>> From: Xu, Min M >>> Sent: Tuesday, April 19, 2022 12:39 PM >>> To: Yao, Jiewen ; devel@edk2.groups.io >>> Cc: Brijesh Singh ; Aktas, Erdem >>> ; James Bottomley ; Tom >>> Lendacky >>> Subject: RE: [edk2-devel] [PATCH] OvmfPkg: Set PciLib for TdxDxe driver >>> >>> In AmdSevDxe's entry point it clears the C-bit from PcdPciExpressBaseAddress >>> and other memory spaces if needed. Please see >>> >> https://github.com/tianocore/edk2/blob/master/OvmfPkg/AmdSevDxe/AmdSev >>> Dxe.c#L81-L95. After that OVMF can use PCI express. >>> >>> This broken is caused by the call sequence of TdxDxe driver and AmdSevDxe >>> driver. Currently TdxDxe driver is loaded before AmdSevDxe, so in SEV-ES guest >>> the C-bit of PcdPciExpressBaseAddress hasn't been cleared. In this situation >> the >>> access to PciExpressBaseAddress trigger exceptions (lib constructor in TdxDxe). >>> >>> There are 2 options to fix this issue. >>> 1. Adjust the load sequence of AmdSevDxe and TdxDxe (Load AmdSevDxe >> before >>> TdxDxe) >>> 2. Make TdxDxe to import BasePciLibCf8.inf instead of DxePciLibI440FxQ35.inf >>> (just like AmdSevDxe) >>> >>> Tom and I tested above 2 options in SEV and TDX and all work. >>> >>>> -----Original Message----- >>>> From: Yao, Jiewen >>>> Sent: Tuesday, April 19, 2022 12:16 PM >>>> To: Xu, Min M ; devel@edk2.groups.io >>>> Cc: Brijesh Singh ; Aktas, Erdem >>>> ; James Bottomley ; Tom >>>> Lendacky >>>> Subject: RE: [edk2-devel] [PATCH] OvmfPkg: Set PciLib for TdxDxe driver >>>> >>>> Do you mean, with SEV introduced, OVMF cannot use PCI express any more? >>>> >>>> Thank you >>>> Yao Jiewen >>>> >>>> >>>>> -----Original Message----- >>>>> From: Xu, Min M >>>>> Sent: Tuesday, April 19, 2022 11:05 AM >>>>> To: Yao, Jiewen ; devel@edk2.groups.io >>>>> Cc: Brijesh Singh ; Aktas, Erdem >>>>> ; James Bottomley ; >> Tom >>>>> Lendacky >>>>> Subject: RE: [edk2-devel] [PATCH] OvmfPkg: Set PciLib for TdxDxe >>>>> driver >>>>> >>>>> On April 19, 2022 10:54 AM, Yao Jiewen wrote: >>>>>> >>>>>> Why does TdxDxe call TdxMailbox in an SEV platform? >>>>>> Or why does TdxMailbox call SynchronizationLib in an SEV platform? >>>>>> >>>>> TdxDxe will not call TdxMailbox/SynchronizationLib in SEV platform. >>>>> The problem is in the lib constructor. When TdxDxe driver is loaded, >>>>> before its entry point is called, the lib constructors will be called even in a >>>> SEV platform. >>>>>> >>>>>> There are many places we can do CcProbe to stop action. Why we need >>>>>> do it in DSC? >>>>> So we cannot stop the lib constructor with CcProbe in this case. >>>>> >>>>> Thanks >>>>> Min >> >> >> >> >