From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from NAM12-DM6-obe.outbound.protection.outlook.com (NAM12-DM6-obe.outbound.protection.outlook.com [40.107.243.72]) by mx.groups.io with SMTP id smtpd.web12.10241.1651151759633130644 for ; Thu, 28 Apr 2022 06:16:00 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=@amd.com header.s=selector1 header.b=qhFcQGmr; spf=permerror, err=parse error for token &{10 18 %{i}._ip.%{h}._ehlo.%{d}._spf.vali.email}: invalid domain name (domain: amd.com, ip: 40.107.243.72, mailfrom: thomas.lendacky@amd.com) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=U8NMLRa5WBUxi6DSJrn/UziVCBus5edQv8Q1CvcFSY11+RTiLLjGyHYPOj5teJ90BZQEprB46kPITK/KUnRFJ0uq1yDjXZRneclqwkn+U0U76xXrz8GEuQmI8NtRVSLbs91JtqGnQeGJGDoLIWbo8EEWc2Lv3MRAvp/NezJvViaawORK5sBpJmJij52zKR0+es3vmA/2UlIGNABY1XOzqZ6QhUgp/XFDSPnUj3J7J71jwDLTcotor8uAlFvf9kAQkf2la2XrBBZq4n5k/YfxzTj+fMw6xoY1cfdMa32eVHaB9TJ2W6HFCfXdQotlCtEvflAkJXKXdwZTfmpkDDbv7g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=im6KB4oHiECvHbmBJlAWxiJKpatqfULcpHYFAUx793I=; b=L09sDgWOtw+jE17naXHn/tpPBKngTgP4IVCiHamCoXssEdY0rUG8CTBFu5o0u013oF7f2mE66cpquzAZ1YYEQNw+WeozkELqoD10ALAXVIPVQGCHqPU+0aLyyCIjUtCVO9IW/RvV2oe6OZ7j9kqtYEjo0kmnTlSWmWyq4OFvSo1A3WvygIGzsuRzLgpcZT8Al9ik729o/L9IvtBewKOpvfZf0EJhSUPYSi+XlnwmRH7O5jsj3iViI4Jv9CXdnL2UtDdAXpIOi9Xn9BA9QEOToxiShMzb41HH3S0YTqMJspThDd/315L/DHuyE9QPjDgsq2UIAl4FJzP8mKSJwyY6jw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=amd.com; dmarc=pass action=none header.from=amd.com; dkim=pass header.d=amd.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=im6KB4oHiECvHbmBJlAWxiJKpatqfULcpHYFAUx793I=; b=qhFcQGmrXT+RYdKnTc07XT1lEU7yleNNnmD0ll/KtxyWrRFVTlZl7gIcSaZjcTwgs1UAm0EsW510m54Uw27/WohioorfytcPc5/0FMPZEE1vUJgwJBhWrm1muT/IPjqJM1PFqvP7is9sLrUHrKa1thZqq/gK6YONmsvJoQ7ri/I= Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=amd.com; Received: from BL1PR12MB5221.namprd12.prod.outlook.com (2603:10b6:208:30b::9) by SJ0PR12MB5661.namprd12.prod.outlook.com (2603:10b6:a03:422::12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5186.18; Thu, 28 Apr 2022 13:15:57 +0000 Received: from BL1PR12MB5221.namprd12.prod.outlook.com ([fe80::e4fe:e7c1:736c:d792]) by BL1PR12MB5221.namprd12.prod.outlook.com ([fe80::e4fe:e7c1:736c:d792%7]) with mapi id 15.20.5186.021; Thu, 28 Apr 2022 13:15:57 +0000 Message-ID: <8c560b9e-abd5-1e1b-e072-88efa56b7002@amd.com> Date: Thu, 28 Apr 2022 08:15:55 -0500 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101 Thunderbird/91.7.0 Subject: Re: [PATCH V12 14/47] UefiCpuPkg: Enable Tdx support in MpInitLib To: Min Xu , devel@edk2.groups.io Cc: Erdem Aktas , James Bottomley , Jiewen Yao , Eric Dong , Ray Ni , Rahul Kumar , Gerd Hoffmann References: <4bea523a89e872e3abe72eb5e800a0a245282480.1648555175.git.min.m.xu@intel.com> From: "Lendacky, Thomas" In-Reply-To: <4bea523a89e872e3abe72eb5e800a0a245282480.1648555175.git.min.m.xu@intel.com> X-ClientProxiedBy: MN2PR18CA0003.namprd18.prod.outlook.com (2603:10b6:208:23c::8) To BL1PR12MB5221.namprd12.prod.outlook.com (2603:10b6:208:30b::9) Return-Path: Thomas.Lendacky@amd.com MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 93201879-96ff-4d82-c388-08da29193b4d X-MS-TrafficTypeDiagnostic: SJ0PR12MB5661:EE_ X-Microsoft-Antispam-PRVS: X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: F0QS7/eXW3D52/eOSVAmhnH/1X3uvYwAMQs003jnTJcIp73NudyFLuAJkbZ4/YdEEX446jV5AYIDF5j5uEovUaW7XgdO+gR7vTMf6DGEjBvgmdAeC6KuEo3MQ5v7NAQIUWsf97WBBVip9uDMzoTkHcfPPMJWK8bCdOoMDULRfN7mfuDykfioajBlfWWfFRSV+WdzuChBECld+5Jpr2SJBzDDZXl7HBQJA3/up+MlCNH7lrnAe0jITYS54f2cHTrxVA6Vs66dWZkVvEfNs01ck2cTa1YzfZXpRp1ruWZ/m7QUl9W6TbIOHTiB2NvJTeHNkS+re7XJdVKcap2xOu3Jj2A2Zw+yl8wwkEQupMrBxVFs3hasoAwcuf+aJwSamWZkIh3wZHfCxwAAz9xFYX/NtG5iydFGKaStHBKOam8PjChKzFpX7k94hElz6sl8mM8zXdLwHDUFXfp1GdxtIhxHDEu/K9ESkgC2oCotCAYfjvGjiKf2E2ve200TOQiMIN5KEKC5e9kXSkPCiBIdnD6udkJtxwUPkqk/2yQfnkryCBUapr98c3MdpG62IK/50kVlKig6R+khGNPCbV+wbbTssvwrmG4w8ED7uShpoy2fHNfFT+gnsxI9TJBHdNvQCWdGNjRGyJ3W/YlI3kpzIqI7RpWotIb+Aynjwo2zh8bTYO0r963/vbloeMIGmBgxnYNUoeOBKYtbpycc4g0PQVZjb5vJx8P5DGlUEmIuyTsKk1Pz3O49cpkARg6ZiBfLLR4YlKApXV8CbOl2jZNouwuFqsq2eKEht26lMtToN5z7V0w1KrP6M1kP87Lsme/wX71ILKUA1z1VN30pX1cC2EbRNiiiS58v8LTEpjudHZwMmY8= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BL1PR12MB5221.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230001)(6029001)(4636009)(366004)(53546011)(6486002)(966005)(26005)(186003)(66556008)(66476007)(8676002)(4326008)(6506007)(316002)(54906003)(36756003)(31686004)(8936002)(31696002)(66946007)(6512007)(5660300002)(508600001)(2906002)(83380400001)(2616005)(86362001)(38100700002)(21314003)(43740500002)(45980500001);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?bk1mWnV3dDUwaHgzVndTb0xBWU5Qdi9lVTNFU2hlYjc5bGRMZThCUzlqQlcw?= =?utf-8?B?NUtrVG9VV3NDazRiU2pzTlBzRUR5VU5KZlFPdkZ0UkEyS0o5Zlo1a1k5VlpX?= =?utf-8?B?VVIvbFpES2JPWWpkUklFM2huWTQzeEIvRVhjQnFJVnpydFU3M0Q3YXZCUitK?= =?utf-8?B?WWs2by8zTXVYVGZjcmkweGpyOTR1RGVQZ2ZQaXJUa1VqL050TmJzU3Q5c3NV?= =?utf-8?B?emdiYUpQcGZKRFNPMHNOcExkVTZnNCsvQVJwb1VjTnpnR0E4L29EamhRTklJ?= =?utf-8?B?SlhmQ200SzlKeEdhTThjbFE5c3g2ajEzOWJhcy9YQm5xa09qdWVTN29vd3hW?= =?utf-8?B?bVorczd0NGZ2bWFNc0pzZmczakliNHAvUDFnWktLZVJ6N3JxMFVlTGlBczVD?= =?utf-8?B?bWRZRUlqR0M1d2NldzlDYmhqdDhjRGhWMVpwVEROUDE1c09WYXdKN2lCSnZJ?= =?utf-8?B?SFFZTVhkZ1NiUUd2eVo1Zllyb0ljMUhlaVprMVRBMjUyQnJBZzRoa005YnJn?= =?utf-8?B?RFpSK0NLK0JpVkFwWVVBS2RoMExBQVlXdnFFcG15TzNEdGpySnAxSktuREtj?= =?utf-8?B?TlU2MVdQYVROQTFQZk5XSmh4OU05WFk0Qzl1a0syUW8vdnU5aElNSzNydHR2?= =?utf-8?B?YTBsZzVOV2VZTnVCY1owdGV4Q0JtM2VHbElJSk1EaS85SVJOUnRaTEdpY1l5?= =?utf-8?B?VWJLR3VNc2lOZUdxbUxUOGFoTHVHclp5ck41UGxLSzd5ckZkTEIzQ2pXUElj?= =?utf-8?B?NVNkRWZUUnZEanRSVThOMTJwSEVCUWYveUMzdzIzb2FET0IxdnYxYjk1Qnhi?= =?utf-8?B?YkhHZGpKdlNRQTBCZFlMMzd2MjRqbm9uRThHS1pyY1cxRUR1MGNVOXVNNE8r?= =?utf-8?B?aForckoxTXJiQ2tXclFWeUpzZUtXZ01JamphMlQvRDRWNTZzNkFna1dxZW5S?= =?utf-8?B?SGZ0SE05djJoZ2RiOUNuRmVjdzBUKzl4V0swYVE4WjFlaStnc21lRkZmSXRk?= =?utf-8?B?ZVJHaGFIOVZMZFl1Rkc3ZTBzTVlJZ0puYU8wN2MvY1FxTXI1RzJCMloyazV3?= =?utf-8?B?N0RyY2tueG4xT29zU2g0amlQMUlvMTU2Tlp1R1owdlhwS204RHg4YkJlb2Iz?= =?utf-8?B?dDExaVkzN1pDTHNjaGJObEZ4TENXUTNYWVJ4NzJra0tvMkVOczJaMWloWHNT?= =?utf-8?B?UmFqODJEZUtFYkw2Nm1MN1VLOVdla0FLWE5IRXdKMzFMZElDWmN6aVdheFVp?= =?utf-8?B?b3IvOUhwWjFaeTBUZ2Q3NG8xOTA0YUN0eS93bnhsY0RoeTk3OVBuQUFvNjI5?= =?utf-8?B?UVdHek1PWVdVdUNaWHF4bWVxYVdZU0pLUnp2TUpkNW5yT1V5VGZWMlA5K3RS?= =?utf-8?B?TkJOTlF4TWYvMjd1Y3JUa1V0Y2lSYjR3RTFVditCa1RMY1JWTnlXV0lLUFRS?= =?utf-8?B?YldwSXpnS2RqQ0NOUTZ2N3BuUGsvUWM1NnZ0UFlYYUUvUVdsV1laTUVZMDl1?= =?utf-8?B?VTZPZjRzQmZEdnpqM3Z2MWozclNYa0ljc0ptZUVFK0hRVDg1cnJ6N0xCSkQ1?= =?utf-8?B?MFBSVXMvQ0Z3U09KS09hRjFuTnhMUXpDcWl1azZMeU1XRDFDVU0vMm5Ud1FJ?= =?utf-8?B?K2s0d1o2akQ4d1RMYm5vQ1hhbUtRak1CN3ljRGhlbXJqZ3I3WVlRQ21Sa1BN?= =?utf-8?B?V0ZGbzdRdVZUR0RhQXk1QWp0YytPRzhoZ1FEbTRuRUFFMU9WSUFsRk4vOUN3?= =?utf-8?B?ODFvSU1YRVVnUWdXdkNab0lhL2lhY1Rjbk01ZXZLUFZVS3B1c25NNjBnVDM0?= =?utf-8?B?SDQvdE1GTnAwYk5MVHFGNEUrNDIwSWtzU0JIbnNUTXBKOEdMMENTZ2Y0b0JN?= =?utf-8?B?VlMySWphL2NnR3lHNFU3Q2NuMXdBSkZhNlpnU1drVEc3SXJHWGRYc2JITnhw?= =?utf-8?B?VnN2a2RpYkFQcHJoWWJqOVpEc0F0ZGhNZ3VqeTNuQ01mSlg2aUtVWmdFZkRF?= =?utf-8?B?NndXMUZIV3MvY0JUNHd1T0p5dEFBRC80VlcvMnhGSGNhTW1LSktpQzc5QmxI?= =?utf-8?B?QnFlRnJaZnhUQlVJZ0l3ZkFISUIxUktOSEZyZnJCcXRKeFFIN2VtZFppSnpu?= =?utf-8?B?azAvRkRTZWIvaDE0cUJkRlkvSjNRSmI2TDg3dTJVVkhCbUVRbitydE9oYzRD?= =?utf-8?B?SlNkQTJkQmhjVm9iL1AxMDhXTzdSZmJiNjdqL3hpSytTZFBrc2tmUjJZOWln?= =?utf-8?B?dGZEekxKeFpONzNlMFlDUDlESGJtYSt0Nk1KbWF4SEY1QlEzTUM3Q3RScUZl?= =?utf-8?B?S0NlMEl3aFRHRHUrUXluUC91bGJjWENSWWlOa3FVR05xSm1mSmp6Zz09?= X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-Network-Message-Id: 93201879-96ff-4d82-c388-08da29193b4d X-MS-Exchange-CrossTenant-AuthSource: BL1PR12MB5221.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 28 Apr 2022 13:15:57.4875 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: ffEfNOiJQAlGYlzpQVQC46mOJlAFk+QURnp6VTGEx78cHhoGMzCt1zvUWmJnY9Mo0GyQ/nj8O4qvAamEw5gpgg== X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ0PR12MB5661 Content-Language: en-US Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit On 3/29/22 18:46, Min Xu wrote: > RFC: https://bugzilla.tianocore.org/show_bug.cgi?id=3429 > > In TDVF BSP and APs are simplified. BSP is the vCPU-0, while the others > are treated as APs. > > So MP intialization is rather simple. ApWorker is not supported, BSP is > always the working processor, while the APs are just in a > wait-for-precedure state. > > Cc: Brijesh Singh > Cc: Erdem Aktas > Cc: James Bottomley > Cc: Jiewen Yao > Cc: Tom Lendacky > Cc: Eric Dong > Cc: Ray Ni > Cc: Rahul Kumar > Cc: Gerd Hoffmann > Acked-by: Gerd Hoffmann > Signed-off-by: Min Xu > --- > UefiCpuPkg/Library/MpInitLib/DxeMpInitLib.inf | 3 + > UefiCpuPkg/Library/MpInitLib/MpIntelTdx.h | 69 ++++++++++++ > UefiCpuPkg/Library/MpInitLib/MpLib.c | 63 ++++++++++- > UefiCpuPkg/Library/MpInitLib/MpLibTdx.c | 106 ++++++++++++++++++ > UefiCpuPkg/Library/MpInitLib/MpLibTdxNull.c | 69 ++++++++++++ > UefiCpuPkg/Library/MpInitLib/PeiMpInitLib.inf | 3 + > 6 files changed, 308 insertions(+), 5 deletions(-) > create mode 100644 UefiCpuPkg/Library/MpInitLib/MpIntelTdx.h > create mode 100644 UefiCpuPkg/Library/MpInitLib/MpLibTdx.c > create mode 100644 UefiCpuPkg/Library/MpInitLib/MpLibTdxNull.c > > // > @@ -2367,6 +2385,11 @@ MpInitLibWhoAmI ( > return EFI_INVALID_PARAMETER; > } > > + if (CC_GUEST_IS_TDX (PcdGet64 (PcdConfidentialComputingGuestAttr))) { > + *ProcessorNumber = 0; > + return EFI_SUCCESS; > + } > + I've narrowed down this change as causing issues when booting multiple vCPU guests (regular and SEV). This issues consist of EfiAcquireLock() / EfiReleaseLock() ASSERTS and TPL level ASSERTS that occur during ExitBootServices when the APs are being parked by RelocateApLoop(). The PCD accesses use locking which is not SMP safe. I believe PCD calls are not supposed to be issued by APs because of this (or at least not APs executing in parallel). This check is spread throughout the MpLib code and I didn't look to see how many of those calls can be done by an AP. I think the PCD usage can be reduced to getting the PcdConfidentialComputingGuestAttr value at init and caching it in a STATIC variable, but I would feel better if @Min Xu could verify that and submit a patch accordingly. Otherwise, maybe this TDX guest setting could be added to the CpuMpData (similar to the SevEsIsEnabled, etc. fields). A future cleanup could cache PcdConfidentialComputingGuestAttr in CpuMpData and use CC_GUEST_IS_XXX() calls against that. Thanks, Tom > CpuMpData = GetCpuMpData (); > > return GetProcessorNumber (CpuMpData, ProcessorNumber);