From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mga18.intel.com (mga18.intel.com [134.134.136.126]) by mx.groups.io with SMTP id smtpd.web08.3065.1611378644815278326 for ; Fri, 22 Jan 2021 21:10:45 -0800 Authentication-Results: mx.groups.io; dkim=pass header.i=@intel.onmicrosoft.com header.s=selector2-intel-onmicrosoft-com header.b=mOQImKkJ; spf=pass (domain: intel.com, ip: 134.134.136.126, mailfrom: eric.dong@intel.com) IronPort-SDR: giRnJ9Xz/fcnQl4TzPJWXFKF9hvR06OeWsu7Oe6Ucm2Qs7oTzuYkG86c1gK/KoJwRG/xPcjWew EDyHfmsD6iPg== X-IronPort-AV: E=McAfee;i="6000,8403,9872"; a="167228766" X-IronPort-AV: E=Sophos;i="5.79,368,1602572400"; d="scan'208";a="167228766" Received: from fmsmga008.fm.intel.com ([10.253.24.58]) by orsmga106.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 22 Jan 2021 21:10:43 -0800 IronPort-SDR: Zr+pdktTUJwtHc9Ou9+cZ0DiorXZdirXwPg2o8rL4Id1ZjVQfdVV8asVtVExDBYV+KNXR7jK5x fxgDh2AmsBUA== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.79,368,1602572400"; d="scan'208";a="357725796" Received: from orsmsx601.amr.corp.intel.com ([10.22.229.14]) by fmsmga008.fm.intel.com with ESMTP; 22 Jan 2021 21:10:43 -0800 Received: from orsmsx608.amr.corp.intel.com (10.22.229.21) by ORSMSX601.amr.corp.intel.com (10.22.229.14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.1713.5; Fri, 22 Jan 2021 21:10:42 -0800 Received: from orsedg603.ED.cps.intel.com (10.7.248.4) by orsmsx608.amr.corp.intel.com (10.22.229.21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.1713.5 via Frontend Transport; Fri, 22 Jan 2021 21:10:42 -0800 Received: from NAM12-DM6-obe.outbound.protection.outlook.com (104.47.59.175) by edgegateway.intel.com (134.134.137.100) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.1.1713.5; Fri, 22 Jan 2021 21:10:41 -0800 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=dxqzfqty3orstloKYHHqM5F6uj/e0TDTP1j+B3IMzfZLJmBNXBT2Hb7Vg6Q5A84NAX0ZW/834IQozOU21rEiSUBHdW8EHWfvtfG2nvZ/G0dcStnEDB+0zJN4ZTosd6swe3MzkBZFDoi5FGFlk8/JUTTNVQGPtYCmx4YWNr4Qvi/SSz9rd7h2uwL1tbpgpPf9pajRTFfJxEgGvTL5AogopOCqc9wgaRb5YUMOZ7IXoIwtCavw6qoZ9F2uKcUNh7FM5/kWXjaH6LLX1XtLHNmKnn91oC5Ls6TjgqhLJpSRvk8nF662kvDMAWfDmLQ8xGnLFhZ7qAIC3nRrgfnEvF+YdA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=uUz7+RzAh61CjyR8a35Ik7K7xyN1tSwUcURO71NGY9c=; b=TABW+gJT8+7cvTzqnlZMdeMvUJQJdoRIi7fcgwcYpPw9q9j53/VzbzZFCLi2/rCmakErlrH6cbvy16KZnfqKbLI6dEdyvo+5sobFYo4IRsuM8OHlw3vYAm2qNXvKqqscfg25ZnkCOiDZo/oolI1dmeI+O6mrpDF6nMoVKyEFfKGN4ApU8nh15S0XkbpJKb5FUoubF7/KH29CSQYG8DGVNExMqA2MAgtUAG65Nyg7auop3J0k+oLAgkStvDpVT9+b5QpFwbYRChwjoDeBzMzw1mK4GrIYgScng3EeFIbl+ixKWhVQpA0mCTELi1GDVttyaOmr15kmihqeLKl3LDnE9g== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com; dkim=pass header.d=intel.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=intel.onmicrosoft.com; s=selector2-intel-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=uUz7+RzAh61CjyR8a35Ik7K7xyN1tSwUcURO71NGY9c=; b=mOQImKkJbZ3oJ3z8TbJ+phvqEJQ5XMixbt12psNybAWUIrSqKWTf7C9KHTwViBBn7QorfccQU1WosCJcOJ7S4q0BewH27RogtFRJsR/JO1R7tUmZXS/qLmRyiCBoHJV+8PsJwtRNlOXybvqemd6bA36bCVkguAaANh2RdiGMtjg= Received: from CY4PR11MB1272.namprd11.prod.outlook.com (2603:10b6:903:29::9) by CY4PR1101MB2360.namprd11.prod.outlook.com (2603:10b6:903:b9::9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3784.12; Sat, 23 Jan 2021 05:10:38 +0000 Received: from CY4PR11MB1272.namprd11.prod.outlook.com ([fe80::59b2:5d29:472d:1d36]) by CY4PR11MB1272.namprd11.prod.outlook.com ([fe80::59b2:5d29:472d:1d36%5]) with mapi id 15.20.3784.016; Sat, 23 Jan 2021 05:10:38 +0000 From: "Dong, Eric" To: "Kinney, Michael D" , "devel@edk2.groups.io" CC: "Ni, Ray" , Laszlo Ersek , "Kumar, Rahul1" Subject: Re: [Patch 1/1] UefiCpuPkg/Library/MpInitLib: Fix AP VolatileRegisters race condition Thread-Topic: [Patch 1/1] UefiCpuPkg/Library/MpInitLib: Fix AP VolatileRegisters race condition Thread-Index: AQHW8OGJhead8ruChES7htf/FH5KE6o0qk2A Date: Sat, 23 Jan 2021 05:10:38 +0000 Message-ID: References: <20210122171020.589-1-michael.d.kinney@intel.com> In-Reply-To: <20210122171020.589-1-michael.d.kinney@intel.com> Accept-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: intel.com; dkim=none (message not signed) header.d=none;intel.com; dmarc=none action=none header.from=intel.com; x-originating-ip: [192.102.204.36] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 84c640e3-3ca1-4223-729f-08d8bf5d3925 x-ms-traffictypediagnostic: CY4PR1101MB2360: x-ms-exchange-transport-forked: True x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:4303; x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: Z7gM1JzAwNnrHujNGjAWJZlVlEYrEGPZUdG2+BuHzuDporsYM5RsXlEOGRZrrtz/HuPH53IawGIHALyiG5kQkU3tAt/RLMd+1BF+SkPlwzgSm8ivPQyIEXA93+xcp7PXWtmNIS/LWyQ/luqaaRttYg9WAYfFIM6i4Tc60lD1Cs9jhihTW2ev13m/l5o484F+iEgKHDGrzW+3q1ojBKP5sbCzEJEO81tjexsfzaxAtnHnImyiNM0pQvWWF9f2WnJUX6StjzJEGdyDX4Xu1eQWQdcFOvqhNCggQYkoCbeVOuD2MIVi1d130A20vUpMUeiu+HHhBPd7nUk/jw7ZQNnDqLteQl/piPmUd89weSWsIDiKMKUEFMsU+8MIo8m16/M0jpOk40zwDgT8bPH3i7EBCX8IOQosYD70BsbuSJQetFxaYGJhjmMX4wKDNCgztQHN+gIaanrIqHGbR8f8hNoVCg== x-forefront-antispam-report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:CY4PR11MB1272.namprd11.prod.outlook.com;PTR:;CAT:NONE;SFS:(6029001)(4636009)(136003)(39860400002)(366004)(396003)(346002)(376002)(186003)(5660300002)(76116006)(8936002)(86362001)(55016002)(26005)(478600001)(53546011)(6506007)(66446008)(64756008)(66476007)(66556008)(66946007)(8676002)(52536014)(316002)(83380400001)(2906002)(71200400001)(33656002)(7696005)(107886003)(966005)(4326008)(54906003)(9686003)(110136005);DIR:OUT;SFP:1102; x-ms-exchange-antispam-messagedata: =?us-ascii?Q?P5mdf/1gVR7zkvqhnbrThmSfarlhTmRN2EYOXTBtWZgYQS1DcaNdzBACPWEo?= =?us-ascii?Q?xE3ZFB+8IdOh6eCNQZgfMC1gOLa+DCkWoR2JAlfAW8VYTJ2nPh2Tto62NlCu?= =?us-ascii?Q?RXN0ajCNUzjB2qzKCz2PX4GPCQUFvLvCnp2Owqktji9kQZkUzdpocBfB+hKG?= =?us-ascii?Q?7sOsZOee+nlwHYV6isoNggR8Wg8mJgwmmIGYYeMZJbzlvtJnGvv7P/nYqM5E?= =?us-ascii?Q?7QplKvEqS/5ZO1Xbid4ALMho4J55UwIVZifxuvcDut1p2ricYcjD/x+zJIo2?= =?us-ascii?Q?KhHj5qhaGZGlPBBAFvs/qv0qXq4yV3tYGey4xEHEkykkHWq6vS/lVKZl9dz9?= =?us-ascii?Q?0cR6OZKJQnsEKTJhBj8aBc5IRcKlG07dhK41+oP2aYNg6pTG+q75YuAKv4vs?= =?us-ascii?Q?Qor0L+YgtDj0dgqPAXqlWcW0BxkLS+EsJvuwcrds6bhRS5mKuvECh5/u6+aS?= =?us-ascii?Q?RTtez7VmlK/vLg2/5WKNIZ+6hvZqSEaSiwjoYSFQTSvaUCNRZwe4xu5Aa2H1?= =?us-ascii?Q?fNLw0Whujo+llbYgg94aXXT4oA07l52v0dA+h7OFe29j87A3KLeTaNArxKs0?= =?us-ascii?Q?9zqOlRQ3YoUf8C3Lucz1eNESfJFzL5mUX+ZYsC3t2No8o5kSiXNOLBLLN9g0?= =?us-ascii?Q?+yKEkZLien5eXCrIX8C+S9QzFowMuwcLU6/6sfcWDEcxBr4JWB8ttWOTg67f?= =?us-ascii?Q?9VsNEzxNAE/Ty382J4UWVZPF+QJLdH4nejmiCVVivCjt975W34/ZjqfBDv2F?= =?us-ascii?Q?2AZXd27Ci64AnBZWTTo4AeIOVS7smpKmBz9K4eD99XCJms0dgMdAa9WARU/9?= =?us-ascii?Q?mNHV6T2GZjWXl6DJwHoEdR1IJ17A6lb+c9uMrup7P8xdCELEkMd6hcVIj/xj?= =?us-ascii?Q?KNRCsEcejn5YEYIv81aXpPo21p4fJ7hXdKrs7pwctgExlbh2DeCEq5z+fiaR?= =?us-ascii?Q?hTqhWJY43kQxGcZWNlWCBcfzO4Eo8lobdieR2cPOefF7cNDDyj5nm18WfsgY?= =?us-ascii?Q?UVONOyO7ij2Tj/f609RlVvGzhUgYZexZnIYzqmrSPeItSkbHCdgvVWfbpRb+?= =?us-ascii?Q?LGUzGCr1?= MIME-Version: 1.0 X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: CY4PR11MB1272.namprd11.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: 84c640e3-3ca1-4223-729f-08d8bf5d3925 X-MS-Exchange-CrossTenant-originalarrivaltime: 23 Jan 2021 05:10:38.1925 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 46c98d88-e344-4ed4-8496-4ed7712e255d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: 1Bpm01+9/ShVA+L5f0qLdbdzEWw9iYqMyIywmAnsHZQ41tbHYv2NMr0Rp+ZWS7C0aXs6VoDdagIcd2023vX0XA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY4PR1101MB2360 Return-Path: eric.dong@intel.com X-OriginatorOrg: intel.com Content-Language: en-US Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Reviewed-by: Eric Dong -----Original Message----- From: Michael D Kinney =20 Sent: Saturday, January 23, 2021 1:10 AM To: devel@edk2.groups.io Cc: Dong, Eric ; Ni, Ray ; Laszlo Er= sek ; Kumar, Rahul1 Subject: [Patch 1/1] UefiCpuPkg/Library/MpInitLib: Fix AP VolatileRegisters= race condition REF: https://bugzilla.tianocore.org/show_bug.cgi?id=3D3182 Fix the order of operations in ApWakeupFunction() when PcdCpuApLoopMode is set to HLT mode that uses INIT-SIPI-SIPI to wake APs. In this mode, volatile state is restored and saved each time a INIT-SIPI-SIPI is sent to an AP to request a function to be executed on the AP. When the function is completed the volatile state of the AP is saved. However, the counters NumApsExecuting and FinishedCount are updated before the volatile state is saved. This allows for a race condition window for the BSP that is waiting on these counters to request a new INIT-SIPI-SIPI before all the APs have completely saved their volatile state. The fix is to save the AP volatile state before updating the NumApsExecuting and FinishedCount counters. Cc: Eric Dong Cc: Ray Ni Cc: Laszlo Ersek Cc: Rahul Kumar Signed-off-by: Michael D Kinney --- UefiCpuPkg/Library/MpInitLib/MpLib.c | 31 ++++++++++++++++------------ 1 file changed, 18 insertions(+), 13 deletions(-) diff --git a/UefiCpuPkg/Library/MpInitLib/MpLib.c b/UefiCpuPkg/Library/MpIn= itLib/MpLib.c index 681fa79b4cff..8b1f7f84bad6 100644 --- a/UefiCpuPkg/Library/MpInitLib/MpLib.c +++ b/UefiCpuPkg/Library/MpInitLib/MpLib.c @@ -769,15 +769,6 @@ ApWakeupFunction ( RestoreVolatileRegisters (&CpuMpData->CpuData[0].VolatileRegisters, = FALSE); InitializeApData (CpuMpData, ProcessorNumber, BistData, ApTopOfStack= ); ApStartupSignalBuffer =3D CpuMpData->CpuData[ProcessorNumber].Startu= pApSignal; - - // - // Delay decrementing the APs executing count when SEV-ES is enabled - // to allow the APs to issue an AP_RESET_HOLD before the BSP possibl= y - // performs another INIT-SIPI-SIPI sequence. - // - if (!CpuMpData->SevEsIsEnabled) { - InterlockedDecrement ((UINT32 *) &CpuMpData->MpCpuExchangeInfo->Nu= mApsExecuting); - } } else { // // Execute AP function if AP is ready @@ -866,19 +857,33 @@ ApWakeupFunction ( } } =20 + if (CpuMpData->ApLoopMode =3D=3D ApInHltLoop) { + // + // Save AP volatile registers + // + SaveVolatileRegisters (&CpuMpData->CpuData[ProcessorNumber].Volatile= Registers); + } + // // AP finished executing C code // InterlockedIncrement ((UINT32 *) &CpuMpData->FinishedCount); =20 + if (CpuMpData->InitFlag =3D=3D ApInitConfig) { + // + // Delay decrementing the APs executing count when SEV-ES is enabled + // to allow the APs to issue an AP_RESET_HOLD before the BSP possibl= y + // performs another INIT-SIPI-SIPI sequence. + // + if (!CpuMpData->SevEsIsEnabled) { + InterlockedDecrement ((UINT32 *) &CpuMpData->MpCpuExchangeInfo->Nu= mApsExecuting); + } + } + // // Place AP is specified loop mode // if (CpuMpData->ApLoopMode =3D=3D ApInHltLoop) { - // - // Save AP volatile registers - // - SaveVolatileRegisters (&CpuMpData->CpuData[ProcessorNumber].Volatile= Registers); // // Place AP in HLT-loop // --=20 2.29.2.windows.2