From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from aserp2130.oracle.com (aserp2130.oracle.com [141.146.126.79]) by mx.groups.io with SMTP id smtpd.web10.5595.1612332799800824940 for ; Tue, 02 Feb 2021 22:13:20 -0800 Authentication-Results: mx.groups.io; dkim=pass header.i=@oracle.com header.s=corp-2020-01-29 header.b=IIgzTBKG; spf=pass (domain: oracle.com, ip: 141.146.126.79, mailfrom: ankur.a.arora@oracle.com) Received: from pps.filterd (aserp2130.oracle.com [127.0.0.1]) by aserp2130.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 11369grA071551; Wed, 3 Feb 2021 06:13:13 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=subject : to : cc : references : from : message-id : date : in-reply-to : content-type : content-transfer-encoding : mime-version; s=corp-2020-01-29; bh=YvXqnYr3B1ZYNuQnoeku5EGgRb8asEe6KZWxApCMfVU=; b=IIgzTBKGyUc8wUkpwNpHogwjfsraKZe5+xnWw8F6olIK+LCm9SV68s86HcaPIMOUjO3f ERREd/efcoR8/ELH/9pLnLbD62eK2RpuOPPdeebZwV8k8bWfzbxLIPsWiLOp/Zqt8DeF FVrK/vGnFco9ez6GTvgovtdy+qkgqpzk7vnHaSdN24ryCpw3fVwuYYkoNHRmee5/h1wt A9QMETe8M0WylYo733ch6ItGzunfb8KO0MO3RsI0wtm9tKoYr8j30B8F8vQp890ZbgtJ cfUBBsOCP7nYrHr1cCAgS85oGDcF520HfztoNqxuZyeF/2HdGkk2ziqjP0qRoWCwUQmh aA== Received: from userp3030.oracle.com (userp3030.oracle.com [156.151.31.80]) by aserp2130.oracle.com with ESMTP id 36cvyaxfcu-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 03 Feb 2021 06:13:13 +0000 Received: from pps.filterd (userp3030.oracle.com [127.0.0.1]) by userp3030.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 11366bmb014282; Wed, 3 Feb 2021 06:13:12 GMT Received: from nam11-dm6-obe.outbound.protection.outlook.com (mail-dm6nam11lp2169.outbound.protection.outlook.com [104.47.57.169]) by userp3030.oracle.com with ESMTP id 36dhcy0cev-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 03 Feb 2021 06:13:12 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=kdy/qRBpqEh4ANQlx4mPC+jce6uyQIofPcBmrhaOk3Ltk0S4DqA1EtZdsFo0bTihnWJFs7CRjMNakzq16X/0aUt7jizjIJxB/tJoiD1+CgENXjgE2BY4nOcPSXdAVhnsmpmH70UM2P1XUQ+P87LLPBXHTFSBB7FQaMxH5aS/IJhbimGrZJEGig43lr/+kojXX9cCbbDvPLDI/r6N6cz/5H5yYzKp9qufUXjrpjLSeDqhRWtY1IpkrMsUXwwiVlWOd82LoSO/xQAGlVx99atPeuzvfWs7h3KTbQ4R2BAnqz8KOETEFuwB2leBsj3Ovn80pdv1MMajEwV2H63/xryDLw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=YvXqnYr3B1ZYNuQnoeku5EGgRb8asEe6KZWxApCMfVU=; b=jpdAqxIsRz73OEAO6R8lSLnWEYu4pF8wVljPnQlvcipb8SOK9f76GcZpFqIM5hrbkpJgkxyb8l8P4Vnj6ajg1PWS0rsmaUvHzC5Iz+lEhhAM7IHSzUztQ+CUH+gIxP5g1Tu5oJOfMQXIIc/bjlAfMJxn6m1oO2Dgt/X3TmHKevz3zm6bukDfZeVxcbsgWmPNb3jRWSR0/Yk5UIDbOV+9wLQ9/2SI5PGlqsrRc0racyVdwAcPT16t38QapM5fTNyiuVAPyGXQe4gQq3je6I1e/Peo1TU1g37FLxGWCz3rg5Qqh1ees2N12sMb8CDJQawKiiXPpCgaVutLFFS2gkXy0A== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=YvXqnYr3B1ZYNuQnoeku5EGgRb8asEe6KZWxApCMfVU=; b=jYJwNPXiSYKjHELgnR+gAqENveaLgxvRNdnNU6w43IsH+gC5voa1ZKAr5+PDT/k1YkfuG7E0BJKebfGsJl3CpX5InppcBdIh7crX0C0Zbh2f521IxlOOqIXfCcjtEhH7S2+ExCE7zhMrk7sbYre0e8uNSI8ukSJYb6C5glBMYmY= Received: from SJ0PR10MB4605.namprd10.prod.outlook.com (2603:10b6:a03:2d9::24) by SJ0PR10MB4656.namprd10.prod.outlook.com (2603:10b6:a03:2d1::20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3805.16; Wed, 3 Feb 2021 06:13:10 +0000 Received: from SJ0PR10MB4605.namprd10.prod.outlook.com ([fe80::b1e9:811d:aa8e:f62a]) by SJ0PR10MB4605.namprd10.prod.outlook.com ([fe80::b1e9:811d:aa8e:f62a%6]) with mapi id 15.20.3805.028; Wed, 3 Feb 2021 06:13:10 +0000 Subject: Re: [PATCH v6 7/9] OvmfPkg/CpuHotplugSmm: add CpuEject() To: Laszlo Ersek , devel@edk2.groups.io Cc: imammedo@redhat.com, boris.ostrovsky@oracle.com, Jordan Justen , Ard Biesheuvel , Aaron Young References: <20210129005950.467638-1-ankur.a.arora@oracle.com> <20210129005950.467638-8-ankur.a.arora@oracle.com> <14068eb6-dc43-c244-5985-709d685fc750@oracle.com> From: "Ankur Arora" Message-ID: <3fc5ff97-6ea9-e943-523f-9a7462072c59@oracle.com> Date: Tue, 2 Feb 2021 22:13:05 -0800 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Thunderbird/78.6.0 In-Reply-To: X-Originating-IP: [148.87.23.11] X-ClientProxiedBy: MWHPR15CA0059.namprd15.prod.outlook.com (2603:10b6:301:4c::21) To SJ0PR10MB4605.namprd10.prod.outlook.com (2603:10b6:a03:2d9::24) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from [192.168.0.108] (148.87.23.11) by MWHPR15CA0059.namprd15.prod.outlook.com (2603:10b6:301:4c::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3805.17 via Frontend Transport; Wed, 3 Feb 2021 06:13:08 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: f78c1956-7c8c-40ef-ea50-08d8c80ac805 X-MS-TrafficTypeDiagnostic: SJ0PR10MB4656: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:9508; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: s/qIwawa8EvbBYWGUi4CVY7Vy/b1nj3hfGGactkY3P2F34TqmTY4Fee83h2npv40sw6Q8D27EF1Ij9REbqccx9crQd6YBZuGE0tOaReFohODPwLlQmPjqxEmejH+A2qw7mnw3I1weul0J55FiRLr+DAHsSJhg40AmsuKbx/x4jxv/HWlxGu/0Tg0m87tsiaU93MSmgrFdkqMLE5aRlKba0SXKUcbDdJY216eXc30VRAWVAstkcRJzE5H/ZRYObb5RdmhzxzRZkFnsqf4GMhaG5LqtP/cI8ALePCh0SNREBQtrrMewtNRtH1wB6AwA/TSoA2QRLh3TtDOFungj4MpFL8WZUXfH7TW3W3M9AJXLGn4IpCGVFBnTcmLIkrW/cCvcSEwCrrYSJc3CHxT4TYMTtFxX9lIJS49mKXbECJ+35baFrB6/ILOhW26ORfiCBpt0oYLjyahZSHk1iyksX8IRSFCvAJBOJREmHnOLQJMxBrUSI7YbRAZT7hxgKHn3B4LYJt/p6CK/JUENoKuV6DqtLpbKr7GXReqrfOZDHh1bUOV6mlmLXTs2lDdv/VMYS7ugvc1VWIlLKjMsEh3zqxppfeNnStxTQ5JaEV3+peBmJM= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:SJ0PR10MB4605.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(39860400002)(396003)(136003)(346002)(366004)(376002)(8936002)(186003)(16526019)(26005)(31686004)(6666004)(316002)(54906003)(86362001)(478600001)(31696002)(66946007)(66476007)(53546011)(2616005)(4326008)(107886003)(2906002)(5660300002)(16576012)(6486002)(83380400001)(956004)(36756003)(66556008)(8676002)(45980500001)(43740500002);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData: =?utf-8?B?MGxSMnFxQituR2tvL0pla2EyT2liaWVTcEhweDJtWVFqa0Rybk1Mc3dqS3hU?= =?utf-8?B?Z3o0VnBtU0RhNmltRG1pL1o1K3FldGxNbmdtS2xXUlRMZDU1MW05bG5RYW9T?= =?utf-8?B?aUdjZFF2NCtYTWxvZG0zaFgzZkl1ZDJ5bjlQTExKeHV3RXBaV2tUdFhPdWNi?= =?utf-8?B?R0NKWE9GVWJpVlFXalpUME5NZlplM3hzRUJPZ1hCSEpLbWtiRy9OWXZzUk82?= =?utf-8?B?T1c1TVBYMlZoM3JGck1ZYi9LdUV6YitaeXgzRHJqYWNoejQ4V3hpTkYyNDZC?= =?utf-8?B?MEZlN2w5RDIwNHJ6T3p3Vjh4TXRUN2pMUWhZVTVRd0xJbXFrek9RZFNkVWU5?= =?utf-8?B?czNlRUtqRFdoYlVXSm90WTVPWWt1R09DZW1QMExkUVIxbDZIYVRwZDRpUEtw?= =?utf-8?B?bFk3ZndFWFFkY2lIOHhkR1pmd3RIRmkvOGwzQUEyZE5hQXdzejRvL3dBQWpC?= =?utf-8?B?WmtPYy9BMVhsNDVmTmlqeE1wUGJyQTdXYUtnSGQ0NUtNTmJQWTRua0txT2py?= =?utf-8?B?THhuUStqOEh1TTNTYWltb1NtV2pPUnZydW5EeVY5ZThKZjNrOEpqUVIzV0dC?= =?utf-8?B?eXIybXNwc2FFblhEMkcwRUNLcHY3NGlBclNFVitKVzZkU2JEbGpTTUE0N0xB?= =?utf-8?B?UUlKVGxNbEp1cEc1U1lrb1BOV2hyWEhWMDlpai9ZNG4xS21CcTFHRjk4SStv?= =?utf-8?B?SlJ3MnRnRXozZjRYeWh5di91WGpNZytmZGNiWkdkczhNakxlazE4SVBpYXFy?= =?utf-8?B?QU1rUmtraS9aV3ZrT0lJbnhYdlJYckRzQjFPa0VaUUNuK0lRVWE0NjVOb2tU?= =?utf-8?B?Nnh4WmpzZW9iN00rZ1VWZzBJMzZDYXQwWWdKMjYyTlhubnlER1JUQ091NUsw?= =?utf-8?B?L3NibXo5MEl6VVRkVGxmNW51SkdjU0FYSnczMyt1SWRpNmhJNUpqMkIweitC?= =?utf-8?B?bkNST0VWTnpQUWdyWW5qUFRPTHJUZk85a2Uyc0lJaFhNRnl4bTJlRlRCM1hO?= =?utf-8?B?WWJaVGxPSzIydHRUUXhQUHJRakRTLzZBbjlZS2owT3dqOHhRMnVweVBIc2FN?= =?utf-8?B?U1R6V0MrOFhHQ1FOTGRGcUp6Q0h3K3kxQnhMdWF2NllicHBQbGFTL3pzWkRW?= =?utf-8?B?WHJ6SWV3L3BNOEtNZEdjaE1aaERlSWtQaVdzbEQ0d0p1cnZRMHB0czlCbUp1?= =?utf-8?B?clJvY1kvSVJRY0VWRW1sQXRyL1lkNUdVdnQ0VEI0RWhNMXdHZmNQV0VXQ0Ja?= =?utf-8?B?eWU2T0VmODVtaFZVRFNQOFo1WWJLdlhYLzVzTS94R2RNeHk0TjI1WTNIM1Vw?= =?utf-8?B?QlMrVDkwUmwwelR0NW1ZYngrcHVuNUNYZFoyVDBIR1pxODhsd25nc2pHU0VL?= =?utf-8?B?bFd0N21NaFVNSW5xRW1leDV4U0dERmFtRThoQ3gzdlRWcS80NEZUVUN3N2Mv?= =?utf-8?Q?DWXUl51K?= X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: f78c1956-7c8c-40ef-ea50-08d8c80ac805 X-MS-Exchange-CrossTenant-AuthSource: SJ0PR10MB4605.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 03 Feb 2021 06:13:10.5836 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: itnbCFBvEdFD30udkdlEs5m6dVDJUClUlUgR4M7Ua0lSYlHizU/uIOjr789ZriEK3Q+xqH5tzJhflGS0MqH6LqJD9EaYCaT1tSnlyLh46Y8= X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ0PR10MB4656 X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=9883 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxscore=0 mlxlogscore=999 phishscore=0 spamscore=0 suspectscore=0 malwarescore=0 adultscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2009150000 definitions=main-2102030036 X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=9883 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 clxscore=1015 impostorscore=0 mlxscore=0 spamscore=0 bulkscore=0 priorityscore=1501 adultscore=0 lowpriorityscore=0 malwarescore=0 phishscore=0 mlxlogscore=999 suspectscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2009150000 definitions=main-2102030036 Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit On 2021-02-02 6:00 a.m., Laszlo Ersek wrote: > On 02/01/21 21:12, Ankur Arora wrote: >> On 2021-02-01 11:08 a.m., Laszlo Ersek wrote: >>> apologies, I've got more comments here: >>> >>> On 01/29/21 01:59, Ankur Arora wrote: >>> >>>> /** >>>> + CPU Hot-eject handler, called from SmmCpuFeaturesRendezvousExit(), >>>> + on each CPU at exit from SMM. >>>> + >>>> + If, the executing CPU is not being ejected, nothing to be done. >>>> + If, the executing CPU is being ejected, wait in a CpuDeadLoop() >>>> + until ejected. >>>> + >>>> + @param[in] ProcessorNum Index of executing CPU. >>>> + >>>> +**/ >>>> +VOID >>>> +EFIAPI >>>> +CpuEject ( >>>> + IN UINTN ProcessorNum >>>> + ) >>>> +{ >>>> + // >>>> + // APIC ID is UINT32, but mCpuHotEjectData->ApicIdMap[] is UINT64 >>>> + // so use UINT64 throughout. >>>> + // >>>> + UINT64 ApicId; >>>> + >>>> + ApicId = mCpuHotEjectData->ApicIdMap[ProcessorNum]; >>>> + if (ApicId == CPU_EJECT_INVALID) { >>>> + return; >>>> + } >>>> + >>>> + // >>>> + // CPU(s) being unplugged get here from >>>> SmmCpuFeaturesSmiRendezvousExit() >>>> + // after having been cleared to exit the SMI by the monarch and >>>> thus have >>>> + // no SMM processing remaining. >>>> + // >>>> + // Given that we cannot allow them to escape to the guest, we pen >>>> them >>>> + // here until the SMM monarch tells the HW to unplug them. >>>> + // >>>> + CpuDeadLoop (); >>>> +} >>> >>> (15) There is no such function as SmmCpuFeaturesSmiRendezvousExit() -- >>> it's SmmCpuFeaturesRendezvousExit(). >>> >>> (16) This function uses a data structure for communication between BSP >>> and APs -- mCpuHotEjectData->ApicIdMap is modified in UnplugCpus() on >>> the BSP, and checked above by the APs (too). >>> >>> What guarantees the visibility of mCpuHotEjectData->ApicIdMap? >> >> I was banking on SmiRendezvous() explicitly signalling that all >> processing on the BSP was done before any AP will look at >> mCpuHotEjectData in SmmCpuFeaturesRendezvousExit(). >> >> 1716 // >> 1717 // Wait for BSP's signal to exit SMI >> 1718 // >> 1719 while (*mSmmMpSyncData->AllCpusInSync) { >> 1720 CpuPause (); >> 1721 } >> 1722 } >> 1723 >> 1724 Exit: >> 1725 SmmCpuFeaturesRendezvousExit (CpuIndex); > > Right; it's a general pattern in edk2: volatile UINT8 (aka BOOLEAN) > objects are considered atomic. (See > SMM_DISPATCHER_MP_SYNC_DATA.AllCpusInSync -- it's a pointer to a > volatile BOOLEAN.) > > But our UINT64 values are neither volatile nor UINT8, and I got suddenly > doubtful about "AllCpusInSync" working as a multiprocessor barrier. > > (I could be unjustifiedly worried, as a bunch of other fields in > SMM_DISPATCHER_MP_SYNC_DATA are volatile, wider than UINT8, and *not* > accessed with InterlockedCompareExchageXx().) Thanks for pointing me to this code. There's a curious comment in about making this structure uncache-able in the declaration here (though I couldn't figure out how that is done): 418 typedef struct { 419 // 420 // Pointer to an array. The array should be located immediately after this structure 421 // so that UC cache-ability can be set together. 422 // 423 SMM_CPU_DATA_BLOCK *CpuData; 424 volatile UINT32 *Counter; 425 volatile UINT32 BspIndex; 426 volatile BOOLEAN *InsideSmm; 427 volatile BOOLEAN *AllCpusInSync; 428 volatile SMM_CPU_SYNC_MODE EffectiveSyncMode; 429 volatile BOOLEAN SwitchBsp; 430 volatile BOOLEAN *CandidateBsp; 431 EFI_AP_PROCEDURE StartupProcedure; 432 VOID *StartupProcArgs; 433 } SMM_DISPATCHER_MP_SYNC_DATA; Also, is there an expectation that these fields (at least some of them) switch over when a new leader is chosen? Otherwise I'm not sure why for instance, AllCpusInSync would be a pointer. > > >> >>> >>> I think we might want to use InterlockedCompareExchange64() in both >>> EjectCpu() and UnplugCpus() (and make "ApicIdMap" volatile, in >>> addition). InterlockedCompareExchange64() can be used just for >>> comparison as well, by passing ExchangeValue=CompareValue. >> >> >> Speaking specifically about the ApicIdMap, I'm not sure I fully >> agree (assuming my comment just above is correct.) >> >> >> The only AP (reader) ApicIdMap deref is here: >> >> CpuEject(): >> 218 ApicId = mCpuHotEjectData->ApicIdMap[ProcessorNum]; >> >> For the to-be-ejected-AP, this value can only move from >> valid-APIC-ID (=> wait in CpuDeadLoop()) -> CPU_EJECT_INVALID. >> >> Given that, by the time the worker does the write on line 254, this >> AP is guaranteed to be dead already, I don't think there's any >> scenario where the to-be-ejected-AP can see anything other than >> a valid-APIC-ID. > > The scenario I had in mind was different: what guarantees that the > effect of > > 375 mCpuHotEjectData->ApicIdMap[ProcessorNum] = (UINT64)RemoveApicId; > > which is performed by the BSP in UnplugCpus(), is visible by the AP on > line 218 (see your quote above)? > > What if the AP gets to line 218 before the BSP's write on line 375 > *propagates* sufficiently? I understand. That does make sense. And, as you said elsewhere, a real memory fence would come in useful here. We could use AsmCpuid() as a poor man's mfence, but that seems overkill given that x86 at least guarantees store-order. Ankur > > There's no question that the BSP writes before the AP reads, but I'm > uncertain if that suffices for the *effect* of the write to be visible > to the AP. My concern is not whether the AP sees a partial vs. a settled > update; my concern is if the AP could see an entirely *stale* value. > > The consequence of that problem would be that an AP that the BSP were > about to eject would return from CpuEject() to > SmmCpuFeaturesRendezvousExit() to SmiRendezvous(). > > ... I guess that volatile-qualifying both CPU_HOT_EJECT_DATA, and the > array pointed-to by CPU_HOT_EJECT_DATA.ApicIdMap, should suffice. In > combination with the sync-up point that you quoted. This seems to match > existing practice in PiSmmCpuDxeSmm -- there are no concurrent accesses, > so atomicity is not a concern, and serializing the instruction streams > coarsely, with the sync-up, in combination with volatile accesses, > should presumably guarantee visibility (on x86 anyway). > > Thanks > Laszlo > > >> >> 241 QemuCpuhpWriteCpuSelector (mMmCpuIo, (APIC_ID) RemoveApicId); >> 242 QemuCpuhpWriteCpuStatus (mMmCpuIo, QEMU_CPUHP_STAT_EJECTED); >> 243 >> 244 // >> 245 // Compiler barrier to ensure the next store isn't reordered >> 246 // >> 247 MemoryFence (); >> 248 >> 249 // >> 250 // Clear the eject status for CpuIndex to ensure that an >> invalid >> 251 // SMI later does not end up trying to eject it or a newly >> 252 // hotplugged CpuIndex does not go into the dead loop. >> 253 // >> 254 mCpuHotEjectData->ApicIdMap[CpuIndex] = CPU_EJECT_INVALID; >> For APs that are not being ejected, they will always see >> CPU_EJECT_INVALID >> since the writer never changes that. >> >> The one scenario in which bad things could happen is if entries in the >> ApicIdMap are unaligned (or if the compiler or cpu-arch tears aligned >> writes). >> >>> >>> (17) I think a similar observation applies to the "Handler" field too, >>> as APs call it, while the BSP keeps flipping it between NULL and a real >>> function address. We might have to turn that field into an >> From a real function address, to NULL is the problem part right? >> >> (Same argument as above for the transition in UnplugCpus() from >> NULL -> function-address.) >> >> >>> EFI_PHYSICAL_ADDRESS (just a fancy name for UINT64), and use >>> InterlockedCompareExchange64() again. >> >> AFAICS, these are the problematic derefs: >> >> SmmCpuFeaturesRendezvousExit(): >> >> 450 if (mCpuHotEjectData == NULL || >> 451 mCpuHotEjectData->Handler == NULL) { >> 452 return; >> >> and problematic assignments: >> >> 266 // >> 267 // We are done until the next hot-unplug; clear the handler. >> 268 // >> 269 mCpuHotEjectData->Handler = NULL; >> 270 return; >> 271 } >> >> Here as well, I've been banking on aligned writes such that the APs would >> only see the before or after value not an intermediate value. >> >> Thanks >> Ankur >> >>> >>> Thanks >>> Laszlo >>> >> >