From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from aserp2120.oracle.com (aserp2120.oracle.com [141.146.126.78]) by mx.groups.io with SMTP id smtpd.web11.2146.1610049335374482640 for ; Thu, 07 Jan 2021 11:55:35 -0800 Authentication-Results: mx.groups.io; dkim=pass header.i=@oracle.com header.s=corp-2020-01-29 header.b=NsCsv2fB; spf=pass (domain: oracle.com, ip: 141.146.126.78, mailfrom: ankur.a.arora@oracle.com) Received: from pps.filterd (aserp2120.oracle.com [127.0.0.1]) by aserp2120.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 107JtVks013664; Thu, 7 Jan 2021 19:55:32 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=corp-2020-01-29; bh=blMXL3IxUMJ+MCYZMVyGruZTP5THd4K/LEEn/iRBrkY=; b=NsCsv2fBtnNHGcyK2P5xIAMsIgNK0lkr+jA/Z6iAhgVc6nlIvdyt7/0crKbfkcW5H772 UqKELik8i5iaw3KkiMJh0WKeorXPTWB9kKKdYheXM2WHlPELkfdsIYcIDU8RH7xxLf4D CUz8thLfGRMjnONJH0qgXIQd74w9gtIRcxTyfdBPNRWI+4HIuf+ZUPnZJo6udUVyNiJv sFSTfImHL8IIucvs+n2YcsIooNZL/9C/Ne57i5nrWncdsEkdjmwRF8giT5a5PmQLeM2h smoO8UMjSkxoxkn3rfNk/o+ZosThuMBozXprt5CDK1B7QNByYaw5j5RByHCkaGd5x0La gw== Received: from userp3030.oracle.com (userp3030.oracle.com [156.151.31.80]) by aserp2120.oracle.com with ESMTP id 35wepme2jw-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=FAIL); Thu, 07 Jan 2021 19:55:32 +0000 Received: from pps.filterd (userp3030.oracle.com [127.0.0.1]) by userp3030.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 107JtSh4109071; Thu, 7 Jan 2021 19:55:31 GMT Received: from userv0122.oracle.com (userv0122.oracle.com [156.151.31.75]) by userp3030.oracle.com with ESMTP id 35w3g3835t-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 07 Jan 2021 19:55:31 +0000 Received: from abhmp0002.oracle.com (abhmp0002.oracle.com [141.146.116.8]) by userv0122.oracle.com (8.14.4/8.14.4) with ESMTP id 107JtU30021843; Thu, 7 Jan 2021 19:55:30 GMT Received: from localhost.localdomain (/70.36.60.91) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Thu, 07 Jan 2021 11:55:30 -0800 From: "Ankur Arora" To: devel@edk2.groups.io Cc: lersek@redhat.com, imammedo@redhat.com, boris.ostrovsky@oracle.com, Ankur Arora , Jordan Justen , Ard Biesheuvel , Eric Dong , Ray Ni , Rahul Kumar , Aaron Young Subject: [PATCH v2 08/10] OvmfCpuPkg/CpuHotplug: add a hot-unplug handler called at SMI exit Date: Thu, 7 Jan 2021 11:55:13 -0800 Message-Id: <20210107195515.106158-9-ankur.a.arora@oracle.com> X-Mailer: git-send-email 2.25.4 In-Reply-To: <20210107195515.106158-1-ankur.a.arora@oracle.com> References: <20210107195515.106158-1-ankur.a.arora@oracle.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=nai engine=6000 definitions=9857 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxscore=0 malwarescore=0 adultscore=0 phishscore=0 spamscore=0 mlxlogscore=999 suspectscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2009150000 definitions=main-2101070115 X-Proofpoint-Virus-Version: vendor=nai engine=6000 definitions=9857 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 adultscore=0 bulkscore=0 spamscore=0 impostorscore=0 phishscore=0 lowpriorityscore=0 suspectscore=0 priorityscore=1501 mlxscore=0 malwarescore=0 clxscore=1015 mlxlogscore=999 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2009150000 definitions=main-2101070115 Content-Transfer-Encoding: quoted-printable Add CpuUnplugExitWork(), to be called from SmmCpuFeaturesRendezvousExit() to do the final ejection as part of CPU hot-unplug. On the BSP, CpuUnplugExitWork() calls QEMU to do the ejection for each CPU that is unplugged. QEMU handles this by signalling the remote VCPU thread which forces the SMI AP to context switch out of the SMI and with its QEMU state destroyed. On the AP, CpuUnplugExitWork() provides a holding area where the CPU spins until context switched out by QEMU via the BSP. Given that the context switch would end up with the AP state being cleaned up, this means that the AP CPU will never return to finish the SMI handling, and thus would not restore some of the CPU state that it ordinarily would (in SmiRendezvous() and in SmiEntry.nasm::CommonHandler). This unrestored state includes FPU state, CET enable, stuffing of RSB and the final RSM. Given that the CPU state is destroyed by QEMU on unplug, this should be okay. Cc: Laszlo Ersek Cc: Jordan Justen Cc: Ard Biesheuvel Cc: Eric Dong Cc: Ray Ni Cc: Rahul Kumar Cc: Igor Mammedov Cc: Boris Ostrovsky Cc: Aaron Young Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=3D3132 Signed-off-by: Ankur Arora --- OvmfPkg/CpuHotplugSmm/CpuHotplug.c | 68 +++++++++++++++++++++++++++++++= ++++ UefiCpuPkg/PiSmmCpuDxeSmm/MpService.c | 6 ++++ 2 files changed, 74 insertions(+) diff --git a/OvmfPkg/CpuHotplugSmm/CpuHotplug.c b/OvmfPkg/CpuHotplugSmm/Cpu= Hotplug.c index 20d92a35da39..379c9a66f261 100644 --- a/OvmfPkg/CpuHotplugSmm/CpuHotplug.c +++ b/OvmfPkg/CpuHotplugSmm/CpuHotplug.c @@ -421,6 +421,69 @@ Fatal: return EFI_INTERRUPT_PENDING;=0D }=0D =0D +EFI_STATUS=0D +EFIAPI=0D +CpuUnplugExitWork(=0D + IN UINTN CpuIndex,=0D + IN BOOLEAN IsBSP=0D + )=0D +{=0D + APIC_ID RemoveApicId;=0D +=0D + RemoveApicId =3D mHotUnplugWork[CpuIndex];=0D +=0D + if (!IsBSP && RemoveApicId =3D=3D MAX_UINT32) {=0D + return EFI_SUCCESS;=0D + }=0D +=0D + if (IsBSP) {=0D + UINT32 Idx;=0D + for (Idx =3D 0; Idx < mCpuHotPlugData->ArrayLength; Idx++) {=0D + RemoveApicId =3D mHotUnplugWork[Idx];=0D +=0D + if (RemoveApicId !=3D MAX_UINT32) {=0D + //=0D + // The CPU(s) to be unplugged have received the BSP's signal to exit the= =0D + // SMI and either will execute SmmCpuFeaturesSmiRendezvousExit() and this= =0D + // callback or are waiting here.=0D + //=0D + // Tell HW to put it out of its misery.=0D + //=0D + QemuCpuhpWriteCpuSelector (mMmCpuIo, RemoveApicId);=0D + QemuCpuhpWriteCpuStatus (mMmCpuIo, QEMU_CPUHP_STAT_EJECTED);=0D +=0D + //=0D + // Barrier to ensure that the compiler doesn't reorder the next store=0D + //=0D + MemoryFence();=0D +=0D + //=0D + // Clear the unplug status to make sure that an invalid SMI later=0D + // does not try to do an unplug or go to the dead loop.=0D + //=0D + mHotUnplugWork[Idx] =3D MAX_UINT32;=0D +=0D + DEBUG ((DEBUG_INFO, "%a: Unplugged CPU " FMT_APIC_ID "\n",=0D + __FUNCTION__, RemoveApicId));=0D + }=0D + }=0D + return EFI_SUCCESS;=0D + }=0D + =0D + //=0D + // CPU(s) being unplugged get here from SmmCpuFeaturesSmiRendezvousExit(= )=0D + // after having been cleared to exit the SMI by the monarch and thus hav= e=0D + // no SMM processing remaining.=0D + //=0D + // Given that we cannot allow them to escape to the guest, we pen them=0D + // here until the SMM monarch tells the HW to unplug them.=0D + //=0D +=0D + CpuDeadLoop ();=0D +=0D + return EFI_ABORTED;=0D +}=0D +=0D =0D //=0D // Entry point function of this driver.=0D @@ -573,6 +636,11 @@ CpuHotplugEntry ( }=0D =0D //=0D + // Register handler for hot-unplugging an AP.=0D + //=0D + MmRegisterShutdownInterface(CpuUnplugExitWork);=0D +=0D + //=0D // Register the handler for the CPU Hotplug MMI.=0D //=0D Status =3D gMmst->MmiHandlerRegister (=0D diff --git a/UefiCpuPkg/PiSmmCpuDxeSmm/MpService.c b/UefiCpuPkg/PiSmmCpuDxe= Smm/MpService.c index fb6aab17de37..f246d730d1e2 100644 --- a/UefiCpuPkg/PiSmmCpuDxeSmm/MpService.c +++ b/UefiCpuPkg/PiSmmCpuDxeSmm/MpService.c @@ -1727,6 +1727,12 @@ SmiRendezvous ( }=0D }=0D =0D + //=0D + // Note that the BSP will unplug any CPUs that have been marked for=0D + // hot-unplug at any point after it sets AllCpusInSync =3D FALSE=0D + // so it cannot depend on an AP executing code post that point.=0D + //=0D +=0D Exit:=0D SmmCpuFeaturesRendezvousExit (CpuIndex, IsBsp);=0D =0D --=20 2.9.3