From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from aserp2120.oracle.com (aserp2120.oracle.com [141.146.126.78]) by mx.groups.io with SMTP id smtpd.web12.30058.1610951832112032695 for ; Sun, 17 Jan 2021 22:37:12 -0800 Authentication-Results: mx.groups.io; dkim=pass header.i=@oracle.com header.s=corp-2020-01-29 header.b=I6FJrmVA; spf=pass (domain: oracle.com, ip: 141.146.126.78, mailfrom: ankur.a.arora@oracle.com) Received: from pps.filterd (aserp2120.oracle.com [127.0.0.1]) by aserp2120.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 10I6Y8Hh093275; Mon, 18 Jan 2021 06:37:09 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=corp-2020-01-29; bh=00CGxbdOEMP/6mNiVCFEwPCvKjehiK0y/BIsN8tcszs=; b=I6FJrmVARSdWTz5E4NiHUuCAaeEPmvtZ8dLAM/LmjdV7LQNxahHt//mkPjT3XyuYATgp bKThSo1xsGWz0OTxnhQUzGw2PsCwgEbcQGFQQdt+LoiPjJsXI2rhXPmleS5ll9u3o6vn sQmB82BLRTUTNqFe5ajcJmreagTXx/GG0AYvH1L63Tq+IyQZ6ypN2E4Tb/6jEWrwKpFU keSoMaT/rG7zOg4+0CoP+kKC5ADlaCo469zcSGaVhCFL2IM1FfTkWrJlEzmYREyoWJtA JWSFrxB62e8IH+CsHWtHFZTWIeB/ZkwUZ0J9+p0V0awJW7u7/06RbxKk+KKxVy4IDASI lQ== Received: from aserp3020.oracle.com (aserp3020.oracle.com [141.146.126.70]) by aserp2120.oracle.com with ESMTP id 363r3kkm9m-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 18 Jan 2021 06:37:09 +0000 Received: from pps.filterd (aserp3020.oracle.com [127.0.0.1]) by aserp3020.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 10I6TuBO099000; Mon, 18 Jan 2021 06:35:09 GMT Received: from aserv0122.oracle.com (aserv0122.oracle.com [141.146.126.236]) by aserp3020.oracle.com with ESMTP id 364a1vywpt-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 18 Jan 2021 06:35:09 +0000 Received: from abhmp0016.oracle.com (abhmp0016.oracle.com [141.146.116.22]) by aserv0122.oracle.com (8.14.4/8.14.4) with ESMTP id 10I6Z8ML001699; Mon, 18 Jan 2021 06:35:08 GMT Received: from prion.us.oracle.com (/10.159.226.191) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Sun, 17 Jan 2021 22:35:08 -0800 From: "Ankur Arora" To: devel@edk2.groups.io Cc: lersek@redhat.com, imammedo@redhat.com, boris.ostrovsky@oracle.com, Ankur Arora , Jordan Justen , Ard Biesheuvel , Aaron Young Subject: [PATCH v4 8/9] OvmfPkg/CpuHotplugSmm: add worker to do CPU ejection Date: Sun, 17 Jan 2021 22:34:56 -0800 Message-Id: <20210118063457.358581-9-ankur.a.arora@oracle.com> X-Mailer: git-send-email 2.25.4 In-Reply-To: <20210118063457.358581-1-ankur.a.arora@oracle.com> References: <20210118063457.358581-1-ankur.a.arora@oracle.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=nai engine=6000 definitions=9867 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 bulkscore=0 malwarescore=0 adultscore=0 mlxscore=0 spamscore=0 suspectscore=0 mlxlogscore=904 phishscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2009150000 definitions=main-2101180038 X-Proofpoint-Virus-Version: vendor=nai engine=6000 definitions=9867 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxlogscore=935 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 priorityscore=1501 mlxscore=0 malwarescore=0 phishscore=0 suspectscore=0 impostorscore=0 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2009150000 definitions=main-2101180038 Content-Transfer-Encoding: quoted-printable Designate a worker CPU (we use the one executing the root MMI handler), which will do the actual ejection via QEMU in CpuEject(). CpuEject(), on the worker CPU, ejects each marked CPU by first selecting its APIC ID and then sending the QEMU "eject" command. QEMU in-turn signals the remote VCPU thread which context-switches it out of the SMI. CpuEject(), on the CPU being ejected, spins around in its holding area until this final context-switch. This does mean that there is some CPU state that would ordinarily be restored (in SmiRendezvous() and in SmiEntry.nasm::CommonHandler), but will not be anymore. This unrestored state includes FPU state, CET enable, stuffing of RSB and the final RSM. Since the CPU state is destroyed by QEMU, this should be okay. Cc: Laszlo Ersek Cc: Jordan Justen Cc: Ard Biesheuvel Cc: Igor Mammedov Cc: Boris Ostrovsky Cc: Aaron Young Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=3D3132 Signed-off-by: Ankur Arora --- OvmfPkg/CpuHotplugSmm/CpuHotplug.c | 66 ++++++++++++++++++++++++++++++++++= +--- 1 file changed, 61 insertions(+), 5 deletions(-) diff --git a/OvmfPkg/CpuHotplugSmm/CpuHotplug.c b/OvmfPkg/CpuHotplugSmm/Cpu= Hotplug.c index e8ba9ae59e69..27fd982d6771 100644 --- a/OvmfPkg/CpuHotplugSmm/CpuHotplug.c +++ b/OvmfPkg/CpuHotplugSmm/CpuHotplug.c @@ -200,6 +200,54 @@ CpuEject( return; } =20 + if (ApicId =3D=3D CPU_EJECT_WORKER) { + UINT32 CpuIndex; + + for (CpuIndex =3D 0; CpuIndex < mCpuHotEjectData->ArrayLength; CpuInde= x++) { + UINT64 RemoveApicId =3D mCpuHotEjectData->ApicIdMap[CpuIndex]; + + if ((RemoveApicId !=3D CPU_EJECT_INVALID && + RemoveApicId !=3D CPU_EJECT_WORKER)) { + // + // This to-be-ejected-CPU has already received the BSP's SMI exit + // signal and, will execute SmmCpuFeaturesSmiRendezvousExit() + // followed by this callback or is already waiting in the + // CpuDeadLoop() below. + // + // Tell QEMU to context-switch it out. + // + QemuCpuhpWriteCpuSelector (mMmCpuIo, RemoveApicId); + QemuCpuhpWriteCpuStatus (mMmCpuIo, QEMU_CPUHP_STAT_EJECTED); + + // + // Compiler barrier to ensure the next store isn't reordered + // + MemoryFence(); + + // + // Clear the eject status for CpuIndex to ensure that an invalid + // SMI later does not end up trying to eject it or a newly + // hotplugged CpuIndex does not go into the dead loop. + // + mCpuHotEjectData->ApicIdMap[CpuIndex] =3D CPU_EJECT_INVALID; + + DEBUG ((DEBUG_INFO, "%a: Unplugged CPU %u -> " FMT_APIC_ID "\n", + __FUNCTION__, CpuIndex, RemoveApicId)); + } + } + + // + // Clear our own worker status. + // + mCpuHotEjectData->ApicIdMap[ProcessorNum] =3D CPU_EJECT_INVALID; + + // + // We are done until the next hot-unplug; clear the handler. + // + mCpuHotEjectData->Handler =3D NULL; + return; + } + // // CPU(s) being unplugged get here from SmmCpuFeaturesSmiRendezvousExit() // after having been cleared to exit the SMI by the monarch and thus have @@ -296,6 +344,19 @@ UnplugCpus( } =20 if (EjectCount) { + UINTN Worker; + + Status =3D mMmCpuService->WhoAmI(mMmCpuService, &Worker); + ASSERT_EFI_ERROR(Status); + // + // UnplugCpus() is called via the root MMI handler and thus we are + // executing in the BSP context. + // + // Mark ourselves as the worker CPU. + // + ASSERT (mCpuHotEjectData->ApicIdMap[Worker] =3D=3D CPU_EJECT_INVALID); + mCpuHotEjectData->ApicIdMap[Worker] =3D CPU_EJECT_WORKER; + // // We have processors to be ejected; install the handler. // @@ -419,11 +480,6 @@ CpuHotplugMmi ( if (EFI_ERROR (Status)) {=0D goto Fatal;=0D }=0D - if (ToUnplugCount > 0) {=0D - DEBUG ((DEBUG_ERROR, "%a: hot-unplug is not supported yet\n",=0D - __FUNCTION__));=0D - goto Fatal;=0D - }=0D =0D if (PluggedCount > 0) { Status =3D PlugCpus(mPluggedApicIds, PluggedCount); --=20 2.9.3