From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from aserp2130.oracle.com (aserp2130.oracle.com [141.146.126.79]) by mx.groups.io with SMTP id smtpd.web08.1021.1610696751173219037 for ; Thu, 14 Jan 2021 23:45:51 -0800 Authentication-Results: mx.groups.io; dkim=pass header.i=@oracle.com header.s=corp-2020-01-29 header.b=OfNAKyeR; spf=pass (domain: oracle.com, ip: 141.146.126.79, mailfrom: ankur.a.arora@oracle.com) Received: from pps.filterd (aserp2130.oracle.com [127.0.0.1]) by aserp2130.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 10F7hmoR185509; Fri, 15 Jan 2021 07:45:48 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=corp-2020-01-29; bh=YEaDLUO1i024jjzc4a7NMFoS7ycCIg4C9FWTLnVuTyY=; b=OfNAKyeRcIgp2eTQWF8XcXjIyHY1kukqMCCgNnpyXN9mxeBqT+NLQmC1EZQEaR1J3VKp ntv3IdOFALGw9AqwqHzIC823t/sd3rieJ0AnUVZvDnkaXDyo/Ml0oKmm7kA1i8GSvt6h jTsfiM3OiBxXYfDzGxqqtiRoHv9jtiXEzxzsLbVzGiQvbmWBC7TTZUN1OAt0qKz7qfCT 8w5FUKEBpCV70q8rOUms2IOpVo9v/7+W0GlWqAz9uAqHPO/bcbK/Jq3T+igZi0cq4Hke K2HK/h6u9MS2uMXYt1w/F8iksn9OaIjs3iep122+DU4Rk7QZAq/5CxgM7SANIy1hhHwK ug== Received: from aserp3020.oracle.com (aserp3020.oracle.com [141.146.126.70]) by aserp2130.oracle.com with ESMTP id 360kg23rep-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 15 Jan 2021 07:45:47 +0000 Received: from pps.filterd (aserp3020.oracle.com [127.0.0.1]) by aserp3020.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 10F7dkOF132020; Fri, 15 Jan 2021 07:45:47 GMT Received: from aserv0122.oracle.com (aserv0122.oracle.com [141.146.126.236]) by aserp3020.oracle.com with ESMTP id 360keavrex-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 15 Jan 2021 07:45:47 +0000 Received: from abhmp0019.oracle.com (abhmp0019.oracle.com [141.146.116.25]) by aserv0122.oracle.com (8.14.4/8.14.4) with ESMTP id 10F7jkPn008893; Fri, 15 Jan 2021 07:45:46 GMT Received: from localhost.localdomain (/70.36.60.91) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Thu, 14 Jan 2021 23:45:46 -0800 From: "Ankur Arora" To: devel@edk2.groups.io Cc: imammedo@redhat.com, lersek@redhat.com, Ankur Arora , Jordan Justen , Ard Biesheuvel , Boris Ostrovsky , Aaron Young Subject: [PATCH v3 08/10] OvmfPkg/CpuHotplugSmm: add worker to do CPU ejection Date: Thu, 14 Jan 2021 23:45:31 -0800 Message-Id: <20210115074533.277448-9-ankur.a.arora@oracle.com> X-Mailer: git-send-email 2.25.4 In-Reply-To: <20210115074533.277448-1-ankur.a.arora@oracle.com> References: <20210115074533.277448-1-ankur.a.arora@oracle.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=nai engine=6000 definitions=9864 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 adultscore=0 suspectscore=0 spamscore=0 mlxlogscore=999 malwarescore=0 bulkscore=0 mlxscore=0 phishscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2009150000 definitions=main-2101150044 X-Proofpoint-Virus-Version: vendor=nai engine=6000 definitions=9864 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 malwarescore=0 suspectscore=0 clxscore=1015 impostorscore=0 spamscore=0 priorityscore=1501 mlxscore=0 phishscore=0 mlxlogscore=999 bulkscore=0 adultscore=0 lowpriorityscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2009150000 definitions=main-2101150044 Content-Transfer-Encoding: quoted-printable Designate a worker CPU (we use the one executing the root MMI handler), which will do the final CPU ejection. This happens via CpuEject(). On the worker CPU, CpuEject() calls QEMU to do the ejection for each CPU that is unplugged. QEMU handles this by signalling the remote VCPU thread which forces the SMI AP to context switch out of the SMI, ending with its QEMU state destroyed. On the AP, CpuEject() spins around in its holding area until the context-switch happens. Given that the context switch would end up with the AP state being cleaned up, this means that the AP will never return to finish the SMI handling, and thus would not restore some of the CPU state that it ordinarily would (in SmiRendezvous() and in SmiEntry.nasm::CommonHandler). This unrestored state includes FPU state, CET enable, stuffing of RSB and the final RSM. Given that the CPU state is destroyed by QEMU on unplug, this should be okay. Cc: Laszlo Ersek Cc: Jordan Justen Cc: Ard Biesheuvel Cc: Igor Mammedov Cc: Boris Ostrovsky Cc: Aaron Young Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=3D3132 Signed-off-by: Ankur Arora --- OvmfPkg/CpuHotplugSmm/CpuHotplug.c | 61 ++++++++++++++++++++++++++++++++++= ---- 1 file changed, 56 insertions(+), 5 deletions(-) diff --git a/OvmfPkg/CpuHotplugSmm/CpuHotplug.c b/OvmfPkg/CpuHotplugSmm/Cpu= Hotplug.c index 4048490783e4..8aa52ebe5dd1 100644 --- a/OvmfPkg/CpuHotplugSmm/CpuHotplug.c +++ b/OvmfPkg/CpuHotplugSmm/CpuHotplug.c @@ -266,6 +266,20 @@ UnplugCpus( ToUnplugIdx++; } =20 + if (EjectCount) { + UINTN Worker; + Status =3D mMmCpuService->WhoAmI(mMmCpuService, &Worker); + ASSERT_EFI_ERROR(Status); + // + // UnplugCpus() is called via the root MMI handler and thus we are in = the + // BSP context. Accordingly, mark ourselves as the ejecting CPU. + // Note that, the QEMU eject protocol does not specify that only the B= SP + // can do the ejection, so this should be safe on any CPU (that is not= itself + // being unplugged.) + // + mCpuHotEjectData->ApicIdMap[Worker] =3D CPU_EJECT_WORKER; + } + // // We've handled this unplug. // @@ -383,11 +397,6 @@ CpuHotplugMmi ( if (EFI_ERROR (Status)) {=0D goto Fatal;=0D }=0D - if (ToUnplugCount > 0) {=0D - DEBUG ((DEBUG_ERROR, "%a: hot-unplug is not supported yet\n",=0D - __FUNCTION__));=0D - goto Fatal;=0D - }=0D =0D if (PluggedCount > 0) { Status =3D PlugCpus(mPluggedApicIds, PluggedCount); @@ -430,6 +439,48 @@ CpuEject( return; } =20 + if (ApicId =3D=3D CPU_EJECT_WORKER) { + UINT32 CpuIndex; + for (CpuIndex =3D 0; CpuIndex < mCpuHotEjectData->ArrayLength; CpuInde= x++) { + UINT64 RemoveApicId =3D mCpuHotEjectData->ApicIdMap[CpuIndex]; + + if ((RemoveApicId !=3D CPU_EJECT_INVALID && RemoveApicId !=3D CPU_EJ= ECT_WORKER)) { + + // + // The CPUs to be unplugged have received the BSP's signal to exit= the + // SMI and either will execute SmmCpuFeaturesSmiRendezvousExit() + // followed by this callback or are already waiting in the CpuDead= Loop() + // below. + // + // Tell QEMU to put them out of their misery. + // + QemuCpuhpWriteCpuSelector (mMmCpuIo, RemoveApicId); + QemuCpuhpWriteCpuStatus (mMmCpuIo, QEMU_CPUHP_STAT_EJECTED); + + // + // Compiler memory barrier to ensure the next store isn't reorder= ed + // + MemoryFence(); + + // Clear the unplug status for CpuIndex to ensure that an invalid = SMI + // later does not end up trying to unplug it or the newly hotplugg= ed + // CpuIndex does not go into the dead loop. + // + mCpuHotEjectData->ApicIdMap[CpuIndex] =3D CPU_EJECT_INVALID; + + DEBUG ((DEBUG_INFO, "%a: Unplugged CPU " FMT_APIC_ID "\n", + __FUNCTION__, RemoveApicId)); + } + } + + // + // Clear our own CPU status to ensure that we don't needlessly enter + // the this loop on the next SMI. + // + mCpuHotEjectData->ApicIdMap[ProcessorNum] =3D CPU_EJECT_INVALID; + return; + } + // // CPU(s) being unplugged get here from SmmCpuFeaturesSmiRendezvousExit() // after having been cleared to exit the SMI by the monarch and thus have --=20 2.9.3