From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from aserp2120.oracle.com (aserp2120.oracle.com [141.146.126.78]) by mx.groups.io with SMTP id smtpd.web11.20112.1607630897819988190 for ; Thu, 10 Dec 2020 12:08:17 -0800 Authentication-Results: mx.groups.io; dkim=pass header.i=@oracle.com header.s=corp-2020-01-29 header.b=Au9UnfSE; spf=pass (domain: oracle.com, ip: 141.146.126.78, mailfrom: ankur.a.arora@oracle.com) Received: from pps.filterd (aserp2120.oracle.com [127.0.0.1]) by aserp2120.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 0BAK5PC5070967; Thu, 10 Dec 2020 20:08:17 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=subject : to : cc : references : from : message-id : date : mime-version : in-reply-to : content-type : content-transfer-encoding; s=corp-2020-01-29; bh=RrI7EOoCOchBOA1vnctabeeL0Cu7euaa+atKbwXPLug=; b=Au9UnfSE2SbakviEzzR3t1RkJWQXZWT0CjD3R0DpU17cLXwc335HYFatKdGpUXc/0Qxu 3z6OXEQ0E5bA2iN1/5BD8Ub2z2enTJci9tIo6CYUKN9EmU4KGFXglpKhAItifQ8GLwBa 1ZV0M0a5f+llNkIrBgDFLDzqLKM0KQy8W5fuyiUpBfhUFpvJe+ana4wuQ2z1VT1jWber mIatFCZLOp0QhfaqbuDDwu71oyUUqKKAdRaSm/QYlKuc7QazyXHHUNFQUjA9hvcZQP29 zn/1uPvutYfyqs7lewuW8NaYjYqIiidyFQ8iKAVy+o+yxI1gN07fy/WanNyDnoXK4wTc cQ== Received: from userp3030.oracle.com (userp3030.oracle.com [156.151.31.80]) by aserp2120.oracle.com with ESMTP id 35825mfce8-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=FAIL); Thu, 10 Dec 2020 20:08:16 +0000 Received: from pps.filterd (userp3030.oracle.com [127.0.0.1]) by userp3030.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 0BAK5pEB030157; Thu, 10 Dec 2020 20:08:16 GMT Received: from userv0122.oracle.com (userv0122.oracle.com [156.151.31.75]) by userp3030.oracle.com with ESMTP id 358m52qfwd-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 10 Dec 2020 20:08:16 +0000 Received: from abhmp0018.oracle.com (abhmp0018.oracle.com [141.146.116.24]) by userv0122.oracle.com (8.14.4/8.14.4) with ESMTP id 0BAK8FFt020362; Thu, 10 Dec 2020 20:08:15 GMT Received: from [10.159.154.97] (/10.159.154.97) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Thu, 10 Dec 2020 12:08:14 -0800 Subject: Re: [RFC PATCH 0/5] support CPU hot-unplug To: Laszlo Ersek , devel@edk2.groups.io Cc: imammedo@redhat.com, boris.ostrovsky@oracle.com References: <20201208053432.2690694-1-ankur.a.arora@oracle.com> <58d0ab9c-d50e-125f-d439-70a409bf95f0@redhat.com> From: "Ankur Arora" Message-ID: Date: Thu, 10 Dec 2020 12:08:13 -0800 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101 Thunderbird/68.11.0 MIME-Version: 1.0 In-Reply-To: <58d0ab9c-d50e-125f-d439-70a409bf95f0@redhat.com> X-Proofpoint-Virus-Version: vendor=nai engine=6000 definitions=9831 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxscore=0 spamscore=0 suspectscore=0 bulkscore=0 malwarescore=0 phishscore=0 adultscore=0 mlxlogscore=999 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2009150000 definitions=main-2012100126 X-Proofpoint-Virus-Version: vendor=nai engine=6000 definitions=9831 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=0 adultscore=0 bulkscore=0 phishscore=0 mlxlogscore=999 clxscore=1015 priorityscore=1501 mlxscore=0 spamscore=0 lowpriorityscore=0 malwarescore=0 impostorscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2009150000 definitions=main-2012100126 Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit On 2020-12-10 1:21 a.m., Laszlo Ersek wrote: > Hi Ankur, > > On 12/08/20 06:34, Ankur Arora wrote: >> [ Resending to the correct edk2 alias this time. ] >> >> Hi, >> >> This series adds support for CPU hot-unplug with OVMF. >> >> Please see this in conjunction with the QEMU v2 series posted here: >> https://lore.kernel.org/qemu-devel/20201207140739.3829993-1-imammedo@redhat.com/ >> >> In particular, would be glad for comments on Patch 4, specifically >> where we should be ejecting the CPU. >> >> Right now the ejection happens in UnplugCpus() (called from >> CpuHotplugMmi()): >> + QemuCpuhpWriteCpuSelector (mMmCpuIo, RemoveApicId); >> + QemuCpuhpWriteCpuStatus (mMmCpuIo, QEMU_CPUHP_STAT_EJECTED); >> >> That is way too early -- given that the actual unplug will happen >> in SmmCpuUpdate() and given that the BSPHandler() would have waited >> for the APs multiple times before then. >> >> Another possibility is that the actual ejection be deferred to the >> _EJ0 method after the return from the SMI. But, that seems like a >> hack. Additionally, Igor points out here that this approach has problems: >> https://lore.kernel.org/qemu-devel/20201204170939.1815522-1-imammedo@redhat.com/ >> >> Please review. > > thanks for the patches; I'm confirming I've got them. > > I'll need a non-trivial amount of time before I come to these patches > (and to the QEMU patches posted by Igor). > > I'm working very busily on > and my brain is > full of other stuff. Thanks for letting me know. I empathize with not wanting to context switch out all of your built up virtio-fs/ARM state. > > We had the reverse situation earlier this year, I think, when -- in > relation to hotplug -- Igor was occupied with a more pressing QEMU > development (NUMA I think?), for a significant amount of time. > > What's important is that I want to do both Igor's patches and your > patches *justice*, with my review, and at this time I just cannot sit > down with them alone for a day. These patches deserve a deep looking-at, > rather than a skim, and I cannot afford the former at the moment. I > prefer doing a (hopefully) thorough review, later, to rushing a review now. I'll look forward to it. Anyway I think a deep look at these patches might be wasted at the current stage. In particular there's a glaring hole in this patch set which is how to handle the actual unplug (setting of QEMU_CPUHP_STAT_EJECTED). That's one thing I would be glad for a comment on: not right away, please come back to this when you have thinking room. So the problem is that my current approach -- setting QEMU_CPUHP_STAT_EJECTED via the CpuHotplugMmi() handler definitely doesn't work given that it removes an AP immediately while the SMI handler is still using it. The two alternatives are: - do this in SmmCpuFeaturesLib::SmmCpuFeaturesRendezvousExit() while exiting SMI. That means that the only thing we will not do on the AP being unplugged is restoring debug registers and a bunch of MSRs. Which AFAICS would be okay, since the next time this AP is plugged in it will start from a clean slate anyway. - Qemu marks the hot-unplug when QEMU_CPUHP_STAT_EJECTED is set and defers it until the SMI exit. AFAICS, both ought to work. But, assuming it works (I haven't tried it out yet) the first seems cleaner. Ankur > > I hope that's tolerable. > > Thank you, > Laszlo >