From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from userp2130.oracle.com (userp2130.oracle.com [156.151.31.86]) by mx.groups.io with SMTP id smtpd.web09.8843.1611643504107837252 for ; Mon, 25 Jan 2021 22:45:04 -0800 Authentication-Results: mx.groups.io; dkim=pass header.i=@oracle.com header.s=corp-2020-01-29 header.b=b6w9bm5/; spf=pass (domain: oracle.com, ip: 156.151.31.86, mailfrom: ankur.a.arora@oracle.com) Received: from pps.filterd (userp2130.oracle.com [127.0.0.1]) by userp2130.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 10Q6XsGM053694; Tue, 26 Jan 2021 06:45:00 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-transfer-encoding : content-type : mime-version; s=corp-2020-01-29; bh=DGWFRrEXVuBMKJZS5HEyR44jBu4frhxHnnWSVDSsWsc=; b=b6w9bm5/CaLTocwW0pLVBkq8sd11nckiN/tS8aJ0vRVTOKlBk/vgmXZfSkuYuLbAtBJT Enz117z1VRR13Z8BBvP/LI1dVtQpvpmI5Fk4BJqvetmnPy4rvPbQd+2Ifaovk4QzN+eg EeaUsC+Ww4ftzp2yrOKnGnosV8L9WvM1DwjfVBF5OVhHg7vjyYloI4euwxmrai/dMQrt yWwkMB2yZJReB9NR9+pOkPj6rZ0nhVOAvzNjlRK/+QPEBfsHwN8zBYb9rTVSXT/23PKE fHaJQhgEVN5z1HS5W1vCh3dVRC7JT01QQTrxQV2Frm/LhHFuGN8wP2v9/GmyVC5d8IDO UA== Received: from userp3030.oracle.com (userp3030.oracle.com [156.151.31.80]) by userp2130.oracle.com with ESMTP id 368b7qrnuk-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 26 Jan 2021 06:45:00 +0000 Received: from pps.filterd (userp3030.oracle.com [127.0.0.1]) by userp3030.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 10Q6a688097113; Tue, 26 Jan 2021 06:45:00 GMT Received: from nam12-dm6-obe.outbound.protection.outlook.com (mail-dm6nam12lp2177.outbound.protection.outlook.com [104.47.59.177]) by userp3030.oracle.com with ESMTP id 368wqw1qcn-3 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 26 Jan 2021 06:45:00 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=G3MvQ4KWGEKVIcqiRH8Qffa0H73HkSrerjCkJ4YMrNcocgIsLi8DPBP7uu78IMShSwULSycj3hRAtxVQ8Yp3FXUQYMynLiRxBN/lm6E4avLxtwCZG4H9AUZdj7/OIi9LZJotWdvQ3IzcElyXKrQSsg/cWEyijDANkc35ixgeX1FQb/ggBwAUUNaHjFCpsNMTysBvxO8lh9EnrKbeiLCmbqd3QP+tD6HTI/MSFiBwU7iav9qEoIvoGS3G2uOOkO22hKUQv95fDNT7h9RdVn/Pk5/eO7rECRIz5LuLGlu8n+cwSJxA1L9f4eQD51XOEAzBCt/3tmpmJkg5IFht6I1ZBw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=DGWFRrEXVuBMKJZS5HEyR44jBu4frhxHnnWSVDSsWsc=; b=jatxgsc7RLgXdraHuI93MMEWIPTPFCJeKiB/JyXYFYgE7P7/P5/hGzVFXEPOF1IFewqiOJ5zHYhvvUvZc0+ldukwc/KrYFQNNmtxGd8BV4LkSCXgT958ikk/+aQQvFsQSWTA3kqtA/gAlNe6J/d8HvQW293YdE6CklagnaaeYK8Qmsh/7D/4tIZNl5L+5SibbrjqhA7YxazYa2twjGNwD0exkbabbJLj1aJy1ghjjlbm2INE3XINb92MLsk1BrdzJtCbKS3MJn9wc1gs4zT96dGgynBJS+HsgGAH/uvS4YJrqKu+9KkPjQqqPaOFg27f29zfjaY6A3ovy0CtOQDeMw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=DGWFRrEXVuBMKJZS5HEyR44jBu4frhxHnnWSVDSsWsc=; b=MLtssrF9uLwV0cW01/jsX7g2MWpLh/MnTqSZRjfsAo/jpImTwkIc9FmbnichxgNU439ltmq+zi6oiN5o6fbGSP2MoZqZVvBhvOu9+ZnlFwXC+KTtateL19h1+fPvcq3eMFrKI/wPKgltrUV4M8oGHDOd0kjU2o1/VPYkZ+vtgBk= Authentication-Results: edk2.groups.io; dkim=none (message not signed) header.d=none;edk2.groups.io; dmarc=none action=none header.from=oracle.com; Received: from CY4PR10MB1718.namprd10.prod.outlook.com (2603:10b6:910:9::17) by CY4PR1001MB2184.namprd10.prod.outlook.com (2603:10b6:910:45::37) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3763.14; Tue, 26 Jan 2021 06:44:57 +0000 Received: from CY4PR10MB1718.namprd10.prod.outlook.com ([fe80::39dc:6ba7:9397:cb6]) by CY4PR10MB1718.namprd10.prod.outlook.com ([fe80::39dc:6ba7:9397:cb6%6]) with mapi id 15.20.3784.019; Tue, 26 Jan 2021 06:44:57 +0000 From: "Ankur Arora" To: devel@edk2.groups.io Cc: lersek@redhat.com, imammedo@redhat.com, boris.ostrovsky@oracle.com, Ankur Arora , Jordan Justen , Ard Biesheuvel , Aaron Young Subject: [PATCH v5 7/9] OvmfPkg/CpuHotplugSmm: add CpuEject() Date: Mon, 25 Jan 2021 22:44:38 -0800 Message-Id: <20210126064440.299596-8-ankur.a.arora@oracle.com> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20210126064440.299596-1-ankur.a.arora@oracle.com> References: <20210126064440.299596-1-ankur.a.arora@oracle.com> X-Originating-IP: [70.36.60.91] X-ClientProxiedBy: MWHPR18CA0061.namprd18.prod.outlook.com (2603:10b6:300:39::23) To CY4PR10MB1718.namprd10.prod.outlook.com (2603:10b6:910:9::17) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from localhost.localdomain (70.36.60.91) by MWHPR18CA0061.namprd18.prod.outlook.com (2603:10b6:300:39::23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3784.12 via Frontend Transport; Tue, 26 Jan 2021 06:44:54 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 48f426e5-9121-4c47-0013-08d8c1c5e3c4 X-MS-TrafficTypeDiagnostic: CY4PR1001MB2184: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:9508; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: Zwz4Kaf9hLCldwrfZ1I0GJ99THtrulNCfkAXMQHO9D+zoVRkrO/4IDaR+Se7EgJGxfiiBzCJ86kvd1q+vomXSId8F5s0DbuVrU/N1QDxMx/FaBtTGn0Y25hOgtPyciqtIW4bFKH4yMCtYS59gftwzgdMhKL6PpIc7zabS1VCy+ESMUSLh5oD/LQ0evhwcSGDZajaJqAp5eYFNM0/11wdfbQwjuX06zzZKuA4uKtNnS46rziRrFSkDreNugneCqzZm4dw/XlOUoVIrA4BGEJOQ+PBeGMLpVKylwr+Xa/8p5bdf1l9mACCzf50Ov/184yW/yjAkcKoqA47A5RfvtMCqGnABC82dm9mDzuBGWe7C5dGz1+7kdKvrXRvjocpoNYgrXAdMjCXTX8v0AVJOL6P3AVLY4Ea/NLxVD/oehmYOvYpbofdg2qfxTn/xLHF3YmLmsuiipzR1BmLEgPNviZjYpmGiKO1SHbMpPGVzVXcnQaOGWfLkxSymfrY00/MnCwUs9aYEDzRSWGYMozp3fDxDQ== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:CY4PR10MB1718.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(396003)(39860400002)(376002)(346002)(136003)(69590400011)(6916009)(2616005)(4326008)(6512007)(52116002)(956004)(66556008)(6486002)(36756003)(186003)(6506007)(26005)(66476007)(6666004)(478600001)(8676002)(8936002)(83380400001)(966005)(1076003)(2906002)(86362001)(5660300002)(66946007)(103116003)(316002)(54906003)(16526019)(107886003);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData: =?us-ascii?Q?v95tPwGgCZlal9KbNdqYBKUZd3RjIb8DRP7cRq4kZkEpdpu3aDPuM0QbwO/w?= =?us-ascii?Q?Br2G7YhoXfBKphz3cpd6i3tcpnbxXoFBTNXM3CXhA4yoX00pWiHjZBBniVMi?= =?us-ascii?Q?rZvFe6O7zYn19oLfpZ2m2Bt0saoC+xRj/KzUgiWO2J5cwvWgx9jYE+idy5Ix?= =?us-ascii?Q?Z+nLhWN1+pFntRWqyqdDtxdaTFnIG5MSOmZk0ZUQZQj0HP2ImAPzyAFnp0Yn?= =?us-ascii?Q?L/Fkj9wh+6OSpfPqmWS2keXyx9OvFwrzqV53+cAPBvfnECK/1/MAnkPLF9T1?= =?us-ascii?Q?tQGTxdavZ6+ObYuYZxeChQLLA7It6XF73PiKvQY82hBX3eZZenuPs2mUs8lT?= =?us-ascii?Q?AVmVAwN72pVItT+sx/dvwTe7uiPEa8gBjfRRkFBGUCPeTcRVfxtXqIw/TjNG?= =?us-ascii?Q?GXfWVH3foJvs9k6paev9BdWfn5K6TazxOuEqCPcLMgvKY4t29eQvN60mFDAC?= =?us-ascii?Q?hgojvprHdEpKP5mbTKG2DB2WhqfSRh8wULsVOis2Ge/qsTzXPLn6Uu0xroqb?= =?us-ascii?Q?uRQAOM/XfGLkmRYvoqLgtNqtd7ZoD+jbTA+aZsfItKfCBXNXWmS8nTrPuZ1+?= =?us-ascii?Q?0ZxxDmNPnLdtiwsS0uTtC9kyfzYNbSNRPMGRkN6zRZTZ92IMk7mDvVCzLuK3?= =?us-ascii?Q?EnGF3bTWcxVM+ujfGI1wkfUX7sB0s/46yoqgA9iMCHSmzOA0jjWWBUkh1fYU?= =?us-ascii?Q?ajsgju++4FFgFWXEZ9D175bF4SCJ/WtbJVA3D2pkJhOENsfMb7ppCWzQ6Yq6?= =?us-ascii?Q?HjC8vG1t8aTchZ3XsrzsOSkXPIgfr15tWZX9zQQwfjCKSUoTmmfLwb5Titlr?= =?us-ascii?Q?g5PjxllSWRRUJzyqAcPe3mqRNsrLdZVlCbiFRJv1nh/eusbvVLS7QU4QarWm?= =?us-ascii?Q?RD3bsK8U5zCSz3Ykh0Tg8QdeadbjD/m+bFGVRa1Na+x6ebh4lcdZw7zgU/Fz?= =?us-ascii?Q?5kC/12yuFo3ee4bWOmFCWkdRXvOw9XwjHA9cchJMRujlktOoRujq2DJkZ31/?= =?us-ascii?Q?CNb3RKNkRf6ZgSxUPdtiH05IJwStIpL3pwBhroMX9GSO2lQz9ofOWK0j9DMD?= =?us-ascii?Q?auibLwBO?= X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: 48f426e5-9121-4c47-0013-08d8c1c5e3c4 X-MS-Exchange-CrossTenant-AuthSource: CY4PR10MB1718.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 26 Jan 2021 06:44:55.1144 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 8VfexRuORZ7mbjFwsqV5WYFDtn93q8Yz3e0qcT9j44H2cSnyIG7baeQLdcxSFRIADKAkB7ybXRk1+3PZRwjaW5C8hZclttfwsKDUaKGCdtU= X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY4PR1001MB2184 X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=9875 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxscore=0 spamscore=0 phishscore=0 adultscore=0 mlxlogscore=999 malwarescore=0 suspectscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2009150000 definitions=main-2101260035 X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=9875 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=0 spamscore=0 phishscore=0 adultscore=0 impostorscore=0 malwarescore=0 lowpriorityscore=0 bulkscore=0 priorityscore=1501 mlxscore=0 clxscore=1015 mlxlogscore=999 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2009150000 definitions=main-2101260035 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain Add CpuEject(), which handles the CPU ejection, and provides a holding area for said CPUs. It is called via SmmCpuFeaturesRendezvousExit(), at the tail end of the SMI handling. Also UnplugCpus() now stashes APIC IDs of CPUs which need to be ejected in CPU_HOT_EJECT_DATA.ApicIdMap. These are used by CpuEject() to identify such CPUs. Cc: Laszlo Ersek Cc: Jordan Justen Cc: Ard Biesheuvel Cc: Igor Mammedov Cc: Boris Ostrovsky Cc: Aaron Young Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=3D3132 Signed-off-by: Ankur Arora --- OvmfPkg/CpuHotplugSmm/CpuHotplug.c | 94 ++++++++++++++++++++++++++++++++++= +++- 1 file changed, 93 insertions(+), 1 deletion(-) diff --git a/OvmfPkg/CpuHotplugSmm/CpuHotplug.c b/OvmfPkg/CpuHotplugSmm/Cpu= Hotplug.c index d165d6ccea0d..99e6845a12d9 100644 --- a/OvmfPkg/CpuHotplugSmm/CpuHotplug.c +++ b/OvmfPkg/CpuHotplugSmm/CpuHotplug.c @@ -14,6 +14,7 @@ #include // gMmst=0D #include // PcdGetBool()=0D #include // SafeUintnSub()=0D +#include // CPU_HOT_EJECT_DATA=0D #include // EFI_MM_CPU_IO_PROTOCOL=0D #include // EFI_SMM_CPU_SERVICE_PROTOC= OL=0D #include // EFI_STATUS=0D @@ -32,11 +33,12 @@ STATIC EFI_MM_CPU_IO_PROTOCOL *mMmCpuIo; //=0D STATIC EFI_SMM_CPU_SERVICE_PROTOCOL *mMmCpuService;=0D //=0D -// This structure is a communication side-channel between the=0D +// These structures serve as communication side-channels between the=0D // EFI_SMM_CPU_SERVICE_PROTOCOL consumer (i.e., this driver) and provider= =0D // (i.e., PiSmmCpuDxeSmm).=0D //=0D STATIC CPU_HOT_PLUG_DATA *mCpuHotPlugData;=0D +STATIC CPU_HOT_EJECT_DATA *mCpuHotEjectData;=0D //=0D // SMRAM arrays for fetching the APIC IDs of processors with pending event= s (of=0D // known event types), for the time of just one MMI.=0D @@ -182,6 +184,43 @@ Fatal: }=0D =0D /**=0D + CPU Hot-eject handler function.=0D +=0D + If the executing CPU is not being ejected: NOP.=0D + If the executing CPU is being ejected: wait in a CpuDeadLoop()=0D + until ejected.=0D +=0D + @param[in] ProcessorNum Index of executing CPU.=0D +**/=0D +VOID=0D +EFIAPI=0D +CpuEject(=0D + IN UINTN ProcessorNum=0D + )=0D +{=0D + //=0D + // APIC ID is UINT32, but mCpuHotEjectData->ApicIdMap[] is UINT64=0D + // so use UINT64 throughout.=0D + //=0D + UINT64 ApicId;=0D +=0D + ApicId =3D mCpuHotEjectData->ApicIdMap[ProcessorNum];=0D + if (ApicId =3D=3D CPU_EJECT_INVALID) {=0D + return;=0D + }=0D +=0D + //=0D + // CPU(s) being unplugged get here from SmmCpuFeaturesSmiRendezvousExit(= )=0D + // after having been cleared to exit the SMI by the monarch and thus hav= e=0D + // no SMM processing remaining.=0D + //=0D + // Given that we cannot allow them to escape to the guest, we pen them=0D + // here until the SMM monarch tells the HW to unplug them.=0D + //=0D + CpuDeadLoop ();=0D +}=0D +=0D +/**=0D CPU Hot-unplug MMI handler function.=0D =0D @param[in] UnplugApicIds List of APIC IDs to be unplugged.=0D @@ -203,9 +242,11 @@ UnplugCpus( {=0D EFI_STATUS Status;=0D UINT32 ToUnplugIdx;=0D + UINT32 EjectCount;=0D UINTN ProcessorNum;=0D =0D ToUnplugIdx =3D 0;=0D + EjectCount =3D 0;=0D while (ToUnplugIdx < ToUnplugCount) {=0D APIC_ID RemoveApicId;=0D =0D @@ -242,11 +283,35 @@ UnplugCpus( DEBUG ((DEBUG_ERROR, "%a: RemoveProcessor(" FMT_APIC_ID "): %r\n",=0D __FUNCTION__, RemoveApicId, Status));=0D goto Fatal;=0D + } else {=0D + //=0D + // Stash the APIC IDs so we can do the actual unplug later.=0D + //=0D + if (mCpuHotEjectData->ApicIdMap[ProcessorNum] !=3D CPU_EJECT_INVALID= ) {=0D + //=0D + // Since ProcessorNum and APIC-ID map 1-1, so a valid=0D + // mCpuHotEjectData->ApicIdMap[ProcessorNum] means something=0D + // is horribly wrong.=0D + //=0D + DEBUG ((DEBUG_ERROR, "%a: ProcessorNum %u maps to %llx, cannot map= to " FMT_APIC_ID "\n",=0D + __FUNCTION__, ProcessorNum, mCpuHotEjectData->ApicIdMap[Processo= rNum], RemoveApicId));=0D + goto Fatal;=0D + }=0D +=0D + mCpuHotEjectData->ApicIdMap[ProcessorNum] =3D (UINT64)RemoveApicId;= =0D + EjectCount++;=0D }=0D =0D ToUnplugIdx++;=0D }=0D =0D + if (EjectCount !=3D 0) {=0D + //=0D + // We have processors to be ejected; install the handler.=0D + //=0D + mCpuHotEjectData->Handler =3D CpuEject;=0D + }=0D +=0D //=0D // We've handled this unplug.=0D //=0D @@ -445,7 +510,13 @@ CpuHotplugEntry ( // Our DEPEX on EFI_SMM_CPU_SERVICE_PROTOCOL guarantees that PiSmmCpuDxe= Smm=0D // has pointed PcdCpuHotPlugDataAddress to CPU_HOT_PLUG_DATA in SMRAM.=0D //=0D + // Additionally, CPU Hot-unplug is available only if CPU Hotplug is, so= =0D + // the same DEPEX also guarantees that PcdCpuHotEjectDataAddress points= =0D + // to CPU_HOT_EJECT_DATA in SMRAM.=0D + //=0D mCpuHotPlugData =3D (VOID *)(UINTN)PcdGet64 (PcdCpuHotPlugDataAddress);= =0D + mCpuHotEjectData =3D (VOID *)(UINTN)PcdGet64 (PcdCpuHotEjectDataAddress)= ;=0D +=0D if (mCpuHotPlugData =3D=3D NULL) {=0D Status =3D EFI_NOT_FOUND;=0D DEBUG ((DEBUG_ERROR, "%a: CPU_HOT_PLUG_DATA: %r\n", __FUNCTION__, Stat= us));=0D @@ -457,6 +528,9 @@ CpuHotplugEntry ( if (mCpuHotPlugData->ArrayLength =3D=3D 1) {=0D return EFI_UNSUPPORTED;=0D }=0D + ASSERT (mCpuHotEjectData &&=0D + (mCpuHotPlugData->ArrayLength =3D=3D mCpuHotEjectData->ArrayLeng= th));=0D +=0D //=0D // Allocate the data structures that depend on the possible CPU count.=0D //=0D @@ -539,6 +613,24 @@ CpuHotplugEntry ( //=0D SmbaseInstallFirstSmiHandler ();=0D =0D + if (mCpuHotEjectData) {=0D + UINT32 Idx;=0D + //=0D + // For CPU ejection we need to map ProcessorNum -> APIC_ID. By the tim= e=0D + // we do that, however, the Processor's APIC ID has already been remov= ed=0D + // from SMM data structures. So we will use mCpuHotEjectData->ApicIdMa= p=0D + // to map from ProcessorNum -> APIC_ID.=0D + //=0D + for (Idx =3D 0; Idx < mCpuHotEjectData->ArrayLength; Idx++) {=0D + mCpuHotEjectData->ApicIdMap[Idx] =3D CPU_EJECT_INVALID;=0D + }=0D +=0D + //=0D + // Wait to init the handler until an ejection is warranted=0D + //=0D + mCpuHotEjectData->Handler =3D NULL;=0D + }=0D +=0D return EFI_SUCCESS;=0D =0D ReleasePostSmmPen:=0D --=20 2.9.3