From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from aserp2120.oracle.com (aserp2120.oracle.com [141.146.126.78]) by mx.groups.io with SMTP id smtpd.web09.3211.1611882126714735838 for ; Thu, 28 Jan 2021 17:02:06 -0800 Authentication-Results: mx.groups.io; dkim=pass header.i=@oracle.com header.s=corp-2020-01-29 header.b=ZmiwGckR; spf=pass (domain: oracle.com, ip: 141.146.126.78, mailfrom: ankur.a.arora@oracle.com) Received: from pps.filterd (aserp2120.oracle.com [127.0.0.1]) by aserp2120.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 10T0gDlp065042; Fri, 29 Jan 2021 01:02:04 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-transfer-encoding : content-type : mime-version; s=corp-2020-01-29; bh=Guq00U15YcGAAy1mDO2Rk+JcBjl0zhlnNSefjcd/tks=; b=ZmiwGckRKKv1ERmpemDypWHoTbNMf0nMYdNONRVQg8syHEPKNhAquMM9lxXQgWxExkLa UCjQ76w+cPvEoWhYeAChdh57CfjLUp+S15Os5kzrKnVdg/ap1T/AGf2EDn+P0Lf+I8Zp G17bN99h+AF5KD3n5R4Xbr1ItJmkhWkdjq5c0UyMhE8J+q2HM5yx4DTweqtAqCvKa9p4 5CJ6193iK1ZwAPG7cttO/Ou3KBOhBgzbKZ9895kXfyP+4NKtG5LJ4LlyUTVnkxtPEpHZ DtMS9Ln+Lt50NhBzWUsw9CTQ/TeuJS3BmWsNwFlUcdAxGBsS6wtJytglMJUcMDrnfyb0 dQ== Received: from userp3030.oracle.com (userp3030.oracle.com [156.151.31.80]) by aserp2120.oracle.com with ESMTP id 368brkxsbf-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 29 Jan 2021 01:02:03 +0000 Received: from pps.filterd (userp3030.oracle.com [127.0.0.1]) by userp3030.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 10T0eGVF081743; Fri, 29 Jan 2021 01:00:03 GMT Received: from nam11-co1-obe.outbound.protection.outlook.com (mail-co1nam11lp2172.outbound.protection.outlook.com [104.47.56.172]) by userp3030.oracle.com with ESMTP id 368wr13e5c-3 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 29 Jan 2021 01:00:02 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=iBpJyE2cQ731ZWCWY29AyFPQJnKR3YDc6iknz1qA6sXolGUzSaeeEQGWSFerVyb7Fic+WhYYZFrNzYUKjwGsysbC55i2lLgerUSxPjrVnq41aI+L5u4v+cT3buYRX1eV5tTclwoFPRWdXKC75rXgDW+m7DxMXEAx6oF1Xdt8Djcv88Ui8ZVRvhyVtNDnqGkJqv6e2U3CRDXP6OptcIKtsotDqWZSAdr1y4T5bw+WxIU4GHMedv0biRl73kXHiIJB8H+GAqH+PyhgIYd10oz9Pwbf3saDpU38zFsO5OKpCuG0GkpQ4sZr4VPwpimn1LCddZKQyDLN0PKAc1PPHLiELg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Guq00U15YcGAAy1mDO2Rk+JcBjl0zhlnNSefjcd/tks=; b=aQP6JBBhJi0AMX+g6JVJ1KyU+FHA7l/FDbGfQHBUNcQWfiK8x6sMloosLDKVZMwUJgOD5u9rbWsqbKzWOe/dX8ni7TOJ1KiU3aN0paScJz5roQYghB64NhZeCkbeu72xy/pvQDJKHktAZXA3U3fD6hV4v8Bl1ElTaKKlcRPA1TT2gDJwz28eLlnoqKy+oS5hTiHrbc+sP7lu+RihmzuaeEM7lbN+f1Cub16RzTQYaPl6HJnyOGM31hBm2uh60/3O51CfL/gxmKo8qMA5ZmXmB4PHeHfhjPMkwqG5SbKwoHWJsN4mHv19xw1tiGfoOemKfhDi2rPctFr973pHEf4EIA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Guq00U15YcGAAy1mDO2Rk+JcBjl0zhlnNSefjcd/tks=; b=wcbwofr96RSuJPDZCFAxW/HfstRnyDki70ioXMSfSXRdhZu2tQ3R7Q89G46hionrnH9xohlD3gT5ak11AweLD8v1loaFCcns5+kdgNEC3LxaNpCs5ZiSHgOnXMsYwOfcKfnE8vQsYOor6VumnBVRMe/n9KCX5cgJeco1FfjTrRE= Authentication-Results: edk2.groups.io; dkim=none (message not signed) header.d=none;edk2.groups.io; dmarc=none action=none header.from=oracle.com; Received: from CY4PR10MB1718.namprd10.prod.outlook.com (2603:10b6:910:9::17) by CY4PR1001MB2246.namprd10.prod.outlook.com (2603:10b6:910:41::20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3784.16; Fri, 29 Jan 2021 01:00:00 +0000 Received: from CY4PR10MB1718.namprd10.prod.outlook.com ([fe80::39dc:6ba7:9397:cb6]) by CY4PR10MB1718.namprd10.prod.outlook.com ([fe80::39dc:6ba7:9397:cb6%6]) with mapi id 15.20.3805.019; Fri, 29 Jan 2021 01:00:00 +0000 From: "Ankur Arora" To: devel@edk2.groups.io Cc: lersek@redhat.com, imammedo@redhat.com, boris.ostrovsky@oracle.com, Ankur Arora , Jordan Justen , Ard Biesheuvel , Aaron Young Subject: [PATCH v6 2/9] OvmfPkg/CpuHotplugSmm: collect hot-unplug events Date: Thu, 28 Jan 2021 16:59:43 -0800 Message-Id: <20210129005950.467638-3-ankur.a.arora@oracle.com> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20210129005950.467638-1-ankur.a.arora@oracle.com> References: <20210129005950.467638-1-ankur.a.arora@oracle.com> X-Originating-IP: [148.87.23.8] X-ClientProxiedBy: MWHPR1401CA0022.namprd14.prod.outlook.com (2603:10b6:301:4b::32) To CY4PR10MB1718.namprd10.prod.outlook.com (2603:10b6:910:9::17) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from prion.us.oracle.com (148.87.23.8) by MWHPR1401CA0022.namprd14.prod.outlook.com (2603:10b6:301:4b::32) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3805.17 via Frontend Transport; Fri, 29 Jan 2021 00:59:59 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 147cf1a0-c8a9-41b0-dfef-08d8c3f1340f X-MS-TrafficTypeDiagnostic: CY4PR1001MB2246: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:10000; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: ZESUD/2pirqdloXZwLMQqBpelaqOPkOYof9Lryin43f0hjm1nk82Aw3+fOnlxZ28JwNBzT//K2H+9OHgSdsGvZoQZf7CCkLPd5+vXaiXKQ7twIKRQuu9hoOHPnt35YZfdVqImXYutMxMKWbTrjRNKH8NOf8zFdwx/OQveG2s3RqzyUksSBKPmGJvBMYaEl1kpzAEFRKIcbbtZYdBWZzYjkC744AMXYQQ0LseB9mSdjcD7ZUlFIjwhzebH8v5ejEeVTBv9QfGzUQw6MEUB9tdGKdq+TgAKoCXJxJM28ingmKj0PzrtQzniLqWAcc+uiWyrfLk74nzwIU21VvaR64jjH6IqwbBj/S7Grv7ZCjlVCNkJS1ThquBxlcK7jljmA21CFS2lAdINdNj2FIxt5Bq2fmPDp0EStEdVVEyfz7FYqK2DH5OOOkRVIPdw3iXGm/nl4fylxEdYfQ1uBZUI0V0YKKBxNgBiDsrPOv16rUfTDSjpywMAj0NmJFTme2hXfC+jLS9r47LQpsFRv8FVcjWiDw6p6sxbk8wqu3sHROPOxsl0wCUeJxnI+RtJjehsXBAsOGzGo+5RKQZm8dDfNHF8qHPSOCGKgvRR8uycZZcR6M= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:CY4PR10MB1718.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(39860400002)(346002)(376002)(396003)(136003)(8936002)(107886003)(2906002)(66946007)(66476007)(66556008)(36756003)(1076003)(103116003)(16526019)(956004)(966005)(7696005)(478600001)(52116002)(54906003)(6486002)(86362001)(83380400001)(6666004)(8676002)(186003)(5660300002)(4326008)(6916009)(26005)(316002)(2616005);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData: =?us-ascii?Q?RFFDeLOV4N9AhB575GQDPng/Zp79FCiXvUuawAD8wVVthYPo/a98SQzaGZ5b?= =?us-ascii?Q?Cr+dYjZeZ3hywUvfoxIh6z+z9LOYBmtRN2hSxnINjuXCzd7Z6I84CS22irMW?= =?us-ascii?Q?KDq/A+ymOyHFuCpcYkqO8+vztoSmBgk3UoAxu5fqlOV9VuECEhoTU0M4BuoA?= =?us-ascii?Q?X92QQhgKHO1FF/nKeaAUhf0uNHr9o+cUTOfAGG9RP6C7CUNFpmta56j1FtJH?= =?us-ascii?Q?HgHf3BBac4p5MDhWmBGc2Fa4kRXKmC1VQsOJjJizvaYOmv3sorbZ4ankv4X2?= =?us-ascii?Q?78VzjywRyelTpsap0rOsVQODnKIBB93z2GcxyUZiNLKrtED8BISM7D5F/nJ3?= =?us-ascii?Q?XPrEayXfSWWpkkH4g/WStQDnCMHQlZSc5e/cvLAmhOwwSQsV4yiyeO6jcwk5?= =?us-ascii?Q?aZcNqnzFrW0g8k3TCJVunLovJvqiXl4/uzGm1AsJiyXrW7C4OK7LC88omMyk?= =?us-ascii?Q?XE4ksF1705hnVQxTUFK9m1YC6p7iOTA7rxNfX/a9n4As2i27PCDfdhY8lur1?= =?us-ascii?Q?gB7Z+osUqQmK+IX9GrcbL6QJV/Pssi8pONbNOHlU4ceGeI4+UR+dSGu1J/nC?= =?us-ascii?Q?0u8TZKP/UmbmSpKVULpqrngDluKPyl+6sRibNspSBsKqJpGFa2Ek3ducqq9T?= =?us-ascii?Q?5PSD7kRPN4g1zWNNI0bOho3wjRi6oktH0YPzKoyko7wcJ4CBg5BGlglyfEdu?= =?us-ascii?Q?0GOR1dMIMm9i7LCUGyvu8L7FVl1jFECeArxunHNDe3s+42jry3Di5euvv05N?= =?us-ascii?Q?nFb2rl/1LBsUHIOs3r5Aq3RLa3iDnuD8ASXcCqtWxbofHQzblwq942y2whiD?= =?us-ascii?Q?U37IzeZNSYkEZdRtYkAKQquAe9qTSAjSCC9V7idlapJXWimeTxYIqkZxay7e?= =?us-ascii?Q?zpSN+87VNOsHPV9fO+Uz43FWgUK+w62NuXqV+y2WTkYXwb1X1JQ2v+gusf5k?= =?us-ascii?Q?OQs6MyBa9uXc/uabhWgSoB+9cRfXglnZHibsAe438cP4yErP8ZEYiEEdXN1z?= =?us-ascii?Q?2XLYkpThh1N5ZMmUgbdQPwOh4+geppfPJZtubMeLGFDWp9th0zf/t0TbP/Up?= =?us-ascii?Q?4R2k6NsC?= X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: 147cf1a0-c8a9-41b0-dfef-08d8c3f1340f X-MS-Exchange-CrossTenant-AuthSource: CY4PR10MB1718.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 29 Jan 2021 01:00:00.4583 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: F7Ek/8iKEJY/Fxlze3OKy/feHWWIYLc8gXn26CmzTf8Q3HQWGyOUslCiQlt8D8VCymoE861X/NbThs626TCGpuAkbVC9jg9pKz6tjGBsCEo= X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY4PR1001MB2246 X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=9878 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxscore=0 spamscore=0 phishscore=0 adultscore=0 mlxlogscore=999 malwarescore=0 suspectscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2009150000 definitions=main-2101290001 X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=9878 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 adultscore=0 impostorscore=0 phishscore=0 bulkscore=0 priorityscore=1501 mlxlogscore=999 lowpriorityscore=0 spamscore=0 mlxscore=0 suspectscore=0 malwarescore=0 clxscore=1015 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2009150000 definitions=main-2101290001 Content-Transfer-Encoding: 8bit Content-Type: text/plain Process fw_remove events in QemuCpuhpCollectApicIds() and collect corresponding APIC IDs for CPUs that are being hot-unplugged. In addition, we now ignore CPUs which only have remove set. These CPUs haven't been processed by OSPM yet. This is based on the QEMU hot-unplug protocol documented here: https://lore.kernel.org/qemu-devel/20201204170939.1815522-3-imammedo@redhat.com/ Also define QEMU_CPUHP_STAT_EJECTED while we are at it. Cc: Laszlo Ersek Cc: Jordan Justen Cc: Ard Biesheuvel Cc: Igor Mammedov Cc: Boris Ostrovsky Cc: Aaron Young Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=3132 Signed-off-by: Ankur Arora --- Notes: I'm treating events (insert=1, fw_remove=1) below as invalid (return EFI_PROTOCOL_ERROR, which ends up as an assert), but I'm not sure that is correct: if ((CpuStatus & QEMU_CPUHP_STAT_INSERT) != 0) { // // The "insert" event guarantees the "enabled" status; plus it excludes - // the "remove" event. + // the "fw_remove" event. // if ((CpuStatus & QEMU_CPUHP_STAT_ENABLED) == 0 || - (CpuStatus & QEMU_CPUHP_STAT_REMOVE) != 0) { + (CpuStatus & QEMU_CPUHP_STAT_FW_REMOVE) != 0) { DEBUG ((DEBUG_ERROR, "%a: CurrentSelector=%u CpuStatus=0x%x: " "inconsistent CPU status\n", __FUNCTION__, CurrentSelector, CpuStatus)); QEMU's handling in cpu_hotplug_rd() can return both of these: cpu_hotplug_rd() { ... case ACPI_CPU_FLAGS_OFFSET_RW: /* pack and return is_* fields */ val |= cdev->cpu ? 1 : 0; val |= cdev->is_inserting ? 2 : 0; val |= cdev->is_removing ? 4 : 0; val |= cdev->fw_remove ? 16 : 0; ... } and I don't see any code that treats is_inserting and is_removing as exclusive. One specific case where this looks it might be a problem is if the user unplugs a CPU and right after that plugs it. As part of the unplug handling, the ACPI AML would, in the scan loop, asynchronously trigger the notify, which would do the OS unplug, set "fw_remove" and then call the SMI_CMD. The subsequent plug could then come and set the "insert" bit. Assuming what I'm describing could happen, I'm not sure what's the right handling: QEMU could treat these bits as exclusive and then OVMF could justifiably treat it as a protocol error? OvmfPkg/Include/IndustryStandard/QemuCpuHotplug.h | 2 ++ OvmfPkg/CpuHotplugSmm/QemuCpuhp.c | 29 +++++++++++++++++++---- 2 files changed, 26 insertions(+), 5 deletions(-) diff --git a/OvmfPkg/Include/IndustryStandard/QemuCpuHotplug.h b/OvmfPkg/Include/IndustryStandard/QemuCpuHotplug.h index a34a6d3fae61..692e3072598c 100644 --- a/OvmfPkg/Include/IndustryStandard/QemuCpuHotplug.h +++ b/OvmfPkg/Include/IndustryStandard/QemuCpuHotplug.h @@ -34,6 +34,8 @@ #define QEMU_CPUHP_STAT_ENABLED BIT0 #define QEMU_CPUHP_STAT_INSERT BIT1 #define QEMU_CPUHP_STAT_REMOVE BIT2 +#define QEMU_CPUHP_STAT_EJECTED BIT3 +#define QEMU_CPUHP_STAT_FW_REMOVE BIT4 #define QEMU_CPUHP_RW_CMD_DATA 0x8 diff --git a/OvmfPkg/CpuHotplugSmm/QemuCpuhp.c b/OvmfPkg/CpuHotplugSmm/QemuCpuhp.c index 8d4a6693c8d6..f871e50c377b 100644 --- a/OvmfPkg/CpuHotplugSmm/QemuCpuhp.c +++ b/OvmfPkg/CpuHotplugSmm/QemuCpuhp.c @@ -245,10 +245,10 @@ QemuCpuhpCollectApicIds ( if ((CpuStatus & QEMU_CPUHP_STAT_INSERT) != 0) { // // The "insert" event guarantees the "enabled" status; plus it excludes - // the "remove" event. + // the "fw_remove" event. // if ((CpuStatus & QEMU_CPUHP_STAT_ENABLED) == 0 || - (CpuStatus & QEMU_CPUHP_STAT_REMOVE) != 0) { + (CpuStatus & QEMU_CPUHP_STAT_FW_REMOVE) != 0) { DEBUG ((DEBUG_ERROR, "%a: CurrentSelector=%u CpuStatus=0x%x: " "inconsistent CPU status\n", __FUNCTION__, CurrentSelector, CpuStatus)); @@ -260,12 +260,31 @@ QemuCpuhpCollectApicIds ( ExtendIds = PluggedApicIds; ExtendCount = PluggedCount; - } else if ((CpuStatus & QEMU_CPUHP_STAT_REMOVE) != 0) { - DEBUG ((DEBUG_VERBOSE, "%a: CurrentSelector=%u: remove\n", __FUNCTION__, - CurrentSelector)); + } else if ((CpuStatus & QEMU_CPUHP_STAT_FW_REMOVE) != 0) { + // + // "fw_remove" event guarantees "enabled". + // + if ((CpuStatus & QEMU_CPUHP_STAT_ENABLED) == 0) { + DEBUG ((DEBUG_ERROR, "%a: CurrentSelector=%u CpuStatus=0x%x: " + "inconsistent CPU status\n", __FUNCTION__, CurrentSelector, + CpuStatus)); + return EFI_PROTOCOL_ERROR; + } + + DEBUG ((DEBUG_VERBOSE, "%a: CurrentSelector=%u: fw_remove\n", + __FUNCTION__, CurrentSelector)); ExtendIds = ToUnplugApicIds; ExtendCount = ToUnplugCount; + } else if ((CpuStatus & QEMU_CPUHP_STAT_REMOVE) != 0) { + // + // Let the OSPM deal with the "remove" event. + // + DEBUG ((DEBUG_INFO, "%a: CurrentSelector=%u: remove (ignored)\n", + __FUNCTION__, CurrentSelector)); + + CurrentSelector++; + continue; } else { DEBUG ((DEBUG_VERBOSE, "%a: CurrentSelector=%u: no event\n", __FUNCTION__, CurrentSelector)); -- 2.9.3