From: "Laszlo Ersek" <lersek@redhat.com>
To: devel@edk2.groups.io, yuanhao.xie@intel.com
Cc: Ray Ni <ray.ni@intel.com>, Eric Dong <eric.dong@intel.com>,
Rahul Kumar <rahul1.kumar@intel.com>,
Tom Lendacky <thomas.lendacky@amd.com>
Subject: Re: [edk2-devel] [Patch V3] UefiCpuPkg/MpInitLib: Wait for all APs to finish initialization
Date: Thu, 26 Oct 2023 15:36:22 +0200 [thread overview]
Message-ID: <5a82610a-4898-24e9-9687-be985d5cea36@redhat.com> (raw)
In-Reply-To: <20231025114216.2824-1-yuanhao.xie@intel.com>
On 10/25/23 13:42, Yuanhao Xie wrote:
> Aim:
> - To solve the assertion that checks if CpuMpData->FinishedCount
> equals (CpuMpData->CpuCount - 1). The assertion arises from a timing
> discrepancy between the BSP's completion of startup signal checks and
> the APs' incrementation of the FinishedCount.
> - This patch also ensures that "finished" reporting from the APs is as
> later as possible.
>
> More specifially:
>
> In the SwitchApContext() function, the BSP trigers
> the startup signal and check whether the APs have received it. After
> completing this check, the BSP then verifies if the FinishedCount is
> equal to CpuCount-1.
>
> On the AP side, upon receiving the startup signal, they invoke
> SwitchContextPerAp() and increase the FinishedCount to indicate their
> activation. However, even when all APs have received the startup signal,
> they might not have finished incrementing the FinishedCount. This timing
> gap results in the triggering of the assertion.
>
> Solution:
> Instead of assertion, use while loop to waits until all the APs have
> incremented the FinishedCount.
>
> Fixes: 964a4f032dcd
>
> Signed-off-by: Yuanhao Xie <yuanhao.xie@intel.com>
> Cc: Ray Ni <ray.ni@intel.com>
> Cc: Eric Dong <eric.dong@intel.com>
> Cc: Rahul Kumar <rahul1.kumar@intel.com>
> Cc: Tom Lendacky <thomas.lendacky@amd.com>
> ---
> UefiCpuPkg/Library/MpInitLib/MpLib.c | 9 +++++++--
> 1 file changed, 7 insertions(+), 2 deletions(-)
>
> diff --git a/UefiCpuPkg/Library/MpInitLib/MpLib.c b/UefiCpuPkg/Library/MpInitLib/MpLib.c
> index 6f1456cfe1..9a6ec5db5c 100644
> --- a/UefiCpuPkg/Library/MpInitLib/MpLib.c
> +++ b/UefiCpuPkg/Library/MpInitLib/MpLib.c
> @@ -913,8 +913,8 @@ DxeApEntryPoint (
> UINTN ProcessorNumber;
>
> GetProcessorNumber (CpuMpData, &ProcessorNumber);
> - InterlockedIncrement ((UINT32 *)&CpuMpData->FinishedCount);
> RestoreVolatileRegisters (&CpuMpData->CpuData[0].VolatileRegisters, FALSE);
> + InterlockedIncrement ((UINT32 *)&CpuMpData->FinishedCount);
> PlaceAPInMwaitLoopOrRunLoop (
> CpuMpData->ApLoopMode,
> CpuMpData->CpuData[ProcessorNumber].StartupApSignal,
> @@ -2201,7 +2201,12 @@ MpInitLibInitialize (
> // looping process there.
> //
> SwitchApContext (MpHandOff);
> - ASSERT (CpuMpData->FinishedCount == (CpuMpData->CpuCount - 1));
> + //
> + // Wait for all APs finished initialization
> + //
> + while (CpuMpData->FinishedCount < (CpuMpData->CpuCount - 1)) {
> + CpuPause ();
> + }
>
> //
> // Set Apstate as Idle, otherwise Aps cannot be waken-up again.
Reviewed-by: Laszlo Ersek <lersek@redhat.com>
The change is not testable using OVMF, because OVMF (intentionally) uses
ApLoopMode=ApInHltLoop, and in that case, neither hunk is reachable.
(Accordingly, the log message reports WaitLoopExecutionMode as zero.)
I've still regression-tested this change, with my usual configs:
- OVMF IA32 with SMM_REQUIRE, on q35
- OVMF IA32X64 with SMM_REQUIRE, on q35
- OVMF X64 without SMM_REQUIRE, on pc (i440fx)
The test goes like
- boot with 1 cold-plugged plus 2 more hot-pluggable VCPUs
- [*]
- hotplug 2 VCPUs
- [*]
- hot-unplug 2 VCPUs
- [*]
- poweroff
where [*] stands for:
- run efibootmgr, bound to each online VCPU in separation
- ACPI S3 suspend/resume
- run efibootmgr, bound to each online VCPU in separation
I used Fedora and RHEL guests.
So:
Regression-tested-by: Laszlo Ersek <lersek@redhat.com>
Thanks
Laszlo
-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.
View/Reply Online (#110101): https://edk2.groups.io/g/devel/message/110101
Mute This Topic: https://groups.io/mt/102176057/7686176
Group Owner: devel+owner@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/leave/12367111/7686176/1913456212/xyzzy [rebecca@openfw.io]
-=-=-=-=-=-=-=-=-=-=-=-
next prev parent reply other threads:[~2023-10-26 13:36 UTC|newest]
Thread overview: 4+ messages / expand[flat|nested] mbox.gz Atom feed top
2023-10-25 11:42 [edk2-devel] [Patch V3] UefiCpuPkg/MpInitLib: Wait for all APs to finish initialization Yuanhao Xie
2023-10-26 13:36 ` Laszlo Ersek [this message]
2023-10-26 13:41 ` Laszlo Ersek
2023-10-26 18:00 ` Laszlo Ersek
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-list from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=5a82610a-4898-24e9-9687-be985d5cea36@redhat.com \
--to=devel@edk2.groups.io \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox