From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received-SPF: Pass (sender SPF authorized) identity=mailfrom; client-ip=192.55.52.120; helo=mga04.intel.com; envelope-from=ruiyu.ni@intel.com; receiver=edk2-devel@lists.01.org Received: from mga04.intel.com (mga04.intel.com [192.55.52.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ml01.01.org (Postfix) with ESMTPS id 6096021BADAB2 for ; Wed, 18 Jul 2018 21:40:49 -0700 (PDT) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga001.fm.intel.com ([10.253.24.23]) by fmsmga104.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 18 Jul 2018 21:40:49 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.51,373,1526367600"; d="scan'208";a="73597420" Received: from fmsmsx104.amr.corp.intel.com ([10.18.124.202]) by fmsmga001.fm.intel.com with ESMTP; 18 Jul 2018 21:40:37 -0700 Received: from fmsmsx123.amr.corp.intel.com (10.18.125.38) by fmsmsx104.amr.corp.intel.com (10.18.124.202) with Microsoft SMTP Server (TLS) id 14.3.319.2; Wed, 18 Jul 2018 21:40:37 -0700 Received: from shsmsx102.ccr.corp.intel.com (10.239.4.154) by fmsmsx123.amr.corp.intel.com (10.18.125.38) with Microsoft SMTP Server (TLS) id 14.3.319.2; Wed, 18 Jul 2018 21:40:37 -0700 Received: from shsmsx104.ccr.corp.intel.com ([169.254.5.81]) by shsmsx102.ccr.corp.intel.com ([169.254.2.124]) with mapi id 14.03.0319.002; Thu, 19 Jul 2018 12:40:34 +0800 From: "Ni, Ruiyu" To: "Dong, Eric" , "edk2-devel@lists.01.org" CC: Laszlo Ersek Thread-Topic: UefiCpuPkg/MpInitLib: Fix S3 resume hang issue. Thread-Index: AQHUHlWnOPZzLVfa+UexUbYPfWWLjqSV+LXQ Date: Thu, 19 Jul 2018 04:40:34 +0000 Message-ID: <734D49CCEBEEF84792F5B80ED585239D5BD7F235@SHSMSX104.ccr.corp.intel.com> References: <20180718051009.19748-1-eric.dong@intel.com> In-Reply-To: <20180718051009.19748-1-eric.dong@intel.com> Accept-Language: en-US, zh-CN X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [10.239.127.40] MIME-Version: 1.0 Subject: Re: UefiCpuPkg/MpInitLib: Fix S3 resume hang issue. X-BeenThere: edk2-devel@lists.01.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: EDK II Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 19 Jul 2018 04:40:49 -0000 Content-Language: en-US Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Reviewed-by: Ruiyu Ni Thanks/Ray > -----Original Message----- > From: Dong, Eric > Sent: Wednesday, July 18, 2018 1:10 PM > To: edk2-devel@lists.01.org > Cc: Laszlo Ersek ; Ni, Ruiyu > Subject: UefiCpuPkg/MpInitLib: Fix S3 resume hang issue. >=20 > When resume from S3 and CPU loop mode is MWait mode, if driver calls APs > to do task at EndOfPei point, the APs can't been wake up and bios hang at > that point. >=20 > The root cause is PiSmmCpuDxeSmm driver wakes up APs with HLT mode > during S3 resume phase to do SMM relocation. > After this task, PiSmmCpuDxeSmm driver not restore APs context which > make CpuMpPei driver saved wake up buffer not works. >=20 > The solution for this issue is let CpuMpPei driver hook S3SmmInitDone ppi > notification. In this notify function, it check whether Cpu Loop mode is = not > HLT mode. If yes, CpuMpPei driver will set a flag to force BSP use INIT-S= IPI - > SIPI command to wake up the APs. >=20 > Cc: Laszlo Ersek > Cc: Ruiyu Ni > Contributed-under: TianoCore Contribution Agreement 1.1 > Signed-off-by: Eric Dong > --- > UefiCpuPkg/Library/MpInitLib/MpLib.c | 16 ++++- > UefiCpuPkg/Library/MpInitLib/MpLib.h | 9 +++ > UefiCpuPkg/Library/MpInitLib/PeiMpInitLib.inf | 4 ++ > UefiCpuPkg/Library/MpInitLib/PeiMpLib.c | 89 > +++++++++++++++++++++++++++ > 4 files changed, 116 insertions(+), 2 deletions(-) >=20 > diff --git a/UefiCpuPkg/Library/MpInitLib/MpLib.c > b/UefiCpuPkg/Library/MpInitLib/MpLib.c > index 722db2a01f..e5c701ddeb 100644 > --- a/UefiCpuPkg/Library/MpInitLib/MpLib.c > +++ b/UefiCpuPkg/Library/MpInitLib/MpLib.c > @@ -985,13 +985,15 @@ WakeUpAP ( > CpuMpData->FinishedCount =3D 0; > ResetVectorRequired =3D FALSE; >=20 > - if (CpuMpData->ApLoopMode =3D=3D ApInHltLoop || > + if (CpuMpData->WakeUpByInitSipiSipi || > CpuMpData->InitFlag !=3D ApInitDone) { > ResetVectorRequired =3D TRUE; > AllocateResetVector (CpuMpData); > FillExchangeInfoData (CpuMpData); > SaveLocalApicTimerSetting (CpuMpData); > - } else if (CpuMpData->ApLoopMode =3D=3D ApInMwaitLoop) { > + } > + > + if (CpuMpData->ApLoopMode =3D=3D ApInMwaitLoop) { > // > // Get AP target C-state each time when waking up AP, > // for it maybe updated by platform again @@ -1076,6 +1078,13 @@ > WakeUpAP ( > if (ResetVectorRequired) { > FreeResetVector (CpuMpData); > } > + > + // > + // After one round of Wakeup Ap actions, need to re-sync ApLoopMode > + with // WakeUpByInitSipiSipi flag. WakeUpByInitSipiSipi flag maybe > + changed by // S3SmmInitDone Ppi. > + // > + CpuMpData->WakeUpByInitSipiSipi =3D (CpuMpData->ApLoopMode =3D=3D > + ApInHltLoop); > } >=20 > /** > @@ -1648,6 +1657,9 @@ MpInitLibInitialize ( > // > CpuMpData->ApLoopMode =3D ApLoopMode; > DEBUG ((DEBUG_INFO, "AP Loop Mode is %d\n", CpuMpData- > >ApLoopMode)); > + > + CpuMpData->WakeUpByInitSipiSipi =3D (CpuMpData->ApLoopMode =3D=3D > + ApInHltLoop); > + > // > // Set up APs wakeup signal buffer > // > diff --git a/UefiCpuPkg/Library/MpInitLib/MpLib.h > b/UefiCpuPkg/Library/MpInitLib/MpLib.h > index 6958080ac1..9d0b866d09 100644 > --- a/UefiCpuPkg/Library/MpInitLib/MpLib.h > +++ b/UefiCpuPkg/Library/MpInitLib/MpLib.h > @@ -250,6 +250,15 @@ struct _CPU_MP_DATA { > UINT32 ProcessorFlags; > UINT64 MicrocodeDataAddress; > UINT32 MicrocodeRevision; > + > + // > + // Whether need to use Init-Sipi-Sipi to wake up the APs. > + // Two cases need to set this value to TRUE. One is in HLT // loop > + mode, the other is resume from S3 which loop mode // will be hardcode > + change to HLT mode by PiSmmCpuDxeSmm // driver. > + // > + BOOLEAN WakeUpByInitSipiSipi; > }; >=20 > extern EFI_GUID mCpuInitMpLibHobGuid; > diff --git a/UefiCpuPkg/Library/MpInitLib/PeiMpInitLib.inf > b/UefiCpuPkg/Library/MpInitLib/PeiMpInitLib.inf > index fa84e392af..43a3b3b036 100644 > --- a/UefiCpuPkg/Library/MpInitLib/PeiMpInitLib.inf > +++ b/UefiCpuPkg/Library/MpInitLib/PeiMpInitLib.inf > @@ -44,6 +44,7 @@ > [Packages] > MdePkg/MdePkg.dec > UefiCpuPkg/UefiCpuPkg.dec > + MdeModulePkg/MdeModulePkg.dec >=20 > [LibraryClasses] > BaseLib > @@ -54,6 +55,7 @@ > CpuLib > UefiCpuLib > SynchronizationLib > + PeiServicesLib >=20 > [Pcd] > gUefiCpuPkgTokenSpaceGuid.PcdCpuMaxLogicalProcessorNumber ## > CONSUMES > @@ -64,3 +66,5 @@ > gUefiCpuPkgTokenSpaceGuid.PcdCpuApLoopMode ## > CONSUMES > gUefiCpuPkgTokenSpaceGuid.PcdCpuApTargetCstate ## > SOMETIMES_CONSUMES >=20 > +[Guids] > + gEdkiiS3SmmInitDoneGuid > \ No newline at end of file > diff --git a/UefiCpuPkg/Library/MpInitLib/PeiMpLib.c > b/UefiCpuPkg/Library/MpInitLib/PeiMpLib.c > index 92f28681e4..06d966b227 100644 > --- a/UefiCpuPkg/Library/MpInitLib/PeiMpLib.c > +++ b/UefiCpuPkg/Library/MpInitLib/PeiMpLib.c > @@ -13,6 +13,87 @@ > **/ >=20 > #include "MpLib.h" > +#include > +#include > + > +/** > + S3 SMM Init Done notification function. > + > + @param PeiServices Indirect reference to the PEI Services Table. > + @param NotifyDesc Address of the notification descriptor data > structure. > + @param InvokePpi Address of the PPI that was invoked. > + > + @retval EFI_SUCCESS The function completes successfully. > + > +**/ > +EFI_STATUS > +EFIAPI > +NotifyOnS3SmmInitDonePpi ( > + IN EFI_PEI_SERVICES **PeiServices, > + IN EFI_PEI_NOTIFY_DESCRIPTOR *NotifyDesc, > + IN VOID *InvokePpi > + ); > + > + > +// > +// Global function > +// > +EFI_PEI_NOTIFY_DESCRIPTOR mS3SmmInitDoneNotifyDesc =3D { > + EFI_PEI_PPI_DESCRIPTOR_NOTIFY_CALLBACK | > +EFI_PEI_PPI_DESCRIPTOR_TERMINATE_LIST, > + &gEdkiiS3SmmInitDoneGuid, > + NotifyOnS3SmmInitDonePpi > +}; > + > +/** > + The function prototype for invoking a function on an Application Proce= ssor. > + > + This definition is used by the UEFI MP Serices Protocol, and the PI > + SMM System Table. > + > + @param[in,out] Buffer The pointer to private data buffer. > +**/ > +VOID > +EmptyApProcedure ( > + IN OUT VOID * Buffer > + ) > +{ > +} > + > +/** > + S3 SMM Init Done notification function. > + > + @param PeiServices Indirect reference to the PEI Services Table. > + @param NotifyDesc Address of the notification descriptor data > structure. > + @param InvokePpi Address of the PPI that was invoked. > + > + @retval EFI_SUCCESS The function completes successfully. > + > +**/ > +EFI_STATUS > +EFIAPI > +NotifyOnS3SmmInitDonePpi ( > + IN EFI_PEI_SERVICES **PeiServices, > + IN EFI_PEI_NOTIFY_DESCRIPTOR *NotifyDesc, > + IN VOID *InvokePpi > + ) > +{ > + CPU_MP_DATA *CpuMpData; > + > + CpuMpData =3D GetCpuMpData (); > + > + // > + // PiSmmCpuDxeSmm driver hardcode change the loop mode to HLT mode. > + // So in this notify function, code need to check the current loop > + // mode, if it is not HLT mode, code need to change loop mode back // > + to the original mode. > + // > + if (CpuMpData->ApLoopMode !=3D ApInHltLoop) { > + CpuMpData->WakeUpByInitSipiSipi =3D TRUE; } > + > + return EFI_SUCCESS; > +} > + >=20 > /** > Enable Debug Agent to support source debugging on AP function. > @@ -240,7 +321,15 @@ InitMpGlobalData ( > IN CPU_MP_DATA *CpuMpData > ) > { > + EFI_STATUS Status; > + > SaveCpuMpData (CpuMpData); > + > + /// > + /// Install Notify > + /// > + Status =3D PeiServicesNotifyPpi (&mS3SmmInitDoneNotifyDesc); > + ASSERT_EFI_ERROR (Status); > } >=20 > /** > -- > 2.15.0.windows.1