From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-pj1-f54.google.com (mail-pj1-f54.google.com [209.85.216.54]) by mx.groups.io with SMTP id smtpd.web12.5692.1639626406573458024 for ; Wed, 15 Dec 2021 19:46:46 -0800 Authentication-Results: mx.groups.io; dkim=pass header.i=@nuviainc-com.20210112.gappssmtp.com header.s=20210112 header.b=64PHDK+t; spf=pass (domain: nuviainc.com, ip: 209.85.216.54, mailfrom: rebecca@nuviainc.com) Received: by mail-pj1-f54.google.com with SMTP id a11-20020a17090a854b00b001b11aae38d6so1872348pjw.2 for ; Wed, 15 Dec 2021 19:46:46 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nuviainc-com.20210112.gappssmtp.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=Y1FdVpu4iw4QZ19lMmRgbn3I2/Cp5Ti27OYijeQqMjw=; b=64PHDK+trmcvhV5iN5brV97SY4VN+mh7ablIhiUOWfMcPJQtMYvpEwuOCCA0mN5tau +2LyBvVGpkfLJsYxjuTU4/VdPNq+vmKGoGOsDCzJoenmuQ+nBRZBNxRbvPEKps1s1FoG JeX+ByZ+VhMpsES4NXGheI5Vgpwvel4W5MB5G5axayluCvUo9TLwrCAVrp5jZrmr8ni9 V8wJorsx6n4JPikWul/pGoWxhQxIZzgoULKGSFOg5iSUIDDxvlCZ7LNU8VA/Vgsoj4AG MH5w4Vrohw/mZaIcGir/d/4jCc0SeIY94K3e8b+B/fvPv44PvYIive86Qh136PCbltQy 6CCQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=Y1FdVpu4iw4QZ19lMmRgbn3I2/Cp5Ti27OYijeQqMjw=; b=SeZPNlfODDvt8ZHKYCXolkHCYsSQPDuZxG6c3+ckMpItPwsBY34yqppb2IGfawi1GK bhNi7JfpFnQo/ILCh1rfi50pJktRjAZR88Uka+N4FJUibu42LmTLZidvTHnNr77+kFrL 06ccJvV0aznBpN5lJAOXTgd/Sx0W7xYUZGEw7QTQb8wPQqGX9EkXyzk5ms79bTaZtV4h ZKnzSAJsDOruIBgqBvpOJNwDUN0Q2EFTSPePfbJdgjYFYU8ZIa4bxfEk0ERFXYvP843Z IRwvp4D3AdRLGFHJihd1aM2SkIISHpZYN16xY/KSCbfE9Lw71sL8rXYIMd+R2GkobuVJ DSvw== X-Gm-Message-State: AOAM530H0hFWbFfmr2mrHKhjOt7tC1CCcqI++6/Zb1svTikwnRJ2OViB fbE3I7OIXpU5kYRHvYkst5LDaE07cnzsLYVMKORlnUvdeLbAt+5ym8kMinyesSPM9oAZq8Szbwc mUMH9/UJBIi7Sn5SP3jZumQbqLBR/H/78C0jg5n/dodZA6oJvmso16/z5u0wlRmow2NobJA== X-Google-Smtp-Source: ABdhPJy8WYJkdkV8hfdu44qZJp5PT9d+Pwve5eERUIFS/yG0ZEjSdBgV7m58z12Bub9JkXYYwAPXyQ== X-Received: by 2002:a17:903:2446:b0:148:a2f7:9d5d with SMTP id l6-20020a170903244600b00148a2f79d5dmr7613783pls.124.1639626404227; Wed, 15 Dec 2021 19:46:44 -0800 (PST) Return-Path: Received: from linbox.ba.nuviainc.com ([2601:681:4300:69e:9e7b:efff:fe2b:884c]) by smtp.gmail.com with ESMTPSA id c21sm4124054pfl.138.2021.12.15.19.46.42 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 15 Dec 2021 19:46:43 -0800 (PST) From: "Rebecca Cran" To: devel@edk2.groups.io, Ard Biesheuvel , Gerd Hoffmann , Samer El-Haj-Mahmoud , Leif Lindholm , Sami Mujawar Cc: Rebecca Cran Subject: [PATCH v3 2/4] ArmPkg: Add Library/MpInitLib to support EFI_MP_SERVICES_PROTOCOL Date: Wed, 15 Dec 2021 20:46:32 -0700 Message-Id: <20211216034634.15468-3-rebecca@nuviainc.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20211216034634.15468-1-rebecca@nuviainc.com> References: <20211216034634.15468-1-rebecca@nuviainc.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Add support for EFI_MP_SERVICES_PROTOCOL during the DXE phase under AArch64. PSCI_CPU_ON is called to power on the core, the supplied procedure is executed and PSCI_CPU_OFF is called to power off the core. Minimal setup is done before calling the supplied procedure: for example the MMU and caches are not enabled. Signed-off-by: Rebecca Cran --- ArmPkg/ArmPkg.dec | 4 + ArmPkg/ArmPkg.dsc | 4 + ArmPkg/Include/Library/MpInitLib.h | 362 +++++ ArmPkg/Library/MpInitLib/AArch64/MpFuncs.S | 65 + ArmPkg/Library/MpInitLib/DxeMpInitLib.inf | 52 + ArmPkg/Library/MpInitLib/DxeMpLib.c | 1477 ++++++++++++++++++++ ArmPkg/Library/MpInitLib/InternalMpInitLib.h | 357 +++++ 7 files changed, 2321 insertions(+) diff --git a/ArmPkg/ArmPkg.dec b/ArmPkg/ArmPkg.dec index 9da1bbc9f216..363eddc57393 100644 --- a/ArmPkg/ArmPkg.dec +++ b/ArmPkg/ArmPkg.dec @@ -75,6 +75,10 @@ # DefaultExceptionHandlerLib|Include/Library/DefaultExceptionHandlerLib.h + ## @libraryclass Provides a MP Services interface. + # + MpInitLib|Include/Library/MpInitLib.h + ## @libraryclass Provides an interface to query miscellaneous OEM # information. # diff --git a/ArmPkg/ArmPkg.dsc b/ArmPkg/ArmPkg.dsc index 59fd8f295d4f..6e053d6ee31d 100644 --- a/ArmPkg/ArmPkg.dsc +++ b/ArmPkg/ArmPkg.dsc @@ -100,6 +100,9 @@ PeiServicesLib|MdePkg/Library/PeiServicesLib/PeiServicesLib.inf PeiServicesTablePointerLib|MdePkg/Library/PeiServicesTablePointerLib/PeiServicesTablePointerLib.inf +[LibraryClasses.AARCH64] + MpInitLib|ArmPkg/Library/MpInitLib/DxeMpInitLib.inf + [LibraryClasses.ARM, LibraryClasses.AARCH64] NULL|ArmPkg/Library/CompilerIntrinsicsLib/CompilerIntrinsicsLib.inf @@ -163,6 +166,7 @@ [Components.AARCH64] ArmPkg/Drivers/MmCommunicationDxe/MmCommunication.inf ArmPkg/Library/ArmMmuLib/ArmMmuPeiLib.inf + ArmPkg/Library/MpInitLib/DxeMpInitLib.inf [Components.AARCH64, Components.ARM] ArmPkg/Library/StandaloneMmMmuLib/ArmMmuStandaloneMmLib.inf diff --git a/ArmPkg/Include/Library/MpInitLib.h b/ArmPkg/Include/Library/MpInitLib.h new file mode 100644 index 000000000000..582bb788fd59 --- /dev/null +++ b/ArmPkg/Include/Library/MpInitLib.h @@ -0,0 +1,362 @@ +/** @file + +Copyright (c) 2021, NUVIA Inc. All rights reserved.
+Copyright (c) 2006 - 2011, Intel Corporation. All rights reserved.
+Portions copyright (c) 2011, Apple Inc. All rights reserved. + +SPDX-License-Identifier: BSD-2-Clause-Patent + +**/ + +#ifndef MP_INITLIB_H_ +#define MP_INITLIB_H_ + +#include +#include +#include +#include +#include + +/** + This service retrieves the number of logical processor in the platform + and the number of those logical processors that are enabled on this boot. + This service may only be called from the BSP. + + @param[in] This A pointer to the + EFI_MP_SERVICES_PROTOCOL instance. + @param[out] NumberOfProcessors Pointer to the total number of logical + processors in the system, including + the BSP and disabled APs. + @param[out] NumberOfEnabledProcessors Pointer to the number of enabled + logical processors that exist in the + system, including the BSP. + + @retval EFI_SUCCESS The number of logical processors and enabled + logical processors was retrieved. + @retval EFI_DEVICE_ERROR The calling processor is an AP. + @retval EFI_INVALID_PARAMETER NumberOfProcessors is NULL. + @retval EFI_INVALID_PARAMETER NumberOfEnabledProcessors is NULL. + +**/ +EFI_STATUS +EFIAPI +MpInitLibGetNumberOfProcessors ( + IN EFI_MP_SERVICES_PROTOCOL *This, + OUT UINTN *NumberOfProcessors, + OUT UINTN *NumberOfEnabledProcessors + ); + +/** + Gets detailed MP-related information on the requested processor at the + instant this call is made. This service may only be called from the BSP. + + @param[in] This A pointer to the EFI_MP_SERVICES_PROTOCOL + instance. + @param[in] ProcessorIndex The index of the processor. + @param[out] ProcessorInfoBuffer A pointer to the buffer where information + for the requested processor is deposited. + + @retval EFI_SUCCESS Processor information was returned. + @retval EFI_DEVICE_ERROR The calling processor is an AP. + @retval EFI_INVALID_PARAMETER ProcessorInfoBuffer is NULL. + @retval EFI_NOT_FOUND The processor with the handle specified by + ProcessorNumber does not exist in the platform. + +**/ +EFI_STATUS +EFIAPI +MpInitLibGetProcessorInfo ( + IN EFI_MP_SERVICES_PROTOCOL *This, + IN UINTN ProcessorIndex, + OUT EFI_PROCESSOR_INFORMATION *ProcessorInfoBuffer + ); + +/** + This service executes a caller provided function on all enabled APs. APs can + run either simultaneously or one at a time in sequence. This service supports + both blocking and non-blocking requests. The non-blocking requests use EFI + events so the BSP can detect when the APs have finished. This service may only + be called from the BSP. + + @param[in] This A pointer to the EFI_MP_SERVICES_PROTOCOL + instance. + @param[in] Procedure A pointer to the function to be run on + enabled APs of the system. See type + EFI_AP_PROCEDURE. + @param[in] SingleThread If TRUE, then all the enabled APs execute + the function specified by Procedure one by + one, in ascending order of processor + handle number. If FALSE, then all the + enabled APs execute the function specified + by Procedure simultaneously. + @param[in] WaitEvent The event created by the caller with + CreateEvent() service. If it is NULL, + then execute in blocking mode. BSP waits + until all APs finish or + TimeoutInMicroseconds expires. If it's + not NULL, then execute in non-blocking + mode. BSP requests the function specified + by Procedure to be started on all the + enabled APs, and go on executing + immediately. If all return from Procedure, + or TimeoutInMicroseconds expires, this + event is signaled. The BSP can use the + CheckEvent() or WaitForEvent() + services to check the state of event. Type + EFI_EVENT is defined in CreateEvent() in + the Unified Extensible Firmware Interface + Specification. + @param[in] TimeoutInMicroseconds Indicates the time limit in microseconds + for APs to return from Procedure, either + for blocking or non-blocking mode. Zero + means infinity. If the timeout expires + before all APs return from Procedure, then + Procedure on the failed APs is terminated. + All enabled APs are available for next + function assigned by + EFI_MP_SERVICES_PROTOCOL.StartupAllAPs() + or EFI_MP_SERVICES_PROTOCOL.StartupThisAP(). + If the timeout expires in blocking mode, + BSP returns EFI_TIMEOUT. If the timeout + expires in non-blocking mode, WaitEvent + is signaled with SignalEvent(). + @param[in] ProcedureArgument The parameter passed into Procedure for + all APs. + @param[out] FailedCpuList If NULL, this parameter is ignored. + Otherwise, if all APs finish successfully, + then its content is set to NULL. If not + all APs finish before timeout expires, + then its content is set to address of the + buffer holding handle numbers of the + failed APs. + The buffer is allocated by MP Service + Protocol, and it's the caller's + responsibility to free the buffer with + FreePool() service. + In blocking mode, it is ready for + consumption when the call returns. In + non-blocking mode, it is ready when + WaitEvent is signaled. The list of failed + CPU is terminated by END_OF_CPU_LIST. + + @retval EFI_SUCCESS In blocking mode, all APs have finished before + the timeout expired. + @retval EFI_SUCCESS In non-blocking mode, function has been + dispatched to all enabled APs. + @retval EFI_UNSUPPORTED A non-blocking mode request was made after the + UEFI event EFI_EVENT_GROUP_READY_TO_BOOT was + signaled. + @retval EFI_DEVICE_ERROR Caller processor is AP. + @retval EFI_NOT_STARTED No enabled APs exist in the system. + @retval EFI_NOT_READY Any enabled APs are busy. + @retval EFI_TIMEOUT In blocking mode, the timeout expired before + all enabled APs have finished. + @retval EFI_INVALID_PARAMETER Procedure is NULL. + +**/ +EFI_STATUS +EFIAPI +MpInitLibStartupAllAPs ( + IN EFI_MP_SERVICES_PROTOCOL *This, + IN EFI_AP_PROCEDURE Procedure, + IN BOOLEAN SingleThread, + IN EFI_EVENT WaitEvent OPTIONAL, + IN UINTN TimeoutInMicroseconds, + IN VOID *ProcedureArgument OPTIONAL, + OUT UINTN **FailedCpuList OPTIONAL + ); + +/** + This service lets the caller get one enabled AP to execute a caller-provided + function. The caller can request the BSP to either wait for the completion + of the AP or just proceed with the next task by using the EFI event mechanism. + See EFI_MP_SERVICES_PROTOCOL.StartupAllAPs() for more details on non-blocking + execution support. This service may only be called from the BSP. + + @param[in] This A pointer to the EFI_MP_SERVICES_PROTOCOL + instance. + @param[in] Procedure A pointer to the function to be run on + enabled APs of the system. See type + EFI_AP_PROCEDURE. + @param[in] ProcessorNumber The handle number of the AP. The range is + from 0 to the total number of logical + processors minus 1. The total number of + logical processors can be retrieved by + EFI_MP_SERVICES_PROTOCOL.GetNumberOfProcessors(). + @param[in] WaitEvent The event created by the caller with CreateEvent() + service. If it is NULL, then execute in + blocking mode. BSP waits until all APs finish + or TimeoutInMicroseconds expires. If it's + not NULL, then execute in non-blocking mode. + BSP requests the function specified by + Procedure to be started on all the enabled + APs, and go on executing immediately. If + all return from Procedure or TimeoutInMicroseconds + expires, this event is signaled. The BSP + can use the CheckEvent() or WaitForEvent() + services to check the state of event. Type + EFI_EVENT is defined in CreateEvent() in + the Unified Extensible Firmware Interface + Specification. + @param[in] TimeoutInMicroseconds Indicates the time limit in microseconds for + APs to return from Procedure, either for + blocking or non-blocking mode. Zero means + infinity. If the timeout expires before + all APs return from Procedure, then Procedure + on the failed APs is terminated. All enabled + APs are available for next function assigned + by EFI_MP_SERVICES_PROTOCOL.StartupAllAPs() + or EFI_MP_SERVICES_PROTOCOL.StartupThisAP(). + If the timeout expires in blocking mode, + BSP returns EFI_TIMEOUT. If the timeout + expires in non-blocking mode, WaitEvent + is signaled with SignalEvent(). + @param[in] ProcedureArgument The parameter passed into Procedure for + all APs. + @param[out] Finished If NULL, this parameter is ignored. In + blocking mode, this parameter is ignored. + In non-blocking mode, if AP returns from + Procedure before the timeout expires, its + content is set to TRUE. Otherwise, the + value is set to FALSE. The caller can + determine if the AP returned from Procedure + by evaluating this value. + + @retval EFI_SUCCESS In blocking mode, specified AP finished before + the timeout expires. + @retval EFI_SUCCESS In non-blocking mode, the function has been + dispatched to specified AP. + @retval EFI_UNSUPPORTED A non-blocking mode request was made after the + UEFI event EFI_EVENT_GROUP_READY_TO_BOOT was + signaled. + @retval EFI_DEVICE_ERROR The calling processor is an AP. + @retval EFI_TIMEOUT In blocking mode, the timeout expired before + the specified AP has finished. + @retval EFI_NOT_READY The specified AP is busy. + @retval EFI_NOT_FOUND The processor with the handle specified by + ProcessorNumber does not exist. + @retval EFI_INVALID_PARAMETER ProcessorNumber specifies the BSP or disabled AP. + @retval EFI_INVALID_PARAMETER Procedure is NULL. + +**/ +EFI_STATUS +EFIAPI +MpInitLibStartupThisAP ( + IN EFI_MP_SERVICES_PROTOCOL *This, + IN EFI_AP_PROCEDURE Procedure, + IN UINTN ProcessorNumber, + IN EFI_EVENT WaitEvent OPTIONAL, + IN UINTN TimeoutInMicroseconds, + IN VOID *ProcedureArgument OPTIONAL, + OUT BOOLEAN *Finished OPTIONAL + ); + +/** + This service switches the requested AP to be the BSP from that point onward. + This service changes the BSP for all purposes. This call can only be + performed by the current BSP. + + @param[in] This A pointer to the EFI_MP_SERVICES_PROTOCOL instance. + @param[in] ProcessorNumber The handle number of AP that is to become the new + BSP. The range is from 0 to the total number of + logical processors minus 1. The total number of + logical processors can be retrieved by + EFI_MP_SERVICES_PROTOCOL.GetNumberOfProcessors(). + @param[in] EnableOldBSP If TRUE, then the old BSP will be listed as an + enabled AP. Otherwise, it will be disabled. + + @retval EFI_SUCCESS BSP successfully switched. + @retval EFI_UNSUPPORTED Switching the BSP cannot be completed prior to + this service returning. + @retval EFI_UNSUPPORTED Switching the BSP is not supported. + @retval EFI_SUCCESS The calling processor is an AP. + @retval EFI_NOT_FOUND The processor with the handle specified by + ProcessorNumber does not exist. + @retval EFI_INVALID_PARAMETER ProcessorNumber specifies the current BSP or + a disabled AP. + @retval EFI_NOT_READY The specified AP is busy. + +**/ +EFI_STATUS +EFIAPI +MpInitLibSwitchBSP ( + IN EFI_MP_SERVICES_PROTOCOL *This, + IN UINTN ProcessorNumber, + IN BOOLEAN EnableOldBSP + ); + +/** + This service lets the caller enable or disable an AP from this point onward. + This service may only be called from the BSP. + + @param[in] This A pointer to the EFI_MP_SERVICES_PROTOCOL instance. + @param[in] ProcessorNumber The handle number of AP that is to become the new + BSP. The range is from 0 to the total number of + logical processors minus 1. The total number of + logical processors can be retrieved by + EFI_MP_SERVICES_PROTOCOL.GetNumberOfProcessors(). + @param[in] EnableAP Specifies the new state for the processor for + enabled, FALSE for disabled. + @param[in] HealthFlag If not NULL, a pointer to a value that specifies + the new health status of the AP. This flag + corresponds to StatusFlag defined in + EFI_MP_SERVICES_PROTOCOL.GetProcessorInfo(). Only + the PROCESSOR_HEALTH_STATUS_BIT is used. All other + bits are ignored. If it is NULL, this parameter + is ignored. + + @retval EFI_SUCCESS The specified AP was enabled or disabled successfully. + @retval EFI_UNSUPPORTED Enabling or disabling an AP cannot be completed + prior to this service returning. + @retval EFI_UNSUPPORTED Enabling or disabling an AP is not supported. + @retval EFI_DEVICE_ERROR The calling processor is an AP. + @retval EFI_NOT_FOUND Processor with the handle specified by ProcessorNumber + does not exist. + @retval EFI_INVALID_PARAMETER ProcessorNumber specifies the BSP. + +**/ +EFI_STATUS +EFIAPI +MpInitLibEnableDisableAP ( + IN EFI_MP_SERVICES_PROTOCOL *This, + IN UINTN ProcessorNumber, + IN BOOLEAN EnableAP, + IN UINT32 *HealthFlag OPTIONAL + ); + +/** + This return the handle number for the calling processor. This service may be + called from the BSP and APs. + + @param[in] This A pointer to the EFI_MP_SERVICES_PROTOCOL instance. + @param[out] ProcessorNumber The handle number of AP that is to become the new + BSP. The range is from 0 to the total number of + logical processors minus 1. The total number of + logical processors can be retrieved by + EFI_MP_SERVICES_PROTOCOL.GetNumberOfProcessors(). + + @retval EFI_SUCCESS The current processor handle number was returned + in ProcessorNumber. + @retval EFI_INVALID_PARAMETER ProcessorNumber is NULL. + +**/ +EFI_STATUS +EFIAPI +MpInitLibWhoAmI ( + IN EFI_MP_SERVICES_PROTOCOL *This, + OUT UINTN *ProcessorNumber + ); + +/** Initializes the MP Services system data + + @param NumberOfProcessors The number of processors, both BSP and AP. + @param CpuInfo CPU information gathered earlier during boot. + +**/ +VOID +MpInitLibInitialize ( + IN UINTN NumberOfProcessors, + IN ARM_PROCESSOR_TABLE *CpuInfo + ); + +#endif /* MP_INITLIB_H_ */ diff --git a/ArmPkg/Library/MpInitLib/AArch64/MpFuncs.S b/ArmPkg/Library/MpInitLib/AArch64/MpFuncs.S new file mode 100644 index 000000000000..8f7019a1c62c --- /dev/null +++ b/ArmPkg/Library/MpInitLib/AArch64/MpFuncs.S @@ -0,0 +1,65 @@ +#=============================================================================== +# Copyright (c) 2021 NUVIA Inc. All rights reserved. +# +# SPDX-License-Identifier: BSD-2-Clause-Patent +#=============================================================================== + +.text +.align 3 + +#include +#include + +#include "InternalMpInitLib.h" + +GCC_ASM_IMPORT (gApStacksBase) +GCC_ASM_IMPORT (gProcessorIDs) +GCC_ASM_IMPORT (ApProcedure) +GCC_ASM_IMPORT (gApStackSize) + +GCC_ASM_EXPORT (ApEntryPoint) + +StartupAddr: .8byte ASM_PFX(ApProcedure) + +// Entry-point for the AP +// VOID +// ApEntryPoint ( +// VOID +// ); +ASM_PFX(ApEntryPoint): + mrs x0, mpidr_el1 + // Mask the non-affinity bits + ldr x1, =0xff00ffffff + and x0, x0, x1 + ldr x1, gProcessorIDs + mov x2, 0 // x2 = processor index + mov x3, 0 // x3 = address offset + +// Find index in gProcessorIDs for current processor +1: + ldr x4, [x1, x3] // x4 = gProcessorIDs + x3 + ldr x5, =0xffffffffff + cmp x4, x5 // check if we've reached the end of gProcessorIDs + beq ProcessorNotFound + add x3, x3, 8 // x3 += sizeof (*gProcessorIDs) + add x2, x2, 1 // x2++ + cmp x0, x4 // if mpidr_el1 != *(gProcessorIDs + x3) then loop + bne 1b + sub x2, x2, 1 + +// Calculate stack address + // x2 contains the index for the current processor + ldr x0, gApStacksBase + ldr x1, gApStackSize + mul x3, x2, x1 // x3 = ProcessorIndex * gApStackSize + add x4, x0, x3 // x4 = gApStacksBase + x3 + add sp, x4, x1 // sp = x4 + gApStackSize + + ldr x0, StartupAddr // ASM_PFX(ApProcedure) + blr x0 // doesn't return + +ProcessorNotFound: +// Turn off the processor + MOV32 (w0, ARM_SMC_ID_PSCI_CPU_OFF) + smc #0 + b . diff --git a/ArmPkg/Library/MpInitLib/DxeMpInitLib.inf b/ArmPkg/Library/MpInitLib/DxeMpInitLib.inf new file mode 100644 index 000000000000..72f8fec7f323 --- /dev/null +++ b/ArmPkg/Library/MpInitLib/DxeMpInitLib.inf @@ -0,0 +1,52 @@ +## @file +# MP Initialization Library instance for DXE driver. +# +# Copyright (c) 2021, NUVIA Inc. All rights reserved.
+# +# SPDX-License-Identifier: BSD-2-Clause-Patent +# +## + +[Defines] + INF_VERSION = 1.29 + BASE_NAME = DxeMpInitLib + FILE_GUID = c9ca773c-8ae4-4b74-82fd-f7345503294e + MODULE_TYPE = DXE_DRIVER + VERSION_STRING = 1.0 + LIBRARY_CLASS = MpInitLib|DXE_DRIVER + +# +# The following information is for reference only and not required by the build tools. +# +# VALID_ARCHITECTURES = AARCH64 +# + +[Sources.AARCH64] + AArch64/MpFuncs.S + +[Sources] + DxeMpLib.c + InternalMpInitLib.h + +[Packages] + ArmPkg/ArmPkg.dec + MdeModulePkg/MdeModulePkg.dec + MdePkg/MdePkg.dec + +[LibraryClasses] + ArmLib + ArmSmcLib + BaseLib + BaseMemoryLib + DebugLib + HobLib + MemoryAllocationLib + UefiBootServicesTableLib + UefiDriverEntryPoint + UefiLib + +[Protocols] + gEfiMpServiceProtocolGuid + +[Guids] + gArmMpCoreInfoGuid diff --git a/ArmPkg/Library/MpInitLib/DxeMpLib.c b/ArmPkg/Library/MpInitLib/DxeMpLib.c new file mode 100644 index 000000000000..b00893a5ea7a --- /dev/null +++ b/ArmPkg/Library/MpInitLib/DxeMpLib.c @@ -0,0 +1,1477 @@ +/** @file + Construct MP Services Protocol. + + Copyright (c) 2021, NUVIA Inc. All rights reserved.
+ Copyright (c) 2006 - 2012, Intel Corporation. All rights reserved.
+ Portions Copyright (c) 2011, Apple Inc. All rights reserved. + + SPDX-License-Identifier: BSD-2-Clause-Patent + +**/ + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include "InternalMpInitLib.h" + +#define POLL_INTERVAL_US 50000 + +#define GET_MPIDR_AFFINITY_BITS(x) ((x) & 0xFF00FFFFFF) + +#define MPIDR_MT_BIT BIT24 + +STATIC CPU_MP_DATA mCpuMpData; +STATIC BOOLEAN mNonBlockingModeAllowed; +UINT64 *gApStacksBase; +UINT64 *gProcessorIDs; +CONST UINT64 gApStackSize = AP_STACK_SIZE; + +/** C entry-point for the AP. + This function gets called from the assembly function ApEntryPoint. + +**/ +VOID +ApProcedure ( + VOID + ) +{ + ARM_SMC_ARGS Args; + EFI_AP_PROCEDURE UserApProcedure; + VOID *UserApParameter; + UINTN ProcessorIndex; + + MpInitLibWhoAmI (&mMpServicesTemplate, &ProcessorIndex); + + /* Fetch the user-supplied procedure and parameter to execute */ + UserApProcedure = mCpuMpData.CpuData[ProcessorIndex].Procedure; + UserApParameter = mCpuMpData.CpuData[ProcessorIndex].Parameter; + + UserApProcedure (UserApParameter); + + mCpuMpData.CpuData[ProcessorIndex].State = CpuStateFinished; + + /* Since we're finished with this AP, turn it off */ + Args.Arg0 = ARM_SMC_ID_PSCI_CPU_OFF; + ArmCallSmc (&Args); + + /* Should never be reached */ + ASSERT (FALSE); + CpuDeadLoop (); +} + +/** Turns on the specified core using PSCI and executes the user-supplied + function that's been configured via a previous call to SetApProcedure. + + @param ProcessorIndex The index of the core to turn on. + + @retval EFI_SUCCESS Success. + @retval EFI_DEVICE_ERROR The processor could not be turned on. + +**/ +STATIC +EFI_STATUS +EFIAPI +DispatchCpu ( + IN UINTN ProcessorIndex + ) +{ + ARM_SMC_ARGS Args; + EFI_STATUS Status; + + Status = EFI_SUCCESS; + + mCpuMpData.CpuData[ProcessorIndex].State = CpuStateBusy; + + /* Turn the AP on */ + if (sizeof (Args.Arg0) == sizeof (UINT32)) { + Args.Arg0 = ARM_SMC_ID_PSCI_CPU_ON_AARCH32; + } else { + Args.Arg0 = ARM_SMC_ID_PSCI_CPU_ON_AARCH64; + } + + Args.Arg1 = gProcessorIDs[ProcessorIndex]; + Args.Arg2 = (UINTN)ApEntryPoint; + + ArmCallSmc (&Args); + + if (Args.Arg0 != ARM_SMC_PSCI_RET_SUCCESS) { + DEBUG ((DEBUG_ERROR, "PSCI_CPU_ON call failed: %d", Args.Arg0)); + Status = EFI_DEVICE_ERROR; + } + + return Status; +} + +/** Returns whether the specified processor is the BSP. + + @param[in] ProcessorIndex The index the processor to check. + + @return TRUE if the processor is the BSP, FALSE otherwise. +**/ +STATIC +BOOLEAN +IsProcessorBSP ( + UINTN ProcessorIndex + ) +{ + EFI_PROCESSOR_INFORMATION *CpuInfo; + + CpuInfo = &mCpuMpData.CpuData[ProcessorIndex].Info; + + return (CpuInfo->StatusFlag & PROCESSOR_AS_BSP_BIT) != 0; +} + +/** Returns whether the processor executing this function is the BSP. + + @return Whether the current processor is the BSP. +**/ +STATIC +BOOLEAN +IsCurrentProcessorBSP ( + VOID + ) +{ + EFI_STATUS Status; + UINTN ProcessorIndex; + + Status = MpInitLibWhoAmI (&mMpServicesTemplate, &ProcessorIndex); + if (EFI_ERROR (Status)) { + ASSERT (0); + return FALSE; + } + + return IsProcessorBSP (ProcessorIndex); +} + +/** Get the Application Processors state. + + @param[in] CpuData The pointer to CPU_AP_DATA of specified AP. + + @return The AP status. +**/ +CPU_STATE +GetApState ( + IN CPU_AP_DATA *CpuData + ) +{ + return CpuData->State; +} + +/** Configures the processor context with the user-supplied procedure and + argument. + + @param CpuData The processor context. + @param Procedure The user-supplied procedure. + @param ProcedureArgument The user-supplied procedure argument. + +**/ +STATIC +VOID +SetApProcedure ( + IN CPU_AP_DATA *CpuData, + IN EFI_AP_PROCEDURE Procedure, + IN VOID *ProcedureArgument + ) +{ + ASSERT (CpuData != NULL); + ASSERT (Procedure != NULL); + + CpuData->Parameter = ProcedureArgument; + CpuData->Procedure = Procedure; +} + +/** Returns the index of the next processor that is blocked. + + @param[out] NextNumber The index of the next blocked processor. + + @retval EFI_SUCCESS Successfully found the next blocked processor. + @retval EFI_NOT_FOUND There are no blocked processors. + +**/ +STATIC +EFI_STATUS +GetNextBlockedNumber ( + OUT UINTN *NextNumber + ) +{ + UINTN Index; + CPU_STATE State; + CPU_AP_DATA *CpuData; + + for (Index = 0; Index < mCpuMpData.NumberOfProcessors; Index++) { + CpuData = &mCpuMpData.CpuData[Index]; + if (IsProcessorBSP (Index)) { + // Skip BSP + continue; + } + + State = CpuData->State; + + if (State == CpuStateBlocked) { + *NextNumber = Index; + return EFI_SUCCESS; + } + } + + return EFI_NOT_FOUND; +} + +/** Stalls the BSP for the minimum of POLL_INTERVAL_US and Timeout. + + @param[in] Timeout The time limit in microseconds remaining for + APs to return from Procedure. + + @retval StallTime Time of execution stall. +**/ +STATIC +UINTN +CalculateAndStallInterval ( + IN UINTN Timeout + ) +{ + UINTN StallTime; + + if ((Timeout < POLL_INTERVAL_US) && (Timeout != 0)) { + StallTime = Timeout; + } else { + StallTime = POLL_INTERVAL_US; + } + + gBS->Stall (StallTime); + + return StallTime; +} + +/** + This service retrieves the number of logical processor in the platform + and the number of those logical processors that are enabled on this boot. + This service may only be called from the BSP. + + @param[in] This A pointer to the + EFI_MP_SERVICES_PROTOCOL instance. + @param[out] NumberOfProcessors Pointer to the total number of logical + processors in the system, including + the BSP and disabled APs. + @param[out] NumberOfEnabledProcessors Pointer to the number of enabled + logical processors that exist in the + system, including the BSP. + + @retval EFI_SUCCESS The number of logical processors and enabled + logical processors was retrieved. + @retval EFI_DEVICE_ERROR The calling processor is an AP. + @retval EFI_INVALID_PARAMETER NumberOfProcessors is NULL. + @retval EFI_INVALID_PARAMETER NumberOfEnabledProcessors is NULL. + +**/ +EFI_STATUS +EFIAPI +MpInitLibGetNumberOfProcessors ( + IN EFI_MP_SERVICES_PROTOCOL *This, + OUT UINTN *NumberOfProcessors, + OUT UINTN *NumberOfEnabledProcessors + ) +{ + if ((NumberOfProcessors == NULL) || (NumberOfEnabledProcessors == NULL)) { + return EFI_INVALID_PARAMETER; + } + + if (!IsCurrentProcessorBSP ()) { + return EFI_DEVICE_ERROR; + } + + *NumberOfProcessors = mCpuMpData.NumberOfProcessors; + *NumberOfEnabledProcessors = mCpuMpData.NumberOfEnabledProcessors; + return EFI_SUCCESS; +} + +/** + Gets detailed MP-related information on the requested processor at the + instant this call is made. This service may only be called from the BSP. + + @param[in] This A pointer to the EFI_MP_SERVICES_PROTOCOL + instance. + @param[in] ProcessorIndex The index of the processor. + @param[out] ProcessorInfoBuffer A pointer to the buffer where information + for the requested processor is deposited. + + @retval EFI_SUCCESS Processor information was returned. + @retval EFI_DEVICE_ERROR The calling processor is an AP. + @retval EFI_INVALID_PARAMETER ProcessorInfoBuffer is NULL. + @retval EFI_NOT_FOUND The processor with the handle specified by + ProcessorNumber does not exist in the platform. + +**/ +EFI_STATUS +EFIAPI +MpInitLibGetProcessorInfo ( + IN EFI_MP_SERVICES_PROTOCOL *This, + IN UINTN ProcessorIndex, + OUT EFI_PROCESSOR_INFORMATION *ProcessorInfoBuffer + ) +{ + if (ProcessorInfoBuffer == NULL) { + return EFI_INVALID_PARAMETER; + } + + if (!IsCurrentProcessorBSP ()) { + return EFI_DEVICE_ERROR; + } + + ProcessorIndex &= ~CPU_V2_EXTENDED_TOPOLOGY; + + if (ProcessorIndex >= mCpuMpData.NumberOfProcessors) { + return EFI_NOT_FOUND; + } + + CopyMem ( + ProcessorInfoBuffer, + &mCpuMpData.CpuData[ProcessorIndex], + sizeof (EFI_PROCESSOR_INFORMATION) + ); + return EFI_SUCCESS; +} + +/** Returns whether the specified processor is enabled. + + @param[in] ProcessorIndex The index of the processor to check. + + @return TRUE if the processor is enabled, FALSE otherwise. +**/ +STATIC +BOOLEAN +IsProcessorEnabled ( + UINTN ProcessorIndex + ) +{ + EFI_PROCESSOR_INFORMATION *CpuInfo; + + CpuInfo = &mCpuMpData.CpuData[ProcessorIndex].Info; + + return (CpuInfo->StatusFlag & PROCESSOR_ENABLED_BIT) != 0; +} + +/** Returns whether all processors are in the idle state. + + @return Whether all the processors are idle. + +**/ +STATIC +BOOLEAN +CheckAllCpusReady ( + VOID + ) +{ + UINTN Index; + CPU_AP_DATA *CpuData; + + for (Index = 0; Index < mCpuMpData.NumberOfProcessors; Index++) { + CpuData = &mCpuMpData.CpuData[Index]; + if (IsProcessorBSP (Index)) { + // Skip BSP + continue; + } + + if (!IsProcessorEnabled (Index)) { + // Skip Disabled processors + continue; + } + + if (GetApState (CpuData) != CpuStateIdle) { + return FALSE; + } + } + + return TRUE; +} + +/** Sets up the state for the StartupAllAPs function. + + @param SingleThread Whether the APs will execute sequentially. + +**/ +STATIC +VOID +StartupAllAPsPrepareState ( + IN BOOLEAN SingleThread + ) +{ + UINTN Index; + CPU_STATE APInitialState; + CPU_AP_DATA *CpuData; + + mCpuMpData.FinishCount = 0; + mCpuMpData.StartCount = 0; + mCpuMpData.SingleThread = SingleThread; + + APInitialState = CpuStateReady; + + for (Index = 0; Index < mCpuMpData.NumberOfProcessors; Index++) { + CpuData = &mCpuMpData.CpuData[Index]; + + // + // Get APs prepared, and put failing APs into FailedCpuList. + // If "SingleThread", only 1 AP will put into ready state, other AP will be + // put into ready state 1 by 1, until the previous 1 finished its task. + // If not "SingleThread", all APs are put into ready state from the + // beginning + // + + if (IsProcessorBSP (Index)) { + // Skip BSP + continue; + } + + if (!IsProcessorEnabled (Index)) { + // Skip Disabled processors + if (mCpuMpData.FailedList != NULL) { + mCpuMpData.FailedList[mCpuMpData.FailedListIndex++] = Index; + } + + continue; + } + + ASSERT (GetApState (CpuData) == CpuStateIdle); + CpuData->State = APInitialState; + + mCpuMpData.StartCount++; + if (SingleThread) { + APInitialState = CpuStateBlocked; + } + } +} + +/** Handles execution of StartupAllAPs when a WaitEvent has been specified. + + @param Procedure The user-supplied procedure. + @param ProcedureArgument The user-supplied procedure argument. + @param WaitEvent The wait event to be signaled when the work is + complete or a timeout has occurred. + @param TimeoutInMicroseconds The timeout for the work to be completed. Zero + indicates an infinite timeout. + + @return EFI_SUCCESS on success. +**/ +STATIC +EFI_STATUS +StartupAllAPsWithWaitEvent ( + IN EFI_AP_PROCEDURE Procedure, + IN VOID *ProcedureArgument, + IN EFI_EVENT WaitEvent, + IN UINTN TimeoutInMicroseconds + ) +{ + EFI_STATUS Status; + UINTN Index; + CPU_AP_DATA *CpuData; + + for (Index = 0; Index < mCpuMpData.NumberOfProcessors; Index++) { + CpuData = &mCpuMpData.CpuData[Index]; + if (IsProcessorBSP (Index)) { + // Skip BSP + continue; + } + + if (!IsProcessorEnabled (Index)) { + // Skip Disabled processors + continue; + } + + if (GetApState (CpuData) == CpuStateReady) { + SetApProcedure (CpuData, Procedure, ProcedureArgument); + } + } + + // + // Save data into private data structure, and create timer to poll AP state + // before exiting + // + mCpuMpData.Procedure = Procedure; + mCpuMpData.ProcedureArgument = ProcedureArgument; + mCpuMpData.WaitEvent = WaitEvent; + mCpuMpData.Timeout = TimeoutInMicroseconds; + mCpuMpData.TimeoutActive = (BOOLEAN)(TimeoutInMicroseconds != 0); + Status = gBS->SetTimer ( + mCpuMpData.CheckAllAPsEvent, + TimerPeriodic, + POLL_INTERVAL_US + ); + return Status; +} + +/** Handles execution of StartupAllAPs when no wait event has been specified. + + @param Procedure The user-supplied procedure. + @param ProcedureArgument The user-supplied procedure argument. + @param TimeoutInMicroseconds The timeout for the work to be completed. Zero + indicates an infinite timeout. + @param SingleThread Whether the APs will execute sequentially. + @param FailedCpuList User-supplied pointer for list of failed CPUs. + + @return EFI_SUCCESS on success. +**/ +STATIC +EFI_STATUS +StartupAllAPsNoWaitEvent ( + IN EFI_AP_PROCEDURE Procedure, + IN VOID *ProcedureArgument, + IN UINTN TimeoutInMicroseconds, + IN BOOLEAN SingleThread, + IN UINTN **FailedCpuList + ) +{ + EFI_STATUS Status; + UINTN Index; + UINTN NextIndex; + UINTN Timeout; + CPU_AP_DATA *CpuData; + + Timeout = TimeoutInMicroseconds; + + while (TRUE) { + for (Index = 0; Index < mCpuMpData.NumberOfProcessors; Index++) { + CpuData = &mCpuMpData.CpuData[Index]; + if (IsProcessorBSP (Index)) { + // Skip BSP + continue; + } + + if (!IsProcessorEnabled (Index)) { + // Skip Disabled processors + continue; + } + + switch (GetApState (CpuData)) { + case CpuStateReady: + SetApProcedure (CpuData, Procedure, ProcedureArgument); + Status = DispatchCpu (Index); + if (EFI_ERROR (Status)) { + CpuData->State = CpuStateIdle; + Status = EFI_NOT_READY; + goto Done; + } + + break; + + case CpuStateFinished: + mCpuMpData.FinishCount++; + if (SingleThread) { + Status = GetNextBlockedNumber (&NextIndex); + if (!EFI_ERROR (Status)) { + mCpuMpData.CpuData[NextIndex].State = CpuStateReady; + } + } + + CpuData->State = CpuStateIdle; + break; + + default: + break; + } + } + + if (mCpuMpData.FinishCount == mCpuMpData.StartCount) { + Status = EFI_SUCCESS; + goto Done; + } + + if ((TimeoutInMicroseconds != 0) && (Timeout == 0)) { + Status = EFI_TIMEOUT; + goto Done; + } + + Timeout -= CalculateAndStallInterval (Timeout); + } + +Done: + if (FailedCpuList != NULL) { + if (mCpuMpData.FailedListIndex == 0) { + FreePool (*FailedCpuList); + *FailedCpuList = NULL; + } + } + + return Status; +} + +/** + This service executes a caller provided function on all enabled APs. APs can + run either simultaneously or one at a time in sequence. This service supports + both blocking and non-blocking requests. The non-blocking requests use EFI + events so the BSP can detect when the APs have finished. This service may only + be called from the BSP. + + @param[in] This A pointer to the EFI_MP_SERVICES_PROTOCOL + instance. + @param[in] Procedure A pointer to the function to be run on + enabled APs of the system. See type + EFI_AP_PROCEDURE. + @param[in] SingleThread If TRUE, then all the enabled APs execute + the function specified by Procedure one by + one, in ascending order of processor + handle number. If FALSE, then all the + enabled APs execute the function specified + by Procedure simultaneously. + @param[in] WaitEvent The event created by the caller with + CreateEvent() service. If it is NULL, + then execute in blocking mode. BSP waits + until all APs finish or + TimeoutInMicroseconds expires. If it's + not NULL, then execute in non-blocking + mode. BSP requests the function specified + by Procedure to be started on all the + enabled APs, and go on executing + immediately. If all return from Procedure, + or TimeoutInMicroseconds expires, this + event is signaled. The BSP can use the + CheckEvent() or WaitForEvent() + services to check the state of event. Type + EFI_EVENT is defined in CreateEvent() in + the Unified Extensible Firmware Interface + Specification. + @param[in] TimeoutInMicroseconds Indicates the time limit in microseconds + for APs to return from Procedure, either + for blocking or non-blocking mode. Zero + means infinity. If the timeout expires + before all APs return from Procedure, then + Procedure on the failed APs is terminated. + All enabled APs are available for next + function assigned by + EFI_MP_SERVICES_PROTOCOL.StartupAllAPs() + or EFI_MP_SERVICES_PROTOCOL.StartupThisAP(). + If the timeout expires in blocking mode, + BSP returns EFI_TIMEOUT. If the timeout + expires in non-blocking mode, WaitEvent + is signaled with SignalEvent(). + @param[in] ProcedureArgument The parameter passed into Procedure for + all APs. + @param[out] FailedCpuList If NULL, this parameter is ignored. + Otherwise, if all APs finish successfully, + then its content is set to NULL. If not + all APs finish before timeout expires, + then its content is set to address of the + buffer holding handle numbers of the + failed APs. + The buffer is allocated by MP Service + Protocol, and it's the caller's + responsibility to free the buffer with + FreePool() service. + In blocking mode, it is ready for + consumption when the call returns. In + non-blocking mode, it is ready when + WaitEvent is signaled. The list of failed + CPU is terminated by END_OF_CPU_LIST. + + @retval EFI_SUCCESS In blocking mode, all APs have finished before + the timeout expired. + @retval EFI_SUCCESS In non-blocking mode, function has been + dispatched to all enabled APs. + @retval EFI_UNSUPPORTED A non-blocking mode request was made after the + UEFI event EFI_EVENT_GROUP_READY_TO_BOOT was + signaled. + @retval EFI_DEVICE_ERROR Caller processor is AP. + @retval EFI_NOT_STARTED No enabled APs exist in the system. + @retval EFI_NOT_READY Any enabled APs are busy. + @retval EFI_TIMEOUT In blocking mode, the timeout expired before + all enabled APs have finished. + @retval EFI_INVALID_PARAMETER Procedure is NULL. + +**/ +EFI_STATUS +EFIAPI +MpInitLibStartupAllAPs ( + IN EFI_MP_SERVICES_PROTOCOL *This, + IN EFI_AP_PROCEDURE Procedure, + IN BOOLEAN SingleThread, + IN EFI_EVENT WaitEvent OPTIONAL, + IN UINTN TimeoutInMicroseconds, + IN VOID *ProcedureArgument OPTIONAL, + OUT UINTN **FailedCpuList OPTIONAL + ) +{ + EFI_STATUS Status; + + if (!IsCurrentProcessorBSP ()) { + return EFI_DEVICE_ERROR; + } + + if (mCpuMpData.NumberOfProcessors == 1) { + return EFI_NOT_STARTED; + } + + if (Procedure == NULL) { + return EFI_INVALID_PARAMETER; + } + + if ((WaitEvent != NULL) && !mNonBlockingModeAllowed) { + return EFI_UNSUPPORTED; + } + + if (!CheckAllCpusReady ()) { + return EFI_NOT_READY; + } + + if (FailedCpuList != NULL) { + mCpuMpData.FailedList = AllocatePool ( + (mCpuMpData.NumberOfProcessors + 1) * + sizeof (UINTN) + ); + if (mCpuMpData.FailedList == NULL) { + return EFI_OUT_OF_RESOURCES; + } + + SetMemN ( + mCpuMpData.FailedList, + (mCpuMpData.NumberOfProcessors + 1) * + sizeof (UINTN), + END_OF_CPU_LIST + ); + mCpuMpData.FailedListIndex = 0; + *FailedCpuList = mCpuMpData.FailedList; + } + + StartupAllAPsPrepareState (SingleThread); + + if (WaitEvent != NULL) { + Status = StartupAllAPsWithWaitEvent ( + Procedure, + ProcedureArgument, + WaitEvent, + TimeoutInMicroseconds + ); + } else { + Status = StartupAllAPsNoWaitEvent ( + Procedure, + ProcedureArgument, + TimeoutInMicroseconds, + SingleThread, + FailedCpuList + ); + } + + return Status; +} + +/** + This service lets the caller get one enabled AP to execute a caller-provided + function. The caller can request the BSP to either wait for the completion + of the AP or just proceed with the next task by using the EFI event mechanism. + See EFI_MP_SERVICES_PROTOCOL.StartupAllAPs() for more details on non-blocking + execution support. This service may only be called from the BSP. + + @param[in] This A pointer to the EFI_MP_SERVICES_PROTOCOL + instance. + @param[in] Procedure A pointer to the function to be run on + enabled APs of the system. See type + EFI_AP_PROCEDURE. + @param[in] ProcessorNumber The handle number of the AP. The range is + from 0 to the total number of logical + processors minus 1. The total number of + logical processors can be retrieved by + EFI_MP_SERVICES_PROTOCOL.GetNumberOfProcessors(). + @param[in] WaitEvent The event created by the caller with CreateEvent() + service. If it is NULL, then execute in + blocking mode. BSP waits until all APs finish + or TimeoutInMicroseconds expires. If it's + not NULL, then execute in non-blocking mode. + BSP requests the function specified by + Procedure to be started on all the enabled + APs, and go on executing immediately. If + all return from Procedure or TimeoutInMicroseconds + expires, this event is signaled. The BSP + can use the CheckEvent() or WaitForEvent() + services to check the state of event. Type + EFI_EVENT is defined in CreateEvent() in + the Unified Extensible Firmware Interface + Specification. + @param[in] TimeoutInMicroseconds Indicates the time limit in microseconds for + APs to return from Procedure, either for + blocking or non-blocking mode. Zero means + infinity. If the timeout expires before + all APs return from Procedure, then Procedure + on the failed APs is terminated. All enabled + APs are available for next function assigned + by EFI_MP_SERVICES_PROTOCOL.StartupAllAPs() + or EFI_MP_SERVICES_PROTOCOL.StartupThisAP(). + If the timeout expires in blocking mode, + BSP returns EFI_TIMEOUT. If the timeout + expires in non-blocking mode, WaitEvent + is signaled with SignalEvent(). + @param[in] ProcedureArgument The parameter passed into Procedure for + all APs. + @param[out] Finished If NULL, this parameter is ignored. In + blocking mode, this parameter is ignored. + In non-blocking mode, if AP returns from + Procedure before the timeout expires, its + content is set to TRUE. Otherwise, the + value is set to FALSE. The caller can + determine if the AP returned from Procedure + by evaluating this value. + + @retval EFI_SUCCESS In blocking mode, specified AP finished before + the timeout expires. + @retval EFI_SUCCESS In non-blocking mode, the function has been + dispatched to specified AP. + @retval EFI_UNSUPPORTED A non-blocking mode request was made after the + UEFI event EFI_EVENT_GROUP_READY_TO_BOOT was + signaled. + @retval EFI_DEVICE_ERROR The calling processor is an AP. + @retval EFI_TIMEOUT In blocking mode, the timeout expired before + the specified AP has finished. + @retval EFI_NOT_READY The specified AP is busy. + @retval EFI_NOT_FOUND The processor with the handle specified by + ProcessorNumber does not exist. + @retval EFI_INVALID_PARAMETER ProcessorNumber specifies the BSP or disabled AP. + @retval EFI_INVALID_PARAMETER Procedure is NULL. + +**/ +EFI_STATUS +EFIAPI +MpInitLibStartupThisAP ( + IN EFI_MP_SERVICES_PROTOCOL *This, + IN EFI_AP_PROCEDURE Procedure, + IN UINTN ProcessorNumber, + IN EFI_EVENT WaitEvent OPTIONAL, + IN UINTN TimeoutInMicroseconds, + IN VOID *ProcedureArgument OPTIONAL, + OUT BOOLEAN *Finished OPTIONAL + ) +{ + EFI_STATUS Status; + UINTN Timeout; + CPU_AP_DATA *CpuData; + + if (!IsCurrentProcessorBSP ()) { + return EFI_DEVICE_ERROR; + } + + if (Procedure == NULL) { + return EFI_INVALID_PARAMETER; + } + + if (ProcessorNumber >= mCpuMpData.NumberOfProcessors) { + return EFI_NOT_FOUND; + } + + CpuData = &mCpuMpData.CpuData[ProcessorNumber]; + + if (IsProcessorBSP (ProcessorNumber)) { + return EFI_INVALID_PARAMETER; + } + + if (!IsProcessorEnabled (ProcessorNumber)) { + return EFI_INVALID_PARAMETER; + } + + if (GetApState (CpuData) != CpuStateIdle) { + return EFI_NOT_READY; + } + + if ((WaitEvent != NULL) && !mNonBlockingModeAllowed) { + return EFI_UNSUPPORTED; + } + + Timeout = TimeoutInMicroseconds; + + mCpuMpData.StartCount = 1; + mCpuMpData.FinishCount = 0; + + SetApProcedure ( + CpuData, + Procedure, + ProcedureArgument + ); + + Status = DispatchCpu (ProcessorNumber); + if (EFI_ERROR (Status)) { + CpuData->State = CpuStateIdle; + return EFI_NOT_READY; + } + + if (WaitEvent != NULL) { + // Non Blocking + mCpuMpData.WaitEvent = WaitEvent; + gBS->SetTimer ( + CpuData->CheckThisAPEvent, + TimerPeriodic, + POLL_INTERVAL_US + ); + return EFI_SUCCESS; + } + + // Blocking + while (TRUE) { + if (GetApState (CpuData) == CpuStateFinished) { + CpuData->State = CpuStateIdle; + break; + } + + if ((TimeoutInMicroseconds != 0) && (Timeout == 0)) { + return EFI_TIMEOUT; + } + + Timeout -= CalculateAndStallInterval (Timeout); + } + + return EFI_SUCCESS; +} + +/** + This service switches the requested AP to be the BSP from that point onward. + This service changes the BSP for all purposes. This call can only be + performed by the current BSP. + + @param[in] This A pointer to the EFI_MP_SERVICES_PROTOCOL instance. + @param[in] ProcessorNumber The handle number of AP that is to become the new + BSP. The range is from 0 to the total number of + logical processors minus 1. The total number of + logical processors can be retrieved by + EFI_MP_SERVICES_PROTOCOL.GetNumberOfProcessors(). + @param[in] EnableOldBSP If TRUE, then the old BSP will be listed as an + enabled AP. Otherwise, it will be disabled. + + @retval EFI_SUCCESS BSP successfully switched. + @retval EFI_UNSUPPORTED Switching the BSP cannot be completed prior to + this service returning. + @retval EFI_UNSUPPORTED Switching the BSP is not supported. + @retval EFI_SUCCESS The calling processor is an AP. + @retval EFI_NOT_FOUND The processor with the handle specified by + ProcessorNumber does not exist. + @retval EFI_INVALID_PARAMETER ProcessorNumber specifies the current BSP or + a disabled AP. + @retval EFI_NOT_READY The specified AP is busy. + +**/ +EFI_STATUS +EFIAPI +MpInitLibSwitchBSP ( + IN EFI_MP_SERVICES_PROTOCOL *This, + IN UINTN ProcessorNumber, + IN BOOLEAN EnableOldBSP + ) +{ + // Skip for now as we need switch a bunch of stack stuff around and it's + // complex. May not be worth it? + return EFI_UNSUPPORTED; +} + +/** + This service lets the caller enable or disable an AP from this point onward. + This service may only be called from the BSP. + + @param[in] This A pointer to the EFI_MP_SERVICES_PROTOCOL instance. + @param[in] ProcessorNumber The handle number of AP that is to become the new + BSP. The range is from 0 to the total number of + logical processors minus 1. The total number of + logical processors can be retrieved by + EFI_MP_SERVICES_PROTOCOL.GetNumberOfProcessors(). + @param[in] EnableAP Specifies the new state for the processor for + enabled, FALSE for disabled. + @param[in] HealthFlag If not NULL, a pointer to a value that specifies + the new health status of the AP. This flag + corresponds to StatusFlag defined in + EFI_MP_SERVICES_PROTOCOL.GetProcessorInfo(). Only + the PROCESSOR_HEALTH_STATUS_BIT is used. All other + bits are ignored. If it is NULL, this parameter + is ignored. + + @retval EFI_SUCCESS The specified AP was enabled or disabled successfully. + @retval EFI_UNSUPPORTED Enabling or disabling an AP cannot be completed + prior to this service returning. + @retval EFI_UNSUPPORTED Enabling or disabling an AP is not supported. + @retval EFI_DEVICE_ERROR The calling processor is an AP. + @retval EFI_NOT_FOUND Processor with the handle specified by ProcessorNumber + does not exist. + @retval EFI_INVALID_PARAMETER ProcessorNumber specifies the BSP. + +**/ +EFI_STATUS +EFIAPI +MpInitLibEnableDisableAP ( + IN EFI_MP_SERVICES_PROTOCOL *This, + IN UINTN ProcessorNumber, + IN BOOLEAN EnableAP, + IN UINT32 *HealthFlag OPTIONAL + ) +{ + UINTN StatusFlag; + CPU_AP_DATA *CpuData; + + StatusFlag = mCpuMpData.CpuData[ProcessorNumber].Info.StatusFlag; + CpuData = &mCpuMpData.CpuData[ProcessorNumber]; + + if (!IsCurrentProcessorBSP ()) { + return EFI_DEVICE_ERROR; + } + + if (ProcessorNumber >= mCpuMpData.NumberOfProcessors) { + return EFI_NOT_FOUND; + } + + if (IsProcessorBSP (ProcessorNumber)) { + return EFI_INVALID_PARAMETER; + } + + if (GetApState (CpuData) != CpuStateIdle) { + return EFI_UNSUPPORTED; + } + + if (EnableAP) { + if (!IsProcessorEnabled (ProcessorNumber)) { + mCpuMpData.NumberOfEnabledProcessors++; + } + + StatusFlag |= PROCESSOR_ENABLED_BIT; + } else { + if (IsProcessorEnabled (ProcessorNumber)) { + mCpuMpData.NumberOfEnabledProcessors--; + } + + StatusFlag &= ~PROCESSOR_ENABLED_BIT; + } + + if (HealthFlag != NULL) { + StatusFlag &= ~PROCESSOR_HEALTH_STATUS_BIT; + StatusFlag |= (*HealthFlag & PROCESSOR_HEALTH_STATUS_BIT); + } + + return EFI_SUCCESS; +} + +/** + This return the handle number for the calling processor. This service may be + called from the BSP and APs. + + @param[in] This A pointer to the EFI_MP_SERVICES_PROTOCOL instance. + @param[out] ProcessorNumber The handle number of AP that is to become the new + BSP. The range is from 0 to the total number of + logical processors minus 1. The total number of + logical processors can be retrieved by + EFI_MP_SERVICES_PROTOCOL.GetNumberOfProcessors(). + + @retval EFI_SUCCESS The current processor handle number was returned + in ProcessorNumber. + @retval EFI_INVALID_PARAMETER ProcessorNumber is NULL. + +**/ +EFI_STATUS +EFIAPI +MpInitLibWhoAmI ( + IN EFI_MP_SERVICES_PROTOCOL *This, + OUT UINTN *ProcessorNumber + ) +{ + UINTN Index; + UINT64 ProcessorId; + CPU_AP_DATA *CpuData; + + if (ProcessorNumber == NULL) { + return EFI_INVALID_PARAMETER; + } + + ProcessorId = GET_MPIDR_AFFINITY_BITS (ArmReadMpidr ()); + for (Index = 0; Index < mCpuMpData.NumberOfProcessors; Index++) { + CpuData = &mCpuMpData.CpuData[Index]; + if (CpuData->Info.ProcessorId == ProcessorId) { + break; + } + } + + *ProcessorNumber = Index; + return EFI_SUCCESS; +} + +/** Adds the specified processor the list of failed processors. + + @param ProcessorIndex The processor index to add. + @param ApState Processor state. + +**/ +STATIC +VOID +AddProcessorToFailedList ( + UINTN ProcessorIndex, + CPU_STATE ApState + ) +{ + UINTN Index; + BOOLEAN Found; + + Found = FALSE; + + if (ApState == CpuStateIdle) { + return; + } + + // If we are retrying make sure we don't double count + for (Index = 0; Index < mCpuMpData.FailedListIndex; Index++) { + if (mCpuMpData.FailedList[Index] == ProcessorIndex) { + Found = TRUE; + break; + } + } + + /* If the CPU isn't already in the FailedList, add it */ + if (!Found) { + mCpuMpData.FailedList[mCpuMpData.FailedListIndex++] = Index; + } +} + +/** Handles the StartupAllAPs case where the timeout has occurred. + +**/ +STATIC +VOID +ProcessStartupAllAPsTimeout ( + VOID + ) +{ + CPU_AP_DATA *CpuData; + UINTN Index; + + if (mCpuMpData.FailedList == NULL) { + return; + } + + for (Index = 0; Index < mCpuMpData.NumberOfProcessors; Index++) { + CpuData = &mCpuMpData.CpuData[Index]; + if (IsProcessorBSP (Index)) { + // Skip BSP + continue; + } + + if (!IsProcessorEnabled (Index)) { + // Skip Disabled processors + continue; + } + + CpuData = &mCpuMpData.CpuData[Index]; + AddProcessorToFailedList (Index, GetApState (CpuData)); + } +} + +/** Updates the status of the APs. + + @param[in] ProcessorIndex The index of the AP to update. +**/ +STATIC +VOID +UpdateApStatus ( + IN UINTN ProcessorIndex + ) +{ + EFI_STATUS Status; + CPU_AP_DATA *CpuData; + CPU_AP_DATA *NextCpuData; + CPU_STATE State; + UINTN NextNumber; + + CpuData = &mCpuMpData.CpuData[ProcessorIndex]; + + if (IsProcessorBSP (ProcessorIndex)) { + // Skip BSP + return; + } + + if (!IsProcessorEnabled (ProcessorIndex)) { + // Skip Disabled processors + return; + } + + State = GetApState (CpuData); + + switch (State) { + case CpuStateFinished: + if (mCpuMpData.SingleThread) { + Status = GetNextBlockedNumber (&NextNumber); + if (!EFI_ERROR (Status)) { + NextCpuData = &mCpuMpData.CpuData[NextNumber]; + + NextCpuData->State = CpuStateReady; + + SetApProcedure ( + NextCpuData, + mCpuMpData.Procedure, + mCpuMpData.ProcedureArgument + ); + } + } + + CpuData->State = CpuStateIdle; + mCpuMpData.FinishCount++; + break; + + default: + break; + } +} + +/** + If a timeout is specified in StartupAllAps(), a timer is set, which invokes + this procedure periodically to check whether all APs have finished. + + @param[in] Event The WaitEvent the user supplied. + @param[in] Context The event context. +**/ +STATIC +VOID +EFIAPI +CheckAllAPsStatus ( + IN EFI_EVENT Event, + IN VOID *Context + ) +{ + UINTN Index; + + if (mCpuMpData.TimeoutActive) { + mCpuMpData.Timeout -= CalculateAndStallInterval (mCpuMpData.Timeout); + } + + for (Index = 0; Index < mCpuMpData.NumberOfProcessors; Index++) { + UpdateApStatus (Index); + } + + if (mCpuMpData.TimeoutActive && (mCpuMpData.Timeout == 0)) { + ProcessStartupAllAPsTimeout (); + + // Force terminal exit + mCpuMpData.FinishCount = mCpuMpData.StartCount; + } + + if (mCpuMpData.FinishCount != mCpuMpData.StartCount) { + return; + } + + gBS->SetTimer ( + mCpuMpData.CheckAllAPsEvent, + TimerCancel, + 0 + ); + + if (mCpuMpData.FailedListIndex == 0) { + if (mCpuMpData.FailedList != NULL) { + FreePool (mCpuMpData.FailedList); + mCpuMpData.FailedList = NULL; + } + } + + gBS->SignalEvent (mCpuMpData.WaitEvent); +} + +/** Invoked periodically via a timer to check the state of the processor. + + @param Event The event supplied by the timer expiration. + @param Context The processor context. + +**/ +STATIC +VOID +EFIAPI +CheckThisAPStatus ( + IN EFI_EVENT Event, + IN VOID *Context + ) +{ + EFI_STATUS Status; + CPU_AP_DATA *CpuData; + CPU_STATE State; + + CpuData = Context; + CpuData->TimeTaken += POLL_INTERVAL_US; + + State = GetApState (CpuData); + + if (State == CpuStateFinished) { + Status = gBS->SetTimer (CpuData->CheckThisAPEvent, TimerCancel, 0); + ASSERT_EFI_ERROR (Status); + + if (mCpuMpData.WaitEvent != NULL) { + Status = gBS->SignalEvent (mCpuMpData.WaitEvent); + ASSERT_EFI_ERROR (Status); + } + + CpuData->State = CpuStateIdle; + } + + if (CpuData->TimeTaken > CpuData->Timeout) { + if (mCpuMpData.WaitEvent != NULL) { + Status = gBS->SignalEvent (mCpuMpData.WaitEvent); + ASSERT_EFI_ERROR (Status); + } + } +} + +/** + This function is called by all processors (both BSP and AP) once and collects + MP related data. + + @param BSP TRUE if the processor is the BSP. + @param Mpidr The MPIDR for the specified processor. This should be + the full MPIDR and not only the affinity bits. + @param ProcessorIndex The index of the processor. + + @return EFI_SUCCESS if the data for the processor collected and filled in. + +**/ +STATIC +EFI_STATUS +FillInProcessorInformation ( + IN BOOLEAN BSP, + IN UINTN Mpidr, + IN UINTN ProcessorIndex + ) +{ + EFI_PROCESSOR_INFORMATION *CpuInfo; + + CpuInfo = &mCpuMpData.CpuData[ProcessorIndex].Info; + + CpuInfo->ProcessorId = GET_MPIDR_AFFINITY_BITS (Mpidr); + CpuInfo->StatusFlag = PROCESSOR_ENABLED_BIT | PROCESSOR_HEALTH_STATUS_BIT; + + if (BSP) { + CpuInfo->StatusFlag |= PROCESSOR_AS_BSP_BIT; + } + + if ((Mpidr & MPIDR_MT_BIT) > 0) { + CpuInfo->Location.Package = GET_MPIDR_AFF2 (Mpidr); + CpuInfo->Location.Core = GET_MPIDR_AFF1 (Mpidr); + CpuInfo->Location.Thread = GET_MPIDR_AFF0 (Mpidr); + + CpuInfo->ExtendedInformation.Location2.Package = GET_MPIDR_AFF3 (Mpidr); + CpuInfo->ExtendedInformation.Location2.Die = GET_MPIDR_AFF2 (Mpidr); + CpuInfo->ExtendedInformation.Location2.Core = GET_MPIDR_AFF1 (Mpidr); + CpuInfo->ExtendedInformation.Location2.Thread = GET_MPIDR_AFF0 (Mpidr); + } else { + CpuInfo->Location.Package = GET_MPIDR_AFF1 (Mpidr); + CpuInfo->Location.Core = GET_MPIDR_AFF0 (Mpidr); + CpuInfo->Location.Thread = 0; + + CpuInfo->ExtendedInformation.Location2.Package = GET_MPIDR_AFF2 (Mpidr); + CpuInfo->ExtendedInformation.Location2.Die = GET_MPIDR_AFF1 (Mpidr); + CpuInfo->ExtendedInformation.Location2.Core = GET_MPIDR_AFF0 (Mpidr); + CpuInfo->ExtendedInformation.Location2.Thread = 0; + } + + CpuInfo->ExtendedInformation.Location2.Package = 0; + CpuInfo->ExtendedInformation.Location2.Module = 0; + CpuInfo->ExtendedInformation.Location2.Tile = 0; + + mCpuMpData.CpuData[ProcessorIndex].State = BSP ? CpuStateBusy : CpuStateIdle; + + mCpuMpData.CpuData[ProcessorIndex].Procedure = NULL; + mCpuMpData.CpuData[ProcessorIndex].Parameter = NULL; + + return EFI_SUCCESS; +} + +/** Initializes the MP Services system data + + @param NumberOfProcessors The number of processors, both BSP and AP. + @param CpuInfo CPU information gathered earlier during boot. + +**/ +VOID +MpInitLibInitialize ( + IN UINTN NumberOfProcessors, + IN ARM_PROCESSOR_TABLE *CpuInfo + ) +{ + EFI_STATUS Status; + UINTN Index; + EFI_EVENT ReadyToBootEvent; + + // + // Clear the data structure area first. + // + ZeroMem (&mCpuMpData, sizeof (CPU_MP_DATA)); + // + // First BSP fills and inits all known values, including its own records. + // + mCpuMpData.NumberOfProcessors = NumberOfProcessors; + mCpuMpData.NumberOfEnabledProcessors = NumberOfProcessors; + + mCpuMpData.CpuData = AllocateZeroPool ( + mCpuMpData.NumberOfProcessors * + sizeof (CPU_AP_DATA) + ); + ASSERT (mCpuMpData.CpuData != NULL); + + /* Allocate one extra for the NULL entry at the end */ + gProcessorIDs = AllocatePool ((mCpuMpData.NumberOfProcessors + 1) * sizeof (UINT64)); + ASSERT (gProcessorIDs != NULL); + + FillInProcessorInformation (TRUE, CpuInfo->ArmCpus[0].Mpidr, 0); + gProcessorIDs[0] = mCpuMpData.CpuData[0].Info.ProcessorId; + + Status = gBS->CreateEvent ( + EVT_TIMER | EVT_NOTIFY_SIGNAL, + TPL_CALLBACK, + CheckAllAPsStatus, + NULL, + &mCpuMpData.CheckAllAPsEvent + ); + ASSERT_EFI_ERROR (Status); + + gApStacksBase = AllocatePool ( + mCpuMpData.NumberOfProcessors * + gApStackSize + ); + ASSERT (gApStacksBase != NULL); + + for (Index = 0; Index < mCpuMpData.NumberOfProcessors; Index++) { + if (IsProcessorBSP (Index)) { + /* Skip BSP */ + continue; + } + + FillInProcessorInformation (FALSE, CpuInfo->ArmCpus[Index].Mpidr, Index); + + gProcessorIDs[Index] = mCpuMpData.CpuData[Index].Info.ProcessorId; + + Status = gBS->CreateEvent ( + EVT_TIMER | EVT_NOTIFY_SIGNAL, + TPL_CALLBACK, + CheckThisAPStatus, + (VOID *)&mCpuMpData.CpuData[Index], + &mCpuMpData.CpuData[Index].CheckThisAPEvent + ); + ASSERT (Status == EFI_SUCCESS); + } + + Status = EfiCreateEventReadyToBootEx ( + TPL_CALLBACK, + ReadyToBootSignaled, + NULL, + &ReadyToBootEvent + ); + ASSERT_EFI_ERROR (Status); + + gProcessorIDs[Index] = MAX_UINT32; +} + +/** + Event notification function called when the EFI_EVENT_GROUP_READY_TO_BOOT is + signaled. After this point, non-blocking mode is no longer allowed. + + @param Event Event whose notification function is being invoked. + @param Context The pointer to the notification function's context, + which is implementation-dependent. + +**/ +STATIC +VOID +EFIAPI +ReadyToBootSignaled ( + IN EFI_EVENT Event, + IN VOID *Context + ) +{ + mNonBlockingModeAllowed = FALSE; +} diff --git a/ArmPkg/Library/MpInitLib/InternalMpInitLib.h b/ArmPkg/Library/MpInitLib/InternalMpInitLib.h new file mode 100644 index 000000000000..ccae24ecaf4b --- /dev/null +++ b/ArmPkg/Library/MpInitLib/InternalMpInitLib.h @@ -0,0 +1,357 @@ +/** @file + +Copyright (c) 2021, NUVIA Inc. All rights reserved.
+Copyright (c) 2006 - 2011, Intel Corporation. All rights reserved.
+Portions copyright (c) 2011, Apple Inc. All rights reserved. + +SPDX-License-Identifier: BSD-2-Clause-Patent + +**/ + +#ifndef MP_INTERNAL_INIT_LIB_H_ +#define MP_INTERNAL_INIT_LIB_H_ + +#include +#include + +#include +#include + +#define AP_STACK_SIZE 0x1000 + +// +// Internal Data Structures +// + +// +// AP state +// +// The state transitions for an AP when it process a procedure are: +// Idle ----> Ready ----> Busy ----> Idle +// [BSP] [AP] [AP] +// +typedef enum { + CpuStateIdle, + CpuStateReady, + CpuStateBlocked, + CpuStateBusy, + CpuStateFinished, + CpuStateDisabled +} CPU_STATE; + +// +// Define Individual Processor Data block. +// +typedef struct { + EFI_PROCESSOR_INFORMATION Info; + EFI_AP_PROCEDURE Procedure; + VOID *Parameter; + CPU_STATE State; + EFI_EVENT CheckThisAPEvent; + UINTN Timeout; + UINTN TimeTaken; +} CPU_AP_DATA; + +// +// Define MP data block which consumes individual processor block. +// +typedef struct { + UINTN NumberOfProcessors; + UINTN NumberOfEnabledProcessors; + EFI_EVENT CheckAllAPsEvent; + EFI_EVENT WaitEvent; + UINTN FinishCount; + UINTN StartCount; + EFI_AP_PROCEDURE Procedure; + VOID *ProcedureArgument; + BOOLEAN SingleThread; + UINTN StartedNumber; + CPU_AP_DATA *CpuData; + UINTN Timeout; + UINTN *FailedList; + UINTN FailedListIndex; + BOOLEAN TimeoutActive; +} CPU_MP_DATA; + +EFI_STATUS +EFIAPI +CpuMpServicesInit ( + IN EFI_HANDLE ImageHandle, + IN EFI_SYSTEM_TABLE *SystemTable + ); + +extern EFI_MP_SERVICES_PROTOCOL mMpServicesTemplate; + +/** Secondary core entry point. + +**/ +VOID +ApEntryPoint ( + VOID + ); + +/** C entry-point for the AP. + This function gets called from the assembly function ApEntryPoint. +**/ +VOID +ApProcedure ( + VOID + ); + +/** Turns on the specified core using PSCI and executes the user-supplied + function that's been configured via a previous call to SetApProcedure. + + @param ProcessorIndex The index of the core to turn on. + + @retval EFI_SUCCESS The processor was successfully turned on. + @retval EFI_DEVICE_ERROR An error occurred turning the processor on. + +**/ +STATIC +EFI_STATUS +EFIAPI +DispatchCpu ( + IN UINTN ProcessorIndex + ); + +/** Returns whether the specified processor is the BSP. + + @param[in] ProcessorIndex The index the processor to check. + + @return TRUE if the processor is the BSP, FALSE otherwise. +**/ +STATIC +BOOLEAN +IsProcessorBSP ( + UINTN ProcessorIndex + ); + +/** Returns whether the processor executing this function is the BSP. + + @return Whether the current processor is the BSP. +**/ +STATIC +BOOLEAN +IsCurrentProcessorBSP ( + VOID + ); + +/** Returns whether the specified processor is enabled. + + @param[in] ProcessorIndex The index of the processor to check. + + @return TRUE if the processor is enabled, FALSE otherwise. +**/ +STATIC +BOOLEAN +IsProcessorEnabled ( + UINTN ProcessorIndex + ); + +/** Configures the processor context with the user-supplied procedure and + argument. + + @param CpuData The processor context. + @param Procedure The user-supplied procedure. + @param ProcedureArgument The user-supplied procedure argument. + +**/ +STATIC +VOID +SetApProcedure ( + IN CPU_AP_DATA *CpuData, + IN EFI_AP_PROCEDURE Procedure, + IN VOID *ProcedureArgument + ); + +/** + Get the Application Processors state. + + @param[in] CpuData The pointer to CPU_AP_DATA of specified AP + + @return The AP status +**/ +CPU_STATE +GetApState ( + IN CPU_AP_DATA *CpuData + ); + +/** Returns the index of the next processor that is blocked. + + @param[out] NextNumber The index of the next blocked processor. + + @retval EFI_SUCCESS Successfully found the next blocked processor. + @retval EFI_NOT_FOUND There are no blocked processors. + +**/ +STATIC +EFI_STATUS +GetNextBlockedNumber ( + OUT UINTN *NextNumber + ); + +/** Stalls the BSP for the minimum of gPollInterval and Timeout. + + @param[in] Timeout The time limit in microseconds remaining for + APs to return from Procedure. + + @retval StallTime Time of execution stall. +**/ +STATIC +UINTN +CalculateAndStallInterval ( + IN UINTN Timeout + ); + +/** Returns whether all processors are in the idle state. + + @return Whether all the processors are idle. + +**/ +STATIC +BOOLEAN +CheckAllCpusReady ( + VOID + ); + +/** Sets up the state for the StartupAllAPs function. + + @param SingleThread Whether the APs will execute sequentially. + +**/ +STATIC +VOID +StartupAllAPsPrepareState ( + IN BOOLEAN SingleThread + ); + +/** Handles execution of StartupAllAPs when a WaitEvent has been specified. + + @param Procedure The user-supplied procedure. + @param ProcedureArgument The user-supplied procedure argument. + @param WaitEvent The wait event to be signaled when the work is + complete or a timeout has occurred. + @param TimeoutInMicroseconds The timeout for the work to be completed. Zero + indicates an infinite timeout. + + @return EFI_SUCCESS on success. +**/ +STATIC +EFI_STATUS +StartupAllAPsWithWaitEvent ( + IN EFI_AP_PROCEDURE Procedure, + IN VOID *ProcedureArgument, + IN EFI_EVENT WaitEvent, + IN UINTN TimeoutInMicroseconds + ); + +/** Handles execution of StartupAllAPs when no wait event has been specified. + + @param Procedure The user-supplied procedure. + @param ProcedureArgument The user-supplied procedure argument. + @param TimeoutInMicroseconds The timeout for the work to be completed. Zero + indicates an infinite timeout. + @param SingleThread Whether the APs will execute sequentially. + @param FailedCpuList User-supplied pointer for list of failed CPUs. + + @return EFI_SUCCESS on success. +**/ +STATIC +EFI_STATUS +StartupAllAPsNoWaitEvent ( + IN EFI_AP_PROCEDURE Procedure, + IN VOID *ProcedureArgument, + IN UINTN TimeoutInMicroseconds, + IN BOOLEAN SingleThread, + IN UINTN **FailedCpuList + ); + +/** Adds the specified processor the list of failed processors. + + @param ProcessorIndex The processor index to add. + @param ApState Processor state. + +**/ +STATIC +VOID +AddProcessorToFailedList ( + UINTN ProcessorIndex, + CPU_STATE ApState + ); + +/** Handles the StartupAllAPs case where the timeout has occurred. + +**/ +STATIC +VOID +ProcessStartupAllAPsTimeout ( + VOID + ); + +/** + If a timeout is specified in StartupAllAps(), a timer is set, which invokes + this procedure periodically to check whether all APs have finished. + + @param[in] Event The WaitEvent the user supplied. + @param[in] Context The event context. +**/ +STATIC +VOID +EFIAPI +CheckAllAPsStatus ( + IN EFI_EVENT Event, + IN VOID *Context + ); + +/** Invoked periodically via a timer to check the state of the processor. + + @param Event The event supplied by the timer expiration. + @param Context The processor context. + +**/ +STATIC +VOID +EFIAPI +CheckThisAPStatus ( + IN EFI_EVENT Event, + IN VOID *Context + ); + +/** + This function is called by all processors (both BSP and AP) once and collects + MP related data. + + @param BSP TRUE if the processor is the BSP. + @param Mpidr The MPIDR for the specified processor. This should be + the full MPIDR and not only the affinity bits. + @param ProcessorIndex The index of the processor. + + @return EFI_SUCCESS if the data for the processor collected and filled in. + +**/ +STATIC +EFI_STATUS +FillInProcessorInformation ( + IN BOOLEAN BSP, + IN UINTN Mpidr, + IN UINTN ProcessorIndex + ); + +/** + Event notification function called when the EFI_EVENT_GROUP_READY_TO_BOOT is + signaled. After this point, non-blocking mode is no longer allowed. + + @param Event Event whose notification function is being invoked. + @param Context The pointer to the notification function's context, + which is implementation-dependent. + +**/ +STATIC +VOID +EFIAPI +ReadyToBootSignaled ( + IN EFI_EVENT Event, + IN VOID *Context + ); + +#endif /* MP_INTERNAL_INIT_LIB_H_ */ -- 2.31.1