public inbox for devel@edk2.groups.io
 help / color / mirror / Atom feed
From: "Rebecca Cran" <rebecca@nuviainc.com>
To: devel@edk2.groups.io, Ard Biesheuvel <ardb+tianocore@kernel.org>,
	Gerd Hoffmann <kraxel@redhat.com>,
	Samer El-Haj-Mahmoud <samer.el-haj-mahmoud@arm.com>,
	Leif Lindholm <leif@nuviainc.com>,
	Sami Mujawar <sami.mujawar@arm.com>
Cc: Rebecca Cran <rebecca@nuviainc.com>
Subject: [PATCH v3 2/4] ArmPkg: Add Library/MpInitLib to support EFI_MP_SERVICES_PROTOCOL
Date: Wed, 15 Dec 2021 20:46:32 -0700	[thread overview]
Message-ID: <20211216034634.15468-3-rebecca@nuviainc.com> (raw)
In-Reply-To: <20211216034634.15468-1-rebecca@nuviainc.com>

Add support for EFI_MP_SERVICES_PROTOCOL during the DXE phase under
AArch64.

PSCI_CPU_ON is called to power on the core, the supplied procedure is
executed and PSCI_CPU_OFF is called to power off the core.

Minimal setup is done before calling the supplied procedure: for example
the MMU and caches are not enabled.

Signed-off-by: Rebecca Cran <rebecca@nuviainc.com>
---
 ArmPkg/ArmPkg.dec                            |    4 +
 ArmPkg/ArmPkg.dsc                            |    4 +
 ArmPkg/Include/Library/MpInitLib.h           |  362 +++++
 ArmPkg/Library/MpInitLib/AArch64/MpFuncs.S   |   65 +
 ArmPkg/Library/MpInitLib/DxeMpInitLib.inf    |   52 +
 ArmPkg/Library/MpInitLib/DxeMpLib.c          | 1477 ++++++++++++++++++++
 ArmPkg/Library/MpInitLib/InternalMpInitLib.h |  357 +++++
 7 files changed, 2321 insertions(+)

diff --git a/ArmPkg/ArmPkg.dec b/ArmPkg/ArmPkg.dec
index 9da1bbc9f216..363eddc57393 100644
--- a/ArmPkg/ArmPkg.dec
+++ b/ArmPkg/ArmPkg.dec
@@ -75,6 +75,10 @@
   #
   DefaultExceptionHandlerLib|Include/Library/DefaultExceptionHandlerLib.h
 
+  ##  @libraryclass  Provides a MP Services interface.
+  #
+  MpInitLib|Include/Library/MpInitLib.h
+
   ##  @libraryclass  Provides an interface to query miscellaneous OEM
   #   information.
   #
diff --git a/ArmPkg/ArmPkg.dsc b/ArmPkg/ArmPkg.dsc
index 59fd8f295d4f..6e053d6ee31d 100644
--- a/ArmPkg/ArmPkg.dsc
+++ b/ArmPkg/ArmPkg.dsc
@@ -100,6 +100,9 @@
   PeiServicesLib|MdePkg/Library/PeiServicesLib/PeiServicesLib.inf
   PeiServicesTablePointerLib|MdePkg/Library/PeiServicesTablePointerLib/PeiServicesTablePointerLib.inf
 
+[LibraryClasses.AARCH64]
+  MpInitLib|ArmPkg/Library/MpInitLib/DxeMpInitLib.inf
+
 [LibraryClasses.ARM, LibraryClasses.AARCH64]
   NULL|ArmPkg/Library/CompilerIntrinsicsLib/CompilerIntrinsicsLib.inf
 
@@ -163,6 +166,7 @@
 [Components.AARCH64]
   ArmPkg/Drivers/MmCommunicationDxe/MmCommunication.inf
   ArmPkg/Library/ArmMmuLib/ArmMmuPeiLib.inf
+  ArmPkg/Library/MpInitLib/DxeMpInitLib.inf
 
 [Components.AARCH64, Components.ARM]
   ArmPkg/Library/StandaloneMmMmuLib/ArmMmuStandaloneMmLib.inf
diff --git a/ArmPkg/Include/Library/MpInitLib.h b/ArmPkg/Include/Library/MpInitLib.h
new file mode 100644
index 000000000000..582bb788fd59
--- /dev/null
+++ b/ArmPkg/Include/Library/MpInitLib.h
@@ -0,0 +1,362 @@
+/** @file
+
+Copyright (c) 2021, NUVIA Inc. All rights reserved.<BR>
+Copyright (c) 2006 - 2011, Intel Corporation. All rights reserved.<BR>
+Portions copyright (c) 2011, Apple Inc. All rights reserved.
+
+SPDX-License-Identifier: BSD-2-Clause-Patent
+
+**/
+
+#ifndef MP_INITLIB_H_
+#define MP_INITLIB_H_
+
+#include <Protocol/Cpu.h>
+#include <Protocol/MpService.h>
+#include <Library/BaseLib.h>
+#include <Library/UefiLib.h>
+#include <Guid/ArmMpCoreInfo.h>
+
+/**
+  This service retrieves the number of logical processor in the platform
+  and the number of those logical processors that are enabled on this boot.
+  This service may only be called from the BSP.
+
+  @param[in]  This                        A pointer to the
+                                          EFI_MP_SERVICES_PROTOCOL instance.
+  @param[out] NumberOfProcessors          Pointer to the total number of logical
+                                          processors in the system, including
+                                          the BSP and disabled APs.
+  @param[out] NumberOfEnabledProcessors   Pointer to the number of enabled
+                                          logical processors that exist in the
+                                          system, including the BSP.
+
+  @retval EFI_SUCCESS             The number of logical processors and enabled
+                                  logical processors was retrieved.
+  @retval EFI_DEVICE_ERROR        The calling processor is an AP.
+  @retval EFI_INVALID_PARAMETER   NumberOfProcessors is NULL.
+  @retval EFI_INVALID_PARAMETER   NumberOfEnabledProcessors is NULL.
+
+**/
+EFI_STATUS
+EFIAPI
+MpInitLibGetNumberOfProcessors (
+  IN  EFI_MP_SERVICES_PROTOCOL  *This,
+  OUT UINTN                     *NumberOfProcessors,
+  OUT UINTN                     *NumberOfEnabledProcessors
+  );
+
+/**
+  Gets detailed MP-related information on the requested processor at the
+  instant this call is made. This service may only be called from the BSP.
+
+  @param[in]  This                  A pointer to the EFI_MP_SERVICES_PROTOCOL
+                                    instance.
+  @param[in]  ProcessorIndex        The index of the processor.
+  @param[out] ProcessorInfoBuffer   A pointer to the buffer where information
+                                    for the requested processor is deposited.
+
+  @retval EFI_SUCCESS             Processor information was returned.
+  @retval EFI_DEVICE_ERROR        The calling processor is an AP.
+  @retval EFI_INVALID_PARAMETER   ProcessorInfoBuffer is NULL.
+  @retval EFI_NOT_FOUND           The processor with the handle specified by
+                                  ProcessorNumber does not exist in the platform.
+
+**/
+EFI_STATUS
+EFIAPI
+MpInitLibGetProcessorInfo (
+  IN  EFI_MP_SERVICES_PROTOCOL   *This,
+  IN  UINTN                      ProcessorIndex,
+  OUT EFI_PROCESSOR_INFORMATION  *ProcessorInfoBuffer
+  );
+
+/**
+  This service executes a caller provided function on all enabled APs. APs can
+  run either simultaneously or one at a time in sequence. This service supports
+  both blocking and non-blocking requests. The non-blocking requests use EFI
+  events so the BSP can detect when the APs have finished. This service may only
+  be called from the BSP.
+
+  @param[in]  This                    A pointer to the EFI_MP_SERVICES_PROTOCOL
+                                      instance.
+  @param[in]  Procedure               A pointer to the function to be run on
+                                      enabled APs of the system. See type
+                                      EFI_AP_PROCEDURE.
+  @param[in]  SingleThread            If TRUE, then all the enabled APs execute
+                                      the function specified by Procedure one by
+                                      one, in ascending order of processor
+                                      handle number.  If FALSE, then all the
+                                      enabled APs execute the function specified
+                                      by Procedure simultaneously.
+  @param[in]  WaitEvent               The event created by the caller with
+                                      CreateEvent() service.  If it is NULL,
+                                      then execute in blocking mode. BSP waits
+                                      until all APs finish or
+                                      TimeoutInMicroseconds expires.  If it's
+                                      not NULL, then execute in non-blocking
+                                      mode. BSP requests the function specified
+                                      by Procedure to be started on all the
+                                      enabled APs, and go on executing
+                                      immediately. If all return from Procedure,
+                                      or TimeoutInMicroseconds expires, this
+                                      event is signaled. The BSP can use the
+                                      CheckEvent() or WaitForEvent()
+                                      services to check the state of event. Type
+                                      EFI_EVENT is defined in CreateEvent() in
+                                      the Unified Extensible Firmware Interface
+                                      Specification.
+  @param[in]  TimeoutInMicroseconds   Indicates the time limit in microseconds
+                                      for APs to return from Procedure, either
+                                      for blocking or non-blocking mode. Zero
+                                      means infinity.  If the timeout expires
+                                      before all APs return from Procedure, then
+                                      Procedure on the failed APs is terminated.
+                                      All enabled APs are available for next
+                                      function assigned by
+                                      EFI_MP_SERVICES_PROTOCOL.StartupAllAPs()
+                                      or EFI_MP_SERVICES_PROTOCOL.StartupThisAP().
+                                      If the timeout expires in blocking mode,
+                                      BSP returns EFI_TIMEOUT.  If the timeout
+                                      expires in non-blocking mode, WaitEvent
+                                      is signaled with SignalEvent().
+  @param[in]  ProcedureArgument       The parameter passed into Procedure for
+                                      all APs.
+  @param[out] FailedCpuList           If NULL, this parameter is ignored.
+                                      Otherwise, if all APs finish successfully,
+                                      then its content is set to NULL. If not
+                                      all APs finish before timeout expires,
+                                      then its content is set to address of the
+                                      buffer holding handle numbers of the
+                                      failed APs.
+                                      The buffer is allocated by MP Service
+                                      Protocol, and it's the caller's
+                                      responsibility to free the buffer with
+                                      FreePool() service.
+                                      In blocking mode, it is ready for
+                                      consumption when the call returns. In
+                                      non-blocking mode, it is ready when
+                                      WaitEvent is signaled. The list of failed
+                                      CPU is terminated by  END_OF_CPU_LIST.
+
+  @retval EFI_SUCCESS             In blocking mode, all APs have finished before
+                                  the timeout expired.
+  @retval EFI_SUCCESS             In non-blocking mode, function has been
+                                  dispatched to all enabled APs.
+  @retval EFI_UNSUPPORTED         A non-blocking mode request was made after the
+                                  UEFI event EFI_EVENT_GROUP_READY_TO_BOOT was
+                                  signaled.
+  @retval EFI_DEVICE_ERROR        Caller processor is AP.
+  @retval EFI_NOT_STARTED         No enabled APs exist in the system.
+  @retval EFI_NOT_READY           Any enabled APs are busy.
+  @retval EFI_TIMEOUT             In blocking mode, the timeout expired before
+                                  all enabled APs have finished.
+  @retval EFI_INVALID_PARAMETER   Procedure is NULL.
+
+**/
+EFI_STATUS
+EFIAPI
+MpInitLibStartupAllAPs (
+  IN  EFI_MP_SERVICES_PROTOCOL  *This,
+  IN  EFI_AP_PROCEDURE          Procedure,
+  IN  BOOLEAN                   SingleThread,
+  IN  EFI_EVENT                 WaitEvent               OPTIONAL,
+  IN  UINTN                     TimeoutInMicroseconds,
+  IN  VOID                      *ProcedureArgument      OPTIONAL,
+  OUT UINTN                     **FailedCpuList         OPTIONAL
+  );
+
+/**
+  This service lets the caller get one enabled AP to execute a caller-provided
+  function. The caller can request the BSP to either wait for the completion
+  of the AP or just proceed with the next task by using the EFI event mechanism.
+  See EFI_MP_SERVICES_PROTOCOL.StartupAllAPs() for more details on non-blocking
+  execution support.  This service may only be called from the BSP.
+
+  @param[in]  This                    A pointer to the EFI_MP_SERVICES_PROTOCOL
+                                      instance.
+  @param[in]  Procedure               A pointer to the function to be run on
+                                      enabled APs of the system. See type
+                                      EFI_AP_PROCEDURE.
+  @param[in]  ProcessorNumber         The handle number of the AP. The range is
+                                      from 0 to the total number of logical
+                                      processors minus 1. The total number of
+                                      logical processors can be retrieved by
+                                      EFI_MP_SERVICES_PROTOCOL.GetNumberOfProcessors().
+  @param[in]  WaitEvent               The event created by the caller with CreateEvent()
+                                      service.  If it is NULL, then execute in
+                                      blocking mode. BSP waits until all APs finish
+                                      or TimeoutInMicroseconds expires.  If it's
+                                      not NULL, then execute in non-blocking mode.
+                                      BSP requests the function specified by
+                                      Procedure to be started on all the enabled
+                                      APs, and go on executing immediately. If
+                                      all return from Procedure or TimeoutInMicroseconds
+                                      expires, this event is signaled. The BSP
+                                      can use the CheckEvent() or WaitForEvent()
+                                      services to check the state of event.  Type
+                                      EFI_EVENT is defined in CreateEvent() in
+                                      the Unified Extensible Firmware Interface
+                                      Specification.
+  @param[in]  TimeoutInMicroseconds   Indicates the time limit in microseconds for
+                                      APs to return from Procedure, either for
+                                      blocking or non-blocking mode. Zero means
+                                      infinity.  If the timeout expires before
+                                      all APs return from Procedure, then Procedure
+                                      on the failed APs is terminated. All enabled
+                                      APs are available for next function assigned
+                                      by EFI_MP_SERVICES_PROTOCOL.StartupAllAPs()
+                                      or EFI_MP_SERVICES_PROTOCOL.StartupThisAP().
+                                      If the timeout expires in blocking mode,
+                                      BSP returns EFI_TIMEOUT.  If the timeout
+                                      expires in non-blocking mode, WaitEvent
+                                      is signaled with SignalEvent().
+  @param[in]  ProcedureArgument       The parameter passed into Procedure for
+                                      all APs.
+  @param[out] Finished                If NULL, this parameter is ignored.  In
+                                      blocking mode, this parameter is ignored.
+                                      In non-blocking mode, if AP returns from
+                                      Procedure before the timeout expires, its
+                                      content is set to TRUE. Otherwise, the
+                                      value is set to FALSE. The caller can
+                                      determine if the AP returned from Procedure
+                                      by evaluating this value.
+
+  @retval EFI_SUCCESS             In blocking mode, specified AP finished before
+                                  the timeout expires.
+  @retval EFI_SUCCESS             In non-blocking mode, the function has been
+                                  dispatched to specified AP.
+  @retval EFI_UNSUPPORTED         A non-blocking mode request was made after the
+                                  UEFI event EFI_EVENT_GROUP_READY_TO_BOOT was
+                                  signaled.
+  @retval EFI_DEVICE_ERROR        The calling processor is an AP.
+  @retval EFI_TIMEOUT             In blocking mode, the timeout expired before
+                                  the specified AP has finished.
+  @retval EFI_NOT_READY           The specified AP is busy.
+  @retval EFI_NOT_FOUND           The processor with the handle specified by
+                                  ProcessorNumber does not exist.
+  @retval EFI_INVALID_PARAMETER   ProcessorNumber specifies the BSP or disabled AP.
+  @retval EFI_INVALID_PARAMETER   Procedure is NULL.
+
+**/
+EFI_STATUS
+EFIAPI
+MpInitLibStartupThisAP (
+  IN  EFI_MP_SERVICES_PROTOCOL  *This,
+  IN  EFI_AP_PROCEDURE          Procedure,
+  IN  UINTN                     ProcessorNumber,
+  IN  EFI_EVENT                 WaitEvent               OPTIONAL,
+  IN  UINTN                     TimeoutInMicroseconds,
+  IN  VOID                      *ProcedureArgument      OPTIONAL,
+  OUT BOOLEAN                   *Finished               OPTIONAL
+  );
+
+/**
+  This service switches the requested AP to be the BSP from that point onward.
+  This service changes the BSP for all purposes.   This call can only be
+  performed by the current BSP.
+
+  @param[in] This              A pointer to the EFI_MP_SERVICES_PROTOCOL instance.
+  @param[in] ProcessorNumber   The handle number of AP that is to become the new
+                               BSP. The range is from 0 to the total number of
+                               logical processors minus 1. The total number of
+                               logical processors can be retrieved by
+                               EFI_MP_SERVICES_PROTOCOL.GetNumberOfProcessors().
+  @param[in] EnableOldBSP      If TRUE, then the old BSP will be listed as an
+                               enabled AP. Otherwise, it will be disabled.
+
+  @retval EFI_SUCCESS             BSP successfully switched.
+  @retval EFI_UNSUPPORTED         Switching the BSP cannot be completed prior to
+                                  this service returning.
+  @retval EFI_UNSUPPORTED         Switching the BSP is not supported.
+  @retval EFI_SUCCESS             The calling processor is an AP.
+  @retval EFI_NOT_FOUND           The processor with the handle specified by
+                                  ProcessorNumber does not exist.
+  @retval EFI_INVALID_PARAMETER   ProcessorNumber specifies the current BSP or
+                                  a disabled AP.
+  @retval EFI_NOT_READY           The specified AP is busy.
+
+**/
+EFI_STATUS
+EFIAPI
+MpInitLibSwitchBSP (
+  IN EFI_MP_SERVICES_PROTOCOL  *This,
+  IN  UINTN                    ProcessorNumber,
+  IN  BOOLEAN                  EnableOldBSP
+  );
+
+/**
+  This service lets the caller enable or disable an AP from this point onward.
+  This service may only be called from the BSP.
+
+  @param[in] This              A pointer to the EFI_MP_SERVICES_PROTOCOL instance.
+  @param[in] ProcessorNumber   The handle number of AP that is to become the new
+                               BSP. The range is from 0 to the total number of
+                               logical processors minus 1. The total number of
+                               logical processors can be retrieved by
+                               EFI_MP_SERVICES_PROTOCOL.GetNumberOfProcessors().
+  @param[in] EnableAP          Specifies the new state for the processor for
+                               enabled, FALSE for disabled.
+  @param[in] HealthFlag        If not NULL, a pointer to a value that specifies
+                               the new health status of the AP. This flag
+                               corresponds to StatusFlag defined in
+                               EFI_MP_SERVICES_PROTOCOL.GetProcessorInfo(). Only
+                               the PROCESSOR_HEALTH_STATUS_BIT is used. All other
+                               bits are ignored.  If it is NULL, this parameter
+                               is ignored.
+
+  @retval EFI_SUCCESS             The specified AP was enabled or disabled successfully.
+  @retval EFI_UNSUPPORTED         Enabling or disabling an AP cannot be completed
+                                  prior to this service returning.
+  @retval EFI_UNSUPPORTED         Enabling or disabling an AP is not supported.
+  @retval EFI_DEVICE_ERROR        The calling processor is an AP.
+  @retval EFI_NOT_FOUND           Processor with the handle specified by ProcessorNumber
+                                  does not exist.
+  @retval EFI_INVALID_PARAMETER   ProcessorNumber specifies the BSP.
+
+**/
+EFI_STATUS
+EFIAPI
+MpInitLibEnableDisableAP (
+  IN  EFI_MP_SERVICES_PROTOCOL  *This,
+  IN  UINTN                     ProcessorNumber,
+  IN  BOOLEAN                   EnableAP,
+  IN  UINT32                    *HealthFlag OPTIONAL
+  );
+
+/**
+  This return the handle number for the calling processor.  This service may be
+  called from the BSP and APs.
+
+  @param[in] This              A pointer to the EFI_MP_SERVICES_PROTOCOL instance.
+  @param[out] ProcessorNumber  The handle number of AP that is to become the new
+                               BSP. The range is from 0 to the total number of
+                               logical processors minus 1. The total number of
+                               logical processors can be retrieved by
+                               EFI_MP_SERVICES_PROTOCOL.GetNumberOfProcessors().
+
+  @retval EFI_SUCCESS             The current processor handle number was returned
+                                  in ProcessorNumber.
+  @retval EFI_INVALID_PARAMETER   ProcessorNumber is NULL.
+
+**/
+EFI_STATUS
+EFIAPI
+MpInitLibWhoAmI (
+  IN EFI_MP_SERVICES_PROTOCOL  *This,
+  OUT UINTN                    *ProcessorNumber
+  );
+
+/** Initializes the MP Services system data
+
+   @param NumberOfProcessors The number of processors, both BSP and AP.
+   @param CpuInfo            CPU information gathered earlier during boot.
+
+**/
+VOID
+MpInitLibInitialize (
+  IN   UINTN                NumberOfProcessors,
+  IN   ARM_PROCESSOR_TABLE  *CpuInfo
+  );
+
+#endif /* MP_INITLIB_H_ */
diff --git a/ArmPkg/Library/MpInitLib/AArch64/MpFuncs.S b/ArmPkg/Library/MpInitLib/AArch64/MpFuncs.S
new file mode 100644
index 000000000000..8f7019a1c62c
--- /dev/null
+++ b/ArmPkg/Library/MpInitLib/AArch64/MpFuncs.S
@@ -0,0 +1,65 @@
+#===============================================================================
+#  Copyright (c) 2021 NUVIA Inc. All rights reserved.
+#
+#  SPDX-License-Identifier: BSD-2-Clause-Patent
+#===============================================================================
+
+.text
+.align 3
+
+#include <AsmMacroIoLibV8.h>
+#include <IndustryStandard/ArmStdSmc.h>
+
+#include "InternalMpInitLib.h"
+
+GCC_ASM_IMPORT (gApStacksBase)
+GCC_ASM_IMPORT (gProcessorIDs)
+GCC_ASM_IMPORT (ApProcedure)
+GCC_ASM_IMPORT (gApStackSize)
+
+GCC_ASM_EXPORT (ApEntryPoint)
+
+StartupAddr:        .8byte ASM_PFX(ApProcedure)
+
+// Entry-point for the AP
+// VOID
+// ApEntryPoint (
+//   VOID
+//   );
+ASM_PFX(ApEntryPoint):
+  mrs x0, mpidr_el1
+  // Mask the non-affinity bits
+  ldr x1, =0xff00ffffff
+  and x0, x0, x1
+  ldr x1, gProcessorIDs
+  mov x2, 0                   // x2 = processor index
+  mov x3, 0                   // x3 = address offset
+
+// Find index in gProcessorIDs for current processor
+1:
+  ldr x4, [x1, x3]            // x4 = gProcessorIDs + x3
+  ldr x5, =0xffffffffff
+  cmp x4, x5                  // check if we've reached the end of gProcessorIDs
+  beq ProcessorNotFound
+  add x3, x3, 8               // x3 += sizeof (*gProcessorIDs)
+  add x2, x2, 1               // x2++
+  cmp x0, x4                  // if mpidr_el1 != *(gProcessorIDs + x3) then loop
+  bne 1b
+  sub x2, x2, 1
+
+// Calculate stack address
+  // x2 contains the index for the current processor
+  ldr x0, gApStacksBase
+  ldr x1, gApStackSize
+  mul x3, x2, x1              // x3 = ProcessorIndex * gApStackSize
+  add x4, x0, x3              // x4 = gApStacksBase + x3
+  add sp, x4, x1              // sp = x4 + gApStackSize
+
+  ldr x0, StartupAddr         // ASM_PFX(ApProcedure)
+  blr x0                      // doesn't return
+
+ProcessorNotFound:
+// Turn off the processor
+  MOV32 (w0, ARM_SMC_ID_PSCI_CPU_OFF)
+  smc #0
+  b .
diff --git a/ArmPkg/Library/MpInitLib/DxeMpInitLib.inf b/ArmPkg/Library/MpInitLib/DxeMpInitLib.inf
new file mode 100644
index 000000000000..72f8fec7f323
--- /dev/null
+++ b/ArmPkg/Library/MpInitLib/DxeMpInitLib.inf
@@ -0,0 +1,52 @@
+## @file
+# MP Initialization Library instance for DXE driver.
+#
+# Copyright (c) 2021, NUVIA Inc. All rights reserved.<BR>
+#
+# SPDX-License-Identifier: BSD-2-Clause-Patent
+#
+##
+
+[Defines]
+  INF_VERSION                    = 1.29
+  BASE_NAME                      = DxeMpInitLib
+  FILE_GUID                      = c9ca773c-8ae4-4b74-82fd-f7345503294e
+  MODULE_TYPE                    = DXE_DRIVER
+  VERSION_STRING                 = 1.0
+  LIBRARY_CLASS                  = MpInitLib|DXE_DRIVER
+
+#
+# The following information is for reference only and not required by the build tools.
+#
+#  VALID_ARCHITECTURES           = AARCH64
+#
+
+[Sources.AARCH64]
+  AArch64/MpFuncs.S
+
+[Sources]
+  DxeMpLib.c
+  InternalMpInitLib.h
+
+[Packages]
+  ArmPkg/ArmPkg.dec
+  MdeModulePkg/MdeModulePkg.dec
+  MdePkg/MdePkg.dec
+
+[LibraryClasses]
+  ArmLib
+  ArmSmcLib
+  BaseLib
+  BaseMemoryLib
+  DebugLib
+  HobLib
+  MemoryAllocationLib
+  UefiBootServicesTableLib
+  UefiDriverEntryPoint
+  UefiLib
+
+[Protocols]
+  gEfiMpServiceProtocolGuid
+
+[Guids]
+  gArmMpCoreInfoGuid
diff --git a/ArmPkg/Library/MpInitLib/DxeMpLib.c b/ArmPkg/Library/MpInitLib/DxeMpLib.c
new file mode 100644
index 000000000000..b00893a5ea7a
--- /dev/null
+++ b/ArmPkg/Library/MpInitLib/DxeMpLib.c
@@ -0,0 +1,1477 @@
+/** @file
+  Construct MP Services Protocol.
+
+  Copyright (c) 2021, NUVIA Inc. All rights reserved.<BR>
+  Copyright (c) 2006 - 2012, Intel Corporation. All rights reserved.<BR>
+  Portions Copyright (c) 2011, Apple Inc. All rights reserved.
+
+  SPDX-License-Identifier: BSD-2-Clause-Patent
+
+**/
+
+#include <Ppi/ArmMpCoreInfo.h>
+#include <Library/ArmLib.h>
+#include <Library/ArmSmcLib.h>
+#include <Library/BaseMemoryLib.h>
+#include <Library/DebugLib.h>
+#include <Library/MemoryAllocationLib.h>
+#include <Library/MpInitLib.h>
+#include <Library/UefiBootServicesTableLib.h>
+#include <Library/UefiDriverEntryPoint.h>
+#include <IndustryStandard/ArmStdSmc.h>
+
+#include "InternalMpInitLib.h"
+
+#define POLL_INTERVAL_US  50000
+
+#define GET_MPIDR_AFFINITY_BITS(x)  ((x) & 0xFF00FFFFFF)
+
+#define MPIDR_MT_BIT  BIT24
+
+STATIC CPU_MP_DATA  mCpuMpData;
+STATIC BOOLEAN      mNonBlockingModeAllowed;
+UINT64              *gApStacksBase;
+UINT64              *gProcessorIDs;
+CONST UINT64        gApStackSize = AP_STACK_SIZE;
+
+/** C entry-point for the AP.
+    This function gets called from the assembly function ApEntryPoint.
+
+**/
+VOID
+ApProcedure (
+  VOID
+  )
+{
+  ARM_SMC_ARGS      Args;
+  EFI_AP_PROCEDURE  UserApProcedure;
+  VOID              *UserApParameter;
+  UINTN             ProcessorIndex;
+
+  MpInitLibWhoAmI (&mMpServicesTemplate, &ProcessorIndex);
+
+  /* Fetch the user-supplied procedure and parameter to execute */
+  UserApProcedure = mCpuMpData.CpuData[ProcessorIndex].Procedure;
+  UserApParameter = mCpuMpData.CpuData[ProcessorIndex].Parameter;
+
+  UserApProcedure (UserApParameter);
+
+  mCpuMpData.CpuData[ProcessorIndex].State = CpuStateFinished;
+
+  /* Since we're finished with this AP, turn it off */
+  Args.Arg0 = ARM_SMC_ID_PSCI_CPU_OFF;
+  ArmCallSmc (&Args);
+
+  /* Should never be reached */
+  ASSERT (FALSE);
+  CpuDeadLoop ();
+}
+
+/** Turns on the specified core using PSCI and executes the user-supplied
+    function that's been configured via a previous call to SetApProcedure.
+
+    @param ProcessorIndex The index of the core to turn on.
+
+    @retval EFI_SUCCESS      Success.
+    @retval EFI_DEVICE_ERROR The processor could not be turned on.
+
+**/
+STATIC
+EFI_STATUS
+EFIAPI
+DispatchCpu (
+  IN UINTN  ProcessorIndex
+  )
+{
+  ARM_SMC_ARGS  Args;
+  EFI_STATUS    Status;
+
+  Status = EFI_SUCCESS;
+
+  mCpuMpData.CpuData[ProcessorIndex].State = CpuStateBusy;
+
+  /* Turn the AP on */
+  if (sizeof (Args.Arg0) == sizeof (UINT32)) {
+    Args.Arg0 = ARM_SMC_ID_PSCI_CPU_ON_AARCH32;
+  } else {
+    Args.Arg0 = ARM_SMC_ID_PSCI_CPU_ON_AARCH64;
+  }
+
+  Args.Arg1 = gProcessorIDs[ProcessorIndex];
+  Args.Arg2 = (UINTN)ApEntryPoint;
+
+  ArmCallSmc (&Args);
+
+  if (Args.Arg0 != ARM_SMC_PSCI_RET_SUCCESS) {
+    DEBUG ((DEBUG_ERROR, "PSCI_CPU_ON call failed: %d", Args.Arg0));
+    Status = EFI_DEVICE_ERROR;
+  }
+
+  return Status;
+}
+
+/** Returns whether the specified processor is the BSP.
+
+  @param[in] ProcessorIndex The index the processor to check.
+
+  @return TRUE if the processor is the BSP, FALSE otherwise.
+**/
+STATIC
+BOOLEAN
+IsProcessorBSP (
+  UINTN  ProcessorIndex
+  )
+{
+  EFI_PROCESSOR_INFORMATION  *CpuInfo;
+
+  CpuInfo = &mCpuMpData.CpuData[ProcessorIndex].Info;
+
+  return (CpuInfo->StatusFlag & PROCESSOR_AS_BSP_BIT) != 0;
+}
+
+/** Returns whether the processor executing this function is the BSP.
+
+    @return Whether the current processor is the BSP.
+**/
+STATIC
+BOOLEAN
+IsCurrentProcessorBSP (
+  VOID
+  )
+{
+  EFI_STATUS  Status;
+  UINTN       ProcessorIndex;
+
+  Status = MpInitLibWhoAmI (&mMpServicesTemplate, &ProcessorIndex);
+  if (EFI_ERROR (Status)) {
+    ASSERT (0);
+    return FALSE;
+  }
+
+  return IsProcessorBSP (ProcessorIndex);
+}
+
+/** Get the Application Processors state.
+
+  @param[in]  CpuData    The pointer to CPU_AP_DATA of specified AP.
+
+  @return The AP status.
+**/
+CPU_STATE
+GetApState (
+  IN  CPU_AP_DATA  *CpuData
+  )
+{
+  return CpuData->State;
+}
+
+/** Configures the processor context with the user-supplied procedure and
+    argument.
+
+   @param CpuData           The processor context.
+   @param Procedure         The user-supplied procedure.
+   @param ProcedureArgument The user-supplied procedure argument.
+
+**/
+STATIC
+VOID
+SetApProcedure (
+  IN   CPU_AP_DATA       *CpuData,
+  IN   EFI_AP_PROCEDURE  Procedure,
+  IN   VOID              *ProcedureArgument
+  )
+{
+  ASSERT (CpuData != NULL);
+  ASSERT (Procedure != NULL);
+
+  CpuData->Parameter = ProcedureArgument;
+  CpuData->Procedure = Procedure;
+}
+
+/** Returns the index of the next processor that is blocked.
+
+   @param[out] NextNumber The index of the next blocked processor.
+
+   @retval EFI_SUCCESS   Successfully found the next blocked processor.
+   @retval EFI_NOT_FOUND There are no blocked processors.
+
+**/
+STATIC
+EFI_STATUS
+GetNextBlockedNumber (
+  OUT UINTN  *NextNumber
+  )
+{
+  UINTN        Index;
+  CPU_STATE    State;
+  CPU_AP_DATA  *CpuData;
+
+  for (Index = 0; Index < mCpuMpData.NumberOfProcessors; Index++) {
+    CpuData = &mCpuMpData.CpuData[Index];
+    if (IsProcessorBSP (Index)) {
+      // Skip BSP
+      continue;
+    }
+
+    State = CpuData->State;
+
+    if (State == CpuStateBlocked) {
+      *NextNumber = Index;
+      return EFI_SUCCESS;
+    }
+  }
+
+  return EFI_NOT_FOUND;
+}
+
+/** Stalls the BSP for the minimum of POLL_INTERVAL_US and Timeout.
+
+   @param[in]  Timeout    The time limit in microseconds remaining for
+                          APs to return from Procedure.
+
+   @retval     StallTime  Time of execution stall.
+**/
+STATIC
+UINTN
+CalculateAndStallInterval (
+  IN UINTN  Timeout
+  )
+{
+  UINTN  StallTime;
+
+  if ((Timeout < POLL_INTERVAL_US) && (Timeout != 0)) {
+    StallTime = Timeout;
+  } else {
+    StallTime = POLL_INTERVAL_US;
+  }
+
+  gBS->Stall (StallTime);
+
+  return StallTime;
+}
+
+/**
+  This service retrieves the number of logical processor in the platform
+  and the number of those logical processors that are enabled on this boot.
+  This service may only be called from the BSP.
+
+  @param[in]  This                        A pointer to the
+                                          EFI_MP_SERVICES_PROTOCOL instance.
+  @param[out] NumberOfProcessors          Pointer to the total number of logical
+                                          processors in the system, including
+                                          the BSP and disabled APs.
+  @param[out] NumberOfEnabledProcessors   Pointer to the number of enabled
+                                          logical processors that exist in the
+                                          system, including the BSP.
+
+  @retval EFI_SUCCESS             The number of logical processors and enabled
+                                  logical processors was retrieved.
+  @retval EFI_DEVICE_ERROR        The calling processor is an AP.
+  @retval EFI_INVALID_PARAMETER   NumberOfProcessors is NULL.
+  @retval EFI_INVALID_PARAMETER   NumberOfEnabledProcessors is NULL.
+
+**/
+EFI_STATUS
+EFIAPI
+MpInitLibGetNumberOfProcessors (
+  IN  EFI_MP_SERVICES_PROTOCOL  *This,
+  OUT UINTN                     *NumberOfProcessors,
+  OUT UINTN                     *NumberOfEnabledProcessors
+  )
+{
+  if ((NumberOfProcessors == NULL) || (NumberOfEnabledProcessors == NULL)) {
+    return EFI_INVALID_PARAMETER;
+  }
+
+  if (!IsCurrentProcessorBSP ()) {
+    return EFI_DEVICE_ERROR;
+  }
+
+  *NumberOfProcessors        = mCpuMpData.NumberOfProcessors;
+  *NumberOfEnabledProcessors = mCpuMpData.NumberOfEnabledProcessors;
+  return EFI_SUCCESS;
+}
+
+/**
+  Gets detailed MP-related information on the requested processor at the
+  instant this call is made. This service may only be called from the BSP.
+
+  @param[in]  This                  A pointer to the EFI_MP_SERVICES_PROTOCOL
+                                    instance.
+  @param[in]  ProcessorIndex        The index of the processor.
+  @param[out] ProcessorInfoBuffer   A pointer to the buffer where information
+                                    for the requested processor is deposited.
+
+  @retval EFI_SUCCESS             Processor information was returned.
+  @retval EFI_DEVICE_ERROR        The calling processor is an AP.
+  @retval EFI_INVALID_PARAMETER   ProcessorInfoBuffer is NULL.
+  @retval EFI_NOT_FOUND           The processor with the handle specified by
+                                  ProcessorNumber does not exist in the platform.
+
+**/
+EFI_STATUS
+EFIAPI
+MpInitLibGetProcessorInfo (
+  IN  EFI_MP_SERVICES_PROTOCOL   *This,
+  IN  UINTN                      ProcessorIndex,
+  OUT EFI_PROCESSOR_INFORMATION  *ProcessorInfoBuffer
+  )
+{
+  if (ProcessorInfoBuffer == NULL) {
+    return EFI_INVALID_PARAMETER;
+  }
+
+  if (!IsCurrentProcessorBSP ()) {
+    return EFI_DEVICE_ERROR;
+  }
+
+  ProcessorIndex &= ~CPU_V2_EXTENDED_TOPOLOGY;
+
+  if (ProcessorIndex >= mCpuMpData.NumberOfProcessors) {
+    return EFI_NOT_FOUND;
+  }
+
+  CopyMem (
+    ProcessorInfoBuffer,
+    &mCpuMpData.CpuData[ProcessorIndex],
+    sizeof (EFI_PROCESSOR_INFORMATION)
+    );
+  return EFI_SUCCESS;
+}
+
+/** Returns whether the specified processor is enabled.
+
+   @param[in] ProcessorIndex The index of the processor to check.
+
+   @return TRUE if the processor is enabled, FALSE otherwise.
+**/
+STATIC
+BOOLEAN
+IsProcessorEnabled (
+  UINTN  ProcessorIndex
+  )
+{
+  EFI_PROCESSOR_INFORMATION  *CpuInfo;
+
+  CpuInfo = &mCpuMpData.CpuData[ProcessorIndex].Info;
+
+  return (CpuInfo->StatusFlag & PROCESSOR_ENABLED_BIT) != 0;
+}
+
+/** Returns whether all processors are in the idle state.
+
+   @return Whether all the processors are idle.
+
+**/
+STATIC
+BOOLEAN
+CheckAllCpusReady (
+  VOID
+  )
+{
+  UINTN        Index;
+  CPU_AP_DATA  *CpuData;
+
+  for (Index = 0; Index < mCpuMpData.NumberOfProcessors; Index++) {
+    CpuData = &mCpuMpData.CpuData[Index];
+    if (IsProcessorBSP (Index)) {
+      // Skip BSP
+      continue;
+    }
+
+    if (!IsProcessorEnabled (Index)) {
+      // Skip Disabled processors
+      continue;
+    }
+
+    if (GetApState (CpuData) != CpuStateIdle) {
+      return FALSE;
+    }
+  }
+
+  return TRUE;
+}
+
+/** Sets up the state for the StartupAllAPs function.
+
+   @param SingleThread Whether the APs will execute sequentially.
+
+**/
+STATIC
+VOID
+StartupAllAPsPrepareState (
+  IN BOOLEAN  SingleThread
+  )
+{
+  UINTN        Index;
+  CPU_STATE    APInitialState;
+  CPU_AP_DATA  *CpuData;
+
+  mCpuMpData.FinishCount  = 0;
+  mCpuMpData.StartCount   = 0;
+  mCpuMpData.SingleThread = SingleThread;
+
+  APInitialState = CpuStateReady;
+
+  for (Index = 0; Index < mCpuMpData.NumberOfProcessors; Index++) {
+    CpuData = &mCpuMpData.CpuData[Index];
+
+    //
+    // Get APs prepared, and put failing APs into FailedCpuList.
+    // If "SingleThread", only 1 AP will put into ready state, other AP will be
+    // put into ready state 1 by 1, until the previous 1 finished its task.
+    // If not "SingleThread", all APs are put into ready state from the
+    // beginning
+    //
+
+    if (IsProcessorBSP (Index)) {
+      // Skip BSP
+      continue;
+    }
+
+    if (!IsProcessorEnabled (Index)) {
+      // Skip Disabled processors
+      if (mCpuMpData.FailedList != NULL) {
+        mCpuMpData.FailedList[mCpuMpData.FailedListIndex++] = Index;
+      }
+
+      continue;
+    }
+
+    ASSERT (GetApState (CpuData) == CpuStateIdle);
+    CpuData->State = APInitialState;
+
+    mCpuMpData.StartCount++;
+    if (SingleThread) {
+      APInitialState = CpuStateBlocked;
+    }
+  }
+}
+
+/** Handles execution of StartupAllAPs when a WaitEvent has been specified.
+
+   @param Procedure         The user-supplied procedure.
+   @param ProcedureArgument The user-supplied procedure argument.
+   @param WaitEvent         The wait event to be signaled when the work is
+                            complete or a timeout has occurred.
+   @param TimeoutInMicroseconds The timeout for the work to be completed. Zero
+                                indicates an infinite timeout.
+
+   @return EFI_SUCCESS on success.
+**/
+STATIC
+EFI_STATUS
+StartupAllAPsWithWaitEvent (
+  IN EFI_AP_PROCEDURE  Procedure,
+  IN VOID              *ProcedureArgument,
+  IN EFI_EVENT         WaitEvent,
+  IN UINTN             TimeoutInMicroseconds
+  )
+{
+  EFI_STATUS   Status;
+  UINTN        Index;
+  CPU_AP_DATA  *CpuData;
+
+  for (Index = 0; Index < mCpuMpData.NumberOfProcessors; Index++) {
+    CpuData = &mCpuMpData.CpuData[Index];
+    if (IsProcessorBSP (Index)) {
+      // Skip BSP
+      continue;
+    }
+
+    if (!IsProcessorEnabled (Index)) {
+      // Skip Disabled processors
+      continue;
+    }
+
+    if (GetApState (CpuData) == CpuStateReady) {
+      SetApProcedure (CpuData, Procedure, ProcedureArgument);
+    }
+  }
+
+  //
+  // Save data into private data structure, and create timer to poll AP state
+  // before exiting
+  //
+  mCpuMpData.Procedure         = Procedure;
+  mCpuMpData.ProcedureArgument = ProcedureArgument;
+  mCpuMpData.WaitEvent         = WaitEvent;
+  mCpuMpData.Timeout           = TimeoutInMicroseconds;
+  mCpuMpData.TimeoutActive     = (BOOLEAN)(TimeoutInMicroseconds != 0);
+  Status                       = gBS->SetTimer (
+                                        mCpuMpData.CheckAllAPsEvent,
+                                        TimerPeriodic,
+                                        POLL_INTERVAL_US
+                                        );
+  return Status;
+}
+
+/** Handles execution of StartupAllAPs when no wait event has been specified.
+
+   @param Procedure             The user-supplied procedure.
+   @param ProcedureArgument     The user-supplied procedure argument.
+   @param TimeoutInMicroseconds The timeout for the work to be completed. Zero
+                                indicates an infinite timeout.
+   @param SingleThread          Whether the APs will execute sequentially.
+   @param FailedCpuList         User-supplied pointer for list of failed CPUs.
+
+   @return EFI_SUCCESS on success.
+**/
+STATIC
+EFI_STATUS
+StartupAllAPsNoWaitEvent (
+  IN EFI_AP_PROCEDURE  Procedure,
+  IN VOID              *ProcedureArgument,
+  IN UINTN             TimeoutInMicroseconds,
+  IN BOOLEAN           SingleThread,
+  IN UINTN             **FailedCpuList
+  )
+{
+  EFI_STATUS   Status;
+  UINTN        Index;
+  UINTN        NextIndex;
+  UINTN        Timeout;
+  CPU_AP_DATA  *CpuData;
+
+  Timeout = TimeoutInMicroseconds;
+
+  while (TRUE) {
+    for (Index = 0; Index < mCpuMpData.NumberOfProcessors; Index++) {
+      CpuData = &mCpuMpData.CpuData[Index];
+      if (IsProcessorBSP (Index)) {
+        // Skip BSP
+        continue;
+      }
+
+      if (!IsProcessorEnabled (Index)) {
+        // Skip Disabled processors
+        continue;
+      }
+
+      switch (GetApState (CpuData)) {
+        case CpuStateReady:
+          SetApProcedure (CpuData, Procedure, ProcedureArgument);
+          Status = DispatchCpu (Index);
+          if (EFI_ERROR (Status)) {
+            CpuData->State = CpuStateIdle;
+            Status         = EFI_NOT_READY;
+            goto Done;
+          }
+
+          break;
+
+        case CpuStateFinished:
+          mCpuMpData.FinishCount++;
+          if (SingleThread) {
+            Status = GetNextBlockedNumber (&NextIndex);
+            if (!EFI_ERROR (Status)) {
+              mCpuMpData.CpuData[NextIndex].State = CpuStateReady;
+            }
+          }
+
+          CpuData->State = CpuStateIdle;
+          break;
+
+        default:
+          break;
+      }
+    }
+
+    if (mCpuMpData.FinishCount == mCpuMpData.StartCount) {
+      Status = EFI_SUCCESS;
+      goto Done;
+    }
+
+    if ((TimeoutInMicroseconds != 0) && (Timeout == 0)) {
+      Status = EFI_TIMEOUT;
+      goto Done;
+    }
+
+    Timeout -= CalculateAndStallInterval (Timeout);
+  }
+
+Done:
+  if (FailedCpuList != NULL) {
+    if (mCpuMpData.FailedListIndex == 0) {
+      FreePool (*FailedCpuList);
+      *FailedCpuList = NULL;
+    }
+  }
+
+  return Status;
+}
+
+/**
+  This service executes a caller provided function on all enabled APs. APs can
+  run either simultaneously or one at a time in sequence. This service supports
+  both blocking and non-blocking requests. The non-blocking requests use EFI
+  events so the BSP can detect when the APs have finished. This service may only
+  be called from the BSP.
+
+  @param[in]  This                    A pointer to the EFI_MP_SERVICES_PROTOCOL
+                                      instance.
+  @param[in]  Procedure               A pointer to the function to be run on
+                                      enabled APs of the system. See type
+                                      EFI_AP_PROCEDURE.
+  @param[in]  SingleThread            If TRUE, then all the enabled APs execute
+                                      the function specified by Procedure one by
+                                      one, in ascending order of processor
+                                      handle number.  If FALSE, then all the
+                                      enabled APs execute the function specified
+                                      by Procedure simultaneously.
+  @param[in]  WaitEvent               The event created by the caller with
+                                      CreateEvent() service.  If it is NULL,
+                                      then execute in blocking mode. BSP waits
+                                      until all APs finish or
+                                      TimeoutInMicroseconds expires.  If it's
+                                      not NULL, then execute in non-blocking
+                                      mode. BSP requests the function specified
+                                      by Procedure to be started on all the
+                                      enabled APs, and go on executing
+                                      immediately. If all return from Procedure,
+                                      or TimeoutInMicroseconds expires, this
+                                      event is signaled. The BSP can use the
+                                      CheckEvent() or WaitForEvent()
+                                      services to check the state of event. Type
+                                      EFI_EVENT is defined in CreateEvent() in
+                                      the Unified Extensible Firmware Interface
+                                      Specification.
+  @param[in]  TimeoutInMicroseconds   Indicates the time limit in microseconds
+                                      for APs to return from Procedure, either
+                                      for blocking or non-blocking mode. Zero
+                                      means infinity.  If the timeout expires
+                                      before all APs return from Procedure, then
+                                      Procedure on the failed APs is terminated.
+                                      All enabled APs are available for next
+                                      function assigned by
+                                      EFI_MP_SERVICES_PROTOCOL.StartupAllAPs()
+                                      or EFI_MP_SERVICES_PROTOCOL.StartupThisAP().
+                                      If the timeout expires in blocking mode,
+                                      BSP returns EFI_TIMEOUT.  If the timeout
+                                      expires in non-blocking mode, WaitEvent
+                                      is signaled with SignalEvent().
+  @param[in]  ProcedureArgument       The parameter passed into Procedure for
+                                      all APs.
+  @param[out] FailedCpuList           If NULL, this parameter is ignored.
+                                      Otherwise, if all APs finish successfully,
+                                      then its content is set to NULL. If not
+                                      all APs finish before timeout expires,
+                                      then its content is set to address of the
+                                      buffer holding handle numbers of the
+                                      failed APs.
+                                      The buffer is allocated by MP Service
+                                      Protocol, and it's the caller's
+                                      responsibility to free the buffer with
+                                      FreePool() service.
+                                      In blocking mode, it is ready for
+                                      consumption when the call returns. In
+                                      non-blocking mode, it is ready when
+                                      WaitEvent is signaled. The list of failed
+                                      CPU is terminated by  END_OF_CPU_LIST.
+
+  @retval EFI_SUCCESS             In blocking mode, all APs have finished before
+                                  the timeout expired.
+  @retval EFI_SUCCESS             In non-blocking mode, function has been
+                                  dispatched to all enabled APs.
+  @retval EFI_UNSUPPORTED         A non-blocking mode request was made after the
+                                  UEFI event EFI_EVENT_GROUP_READY_TO_BOOT was
+                                  signaled.
+  @retval EFI_DEVICE_ERROR        Caller processor is AP.
+  @retval EFI_NOT_STARTED         No enabled APs exist in the system.
+  @retval EFI_NOT_READY           Any enabled APs are busy.
+  @retval EFI_TIMEOUT             In blocking mode, the timeout expired before
+                                  all enabled APs have finished.
+  @retval EFI_INVALID_PARAMETER   Procedure is NULL.
+
+**/
+EFI_STATUS
+EFIAPI
+MpInitLibStartupAllAPs (
+  IN  EFI_MP_SERVICES_PROTOCOL  *This,
+  IN  EFI_AP_PROCEDURE          Procedure,
+  IN  BOOLEAN                   SingleThread,
+  IN  EFI_EVENT                 WaitEvent               OPTIONAL,
+  IN  UINTN                     TimeoutInMicroseconds,
+  IN  VOID                      *ProcedureArgument      OPTIONAL,
+  OUT UINTN                     **FailedCpuList         OPTIONAL
+  )
+{
+  EFI_STATUS  Status;
+
+  if (!IsCurrentProcessorBSP ()) {
+    return EFI_DEVICE_ERROR;
+  }
+
+  if (mCpuMpData.NumberOfProcessors == 1) {
+    return EFI_NOT_STARTED;
+  }
+
+  if (Procedure == NULL) {
+    return EFI_INVALID_PARAMETER;
+  }
+
+  if ((WaitEvent != NULL) && !mNonBlockingModeAllowed) {
+    return EFI_UNSUPPORTED;
+  }
+
+  if (!CheckAllCpusReady ()) {
+    return EFI_NOT_READY;
+  }
+
+  if (FailedCpuList != NULL) {
+    mCpuMpData.FailedList = AllocatePool (
+                              (mCpuMpData.NumberOfProcessors + 1) *
+                              sizeof (UINTN)
+                              );
+    if (mCpuMpData.FailedList == NULL) {
+      return EFI_OUT_OF_RESOURCES;
+    }
+
+    SetMemN (
+      mCpuMpData.FailedList,
+      (mCpuMpData.NumberOfProcessors + 1) *
+      sizeof (UINTN),
+      END_OF_CPU_LIST
+      );
+    mCpuMpData.FailedListIndex = 0;
+    *FailedCpuList             = mCpuMpData.FailedList;
+  }
+
+  StartupAllAPsPrepareState (SingleThread);
+
+  if (WaitEvent != NULL) {
+    Status = StartupAllAPsWithWaitEvent (
+               Procedure,
+               ProcedureArgument,
+               WaitEvent,
+               TimeoutInMicroseconds
+               );
+  } else {
+    Status = StartupAllAPsNoWaitEvent (
+               Procedure,
+               ProcedureArgument,
+               TimeoutInMicroseconds,
+               SingleThread,
+               FailedCpuList
+               );
+  }
+
+  return Status;
+}
+
+/**
+  This service lets the caller get one enabled AP to execute a caller-provided
+  function. The caller can request the BSP to either wait for the completion
+  of the AP or just proceed with the next task by using the EFI event mechanism.
+  See EFI_MP_SERVICES_PROTOCOL.StartupAllAPs() for more details on non-blocking
+  execution support.  This service may only be called from the BSP.
+
+  @param[in]  This                    A pointer to the EFI_MP_SERVICES_PROTOCOL
+                                      instance.
+  @param[in]  Procedure               A pointer to the function to be run on
+                                      enabled APs of the system. See type
+                                      EFI_AP_PROCEDURE.
+  @param[in]  ProcessorNumber         The handle number of the AP. The range is
+                                      from 0 to the total number of logical
+                                      processors minus 1. The total number of
+                                      logical processors can be retrieved by
+                                      EFI_MP_SERVICES_PROTOCOL.GetNumberOfProcessors().
+  @param[in]  WaitEvent               The event created by the caller with CreateEvent()
+                                      service.  If it is NULL, then execute in
+                                      blocking mode. BSP waits until all APs finish
+                                      or TimeoutInMicroseconds expires.  If it's
+                                      not NULL, then execute in non-blocking mode.
+                                      BSP requests the function specified by
+                                      Procedure to be started on all the enabled
+                                      APs, and go on executing immediately. If
+                                      all return from Procedure or TimeoutInMicroseconds
+                                      expires, this event is signaled. The BSP
+                                      can use the CheckEvent() or WaitForEvent()
+                                      services to check the state of event.  Type
+                                      EFI_EVENT is defined in CreateEvent() in
+                                      the Unified Extensible Firmware Interface
+                                      Specification.
+  @param[in]  TimeoutInMicroseconds   Indicates the time limit in microseconds for
+                                      APs to return from Procedure, either for
+                                      blocking or non-blocking mode. Zero means
+                                      infinity.  If the timeout expires before
+                                      all APs return from Procedure, then Procedure
+                                      on the failed APs is terminated. All enabled
+                                      APs are available for next function assigned
+                                      by EFI_MP_SERVICES_PROTOCOL.StartupAllAPs()
+                                      or EFI_MP_SERVICES_PROTOCOL.StartupThisAP().
+                                      If the timeout expires in blocking mode,
+                                      BSP returns EFI_TIMEOUT.  If the timeout
+                                      expires in non-blocking mode, WaitEvent
+                                      is signaled with SignalEvent().
+  @param[in]  ProcedureArgument       The parameter passed into Procedure for
+                                      all APs.
+  @param[out] Finished                If NULL, this parameter is ignored.  In
+                                      blocking mode, this parameter is ignored.
+                                      In non-blocking mode, if AP returns from
+                                      Procedure before the timeout expires, its
+                                      content is set to TRUE. Otherwise, the
+                                      value is set to FALSE. The caller can
+                                      determine if the AP returned from Procedure
+                                      by evaluating this value.
+
+  @retval EFI_SUCCESS             In blocking mode, specified AP finished before
+                                  the timeout expires.
+  @retval EFI_SUCCESS             In non-blocking mode, the function has been
+                                  dispatched to specified AP.
+  @retval EFI_UNSUPPORTED         A non-blocking mode request was made after the
+                                  UEFI event EFI_EVENT_GROUP_READY_TO_BOOT was
+                                  signaled.
+  @retval EFI_DEVICE_ERROR        The calling processor is an AP.
+  @retval EFI_TIMEOUT             In blocking mode, the timeout expired before
+                                  the specified AP has finished.
+  @retval EFI_NOT_READY           The specified AP is busy.
+  @retval EFI_NOT_FOUND           The processor with the handle specified by
+                                  ProcessorNumber does not exist.
+  @retval EFI_INVALID_PARAMETER   ProcessorNumber specifies the BSP or disabled AP.
+  @retval EFI_INVALID_PARAMETER   Procedure is NULL.
+
+**/
+EFI_STATUS
+EFIAPI
+MpInitLibStartupThisAP (
+  IN  EFI_MP_SERVICES_PROTOCOL  *This,
+  IN  EFI_AP_PROCEDURE          Procedure,
+  IN  UINTN                     ProcessorNumber,
+  IN  EFI_EVENT                 WaitEvent               OPTIONAL,
+  IN  UINTN                     TimeoutInMicroseconds,
+  IN  VOID                      *ProcedureArgument      OPTIONAL,
+  OUT BOOLEAN                   *Finished               OPTIONAL
+  )
+{
+  EFI_STATUS   Status;
+  UINTN        Timeout;
+  CPU_AP_DATA  *CpuData;
+
+  if (!IsCurrentProcessorBSP ()) {
+    return EFI_DEVICE_ERROR;
+  }
+
+  if (Procedure == NULL) {
+    return EFI_INVALID_PARAMETER;
+  }
+
+  if (ProcessorNumber >= mCpuMpData.NumberOfProcessors) {
+    return EFI_NOT_FOUND;
+  }
+
+  CpuData = &mCpuMpData.CpuData[ProcessorNumber];
+
+  if (IsProcessorBSP (ProcessorNumber)) {
+    return EFI_INVALID_PARAMETER;
+  }
+
+  if (!IsProcessorEnabled (ProcessorNumber)) {
+    return EFI_INVALID_PARAMETER;
+  }
+
+  if (GetApState (CpuData) != CpuStateIdle) {
+    return EFI_NOT_READY;
+  }
+
+  if ((WaitEvent != NULL) && !mNonBlockingModeAllowed) {
+    return EFI_UNSUPPORTED;
+  }
+
+  Timeout = TimeoutInMicroseconds;
+
+  mCpuMpData.StartCount  = 1;
+  mCpuMpData.FinishCount = 0;
+
+  SetApProcedure (
+    CpuData,
+    Procedure,
+    ProcedureArgument
+    );
+
+  Status = DispatchCpu (ProcessorNumber);
+  if (EFI_ERROR (Status)) {
+    CpuData->State = CpuStateIdle;
+    return EFI_NOT_READY;
+  }
+
+  if (WaitEvent != NULL) {
+    // Non Blocking
+    mCpuMpData.WaitEvent = WaitEvent;
+    gBS->SetTimer (
+           CpuData->CheckThisAPEvent,
+           TimerPeriodic,
+           POLL_INTERVAL_US
+           );
+    return EFI_SUCCESS;
+  }
+
+  // Blocking
+  while (TRUE) {
+    if (GetApState (CpuData) == CpuStateFinished) {
+      CpuData->State = CpuStateIdle;
+      break;
+    }
+
+    if ((TimeoutInMicroseconds != 0) && (Timeout == 0)) {
+      return EFI_TIMEOUT;
+    }
+
+    Timeout -= CalculateAndStallInterval (Timeout);
+  }
+
+  return EFI_SUCCESS;
+}
+
+/**
+  This service switches the requested AP to be the BSP from that point onward.
+  This service changes the BSP for all purposes.   This call can only be
+  performed by the current BSP.
+
+  @param[in] This              A pointer to the EFI_MP_SERVICES_PROTOCOL instance.
+  @param[in] ProcessorNumber   The handle number of AP that is to become the new
+                               BSP. The range is from 0 to the total number of
+                               logical processors minus 1. The total number of
+                               logical processors can be retrieved by
+                               EFI_MP_SERVICES_PROTOCOL.GetNumberOfProcessors().
+  @param[in] EnableOldBSP      If TRUE, then the old BSP will be listed as an
+                               enabled AP. Otherwise, it will be disabled.
+
+  @retval EFI_SUCCESS             BSP successfully switched.
+  @retval EFI_UNSUPPORTED         Switching the BSP cannot be completed prior to
+                                  this service returning.
+  @retval EFI_UNSUPPORTED         Switching the BSP is not supported.
+  @retval EFI_SUCCESS             The calling processor is an AP.
+  @retval EFI_NOT_FOUND           The processor with the handle specified by
+                                  ProcessorNumber does not exist.
+  @retval EFI_INVALID_PARAMETER   ProcessorNumber specifies the current BSP or
+                                  a disabled AP.
+  @retval EFI_NOT_READY           The specified AP is busy.
+
+**/
+EFI_STATUS
+EFIAPI
+MpInitLibSwitchBSP (
+  IN EFI_MP_SERVICES_PROTOCOL  *This,
+  IN  UINTN                    ProcessorNumber,
+  IN  BOOLEAN                  EnableOldBSP
+  )
+{
+  // Skip for now as we need switch a bunch of stack stuff around and it's
+  // complex. May not be worth it?
+  return EFI_UNSUPPORTED;
+}
+
+/**
+  This service lets the caller enable or disable an AP from this point onward.
+  This service may only be called from the BSP.
+
+  @param[in] This              A pointer to the EFI_MP_SERVICES_PROTOCOL instance.
+  @param[in] ProcessorNumber   The handle number of AP that is to become the new
+                               BSP. The range is from 0 to the total number of
+                               logical processors minus 1. The total number of
+                               logical processors can be retrieved by
+                               EFI_MP_SERVICES_PROTOCOL.GetNumberOfProcessors().
+  @param[in] EnableAP          Specifies the new state for the processor for
+                               enabled, FALSE for disabled.
+  @param[in] HealthFlag        If not NULL, a pointer to a value that specifies
+                               the new health status of the AP. This flag
+                               corresponds to StatusFlag defined in
+                               EFI_MP_SERVICES_PROTOCOL.GetProcessorInfo(). Only
+                               the PROCESSOR_HEALTH_STATUS_BIT is used. All other
+                               bits are ignored.  If it is NULL, this parameter
+                               is ignored.
+
+  @retval EFI_SUCCESS             The specified AP was enabled or disabled successfully.
+  @retval EFI_UNSUPPORTED         Enabling or disabling an AP cannot be completed
+                                  prior to this service returning.
+  @retval EFI_UNSUPPORTED         Enabling or disabling an AP is not supported.
+  @retval EFI_DEVICE_ERROR        The calling processor is an AP.
+  @retval EFI_NOT_FOUND           Processor with the handle specified by ProcessorNumber
+                                  does not exist.
+  @retval EFI_INVALID_PARAMETER   ProcessorNumber specifies the BSP.
+
+**/
+EFI_STATUS
+EFIAPI
+MpInitLibEnableDisableAP (
+  IN  EFI_MP_SERVICES_PROTOCOL  *This,
+  IN  UINTN                     ProcessorNumber,
+  IN  BOOLEAN                   EnableAP,
+  IN  UINT32                    *HealthFlag OPTIONAL
+  )
+{
+  UINTN        StatusFlag;
+  CPU_AP_DATA  *CpuData;
+
+  StatusFlag = mCpuMpData.CpuData[ProcessorNumber].Info.StatusFlag;
+  CpuData    = &mCpuMpData.CpuData[ProcessorNumber];
+
+  if (!IsCurrentProcessorBSP ()) {
+    return EFI_DEVICE_ERROR;
+  }
+
+  if (ProcessorNumber >= mCpuMpData.NumberOfProcessors) {
+    return EFI_NOT_FOUND;
+  }
+
+  if (IsProcessorBSP (ProcessorNumber)) {
+    return EFI_INVALID_PARAMETER;
+  }
+
+  if (GetApState (CpuData) != CpuStateIdle) {
+    return EFI_UNSUPPORTED;
+  }
+
+  if (EnableAP) {
+    if (!IsProcessorEnabled (ProcessorNumber)) {
+      mCpuMpData.NumberOfEnabledProcessors++;
+    }
+
+    StatusFlag |= PROCESSOR_ENABLED_BIT;
+  } else {
+    if (IsProcessorEnabled (ProcessorNumber)) {
+      mCpuMpData.NumberOfEnabledProcessors--;
+    }
+
+    StatusFlag &= ~PROCESSOR_ENABLED_BIT;
+  }
+
+  if (HealthFlag != NULL) {
+    StatusFlag &= ~PROCESSOR_HEALTH_STATUS_BIT;
+    StatusFlag |= (*HealthFlag & PROCESSOR_HEALTH_STATUS_BIT);
+  }
+
+  return EFI_SUCCESS;
+}
+
+/**
+  This return the handle number for the calling processor.  This service may be
+  called from the BSP and APs.
+
+  @param[in] This              A pointer to the EFI_MP_SERVICES_PROTOCOL instance.
+  @param[out] ProcessorNumber  The handle number of AP that is to become the new
+                               BSP. The range is from 0 to the total number of
+                               logical processors minus 1. The total number of
+                               logical processors can be retrieved by
+                               EFI_MP_SERVICES_PROTOCOL.GetNumberOfProcessors().
+
+  @retval EFI_SUCCESS             The current processor handle number was returned
+                                  in ProcessorNumber.
+  @retval EFI_INVALID_PARAMETER   ProcessorNumber is NULL.
+
+**/
+EFI_STATUS
+EFIAPI
+MpInitLibWhoAmI (
+  IN EFI_MP_SERVICES_PROTOCOL  *This,
+  OUT UINTN                    *ProcessorNumber
+  )
+{
+  UINTN        Index;
+  UINT64       ProcessorId;
+  CPU_AP_DATA  *CpuData;
+
+  if (ProcessorNumber == NULL) {
+    return EFI_INVALID_PARAMETER;
+  }
+
+  ProcessorId = GET_MPIDR_AFFINITY_BITS (ArmReadMpidr ());
+  for (Index = 0; Index < mCpuMpData.NumberOfProcessors; Index++) {
+    CpuData = &mCpuMpData.CpuData[Index];
+    if (CpuData->Info.ProcessorId == ProcessorId) {
+      break;
+    }
+  }
+
+  *ProcessorNumber = Index;
+  return EFI_SUCCESS;
+}
+
+/** Adds the specified processor the list of failed processors.
+
+   @param ProcessorIndex The processor index to add.
+   @param ApState        Processor state.
+
+**/
+STATIC
+VOID
+AddProcessorToFailedList (
+  UINTN      ProcessorIndex,
+  CPU_STATE  ApState
+  )
+{
+  UINTN    Index;
+  BOOLEAN  Found;
+
+  Found = FALSE;
+
+  if (ApState == CpuStateIdle) {
+    return;
+  }
+
+  // If we are retrying make sure we don't double count
+  for (Index = 0; Index < mCpuMpData.FailedListIndex; Index++) {
+    if (mCpuMpData.FailedList[Index] == ProcessorIndex) {
+      Found = TRUE;
+      break;
+    }
+  }
+
+  /* If the CPU isn't already in the FailedList, add it */
+  if (!Found) {
+    mCpuMpData.FailedList[mCpuMpData.FailedListIndex++] = Index;
+  }
+}
+
+/** Handles the StartupAllAPs case where the timeout has occurred.
+
+**/
+STATIC
+VOID
+ProcessStartupAllAPsTimeout (
+  VOID
+  )
+{
+  CPU_AP_DATA  *CpuData;
+  UINTN        Index;
+
+  if (mCpuMpData.FailedList == NULL) {
+    return;
+  }
+
+  for (Index = 0; Index < mCpuMpData.NumberOfProcessors; Index++) {
+    CpuData = &mCpuMpData.CpuData[Index];
+    if (IsProcessorBSP (Index)) {
+      // Skip BSP
+      continue;
+    }
+
+    if (!IsProcessorEnabled (Index)) {
+      // Skip Disabled processors
+      continue;
+    }
+
+    CpuData = &mCpuMpData.CpuData[Index];
+    AddProcessorToFailedList (Index, GetApState (CpuData));
+  }
+}
+
+/** Updates the status of the APs.
+
+   @param[in] ProcessorIndex The index of the AP to update.
+**/
+STATIC
+VOID
+UpdateApStatus (
+  IN UINTN  ProcessorIndex
+  )
+{
+  EFI_STATUS   Status;
+  CPU_AP_DATA  *CpuData;
+  CPU_AP_DATA  *NextCpuData;
+  CPU_STATE    State;
+  UINTN        NextNumber;
+
+  CpuData = &mCpuMpData.CpuData[ProcessorIndex];
+
+  if (IsProcessorBSP (ProcessorIndex)) {
+    // Skip BSP
+    return;
+  }
+
+  if (!IsProcessorEnabled (ProcessorIndex)) {
+    // Skip Disabled processors
+    return;
+  }
+
+  State = GetApState (CpuData);
+
+  switch (State) {
+    case CpuStateFinished:
+      if (mCpuMpData.SingleThread) {
+        Status = GetNextBlockedNumber (&NextNumber);
+        if (!EFI_ERROR (Status)) {
+          NextCpuData = &mCpuMpData.CpuData[NextNumber];
+
+          NextCpuData->State = CpuStateReady;
+
+          SetApProcedure (
+            NextCpuData,
+            mCpuMpData.Procedure,
+            mCpuMpData.ProcedureArgument
+            );
+        }
+      }
+
+      CpuData->State = CpuStateIdle;
+      mCpuMpData.FinishCount++;
+      break;
+
+    default:
+      break;
+  }
+}
+
+/**
+  If a timeout is specified in StartupAllAps(), a timer is set, which invokes
+  this procedure periodically to check whether all APs have finished.
+
+  @param[in] Event   The WaitEvent the user supplied.
+  @param[in] Context The event context.
+**/
+STATIC
+VOID
+EFIAPI
+CheckAllAPsStatus (
+  IN  EFI_EVENT  Event,
+  IN  VOID       *Context
+  )
+{
+  UINTN  Index;
+
+  if (mCpuMpData.TimeoutActive) {
+    mCpuMpData.Timeout -= CalculateAndStallInterval (mCpuMpData.Timeout);
+  }
+
+  for (Index = 0; Index < mCpuMpData.NumberOfProcessors; Index++) {
+    UpdateApStatus (Index);
+  }
+
+  if (mCpuMpData.TimeoutActive && (mCpuMpData.Timeout == 0)) {
+    ProcessStartupAllAPsTimeout ();
+
+    // Force terminal exit
+    mCpuMpData.FinishCount = mCpuMpData.StartCount;
+  }
+
+  if (mCpuMpData.FinishCount != mCpuMpData.StartCount) {
+    return;
+  }
+
+  gBS->SetTimer (
+         mCpuMpData.CheckAllAPsEvent,
+         TimerCancel,
+         0
+         );
+
+  if (mCpuMpData.FailedListIndex == 0) {
+    if (mCpuMpData.FailedList != NULL) {
+      FreePool (mCpuMpData.FailedList);
+      mCpuMpData.FailedList = NULL;
+    }
+  }
+
+  gBS->SignalEvent (mCpuMpData.WaitEvent);
+}
+
+/** Invoked periodically via a timer to check the state of the processor.
+
+   @param Event   The event supplied by the timer expiration.
+   @param Context The processor context.
+
+**/
+STATIC
+VOID
+EFIAPI
+CheckThisAPStatus (
+  IN  EFI_EVENT  Event,
+  IN  VOID       *Context
+  )
+{
+  EFI_STATUS   Status;
+  CPU_AP_DATA  *CpuData;
+  CPU_STATE    State;
+
+  CpuData             = Context;
+  CpuData->TimeTaken += POLL_INTERVAL_US;
+
+  State = GetApState (CpuData);
+
+  if (State == CpuStateFinished) {
+    Status = gBS->SetTimer (CpuData->CheckThisAPEvent, TimerCancel, 0);
+    ASSERT_EFI_ERROR (Status);
+
+    if (mCpuMpData.WaitEvent != NULL) {
+      Status = gBS->SignalEvent (mCpuMpData.WaitEvent);
+      ASSERT_EFI_ERROR (Status);
+    }
+
+    CpuData->State = CpuStateIdle;
+  }
+
+  if (CpuData->TimeTaken > CpuData->Timeout) {
+    if (mCpuMpData.WaitEvent != NULL) {
+      Status = gBS->SignalEvent (mCpuMpData.WaitEvent);
+      ASSERT_EFI_ERROR (Status);
+    }
+  }
+}
+
+/**
+  This function is called by all processors (both BSP and AP) once and collects
+  MP related data.
+
+  @param BSP            TRUE if the processor is the BSP.
+  @param Mpidr          The MPIDR for the specified processor. This should be
+                        the full MPIDR and not only the affinity bits.
+  @param ProcessorIndex The index of the processor.
+
+  @return EFI_SUCCESS if the data for the processor collected and filled in.
+
+**/
+STATIC
+EFI_STATUS
+FillInProcessorInformation (
+  IN BOOLEAN  BSP,
+  IN UINTN    Mpidr,
+  IN UINTN    ProcessorIndex
+  )
+{
+  EFI_PROCESSOR_INFORMATION  *CpuInfo;
+
+  CpuInfo = &mCpuMpData.CpuData[ProcessorIndex].Info;
+
+  CpuInfo->ProcessorId = GET_MPIDR_AFFINITY_BITS (Mpidr);
+  CpuInfo->StatusFlag  = PROCESSOR_ENABLED_BIT | PROCESSOR_HEALTH_STATUS_BIT;
+
+  if (BSP) {
+    CpuInfo->StatusFlag |= PROCESSOR_AS_BSP_BIT;
+  }
+
+  if ((Mpidr & MPIDR_MT_BIT) > 0) {
+    CpuInfo->Location.Package = GET_MPIDR_AFF2 (Mpidr);
+    CpuInfo->Location.Core    = GET_MPIDR_AFF1 (Mpidr);
+    CpuInfo->Location.Thread  = GET_MPIDR_AFF0 (Mpidr);
+
+    CpuInfo->ExtendedInformation.Location2.Package = GET_MPIDR_AFF3 (Mpidr);
+    CpuInfo->ExtendedInformation.Location2.Die     = GET_MPIDR_AFF2 (Mpidr);
+    CpuInfo->ExtendedInformation.Location2.Core    = GET_MPIDR_AFF1 (Mpidr);
+    CpuInfo->ExtendedInformation.Location2.Thread  = GET_MPIDR_AFF0 (Mpidr);
+  } else {
+    CpuInfo->Location.Package = GET_MPIDR_AFF1 (Mpidr);
+    CpuInfo->Location.Core    = GET_MPIDR_AFF0 (Mpidr);
+    CpuInfo->Location.Thread  = 0;
+
+    CpuInfo->ExtendedInformation.Location2.Package = GET_MPIDR_AFF2 (Mpidr);
+    CpuInfo->ExtendedInformation.Location2.Die     = GET_MPIDR_AFF1 (Mpidr);
+    CpuInfo->ExtendedInformation.Location2.Core    = GET_MPIDR_AFF0 (Mpidr);
+    CpuInfo->ExtendedInformation.Location2.Thread  = 0;
+  }
+
+  CpuInfo->ExtendedInformation.Location2.Package = 0;
+  CpuInfo->ExtendedInformation.Location2.Module  = 0;
+  CpuInfo->ExtendedInformation.Location2.Tile    = 0;
+
+  mCpuMpData.CpuData[ProcessorIndex].State = BSP ? CpuStateBusy : CpuStateIdle;
+
+  mCpuMpData.CpuData[ProcessorIndex].Procedure = NULL;
+  mCpuMpData.CpuData[ProcessorIndex].Parameter = NULL;
+
+  return EFI_SUCCESS;
+}
+
+/** Initializes the MP Services system data
+
+   @param NumberOfProcessors The number of processors, both BSP and AP.
+   @param CpuInfo            CPU information gathered earlier during boot.
+
+**/
+VOID
+MpInitLibInitialize (
+  IN   UINTN                NumberOfProcessors,
+  IN   ARM_PROCESSOR_TABLE  *CpuInfo
+  )
+{
+  EFI_STATUS  Status;
+  UINTN       Index;
+  EFI_EVENT   ReadyToBootEvent;
+
+  //
+  // Clear the data structure area first.
+  //
+  ZeroMem (&mCpuMpData, sizeof (CPU_MP_DATA));
+  //
+  // First BSP fills and inits all known values, including its own records.
+  //
+  mCpuMpData.NumberOfProcessors        = NumberOfProcessors;
+  mCpuMpData.NumberOfEnabledProcessors = NumberOfProcessors;
+
+  mCpuMpData.CpuData = AllocateZeroPool (
+                         mCpuMpData.NumberOfProcessors *
+                         sizeof (CPU_AP_DATA)
+                         );
+  ASSERT (mCpuMpData.CpuData != NULL);
+
+  /* Allocate one extra for the NULL entry at the end */
+  gProcessorIDs = AllocatePool ((mCpuMpData.NumberOfProcessors + 1) * sizeof (UINT64));
+  ASSERT (gProcessorIDs != NULL);
+
+  FillInProcessorInformation (TRUE, CpuInfo->ArmCpus[0].Mpidr, 0);
+  gProcessorIDs[0] = mCpuMpData.CpuData[0].Info.ProcessorId;
+
+  Status = gBS->CreateEvent (
+                  EVT_TIMER | EVT_NOTIFY_SIGNAL,
+                  TPL_CALLBACK,
+                  CheckAllAPsStatus,
+                  NULL,
+                  &mCpuMpData.CheckAllAPsEvent
+                  );
+  ASSERT_EFI_ERROR (Status);
+
+  gApStacksBase = AllocatePool (
+                    mCpuMpData.NumberOfProcessors *
+                    gApStackSize
+                    );
+  ASSERT (gApStacksBase != NULL);
+
+  for (Index = 0; Index < mCpuMpData.NumberOfProcessors; Index++) {
+    if (IsProcessorBSP (Index)) {
+      /* Skip BSP */
+      continue;
+    }
+
+    FillInProcessorInformation (FALSE, CpuInfo->ArmCpus[Index].Mpidr, Index);
+
+    gProcessorIDs[Index] = mCpuMpData.CpuData[Index].Info.ProcessorId;
+
+    Status = gBS->CreateEvent (
+                    EVT_TIMER | EVT_NOTIFY_SIGNAL,
+                    TPL_CALLBACK,
+                    CheckThisAPStatus,
+                    (VOID *)&mCpuMpData.CpuData[Index],
+                    &mCpuMpData.CpuData[Index].CheckThisAPEvent
+                    );
+    ASSERT (Status == EFI_SUCCESS);
+  }
+
+  Status = EfiCreateEventReadyToBootEx (
+             TPL_CALLBACK,
+             ReadyToBootSignaled,
+             NULL,
+             &ReadyToBootEvent
+             );
+  ASSERT_EFI_ERROR (Status);
+
+  gProcessorIDs[Index] = MAX_UINT32;
+}
+
+/**
+  Event notification function called when the EFI_EVENT_GROUP_READY_TO_BOOT is
+  signaled. After this point, non-blocking mode is no longer allowed.
+
+  @param  Event     Event whose notification function is being invoked.
+  @param  Context   The pointer to the notification function's context,
+                    which is implementation-dependent.
+
+**/
+STATIC
+VOID
+EFIAPI
+ReadyToBootSignaled (
+  IN  EFI_EVENT  Event,
+  IN  VOID       *Context
+  )
+{
+  mNonBlockingModeAllowed = FALSE;
+}
diff --git a/ArmPkg/Library/MpInitLib/InternalMpInitLib.h b/ArmPkg/Library/MpInitLib/InternalMpInitLib.h
new file mode 100644
index 000000000000..ccae24ecaf4b
--- /dev/null
+++ b/ArmPkg/Library/MpInitLib/InternalMpInitLib.h
@@ -0,0 +1,357 @@
+/** @file
+
+Copyright (c) 2021, NUVIA Inc. All rights reserved.<BR>
+Copyright (c) 2006 - 2011, Intel Corporation. All rights reserved.<BR>
+Portions copyright (c) 2011, Apple Inc. All rights reserved.
+
+SPDX-License-Identifier: BSD-2-Clause-Patent
+
+**/
+
+#ifndef MP_INTERNAL_INIT_LIB_H_
+#define MP_INTERNAL_INIT_LIB_H_
+
+#include <Protocol/Cpu.h>
+#include <Protocol/MpService.h>
+
+#include <Library/BaseLib.h>
+#include <Library/UefiLib.h>
+
+#define AP_STACK_SIZE  0x1000
+
+//
+// Internal Data Structures
+//
+
+//
+// AP state
+//
+// The state transitions for an AP when it process a procedure are:
+//  Idle ----> Ready ----> Busy ----> Idle
+//       [BSP]       [AP]       [AP]
+//
+typedef enum {
+  CpuStateIdle,
+  CpuStateReady,
+  CpuStateBlocked,
+  CpuStateBusy,
+  CpuStateFinished,
+  CpuStateDisabled
+} CPU_STATE;
+
+//
+// Define Individual Processor Data block.
+//
+typedef struct {
+  EFI_PROCESSOR_INFORMATION    Info;
+  EFI_AP_PROCEDURE             Procedure;
+  VOID                         *Parameter;
+  CPU_STATE                    State;
+  EFI_EVENT                    CheckThisAPEvent;
+  UINTN                        Timeout;
+  UINTN                        TimeTaken;
+} CPU_AP_DATA;
+
+//
+// Define MP data block which consumes individual processor block.
+//
+typedef struct {
+  UINTN               NumberOfProcessors;
+  UINTN               NumberOfEnabledProcessors;
+  EFI_EVENT           CheckAllAPsEvent;
+  EFI_EVENT           WaitEvent;
+  UINTN               FinishCount;
+  UINTN               StartCount;
+  EFI_AP_PROCEDURE    Procedure;
+  VOID                *ProcedureArgument;
+  BOOLEAN             SingleThread;
+  UINTN               StartedNumber;
+  CPU_AP_DATA         *CpuData;
+  UINTN               Timeout;
+  UINTN               *FailedList;
+  UINTN               FailedListIndex;
+  BOOLEAN             TimeoutActive;
+} CPU_MP_DATA;
+
+EFI_STATUS
+EFIAPI
+CpuMpServicesInit (
+  IN EFI_HANDLE        ImageHandle,
+  IN EFI_SYSTEM_TABLE  *SystemTable
+  );
+
+extern EFI_MP_SERVICES_PROTOCOL  mMpServicesTemplate;
+
+/** Secondary core entry point.
+
+**/
+VOID
+ApEntryPoint (
+  VOID
+  );
+
+/** C entry-point for the AP.
+    This function gets called from the assembly function ApEntryPoint.
+**/
+VOID
+ApProcedure (
+  VOID
+  );
+
+/** Turns on the specified core using PSCI and executes the user-supplied
+    function that's been configured via a previous call to SetApProcedure.
+
+   @param ProcessorIndex The index of the core to turn on.
+
+   @retval EFI_SUCCESS       The processor was successfully turned on.
+   @retval EFI_DEVICE_ERROR  An error occurred turning the processor on.
+
+**/
+STATIC
+EFI_STATUS
+EFIAPI
+DispatchCpu (
+  IN UINTN  ProcessorIndex
+  );
+
+/** Returns whether the specified processor is the BSP.
+
+   @param[in] ProcessorIndex The index the processor to check.
+
+   @return TRUE if the processor is the BSP, FALSE otherwise.
+**/
+STATIC
+BOOLEAN
+IsProcessorBSP (
+  UINTN  ProcessorIndex
+  );
+
+/** Returns whether the processor executing this function is the BSP.
+
+   @return Whether the current processor is the BSP.
+**/
+STATIC
+BOOLEAN
+IsCurrentProcessorBSP (
+  VOID
+  );
+
+/** Returns whether the specified processor is enabled.
+
+   @param[in] ProcessorIndex The index of the processor to check.
+
+   @return TRUE if the processor is enabled, FALSE otherwise.
+**/
+STATIC
+BOOLEAN
+IsProcessorEnabled (
+  UINTN  ProcessorIndex
+  );
+
+/** Configures the processor context with the user-supplied procedure and
+    argument.
+
+   @param CpuData           The processor context.
+   @param Procedure         The user-supplied procedure.
+   @param ProcedureArgument The user-supplied procedure argument.
+
+**/
+STATIC
+VOID
+SetApProcedure (
+  IN   CPU_AP_DATA       *CpuData,
+  IN   EFI_AP_PROCEDURE  Procedure,
+  IN   VOID              *ProcedureArgument
+  );
+
+/**
+  Get the Application Processors state.
+
+  @param[in]  CpuData    The pointer to CPU_AP_DATA of specified AP
+
+  @return  The AP status
+**/
+CPU_STATE
+GetApState (
+  IN  CPU_AP_DATA  *CpuData
+  );
+
+/** Returns the index of the next processor that is blocked.
+
+   @param[out] NextNumber The index of the next blocked processor.
+
+   @retval EFI_SUCCESS   Successfully found the next blocked processor.
+   @retval EFI_NOT_FOUND There are no blocked processors.
+
+**/
+STATIC
+EFI_STATUS
+GetNextBlockedNumber (
+  OUT UINTN  *NextNumber
+  );
+
+/** Stalls the BSP for the minimum of gPollInterval and Timeout.
+
+   @param[in]  Timeout    The time limit in microseconds remaining for
+                          APs to return from Procedure.
+
+   @retval     StallTime  Time of execution stall.
+**/
+STATIC
+UINTN
+CalculateAndStallInterval (
+  IN UINTN  Timeout
+  );
+
+/** Returns whether all processors are in the idle state.
+
+   @return Whether all the processors are idle.
+
+**/
+STATIC
+BOOLEAN
+CheckAllCpusReady (
+  VOID
+  );
+
+/** Sets up the state for the StartupAllAPs function.
+
+   @param SingleThread Whether the APs will execute sequentially.
+
+**/
+STATIC
+VOID
+StartupAllAPsPrepareState (
+  IN BOOLEAN  SingleThread
+  );
+
+/** Handles execution of StartupAllAPs when a WaitEvent has been specified.
+
+   @param Procedure         The user-supplied procedure.
+   @param ProcedureArgument The user-supplied procedure argument.
+   @param WaitEvent         The wait event to be signaled when the work is
+                            complete or a timeout has occurred.
+   @param TimeoutInMicroseconds The timeout for the work to be completed. Zero
+                                indicates an infinite timeout.
+
+   @return EFI_SUCCESS on success.
+**/
+STATIC
+EFI_STATUS
+StartupAllAPsWithWaitEvent (
+  IN EFI_AP_PROCEDURE  Procedure,
+  IN VOID              *ProcedureArgument,
+  IN EFI_EVENT         WaitEvent,
+  IN UINTN             TimeoutInMicroseconds
+  );
+
+/** Handles execution of StartupAllAPs when no wait event has been specified.
+
+   @param Procedure             The user-supplied procedure.
+   @param ProcedureArgument     The user-supplied procedure argument.
+   @param TimeoutInMicroseconds The timeout for the work to be completed. Zero
+                                indicates an infinite timeout.
+   @param SingleThread          Whether the APs will execute sequentially.
+   @param FailedCpuList         User-supplied pointer for list of failed CPUs.
+
+   @return EFI_SUCCESS on success.
+**/
+STATIC
+EFI_STATUS
+StartupAllAPsNoWaitEvent (
+  IN EFI_AP_PROCEDURE  Procedure,
+  IN VOID              *ProcedureArgument,
+  IN UINTN             TimeoutInMicroseconds,
+  IN BOOLEAN           SingleThread,
+  IN UINTN             **FailedCpuList
+  );
+
+/** Adds the specified processor the list of failed processors.
+
+   @param ProcessorIndex The processor index to add.
+   @param ApState         Processor state.
+
+**/
+STATIC
+VOID
+AddProcessorToFailedList (
+  UINTN      ProcessorIndex,
+  CPU_STATE  ApState
+  );
+
+/** Handles the StartupAllAPs case where the timeout has occurred.
+
+**/
+STATIC
+VOID
+ProcessStartupAllAPsTimeout (
+  VOID
+  );
+
+/**
+  If a timeout is specified in StartupAllAps(), a timer is set, which invokes
+  this procedure periodically to check whether all APs have finished.
+
+  @param[in] Event   The WaitEvent the user supplied.
+  @param[in] Context The event context.
+**/
+STATIC
+VOID
+EFIAPI
+CheckAllAPsStatus (
+  IN  EFI_EVENT  Event,
+  IN  VOID       *Context
+  );
+
+/** Invoked periodically via a timer to check the state of the processor.
+
+   @param Event   The event supplied by the timer expiration.
+   @param Context The processor context.
+
+**/
+STATIC
+VOID
+EFIAPI
+CheckThisAPStatus (
+  IN  EFI_EVENT  Event,
+  IN  VOID       *Context
+  );
+
+/**
+  This function is called by all processors (both BSP and AP) once and collects
+  MP related data.
+
+  @param BSP            TRUE if the processor is the BSP.
+  @param Mpidr          The MPIDR for the specified processor. This should be
+                        the full MPIDR and not only the affinity bits.
+  @param ProcessorIndex The index of the processor.
+
+  @return EFI_SUCCESS if the data for the processor collected and filled in.
+
+**/
+STATIC
+EFI_STATUS
+FillInProcessorInformation (
+  IN BOOLEAN  BSP,
+  IN UINTN    Mpidr,
+  IN UINTN    ProcessorIndex
+  );
+
+/**
+  Event notification function called when the EFI_EVENT_GROUP_READY_TO_BOOT is
+  signaled. After this point, non-blocking mode is no longer allowed.
+
+  @param  Event     Event whose notification function is being invoked.
+  @param  Context   The pointer to the notification function's context,
+                    which is implementation-dependent.
+
+**/
+STATIC
+VOID
+EFIAPI
+ReadyToBootSignaled (
+  IN  EFI_EVENT  Event,
+  IN  VOID       *Context
+  );
+
+#endif /* MP_INTERNAL_INIT_LIB_H_ */
-- 
2.31.1


  parent reply	other threads:[~2021-12-16  3:46 UTC|newest]

Thread overview: 11+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-12-16  3:46 [PATCH v3 0/4] ArmPkg,ArmVirtPkg: Add support EFI_MP_SERVICES_PROTOCOL on AARCH64 Rebecca Cran
2021-12-16  3:46 ` [PATCH v3 1/4] ArmPkg: Replace CoreId and ClusterId with Mpidr in ARM_CORE_INFO struct Rebecca Cran
2022-01-30 10:44   ` Ard Biesheuvel
2022-01-30 23:22     ` Ard Biesheuvel
2022-01-31 11:42       ` Ard Biesheuvel
2022-02-04 19:13         ` Rebecca Cran
2021-12-16  3:46 ` Rebecca Cran [this message]
2021-12-16  3:46 ` [PATCH v3 3/4] ArmVirtPkg: Add MpInitLib, which is dependency for CpuDxe consumers Rebecca Cran
2021-12-16  3:46 ` [PATCH v3 4/4] ArmPkg: Update Drivers/CpuDxe to initialize MpInitLib Rebecca Cran
2021-12-17 18:08 ` [PATCH v3 0/4] ArmPkg,ArmVirtPkg: Add support EFI_MP_SERVICES_PROTOCOL on AARCH64 Ard Biesheuvel
2021-12-17 18:40   ` Rebecca Cran

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-list from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20211216034634.15468-3-rebecca@nuviainc.com \
    --to=devel@edk2.groups.io \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox