Re: Stack issue after warm UEFI reset and MMU enabling on an Armv8 platform

[Vladimir Olovyannikov <vladimir.olovyannikov@broadcom.com>]

>From: Ard Biesheuvel [mailto:ard.biesheuvel@linaro.org]
>Sent: Wednesday, September 19, 2018 4:38 PM
>To: Vladimir Olovyannikov
>Cc: edk2-devel@lists.01.org
>Subject: Re: Stack issue after warm UEFI reset and MMU enabling on an Armv8
>platform


>On 19 September 2018 at 15:55, Vladimir Olovyannikov
><vladimir.olovyannikov@broadcom.com> wrote:
>>Hi All,

>>I need UEFI experts help on the problem with Armv8 board on warm UEFI
>>reset.
>>Cold reset works fine.

>>Here is how I set up a warm reset:

>>STATIC
>>EFI_STATUS
>>ShutdownUefiBootServices (
>>  VOID
>>  )
>>{
>>  EFI_STATUS              Status;
>>  UINTN                   MemoryMapSize;
>>  EFI_MEMORY_DESCRIPTOR   *MemoryMap;
>>  UINTN                   MapKey;
>>  UINTN                   DescriptorSize;
>>  UINT32                  DescriptorVersion;
>>  UINTN                   Pages;

>>  MemoryMap = NULL;
>>  MemoryMapSize = 0;
>>  Pages = 0;
>>
>>  do {
>>    Status = gBS->GetMemoryMap (
>>                    &MemoryMapSize,
>>                    MemoryMap,
>>                    &MapKey,
>>                    &DescriptorSize,
>>                    &DescriptorVersion
>>                    );
>>    if (Status == EFI_BUFFER_TOO_SMALL) {
>>
>>      Pages = EFI_SIZE_TO_PAGES (MemoryMapSize) + 1;
>>      MemoryMap = AllocatePages (Pages);
>>
>>      //
>>      // Get System MemoryMap
>>      //
>>      Status = gBS->GetMemoryMap (
>>                      &MemoryMapSize,
>>                      MemoryMap,
>>                      &MapKey,
>>                      &DescriptorSize,
>>                      &DescriptorVersion
>>                      );
>>    }
>>
>>    // Don't do anything between the GetMemoryMap() and ExitBootServices()
>>    if (!EFI_ERROR(Status)) {
>>      Status = gBS->ExitBootServices (gImageHandle, MapKey);
>>      if (EFI_ERROR(Status)) {
>>        FreePages (MemoryMap, Pages);
>>        MemoryMap = NULL;
>>        MemoryMapSize = 0;
>>      }
>>    }
>>  } while (EFI_ERROR(Status));
>>
>>  return Status;
>>}
>>
>>Then perform
>>ArmCleanDataCache ();
>>ArmInvalidateDataCache ();
>>ArmDisableInstructionCache ();
>>ArmInvalidateInstructionCache ();

>These don't do anything useful on ARM. You can only reliably perform cache
>maintenance by virtual address.
So, should I just remove them altogether?

>>ArmDisableMmu ();

>... so after this call returns, all bets are off with regards to whether
>what is popped from the stack is actually what we pushed when we entered
>the function.
OK, thank you for explanation.
But this call returns back into ResetLib implementation as it should, and
then there is a direct jump to the start of FV.
Am I doing anything wrong here?
Then, up to the point of enabling of MMU the stack is OK. But right after
enabling MMU it points at _ModuleEntryPoint end of function in
DxeCoreEntryPoint.c
Am I missing anything? Maybe some stack cleanup before jumping to the start
of FV?

>>Then jump to start of FV:

>>typedef
>>VOID
>> (EFIAPI *START_FV)(
>>  VOID
>>);
>>StartOfFv = (START_FV)(UINTN)PcdGet64(PcdFvBaseAddress);
>>StartOfFv ();
>>
>>Now this is what happens on warm reset:
>>reset -c warm
>>1. Until ArmEnableMmu() gets called, everything works as expected.
>>    Here is the stack right before ArmEnableMmu() is called:
>>     ArmConfigureMmu+0x4f8
>>     InitMmu+0x24
>>     MemoryPeim+0x440
>>     PrePiMain+0x114
>>     PrimaryMain+0x68
>>     CEntryPoint+0xC4
>>     EL2:0x00000000800008BC
>>     -----  End of stack info -----
>>
>>2. Here is the stack as soon as Mmu is enabled with ArmEnableMmu() :
>>    ArmConfigureMmu+0x4fc <-- This one is correct, at line 745 in
>> ArmConfigureMmu() in ArmPkg/Library/ArmMmuLib/AArch64/ArmMmuLibCore.c
>> (return EFI_SUCCESS)
>>   _ModuleEntryPoint+0x24 <-- Wrong. This points directly to
>> ASSERT(FALSE); and to CpuDeadLoop() in DxeCoreEntryPoint.c, lines 59-60.
>>   El2:0x000000008E5E8300 <-- Absolutely bogus
>>    --- End of stack info ---
>>
>>So, as soon as ArmEnableMmu() exits, execution jumps directly to
>>CpuDeadLoop() in DxeCoreEntryPoint of _ModuleEntryPoint().
>>
>>Would be grateful for any advice.
>>
>>Thank you,
>>Vladimir