From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received-SPF: Pass (sender SPF authorized) identity=mailfrom; client-ip=192.103.53.11; helo=mail5.wrs.com; envelope-from=bill.paul@windriver.com; receiver=edk2-devel@lists.01.org Received: from mail5.wrs.com (mail5.windriver.com [192.103.53.11]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ml01.01.org (Postfix) with ESMTPS id A514F21B02822 for ; Wed, 19 Sep 2018 17:21:08 -0700 (PDT) Received: from ALA-HCA.corp.ad.wrs.com (ala-hca.corp.ad.wrs.com [147.11.189.40]) by mail5.wrs.com (8.15.2/8.15.2) with ESMTPS id w8K0KE93025248 (version=TLSv1 cipher=AES128-SHA bits=128 verify=FAIL); Wed, 19 Sep 2018 17:20:25 -0700 Received: from ala-wpaul-lx1.wrs.com (147.11.157.242) by ALA-HCA.corp.ad.wrs.com (147.11.189.50) with Microsoft SMTP Server (TLS) id 14.3.408.0; Wed, 19 Sep 2018 17:20:04 -0700 From: Bill Paul Organization: Wind River Systems To: Date: Wed, 19 Sep 2018 16:56:52 -0700 User-Agent: KMail/1.13.5 (Linux/2.6.32-28-generic; KDE/4.4.5; x86_64; ; ) CC: Vladimir Olovyannikov , Ard Biesheuvel References: <0203a14be46555436db1c8d5e58064ae@mail.gmail.com> In-Reply-To: <0203a14be46555436db1c8d5e58064ae@mail.gmail.com> MIME-Version: 1.0 Message-ID: <201809191656.52349.wpaul@windriver.com> Subject: Re: Stack issue after warm UEFI reset and MMU enabling on an Armv8 platform X-BeenThere: edk2-devel@lists.01.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: EDK II Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 20 Sep 2018 00:21:08 -0000 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Of all the gin joints in all the towns in all the world, Vladimir Olovyannikov had to walk into mine at 16:58 on Wednesday 19 September 2018 and say: > >From: Ard Biesheuvel [mailto:ard.biesheuvel@linaro.org] > >Sent: Wednesday, September 19, 2018 4:38 PM > >To: Vladimir Olovyannikov > >Cc: edk2-devel@lists.01.org > >Subject: Re: Stack issue after warm UEFI reset and MMU enabling on an > >Armv8 platform > > > > > >On 19 September 2018 at 15:55, Vladimir Olovyannikov > > > > wrote: > >>Hi All, > >> > >>I need UEFI experts help on the problem with Armv8 board on warm UEFI > >>reset. > >>Cold reset works fine. > >> > >>Here is how I set up a warm reset: > >> > >>STATIC > >>EFI_STATUS > >>ShutdownUefiBootServices ( > >> > >> VOID > >> ) > >> > >>{ > >> > >> EFI_STATUS Status; > >> UINTN MemoryMapSize; > >> EFI_MEMORY_DESCRIPTOR *MemoryMap; > >> UINTN MapKey; > >> UINTN DescriptorSize; > >> UINT32 DescriptorVersion; > >> UINTN Pages; > >> > >> MemoryMap = NULL; > >> MemoryMapSize = 0; > >> Pages = 0; > >> > >> do { > >> > >> Status = gBS->GetMemoryMap ( > >> > >> &MemoryMapSize, > >> MemoryMap, > >> &MapKey, > >> &DescriptorSize, > >> &DescriptorVersion > >> ); > >> > >> if (Status == EFI_BUFFER_TOO_SMALL) { > >> > >> Pages = EFI_SIZE_TO_PAGES (MemoryMapSize) + 1; > >> MemoryMap = AllocatePages (Pages); > >> > >> // > >> // Get System MemoryMap > >> // > >> Status = gBS->GetMemoryMap ( > >> > >> &MemoryMapSize, > >> MemoryMap, > >> &MapKey, > >> &DescriptorSize, > >> &DescriptorVersion > >> ); > >> > >> } > >> > >> // Don't do anything between the GetMemoryMap() and > >> ExitBootServices() if (!EFI_ERROR(Status)) { > >> > >> Status = gBS->ExitBootServices (gImageHandle, MapKey); > >> if (EFI_ERROR(Status)) { > >> > >> FreePages (MemoryMap, Pages); > >> MemoryMap = NULL; > >> MemoryMapSize = 0; > >> > >> } > >> > >> } > >> > >> } while (EFI_ERROR(Status)); > >> > >> return Status; > >> > >>} > >> > >>Then perform > >>ArmCleanDataCache (); > >>ArmInvalidateDataCache (); > >>ArmDisableInstructionCache (); > >>ArmInvalidateInstructionCache (); > > > >These don't do anything useful on ARM. You can only reliably perform cache > >maintenance by virtual address. > > So, should I just remove them altogether? > > >>ArmDisableMmu (); > > > >... so after this call returns, all bets are off with regards to whether > >what is popped from the stack is actually what we pushed when we entered > >the function. > > OK, thank you for explanation. > But this call returns back into ResetLib implementation as it should, and > then there is a direct jump to the start of FV. > Am I doing anything wrong here? > Then, up to the point of enabling of MMU the stack is OK. But right after > enabling MMU it points at _ModuleEntryPoint end of function in > DxeCoreEntryPoint.c > Am I missing anything? Maybe some stack cleanup before jumping to the start > of FV? When the MMU is enabled, does the mapping for the stack pages change? That is, could the stack now be mapped to different physical page now? Instead of showing a stack trace, can you dump the stack pages and compare the before and after contents? Assuming the same physical memory pages are still being used, then there could be a cache flushing problem. What could happen is: - some stack memory has been touched recently and is now in the data cache - changes are made, which are written to the cache, but not yet flushed out to RAM - enabling the MMU causes a full invalidate of the cache Now when you look at the stack, you see the earlier contents that were in RAM -- the changes previously only written to the cache have been lost. Enabling/disabling caches and MMU is always tricky. I can't say for sure, but I wouldn't be surprised if there's some subtle bug that causes a flush operation to be missed and things may just work by coincidence in the cold start case. -Bill > >>Then jump to start of FV: > >> > >>typedef > >>VOID > >> > >> (EFIAPI *START_FV)( > >> > >> VOID > >> > >>); > >>StartOfFv = (START_FV)(UINTN)PcdGet64(PcdFvBaseAddress); > >>StartOfFv (); > >> > >>Now this is what happens on warm reset: > >>reset -c warm > >>1. Until ArmEnableMmu() gets called, everything works as expected. > >> > >> Here is the stack right before ArmEnableMmu() is called: > >> ArmConfigureMmu+0x4f8 > >> InitMmu+0x24 > >> MemoryPeim+0x440 > >> PrePiMain+0x114 > >> PrimaryMain+0x68 > >> CEntryPoint+0xC4 > >> EL2:0x00000000800008BC > >> ----- End of stack info ----- > >> > >>2. Here is the stack as soon as Mmu is enabled with ArmEnableMmu() : > >> ArmConfigureMmu+0x4fc <-- This one is correct, at line 745 in > >> > >> ArmConfigureMmu() in ArmPkg/Library/ArmMmuLib/AArch64/ArmMmuLibCore.c > >> (return EFI_SUCCESS) > >> > >> _ModuleEntryPoint+0x24 <-- Wrong. This points directly to > >> > >> ASSERT(FALSE); and to CpuDeadLoop() in DxeCoreEntryPoint.c, lines 59-60. > >> > >> El2:0x000000008E5E8300 <-- Absolutely bogus > >> > >> --- End of stack info --- > >> > >>So, as soon as ArmEnableMmu() exits, execution jumps directly to > >>CpuDeadLoop() in DxeCoreEntryPoint of _ModuleEntryPoint(). > >> > >>Would be grateful for any advice. > >> > >>Thank you, > >>Vladimir > > _______________________________________________ > edk2-devel mailing list > edk2-devel@lists.01.org > https://lists.01.org/mailman/listinfo/edk2-devel -- ============================================================================= -Bill Paul (510) 749-2329 | Senior Member of Technical Staff, wpaul@windriver.com | Master of Unix-Fu - Wind River Systems ============================================================================= "I put a dollar in a change machine. Nothing changed." - George Carlin =============================================================================