Re: [RFC v3 1/3] UefiCpuPkg/CpuExceptionHandlerLib/X64: Add stack trace support

[Andrew Fish <afish@apple.com>]

> On Nov 23, 2017, at 6:27 AM, Fan Jeff <vanjeff_919@hotmail.com <mailto:vanjeff_919@hotmail.com>> wrote:
> 
> Hi Paulo,
> 
> 
> 
> I’d like to clarify my question.
> 
> The following call trace information is abstracted from your x64 dump contents.
> 
> Call trace:
> 
> 0 0x7DBCD580 @ 0x7DBCD000+0x57F (0x7EEC8DDC) in PartitionDxe.dll
> 
> 1 0x7DBD41BE @ 0x7DBCD000+0x71BD (0x7EEC8DFC) in PartitionDxe.dll
> 
> 
> 
> I guess you used CpuBreakpoint() to do your validation.
> 
> For 0x7DBCD580, it is return address followed by “int 3” from your case.
> 
> If we dump obj file, we would see the following asm code.
> 
>  000000007DBCD57F: CC                 int         3
> 
>  000000007DBCD580: C3                 ret
> 

Jeff,

Can you walk the stack without a frame pointer? I don't think the common nasm code supports that. For x86 GCC and clang use %rbp as the frame pointer. The common nasm code may be why his walk is off?

As you can see when emit frame pointer is on you get:
	pushq	%rbp
	movq	%rsp, %rbp
...
	popq	%rbp
	retq

vs. this without the frame pointer, like your example above. 
...
	retq

Without a frame pointer you need debug symbols as you don't know how deep the return address is on the stack in any given location in a function. 

Simple frame pointer example with clang:
~/work/Compiler>cat breakpoint.c
void
CpuBreakpoint (
  void
  )
{
  __asm__ __volatile__ ("int $3");
}
~/work/Compiler>clang breakpoint.c -S
~/work/Compiler>cat breakpoint.S
	.section	__TEXT,__text,regular,pure_instructions
	.macosx_version_min 10, 12
	.globl	_CpuBreakpoint
	.p2align	4, 0x90
_CpuBreakpoint:                         ## @CpuBreakpoint
	.cfi_startproc
## BB#0:
	pushq	%rbp
Lcfi0:
	.cfi_def_cfa_offset 16
Lcfi1:
	.cfi_offset %rbp, -16
	movq	%rsp, %rbp
Lcfi2:
	.cfi_def_cfa_register %rbp
	## InlineAsm Start
	int3
	## InlineAsm End
	popq	%rbp
	retq
	.cfi_endproc


.subsections_via_symbols
~/work/Compiler>

Thanks,

Andrew Fish

PS some lldb Python that walks an x86 stack frame, assuming you have a stack pointer. Given dereferencing a non-canonical addresses causes a General Protection fault it is good to error check for them if your stack walk code can not tolerate exceptions. EFI_BAD_POINTER, aka 0xAFAFAFAFAFAFAFAF, is the most common thing you hit (Thanks to Vincent Zimmer making sure EFI faults have my initials in them, not to mention the header for TE images is VZ). 

def NotCanonicalAddress(addr, start=0x00007FFFFFFFFFFF, end=0xFF800000000000000):
  return addr > start and addr < end

def print_raw_stacktrace(debugger, fp, pc, address = 0, verbose = False):
  # Do a raw stack trace
  
  stride =4  
  SBTarget  = debugger.GetSelectedTarget()
  if SBTarget:
    Triple = SBTarget.triple
    if Triple.find ("x86_64") != -1:
      stride = 8

  frame_num  = 0
  if address != 0:
    frame_addr = readPointer (debugger, address + 0)
    frame_pc   = readPointer (debugger, address + stride)
  else:
    frame_addr = fp
    frame_pc   = pc
  while frame_num < 50:
    print "0x%x: 0x%x: %s" % (frame_addr, frame_pc, disassembleInstruction (debugger, frame_pc))
    if verbose:
      str = efiSymbolicate (debugger, None, frame_pc, False)
      if str != '':
        print "  %s" % str

    frame_pc   = readPointer (debugger, frame_addr + stride)
    frame_addr = readPointer (debugger, frame_addr + 0)
    frame_num += 1
    if frame_pc == 0:
      break
    if NotCanonicalAddress(frame_pc) or NotCanonicalAddress(frame_addr):
      break
  
  return frame_num



>  (On case 0, the code at offset 0x57F is int 3)
> 
>  000000007DBD41B9: E8 AF 2A 00 00     call        CpuBreakpoint
> 
>  000000007DBD41BE: XX XX XX XX         XXX    XXXXXX
> 
>  (On case 1, what’s the code at offset 0x71BD??)
> 
> 
> 
> If the upper asm code is not correct, please copy your obj file here.
> 
> 
> 
> If the upper asm code is correct, I think we should show the return address as below, since we cannot calculate the calling IP address on most cases. (return address – 1 is not always the calling IP address on IA arch).
> 
> 0 0x7DBCD580 @ 0x7DBCD000+0x580 (0x7EEC8DDC) in PartitionDxe.dll
> 
> 1 0x7DBD41BE @ 0x7DBCD000+0x71BE (0x7EEC8DFC) in PartitionDxe.dll
> 
> 
> 
> Thanks!
> 
> Jeff
> 
> 
> 
> ________________________________
> From: Paulo Alcantara <pcacjr@zytor.com <mailto:pcacjr@zytor.com>>
> Sent: Monday, November 20, 2017 10:59:41 PM
> To: Fan Jeff; edk2-devel@lists.01.org <mailto:edk2-devel@lists.01.org>
> Cc: Laszlo Ersek; Eric Dong
> Subject: Re: 答复: [edk2] [RFC v3 1/3] UefiCpuPkg/CpuExceptionHandlerLib/X64: Add stack trace support
> 
> Hi Jeff,
> 
> (sorry for the late response)
> 
> On 11/17/2017 5:24 AM, Fan Jeff wrote:
>> Paulo,
>> 
>> 
>> I don't understand why you - 1 when calculating EIP offset in image, it
>> confused me.
> 
> That's an offset relative to the PE/COFF image base: 0 - (ImageBase +
> ImageBaseSize - 1)
> 
> Doesn't that look right to you?
> 
> Thanks
> Paulo
> 
>> 
>> 
>> +  for (;;) {
>> +    //
>> +    // Print stack frame in the following format:
>> +    //
>> +    // # <RIP> @ <ImageBase>+<RelOffset> (RBP) in [<ModuleName> | ????]
>> +    //
>> +    InternalPrintMessage (
>> +      "%d 0x%016lx @ 0x%016lx+0x%x (0x%016lx) in %a\n",
>> +      *UnwoundStacksCount - 1,
>> +      Rip,
>> +      ImageBase,
>> +      Rip - ImageBase - 1,   // ????
>> +      Rbp,
>> +      PdbFileName
>> +      );
>> +
>> 
>> Jeff
>> 
>> 
>> 
>> 
>> ------------------------------------------------------------------------
>> *发件人:* edk2-devel <edk2-devel-bounces@lists.01.org <mailto:edk2-devel-bounces@lists.01.org>> 代表 Paulo
>> Alcantara <pcacjr@zytor.com <mailto:pcacjr@zytor.com>>
>> *发送时间:* 2017年11月17日 5:56
>> *收件人:* edk2-devel@lists.01.org <mailto:edk2-devel@lists.01.org>
>> *抄送:* Laszlo Ersek; Eric Dong
>> *主题:* [edk2] [RFC v3 1/3] UefiCpuPkg/CpuExceptionHandlerLib/X64: Add
>> stack trace support
>> This patch adds stack trace support during a X64 CPU exception.
>> 
>> It will dump out back trace, stack contents as well as image module
>> names that were part of the call stack.
>> 
>> Contributed-under: TianoCore Contribution Agreement 1.1
>> Cc: Eric Dong <eric.dong@intel.com <mailto:eric.dong@intel.com>>
>> Cc: Laszlo Ersek <lersek@redhat.com <mailto:lersek@redhat.com>>
>> Signed-off-by: Paulo Alcantara <pcacjr@zytor.com <mailto:pcacjr@zytor.com>>
>> ---
>>  UefiCpuPkg/Library/CpuExceptionHandlerLib/X64/ArchExceptionHandler.c |
>> 376 +++++++++++++++++++-
>>  1 file changed, 374 insertions(+), 2 deletions(-)
>> 
>> diff --git
>> a/UefiCpuPkg/Library/CpuExceptionHandlerLib/X64/ArchExceptionHandler.c
>> b/UefiCpuPkg/Library/CpuExceptionHandlerLib/X64/ArchExceptionHandler.c
>> index 65f0cff680..fe776ccc2d 100644
>> --- a/UefiCpuPkg/Library/CpuExceptionHandlerLib/X64/ArchExceptionHandler.c
>> +++ b/UefiCpuPkg/Library/CpuExceptionHandlerLib/X64/ArchExceptionHandler.c
>> @@ -14,6 +14,11 @@
>> 
>>  #include "CpuExceptionCommon.h"
>> 
>> +//
>> +// Unknown PDB file name
>> +//
>> +GLOBAL_REMOVE_IF_UNREFERENCED CONST CHAR8 *mUnknownPdbFileName = "????";
>> +
>>  /**
>>    Return address map of exception handler template so that C code can
>> generate
>>    exception tables.
>> @@ -242,6 +247,357 @@ DumpCpuContext (
>>      );
>>  }
>> 
>> +/**
>> +  Get absolute path and file name of PDB file in PE/COFF image.
>> +
>> +  @param[in]  ImageBase            Base address of PE/COFF image.
>> +  @param[out] PdbAbsoluteFilePath  Absolute path of PDB file.
>> +  @param[out] PdbFileName          File name of PDB file.
>> +**/
>> +STATIC
>> +VOID
>> +GetPdbFileName (
>> +  IN  UINTN    ImageBase,
>> +  OUT CHAR8    **PdbAbsoluteFilePath,
>> +  OUT CHAR8    **PdbFileName
>> +  )
>> +{
>> +  VOID   *PdbPointer;
>> +  CHAR8  *Str;
>> +
>> +  //
>> +  // Get PDB file name from PE/COFF image
>> +  //
>> +  PdbPointer = PeCoffLoaderGetPdbPointer ((VOID *)ImageBase);
>> +  if (PdbPointer == NULL) {
>> +    //
>> +    // No PDB file name found. Set it to an unknown file name.
>> +    //
>> +    *PdbFileName = (CHAR8 *)mUnknownPdbFileName;
>> +    if (PdbAbsoluteFilePath != NULL) {
>> +      *PdbAbsoluteFilePath = NULL;
>> +    }
>> +  } else {
>> +    //
>> +    // Get file name portion out of PDB file in PE/COFF image
>> +    //
>> +    Str = (CHAR8 *)((UINTN)PdbPointer +
>> +                    AsciiStrLen ((CHAR8 *)PdbPointer) - sizeof *Str);
>> +    for (; *Str != '/' && *Str != '\\'; Str--) {
>> +      ;
>> +    }
>> +
>> +    //
>> +    // Set PDB file name (also skip trailing path separator: '/' or '\\')
>> +    //
>> +    *PdbFileName = Str + 1;
>> +
>> +    if (PdbAbsoluteFilePath != NULL) {
>> +      //
>> +      // Set absolute file path of PDB file
>> +      //
>> +      *PdbAbsoluteFilePath = PdbPointer;
>> +    }
>> +  }
>> +}
>> +
>> +/**
>> +  Dump stack contents.
>> +
>> +  @param[in]  CurrentRsp         Current stack pointer address.
>> +  @param[in]  UnwoundStacksCount  Count of unwound stack frames.
>> +**/
>> +STATIC
>> +VOID
>> +DumpStackContents (
>> +  IN UINT64  CurrentRsp,
>> +  IN INTN    UnwoundStacksCount
>> +  )
>> +{
>> +  //
>> +  // Check for proper stack pointer alignment
>> +  //
>> +  if (((UINTN)CurrentRsp & (CPU_STACK_ALIGNMENT - 1)) != 0) {
>> +    InternalPrintMessage ("!!!! Unaligned stack pointer. !!!!\n");
>> +    return;
>> +  }
>> +
>> +  //
>> +  // Dump out stack contents
>> +  //
>> +  InternalPrintMessage ("\nStack dump:\n");
>> +  while (UnwoundStacksCount-- > 0) {
>> +    InternalPrintMessage (
>> +      "0x%016lx: %016lx %016lx\n",
>> +      CurrentRsp,
>> +      *(UINT64 *)CurrentRsp,
>> +      *(UINT64 *)((UINTN)CurrentRsp + 8)
>> +      );
>> +
>> +    //
>> +    // Point to next stack
>> +    //
>> +    CurrentRsp += CPU_STACK_ALIGNMENT;
>> +  }
>> +}
>> +
>> +/**
>> +  Dump all image module names from call stack.
>> +
>> +  @param[in]  SystemContext  Pointer to EFI_SYSTEM_CONTEXT.
>> +**/
>> +STATIC
>> +VOID
>> +DumpImageModuleNames (
>> +  IN EFI_SYSTEM_CONTEXT   SystemContext
>> +  )
>> +{
>> +  EFI_STATUS  Status;
>> +  UINT64      Rip;
>> +  UINTN       ImageBase;
>> +  VOID        *EntryPoint;
>> +  CHAR8       *PdbAbsoluteFilePath;
>> +  CHAR8       *PdbFileName;
>> +  UINT64      Rbp;
>> +  UINTN       LastImageBase;
>> +
>> +  //
>> +  // Set current RIP address
>> +  //
>> +  Rip = SystemContext.SystemContextX64->Rip;
>> +
>> +  //
>> +  // Set current frame pointer address
>> +  //
>> +  Rbp = SystemContext.SystemContextX64->Rbp;
>> +
>> +  //
>> +  // Check for proper frame pointer alignment
>> +  //
>> +  if (((UINTN)Rbp & (CPU_STACK_ALIGNMENT - 1)) != 0) {
>> +    InternalPrintMessage ("!!!! Unaligned frame pointer. !!!!\n");
>> +    return;
>> +  }
>> +
>> +  //
>> +  // Get initial PE/COFF image base address from current RIP
>> +  //
>> +  ImageBase = PeCoffSearchImageBase (Rip);
>> +  if (ImageBase == 0) {
>> +    InternalPrintMessage ("!!!! Could not find image module names. !!!!");
>> +    return;
>> +  }
>> +
>> +  //
>> +  // Set last PE/COFF image base address
>> +  //
>> +  LastImageBase = ImageBase;
>> +
>> +  //
>> +  // Get initial PE/COFF image's entry point
>> +  //
>> +  Status = PeCoffLoaderGetEntryPoint ((VOID *)ImageBase, &EntryPoint);
>> +  if (EFI_ERROR (Status)) {
>> +    EntryPoint = NULL;
>> +  }
>> +
>> +  //
>> +  // Get file name and absolute path of initial PDB file
>> +  //
>> +  GetPdbFileName (ImageBase, &PdbAbsoluteFilePath, &PdbFileName);
>> +
>> +  //
>> +  // Print out initial image module name (if any)
>> +  //
>> +  if (PdbAbsoluteFilePath != NULL) {
>> +    InternalPrintMessage (
>> +      "\n%a (ImageBase=0x%016lx, EntryPoint=0x%016lx):\n",
>> +      PdbFileName,
>> +      ImageBase,
>> +      (UINTN)EntryPoint
>> +      );
>> +    InternalPrintMessage ("%a\n", PdbAbsoluteFilePath);
>> +  }
>> +
>> +  //
>> +  // Walk through call stack and find next module names
>> +  //
>> +  for (;;) {
>> +    //
>> +    // Set RIP with return address from current stack frame
>> +    //
>> +    Rip = *(UINT64 *)((UINTN)Rbp + 8);
>> +
>> +    //
>> +    // If RIP is zero, then stop unwinding the stack
>> +    //
>> +    if (Rip == 0) {
>> +      break;
>> +    }
>> +
>> +    //
>> +    // Search for the respective PE/COFF image based on RIP
>> +    //
>> +    ImageBase = PeCoffSearchImageBase (Rip);
>> +    if (ImageBase == 0) {
>> +      //
>> +      // Stop stack trace
>> +      //
>> +      break;
>> +    }
>> +
>> +    //
>> +    // If RIP points to another PE/COFF image, then find its respective
>> PDB file
>> +    // name.
>> +    //
>> +    if (LastImageBase != ImageBase) {
>> +      //
>> +      // Get PE/COFF image's entry point
>> +      //
>> +      Status = PeCoffLoaderGetEntryPoint ((VOID *)ImageBase, &EntryPoint);
>> +      if (EFI_ERROR (Status)) {
>> +        EntryPoint = NULL;
>> +      }
>> +
>> +      //
>> +      // Get file name and absolute path of PDB file
>> +      //
>> +      GetPdbFileName (ImageBase, &PdbAbsoluteFilePath, &PdbFileName);
>> +
>> +      //
>> +      // Print out image module name (if any)
>> +      //
>> +      if (PdbAbsoluteFilePath != NULL) {
>> +        InternalPrintMessage (
>> +          "%a (ImageBase=0x%016lx, EntryPoint=0x%016lx):\n",
>> +          PdbFileName,
>> +          ImageBase,
>> +          (UINTN)EntryPoint
>> +          );
>> +        InternalPrintMessage ("%a\n", PdbAbsoluteFilePath);
>> +      }
>> +
>> +      //
>> +      // Save last PE/COFF image base address
>> +      //
>> +      LastImageBase = ImageBase;
>> +    }
>> +
>> +    //
>> +    // Unwind the stack
>> +    //
>> +    Rbp = *(UINT64 *)(UINTN)Rbp;
>> +  }
>> +}
>> +
>> +/**
>> +  Dump stack trace.
>> +
>> +  @param[in]  SystemContext      Pointer to EFI_SYSTEM_CONTEXT.
>> +  @param[out] UnwoundStacksCount  Count of unwound stack frames.
>> +**/
>> +STATIC
>> +VOID
>> +DumpStackTrace (
>> +  IN  EFI_SYSTEM_CONTEXT   SystemContext,
>> +  OUT INTN                 *UnwoundStacksCount
>> +  )
>> +{
>> +  UINT64  Rip;
>> +  UINT64  Rbp;
>> +  UINTN   ImageBase;
>> +  CHAR8   *PdbFileName;
>> +
>> +  //
>> +  // Set current RIP address
>> +  //
>> +  Rip = SystemContext.SystemContextX64->Rip;
>> +
>> +  //
>> +  // Set current frame pointer address
>> +  //
>> +  Rbp = SystemContext.SystemContextX64->Rbp;
>> +
>> +  //
>> +  // Get initial PE/COFF image base address from current RIP
>> +  //
>> +  ImageBase = PeCoffSearchImageBase (Rip);
>> +  if (ImageBase == 0) {
>> +    InternalPrintMessage ("!!!! Could not find backtrace information.
>> !!!!");
>> +    return;
>> +  }
>> +
>> +  //
>> +  // Get PDB file name from initial PE/COFF image
>> +  //
>> +  GetPdbFileName (ImageBase, NULL, &PdbFileName);
>> +
>> +  //
>> +  // Initialize count of unwound stacks
>> +  //
>> +  *UnwoundStacksCount = 1;
>> +
>> +  //
>> +  // Print out back trace
>> +  //
>> +  InternalPrintMessage ("\nCall trace:\n");
>> +
>> +  for (;;) {
>> +    //
>> +    // Print stack frame in the following format:
>> +    //
>> +    // # <RIP> @ <ImageBase>+<RelOffset> (RBP) in [<ModuleName> | ????]
>> +    //
>> +    InternalPrintMessage (
>> +      "%d 0x%016lx @ 0x%016lx+0x%x (0x%016lx) in %a\n",
>> +      *UnwoundStacksCount - 1,
>> +      Rip,
>> +      ImageBase,
>> +      Rip - ImageBase - 1,
>> +      Rbp,
>> +      PdbFileName
>> +      );
>> +
>> +    //
>> +    // Set RIP with return address from current stack frame
>> +    //
>> +    Rip = *(UINT64 *)((UINTN)Rbp + 8);
>> +
>> +    //
>> +    // If RIP is zero, then stop unwinding the stack
>> +    //
>> +    if (Rip == 0) {
>> +      break;
>> +    }
>> +
>> +    //
>> +    // Search for the respective PE/COFF image based on RIP
>> +    //
>> +    ImageBase = PeCoffSearchImageBase (Rip);
>> +    if (ImageBase == 0) {
>> +      //
>> +      // Stop stack trace
>> +      //
>> +      break;
>> +    }
>> +
>> +    //
>> +    // Get PDB file name
>> +    //
>> +    GetPdbFileName (ImageBase, NULL, &PdbFileName);
>> +
>> +    //
>> +    // Unwind the stack
>> +    //
>> +    Rbp = *(UINT64 *)(UINTN)Rbp;
>> +
>> +    //
>> +    // Increment count of unwound stacks
>> +    //
>> +    (*UnwoundStacksCount)++;
>> +  }
>> +}
>> +
>>  /**
>>    Display CPU information.
>> 
>> @@ -254,9 +610,25 @@ DumpImageAndCpuContent (
>>    IN EFI_SYSTEM_CONTEXT   SystemContext
>>    )
>>  {
>> +  INTN UnwoundStacksCount;
>> +
>> +  //
>> +  // Dump CPU context
>> +  //
>>    DumpCpuContext (ExceptionType, SystemContext);
>> +
>> +  //
>> +  // Dump stack trace
>> +  //
>> +  DumpStackTrace (SystemContext, &UnwoundStacksCount);
>> +
>> +  //
>> +  // Dump image module names
>> +  //
>> +  DumpImageModuleNames (SystemContext);
>> +
>>    //
>> -  // Dump module image base and module entry point by RIP
>> +  // Dump stack contents
>>    //
>> -  DumpModuleImageInfo (SystemContext.SystemContextX64->Rip);
>> +  DumpStackContents (SystemContext.SystemContextX64->Rsp,
>> UnwoundStacksCount);
>>  }
>> --
>> 2.14.3
>> 
>> _______________________________________________
>> edk2-devel mailing list
>> edk2-devel@lists.01.org <mailto:edk2-devel@lists.01.org>
>> https://lists.01.org/mailman/listinfo/edk2-devel
>> 
>> edk2-devel Info Page - 01.org
>> <https://lists.01.org/mailman/listinfo/edk2-devel>
>> lists.01.org
>> Your email address: Your name (optional): You may enter a privacy
>> password below. This provides only mild security, but should prevent
>> others from messing ...
>> 
>> 
>> 
> _______________________________________________
> edk2-devel mailing list
> edk2-devel@lists.01.org <mailto:edk2-devel@lists.01.org>
> https://lists.01.org/mailman/listinfo/edk2-devel