CLANGPDB binary debugging

CLANGPDB binary debugging

[Vitaly Cheptsov]

[-- Attachment #1.1: Type: text/plain, Size: 1167 bytes --]

Hello,

We noticed that the original bugzilla, which intended to add new LLVM toolchain support[1], also wanted to bring ELF format support with DWARF debugging information. For some reason this did not make its way into EDK II, and we are currently wondering, how can one debug binaries built with LLVM 9.0.

For macOS and XCODE5 toolchain we use GDB scripts based on Andrei Warkentin’s work, which allow us to integrate with QEMU and VMware[2]. It is likely that they should work with little to no work on Linux with CLANG38/GCC5 with GDB once again. However, CLANGPDB apparently is using PDB debugging information, which I believe is not handled with GDB.

Could you please provide the details on the matter and let us know about the recommended route?
— Is dropping CLANGELF just a temporary measure and it should be resubmitted again?
— Should LLDB, which seems to be aware of PDB, be used instead of GDB, when building with CLANGPDB? If so, did anybody try that?

Thanks!

Best regards,
Vitaly

[1] https://bugzilla.tianocore.org/show_bug.cgi?id=1603
[2] https://github.com/acidanthera/OpenCorePkg/blob/master/Debug/Scripts/gdb_uefi.py


[-- Attachment #1.2: Type: text/html, Size: 2743 bytes --]

[-- Attachment #2: Message signed with OpenPGP --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [edk2-devel] CLANGPDB binary debugging

[Andrew Fish]

[-- Attachment #1: Type: text/plain, Size: 1851 bytes --]



> On Mar 20, 2020, at 8:13 AM, Vitaly Cheptsov <cheptsov@ispras.ru> wrote:
> 
> Hello,
> 
> We noticed that the original bugzilla, which intended to add new LLVM toolchain support[1], also wanted to bring ELF format support with DWARF debugging information. For some reason this did not make its way into EDK II, and we are currently wondering, how can one debug binaries built with LLVM 9.0.
> 
> For macOS and XCODE5 toolchain we use GDB scripts based on Andrei Warkentin’s work, which allow us to integrate with QEMU and VMware[2]. It is likely that they should work with little to no work on Linux with CLANG38/GCC5 with GDB once again. However, CLANGPDB apparently is using PDB debugging information, which I believe is not handled with GDB.
> 
> Could you please provide the details on the matter and let us know about the recommended route?
> — Is dropping CLANGELF just a temporary measure and it should be resubmitted again?
> — Should LLDB, which seems to be aware of PDB, be used instead of GDB, when building with CLANGPDB? If so, did anybody try that?
> 

Vitaly,

I've not tried the CLANGPDB path, but if you want to connect lldb to QEMU you need to set  plugin.process.gdb-remote.target-definition-file [1] to [2]. 

[1]  lldb -o "settings set plugin.process.gdb-remote.target-definition-file x86_64_target_definition.py" -o "gdb-remote 9000"
[2] https://github.com/llvm-mirror/lldb/blob/master/examples/python/x86_64_target_definition.py

Thanks,

Andrew Fish

> Thanks!
> 
> Best regards,
> Vitaly
> 
> [1] https://bugzilla.tianocore.org/show_bug.cgi?id=1603 <https://bugzilla.tianocore.org/show_bug.cgi?id=1603>
> [2] https://github.com/acidanthera/OpenCorePkg/blob/master/Debug/Scripts/gdb_uefi.py <https://github.com/acidanthera/OpenCorePkg/blob/master/Debug/Scripts/gdb_uefi.py>
> 
> 


[-- Attachment #2: Type: text/html, Size: 3946 bytes --]

Re: [edk2-devel] CLANGPDB binary debugging

[Vitaly Cheptsov]

[-- Attachment #1.1: Type: text/plain, Size: 5150 bytes --]

Hello,

Andrey, thanks for the hint, it was very helpful. I rewrote the GDB scripts to work with LLDB[1] and was able to debug OVMF built with CLANGPDB. While it is still quite dirty, at the very least it works.

Unfortunately the experience was close to terrible. I may certainly do something wrong, but it is clear that PDB and LLDB do not support each other well enough. After spending several hours on playing with the tools my conclusion is that LLDB is simply not suited for UEFI PDB debugging, and we really want DWARF  as there is no other opensource debugger that supports PDB on macOS and Linux

In case somebody knows workarounds here are the issues I faced:

1. All integer alias typedefs are discarded in favour of underlying types. This way EFI_STATUS and EFI_TPL become unsigned long long, CHAR8 becomes char, and CHAR16 becomes unsigned short. It does not look like LLDB has the original types anywhere at all, and it also does not have them registered.

    frame #0: 0x000000007fe242aa DxeCore.dll`CoreAllocatePoolPagesI(PoolType=EfiBootServicesData, NoPages=1, Granularity=4096, NeedGuard='\0') at Pool.c:322
   319 	    return NULL;
   320 	  }
   321
-> 322 	  Buffer = CoreAllocatePoolPages (PoolType, NoPages, Granularity, NeedGuard);
   323 	  CoreReleaseMemoryLock ();
   324
   325 	  if (Buffer != NULL) {
(lldb) p Status
(unsigned long long) $3 = 0

Structures work more or less fine, but for simpler types like strings we are out of even potential pretty-printing.

2. Global variables are not accessible. I am not sure what happens, but they either seem to not relocate or conflict with the other names:

(lldb) p gST
error: Couldn't materialize: couldn't get the value of variable ::gST: read memory from 0x6e18 failed
error: errored out in DoExecute, couldn't PrepareToExecuteJITExpression
(lldb) p &gST
error: Couldn't materialize: couldn't get the value of variable ::gST: read memory from 0x6e18 failed
error: errored out in DoExecute, couldn't PrepareToExecuteJITExpression

3. Quite a number of crashes.

In most cases autocompletion by tab press causes a crash. E.g.

b I<TAB>

So will do printing of a GUID, e.g. p gEfiGlobalVariableGuid.

This may have to do with Python compatibility as Xcode 11 LLDB that uses Python 3 generally crashes more often than MacPorts LLDB 9.0. Surprisingly structures work more or less fine.

4. Ctrl+C does not produce a valid backtrace. When I break with a breakpoint, I see a proper stacktrace with more than one entry, with function prototypes and values. When I break with Ctrl+C I only see some weird backtrace with most of the entries missing regardless of frame position:

(lldb) bt
* thread #1, stop reason = signal SIGTRAP
  * frame #0: 0x000000007fe4c5f3 DxeCore.dll

Probably more and all the unintuitive stuff like the lack of more functional TUI, but it is hard to remember all the trials.

[1] https://github.com/acidanthera/OpenCorePkg/blob/master/Debug/Scripts/lldb_uefi.py

Best wishes,
Vitaly

> 20 марта 2020 г., в 22:14, Andrew Fish <afish@apple.com> написал(а):
> 
> 
> 
>> On Mar 20, 2020, at 8:13 AM, Vitaly Cheptsov <cheptsov@ispras.ru <mailto:cheptsov@ispras.ru>> wrote:
>> 
>> Hello,
>> 
>> We noticed that the original bugzilla, which intended to add new LLVM toolchain support[1], also wanted to bring ELF format support with DWARF debugging information. For some reason this did not make its way into EDK II, and we are currently wondering, how can one debug binaries built with LLVM 9.0.
>> 
>> For macOS and XCODE5 toolchain we use GDB scripts based on Andrei Warkentin’s work, which allow us to integrate with QEMU and VMware[2]. It is likely that they should work with little to no work on Linux with CLANG38/GCC5 with GDB once again. However, CLANGPDB apparently is using PDB debugging information, which I believe is not handled with GDB.
>> 
>> Could you please provide the details on the matter and let us know about the recommended route?
>> — Is dropping CLANGELF just a temporary measure and it should be resubmitted again?
>> — Should LLDB, which seems to be aware of PDB, be used instead of GDB, when building with CLANGPDB? If so, did anybody try that?
>> 
> 
> Vitaly,
> 
> I've not tried the CLANGPDB path, but if you want to connect lldb to QEMU you need to set  plugin.process.gdb-remote.target-definition-file [1] to [2].
> 
> [1]  lldb -o "settings set plugin.process.gdb-remote.target-definition-file x86_64_target_definition.py" -o "gdb-remote 9000"
> [2] https://github.com/llvm-mirror/lldb/blob/master/examples/python/x86_64_target_definition.py <https://github.com/llvm-mirror/lldb/blob/master/examples/python/x86_64_target_definition.py>
> 
> Thanks,
> 
> Andrew Fish
> 
>> Thanks!
>> 
>> Best regards,
>> Vitaly
>> 
>> [1] https://bugzilla.tianocore.org/show_bug.cgi?id=1603 <https://bugzilla.tianocore.org/show_bug.cgi?id=1603>
>> [2] https://github.com/acidanthera/OpenCorePkg/blob/master/Debug/Scripts/gdb_uefi.py <https://github.com/acidanthera/OpenCorePkg/blob/master/Debug/Scripts/gdb_uefi.py>
>> 
>> 


[-- Attachment #1.2: Type: text/html, Size: 14825 bytes --]

[-- Attachment #2: Message signed with OpenPGP --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [edk2-devel] CLANGPDB binary debugging

[Andrew Fish]

[-- Attachment #1: Type: text/plain, Size: 7462 bytes --]



> On Mar 21, 2020, at 3:28 AM, Vitaly Cheptsov <cheptsov@ispras.ru> wrote:
> 
> Hello,
> 
> Andrey, thanks for the hint, it was very helpful. I rewrote the GDB scripts to work with LLDB[1] and was able to debug OVMF built with CLANGPDB. While it is still quite dirty, at the very least it works.
> 
> Unfortunately the experience was close to terrible. I may certainly do something wrong, but it is clear that PDB and LLDB do not support each other well enough. After spending several hours on playing with the tools my conclusion is that LLDB is simply not suited for UEFI PDB debugging, and we really want DWARF  as there is no other opensource debugger that supports PDB on macOS and Linux
> 
> In case somebody knows workarounds here are the issues I faced:
> 
> 1. All integer alias typedefs are discarded in favour of underlying types. This way EFI_STATUS and EFI_TPL become unsigned long long, CHAR8 becomes char, and CHAR16 becomes unsigned short. It does not look like LLDB has the original types anywhere at all, and it also does not have them registered.
> 
>     frame #0: 0x000000007fe242aa DxeCore.dll`CoreAllocatePoolPagesI(PoolType=EfiBootServicesData, NoPages=1, Granularity=4096, NeedGuard='\0') at Pool.c:322
>    319 	    return NULL;
>    320 	  }
>    321 	
> -> 322 	  Buffer = CoreAllocatePoolPages (PoolType, NoPages, Granularity, NeedGuard);
>    323 	  CoreReleaseMemoryLock ();
>    324 	
>    325 	  if (Buffer != NULL) {
> (lldb) p Status
> (unsigned long long) $3 = 0
> 
> Structures work more or less fine, but for simpler types like strings we are out of even potential pretty-printing.
> 

Vitaly,

You can teach lldb about types. There is some example code here: https://github.com/tianocore/edk2/blob/master/EmulatorPkg/Unix/lldbefi.py

> 2. Global variables are not accessible. I am not sure what happens, but they either seem to not relocate or conflict with the other names:
> 
> (lldb) p gST
> error: Couldn't materialize: couldn't get the value of variable ::gST: read memory from 0x6e18 failed
> error: errored out in DoExecute, couldn't PrepareToExecuteJITExpression
> (lldb) p &gST
> error: Couldn't materialize: couldn't get the value of variable ::gST: read memory from 0x6e18 failed
> error: errored out in DoExecute, couldn't PrepareToExecuteJITExpression
> 

That is strange as globals usually work best? The common issue I've seen is getting the slide wrong. The EFI modules are linked at a value near zero and relocated into memory, so the slide represents that adjustment. 

You can use `image dump sections` and ` image dump symtab` to see lldb's view of symbols. More info here [1]. 

> 3. Quite a number of crashes.
> 
> In most cases autocompletion by tab press causes a crash. E.g.
> 
> b I<TAB>
> 
> So will do printing of a GUID, e.g. p gEfiGlobalVariableGuid.
> 
> This may have to do with Python compatibility as Xcode 11 LLDB that uses Python 3 generally crashes more often than MacPorts LLDB 9.0. Surprisingly structures work more or less fine.
> 

You can tell lldb to use the older Python like this (from the Terminal.app):
$ defaults write com.apple.dt.lldb DefaultPythonVersion 2

> 4. Ctrl+C does not produce a valid backtrace. When I break with a breakpoint, I see a proper stacktrace with more than one entry, with function prototypes and values. When I break with Ctrl+C I only see some weird backtrace with most of the entries missing regardless of frame position:
> 
> (lldb) bt
> * thread #1, stop reason = signal SIGTRAP
>   * frame #0: 0x000000007fe4c5f3 DxeCore.dll
> 
> Probably more and all the unintuitive stuff like the lack of more functional TUI, but it is hard to remember all the trials.
> 

For the macOS API clang emits frame pointers, so you can walk the stack without symbols. You could try adding the compiler flag to emit the frame pointers. 


> [1] https://github.com/acidanthera/OpenCorePkg/blob/master/Debug/Scripts/lldb_uefi.py <https://github.com/acidanthera/OpenCorePkg/blob/master/Debug/Scripts/lldb_uefi.py>
> 

On macOS the Mach-O and dSYM have a UUID (dwarfdump -u) that is indexed by Spotlight (mdfind "com_apple_xcode_dsym_uuids == *") [2]
This should be the UUID in the debug directory entry and you can use that to lookup the symbols like this:

module = target.AddModule (None, None, uuid)
SBError = target.SetModuleLoadAddress (module, LoadAddress + TeAdjust)

Also lldb has built in help for commands, but it is kind of terse since it is autogenerated from the C++ swig. 
(lldb) script help (lldb.target.AddModule)
Help on method AddModule in module lldb:

AddModule(self, *args) method of lldb.SBTarget instance
    AddModule(SBTarget self, SBModule module) -> bool
    AddModule(SBTarget self, char const * path, char const * triple, char const * uuid) -> SBModule
    AddModule(SBTarget self, char const * path, char const * triple, char const * uuid_cstr, char const * symfile) -> SBModule
    AddModule(SBTarget self, SBModuleSpec module_spec) -> SBModule

The minimum  you need to symbolicate a frame is uuid, LoadAddress, and PC. 

[1] http://lldb.llvm.org/use/map.html
[2] http://lldb.llvm.org/use/symbols.html

Thanks,

Andrew Fish


> Best wishes,
> Vitaly
> 
>> 20 марта 2020 г., в 22:14, Andrew Fish <afish@apple.com <mailto:afish@apple.com>> написал(а):
>> 
>> 
>> 
>>> On Mar 20, 2020, at 8:13 AM, Vitaly Cheptsov <cheptsov@ispras.ru <mailto:cheptsov@ispras.ru>> wrote:
>>> 
>>> Hello,
>>> 
>>> We noticed that the original bugzilla, which intended to add new LLVM toolchain support[1], also wanted to bring ELF format support with DWARF debugging information. For some reason this did not make its way into EDK II, and we are currently wondering, how can one debug binaries built with LLVM 9.0.
>>> 
>>> For macOS and XCODE5 toolchain we use GDB scripts based on Andrei Warkentin’s work, which allow us to integrate with QEMU and VMware[2]. It is likely that they should work with little to no work on Linux with CLANG38/GCC5 with GDB once again. However, CLANGPDB apparently is using PDB debugging information, which I believe is not handled with GDB.
>>> 
>>> Could you please provide the details on the matter and let us know about the recommended route?
>>> — Is dropping CLANGELF just a temporary measure and it should be resubmitted again?
>>> — Should LLDB, which seems to be aware of PDB, be used instead of GDB, when building with CLANGPDB? If so, did anybody try that?
>>> 
>> 
>> Vitaly,
>> 
>> I've not tried the CLANGPDB path, but if you want to connect lldb to QEMU you need to set  plugin.process.gdb-remote.target-definition-file [1] to [2]. 
>> 
>> [1]  lldb -o "settings set plugin.process.gdb-remote.target-definition-file x86_64_target_definition.py" -o "gdb-remote 9000"
>> [2] https://github.com/llvm-mirror/lldb/blob/master/examples/python/x86_64_target_definition.py <https://github.com/llvm-mirror/lldb/blob/master/examples/python/x86_64_target_definition.py>
>> 
>> Thanks,
>> 
>> Andrew Fish
>> 
>>> Thanks!
>>> 
>>> Best regards,
>>> Vitaly
>>> 
>>> [1] https://bugzilla.tianocore.org/show_bug.cgi?id=1603 <https://bugzilla.tianocore.org/show_bug.cgi?id=1603>
>>> [2] https://github.com/acidanthera/OpenCorePkg/blob/master/Debug/Scripts/gdb_uefi.py <https://github.com/acidanthera/OpenCorePkg/blob/master/Debug/Scripts/gdb_uefi.py>
>>> 
> 
> 


[-- Attachment #2: Type: text/html, Size: 32797 bytes --]

Re: [edk2-devel] CLANGPDB binary debugging

[Vitaly Cheptsov]

[-- Attachment #1.1: Type: text/plain, Size: 21635 bytes --]

Andrew,

Thanks once again, but unfortunately it is not that simple. Below I answered inline explaining the particular issues, which mostly seem to be specific to CLANGPDB. LLVM stack emits PDB debug files, and even though LLDB does support them to some level, it is unlikely that this will be working well enough soon. We should really stick to more or less native debug formats, ideally those that have proper open specifications, on all platforms, and for Unix that’s DWARF.

I am pretty sure LLVM can be taught to emit DWARF debug information even for PE files. Perhaps we can either make some option or provide a separate toolchain for this? Another way would be recovering CLANGELF as originally suggested.

> You can teach lldb about types. There is some example code here: https://github.com/tianocore/edk2/blob/master/EmulatorPkg/Unix/lldbefi.py <https://github.com/tianocore/edk2/blob/master/EmulatorPkg/Unix/lldbefi.py>
This code works just fine with LLDB and DWARF (e.g. XCODE5), though I have not yet completed these changes for my scripts for LLDB, only for GDB. However, with CLANGPDB generated files it is not functional. The reason for this is because LLDB is unaware of the underlying type, i.e. it does not know what is EFI_STATUS or UINT32. I can implement pretty-printing when LLDB knows about a typedef, but it is not possible to do this when the debug information is already gone or not parsed:

(lldb) p Status
(unsigned long long) $1 = 0
(lldb) p &Status
(unsigned long long *) $2 = 0x000000007fe19ad8
(lldb) p (EFI_STATUS)Status
error: use of undeclared identifier 'EFI_STATUS'

Just in case I tried using exactly your code, and other stuff like source level debugging works just fine and symbolication works fine, so it should be some bug with PDB in particular.

> That is strange as globals usually work best? The common issue I've seen is getting the slide wrong. The EFI modules are linked at a value near zero and relocated into memory, so the slide represents that adjustment.
> 
> You can use `image dump sections` and ` image dump symtab` to see lldb's view of symbols. More info here [1].

Yes, this one is a bit complicated, once again due to PDB most likely. It knows about global symbols, but does not list them in symtab:

(lldb) image dump symtab
Dumping symbol table for 91 modules.
Symtab, file = GdbSyms/Bin/X64_CLANGPDB/GdbSyms.dll, num_symbols = 0
Symtab, file = /Users/user/Documents/UefiWorkspace/Build/OvmfX64/NOOPT_CLANGPDB/X64/MdeModulePkg/Core/Dxe/DxeMain/DEBUG/DxeCore.dll, num_symbols = 0
Symtab, file = /Users/user/Documents/UefiWorkspace/Build/OvmfX64/NOOPT_CLANGPDB/X64/MdeModulePkg/Universal/DevicePathDxe/DevicePathDxe/DEBUG/DevicePathDxe.dll, num_symbols = 0
…

The slides are correct, but there are two nuances that collide with it.

1. There are multiple instances of the globals with the same name (e.g. gBS), but for some reason LLDB always tries to print the globals from the first module. This happens even when I am source-level debugging, and I see a gBS symbol from another module (e.g. DxeCore) used right at the same line. With GDB the closest symbol is used, but with LLDB it is always coming from the first module. I tried checking expr help to find whether I can pass it a module explicitly, but also failed.

2.  To be able to get EFI types to locate the EFI_SYSTEM_TABLE_POINTER I add a dummy GdbSyms image, which is not loaded to the firmware. So basically I cannot slide what is not in the memory, and this is also my first image. I tried deleting it anyhow, but it failed for me.

(lldb) image dump sections
Dumping sections for 91 modules.
Sections for 'GdbSyms/Bin/X64_CLANGPDB/GdbSyms.dll' (x86_64):
  SectID     Type             Load Address                             Perm File Off.  File Size  Flags      Section Name
  ---------- ---------------- ---------------------------------------  ---- ---------- ---------- ---------- ----------------------------
  0xffffffffffffffff container        [0x0000000000000000-0x0000000000006ec0)* ---  0x00000000 0x00000000 0x00000000 GdbSyms.dll.
  0x00000001 code             [0x0000000000000220-0x0000000000005bd6)* ---  0x00000220 0x000059c0 0x60000020 GdbSyms.dll...text
  0x00000002 data             [0x0000000000005be0-0x0000000000006d79)* ---  0x00005be0 0x000011a0 0x40000040 GdbSyms.dll...rdata
  0x00000003 data             [0x0000000000006d80-0x0000000000006e30)* ---  0x00006d80 0x00000060 0xc0000040 GdbSyms.dll...data
  0x00000004 regular          [0x0000000000006e40-0x0000000000006ea4)* ---  0x00006de0 0x00000080 0x42000040 GdbSyms.dll...reloc
Sections for '/Users/user/Documents/UefiWorkspace/Build/OvmfX64/NOOPT_CLANGPDB/X64/MdeModulePkg/Core/Dxe/DxeMain/DEBUG/DxeCore.dll' (x86_64):
  SectID     Type             Load Address                             Perm File Off.  File Size  Flags      Section Name
  ---------- ---------------- ---------------------------------------  ---- ---------- ---------- ---------- ----------------------------
  0xffffffffffffffff container        [0x0000000000000000-0x00000000000523a0)* ---  0x00000000 0x00000000 0x00000000 DxeCore.dll.
  0x00000001 code             [0x000000007fe1b220-0x000000007fe61e34)  ---  0x00000220 0x00046c20 0x60000020 DxeCore.dll...text
  0x00000002 data             [0x000000007fe61e40-0x000000007fe68065)  ---  0x00046e40 0x00006240 0x40000040 DxeCore.dll...rdata
  0x00000003 data             [0x000000007fe68080-0x000000007fe6d160)  ---  0x0004d080 0x000018a0 0xc0000040 DxeCore.dll...data
  0x00000004 regular          [0x000000007fe6d160-0x000000007fe6d398)  ---  0x0004e920 0x00000240 0x42000040 DxeCore.dll...reloc
Sections for '/Users/user/Documents/UefiWorkspace/Build/OvmfX64/NOOPT_CLANGPDB/X64/MdeModulePkg/Universal/DevicePathDxe/DevicePathDxe/DEBUG/DevicePathDxe.dll' (x86_64):
  SectID     Type             Load Address                             Perm File Off.  File Size  Flags      Section Name
  ---------- ---------------- ---------------------------------------  ---- ---------- ---------- ---------- ----------------------------
  0xffffffffffffffff container        [0x0000000000000000-0x0000000000014420)* ---  0x00000000 0x00000000 0x00000000 DevicePathDxe.dll.
  0x00000001 code             [0x000000007f986220-0x000000007f996cc6)  ---  0x00000220 0x00010ac0 0x60000020 DevicePathDxe.dll...text
  0x00000002 data             [0x000000007f996ce0-0x000000007f999b04)  ---  0x00010ce0 0x00002e40 0x40000040 DevicePathDxe.dll...rdata
  0x00000003 data             [0x000000007f999b20-0x000000007f99a1a2)  ---  0x00013b20 0x00000660 0xc0000040 DevicePathDxe.dll...data
  0x00000004 regular          [0x000000007f99a1c0-0x000000007f99a404)  ---  0x00014180 0x00000260 0x42000040 DevicePathDxe.dll…reloc
…

So, all in all, unique global variables work, but there is no way to access duplicating variables. They either resolve to GdbSyms or just cause a crash:

(lldb) p mDebugInfoTableHeader
(EFI_DEBUG_IMAGE_INFO_TABLE_HEADER) $0 = {
  UpdateStatus = 2
  TableSize = 92
  EfiDebugImageInfoTable = 0x000000007f814018
}
(lldb) p gBS
error: Couldn't materialize: couldn't get the value of variable ::gBS: read memory from 0x6df8 failed
error: errored out in DoExecute, couldn't PrepareToExecuteJITExpression
(lldb) p gEfiGlobalVariableGuid
0  libLLVM.dylib            0x000000010e52ee68 llvm::sys::PrintStackTrace(llvm::raw_ostream&) + 40
1  libLLVM.dylib            0x000000010e52f262 SignalHandler(int) + 188
2  libsystem_platform.dylib 0x00007fff6ca5642d _sigtramp + 29
...

> You can tell lldb to use the older Python like this (from the Terminal.app):
> $ defaults write com.apple.dt.lldb DefaultPythonVersion 2

Thanks, that helped quite a bit, but for some reason Xcode version still crashes more for me. I attached a couple of stack traces if you feel like having a look, but once again it seems that it is all about the PDB plugin.

> For the macOS API clang emits frame pointers, so you can walk the stack without symbols. You could try adding the compiler flag to emit the frame pointers.

I am pretty sure stack frames are not disabled with UEFI, as sometimes backtracing works just fine. To me it looks like debug information parsing randomly breaks in LLDB, and once it happens it forgets about other images:

(lldb) b CoreLocateHandleBuffer
Breakpoint 2: where = DxeCore.dll`CoreLocateHandleBuffer + 31 at Locate.c:649, address = 0x000000007fe36e4f
(lldb) c
Process 1 resuming
Process 1 stopped
* thread #1, stop reason = breakpoint 2.1
    frame #0: 0x000000007fe36e4f DxeCore.dll`CoreLocateHandleBuffer(SearchType=ByProtocol, Protocol=0x000000007f978160, SearchKey=0x0000000000000000, NumberHandles=0x000000007fe19fd8, Buffer=0x000000007fe19fc0) at Locate.c:649
   646 	  EFI_STATUS          Status;
   647 	  UINTN               BufferSize;
   648
-> 649 	  if (NumberHandles == NULL) {
   650 	    return EFI_INVALID_PARAMETER;
   651 	  }
   652
(lldb) bt
* thread #1, stop reason = breakpoint 2.1
  * frame #0: 0x000000007fe36e4f DxeCore.dll`CoreLocateHandleBuffer(SearchType=ByProtocol, Protocol=0x000000007f978160, SearchKey=0x0000000000000000, NumberHandles=0x000000007fe19fd8, Buffer=0x000000007fe19fc0) at Locate.c:649
    frame #1: 0x000000007fe36816 DxeCore.dll`CoreLocateDevicePath(Protocol=0x000000007f978160, DevicePath=0x000000007fe1a060, Device=0x000000007fe1a068) at Locate.c:466
    frame #2: 0x000000007f97479a SecurityStubDxe.dll

———

(lldb) b CopyMem
Breakpoint 3: 70 locations.
(lldb) c
Process 1 resuming
Process 1 stopped
* thread #1, stop reason = breakpoint 2.53 3.53
    frame #0: 0x000000007e5c13b3 MnpDxe.dll`CopyMem(DestinationBuffer=0x000000007fe19b50, SourceBuffer=0x000000007e2aa470, Length=656) at CopyMemWrapper.c:47
   44  	  IN UINTN       Length
   45  	  )
   46  	{
-> 47  	  if (Length == 0) {
   48  	    return DestinationBuffer;
   49  	  }
   50  	  ASSERT ((Length - 1) <= (MAX_ADDRESS - (UINTN)DestinationBuffer));
(lldb) bt
* thread #1, stop reason = breakpoint 2.53 3.53
  * frame #0: 0x000000007e5c13b3 MnpDxe.dll`CopyMem(DestinationBuffer=0x000000007fe19b50, SourceBuffer=0x000000007e2aa470, Length=656) at CopyMemWrapper.c:47
(lldb) finish
error: Could not create return address breakpoint.
(lldb) n
Process 1 stopped
* thread #1, stop reason = step over
    frame #0: 0x000000007e5c13ce MnpDxe.dll`CopyMem(DestinationBuffer=0x000000007fe19b50, SourceBuffer=0x000000007e2aa470, Length=656) at CopyMemWrapper.c:50
   47  	  if (Length == 0) {
   48  	    return DestinationBuffer;
   49  	  }
-> 50  	  ASSERT ((Length - 1) <= (MAX_ADDRESS - (UINTN)DestinationBuffer));
   51  	  ASSERT ((Length - 1) <= (MAX_ADDRESS - (UINTN)SourceBuffer));
   52
   53  	  if (DestinationBuffer == SourceBuffer) {
(lldb)
...
Process 1 stopped
* thread #1, stop reason = step over
    frame #0: 0x000000007e5c14b4 MnpDxe.dll`CopyMem(DestinationBuffer=0x000000007fe19b50, SourceBuffer=0x000000007e2aa470, Length=656) at CopyMemWrapper.c:57
   54  	    return DestinationBuffer;
   55  	  }
   56  	  return InternalMemCopyMem (DestinationBuffer, SourceBuffer, Length);
-> 57  	}
(lldb)
Process 1 stopped
* thread #1, stop reason = step over
    frame #0: 0x000000007e5c726e MnpDxe.dll
->  0x7e5c726e: mov    rax, qword ptr [rsp + 0x60]
    0x7e5c7273: cmp    byte ptr [rax + 0x68], 0x0
    0x7e5c7277: jne    0x7e5c7291
    0x7e5c727d: movabs rax, -0x7fffffffffffffed
(lldb) bt
* thread #1, stop reason = step over
  * frame #0: 0x000000007e5c726e MnpDxe.dll

———

(lldb) c
Process 1 resuming
Process 1 stopped
* thread #1, stop reason = signal SIGINT
    frame #0: 0x000000007fe4d72e DxeCore.dll
->  0x7fe4d72e: cmp    al, 0x0
    0x7fe4d730: je     0x7fe4d765
    0x7fe4d736: mov    rcx, qword ptr [rsp + 0x20]
    0x7fe4d73b: call   0x7fe4c4b0
(lldb) bt
* thread #1, stop reason = signal SIGINT
  * frame #0: 0x000000007fe4d72e DxeCore.dll

> On macOS the Mach-O and dSYM have a UUID (dwarfdump -u) that is indexed by Spotlight (mdfind "com_apple_xcode_dsym_uuids == *") [2]
> This should be the UUID in the debug directory entry and you can use that to lookup the symbols like this:
> 
> module = target.AddModule (None, None, uuid)
> SBError = target.SetModuleLoadAddress (module, LoadAddress + TeAdjust)
> 
> Also lldb has built in help for commands, but it is kind of terse since it is autogenerated from the C++ swig.
> (lldb) script help (lldb.target.AddModule)
> Help on method AddModule in module lldb:
> 
> AddModule(self, *args) method of lldb.SBTarget instance
>     AddModule(SBTarget self, SBModule module) -> bool
>     AddModule(SBTarget self, char const * path, char const * triple, char const * uuid) -> SBModule
>     AddModule(SBTarget self, char const * path, char const * triple, char const * uuid_cstr, char const * symfile) -> SBModule
>     AddModule(SBTarget self, SBModuleSpec module_spec) -> SBModule
> 
> The minimum  you need to symbolicate a frame is uuid, LoadAddress, and PC.
> 
> [1] http://lldb.llvm.org/use/map.html <http://lldb.llvm.org/use/map.html>
> [2] http://lldb.llvm.org/use/symbols.html <http://lldb.llvm.org/use/symbols.html>
Thanks for the links again. Yes, I am using some of these, and in fact for GDB that’s pretty much what I did when I worked with XCODE5. It is very likely that when I get to complete LLDB support for XCODE5 it will work quite fine too. But I am already happy with XCODE5 here, and making it even better will only help myself, but not other people with e.g. Linux or people that want me to use the same compiler with them.

Best regards,
Vitaly



> 21 марта 2020 г., в 20:13, Andrew Fish <afish@apple.com> написал(а):
> 
> 
> 
>> On Mar 21, 2020, at 3:28 AM, Vitaly Cheptsov <cheptsov@ispras.ru <mailto:cheptsov@ispras.ru>> wrote:
>> 
>> Hello,
>> 
>> Andrey, thanks for the hint, it was very helpful. I rewrote the GDB scripts to work with LLDB[1] and was able to debug OVMF built with CLANGPDB. While it is still quite dirty, at the very least it works.
>> 
>> Unfortunately the experience was close to terrible. I may certainly do something wrong, but it is clear that PDB and LLDB do not support each other well enough. After spending several hours on playing with the tools my conclusion is that LLDB is simply not suited for UEFI PDB debugging, and we really want DWARF  as there is no other opensource debugger that supports PDB on macOS and Linux
>> 
>> In case somebody knows workarounds here are the issues I faced:
>> 
>> 1. All integer alias typedefs are discarded in favour of underlying types. This way EFI_STATUS and EFI_TPL become unsigned long long, CHAR8 becomes char, and CHAR16 becomes unsigned short. It does not look like LLDB has the original types anywhere at all, and it also does not have them registered.
>> 
>>     frame #0: 0x000000007fe242aa DxeCore.dll`CoreAllocatePoolPagesI(PoolType=EfiBootServicesData, NoPages=1, Granularity=4096, NeedGuard='\0') at Pool.c:322
>>    319 	    return NULL;
>>    320 	  }
>>    321
>> -> 322 	  Buffer = CoreAllocatePoolPages (PoolType, NoPages, Granularity, NeedGuard);
>>    323 	  CoreReleaseMemoryLock ();
>>    324
>>    325 	  if (Buffer != NULL) {
>> (lldb) p Status
>> (unsigned long long) $3 = 0
>> 
>> Structures work more or less fine, but for simpler types like strings we are out of even potential pretty-printing.
>> 
> 
> Vitaly,
> 
> You can teach lldb about types. There is some example code here: https://github.com/tianocore/edk2/blob/master/EmulatorPkg/Unix/lldbefi.py <https://github.com/tianocore/edk2/blob/master/EmulatorPkg/Unix/lldbefi.py>
>> 2. Global variables are not accessible. I am not sure what happens, but they either seem to not relocate or conflict with the other names:
>> 
>> (lldb) p gST
>> error: Couldn't materialize: couldn't get the value of variable ::gST: read memory from 0x6e18 failed
>> error: errored out in DoExecute, couldn't PrepareToExecuteJITExpression
>> (lldb) p &gST
>> error: Couldn't materialize: couldn't get the value of variable ::gST: read memory from 0x6e18 failed
>> error: errored out in DoExecute, couldn't PrepareToExecuteJITExpression
>> 
> 
> That is strange as globals usually work best? The common issue I've seen is getting the slide wrong. The EFI modules are linked at a value near zero and relocated into memory, so the slide represents that adjustment.
> 
> You can use `image dump sections` and ` image dump symtab` to see lldb's view of symbols. More info here [1].
> 
>> 3. Quite a number of crashes.
>> 
>> In most cases autocompletion by tab press causes a crash. E.g.
>> 
>> b I<TAB>
>> 
>> So will do printing of a GUID, e.g. p gEfiGlobalVariableGuid.
>> 
>> This may have to do with Python compatibility as Xcode 11 LLDB that uses Python 3 generally crashes more often than MacPorts LLDB 9.0. Surprisingly structures work more or less fine.
>> 
> 
> You can tell lldb to use the older Python like this (from the Terminal.app):
> $ defaults write com.apple.dt.lldb DefaultPythonVersion 2
> 
>> 4. Ctrl+C does not produce a valid backtrace. When I break with a breakpoint, I see a proper stacktrace with more than one entry, with function prototypes and values. When I break with Ctrl+C I only see some weird backtrace with most of the entries missing regardless of frame position:
>> 
>> (lldb) bt
>> * thread #1, stop reason = signal SIGTRAP
>>   * frame #0: 0x000000007fe4c5f3 DxeCore.dll
>> 
>> Probably more and all the unintuitive stuff like the lack of more functional TUI, but it is hard to remember all the trials.
>> 
> 
> For the macOS API clang emits frame pointers, so you can walk the stack without symbols. You could try adding the compiler flag to emit the frame pointers.
> 
> 
>> [1] https://github.com/acidanthera/OpenCorePkg/blob/master/Debug/Scripts/lldb_uefi.py <https://github.com/acidanthera/OpenCorePkg/blob/master/Debug/Scripts/lldb_uefi.py>
>> 
> 
> On macOS the Mach-O and dSYM have a UUID (dwarfdump -u) that is indexed by Spotlight (mdfind "com_apple_xcode_dsym_uuids == *") [2]
> This should be the UUID in the debug directory entry and you can use that to lookup the symbols like this:
> 
> module = target.AddModule (None, None, uuid)
> SBError = target.SetModuleLoadAddress (module, LoadAddress + TeAdjust)
> 
> Also lldb has built in help for commands, but it is kind of terse since it is autogenerated from the C++ swig.
> (lldb) script help (lldb.target.AddModule)
> Help on method AddModule in module lldb:
> 
> AddModule(self, *args) method of lldb.SBTarget instance
>     AddModule(SBTarget self, SBModule module) -> bool
>     AddModule(SBTarget self, char const * path, char const * triple, char const * uuid) -> SBModule
>     AddModule(SBTarget self, char const * path, char const * triple, char const * uuid_cstr, char const * symfile) -> SBModule
>     AddModule(SBTarget self, SBModuleSpec module_spec) -> SBModule
> 
> The minimum  you need to symbolicate a frame is uuid, LoadAddress, and PC.
> 
> [1] http://lldb.llvm.org/use/map.html <http://lldb.llvm.org/use/map.html>
> [2] http://lldb.llvm.org/use/symbols.html <http://lldb.llvm.org/use/symbols.html>
> 
> Thanks,
> 
> Andrew Fish
> 
> 
>> Best wishes,
>> Vitaly
>> 
>>> 20 марта 2020 г., в 22:14, Andrew Fish <afish@apple.com <mailto:afish@apple.com>> написал(а):
>>> 
>>> 
>>> 
>>>> On Mar 20, 2020, at 8:13 AM, Vitaly Cheptsov <cheptsov@ispras.ru <mailto:cheptsov@ispras.ru>> wrote:
>>>> 
>>>> Hello,
>>>> 
>>>> We noticed that the original bugzilla, which intended to add new LLVM toolchain support[1], also wanted to bring ELF format support with DWARF debugging information. For some reason this did not make its way into EDK II, and we are currently wondering, how can one debug binaries built with LLVM 9.0.
>>>> 
>>>> For macOS and XCODE5 toolchain we use GDB scripts based on Andrei Warkentin’s work, which allow us to integrate with QEMU and VMware[2]. It is likely that they should work with little to no work on Linux with CLANG38/GCC5 with GDB once again. However, CLANGPDB apparently is using PDB debugging information, which I believe is not handled with GDB.
>>>> 
>>>> Could you please provide the details on the matter and let us know about the recommended route?
>>>> — Is dropping CLANGELF just a temporary measure and it should be resubmitted again?
>>>> — Should LLDB, which seems to be aware of PDB, be used instead of GDB, when building with CLANGPDB? If so, did anybody try that?
>>>> 
>>> 
>>> Vitaly,
>>> 
>>> I've not tried the CLANGPDB path, but if you want to connect lldb to QEMU you need to set  plugin.process.gdb-remote.target-definition-file [1] to [2].
>>> 
>>> [1]  lldb -o "settings set plugin.process.gdb-remote.target-definition-file x86_64_target_definition.py" -o "gdb-remote 9000"
>>> [2] https://github.com/llvm-mirror/lldb/blob/master/examples/python/x86_64_target_definition.py <https://github.com/llvm-mirror/lldb/blob/master/examples/python/x86_64_target_definition.py>
>>> 
>>> Thanks,
>>> 
>>> Andrew Fish
>>> 
>>>> Thanks!
>>>> 
>>>> Best regards,
>>>> Vitaly
>>>> 
>>>> [1] https://bugzilla.tianocore.org/show_bug.cgi?id=1603 <https://bugzilla.tianocore.org/show_bug.cgi?id=1603>
>>>> [2] https://github.com/acidanthera/OpenCorePkg/blob/master/Debug/Scripts/gdb_uefi.py <https://github.com/acidanthera/OpenCorePkg/blob/master/Debug/Scripts/gdb_uefi.py>
>>>> 
>> 
>> 


[-- Attachment #1.2.1: Type: text/html, Size: 70592 bytes --]

[-- Attachment #1.2.2: crashes.txt --]
[-- Type: text/plain, Size: 14456 bytes --]

E.g. this is what happens when I type «b In<TAB>». I cannot reproduce it with stock LLDB.

(lldb) b In0  lldb                     0x0000000102638705 llvm::sys::PrintStackTrace(llvm::raw_ostream&) + 37
1  lldb                     0x0000000102637d77 llvm::sys::RunSignalHandlers() + 39
2  lldb                     0x0000000102638d58 SignalHandler(int) + 264
3  libsystem_platform.dylib 0x00007fff6ca5642d _sigtramp + 29
4  libsystem_platform.dylib 000000000000000000 _sigtramp + 2472188912
5  LLDB                     0x0000000102dbf200 llvm::BinaryStreamReader::BinaryStreamReader(llvm::BinaryStream&) + 32
6  LLDB                     0x00000001030ba527 llvm::pdb::ModuleDebugStreamRef::reload() + 39
7  LLDB                     0x0000000102d589ff lldb_private::npdb::CompileUnitIndex::GetOrCreateCompiland(unsigned short) + 319
8  LLDB                     0x0000000102d72077 lldb_private::npdb::SymbolFileNativePDB::ParseCompileUnitAtIndex(unsigned int) + 119
9  LLDB                     0x00000001029cf71d lldb_private::SymbolVendor::GetCompileUnitAtIndex(unsigned long) + 221
10 LLDB                     0x000000010288659a lldb_private::Module::GetCompileUnitAtIndex(unsigned long) + 106
11 LLDB                     0x000000010289fb10 lldb_private::SearchFilter::DoCUIteration(std::__1::shared_ptr<lldb_private::Module> const&, lldb_private::SymbolContext const&, lldb_private::Searcher&) + 208
12 LLDB                     0x000000010289f7b3 lldb_private::SearchFilter::DoModuleIteration(lldb_private::SymbolContext const&, lldb_private::Searcher&) + 419
13 LLDB                     0x000000010289f5f4 lldb_private::SearchFilter::Search(lldb_private::Searcher&) + 180
14 LLDB                     0x0000000102e5563d lldb_private::CommandCompletions::SourceFileCompleter::DoCompletion(lldb_private::SearchFilter*) + 29
15 LLDB                     0x0000000102e54c28 lldb_private::CommandCompletions::SourceFiles(lldb_private::CommandInterpreter&, lldb_private::CompletionRequest&, lldb_private::SearchFilter*) + 200
16 LLDB                     0x0000000102e555ed lldb_private::CommandCompletions::InvokeCommonCompletionCallbacks(lldb_private::CommandInterpreter&, unsigned int, lldb_private::CompletionRequest&, lldb_private::SearchFilter*) + 93
17 LLDB                     0x000000010293b331 lldb_private::CommandObjectRegexCommand::HandleCompletion(lldb_private::CompletionRequest&) + 33
18 LLDB                     0x00000001029310e8 lldb_private::CommandInterpreter::HandleCompletionMatches(lldb_private::CompletionRequest&) + 424
19 LLDB                     0x000000010293135d lldb_private::CommandInterpreter::HandleCompletion(char const*, char const*, char const*, int, int, lldb_private::StringList&, lldb_private::StringList&) + 349
20 LLDB                     0x0000000102871eaa lldb_private::IOHandlerDelegate::IOHandlerComplete(lldb_private::IOHandler&, char const*, char const*, char const*, int, int, lldb_private::StringList&, lldb_private::StringList&) + 90
21 LLDB                     0x0000000102872547 lldb_private::IOHandlerEditline::AutoCompleteCallback(char const*, char const*, char const*, int, int, lldb_private::StringList&, lldb_private::StringList&, void*) + 55
22 LLDB                     0x00000001029155fe lldb_private::Editline::TabCommand(int) + 110
23 LLDB                     0x0000000102917931 lldb_private::Editline::ConfigureEditor(bool)::$_13::__invoke(editline*, int) + 33
24 libedit.3.dylib          0x00007fff6a3fc7db el_wgets + 484
25 libedit.3.dylib          0x00007fff6a4078bb el_gets + 37
26 LLDB                     0x0000000102916df7 lldb_private::Editline::GetLine(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >&, bool&) + 439
27 LLDB                     0x00000001028727bc lldb_private::IOHandlerEditline::GetLine(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >&, bool&) + 60
28 LLDB                     0x0000000102872f3e lldb_private::IOHandlerEditline::Run() + 190
29 LLDB                     0x000000010285a3cb lldb_private::Debugger::ExecuteIOHandlers() + 123
30 LLDB                     0x000000010293589c lldb_private::CommandInterpreter::RunCommandInterpreter(bool, bool, lldb_private::CommandInterpreterRunOptions&) + 156
31 LLDB                     0x00000001026deb91 lldb::SBDebugger::RunCommandInterpreter(bool, bool) + 209
32 lldb                     0x0000000102623665 Driver::MainLoop() + 2853
33 lldb                     0x00000001026245d2 main + 1634
34 libdyld.dylib            0x00007fff6c85d7fd start + 1
35 libdyld.dylib            0x000000000000000f start + 2474256403
Stack dump:
0.  Program arguments: /Applications/Xcode.app/Contents/Developer/usr/bin/lldb -o settings set plugin.process.gdb-remote.target-definition-file Scripts/x86_64_target_definition.py -o gdb-remote 8864 -o target modules add -s GdbSyms/Bin/X64_CLANGPDB/GdbSyms.pdb GdbSyms/Bin/X64_CLANGPDB/GdbSyms.dll -o command script import Scripts/lldb_uefi.py -o command script add -c lldb_uefi.ReloadUefi reload-uefi -o reload-uefi -o b DebugBreak 
./maclldb.tool: line 33: 37553 Segmentation fault: 11  "$LLDB" -o "settings set plugin.process.gdb-remote.target-definition-file Scripts/x86_64_target_definition.py" -o "gdb-remote $LLDB_PORT" -o "target modules add -s GdbSyms/Bin/X64_CLANGPDB/GdbSyms.pdb GdbSyms/Bin/X64_CLANGPDB/GdbSyms.dll" -o "command script import Scripts/lldb_uefi.py" -o "command script add -c lldb_uefi.ReloadUefi reload-uefi" -o "reload-uefi" -o "b DebugBreak"

This I cannot reproduce with stock GDB either:
(lldb) p mDebugInfoTableHeader
0  lldb                     0x0000000102e13705 llvm::sys::PrintStackTrace(llvm::raw_ostream&) + 37
1  lldb                     0x0000000102e12d77 llvm::sys::RunSignalHandlers() + 39
2  lldb                     0x0000000102e13d58 SignalHandler(int) + 264
3  libsystem_platform.dylib 0x00007fff6ca5642d _sigtramp + 29
4  libsystem_platform.dylib 0xaae108ecfd74df7c _sigtramp + 2429516652
5  LLDB                     0x000000010359e200 llvm::BinaryStreamReader::BinaryStreamReader(llvm::BinaryStream&) + 32
6  LLDB                     0x0000000103899527 llvm::pdb::ModuleDebugStreamRef::reload() + 39
7  LLDB                     0x00000001035379ff lldb_private::npdb::CompileUnitIndex::GetOrCreateCompiland(unsigned short) + 319
8  LLDB                     0x0000000103540a77 lldb_private::npdb::PdbAstBuilder::ParseAllNamespacesPlusChildrenOf(llvm::Optional<llvm::StringRef>) + 663
9  LLDB                     0x00000001035414e2 lldb_private::npdb::PdbAstBuilder::ParseDeclsForContext(clang::DeclContext&) + 82
10 LLDB                     0x0000000103550fef lldb_private::npdb::SymbolFileNativePDB::ParseDeclsForContext(lldb_private::CompilerDeclContext) + 63
11 LLDB                     0x000000010315604c lldb_private::ClangASTContext::DeclContextFindDeclByName(void*, lldb_private::ConstString, bool) + 668
12 LLDB                     0x00000001031627b9 lldb_private::CompilerDeclContext::FindDeclByName(lldb_private::ConstString, bool) + 57
13 LLDB                     0x00000001032f1b76 lldb_private::ClangExpressionDeclMap::FindExternalVisibleDecls(lldb_private::NameSearchContext&, std::__1::shared_ptr<lldb_private::Module>, lldb_private::CompilerDeclContext&, unsigned int) + 1014
14 LLDB                     0x00000001032f1334 lldb_private::ClangExpressionDeclMap::FindExternalVisibleDecls(lldb_private::NameSearchContext&) + 628
15 LLDB                     0x00000001032e63b6 lldb_private::ClangASTSource::FindExternalVisibleDeclsByName(clang::DeclContext const*, clang::DeclarationName) + 1174
16 LLDB                     0x000000010635e802 clang::DeclContext::lookup(clang::DeclarationName) const + 754
17 LLDB                     0x0000000105ff1f66 LookupDirect(clang::Sema&, clang::LookupResult&, clang::DeclContext const*) + 86
18 LLDB                     0x0000000105fee3e9 clang::Sema::CppLookupName(clang::LookupResult&, clang::Scope*) + 3257
19 LLDB                     0x0000000105ff1795 clang::Sema::LookupName(clang::LookupResult&, clang::Scope*, bool) + 725
20 LLDB                     0x0000000105d66a63 clang::Sema::ClassifyName(clang::Scope*, clang::CXXScopeSpec&, clang::IdentifierInfo*&, clang::SourceLocation, clang::Token const&, bool, std::__1::unique_ptr<clang::CorrectionCandidateCallback, std::__1::default_delete<clang::CorrectionCandidateCallback> >) + 419
21 LLDB                     0x0000000105b0ac14 clang::Parser::TryAnnotateName(bool, std::__1::unique_ptr<clang::CorrectionCandidateCallback, std::__1::default_delete<clang::CorrectionCandidateCallback> >) + 484
22 LLDB                     0x0000000105aeb55a clang::Parser::ParseStatementOrDeclarationAfterAttributes(llvm::SmallVector<clang::Stmt*, 32u>&, clang::Parser::AllowedConstructsKind, clang::SourceLocation*, clang::Parser::ParsedAttributesWithRange&) + 394
23 LLDB                     0x0000000105aeb279 clang::Parser::ParseStatementOrDeclaration(llvm::SmallVector<clang::Stmt*, 32u>&, clang::Parser::AllowedConstructsKind, clang::SourceLocation*) + 201
24 LLDB                     0x0000000105af263c clang::Parser::ParseCompoundStatementBody(bool) + 1644
25 LLDB                     0x0000000105af3131 clang::Parser::ParseFunctionStatementBody(clang::Decl*, clang::Parser::ParseScope&) + 161
26 LLDB                     0x0000000105b09780 clang::Parser::ParseFunctionDefinition(clang::ParsingDeclarator&, clang::Parser::ParsedTemplateInfo const&, clang::Parser::LateParsedAttrList*) + 2608
27 LLDB                     0x0000000105a84045 clang::Parser::ParseDeclGroup(clang::ParsingDeclSpec&, clang::DeclaratorContext, clang::SourceLocation*, clang::Parser::ForRangeInit*) + 1781
28 LLDB                     0x0000000105b08a37 clang::Parser::ParseDeclOrFunctionDefInternal(clang::Parser::ParsedAttributesWithRange&, clang::ParsingDeclSpec&, clang::AccessSpecifier) + 695
29 LLDB                     0x0000000105b084e3 clang::Parser::ParseDeclarationOrFunctionDefinition(clang::Parser::ParsedAttributesWithRange&, clang::ParsingDeclSpec*, clang::AccessSpecifier) + 339
30 LLDB                     0x0000000105b06c57 clang::Parser::ParseExternalDeclaration(clang::Parser::ParsedAttributesWithRange&, clang::ParsingDeclSpec*) + 2535
31 LLDB                     0x0000000105b05d97 clang::Parser::ParseTopLevelDecl(clang::OpaquePtr<clang::DeclGroupRef>&) + 679
32 LLDB                     0x0000000105a71855 clang::ParseAST(clang::Sema&, bool, bool) + 453
33 LLDB                     0x0000000105a71672 clang::ParseAST(clang::Preprocessor&, clang::ASTConsumer*, clang::ASTContext&, bool, clang::TranslationUnitKind, clang::CodeCompleteConsumer*, bool) + 194
34 LLDB                     0x00000001033094a1 lldb_private::ClangExpressionParser::ParseInternal(lldb_private::DiagnosticManager&, clang::CodeCompleteConsumer*, unsigned int, unsigned int) + 1729
35 LLDB                     0x0000000103315793 lldb_private::ClangUserExpression::Parse(lldb_private::DiagnosticManager&, lldb_private::ExecutionContext&, lldb_private::ExecutionPolicy, bool, bool) + 499
36 LLDB                     0x00000001030deaf5 lldb_private::UserExpression::Evaluate(lldb_private::ExecutionContext&, lldb_private::EvaluateExpressionOptions const&, llvm::StringRef, llvm::StringRef, lldb_private::SharingPtr<lldb_private::ValueObject>&, lldb_private::Status&, unsigned int, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >*, std::__1::shared_ptr<lldb_private::Module>*) + 1701
37 LLDB                     0x0000000103241542 lldb_private::Target::EvaluateExpression(llvm::StringRef, lldb_private::ExecutionContextScope*, lldb_private::SharingPtr<lldb_private::ValueObject>&, lldb_private::EvaluateExpressionOptions const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >*) + 418
38 LLDB                     0x000000010364dd06 lldb_private::CommandObjectExpression::EvaluateExpression(llvm::StringRef, lldb_private::Stream*, lldb_private::Stream*, lldb_private::CommandReturnObject*) + 566
39 LLDB                     0x000000010364ec56 lldb_private::CommandObjectExpression::DoExecute(llvm::StringRef, lldb_private::CommandReturnObject&) + 1270
40 LLDB                     0x000000010311877b lldb_private::CommandObjectRaw::Execute(char const*, lldb_private::CommandReturnObject&) + 427
41 LLDB                     0x000000010310fd25 lldb_private::CommandInterpreter::HandleCommand(char const*, lldb_private::LazyBool, lldb_private::CommandReturnObject&, lldb_private::ExecutionContext*, bool, bool) + 2805
42 LLDB                     0x0000000103113d61 lldb_private::CommandInterpreter::IOHandlerInputComplete(lldb_private::IOHandler&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >&) + 657
43 LLDB                     0x0000000103051f9d lldb_private::IOHandlerEditline::Run() + 285
44 LLDB                     0x00000001030393cb lldb_private::Debugger::ExecuteIOHandlers() + 123
45 LLDB                     0x000000010311489c lldb_private::CommandInterpreter::RunCommandInterpreter(bool, bool, lldb_private::CommandInterpreterRunOptions&) + 156
46 LLDB                     0x0000000102ebdb91 lldb::SBDebugger::RunCommandInterpreter(bool, bool) + 209
47 lldb                     0x0000000102dfe665 Driver::MainLoop() + 2853
48 lldb                     0x0000000102dff5d2 main + 1634
49 libdyld.dylib            0x00007fff6c85d7fd start + 1
50 libdyld.dylib            0x000000000000000f start + 2474256403
Stack dump:
0.  Program arguments: /Applications/Xcode.app/Contents/Developer/usr/bin/lldb -o settings set plugin.process.gdb-remote.target-definition-file Scripts/x86_64_target_definition.py -o gdb-remote 8864 -o target modules add -s GdbSyms/Bin/X64_CLANGPDB/GdbSyms.pdb GdbSyms/Bin/X64_CLANGPDB/GdbSyms.dll -o command script import Scripts/lldb_uefi.py -o command script add -c lldb_uefi.ReloadUefi reload-uefi -o reload-uefi -o b DebugBreak 
./maclldb.tool: line 33: 37722 Segmentation fault: 11  "$LLDB" -o "settings set plugin.process.gdb-remote.target-definition-file Scripts/x86_64_target_definition.py" -o "gdb-remote $LLDB_PORT" -o "target modules add -s GdbSyms/Bin/X64_CLANGPDB/GdbSyms.pdb GdbSyms/Bin/X64_CLANGPDB/GdbSyms.dll" -o "command script import Scripts/lldb_uefi.py" -o "command script add -c lldb_uefi.ReloadUefi reload-uefi" -o "reload-uefi" -o "b DebugBreak"

[-- Attachment #2: Message signed with OpenPGP --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [edk2-devel] CLANGPDB binary debugging

[Andrew Fish]

[-- Attachment #1: Type: text/plain, Size: 24294 bytes --]



> On Mar 21, 2020, at 11:36 AM, Vitaly Cheptsov <cheptsov@ispras.ru> wrote:
> 
> Andrew,
> 
> Thanks once again, but unfortunately it is not that simple. Below I answered inline explaining the particular issues, which mostly seem to be specific to CLANGPDB. LLVM stack emits PDB debug files, and even though LLDB does support them to some level, it is unlikely that this will be working well enough soon. We should really stick to more or less native debug formats, ideally those that have proper open specifications, on all platforms, and for Unix that’s DWARF.
> 

Vitaly,

I understand and I use the Xcode clang and not the CLANGPDB, but I use lldb a lot I was just trying to point out what works with Xcode. 


> I am pretty sure LLVM can be taught to emit DWARF debug information even for PE files. Perhaps we can either make some option or provide a separate toolchain for this? Another way would be recovering CLANGELF as originally suggested.
> 

There was a bug recently in the  x86_64-pc-win32-macho triple and we had to add -gdwarf to force it emit dwarf. Not sure what that compiler flag would do to CLANGPDB but it is worth a try? Last flag wins for the compiler. 


>> You can teach lldb about types. There is some example code here: https://github.com/tianocore/edk2/blob/master/EmulatorPkg/Unix/lldbefi.py <https://github.com/tianocore/edk2/blob/master/EmulatorPkg/Unix/lldbefi.py>
> This code works just fine with LLDB and DWARF (e.g. XCODE5), though I have not yet completed these changes for my scripts for LLDB, only for GDB. However, with CLANGPDB generated files it is not functional. The reason for this is because LLDB is unaware of the underlying type, i.e. it does not know what is EFI_STATUS or UINT32. I can implement pretty-printing when LLDB knows about a typedef, but it is not possible to do this when the debug information is already gone or not parsed:
> 
> (lldb) p Status
> (unsigned long long) $1 = 0
> (lldb) p &Status
> (unsigned long long *) $2 = 0x000000007fe19ad8
> (lldb) p (EFI_STATUS)Status
> error: use of undeclared identifier 'EFI_STATUS'
> 
> Just in case I tried using exactly your code, and other stuff like source level debugging works just fine and symbolication works fine, so it should be some bug with PDB in particular.
> 
>> That is strange as globals usually work best? The common issue I've seen is getting the slide wrong. The EFI modules are linked at a value near zero and relocated into memory, so the slide represents that adjustment. 
>> 
>> You can use `image dump sections` and ` image dump symtab` to see lldb's view of symbols. More info here [1]. 
> 
> Yes, this one is a bit complicated, once again due to PDB most likely. It knows about global symbols, but does not list them in symtab:
> 
> (lldb) image dump symtab
> Dumping symbol table for 91 modules.
> Symtab, file = GdbSyms/Bin/X64_CLANGPDB/GdbSyms.dll, num_symbols = 0
> Symtab, file = /Users/user/Documents/UefiWorkspace/Build/OvmfX64/NOOPT_CLANGPDB/X64/MdeModulePkg/Core/Dxe/DxeMain/DEBUG/DxeCore.dll, num_symbols = 0
> Symtab, file = /Users/user/Documents/UefiWorkspace/Build/OvmfX64/NOOPT_CLANGPDB/X64/MdeModulePkg/Universal/DevicePathDxe/DevicePathDxe/DEBUG/DevicePathDxe.dll, num_symbols = 0
> …
> 
> The slides are correct, but there are two nuances that collide with it.
> 
> 1. There are multiple instances of the globals with the same name (e.g. gBS), but for some reason LLDB always tries to print the globals from the first module. This happens even when I am source-level debugging, and I see a gBS symbol from another module (e.g. DxeCore) used right at the same line. With GDB the closest symbol is used, but with LLDB it is always coming from the first module. I tried checking expr help to find whether I can pass it a module explicitly, but also failed.
> 

Usually what happens with lldb is you get the global that is in scope for the current frame. 

> 2.  To be able to get EFI types to locate the EFI_SYSTEM_TABLE_POINTER I add a dummy GdbSyms image, which is not loaded to the firmware. So basically I cannot slide what is not in the memory, and this is also my first image. I tried deleting it anyhow, but it failed for me.
> 

I've not used the fake image to get things done so I can't speak to that. I have used a fake target so I could have XIP PEIM and shadowed PEIM address available at the same time. You can't have a module loaded at 2 addresses at the same time in llldb. But you might be able to use a fake target for your fake stuff?

Just in case:

          # create a faka target to store info about symbols
          PeiXipTarget = target.debugger.CreateTarget (None, "i386-apple-macosx", "remote-macosx", True, error)

          # make sure the gdb-remote  connection target is the active target
          target.debugger.SetSelectedTarget (target)


> (lldb) image dump sections
> Dumping sections for 91 modules.
> Sections for 'GdbSyms/Bin/X64_CLANGPDB/GdbSyms.dll' (x86_64):
>   SectID     Type             Load Address                             Perm File Off.  File Size  Flags      Section Name
>   ---------- ---------------- ---------------------------------------  ---- ---------- ---------- ---------- ----------------------------
>   0xffffffffffffffff container        [0x0000000000000000-0x0000000000006ec0)* ---  0x00000000 0x00000000 0x00000000 GdbSyms.dll.
>   0x00000001 code             [0x0000000000000220-0x0000000000005bd6)* ---  0x00000220 0x000059c0 0x60000020 GdbSyms.dll...text
>   0x00000002 data             [0x0000000000005be0-0x0000000000006d79)* ---  0x00005be0 0x000011a0 0x40000040 GdbSyms.dll...rdata
>   0x00000003 data             [0x0000000000006d80-0x0000000000006e30)* ---  0x00006d80 0x00000060 0xc0000040 GdbSyms.dll...data
>   0x00000004 regular          [0x0000000000006e40-0x0000000000006ea4)* ---  0x00006de0 0x00000080 0x42000040 GdbSyms.dll...reloc
> Sections for '/Users/user/Documents/UefiWorkspace/Build/OvmfX64/NOOPT_CLANGPDB/X64/MdeModulePkg/Core/Dxe/DxeMain/DEBUG/DxeCore.dll' (x86_64):
>   SectID     Type             Load Address                             Perm File Off.  File Size  Flags      Section Name
>   ---------- ---------------- ---------------------------------------  ---- ---------- ---------- ---------- ----------------------------
>   0xffffffffffffffff container        [0x0000000000000000-0x00000000000523a0)* ---  0x00000000 0x00000000 0x00000000 DxeCore.dll.
>   0x00000001 code             [0x000000007fe1b220-0x000000007fe61e34)  ---  0x00000220 0x00046c20 0x60000020 DxeCore.dll...text
>   0x00000002 data             [0x000000007fe61e40-0x000000007fe68065)  ---  0x00046e40 0x00006240 0x40000040 DxeCore.dll...rdata
>   0x00000003 data             [0x000000007fe68080-0x000000007fe6d160)  ---  0x0004d080 0x000018a0 0xc0000040 DxeCore.dll...data
>   0x00000004 regular          [0x000000007fe6d160-0x000000007fe6d398)  ---  0x0004e920 0x00000240 0x42000040 DxeCore.dll...reloc
> Sections for '/Users/user/Documents/UefiWorkspace/Build/OvmfX64/NOOPT_CLANGPDB/X64/MdeModulePkg/Universal/DevicePathDxe/DevicePathDxe/DEBUG/DevicePathDxe.dll' (x86_64):
>   SectID     Type             Load Address                             Perm File Off.  File Size  Flags      Section Name
>   ---------- ---------------- ---------------------------------------  ---- ---------- ---------- ---------- ----------------------------
>   0xffffffffffffffff container        [0x0000000000000000-0x0000000000014420)* ---  0x00000000 0x00000000 0x00000000 DevicePathDxe.dll.
>   0x00000001 code             [0x000000007f986220-0x000000007f996cc6)  ---  0x00000220 0x00010ac0 0x60000020 DevicePathDxe.dll...text
>   0x00000002 data             [0x000000007f996ce0-0x000000007f999b04)  ---  0x00010ce0 0x00002e40 0x40000040 DevicePathDxe.dll...rdata
>   0x00000003 data             [0x000000007f999b20-0x000000007f99a1a2)  ---  0x00013b20 0x00000660 0xc0000040 DevicePathDxe.dll...data
>   0x00000004 regular          [0x000000007f99a1c0-0x000000007f99a404)  ---  0x00014180 0x00000260 0x42000040 DevicePathDxe.dll…reloc
> …
> 
> So, all in all, unique global variables work, but there is no way to access duplicating variables. They either resolve to GdbSyms or just cause a crash:
> 
> (lldb) p mDebugInfoTableHeader
> (EFI_DEBUG_IMAGE_INFO_TABLE_HEADER) $0 = {
>   UpdateStatus = 2
>   TableSize = 92
>   EfiDebugImageInfoTable = 0x000000007f814018
> }
> (lldb) p gBS
> error: Couldn't materialize: couldn't get the value of variable ::gBS: read memory from 0x6df8 failed
> error: errored out in DoExecute, couldn't PrepareToExecuteJITExpression
> (lldb) p gEfiGlobalVariableGuid
> 0  libLLVM.dylib            0x000000010e52ee68 llvm::sys::PrintStackTrace(llvm::raw_ostream&) + 40
> 1  libLLVM.dylib            0x000000010e52f262 SignalHandler(int) + 188
> 2  libsystem_platform.dylib 0x00007fff6ca5642d _sigtramp + 29
> ...
> 

If you want to inspect globals I think this logic works to get you data, you would need to print it out etc. 

SBValueList = lldb.target.FindGlobalVariables ("gST", 1024)
for SBValue in SBValueList:
	Module = SBValue.GetAddress().GetModule() 
        ModuleStr = SBValue.GetAddress().GetModule().GetFileSpec().GetFilename()
        Start = int (SBValue.GetLocation(), 0)
        End = Start + SBValue.GetByteSize() - 1
        SBDeclaration = SBValue.GetDeclaration()
        Column = SBDeclaration.GetColumn()

I wrote a command in the early days to dump out all the instances of a global. 

You can also try (lldb) image lookup -Av --name gST

>> You can tell lldb to use the older Python like this (from the Terminal.app):
>> $ defaults write com.apple.dt.lldb DefaultPythonVersion 2
> 
> Thanks, that helped quite a bit, but for some reason Xcode version still crashes more for me. I attached a couple of stack traces if you feel like having a look, but once again it seems that it is all about the PDB plugin.
> 
>> For the macOS API clang emits frame pointers, so you can walk the stack without symbols. You could try adding the compiler flag to emit the frame pointers. 
> 

This is easy enough to check as %rpb is the frame pointer so it will get saved/restored on function entry/exit. 

> I am pretty sure stack frames are not disabled with UEFI, as sometimes backtracing works just fine. To me it looks like debug information parsing randomly breaks in LLDB, and once it happens it forgets about other images:
> 
> (lldb) b CoreLocateHandleBuffer
> Breakpoint 2: where = DxeCore.dll`CoreLocateHandleBuffer + 31 at Locate.c:649, address = 0x000000007fe36e4f
> (lldb) c
> Process 1 resuming
> Process 1 stopped
> * thread #1, stop reason = breakpoint 2.1
>     frame #0: 0x000000007fe36e4f DxeCore.dll`CoreLocateHandleBuffer(SearchType=ByProtocol, Protocol=0x000000007f978160, SearchKey=0x0000000000000000, NumberHandles=0x000000007fe19fd8, Buffer=0x000000007fe19fc0) at Locate.c:649
>    646 	  EFI_STATUS          Status;
>    647 	  UINTN               BufferSize;
>    648 	
> -> 649 	  if (NumberHandles == NULL) {
>    650 	    return EFI_INVALID_PARAMETER;
>    651 	  }
>    652 	
> (lldb) bt
> * thread #1, stop reason = breakpoint 2.1
>   * frame #0: 0x000000007fe36e4f DxeCore.dll`CoreLocateHandleBuffer(SearchType=ByProtocol, Protocol=0x000000007f978160, SearchKey=0x0000000000000000, NumberHandles=0x000000007fe19fd8, Buffer=0x000000007fe19fc0) at Locate.c:649
>     frame #1: 0x000000007fe36816 DxeCore.dll`CoreLocateDevicePath(Protocol=0x000000007f978160, DevicePath=0x000000007fe1a060, Device=0x000000007fe1a068) at Locate.c:466
>     frame #2: 0x000000007f97479a SecurityStubDxe.dll
> 
> ———
> 
> (lldb) b CopyMem
> Breakpoint 3: 70 locations.
> (lldb) c
> Process 1 resuming
> Process 1 stopped
> * thread #1, stop reason = breakpoint 2.53 3.53
>     frame #0: 0x000000007e5c13b3 MnpDxe.dll`CopyMem(DestinationBuffer=0x000000007fe19b50, SourceBuffer=0x000000007e2aa470, Length=656) at CopyMemWrapper.c:47
>    44  	  IN UINTN       Length
>    45  	  )
>    46  	{
> -> 47  	  if (Length == 0) {
>    48  	    return DestinationBuffer;
>    49  	  }
>    50  	  ASSERT ((Length - 1) <= (MAX_ADDRESS - (UINTN)DestinationBuffer));
> (lldb) bt
> * thread #1, stop reason = breakpoint 2.53 3.53
>   * frame #0: 0x000000007e5c13b3 MnpDxe.dll`CopyMem(DestinationBuffer=0x000000007fe19b50, SourceBuffer=0x000000007e2aa470, Length=656) at CopyMemWrapper.c:47
> (lldb) finish
> error: Could not create return address breakpoint.
> (lldb) n
> Process 1 stopped
> * thread #1, stop reason = step over
>     frame #0: 0x000000007e5c13ce MnpDxe.dll`CopyMem(DestinationBuffer=0x000000007fe19b50, SourceBuffer=0x000000007e2aa470, Length=656) at CopyMemWrapper.c:50
>    47  	  if (Length == 0) {
>    48  	    return DestinationBuffer;
>    49  	  }
> -> 50  	  ASSERT ((Length - 1) <= (MAX_ADDRESS - (UINTN)DestinationBuffer));
>    51  	  ASSERT ((Length - 1) <= (MAX_ADDRESS - (UINTN)SourceBuffer));
>    52  	
>    53  	  if (DestinationBuffer == SourceBuffer) {
> (lldb)
> ...
> Process 1 stopped
> * thread #1, stop reason = step over
>     frame #0: 0x000000007e5c14b4 MnpDxe.dll`CopyMem(DestinationBuffer=0x000000007fe19b50, SourceBuffer=0x000000007e2aa470, Length=656) at CopyMemWrapper.c:57
>    54  	    return DestinationBuffer;
>    55  	  }
>    56  	  return InternalMemCopyMem (DestinationBuffer, SourceBuffer, Length);
> -> 57  	}
> (lldb)
> Process 1 stopped
> * thread #1, stop reason = step over
>     frame #0: 0x000000007e5c726e MnpDxe.dll
> ->  0x7e5c726e: mov    rax, qword ptr [rsp + 0x60]
>     0x7e5c7273: cmp    byte ptr [rax + 0x68], 0x0
>     0x7e5c7277: jne    0x7e5c7291
>     0x7e5c727d: movabs rax, -0x7fffffffffffffed
> (lldb) bt
> * thread #1, stop reason = step over
>   * frame #0: 0x000000007e5c726e MnpDxe.dll 
> 
> ———
> 
> (lldb) c
> Process 1 resuming
> Process 1 stopped
> * thread #1, stop reason = signal SIGINT
>     frame #0: 0x000000007fe4d72e DxeCore.dll
> ->  0x7fe4d72e: cmp    al, 0x0
>     0x7fe4d730: je     0x7fe4d765
>     0x7fe4d736: mov    rcx, qword ptr [rsp + 0x20]
>     0x7fe4d73b: call   0x7fe4c4b0
> (lldb) bt
> * thread #1, stop reason = signal SIGINT
>   * frame #0: 0x000000007fe4d72e DxeCore.dll
> 
>> On macOS the Mach-O and dSYM have a UUID (dwarfdump -u) that is indexed by Spotlight (mdfind "com_apple_xcode_dsym_uuids == *") [2]
>> This should be the UUID in the debug directory entry and you can use that to lookup the symbols like this:
>> 
>> module = target.AddModule (None, None, uuid)
>> SBError = target.SetModuleLoadAddress (module, LoadAddress + TeAdjust)
>> 
>> Also lldb has built in help for commands, but it is kind of terse since it is autogenerated from the C++ swig. 
>> (lldb) script help (lldb.target.AddModule)
>> Help on method AddModule in module lldb:
>> 
>> AddModule(self, *args) method of lldb.SBTarget instance
>>     AddModule(SBTarget self, SBModule module) -> bool
>>     AddModule(SBTarget self, char const * path, char const * triple, char const * uuid) -> SBModule
>>     AddModule(SBTarget self, char const * path, char const * triple, char const * uuid_cstr, char const * symfile) -> SBModule
>>     AddModule(SBTarget self, SBModuleSpec module_spec) -> SBModule
>> 
>> The minimum  you need to symbolicate a frame is uuid, LoadAddress, and PC. 
>> 
>> [1] http://lldb.llvm.org/use/map.html <http://lldb.llvm.org/use/map.html>
>> [2] http://lldb.llvm.org/use/symbols.html <http://lldb.llvm.org/use/symbols.html>
> Thanks for the links again. Yes, I am using some of these, and in fact for GDB that’s pretty much what I did when I worked with XCODE5. It is very likely that when I get to complete LLDB support for XCODE5 it will work quite fine too. But I am already happy with XCODE5 here, and making it even better will only help myself, but not other people with e.g. Linux or people that want me to use the same compiler with them.
> 

Thanks for looking out for others. 

Thanks,

Andrew Fish

> Best regards,
> Vitaly
> 
> 
>> 21 марта 2020 г., в 20:13, Andrew Fish <afish@apple.com <mailto:afish@apple.com>> написал(а):
>> 
>> 
>> 
>>> On Mar 21, 2020, at 3:28 AM, Vitaly Cheptsov <cheptsov@ispras.ru <mailto:cheptsov@ispras.ru>> wrote:
>>> 
>>> Hello,
>>> 
>>> Andrey, thanks for the hint, it was very helpful. I rewrote the GDB scripts to work with LLDB[1] and was able to debug OVMF built with CLANGPDB. While it is still quite dirty, at the very least it works.
>>> 
>>> Unfortunately the experience was close to terrible. I may certainly do something wrong, but it is clear that PDB and LLDB do not support each other well enough. After spending several hours on playing with the tools my conclusion is that LLDB is simply not suited for UEFI PDB debugging, and we really want DWARF  as there is no other opensource debugger that supports PDB on macOS and Linux
>>> 
>>> In case somebody knows workarounds here are the issues I faced:
>>> 
>>> 1. All integer alias typedefs are discarded in favour of underlying types. This way EFI_STATUS and EFI_TPL become unsigned long long, CHAR8 becomes char, and CHAR16 becomes unsigned short. It does not look like LLDB has the original types anywhere at all, and it also does not have them registered.
>>> 
>>>     frame #0: 0x000000007fe242aa DxeCore.dll`CoreAllocatePoolPagesI(PoolType=EfiBootServicesData, NoPages=1, Granularity=4096, NeedGuard='\0') at Pool.c:322
>>>    319 	    return NULL;
>>>    320 	  }
>>>    321 	
>>> -> 322 	  Buffer = CoreAllocatePoolPages (PoolType, NoPages, Granularity, NeedGuard);
>>>    323 	  CoreReleaseMemoryLock ();
>>>    324 	
>>>    325 	  if (Buffer != NULL) {
>>> (lldb) p Status
>>> (unsigned long long) $3 = 0
>>> 
>>> Structures work more or less fine, but for simpler types like strings we are out of even potential pretty-printing.
>>> 
>> 
>> Vitaly,
>> 
>> You can teach lldb about types. There is some example code here: https://github.com/tianocore/edk2/blob/master/EmulatorPkg/Unix/lldbefi.py <https://github.com/tianocore/edk2/blob/master/EmulatorPkg/Unix/lldbefi.py>
>>> 2. Global variables are not accessible. I am not sure what happens, but they either seem to not relocate or conflict with the other names:
>>> 
>>> (lldb) p gST
>>> error: Couldn't materialize: couldn't get the value of variable ::gST: read memory from 0x6e18 failed
>>> error: errored out in DoExecute, couldn't PrepareToExecuteJITExpression
>>> (lldb) p &gST
>>> error: Couldn't materialize: couldn't get the value of variable ::gST: read memory from 0x6e18 failed
>>> error: errored out in DoExecute, couldn't PrepareToExecuteJITExpression
>>> 
>> 
>> That is strange as globals usually work best? The common issue I've seen is getting the slide wrong. The EFI modules are linked at a value near zero and relocated into memory, so the slide represents that adjustment. 
>> 
>> You can use `image dump sections` and ` image dump symtab` to see lldb's view of symbols. More info here [1]. 
>> 
>>> 3. Quite a number of crashes.
>>> 
>>> In most cases autocompletion by tab press causes a crash. E.g.
>>> 
>>> b I<TAB>
>>> 
>>> So will do printing of a GUID, e.g. p gEfiGlobalVariableGuid.
>>> 
>>> This may have to do with Python compatibility as Xcode 11 LLDB that uses Python 3 generally crashes more often than MacPorts LLDB 9.0. Surprisingly structures work more or less fine.
>>> 
>> 
>> You can tell lldb to use the older Python like this (from the Terminal.app):
>> $ defaults write com.apple.dt.lldb DefaultPythonVersion 2
>> 
>>> 4. Ctrl+C does not produce a valid backtrace. When I break with a breakpoint, I see a proper stacktrace with more than one entry, with function prototypes and values. When I break with Ctrl+C I only see some weird backtrace with most of the entries missing regardless of frame position:
>>> 
>>> (lldb) bt
>>> * thread #1, stop reason = signal SIGTRAP
>>>   * frame #0: 0x000000007fe4c5f3 DxeCore.dll
>>> 
>>> Probably more and all the unintuitive stuff like the lack of more functional TUI, but it is hard to remember all the trials.
>>> 
>> 
>> For the macOS API clang emits frame pointers, so you can walk the stack without symbols. You could try adding the compiler flag to emit the frame pointers. 
>> 
>> 
>>> [1] https://github.com/acidanthera/OpenCorePkg/blob/master/Debug/Scripts/lldb_uefi.py <https://github.com/acidanthera/OpenCorePkg/blob/master/Debug/Scripts/lldb_uefi.py>
>>> 
>> 
>> On macOS the Mach-O and dSYM have a UUID (dwarfdump -u) that is indexed by Spotlight (mdfind "com_apple_xcode_dsym_uuids == *") [2]
>> This should be the UUID in the debug directory entry and you can use that to lookup the symbols like this:
>> 
>> module = target.AddModule (None, None, uuid)
>> SBError = target.SetModuleLoadAddress (module, LoadAddress + TeAdjust)
>> 
>> Also lldb has built in help for commands, but it is kind of terse since it is autogenerated from the C++ swig. 
>> (lldb) script help (lldb.target.AddModule)
>> Help on method AddModule in module lldb:
>> 
>> AddModule(self, *args) method of lldb.SBTarget instance
>>     AddModule(SBTarget self, SBModule module) -> bool
>>     AddModule(SBTarget self, char const * path, char const * triple, char const * uuid) -> SBModule
>>     AddModule(SBTarget self, char const * path, char const * triple, char const * uuid_cstr, char const * symfile) -> SBModule
>>     AddModule(SBTarget self, SBModuleSpec module_spec) -> SBModule
>> 
>> The minimum  you need to symbolicate a frame is uuid, LoadAddress, and PC. 
>> 
>> [1] http://lldb.llvm.org/use/map.html <http://lldb.llvm.org/use/map.html>
>> [2] http://lldb.llvm.org/use/symbols.html <http://lldb.llvm.org/use/symbols.html>
>> 
>> Thanks,
>> 
>> Andrew Fish
>> 
>> 
>>> Best wishes,
>>> Vitaly
>>> 
>>>> 20 марта 2020 г., в 22:14, Andrew Fish <afish@apple.com <mailto:afish@apple.com>> написал(а):
>>>> 
>>>> 
>>>> 
>>>>> On Mar 20, 2020, at 8:13 AM, Vitaly Cheptsov <cheptsov@ispras.ru <mailto:cheptsov@ispras.ru>> wrote:
>>>>> 
>>>>> Hello,
>>>>> 
>>>>> We noticed that the original bugzilla, which intended to add new LLVM toolchain support[1], also wanted to bring ELF format support with DWARF debugging information. For some reason this did not make its way into EDK II, and we are currently wondering, how can one debug binaries built with LLVM 9.0.
>>>>> 
>>>>> For macOS and XCODE5 toolchain we use GDB scripts based on Andrei Warkentin’s work, which allow us to integrate with QEMU and VMware[2]. It is likely that they should work with little to no work on Linux with CLANG38/GCC5 with GDB once again. However, CLANGPDB apparently is using PDB debugging information, which I believe is not handled with GDB.
>>>>> 
>>>>> Could you please provide the details on the matter and let us know about the recommended route?
>>>>> — Is dropping CLANGELF just a temporary measure and it should be resubmitted again?
>>>>> — Should LLDB, which seems to be aware of PDB, be used instead of GDB, when building with CLANGPDB? If so, did anybody try that?
>>>>> 
>>>> 
>>>> Vitaly,
>>>> 
>>>> I've not tried the CLANGPDB path, but if you want to connect lldb to QEMU you need to set  plugin.process.gdb-remote.target-definition-file [1] to [2]. 
>>>> 
>>>> [1]  lldb -o "settings set plugin.process.gdb-remote.target-definition-file x86_64_target_definition.py" -o "gdb-remote 9000"
>>>> [2] https://github.com/llvm-mirror/lldb/blob/master/examples/python/x86_64_target_definition.py <https://github.com/llvm-mirror/lldb/blob/master/examples/python/x86_64_target_definition.py>
>>>> 
>>>> Thanks,
>>>> 
>>>> Andrew Fish
>>>> 
>>>>> Thanks!
>>>>> 
>>>>> Best regards,
>>>>> Vitaly
>>>>> 
>>>>> [1] https://bugzilla.tianocore.org/show_bug.cgi?id=1603 <https://bugzilla.tianocore.org/show_bug.cgi?id=1603>
>>>>> [2] https://github.com/acidanthera/OpenCorePkg/blob/master/Debug/Scripts/gdb_uefi.py <https://github.com/acidanthera/OpenCorePkg/blob/master/Debug/Scripts/gdb_uefi.py>
>>>>> 
>>> 
> 
> 
> <crashes.txt>


[-- Attachment #2: Type: text/html, Size: 79857 bytes --]

Re: [edk2-devel] CLANGPDB binary debugging

[Vitaly Cheptsov]

[-- Attachment #1.1: Type: text/plain, Size: 46892 bytes --]

Andrew,

I investigated further and was able to make some progress. I got rid of PDB, and changed debug information generation to DWARF[1]. This resolved most of LLDB crashes and made pretty-printing work just fine as type aliases like EFI_STATUS now work correctly (I am still to commit this). Unfortunately there are several downsides. Dead code stripping in LLD linker is broken when generating PE with DWARF, so I had to patch it[2]. Debug link implementation in llvm-objcopy is also quite ugly, we do not get MTOC or anything alike[3], just some section with a random name containing base filename.

I can submit a more proper patch to EDK II properly implementing CLANGDWARF toolchain instead of patching CLANGPDB like in [1], but it is questionable whether it is worth it. LLVM 10 will land very soon, and the necessary LLD patches[2] will definitely not make their way in this release. We will have to wait for LLVM 11 at the very least, and I am not sure I am ready to drive llvm-objcopy changes either. Liming, what do you think?

> I've not used the fake image to get things done so I can't speak to that. I have used a fake target so I could have XIP PEIM and shadowed PEIM address available at the same time. You can't have a module loaded at 2 addresses at the same time in llldb. But you might be able to use a fake target for your fake stuff?
> 
> Just in case:
> 
>           # create a faka target to store info about symbols
>           PeiXipTarget = target.debugger.CreateTarget (None, "i386-apple-macosx", "remote-macosx", True, error)
> 
>           # make sure the gdb-remote  connection target is the active target
>           target.debugger.SetSelectedTarget (target)

Yes, that was actually a smart idea. I initially did it that way, as I had mixing Mach-O and PE triples, but later removed, which was actually a mistake. Now I have a fake target for GdbSyms and a proper target for other images[4]. This somewhat solves the issue with invalid addresses for gST, as gST from DxeCore is now picked. However, I cannot access global variables from other modules for some reason still.

> If you want to inspect globals I think this logic works to get you data, you would need to print it out etc.
> 
> SBValueList = lldb.target.FindGlobalVariables ("gST", 1024)
> for SBValue in SBValueList:
> 	Module = SBValue.GetAddress().GetModule()
>         ModuleStr = SBValue.GetAddress().GetModule().GetFileSpec().GetFilename()
>         Start = int (SBValue.GetLocation(), 0)
>         End = Start + SBValue.GetByteSize() - 1
>         SBDeclaration = SBValue.GetDeclaration()
>         Column = SBDeclaration.GetColumn()
> 
> I wrote a command in the early days to dump out all the instances of a global.
> 
> You can also try (lldb) image lookup -Av --name gST


Something is really weird, gST from the first image shows, but lldb believes that it does not exist. lldb.target.FindGlobalVariables ("gST", 1024) also returns an empty list.

I noticed that LLDB shows invalid type for gST in the symtab, there also is no address. I started to wonder whether this is the cause and decided to check XCODE5, as the the issue could have been with PE format. Unfortunately even after I tried XCODE5 support with LLDB, I got exactly the same problem. I am quite clueless what is going on. Below I included two transcripts: for CLANGDWARF and XCODE5.

(lldb) p gST
(EFI_SYSTEM_TABLE *) $0 = 0x000000007f9ee018
(lldb) p *gST
(EFI_SYSTEM_TABLE) $1 = {
  Hdr = (Signature = 0x5453595320494249, Revision = 0x00020046, HeaderSize = 0x00000078, CRC32 = 0x3d5e9b5d, Reserved = 0x00000000)
  FirmwareVendor = 0x000000007f9e9c98
  FirmwareRevision = 0x00010000
  ConsoleInHandle = 0x000000007ed58f18
  ConIn = 0x000000007e6edcb8
  ConsoleOutHandle = 0x000000007ed57018
  ConOut = 0x000000007e6ede40
  StandardErrorHandle = 0x000000007ed57a98
  StdErr = 0x000000007e6ee030
  RuntimeServices = 0x000000007f9eeb98
  BootServices = 0x000000007fe689e8
  NumberOfTableEntries = 0x000000000000000a
  ConfigurationTable = 0x000000007f9eec98
}
(lldb) image lookup -Av --name gST
(lldb) image dump sections
Dumping sections for 90 modules.
Sections for '/Users/user/Documents/UefiWorkspace/Build/OvmfX64/NOOPT_CLANGPDB/X64/DxeCore.debug' (x86_64):
  SectID     Type             Load Address                             Perm File Off.  File Size  Flags      Section Name
  ---------- ---------------- ---------------------------------------  ---- ---------- ---------- ---------- ----------------------------
  0xffffffffffffffff container        [0x000000007fe1b000-0x000000007ff38ac0)  ---  0x00000000 0x00000000 0x00000000 DxeCore.debug.
  0x00000001 code             [0x000000007fe1b320-0x000000007fe61f34)  ---  0x00000320 0x00046c20 0x60000020 DxeCore.debug...text
  0x00000002 data             [0x000000007fe61f40-0x000000007fe680f1)  ---  0x00046f40 0x000061c0 0x40000040 DxeCore.debug...rdata
  0x00000003 data             [0x000000007fe68100-0x000000007fe6d1e0)  ---  0x0004d100 0x000018a0 0xc0000040 DxeCore.debug...data
  0x00000004 regular          [0x000000007fe6d1e0-0x000000007fe6d424)  ---  0x0004e9a0 0x00000260 0x42000040 DxeCore.debug...reloc
  0x00000005 dwarf-abbrev     [0x000000007fe6d440-0x000000007fe74059)  ---  0x0004ec00 0x00006c20 0x42000040 DxeCore.debug...debug_abbrev
  0x00000006 dwarf-info       [0x000000007fe74060-0x000000007fec1355)  ---  0x00055820 0x0004d300 0x42000040 DxeCore.debug...debug_info
  0x00000007 dwarf-line       [0x000000007fec1360-0x000000007fef9fd4)  ---  0x000a2b20 0x00038c80 0x42000040 DxeCore.debug...debug_line
  0x00000008 dwarf-macinfo    [0x000000007fef9fe0-0x000000007fefa056)  ---  0x000db7a0 0x00000080 0x42000040 DxeCore.debug...debug_macinfo
  0x00000009 dwarf-ranges     [0x000000007fefa060-0x000000007feff610)  ---  0x000db820 0x000055c0 0x42000040 DxeCore.debug...debug_ranges
  0x0000000a dwarf-str        [0x000000007feff620-0x000000007ff38abd)  ---  0x000e0de0 0x000394a0 0x42000040 DxeCore.debug...debug_str
Sections for '/Users/user/Documents/UefiWorkspace/Build/OvmfX64/NOOPT_CLANGPDB/X64/DevicePathDxe.debug' (x86_64):
  SectID     Type             Load Address                             Perm File Off.  File Size  Flags      Section Name
  ---------- ---------------- ---------------------------------------  ---- ---------- ---------- ---------- ----------------------------
  0xffffffffffffffff container        [0x000000007f986000-0x000000007f9cb0a0)  ---  0x00000000 0x00000000 0x00000000 DevicePathDxe.debug.
  0x00000001 code             [0x000000007f986320-0x000000007f996dc6)  ---  0x00000320 0x00010ac0 0x60000020 DevicePathDxe.debug...text
  0x00000002 data             [0x000000007f996de0-0x000000007f999b75)  ---  0x00010de0 0x00002da0 0x40000040 DevicePathDxe.debug...rdata
  0x00000003 data             [0x000000007f999b80-0x000000007f99a202)  ---  0x00013b80 0x00000660 0xc0000040 DevicePathDxe.debug...data
  0x00000004 regular          [0x000000007f99a220-0x000000007f99a464)  ---  0x000141e0 0x00000260 0x42000040 DevicePathDxe.debug...reloc
  0x00000005 dwarf-abbrev     [0x000000007f99a480-0x000000007f99bd09)  ---  0x00014440 0x000018a0 0x42000040 DevicePathDxe.debug...debug_abbrev
  0x00000006 dwarf-info       [0x000000007f99bd20-0x000000007f9ad8b7)  ---  0x00015ce0 0x00011ba0 0x42000040 DevicePathDxe.debug...debug_info
  0x00000007 dwarf-line       [0x000000007f9ad8c0-0x000000007f9bc1f7)  ---  0x00027880 0x0000e940 0x42000040 DevicePathDxe.debug...debug_line
  0x00000008 dwarf-macinfo    [0x000000007f9bc200-0x000000007f9bc223)  ---  0x000361c0 0x00000040 0x42000040 DevicePathDxe.debug...debug_macinfo
  0x00000009 dwarf-ranges     [0x000000007f9bc240-0x000000007f9be020)  ---  0x00036200 0x00001de0 0x42000040 DevicePathDxe.debug...debug_ranges
  0x0000000a dwarf-str        [0x000000007f9be020-0x000000007f9cb099)  ---  0x00037fe0 0x0000d080 0x42000040 DevicePathDxe.debug...debug_str
…
(lldb) image dump symtab
Dumping symbol table for 90 modules.
Symtab, file = /Users/user/Documents/UefiWorkspace/Build/OvmfX64/NOOPT_CLANGPDB/X64/DxeCore.debug, num_symbols = 4343:
               Debug symbol
               |Synthetic symbol
               ||Externally Visible
               |||
Index   UserID DSX Type            File Address/Value Load Address       Size               Flags      Name
------- ------ --- --------------- ------------------ ------------------ ------------------ ---------- ----------------------------------
[    0] 4294967295     Invalid         0x0000000000000000 0x000000007fe1b000 0x000000000000003a 0x00000000 .text
[    1] 4294967295     Invalid         0x0000000000000000                    0x0000000000000000 0x00000000 .data
[    2] 4294967295     Invalid         0x0000000000001898                    0x0000000000000000 0x00000000 .bss
[    3] 4294967295     Invalid         0x0000000000046bc0 0x000000007fe61bc0 0x0000000000000374 0x00000000 .text$_ModuleEntryPoint
[    4] 4294967295     Code            0x0000000000046bc0 0x000000007fe61bc0 0x0000000000000374 0x00000000 _ModuleEntryPoint
[    5] 4294967295     Invalid         0x0000000000001b08                    0x0000000000000000 0x00000000 .bss$gHobList
[    6] 4294967295     Invalid         0x0000000000001b08                    0x0000000000000000 0x00000000 gHobList
[    7] 4294967295     Invalid         0x0000000000000000                    0x0000000000000000 0x00000000 .rdata
[    8] 4294967295     Invalid         0x0000000000000000                    0x0000000000000000 0x00000000 .str
[    9] 4294967295     Invalid         0x0000000000000058                    0x0000000000000000 0x00000000 .rdata
[   10] 4294967295     Invalid         0x0000000000000058                    0x0000000000000000 0x00000000 .str.1
[   11] 4294967295     Invalid         0x0000000000000000                    0x0000000000000000 0x00000000 .debug_str
[   12] 4294967295     Invalid         0x0000000000000000                    0x0000000000000000 0x00000000 .debug_abbrev
[   13] 4294967295     Invalid         0x0000000000000000                    0x0000000000000000 0x00000000 .debug_info
[   14] 4294967295     Invalid         0x0000000000000000                    0x0000000000000000 0x00000000 .debug_ranges
[   15] 4294967295     Invalid         0x0000000000000000                    0x0000000000000000 0x00000000 .debug_macinfo
[   16] 4294967295     Invalid         0x0000000000000000                    0x0000000000000000 0x00000000 .debug_line
[   17] 4294967295     Code            0x000000000003f680 0x000000007fe5a680 0x0000000000000020 0x00000000 ProcessModuleEntryPointList
[   18] 4294967295     Code            0x0000000000023130 0x000000007fe3e130 0x0000000000000020 0x00000000 DebugAssertEnabled
[   19] 4294967295     Code            0x0000000000023060 0x000000007fe3e060 0x00000000000000d0 0x00000000 DebugAssert
[   20] 4294967295     Code            0x0000000000022150 0x000000007fe3d150 0x0000000000000020 0x00000000 CpuDeadLoop
[   21] 4294967295     Invalid         0x0000000000000000 0x000000007fe1b000 0x000000000000003a 0x00000000 .text
[   22] 4294967295     Invalid         0x0000000000000000                    0x0000000000000000 0x00000000 .data
[   23] 4294967295     Invalid         0x0000000000001898                    0x0000000000000000 0x00000000 .bss
[   24] 4294967295     Invalid         0x000000000003f240 0x000000007fe5a240 0x0000000000000440 0x00000000 .text$ProcessLibraryConstructorList
[   25] 4294967295     Code            0x000000000003f240 0x000000007fe5a240 0x0000000000000440 0x00000000 ProcessLibraryConstructorList
[   26] 4294967295     Invalid         0x000000000003f680 0x000000007fe5a680 0x0000000000000020 0x00000000 .text$ProcessModuleEntryPointList
[   27] 4294967295     Invalid         0x0000000000000070                    0x0000000000000000 0x00000000 .data$gEfiCallerIdGuid
[   28] 4294967295     Invalid         0x0000000000000070                    0x0000000000000000 0x00000000 gEfiCallerIdGuid
[   29] 4294967295     Invalid         0x0000000000000148                    0x0000000000000000 0x00000000 .data$gEfiEventMemoryMapChangeGuid
[   30] 4294967295     Invalid         0x0000000000000148                    0x0000000000000000 0x00000000 gEfiEventMemoryMapChangeGuid
[   31] 4294967295     Invalid         0x0000000000000168                    0x0000000000000000 0x00000000 .data$gEfiEventVirtualAddressChangeGuid
[   32] 4294967295     Invalid         0x0000000000000168                    0x0000000000000000 0x00000000 gEfiEventVirtualAddressChangeGuid
[   33] 4294967295     Invalid         0x0000000000000138                    0x0000000000000000 0x00000000 .data$gEfiEventExitBootServicesGuid
[   34] 4294967295     Invalid         0x0000000000000138                    0x0000000000000000 0x00000000 gEfiEventExitBootServicesGuid
….
[  303] 4294967295     Invalid         0x0000000000001b60                    0x0000000000000000 0x00000000 .bss$gST
[  304] 4294967295     Invalid         0x0000000000001b60                    0x0000000000000000 0x00000000 gST
[  305] 4294967295     Invalid         0x00000000000018a0                    0x0000000000000000 0x00000000 .bss$gBS
[  306] 4294967295     Invalid         0x00000000000018a0                    0x0000000000000000 0x00000000 gBS
…

———————————————

(lldb) p gST
(EFI_SYSTEM_TABLE *) $2 = 0x000000007fbee018
(lldb) p &gST
(EFI_SYSTEM_TABLE **) $4 = 0x000000007fe6d788
(lldb) p *gST
(EFI_SYSTEM_TABLE) $3 = {
  Hdr = (Signature = 0x5453595320494249, Revision = 0x00020046, HeaderSize = 0x00000078, CRC32 = 0x5355dbe2, Reserved = 0x00000000)
  FirmwareVendor = 0x000000007fbe9c98
  FirmwareRevision = 0x00010000
  ConsoleInHandle = 0x000000007ede5a98
  ConIn = 0x000000007e95f640
  ConsoleOutHandle = 0x000000007ede4e98
  ConOut = 0x000000007e95f800
  StandardErrorHandle = 0x000000007ede4b98
  StdErr = 0x000000007e95f910
  RuntimeServices = 0x000000007fbeeb98
  BootServices = 0x000000007fe6cde8
  NumberOfTableEntries = 0x0000000000000009
  ConfigurationTable = 0x000000007fbeec98
}
(lldb) image lookup -Av --name gST
(lldb) image dump sections
Dumping sections for 88 modules.
Sections for '/Users/user/Documents/UefiWorkspace/Build/OvmfX64/NOOPT_XCODE5/X64/MdeModulePkg/Core/Dxe/DxeMain/DEBUG/DxeCore.dll' (x86_64):
  SectID     Type             Load Address                             Perm File Off.  File Size  Flags      Section Name
  ---------- ---------------- ---------------------------------------  ---- ---------- ---------- ---------- ----------------------------
  0x00000100 container        [0x000000007fe26240-0x000000007fe6bb40)  r-x  0x000004a0 0x00045900 0x00000000 DxeCore.dll.__TEXT
  0x00000001 code             [0x000000007fe26240-0x000000007fe65bd8)  r-x  0x000004a0 0x0003f998 0x80000400 DxeCore.dll.__TEXT.__text
  0x00000002 data-cstr        [0x000000007fe65bd8-0x000000007fe6ba23)  r-x  0x0003fe38 0x00005e4b 0x00000002 DxeCore.dll.__TEXT.__cstring
  0x00000003 regular          [0x000000007fe6ba24-0x000000007fe6ba6c)  r-x  0x00045c84 0x00000048 0x00000000 DxeCore.dll.__TEXT.__ustring
  0x00000004 regular          [0x000000007fe6ba70-0x000000007fe6bb40)  r-x  0x00045cd0 0x000000d0 0x00000000 DxeCore.dll.__TEXT.__const
  0x00000200 container        [0x000000007fe6bb40-0x000000007fe70e60)  rw-  0x00045da0 0x00001ae0 0x00000000 DxeCore.dll.__DATA
  0x00000005 regular          [0x000000007fe6bb40-0x000000007fe6bd58)  rw-  0x00045da0 0x00000218 0x00000000 DxeCore.dll.__DATA.__const
  0x00000006 data             [0x000000007fe6bd60-0x000000007fe6d610)  rw-  0x00045fc0 0x000018b0 0x00000000 DxeCore.dll.__DATA.__data
  0x00000007 zero-fill        [0x000000007fe6d610-0x000000007fe70e20)  rw-  0x00000000 0x00000000 0x00000001 DxeCore.dll.__DATA.__common
  0x00000008 zero-fill        [0x000000007fe70e20-0x000000007fe70e58)  rw-  0x00000000 0x00000000 0x00000001 DxeCore.dll.__DATA.__bss
  0x00000100 container        [0x000000000004b000-0x00000000000ef000)* rw-  0x00001000 0x000a3a74 0x00000000 DxeCore.dll.__DWARF
  0x00000001 dwarf-line       [0x000000000004b000-0x00000000000723fb)* rw-  0x00001000 0x000273fb 0x00000000 DxeCore.dll.__DWARF.__debug_line
  0x00000002 dwarf-pubnames   [0x00000000000723fb-0x0000000000078429)* rw-  0x000283fb 0x0000602e 0x00000000 DxeCore.dll.__DWARF.__debug_pubnames
  0x00000003 dwarf-pubtypes   [0x0000000000078429-0x000000000008b915)* rw-  0x0002e429 0x000134ec 0x00000000 DxeCore.dll.__DWARF.__debug_pubtypes
  0x00000004 dwarf-aranges    [0x000000000008b915-0x000000000008ea65)* rw-  0x00041915 0x00003150 0x00000000 DxeCore.dll.__DWARF.__debug_aranges
  0x00000005 dwarf-info       [0x000000000008ea65-0x00000000000c78bf)* rw-  0x00044a65 0x00038e5a 0x00000000 DxeCore.dll.__DWARF.__debug_info
  0x00000006 dwarf-frame      [0x00000000000c78bf-0x00000000000cc697)* rw-  0x0007d8bf 0x00004dd8 0x00000000 DxeCore.dll.__DWARF.__debug_frame
  0x00000007 dwarf-ranges     [0x00000000000cc697-0x00000000000cc757)* rw-  0x00082697 0x000000c0 0x00000000 DxeCore.dll.__DWARF.__debug_ranges
  0x00000008 dwarf-abbrev     [0x00000000000cc757-0x00000000000cca9b)* rw-  0x00082757 0x00000344 0x00000000 DxeCore.dll.__DWARF.__debug_abbrev
  0x00000009 dwarf-str        [0x00000000000cca9b-0x00000000000dcc05)* rw-  0x00082a9b 0x0001016a 0x00000000 DxeCore.dll.__DWARF.__debug_str
  0x0000000a apple-names      [0x00000000000dcc05-0x00000000000e2439)* rw-  0x00092c05 0x00005834 0x00000000 DxeCore.dll.__DWARF.__apple_names
  0x0000000b apple-namespaces [0x00000000000e2439-0x00000000000e245d)* rw-  0x00098439 0x00000024 0x00000000 DxeCore.dll.__DWARF.__apple_namespac
  0x0000000c apple-types      [0x00000000000e245d-0x00000000000eea50)* rw-  0x0009845d 0x0000c5f3 0x00000000 DxeCore.dll.__DWARF.__apple_types
  0x0000000d apple-objc       [0x00000000000eea50-0x00000000000eea74)* rw-  0x000a4a50 0x00000024 0x00000000 DxeCore.dll.__DWARF.__apple_objc
Sections for '/Users/user/Documents/UefiWorkspace/Build/OvmfX64/NOOPT_XCODE5/X64/MdeModulePkg/Universal/DevicePathDxe/DevicePathDxe/DEBUG/DevicePathDxe.dll' (x86_64):
…
(lldb) image dump symtab
Dumping symbol table for 88 modules.
Symtab, file = /Users/user/Documents/UefiWorkspace/Build/OvmfX64/NOOPT_XCODE5/X64/MdeModulePkg/Core/Dxe/DxeMain/DEBUG/DxeCore.dll, num_symbols = 1203:
               Debug symbol
               |Synthetic symbol
               ||Externally Visible
               |||
Index   UserID DSX Type            File Address/Value Load Address       Size               Flags      Name
------- ------ --- --------------- ------------------ ------------------ ------------------ ---------- ----------------------------------
[    0]    118 D   SourceFile      0x0000000000000000                    Sibling -> [   20] 0x00640000 /Users/user/Documents/UefiPackages/MdeModulePkg/Core/Dxe/SectionExtraction/CoreSectionExtraction.c
[    1]    120 D   ObjectFile      0x000000005e78588d                    0x0000000000000000 0x00660001 /Users/user/Documents/UefiWorkspace/Build/OvmfX64/NOOPT_XCODE5/X64/MdeModulePkg/Core/Dxe/DxeMain/OUTPUT/DxeCore.lib(CoreSectionExtraction.obj)
[    2]    122 D X Code            0x0000000000000240 0x000000007fe26240 0x0000000000000260 0x000f0000 CustomGuidedSectionExtract
[    3]    126 D X Code            0x00000000000004a0 0x000000007fe264a0 0x0000000000000110 0x000f0000 InitializeSectionExtraction
[    4]    130 D X Code            0x00000000000005b0 0x000000007fe265b0 0x0000000000000140 0x000f0000 IsValidSectionStream
[    5]    134 D X Code            0x00000000000006f0 0x000000007fe266f0 0x0000000000000180 0x000f0000 OpenSectionStreamEx
[    6]    138 D X Code            0x0000000000000870 0x000000007fe26870 0x0000000000000070 0x000f0000 OpenSectionStream
[    7]    142 D X Code            0x00000000000008e0 0x000000007fe268e0 0x0000000000000100 0x000f0000 ChildIsType
…
[  831]   2470 D X Code            0x00000000000306b0 0x000000007fe566b0 0x00000000000000f4 0x000f0000 UefiBootServicesTableLibConstructor
[  832]   2473 D X Data            0x0000000000047780 0x000000007fe6d780 0x0000000000000008 0x000f0000 gImageHandle
[  833]   2474 D X Data            0x0000000000047788 0x000000007fe6d788 0x0000000000000008 0x000f0000 gST
[  834]   2475 D X Data            0x0000000000047790 0x000000007fe6d790 0x0000000000000008 0x000f0000 gBS
…

> This is easy enough to check as %rpb is the frame pointer so it will get saved/restored on function entry/exit.

Actually it is fine. I happened to break in the interrupt routine, which was not mapped for a particular image. After I stepped a little, it soon reached the debug symbols:

(lldb) bt all
* thread #1, stop reason = instruction step over
  * frame #0: 0x000000007fe49f3c DxeCore.dll`CoreRestoreTpl(NewTpl=0x0000000000000008) at Tpl.c:102:12
    frame #1: 0x000000007e518c95
  thread #2
    frame #0: 0x000000007fb3ff88 CpuDxe.dll`CpuSleep + 1
    frame #1: 0x000000007fb4d719 CpuDxe.dll`ApWakeupFunction(ExchangeInfo=0x0000000000087198, ApIndex=0x0000000000000001) at MpLib.c:768:9
    frame #2: 0x000000007fb2d151
  thread #3
    frame #0: 0x000000007fb3ff88 CpuDxe.dll`CpuSleep + 1
    frame #1: 0x000000007fb4d719 CpuDxe.dll`ApWakeupFunction(ExchangeInfo=0x0000000000087198, ApIndex=0x0000000000000002) at MpLib.c:768:9
    frame #2: 0x000000007fb2d151
  thread #4
    frame #0: 0x000000007fb3ff88 CpuDxe.dll`CpuSleep + 1
    frame #1: 0x000000007fb4d719 CpuDxe.dll`ApWakeupFunction(ExchangeInfo=0x0000000000087198, ApIndex=0x0000000000000003) at MpLib.c:768:9
    frame #2: 0x000000007fb2d151

Best wishes,
Vitaly

[1] https://github.com/acidanthera/OpenCorePkg/blob/7684a9f/Debug/ClangDwarf.patch
[2] https://bugs.llvm.org/show_bug.cgi?id=45273
[3] https://bugs.llvm.org/show_bug.cgi?id=45277
[4] https://github.com/acidanthera/OpenCorePkg/commit/3e5f3f3f9a8d3f197834bbfcf5029f771df89754

> 22 марта 2020 г., в 00:06, Andrew Fish <afish@apple.com> написал(а):
> 
> 
> 
>> On Mar 21, 2020, at 11:36 AM, Vitaly Cheptsov <cheptsov@ispras.ru <mailto:cheptsov@ispras.ru>> wrote:
>> 
>> Andrew,
>> 
>> Thanks once again, but unfortunately it is not that simple. Below I answered inline explaining the particular issues, which mostly seem to be specific to CLANGPDB. LLVM stack emits PDB debug files, and even though LLDB does support them to some level, it is unlikely that this will be working well enough soon. We should really stick to more or less native debug formats, ideally those that have proper open specifications, on all platforms, and for Unix that’s DWARF.
>> 
> 
> Vitaly,
> 
> I understand and I use the Xcode clang and not the CLANGPDB, but I use lldb a lot I was just trying to point out what works with Xcode.
> 
> 
>> I am pretty sure LLVM can be taught to emit DWARF debug information even for PE files. Perhaps we can either make some option or provide a separate toolchain for this? Another way would be recovering CLANGELF as originally suggested.
>> 
> 
> There was a bug recently in the  x86_64-pc-win32-macho triple and we had to add -gdwarf to force it emit dwarf. Not sure what that compiler flag would do to CLANGPDB but it is worth a try? Last flag wins for the compiler.
> 
> 
>>> You can teach lldb about types. There is some example code here: https://github.com/tianocore/edk2/blob/master/EmulatorPkg/Unix/lldbefi.py <https://github.com/tianocore/edk2/blob/master/EmulatorPkg/Unix/lldbefi.py>
>> This code works just fine with LLDB and DWARF (e.g. XCODE5), though I have not yet completed these changes for my scripts for LLDB, only for GDB. However, with CLANGPDB generated files it is not functional. The reason for this is because LLDB is unaware of the underlying type, i.e. it does not know what is EFI_STATUS or UINT32. I can implement pretty-printing when LLDB knows about a typedef, but it is not possible to do this when the debug information is already gone or not parsed:
>> 
>> (lldb) p Status
>> (unsigned long long) $1 = 0
>> (lldb) p &Status
>> (unsigned long long *) $2 = 0x000000007fe19ad8
>> (lldb) p (EFI_STATUS)Status
>> error: use of undeclared identifier 'EFI_STATUS'
>> 
>> Just in case I tried using exactly your code, and other stuff like source level debugging works just fine and symbolication works fine, so it should be some bug with PDB in particular.
>> 
>>> That is strange as globals usually work best? The common issue I've seen is getting the slide wrong. The EFI modules are linked at a value near zero and relocated into memory, so the slide represents that adjustment.
>>> 
>>> You can use `image dump sections` and ` image dump symtab` to see lldb's view of symbols. More info here [1].
>> 
>> Yes, this one is a bit complicated, once again due to PDB most likely. It knows about global symbols, but does not list them in symtab:
>> 
>> (lldb) image dump symtab
>> Dumping symbol table for 91 modules.
>> Symtab, file = GdbSyms/Bin/X64_CLANGPDB/GdbSyms.dll, num_symbols = 0
>> Symtab, file = /Users/user/Documents/UefiWorkspace/Build/OvmfX64/NOOPT_CLANGPDB/X64/MdeModulePkg/Core/Dxe/DxeMain/DEBUG/DxeCore.dll, num_symbols = 0
>> Symtab, file = /Users/user/Documents/UefiWorkspace/Build/OvmfX64/NOOPT_CLANGPDB/X64/MdeModulePkg/Universal/DevicePathDxe/DevicePathDxe/DEBUG/DevicePathDxe.dll, num_symbols = 0
>> …
>> 
>> The slides are correct, but there are two nuances that collide with it.
>> 
>> 1. There are multiple instances of the globals with the same name (e.g. gBS), but for some reason LLDB always tries to print the globals from the first module. This happens even when I am source-level debugging, and I see a gBS symbol from another module (e.g. DxeCore) used right at the same line. With GDB the closest symbol is used, but with LLDB it is always coming from the first module. I tried checking expr help to find whether I can pass it a module explicitly, but also failed.
>> 
> 
> Usually what happens with lldb is you get the global that is in scope for the current frame.
> 
>> 2.  To be able to get EFI types to locate the EFI_SYSTEM_TABLE_POINTER I add a dummy GdbSyms image, which is not loaded to the firmware. So basically I cannot slide what is not in the memory, and this is also my first image. I tried deleting it anyhow, but it failed for me.
>> 
> 
> I've not used the fake image to get things done so I can't speak to that. I have used a fake target so I could have XIP PEIM and shadowed PEIM address available at the same time. You can't have a module loaded at 2 addresses at the same time in llldb. But you might be able to use a fake target for your fake stuff?
> 
> Just in case:
> 
>           # create a faka target to store info about symbols
>           PeiXipTarget = target.debugger.CreateTarget (None, "i386-apple-macosx", "remote-macosx", True, error)
> 
>           # make sure the gdb-remote  connection target is the active target
>           target.debugger.SetSelectedTarget (target)
> 
> 
>> (lldb) image dump sections
>> Dumping sections for 91 modules.
>> Sections for 'GdbSyms/Bin/X64_CLANGPDB/GdbSyms.dll' (x86_64):
>>   SectID     Type             Load Address                             Perm File Off.  File Size  Flags      Section Name
>>   ---------- ---------------- ---------------------------------------  ---- ---------- ---------- ---------- ----------------------------
>>   0xffffffffffffffff container        [0x0000000000000000-0x0000000000006ec0)* ---  0x00000000 0x00000000 0x00000000 GdbSyms.dll.
>>   0x00000001 code             [0x0000000000000220-0x0000000000005bd6)* ---  0x00000220 0x000059c0 0x60000020 GdbSyms.dll...text
>>   0x00000002 data             [0x0000000000005be0-0x0000000000006d79)* ---  0x00005be0 0x000011a0 0x40000040 GdbSyms.dll...rdata
>>   0x00000003 data             [0x0000000000006d80-0x0000000000006e30)* ---  0x00006d80 0x00000060 0xc0000040 GdbSyms.dll...data
>>   0x00000004 regular          [0x0000000000006e40-0x0000000000006ea4)* ---  0x00006de0 0x00000080 0x42000040 GdbSyms.dll...reloc
>> Sections for '/Users/user/Documents/UefiWorkspace/Build/OvmfX64/NOOPT_CLANGPDB/X64/MdeModulePkg/Core/Dxe/DxeMain/DEBUG/DxeCore.dll' (x86_64):
>>   SectID     Type             Load Address                             Perm File Off.  File Size  Flags      Section Name
>>   ---------- ---------------- ---------------------------------------  ---- ---------- ---------- ---------- ----------------------------
>>   0xffffffffffffffff container        [0x0000000000000000-0x00000000000523a0)* ---  0x00000000 0x00000000 0x00000000 DxeCore.dll.
>>   0x00000001 code             [0x000000007fe1b220-0x000000007fe61e34)  ---  0x00000220 0x00046c20 0x60000020 DxeCore.dll...text
>>   0x00000002 data             [0x000000007fe61e40-0x000000007fe68065)  ---  0x00046e40 0x00006240 0x40000040 DxeCore.dll...rdata
>>   0x00000003 data             [0x000000007fe68080-0x000000007fe6d160)  ---  0x0004d080 0x000018a0 0xc0000040 DxeCore.dll...data
>>   0x00000004 regular          [0x000000007fe6d160-0x000000007fe6d398)  ---  0x0004e920 0x00000240 0x42000040 DxeCore.dll...reloc
>> Sections for '/Users/user/Documents/UefiWorkspace/Build/OvmfX64/NOOPT_CLANGPDB/X64/MdeModulePkg/Universal/DevicePathDxe/DevicePathDxe/DEBUG/DevicePathDxe.dll' (x86_64):
>>   SectID     Type             Load Address                             Perm File Off.  File Size  Flags      Section Name
>>   ---------- ---------------- ---------------------------------------  ---- ---------- ---------- ---------- ----------------------------
>>   0xffffffffffffffff container        [0x0000000000000000-0x0000000000014420)* ---  0x00000000 0x00000000 0x00000000 DevicePathDxe.dll.
>>   0x00000001 code             [0x000000007f986220-0x000000007f996cc6)  ---  0x00000220 0x00010ac0 0x60000020 DevicePathDxe.dll...text
>>   0x00000002 data             [0x000000007f996ce0-0x000000007f999b04)  ---  0x00010ce0 0x00002e40 0x40000040 DevicePathDxe.dll...rdata
>>   0x00000003 data             [0x000000007f999b20-0x000000007f99a1a2)  ---  0x00013b20 0x00000660 0xc0000040 DevicePathDxe.dll...data
>>   0x00000004 regular          [0x000000007f99a1c0-0x000000007f99a404)  ---  0x00014180 0x00000260 0x42000040 DevicePathDxe.dll…reloc
>> …
>> 
>> So, all in all, unique global variables work, but there is no way to access duplicating variables. They either resolve to GdbSyms or just cause a crash:
>> 
>> (lldb) p mDebugInfoTableHeader
>> (EFI_DEBUG_IMAGE_INFO_TABLE_HEADER) $0 = {
>>   UpdateStatus = 2
>>   TableSize = 92
>>   EfiDebugImageInfoTable = 0x000000007f814018
>> }
>> (lldb) p gBS
>> error: Couldn't materialize: couldn't get the value of variable ::gBS: read memory from 0x6df8 failed
>> error: errored out in DoExecute, couldn't PrepareToExecuteJITExpression
>> (lldb) p gEfiGlobalVariableGuid
>> 0  libLLVM.dylib            0x000000010e52ee68 llvm::sys::PrintStackTrace(llvm::raw_ostream&) + 40
>> 1  libLLVM.dylib            0x000000010e52f262 SignalHandler(int) + 188
>> 2  libsystem_platform.dylib 0x00007fff6ca5642d _sigtramp + 29
>> ...
>> 
> 
> If you want to inspect globals I think this logic works to get you data, you would need to print it out etc.
> 
> SBValueList = lldb.target.FindGlobalVariables ("gST", 1024)
> for SBValue in SBValueList:
> 	Module = SBValue.GetAddress().GetModule()
>         ModuleStr = SBValue.GetAddress().GetModule().GetFileSpec().GetFilename()
>         Start = int (SBValue.GetLocation(), 0)
>         End = Start + SBValue.GetByteSize() - 1
>         SBDeclaration = SBValue.GetDeclaration()
>         Column = SBDeclaration.GetColumn()
> 
> I wrote a command in the early days to dump out all the instances of a global.
> 
> You can also try (lldb) image lookup -Av --name gST
> 
>>> You can tell lldb to use the older Python like this (from the Terminal.app):
>>> $ defaults write com.apple.dt.lldb DefaultPythonVersion 2
>> 
>> Thanks, that helped quite a bit, but for some reason Xcode version still crashes more for me. I attached a couple of stack traces if you feel like having a look, but once again it seems that it is all about the PDB plugin.
>> 
>>> For the macOS API clang emits frame pointers, so you can walk the stack without symbols. You could try adding the compiler flag to emit the frame pointers.
>> 
> 
> This is easy enough to check as %rpb is the frame pointer so it will get saved/restored on function entry/exit.
> 
>> I am pretty sure stack frames are not disabled with UEFI, as sometimes backtracing works just fine. To me it looks like debug information parsing randomly breaks in LLDB, and once it happens it forgets about other images:
>> 
>> (lldb) b CoreLocateHandleBuffer
>> Breakpoint 2: where = DxeCore.dll`CoreLocateHandleBuffer + 31 at Locate.c:649, address = 0x000000007fe36e4f
>> (lldb) c
>> Process 1 resuming
>> Process 1 stopped
>> * thread #1, stop reason = breakpoint 2.1
>>     frame #0: 0x000000007fe36e4f DxeCore.dll`CoreLocateHandleBuffer(SearchType=ByProtocol, Protocol=0x000000007f978160, SearchKey=0x0000000000000000, NumberHandles=0x000000007fe19fd8, Buffer=0x000000007fe19fc0) at Locate.c:649
>>    646 	  EFI_STATUS          Status;
>>    647 	  UINTN               BufferSize;
>>    648
>> -> 649 	  if (NumberHandles == NULL) {
>>    650 	    return EFI_INVALID_PARAMETER;
>>    651 	  }
>>    652
>> (lldb) bt
>> * thread #1, stop reason = breakpoint 2.1
>>   * frame #0: 0x000000007fe36e4f DxeCore.dll`CoreLocateHandleBuffer(SearchType=ByProtocol, Protocol=0x000000007f978160, SearchKey=0x0000000000000000, NumberHandles=0x000000007fe19fd8, Buffer=0x000000007fe19fc0) at Locate.c:649
>>     frame #1: 0x000000007fe36816 DxeCore.dll`CoreLocateDevicePath(Protocol=0x000000007f978160, DevicePath=0x000000007fe1a060, Device=0x000000007fe1a068) at Locate.c:466
>>     frame #2: 0x000000007f97479a SecurityStubDxe.dll
>> 
>> ———
>> 
>> (lldb) b CopyMem
>> Breakpoint 3: 70 locations.
>> (lldb) c
>> Process 1 resuming
>> Process 1 stopped
>> * thread #1, stop reason = breakpoint 2.53 3.53
>>     frame #0: 0x000000007e5c13b3 MnpDxe.dll`CopyMem(DestinationBuffer=0x000000007fe19b50, SourceBuffer=0x000000007e2aa470, Length=656) at CopyMemWrapper.c:47
>>    44  	  IN UINTN       Length
>>    45  	  )
>>    46  	{
>> -> 47  	  if (Length == 0) {
>>    48  	    return DestinationBuffer;
>>    49  	  }
>>    50  	  ASSERT ((Length - 1) <= (MAX_ADDRESS - (UINTN)DestinationBuffer));
>> (lldb) bt
>> * thread #1, stop reason = breakpoint 2.53 3.53
>>   * frame #0: 0x000000007e5c13b3 MnpDxe.dll`CopyMem(DestinationBuffer=0x000000007fe19b50, SourceBuffer=0x000000007e2aa470, Length=656) at CopyMemWrapper.c:47
>> (lldb) finish
>> error: Could not create return address breakpoint.
>> (lldb) n
>> Process 1 stopped
>> * thread #1, stop reason = step over
>>     frame #0: 0x000000007e5c13ce MnpDxe.dll`CopyMem(DestinationBuffer=0x000000007fe19b50, SourceBuffer=0x000000007e2aa470, Length=656) at CopyMemWrapper.c:50
>>    47  	  if (Length == 0) {
>>    48  	    return DestinationBuffer;
>>    49  	  }
>> -> 50  	  ASSERT ((Length - 1) <= (MAX_ADDRESS - (UINTN)DestinationBuffer));
>>    51  	  ASSERT ((Length - 1) <= (MAX_ADDRESS - (UINTN)SourceBuffer));
>>    52
>>    53  	  if (DestinationBuffer == SourceBuffer) {
>> (lldb)
>> ...
>> Process 1 stopped
>> * thread #1, stop reason = step over
>>     frame #0: 0x000000007e5c14b4 MnpDxe.dll`CopyMem(DestinationBuffer=0x000000007fe19b50, SourceBuffer=0x000000007e2aa470, Length=656) at CopyMemWrapper.c:57
>>    54  	    return DestinationBuffer;
>>    55  	  }
>>    56  	  return InternalMemCopyMem (DestinationBuffer, SourceBuffer, Length);
>> -> 57  	}
>> (lldb)
>> Process 1 stopped
>> * thread #1, stop reason = step over
>>     frame #0: 0x000000007e5c726e MnpDxe.dll
>> ->  0x7e5c726e: mov    rax, qword ptr [rsp + 0x60]
>>     0x7e5c7273: cmp    byte ptr [rax + 0x68], 0x0
>>     0x7e5c7277: jne    0x7e5c7291
>>     0x7e5c727d: movabs rax, -0x7fffffffffffffed
>> (lldb) bt
>> * thread #1, stop reason = step over
>>   * frame #0: 0x000000007e5c726e MnpDxe.dll
>> 
>> ———
>> 
>> (lldb) c
>> Process 1 resuming
>> Process 1 stopped
>> * thread #1, stop reason = signal SIGINT
>>     frame #0: 0x000000007fe4d72e DxeCore.dll
>> ->  0x7fe4d72e: cmp    al, 0x0
>>     0x7fe4d730: je     0x7fe4d765
>>     0x7fe4d736: mov    rcx, qword ptr [rsp + 0x20]
>>     0x7fe4d73b: call   0x7fe4c4b0
>> (lldb) bt
>> * thread #1, stop reason = signal SIGINT
>>   * frame #0: 0x000000007fe4d72e DxeCore.dll
>> 
>>> On macOS the Mach-O and dSYM have a UUID (dwarfdump -u) that is indexed by Spotlight (mdfind "com_apple_xcode_dsym_uuids == *") [2]
>>> This should be the UUID in the debug directory entry and you can use that to lookup the symbols like this:
>>> 
>>> module = target.AddModule (None, None, uuid)
>>> SBError = target.SetModuleLoadAddress (module, LoadAddress + TeAdjust)
>>> 
>>> Also lldb has built in help for commands, but it is kind of terse since it is autogenerated from the C++ swig.
>>> (lldb) script help (lldb.target.AddModule)
>>> Help on method AddModule in module lldb:
>>> 
>>> AddModule(self, *args) method of lldb.SBTarget instance
>>>     AddModule(SBTarget self, SBModule module) -> bool
>>>     AddModule(SBTarget self, char const * path, char const * triple, char const * uuid) -> SBModule
>>>     AddModule(SBTarget self, char const * path, char const * triple, char const * uuid_cstr, char const * symfile) -> SBModule
>>>     AddModule(SBTarget self, SBModuleSpec module_spec) -> SBModule
>>> 
>>> The minimum  you need to symbolicate a frame is uuid, LoadAddress, and PC.
>>> 
>>> [1] http://lldb.llvm.org/use/map.html <http://lldb.llvm.org/use/map.html>
>>> [2] http://lldb.llvm.org/use/symbols.html <http://lldb.llvm.org/use/symbols.html>
>> Thanks for the links again. Yes, I am using some of these, and in fact for GDB that’s pretty much what I did when I worked with XCODE5. It is very likely that when I get to complete LLDB support for XCODE5 it will work quite fine too. But I am already happy with XCODE5 here, and making it even better will only help myself, but not other people with e.g. Linux or people that want me to use the same compiler with them.
>> 
> 
> Thanks for looking out for others.
> 
> Thanks,
> 
> Andrew Fish
> 
>> Best regards,
>> Vitaly
>> 
>> 
>>> 21 марта 2020 г., в 20:13, Andrew Fish <afish@apple.com <mailto:afish@apple.com>> написал(а):
>>> 
>>> 
>>> 
>>>> On Mar 21, 2020, at 3:28 AM, Vitaly Cheptsov <cheptsov@ispras.ru <mailto:cheptsov@ispras.ru>> wrote:
>>>> 
>>>> Hello,
>>>> 
>>>> Andrey, thanks for the hint, it was very helpful. I rewrote the GDB scripts to work with LLDB[1] and was able to debug OVMF built with CLANGPDB. While it is still quite dirty, at the very least it works.
>>>> 
>>>> Unfortunately the experience was close to terrible. I may certainly do something wrong, but it is clear that PDB and LLDB do not support each other well enough. After spending several hours on playing with the tools my conclusion is that LLDB is simply not suited for UEFI PDB debugging, and we really want DWARF  as there is no other opensource debugger that supports PDB on macOS and Linux
>>>> 
>>>> In case somebody knows workarounds here are the issues I faced:
>>>> 
>>>> 1. All integer alias typedefs are discarded in favour of underlying types. This way EFI_STATUS and EFI_TPL become unsigned long long, CHAR8 becomes char, and CHAR16 becomes unsigned short. It does not look like LLDB has the original types anywhere at all, and it also does not have them registered.
>>>> 
>>>>     frame #0: 0x000000007fe242aa DxeCore.dll`CoreAllocatePoolPagesI(PoolType=EfiBootServicesData, NoPages=1, Granularity=4096, NeedGuard='\0') at Pool.c:322
>>>>    319 	    return NULL;
>>>>    320 	  }
>>>>    321
>>>> -> 322 	  Buffer = CoreAllocatePoolPages (PoolType, NoPages, Granularity, NeedGuard);
>>>>    323 	  CoreReleaseMemoryLock ();
>>>>    324
>>>>    325 	  if (Buffer != NULL) {
>>>> (lldb) p Status
>>>> (unsigned long long) $3 = 0
>>>> 
>>>> Structures work more or less fine, but for simpler types like strings we are out of even potential pretty-printing.
>>>> 
>>> 
>>> Vitaly,
>>> 
>>> You can teach lldb about types. There is some example code here: https://github.com/tianocore/edk2/blob/master/EmulatorPkg/Unix/lldbefi.py <https://github.com/tianocore/edk2/blob/master/EmulatorPkg/Unix/lldbefi.py>
>>>> 2. Global variables are not accessible. I am not sure what happens, but they either seem to not relocate or conflict with the other names:
>>>> 
>>>> (lldb) p gST
>>>> error: Couldn't materialize: couldn't get the value of variable ::gST: read memory from 0x6e18 failed
>>>> error: errored out in DoExecute, couldn't PrepareToExecuteJITExpression
>>>> (lldb) p &gST
>>>> error: Couldn't materialize: couldn't get the value of variable ::gST: read memory from 0x6e18 failed
>>>> error: errored out in DoExecute, couldn't PrepareToExecuteJITExpression
>>>> 
>>> 
>>> That is strange as globals usually work best? The common issue I've seen is getting the slide wrong. The EFI modules are linked at a value near zero and relocated into memory, so the slide represents that adjustment.
>>> 
>>> You can use `image dump sections` and ` image dump symtab` to see lldb's view of symbols. More info here [1].
>>> 
>>>> 3. Quite a number of crashes.
>>>> 
>>>> In most cases autocompletion by tab press causes a crash. E.g.
>>>> 
>>>> b I<TAB>
>>>> 
>>>> So will do printing of a GUID, e.g. p gEfiGlobalVariableGuid.
>>>> 
>>>> This may have to do with Python compatibility as Xcode 11 LLDB that uses Python 3 generally crashes more often than MacPorts LLDB 9.0. Surprisingly structures work more or less fine.
>>>> 
>>> 
>>> You can tell lldb to use the older Python like this (from the Terminal.app):
>>> $ defaults write com.apple.dt.lldb DefaultPythonVersion 2
>>> 
>>>> 4. Ctrl+C does not produce a valid backtrace. When I break with a breakpoint, I see a proper stacktrace with more than one entry, with function prototypes and values. When I break with Ctrl+C I only see some weird backtrace with most of the entries missing regardless of frame position:
>>>> 
>>>> (lldb) bt
>>>> * thread #1, stop reason = signal SIGTRAP
>>>>   * frame #0: 0x000000007fe4c5f3 DxeCore.dll
>>>> 
>>>> Probably more and all the unintuitive stuff like the lack of more functional TUI, but it is hard to remember all the trials.
>>>> 
>>> 
>>> For the macOS API clang emits frame pointers, so you can walk the stack without symbols. You could try adding the compiler flag to emit the frame pointers.
>>> 
>>> 
>>>> [1] https://github.com/acidanthera/OpenCorePkg/blob/master/Debug/Scripts/lldb_uefi.py <https://github.com/acidanthera/OpenCorePkg/blob/master/Debug/Scripts/lldb_uefi.py>
>>>> 
>>> 
>>> On macOS the Mach-O and dSYM have a UUID (dwarfdump -u) that is indexed by Spotlight (mdfind "com_apple_xcode_dsym_uuids == *") [2]
>>> This should be the UUID in the debug directory entry and you can use that to lookup the symbols like this:
>>> 
>>> module = target.AddModule (None, None, uuid)
>>> SBError = target.SetModuleLoadAddress (module, LoadAddress + TeAdjust)
>>> 
>>> Also lldb has built in help for commands, but it is kind of terse since it is autogenerated from the C++ swig.
>>> (lldb) script help (lldb.target.AddModule)
>>> Help on method AddModule in module lldb:
>>> 
>>> AddModule(self, *args) method of lldb.SBTarget instance
>>>     AddModule(SBTarget self, SBModule module) -> bool
>>>     AddModule(SBTarget self, char const * path, char const * triple, char const * uuid) -> SBModule
>>>     AddModule(SBTarget self, char const * path, char const * triple, char const * uuid_cstr, char const * symfile) -> SBModule
>>>     AddModule(SBTarget self, SBModuleSpec module_spec) -> SBModule
>>> 
>>> The minimum  you need to symbolicate a frame is uuid, LoadAddress, and PC.
>>> 
>>> [1] http://lldb.llvm.org/use/map.html <http://lldb.llvm.org/use/map.html>
>>> [2] http://lldb.llvm.org/use/symbols.html <http://lldb.llvm.org/use/symbols.html>
>>> 
>>> Thanks,
>>> 
>>> Andrew Fish
>>> 
>>> 
>>>> Best wishes,
>>>> Vitaly
>>>> 
>>>>> 20 марта 2020 г., в 22:14, Andrew Fish <afish@apple.com <mailto:afish@apple.com>> написал(а):
>>>>> 
>>>>> 
>>>>> 
>>>>>> On Mar 20, 2020, at 8:13 AM, Vitaly Cheptsov <cheptsov@ispras.ru <mailto:cheptsov@ispras.ru>> wrote:
>>>>>> 
>>>>>> Hello,
>>>>>> 
>>>>>> We noticed that the original bugzilla, which intended to add new LLVM toolchain support[1], also wanted to bring ELF format support with DWARF debugging information. For some reason this did not make its way into EDK II, and we are currently wondering, how can one debug binaries built with LLVM 9.0.
>>>>>> 
>>>>>> For macOS and XCODE5 toolchain we use GDB scripts based on Andrei Warkentin’s work, which allow us to integrate with QEMU and VMware[2]. It is likely that they should work with little to no work on Linux with CLANG38/GCC5 with GDB once again. However, CLANGPDB apparently is using PDB debugging information, which I believe is not handled with GDB.
>>>>>> 
>>>>>> Could you please provide the details on the matter and let us know about the recommended route?
>>>>>> — Is dropping CLANGELF just a temporary measure and it should be resubmitted again?
>>>>>> — Should LLDB, which seems to be aware of PDB, be used instead of GDB, when building with CLANGPDB? If so, did anybody try that?
>>>>>> 
>>>>> 
>>>>> Vitaly,
>>>>> 
>>>>> I've not tried the CLANGPDB path, but if you want to connect lldb to QEMU you need to set  plugin.process.gdb-remote.target-definition-file [1] to [2].
>>>>> 
>>>>> [1]  lldb -o "settings set plugin.process.gdb-remote.target-definition-file x86_64_target_definition.py" -o "gdb-remote 9000"
>>>>> [2] https://github.com/llvm-mirror/lldb/blob/master/examples/python/x86_64_target_definition.py <https://github.com/llvm-mirror/lldb/blob/master/examples/python/x86_64_target_definition.py>
>>>>> 
>>>>> Thanks,
>>>>> 
>>>>> Andrew Fish
>>>>> 
>>>>>> Thanks!
>>>>>> 
>>>>>> Best regards,
>>>>>> Vitaly
>>>>>> 
>>>>>> [1] https://bugzilla.tianocore.org/show_bug.cgi?id=1603 <https://bugzilla.tianocore.org/show_bug.cgi?id=1603>
>>>>>> [2] https://github.com/acidanthera/OpenCorePkg/blob/master/Debug/Scripts/gdb_uefi.py <https://github.com/acidanthera/OpenCorePkg/blob/master/Debug/Scripts/gdb_uefi.py>
>>>>>> 
>>>> 
>> 
>> 
>> <crashes.txt>


[-- Attachment #1.2: Type: text/html, Size: 134156 bytes --]

[-- Attachment #2: Message signed with OpenPGP --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [edk2-devel] CLANGPDB binary debugging

[Liming Gao]

[-- Attachment #1: Type: text/plain, Size: 45730 bytes --]

Vitaly:
 Thanks for your further investigation with LLDB. Now, we have two CLANG tool chain. CLANGPDB and CLANG38. CLANGPDB works with LLVM9.0 or above to generate PE image with PDB debug symbol. CLANG38 can also be used to generate ELF image and convert to PE image with DWARF debug symbol on Linux host. So, these two chains cover both usage models. We don’t need to introduce new CLANGDWARF tool chain. For LLDB issue on PDB debug symbol, I suggest to report them in LLVM.

 Besides, LLVM10.0 is just released.

Thanks
Liming
From: devel@edk2.groups.io <devel@edk2.groups.io> On Behalf Of Vitaly Cheptsov
Sent: 2020年3月23日 17:10
To: Andrew Fish <afish@apple.com>; Gao, Liming <liming.gao@intel.com>
Cc: devel@edk2.groups.io; Marvin Häuser <mhaeuser@outlook.de>
Subject: Re: [edk2-devel] CLANGPDB binary debugging

Andrew,

I investigated further and was able to make some progress. I got rid of PDB, and changed debug information generation to DWARF[1]. This resolved most of LLDB crashes and made pretty-printing work just fine as type aliases like EFI_STATUS now work correctly (I am still to commit this). Unfortunately there are several downsides. Dead code stripping in LLD linker is broken when generating PE with DWARF, so I had to patch it[2]. Debug link implementation in llvm-objcopy is also quite ugly, we do not get MTOC or anything alike[3], just some section with a random name containing base filename.

I can submit a more proper patch to EDK II properly implementing CLANGDWARF toolchain instead of patching CLANGPDB like in [1], but it is questionable whether it is worth it. LLVM 10 will land very soon, and the necessary LLD patches[2] will definitely not make their way in this release. We will have to wait for LLVM 11 at the very least, and I am not sure I am ready to drive llvm-objcopy changes either. Liming, what do you think?

I've not used the fake image to get things done so I can't speak to that. I have used a fake target so I could have XIP PEIM and shadowed PEIM address available at the same time. You can't have a module loaded at 2 addresses at the same time in llldb. But you might be able to use a fake target for your fake stuff?

Just in case:

          # create a faka target to store info about symbols
          PeiXipTarget = target.debugger.CreateTarget (None, "i386-apple-macosx", "remote-macosx", True, error)

          # make sure the gdb-remote  connection target is the active target
          target.debugger.SetSelectedTarget (target)

Yes, that was actually a smart idea. I initially did it that way, as I had mixing Mach-O and PE triples, but later removed, which was actually a mistake. Now I have a fake target for GdbSyms and a proper target for other images[4]. This somewhat solves the issue with invalid addresses for gST, as gST from DxeCore is now picked. However, I cannot access global variables from other modules for some reason still.

If you want to inspect globals I think this logic works to get you data, you would need to print it out etc.

SBValueList = lldb.target.FindGlobalVariables ("gST", 1024)
for SBValue in SBValueList:
Module = SBValue.GetAddress().GetModule()
        ModuleStr = SBValue.GetAddress().GetModule().GetFileSpec().GetFilename()
        Start = int (SBValue.GetLocation(), 0)
        End = Start + SBValue.GetByteSize() - 1
        SBDeclaration = SBValue.GetDeclaration()
        Column = SBDeclaration.GetColumn()

I wrote a command in the early days to dump out all the instances of a global.

You can also try (lldb) image lookup -Av --name gST

Something is really weird, gST from the first image shows, but lldb believes that it does not exist. lldb.target.FindGlobalVariables ("gST", 1024) also returns an empty list.

I noticed that LLDB shows invalid type for gST in the symtab, there also is no address. I started to wonder whether this is the cause and decided to check XCODE5, as the the issue could have been with PE format. Unfortunately even after I tried XCODE5 support with LLDB, I got exactly the same problem. I am quite clueless what is going on. Below I included two transcripts: for CLANGDWARF and XCODE5.

(lldb) p gST
(EFI_SYSTEM_TABLE *) $0 = 0x000000007f9ee018
(lldb) p *gST
(EFI_SYSTEM_TABLE) $1 = {
  Hdr = (Signature = 0x5453595320494249, Revision = 0x00020046, HeaderSize = 0x00000078, CRC32 = 0x3d5e9b5d, Reserved = 0x00000000)
  FirmwareVendor = 0x000000007f9e9c98
  FirmwareRevision = 0x00010000
  ConsoleInHandle = 0x000000007ed58f18
  ConIn = 0x000000007e6edcb8
  ConsoleOutHandle = 0x000000007ed57018
  ConOut = 0x000000007e6ede40
  StandardErrorHandle = 0x000000007ed57a98
  StdErr = 0x000000007e6ee030
  RuntimeServices = 0x000000007f9eeb98
  BootServices = 0x000000007fe689e8
  NumberOfTableEntries = 0x000000000000000a
  ConfigurationTable = 0x000000007f9eec98
}
(lldb) image lookup -Av --name gST
(lldb) image dump sections
Dumping sections for 90 modules.
Sections for '/Users/user/Documents/UefiWorkspace/Build/OvmfX64/NOOPT_CLANGPDB/X64/DxeCore.debug' (x86_64):
  SectID     Type             Load Address                             Perm File Off.  File Size  Flags      Section Name
  ---------- ---------------- ---------------------------------------  ---- ---------- ---------- ---------- ----------------------------
  0xffffffffffffffff container        [0x000000007fe1b000-0x000000007ff38ac0)  ---  0x00000000 0x00000000 0x00000000 DxeCore.debug.
  0x00000001 code             [0x000000007fe1b320-0x000000007fe61f34)  ---  0x00000320 0x00046c20 0x60000020 DxeCore.debug...text
  0x00000002 data             [0x000000007fe61f40-0x000000007fe680f1)  ---  0x00046f40 0x000061c0 0x40000040 DxeCore.debug...rdata
  0x00000003 data             [0x000000007fe68100-0x000000007fe6d1e0)  ---  0x0004d100 0x000018a0 0xc0000040 DxeCore.debug...data
  0x00000004 regular          [0x000000007fe6d1e0-0x000000007fe6d424)  ---  0x0004e9a0 0x00000260 0x42000040 DxeCore.debug...reloc
  0x00000005 dwarf-abbrev     [0x000000007fe6d440-0x000000007fe74059)  ---  0x0004ec00 0x00006c20 0x42000040 DxeCore.debug...debug_abbrev
  0x00000006 dwarf-info       [0x000000007fe74060-0x000000007fec1355)  ---  0x00055820 0x0004d300 0x42000040 DxeCore.debug...debug_info
  0x00000007 dwarf-line       [0x000000007fec1360-0x000000007fef9fd4)  ---  0x000a2b20 0x00038c80 0x42000040 DxeCore.debug...debug_line
  0x00000008 dwarf-macinfo    [0x000000007fef9fe0-0x000000007fefa056)  ---  0x000db7a0 0x00000080 0x42000040 DxeCore.debug...debug_macinfo
  0x00000009 dwarf-ranges     [0x000000007fefa060-0x000000007feff610)  ---  0x000db820 0x000055c0 0x42000040 DxeCore.debug...debug_ranges
  0x0000000a dwarf-str        [0x000000007feff620-0x000000007ff38abd)  ---  0x000e0de0 0x000394a0 0x42000040 DxeCore.debug...debug_str
Sections for '/Users/user/Documents/UefiWorkspace/Build/OvmfX64/NOOPT_CLANGPDB/X64/DevicePathDxe.debug' (x86_64):
  SectID     Type             Load Address                             Perm File Off.  File Size  Flags      Section Name
  ---------- ---------------- ---------------------------------------  ---- ---------- ---------- ---------- ----------------------------
  0xffffffffffffffff container        [0x000000007f986000-0x000000007f9cb0a0)  ---  0x00000000 0x00000000 0x00000000 DevicePathDxe.debug.
  0x00000001 code             [0x000000007f986320-0x000000007f996dc6)  ---  0x00000320 0x00010ac0 0x60000020 DevicePathDxe.debug...text
  0x00000002 data             [0x000000007f996de0-0x000000007f999b75)  ---  0x00010de0 0x00002da0 0x40000040 DevicePathDxe.debug...rdata
  0x00000003 data             [0x000000007f999b80-0x000000007f99a202)  ---  0x00013b80 0x00000660 0xc0000040 DevicePathDxe.debug...data
  0x00000004 regular          [0x000000007f99a220-0x000000007f99a464)  ---  0x000141e0 0x00000260 0x42000040 DevicePathDxe.debug...reloc
  0x00000005 dwarf-abbrev     [0x000000007f99a480-0x000000007f99bd09)  ---  0x00014440 0x000018a0 0x42000040 DevicePathDxe.debug...debug_abbrev
  0x00000006 dwarf-info       [0x000000007f99bd20-0x000000007f9ad8b7)  ---  0x00015ce0 0x00011ba0 0x42000040 DevicePathDxe.debug...debug_info
  0x00000007 dwarf-line       [0x000000007f9ad8c0-0x000000007f9bc1f7)  ---  0x00027880 0x0000e940 0x42000040 DevicePathDxe.debug...debug_line
  0x00000008 dwarf-macinfo    [0x000000007f9bc200-0x000000007f9bc223)  ---  0x000361c0 0x00000040 0x42000040 DevicePathDxe.debug...debug_macinfo
  0x00000009 dwarf-ranges     [0x000000007f9bc240-0x000000007f9be020)  ---  0x00036200 0x00001de0 0x42000040 DevicePathDxe.debug...debug_ranges
  0x0000000a dwarf-str        [0x000000007f9be020-0x000000007f9cb099)  ---  0x00037fe0 0x0000d080 0x42000040 DevicePathDxe.debug...debug_str
…
(lldb) image dump symtab
Dumping symbol table for 90 modules.
Symtab, file = /Users/user/Documents/UefiWorkspace/Build/OvmfX64/NOOPT_CLANGPDB/X64/DxeCore.debug, num_symbols = 4343:
               Debug symbol
               |Synthetic symbol
               ||Externally Visible
               |||
Index   UserID DSX Type            File Address/Value Load Address       Size               Flags      Name
------- ------ --- --------------- ------------------ ------------------ ------------------ ---------- ----------------------------------
[    0] 4294967295     Invalid         0x0000000000000000 0x000000007fe1b000 0x000000000000003a 0x00000000 .text
[    1] 4294967295     Invalid         0x0000000000000000                    0x0000000000000000 0x00000000 .data
[    2] 4294967295     Invalid         0x0000000000001898                    0x0000000000000000 0x00000000 .bss
[    3] 4294967295     Invalid         0x0000000000046bc0 0x000000007fe61bc0 0x0000000000000374 0x00000000 .text$_ModuleEntryPoint
[    4] 4294967295     Code            0x0000000000046bc0 0x000000007fe61bc0 0x0000000000000374 0x00000000 _ModuleEntryPoint
[    5] 4294967295     Invalid         0x0000000000001b08                    0x0000000000000000 0x00000000 .bss$gHobList
[    6] 4294967295     Invalid         0x0000000000001b08                    0x0000000000000000 0x00000000 gHobList
[    7] 4294967295     Invalid         0x0000000000000000                    0x0000000000000000 0x00000000 .rdata
[    8] 4294967295     Invalid         0x0000000000000000                    0x0000000000000000 0x00000000 .str
[    9] 4294967295     Invalid         0x0000000000000058                    0x0000000000000000 0x00000000 .rdata
[   10] 4294967295     Invalid         0x0000000000000058                    0x0000000000000000 0x00000000 .str.1
[   11] 4294967295     Invalid         0x0000000000000000                    0x0000000000000000 0x00000000 .debug_str
[   12] 4294967295     Invalid         0x0000000000000000                    0x0000000000000000 0x00000000 .debug_abbrev
[   13] 4294967295     Invalid         0x0000000000000000                    0x0000000000000000 0x00000000 .debug_info
[   14] 4294967295     Invalid         0x0000000000000000                    0x0000000000000000 0x00000000 .debug_ranges
[   15] 4294967295     Invalid         0x0000000000000000                    0x0000000000000000 0x00000000 .debug_macinfo
[   16] 4294967295     Invalid         0x0000000000000000                    0x0000000000000000 0x00000000 .debug_line
[   17] 4294967295     Code            0x000000000003f680 0x000000007fe5a680 0x0000000000000020 0x00000000 ProcessModuleEntryPointList
[   18] 4294967295     Code            0x0000000000023130 0x000000007fe3e130 0x0000000000000020 0x00000000 DebugAssertEnabled
[   19] 4294967295     Code            0x0000000000023060 0x000000007fe3e060 0x00000000000000d0 0x00000000 DebugAssert
[   20] 4294967295     Code            0x0000000000022150 0x000000007fe3d150 0x0000000000000020 0x00000000 CpuDeadLoop
[   21] 4294967295     Invalid         0x0000000000000000 0x000000007fe1b000 0x000000000000003a 0x00000000 .text
[   22] 4294967295     Invalid         0x0000000000000000                    0x0000000000000000 0x00000000 .data
[   23] 4294967295     Invalid         0x0000000000001898                    0x0000000000000000 0x00000000 .bss
[   24] 4294967295     Invalid         0x000000000003f240 0x000000007fe5a240 0x0000000000000440 0x00000000 .text$ProcessLibraryConstructorList
[   25] 4294967295     Code            0x000000000003f240 0x000000007fe5a240 0x0000000000000440 0x00000000 ProcessLibraryConstructorList
[   26] 4294967295     Invalid         0x000000000003f680 0x000000007fe5a680 0x0000000000000020 0x00000000 .text$ProcessModuleEntryPointList
[   27] 4294967295     Invalid         0x0000000000000070                    0x0000000000000000 0x00000000 .data$gEfiCallerIdGuid
[   28] 4294967295     Invalid         0x0000000000000070                    0x0000000000000000 0x00000000 gEfiCallerIdGuid
[   29] 4294967295     Invalid         0x0000000000000148                    0x0000000000000000 0x00000000 .data$gEfiEventMemoryMapChangeGuid
[   30] 4294967295     Invalid         0x0000000000000148                    0x0000000000000000 0x00000000 gEfiEventMemoryMapChangeGuid
[   31] 4294967295     Invalid         0x0000000000000168                    0x0000000000000000 0x00000000 .data$gEfiEventVirtualAddressChangeGuid
[   32] 4294967295     Invalid         0x0000000000000168                    0x0000000000000000 0x00000000 gEfiEventVirtualAddressChangeGuid
[   33] 4294967295     Invalid         0x0000000000000138                    0x0000000000000000 0x00000000 .data$gEfiEventExitBootServicesGuid
[   34] 4294967295     Invalid         0x0000000000000138                    0x0000000000000000 0x00000000 gEfiEventExitBootServicesGuid
….
[  303] 4294967295     Invalid         0x0000000000001b60                    0x0000000000000000 0x00000000 .bss$gST
[  304] 4294967295     Invalid         0x0000000000001b60                    0x0000000000000000 0x00000000 gST
[  305] 4294967295     Invalid         0x00000000000018a0                    0x0000000000000000 0x00000000 .bss$gBS
[  306] 4294967295     Invalid         0x00000000000018a0                    0x0000000000000000 0x00000000 gBS
…

———————————————

(lldb) p gST
(EFI_SYSTEM_TABLE *) $2 = 0x000000007fbee018
(lldb) p &gST
(EFI_SYSTEM_TABLE **) $4 = 0x000000007fe6d788
(lldb) p *gST
(EFI_SYSTEM_TABLE) $3 = {
  Hdr = (Signature = 0x5453595320494249, Revision = 0x00020046, HeaderSize = 0x00000078, CRC32 = 0x5355dbe2, Reserved = 0x00000000)
  FirmwareVendor = 0x000000007fbe9c98
  FirmwareRevision = 0x00010000
  ConsoleInHandle = 0x000000007ede5a98
  ConIn = 0x000000007e95f640
  ConsoleOutHandle = 0x000000007ede4e98
  ConOut = 0x000000007e95f800
  StandardErrorHandle = 0x000000007ede4b98
  StdErr = 0x000000007e95f910
  RuntimeServices = 0x000000007fbeeb98
  BootServices = 0x000000007fe6cde8
  NumberOfTableEntries = 0x0000000000000009
  ConfigurationTable = 0x000000007fbeec98
}
(lldb) image lookup -Av --name gST
(lldb) image dump sections
Dumping sections for 88 modules.
Sections for '/Users/user/Documents/UefiWorkspace/Build/OvmfX64/NOOPT_XCODE5/X64/MdeModulePkg/Core/Dxe/DxeMain/DEBUG/DxeCore.dll' (x86_64):
  SectID     Type             Load Address                             Perm File Off.  File Size  Flags      Section Name
  ---------- ---------------- ---------------------------------------  ---- ---------- ---------- ---------- ----------------------------
  0x00000100 container        [0x000000007fe26240-0x000000007fe6bb40)  r-x  0x000004a0 0x00045900 0x00000000 DxeCore.dll.__TEXT
  0x00000001 code             [0x000000007fe26240-0x000000007fe65bd8)  r-x  0x000004a0 0x0003f998 0x80000400 DxeCore.dll.__TEXT.__text
  0x00000002 data-cstr        [0x000000007fe65bd8-0x000000007fe6ba23)  r-x  0x0003fe38 0x00005e4b 0x00000002 DxeCore.dll.__TEXT.__cstring
  0x00000003 regular          [0x000000007fe6ba24-0x000000007fe6ba6c)  r-x  0x00045c84 0x00000048 0x00000000 DxeCore.dll.__TEXT.__ustring
  0x00000004 regular          [0x000000007fe6ba70-0x000000007fe6bb40)  r-x  0x00045cd0 0x000000d0 0x00000000 DxeCore.dll.__TEXT.__const
  0x00000200 container        [0x000000007fe6bb40-0x000000007fe70e60)  rw-  0x00045da0 0x00001ae0 0x00000000 DxeCore.dll.__DATA
  0x00000005 regular          [0x000000007fe6bb40-0x000000007fe6bd58)  rw-  0x00045da0 0x00000218 0x00000000 DxeCore.dll.__DATA.__const
  0x00000006 data             [0x000000007fe6bd60-0x000000007fe6d610)  rw-  0x00045fc0 0x000018b0 0x00000000 DxeCore.dll.__DATA.__data
  0x00000007 zero-fill        [0x000000007fe6d610-0x000000007fe70e20)  rw-  0x00000000 0x00000000 0x00000001 DxeCore.dll.__DATA.__common
  0x00000008 zero-fill        [0x000000007fe70e20-0x000000007fe70e58)  rw-  0x00000000 0x00000000 0x00000001 DxeCore.dll.__DATA.__bss
  0x00000100 container        [0x000000000004b000-0x00000000000ef000)* rw-  0x00001000 0x000a3a74 0x00000000 DxeCore.dll.__DWARF
  0x00000001 dwarf-line       [0x000000000004b000-0x00000000000723fb)* rw-  0x00001000 0x000273fb 0x00000000 DxeCore.dll.__DWARF.__debug_line
  0x00000002 dwarf-pubnames   [0x00000000000723fb-0x0000000000078429)* rw-  0x000283fb 0x0000602e 0x00000000 DxeCore.dll.__DWARF.__debug_pubnames
  0x00000003 dwarf-pubtypes   [0x0000000000078429-0x000000000008b915)* rw-  0x0002e429 0x000134ec 0x00000000 DxeCore.dll.__DWARF.__debug_pubtypes
  0x00000004 dwarf-aranges    [0x000000000008b915-0x000000000008ea65)* rw-  0x00041915 0x00003150 0x00000000 DxeCore.dll.__DWARF.__debug_aranges
  0x00000005 dwarf-info       [0x000000000008ea65-0x00000000000c78bf)* rw-  0x00044a65 0x00038e5a 0x00000000 DxeCore.dll.__DWARF.__debug_info
  0x00000006 dwarf-frame      [0x00000000000c78bf-0x00000000000cc697)* rw-  0x0007d8bf 0x00004dd8 0x00000000 DxeCore.dll.__DWARF.__debug_frame
  0x00000007 dwarf-ranges     [0x00000000000cc697-0x00000000000cc757)* rw-  0x00082697 0x000000c0 0x00000000 DxeCore.dll.__DWARF.__debug_ranges
  0x00000008 dwarf-abbrev     [0x00000000000cc757-0x00000000000cca9b)* rw-  0x00082757 0x00000344 0x00000000 DxeCore.dll.__DWARF.__debug_abbrev
  0x00000009 dwarf-str        [0x00000000000cca9b-0x00000000000dcc05)* rw-  0x00082a9b 0x0001016a 0x00000000 DxeCore.dll.__DWARF.__debug_str
  0x0000000a apple-names      [0x00000000000dcc05-0x00000000000e2439)* rw-  0x00092c05 0x00005834 0x00000000 DxeCore.dll.__DWARF.__apple_names
  0x0000000b apple-namespaces [0x00000000000e2439-0x00000000000e245d)* rw-  0x00098439 0x00000024 0x00000000 DxeCore.dll.__DWARF.__apple_namespac
  0x0000000c apple-types      [0x00000000000e245d-0x00000000000eea50)* rw-  0x0009845d 0x0000c5f3 0x00000000 DxeCore.dll.__DWARF.__apple_types
  0x0000000d apple-objc       [0x00000000000eea50-0x00000000000eea74)* rw-  0x000a4a50 0x00000024 0x00000000 DxeCore.dll.__DWARF.__apple_objc
Sections for '/Users/user/Documents/UefiWorkspace/Build/OvmfX64/NOOPT_XCODE5/X64/MdeModulePkg/Universal/DevicePathDxe/DevicePathDxe/DEBUG/DevicePathDxe.dll' (x86_64):
…
(lldb) image dump symtab
Dumping symbol table for 88 modules.
Symtab, file = /Users/user/Documents/UefiWorkspace/Build/OvmfX64/NOOPT_XCODE5/X64/MdeModulePkg/Core/Dxe/DxeMain/DEBUG/DxeCore.dll, num_symbols = 1203:
               Debug symbol
               |Synthetic symbol
               ||Externally Visible
               |||
Index   UserID DSX Type            File Address/Value Load Address       Size               Flags      Name
------- ------ --- --------------- ------------------ ------------------ ------------------ ---------- ----------------------------------
[    0]    118 D   SourceFile      0x0000000000000000                    Sibling -> [   20] 0x00640000 /Users/user/Documents/UefiPackages/MdeModulePkg/Core/Dxe/SectionExtraction/CoreSectionExtraction.c
[    1]    120 D   ObjectFile      0x000000005e78588d                    0x0000000000000000 0x00660001 /Users/user/Documents/UefiWorkspace/Build/OvmfX64/NOOPT_XCODE5/X64/MdeModulePkg/Core/Dxe/DxeMain/OUTPUT/DxeCore.lib(CoreSectionExtraction.obj)
[    2]    122 D X Code            0x0000000000000240 0x000000007fe26240 0x0000000000000260 0x000f0000 CustomGuidedSectionExtract
[    3]    126 D X Code            0x00000000000004a0 0x000000007fe264a0 0x0000000000000110 0x000f0000 InitializeSectionExtraction
[    4]    130 D X Code            0x00000000000005b0 0x000000007fe265b0 0x0000000000000140 0x000f0000 IsValidSectionStream
[    5]    134 D X Code            0x00000000000006f0 0x000000007fe266f0 0x0000000000000180 0x000f0000 OpenSectionStreamEx
[    6]    138 D X Code            0x0000000000000870 0x000000007fe26870 0x0000000000000070 0x000f0000 OpenSectionStream
[    7]    142 D X Code            0x00000000000008e0 0x000000007fe268e0 0x0000000000000100 0x000f0000 ChildIsType
…
[  831]   2470 D X Code            0x00000000000306b0 0x000000007fe566b0 0x00000000000000f4 0x000f0000 UefiBootServicesTableLibConstructor
[  832]   2473 D X Data            0x0000000000047780 0x000000007fe6d780 0x0000000000000008 0x000f0000 gImageHandle
[  833]   2474 D X Data            0x0000000000047788 0x000000007fe6d788 0x0000000000000008 0x000f0000 gST
[  834]   2475 D X Data            0x0000000000047790 0x000000007fe6d790 0x0000000000000008 0x000f0000 gBS
…

This is easy enough to check as %rpb is the frame pointer so it will get saved/restored on function entry/exit.

Actually it is fine. I happened to break in the interrupt routine, which was not mapped for a particular image. After I stepped a little, it soon reached the debug symbols:

(lldb) bt all
* thread #1, stop reason = instruction step over
  * frame #0: 0x000000007fe49f3c DxeCore.dll`CoreRestoreTpl(NewTpl=0x0000000000000008) at Tpl.c:102:12
    frame #1: 0x000000007e518c95
  thread #2
    frame #0: 0x000000007fb3ff88 CpuDxe.dll`CpuSleep + 1
    frame #1: 0x000000007fb4d719 CpuDxe.dll`ApWakeupFunction(ExchangeInfo=0x0000000000087198, ApIndex=0x0000000000000001) at MpLib.c:768:9
    frame #2: 0x000000007fb2d151
  thread #3
    frame #0: 0x000000007fb3ff88 CpuDxe.dll`CpuSleep + 1
    frame #1: 0x000000007fb4d719 CpuDxe.dll`ApWakeupFunction(ExchangeInfo=0x0000000000087198, ApIndex=0x0000000000000002) at MpLib.c:768:9
    frame #2: 0x000000007fb2d151
  thread #4
    frame #0: 0x000000007fb3ff88 CpuDxe.dll`CpuSleep + 1
    frame #1: 0x000000007fb4d719 CpuDxe.dll`ApWakeupFunction(ExchangeInfo=0x0000000000087198, ApIndex=0x0000000000000003) at MpLib.c:768:9
    frame #2: 0x000000007fb2d151

Best wishes,
Vitaly

[1] https://github.com/acidanthera/OpenCorePkg/blob/7684a9f/Debug/ClangDwarf.patch
[2] https://bugs.llvm.org/show_bug.cgi?id=45273
[3] https://bugs.llvm.org/show_bug.cgi?id=45277
[4] https://github.com/acidanthera/OpenCorePkg/commit/3e5f3f3f9a8d3f197834bbfcf5029f771df89754


22 марта 2020 г., в 00:06, Andrew Fish <afish@apple.com<mailto:afish@apple.com>> написал(а):




On Mar 21, 2020, at 11:36 AM, Vitaly Cheptsov <cheptsov@ispras.ru<mailto:cheptsov@ispras.ru>> wrote:

Andrew,

Thanks once again, but unfortunately it is not that simple. Below I answered inline explaining the particular issues, which mostly seem to be specific to CLANGPDB. LLVM stack emits PDB debug files, and even though LLDB does support them to some level, it is unlikely that this will be working well enough soon. We should really stick to more or less native debug formats, ideally those that have proper open specifications, on all platforms, and for Unix that’s DWARF.


Vitaly,

I understand and I use the Xcode clang and not the CLANGPDB, but I use lldb a lot I was just trying to point out what works with Xcode.



I am pretty sure LLVM can be taught to emit DWARF debug information even for PE files. Perhaps we can either make some option or provide a separate toolchain for this? Another way would be recovering CLANGELF as originally suggested.


There was a bug recently in the  x86_64-pc-win32-macho triple and we had to add -gdwarf to force it emit dwarf. Not sure what that compiler flag would do to CLANGPDB but it is worth a try? Last flag wins for the compiler.



You can teach lldb about types. There is some example code here: https://github.com/tianocore/edk2/blob/master/EmulatorPkg/Unix/lldbefi.py

This code works just fine with LLDB and DWARF (e.g. XCODE5), though I have not yet completed these changes for my scripts for LLDB, only for GDB. However, with CLANGPDB generated files it is not functional. The reason for this is because LLDB is unaware of the underlying type, i.e. it does not know what is EFI_STATUS or UINT32. I can implement pretty-printing when LLDB knows about a typedef, but it is not possible to do this when the debug information is already gone or not parsed:

(lldb) p Status
(unsigned long long) $1 = 0
(lldb) p &Status
(unsigned long long *) $2 = 0x000000007fe19ad8
(lldb) p (EFI_STATUS)Status
error: use of undeclared identifier 'EFI_STATUS'

Just in case I tried using exactly your code, and other stuff like source level debugging works just fine and symbolication works fine, so it should be some bug with PDB in particular.


That is strange as globals usually work best? The common issue I've seen is getting the slide wrong. The EFI modules are linked at a value near zero and relocated into memory, so the slide represents that adjustment.

You can use `image dump sections` and ` image dump symtab` to see lldb's view of symbols. More info here [1].

Yes, this one is a bit complicated, once again due to PDB most likely. It knows about global symbols, but does not list them in symtab:

(lldb) image dump symtab
Dumping symbol table for 91 modules.
Symtab, file = GdbSyms/Bin/X64_CLANGPDB/GdbSyms.dll, num_symbols = 0
Symtab, file = /Users/user/Documents/UefiWorkspace/Build/OvmfX64/NOOPT_CLANGPDB/X64/MdeModulePkg/Core/Dxe/DxeMain/DEBUG/DxeCore.dll, num_symbols = 0
Symtab, file = /Users/user/Documents/UefiWorkspace/Build/OvmfX64/NOOPT_CLANGPDB/X64/MdeModulePkg/Universal/DevicePathDxe/DevicePathDxe/DEBUG/DevicePathDxe.dll, num_symbols = 0
…

The slides are correct, but there are two nuances that collide with it.

1. There are multiple instances of the globals with the same name (e.g. gBS), but for some reason LLDB always tries to print the globals from the first module. This happens even when I am source-level debugging, and I see a gBS symbol from another module (e.g. DxeCore) used right at the same line. With GDB the closest symbol is used, but with LLDB it is always coming from the first module. I tried checking expr help to find whether I can pass it a module explicitly, but also failed.


Usually what happens with lldb is you get the global that is in scope for the current frame.


2.  To be able to get EFI types to locate the EFI_SYSTEM_TABLE_POINTER I add a dummy GdbSyms image, which is not loaded to the firmware. So basically I cannot slide what is not in the memory, and this is also my first image. I tried deleting it anyhow, but it failed for me.


I've not used the fake image to get things done so I can't speak to that. I have used a fake target so I could have XIP PEIM and shadowed PEIM address available at the same time. You can't have a module loaded at 2 addresses at the same time in llldb. But you might be able to use a fake target for your fake stuff?

Just in case:

          # create a faka target to store info about symbols
          PeiXipTarget = target.debugger.CreateTarget (None, "i386-apple-macosx", "remote-macosx", True, error)

          # make sure the gdb-remote  connection target is the active target
          target.debugger.SetSelectedTarget (target)



(lldb) image dump sections
Dumping sections for 91 modules.
Sections for 'GdbSyms/Bin/X64_CLANGPDB/GdbSyms.dll' (x86_64):
  SectID     Type             Load Address                             Perm File Off.  File Size  Flags      Section Name
  ---------- ---------------- ---------------------------------------  ---- ---------- ---------- ---------- ----------------------------
  0xffffffffffffffff container        [0x0000000000000000-0x0000000000006ec0)* ---  0x00000000 0x00000000 0x00000000 GdbSyms.dll.
  0x00000001 code             [0x0000000000000220-0x0000000000005bd6)* ---  0x00000220 0x000059c0 0x60000020 GdbSyms.dll...text
  0x00000002 data             [0x0000000000005be0-0x0000000000006d79)* ---  0x00005be0 0x000011a0 0x40000040 GdbSyms.dll...rdata
  0x00000003 data             [0x0000000000006d80-0x0000000000006e30)* ---  0x00006d80 0x00000060 0xc0000040 GdbSyms.dll...data
  0x00000004 regular          [0x0000000000006e40-0x0000000000006ea4)* ---  0x00006de0 0x00000080 0x42000040 GdbSyms.dll...reloc
Sections for '/Users/user/Documents/UefiWorkspace/Build/OvmfX64/NOOPT_CLANGPDB/X64/MdeModulePkg/Core/Dxe/DxeMain/DEBUG/DxeCore.dll' (x86_64):
  SectID     Type             Load Address                             Perm File Off.  File Size  Flags      Section Name
  ---------- ---------------- ---------------------------------------  ---- ---------- ---------- ---------- ----------------------------
  0xffffffffffffffff container        [0x0000000000000000-0x00000000000523a0)* ---  0x00000000 0x00000000 0x00000000 DxeCore.dll.
  0x00000001 code             [0x000000007fe1b220-0x000000007fe61e34)  ---  0x00000220 0x00046c20 0x60000020 DxeCore.dll...text
  0x00000002 data             [0x000000007fe61e40-0x000000007fe68065)  ---  0x00046e40 0x00006240 0x40000040 DxeCore.dll...rdata
  0x00000003 data             [0x000000007fe68080-0x000000007fe6d160)  ---  0x0004d080 0x000018a0 0xc0000040 DxeCore.dll...data
  0x00000004 regular          [0x000000007fe6d160-0x000000007fe6d398)  ---  0x0004e920 0x00000240 0x42000040 DxeCore.dll...reloc
Sections for '/Users/user/Documents/UefiWorkspace/Build/OvmfX64/NOOPT_CLANGPDB/X64/MdeModulePkg/Universal/DevicePathDxe/DevicePathDxe/DEBUG/DevicePathDxe.dll' (x86_64):
  SectID     Type             Load Address                             Perm File Off.  File Size  Flags      Section Name
  ---------- ---------------- ---------------------------------------  ---- ---------- ---------- ---------- ----------------------------
  0xffffffffffffffff container        [0x0000000000000000-0x0000000000014420)* ---  0x00000000 0x00000000 0x00000000 DevicePathDxe.dll.
  0x00000001 code             [0x000000007f986220-0x000000007f996cc6)  ---  0x00000220 0x00010ac0 0x60000020 DevicePathDxe.dll...text
  0x00000002 data             [0x000000007f996ce0-0x000000007f999b04)  ---  0x00010ce0 0x00002e40 0x40000040 DevicePathDxe.dll...rdata
  0x00000003 data             [0x000000007f999b20-0x000000007f99a1a2)  ---  0x00013b20 0x00000660 0xc0000040 DevicePathDxe.dll...data
  0x00000004 regular          [0x000000007f99a1c0-0x000000007f99a404)  ---  0x00014180 0x00000260 0x42000040 DevicePathDxe.dll…reloc
…

So, all in all, unique global variables work, but there is no way to access duplicating variables. They either resolve to GdbSyms or just cause a crash:

(lldb) p mDebugInfoTableHeader
(EFI_DEBUG_IMAGE_INFO_TABLE_HEADER) $0 = {
  UpdateStatus = 2
  TableSize = 92
  EfiDebugImageInfoTable = 0x000000007f814018
}
(lldb) p gBS
error: Couldn't materialize: couldn't get the value of variable ::gBS: read memory from 0x6df8 failed
error: errored out in DoExecute, couldn't PrepareToExecuteJITExpression
(lldb) p gEfiGlobalVariableGuid
0  libLLVM.dylib            0x000000010e52ee68 llvm::sys::PrintStackTrace(llvm::raw_ostream&) + 40
1  libLLVM.dylib            0x000000010e52f262 SignalHandler(int) + 188
2  libsystem_platform.dylib 0x00007fff6ca5642d _sigtramp + 29
...


If you want to inspect globals I think this logic works to get you data, you would need to print it out etc.

SBValueList = lldb.target.FindGlobalVariables ("gST", 1024)
for SBValue in SBValueList:
   Module = SBValue.GetAddress().GetModule()
        ModuleStr = SBValue.GetAddress().GetModule().GetFileSpec().GetFilename()
        Start = int (SBValue.GetLocation(), 0)
        End = Start + SBValue.GetByteSize() - 1
        SBDeclaration = SBValue.GetDeclaration()
        Column = SBDeclaration.GetColumn()

I wrote a command in the early days to dump out all the instances of a global.

You can also try (lldb) image lookup -Av --name gST


You can tell lldb to use the older Python like this (from the Terminal.app):
$ defaults write com.apple.dt.lldb DefaultPythonVersion 2

Thanks, that helped quite a bit, but for some reason Xcode version still crashes more for me. I attached a couple of stack traces if you feel like having a look, but once again it seems that it is all about the PDB plugin.

For the macOS API clang emits frame pointers, so you can walk the stack without symbols. You could try adding the compiler flag to emit the frame pointers.


This is easy enough to check as %rpb is the frame pointer so it will get saved/restored on function entry/exit.


I am pretty sure stack frames are not disabled with UEFI, as sometimes backtracing works just fine. To me it looks like debug information parsing randomly breaks in LLDB, and once it happens it forgets about other images:

(lldb) b CoreLocateHandleBuffer
Breakpoint 2: where = DxeCore.dll`CoreLocateHandleBuffer + 31 at Locate.c:649, address = 0x000000007fe36e4f
(lldb) c
Process 1 resuming
Process 1 stopped
* thread #1, stop reason = breakpoint 2.1
    frame #0: 0x000000007fe36e4f DxeCore.dll`CoreLocateHandleBuffer(SearchType=ByProtocol, Protocol=0x000000007f978160, SearchKey=0x0000000000000000, NumberHandles=0x000000007fe19fd8, Buffer=0x000000007fe19fc0) at Locate.c:649
   646      EFI_STATUS          Status;
   647      UINTN               BufferSize;
   648
-> 649     if (NumberHandles == NULL) {
   650        return EFI_INVALID_PARAMETER;
   651      }
   652
(lldb) bt
* thread #1, stop reason = breakpoint 2.1
  * frame #0: 0x000000007fe36e4f DxeCore.dll`CoreLocateHandleBuffer(SearchType=ByProtocol, Protocol=0x000000007f978160, SearchKey=0x0000000000000000, NumberHandles=0x000000007fe19fd8, Buffer=0x000000007fe19fc0) at Locate.c:649
    frame #1: 0x000000007fe36816 DxeCore.dll`CoreLocateDevicePath(Protocol=0x000000007f978160, DevicePath=0x000000007fe1a060, Device=0x000000007fe1a068) at Locate.c:466
    frame #2: 0x000000007f97479a SecurityStubDxe.dll

———

(lldb) b CopyMem
Breakpoint 3: 70 locations.
(lldb) c
Process 1 resuming
Process 1 stopped
* thread #1, stop reason = breakpoint 2.53 3.53
    frame #0: 0x000000007e5c13b3 MnpDxe.dll`CopyMem(DestinationBuffer=0x000000007fe19b50, SourceBuffer=0x000000007e2aa470, Length=656) at CopyMemWrapper.c:47
   44        IN UINTN       Length
   45        )
   46      {
-> 47       if (Length == 0) {
   48          return DestinationBuffer;
   49        }
   50        ASSERT ((Length - 1) <= (MAX_ADDRESS - (UINTN)DestinationBuffer));
(lldb) bt
* thread #1, stop reason = breakpoint 2.53 3.53
  * frame #0: 0x000000007e5c13b3 MnpDxe.dll`CopyMem(DestinationBuffer=0x000000007fe19b50, SourceBuffer=0x000000007e2aa470, Length=656) at CopyMemWrapper.c:47
(lldb) finish
error: Could not create return address breakpoint.
(lldb) n
Process 1 stopped
* thread #1, stop reason = step over
    frame #0: 0x000000007e5c13ce MnpDxe.dll`CopyMem(DestinationBuffer=0x000000007fe19b50, SourceBuffer=0x000000007e2aa470, Length=656) at CopyMemWrapper.c:50
   47        if (Length == 0) {
   48          return DestinationBuffer;
   49        }
-> 50       ASSERT ((Length - 1) <= (MAX_ADDRESS - (UINTN)DestinationBuffer));
   51        ASSERT ((Length - 1) <= (MAX_ADDRESS - (UINTN)SourceBuffer));
   52
   53        if (DestinationBuffer == SourceBuffer) {
(lldb)
...
Process 1 stopped
* thread #1, stop reason = step over
    frame #0: 0x000000007e5c14b4 MnpDxe.dll`CopyMem(DestinationBuffer=0x000000007fe19b50, SourceBuffer=0x000000007e2aa470, Length=656) at CopyMemWrapper.c:57
   54          return DestinationBuffer;
   55        }
   56        return InternalMemCopyMem (DestinationBuffer, SourceBuffer, Length);
-> 57     }
(lldb)
Process 1 stopped
* thread #1, stop reason = step over
    frame #0: 0x000000007e5c726e MnpDxe.dll
->  0x7e5c726e: mov    rax, qword ptr [rsp + 0x60]
    0x7e5c7273: cmp    byte ptr [rax + 0x68], 0x0
    0x7e5c7277: jne    0x7e5c7291
    0x7e5c727d: movabs rax, -0x7fffffffffffffed
(lldb) bt
* thread #1, stop reason = step over
  * frame #0: 0x000000007e5c726e MnpDxe.dll

———

(lldb) c
Process 1 resuming
Process 1 stopped
* thread #1, stop reason = signal SIGINT
    frame #0: 0x000000007fe4d72e DxeCore.dll
->  0x7fe4d72e: cmp    al, 0x0
    0x7fe4d730: je     0x7fe4d765
    0x7fe4d736: mov    rcx, qword ptr [rsp + 0x20]
    0x7fe4d73b: call   0x7fe4c4b0
(lldb) bt
* thread #1, stop reason = signal SIGINT
  * frame #0: 0x000000007fe4d72e DxeCore.dll

On macOS the Mach-O and dSYM have a UUID (dwarfdump -u) that is indexed by Spotlight (mdfind "com_apple_xcode_dsym_uuids == *") [2]
This should be the UUID in the debug directory entry and you can use that to lookup the symbols like this:

module = target.AddModule (None, None, uuid)
SBError = target.SetModuleLoadAddress (module, LoadAddress + TeAdjust)

Also lldb has built in help for commands, but it is kind of terse since it is autogenerated from the C++ swig.
(lldb) script help (lldb.target.AddModule)
Help on method AddModule in module lldb:

AddModule(self, *args) method of lldb.SBTarget instance
    AddModule(SBTarget self, SBModule module) -> bool
    AddModule(SBTarget self, char const * path, char const * triple, char const * uuid) -> SBModule
    AddModule(SBTarget self, char const * path, char const * triple, char const * uuid_cstr, char const * symfile) -> SBModule
    AddModule(SBTarget self, SBModuleSpec module_spec) -> SBModule


The minimum  you need to symbolicate a frame is uuid, LoadAddress, and PC.

[1] http://lldb.llvm.org/use/map.html
[2] http://lldb.llvm.org/use/symbols.html

Thanks for the links again. Yes, I am using some of these, and in fact for GDB that’s pretty much what I did when I worked with XCODE5. It is very likely that when I get to complete LLDB support for XCODE5 it will work quite fine too. But I am already happy with XCODE5 here, and making it even better will only help myself, but not other people with e.g. Linux or people that want me to use the same compiler with them.


Thanks for looking out for others.

Thanks,

Andrew Fish


Best regards,
Vitaly


21 марта 2020 г., в 20:13, Andrew Fish <afish@apple.com<mailto:afish@apple.com>> написал(а):




On Mar 21, 2020, at 3:28 AM, Vitaly Cheptsov <cheptsov@ispras.ru<mailto:cheptsov@ispras.ru>> wrote:

Hello,

Andrey, thanks for the hint, it was very helpful. I rewrote the GDB scripts to work with LLDB[1] and was able to debug OVMF built with CLANGPDB. While it is still quite dirty, at the very least it works.

Unfortunately the experience was close to terrible. I may certainly do something wrong, but it is clear that PDB and LLDB do not support each other well enough. After spending several hours on playing with the tools my conclusion is that LLDB is simply not suited for UEFI PDB debugging, and we really want DWARF  as there is no other opensource debugger that supports PDB on macOS and Linux

In case somebody knows workarounds here are the issues I faced:

1. All integer alias typedefs are discarded in favour of underlying types. This way EFI_STATUS and EFI_TPL become unsigned long long, CHAR8 becomes char, and CHAR16 becomes unsigned short. It does not look like LLDB has the original types anywhere at all, and it also does not have them registered.

    frame #0: 0x000000007fe242aa DxeCore.dll`CoreAllocatePoolPagesI(PoolType=EfiBootServicesData, NoPages=1, Granularity=4096, NeedGuard='\0') at Pool.c:322
   319             return NULL;
   320           }
   321
-> 322          Buffer = CoreAllocatePoolPages (PoolType, NoPages, Granularity, NeedGuard);
   323    CoreReleaseMemoryLock ();
   324
   325           if (Buffer != NULL) {
(lldb) p Status
(unsigned long long) $3 = 0

Structures work more or less fine, but for simpler types like strings we are out of even potential pretty-printing.


Vitaly,

You can teach lldb about types. There is some example code here: https://github.com/tianocore/edk2/blob/master/EmulatorPkg/Unix/lldbefi.py


2. Global variables are not accessible. I am not sure what happens, but they either seem to not relocate or conflict with the other names:

(lldb) p gST
error: Couldn't materialize: couldn't get the value of variable ::gST: read memory from 0x6e18 failed
error: errored out in DoExecute, couldn't PrepareToExecuteJITExpression
(lldb) p &gST
error: Couldn't materialize: couldn't get the value of variable ::gST: read memory from 0x6e18 failed
error: errored out in DoExecute, couldn't PrepareToExecuteJITExpression


That is strange as globals usually work best? The common issue I've seen is getting the slide wrong. The EFI modules are linked at a value near zero and relocated into memory, so the slide represents that adjustment.

You can use `image dump sections` and ` image dump symtab` to see lldb's view of symbols. More info here [1].


3. Quite a number of crashes.

In most cases autocompletion by tab press causes a crash. E.g.

b I<TAB>

So will do printing of a GUID, e.g. p gEfiGlobalVariableGuid.

This may have to do with Python compatibility as Xcode 11 LLDB that uses Python 3 generally crashes more often than MacPorts LLDB 9.0. Surprisingly structures work more or less fine.


You can tell lldb to use the older Python like this (from the Terminal.app):
$ defaults write com.apple.dt.lldb DefaultPythonVersion 2


4. Ctrl+C does not produce a valid backtrace. When I break with a breakpoint, I see a proper stacktrace with more than one entry, with function prototypes and values. When I break with Ctrl+C I only see some weird backtrace with most of the entries missing regardless of frame position:

(lldb) bt
* thread #1, stop reason = signal SIGTRAP
  * frame #0: 0x000000007fe4c5f3 DxeCore.dll

Probably more and all the unintuitive stuff like the lack of more functional TUI, but it is hard to remember all the trials.


For the macOS API clang emits frame pointers, so you can walk the stack without symbols. You could try adding the compiler flag to emit the frame pointers.


[1] https://github.com/acidanthera/OpenCorePkg/blob/master/Debug/Scripts/lldb_uefi.py


On macOS the Mach-O and dSYM have a UUID (dwarfdump -u) that is indexed by Spotlight (mdfind "com_apple_xcode_dsym_uuids == *") [2]
This should be the UUID in the debug directory entry and you can use that to lookup the symbols like this:

module = target.AddModule (None, None, uuid)
SBError = target.SetModuleLoadAddress (module, LoadAddress + TeAdjust)

Also lldb has built in help for commands, but it is kind of terse since it is autogenerated from the C++ swig.
(lldb) script help (lldb.target.AddModule)
Help on method AddModule in module lldb:

AddModule(self, *args) method of lldb.SBTarget instance
    AddModule(SBTarget self, SBModule module) -> bool
    AddModule(SBTarget self, char const * path, char const * triple, char const * uuid) -> SBModule
    AddModule(SBTarget self, char const * path, char const * triple, char const * uuid_cstr, char const * symfile) -> SBModule
    AddModule(SBTarget self, SBModuleSpec module_spec) -> SBModule


The minimum  you need to symbolicate a frame is uuid, LoadAddress, and PC.

[1] http://lldb.llvm.org/use/map.html
[2] http://lldb.llvm.org/use/symbols.html

Thanks,

Andrew Fish



Best wishes,
Vitaly

20 марта 2020 г., в 22:14, Andrew Fish <afish@apple.com<mailto:afish@apple.com>> написал(а):




On Mar 20, 2020, at 8:13 AM, Vitaly Cheptsov <cheptsov@ispras.ru<mailto:cheptsov@ispras.ru>> wrote:

Hello,

We noticed that the original bugzilla, which intended to add new LLVM toolchain support[1], also wanted to bring ELF format support with DWARF debugging information. For some reason this did not make its way into EDK II, and we are currently wondering, how can one debug binaries built with LLVM 9.0.


For macOS and XCODE5 toolchain we use GDB scripts based on Andrei Warkentin’s work, which allow us to integrate with QEMU and VMware[2]. It is likely that they should work with little to no work on Linux with CLANG38/GCC5 with GDB once again. However, CLANGPDB apparently is using PDB debugging information, which I believe is not handled with GDB.

Could you please provide the details on the matter and let us know about the recommended route?
— Is dropping CLANGELF just a temporary measure and it should be resubmitted again?
— Should LLDB, which seems to be aware of PDB, be used instead of GDB, when building with CLANGPDB? If so, did anybody try that?



Vitaly,

I've not tried the CLANGPDB path, but if you want to connect lldb to QEMU you need to set  plugin.process.gdb-remote.target-definition-file [1] to [2].

[1]  lldb -o "settings set plugin.process.gdb-remote.target-definition-file x86_64_target_definition.py" -o "gdb-remote 9000"
[2] https://github.com/llvm-mirror/lldb/blob/master/examples/python/x86_64_target_definition.py

Thanks,

Andrew Fish


Thanks!


Best regards,
Vitaly

[1] https://bugzilla.tianocore.org/show_bug.cgi?id=1603
[2] https://github.com/acidanthera/OpenCorePkg/blob/master/Debug/Scripts/gdb_uefi.py



<crashes.txt>



[-- Attachment #2: Type: text/html, Size: 118459 bytes --]

Re: [edk2-devel] CLANGPDB binary debugging

[Vitaly Cheptsov]

[-- Attachment #1.1: Type: text/plain, Size: 48699 bytes --]

Liming,

The issue with CLANG38 is that it uses LD BFD, a GNU linker, which is not distributed with LLVM. This makes CLANG38 to be unusable on e.g. macOS or other BSD systems, where GNU linker is not used. Making CLANG38 somehow work with LLD (e.g. by somehow providing it an environment variable to use ld.lld) should most likely solve the issue, but I am not positive how to address it best.

Best regards,
Vitaly

> 25 марта 2020 г., в 16:16, Gao, Liming <liming.gao@intel.com> написал(а):
> 
> Vitaly:
>  Thanks for your further investigation with LLDB. Now, we have two CLANG tool chain. CLANGPDB and CLANG38. CLANGPDB works with LLVM9.0 or above to generate PE image with PDB debug symbol. CLANG38 can also be used to generate ELF image and convert to PE image with DWARF debug symbol on Linux host. So, these two chains cover both usage models. We don’t need to introduce new CLANGDWARF tool chain. For LLDB issue on PDB debug symbol, I suggest to report them in LLVM.
> 
>  Besides, LLVM10.0 is just released.
> 
> Thanks
> Liming
> From: devel@edk2.groups.io <mailto:devel@edk2.groups.io> <devel@edk2.groups.io <mailto:devel@edk2.groups.io>> On Behalf Of Vitaly Cheptsov
> Sent: 2020年3月23日 17:10
> To: Andrew Fish <afish@apple.com <mailto:afish@apple.com>>; Gao, Liming <liming.gao@intel.com <mailto:liming.gao@intel.com>>
> Cc: devel@edk2.groups.io <mailto:devel@edk2.groups.io>; Marvin Häuser <mhaeuser@outlook.de <mailto:mhaeuser@outlook.de>>
> Subject: Re: [edk2-devel] CLANGPDB binary debugging
> 
> Andrew,
> 
> I investigated further and was able to make some progress. I got rid of PDB, and changed debug information generation to DWARF[1]. This resolved most of LLDB crashes and made pretty-printing work just fine as type aliases like EFI_STATUS now work correctly (I am still to commit this). Unfortunately there are several downsides. Dead code stripping in LLD linker is broken when generating PE with DWARF, so I had to patch it[2]. Debug link implementation in llvm-objcopy is also quite ugly, we do not get MTOC or anything alike[3], just some section with a random name containing base filename.
> 
> I can submit a more proper patch to EDK II properly implementing CLANGDWARF toolchain instead of patching CLANGPDB like in [1], but it is questionable whether it is worth it. LLVM 10 will land very soon, and the necessary LLD patches[2] will definitely not make their way in this release. We will have to wait for LLVM 11 at the very least, and I am not sure I am ready to drive llvm-objcopy changes either. Liming, what do you think?
> 
> I've not used the fake image to get things done so I can't speak to that. I have used a fake target so I could have XIP PEIM and shadowed PEIM address available at the same time. You can't have a module loaded at 2 addresses at the same time in llldb. But you might be able to use a fake target for your fake stuff?
> 
> Just in case:
> 
>           # create a faka target to store info about symbols
>           PeiXipTarget = target.debugger.CreateTarget (None, "i386-apple-macosx", "remote-macosx", True, error)
> 
>           # make sure the gdb-remote  connection target is the active target
>           target.debugger.SetSelectedTarget (target)
> 
> Yes, that was actually a smart idea. I initially did it that way, as I had mixing Mach-O and PE triples, but later removed, which was actually a mistake. Now I have a fake target for GdbSyms and a proper target for other images[4]. This somewhat solves the issue with invalid addresses for gST, as gST from DxeCore is now picked. However, I cannot access global variables from other modules for some reason still.
> 
> If you want to inspect globals I think this logic works to get you data, you would need to print it out etc.
> 
> SBValueList = lldb.target.FindGlobalVariables ("gST", 1024)
> for SBValue in SBValueList:
> Module = SBValue.GetAddress().GetModule()
>         ModuleStr = SBValue.GetAddress().GetModule().GetFileSpec().GetFilename()
>         Start = int (SBValue.GetLocation(), 0)
>         End = Start + SBValue.GetByteSize() - 1
>         SBDeclaration = SBValue.GetDeclaration()
>         Column = SBDeclaration.GetColumn()
> 
> I wrote a command in the early days to dump out all the instances of a global.
> 
> You can also try (lldb) image lookup -Av --name gST
> 
> Something is really weird, gST from the first image shows, but lldb believes that it does not exist. lldb.target.FindGlobalVariables ("gST", 1024) also returns an empty list.
> 
> I noticed that LLDB shows invalid type for gST in the symtab, there also is no address. I started to wonder whether this is the cause and decided to check XCODE5, as the the issue could have been with PE format. Unfortunately even after I tried XCODE5 support with LLDB, I got exactly the same problem. I am quite clueless what is going on. Below I included two transcripts: for CLANGDWARF and XCODE5.
> 
> (lldb) p gST
> (EFI_SYSTEM_TABLE *) $0 = 0x000000007f9ee018
> (lldb) p *gST
> (EFI_SYSTEM_TABLE) $1 = {
>   Hdr = (Signature = 0x5453595320494249, Revision = 0x00020046, HeaderSize = 0x00000078, CRC32 = 0x3d5e9b5d, Reserved = 0x00000000)
>   FirmwareVendor = 0x000000007f9e9c98
>   FirmwareRevision = 0x00010000
>   ConsoleInHandle = 0x000000007ed58f18
>   ConIn = 0x000000007e6edcb8
>   ConsoleOutHandle = 0x000000007ed57018
>   ConOut = 0x000000007e6ede40
>   StandardErrorHandle = 0x000000007ed57a98
>   StdErr = 0x000000007e6ee030
>   RuntimeServices = 0x000000007f9eeb98
>   BootServices = 0x000000007fe689e8
>   NumberOfTableEntries = 0x000000000000000a
>   ConfigurationTable = 0x000000007f9eec98
> }
> (lldb) image lookup -Av --name gST
> (lldb) image dump sections
> Dumping sections for 90 modules.
> Sections for '/Users/user/Documents/UefiWorkspace/Build/OvmfX64/NOOPT_CLANGPDB/X64/DxeCore.debug' (x86_64):
>   SectID     Type             Load Address                             Perm File Off.  File Size  Flags      Section Name
>   ---------- ---------------- ---------------------------------------  ---- ---------- ---------- ---------- ----------------------------
>   0xffffffffffffffff container        [0x000000007fe1b000-0x000000007ff38ac0)  ---  0x00000000 0x00000000 0x00000000 DxeCore.debug.
>   0x00000001 code             [0x000000007fe1b320-0x000000007fe61f34)  ---  0x00000320 0x00046c20 0x60000020 DxeCore.debug...text
>   0x00000002 data             [0x000000007fe61f40-0x000000007fe680f1)  ---  0x00046f40 0x000061c0 0x40000040 DxeCore.debug...rdata
>   0x00000003 data             [0x000000007fe68100-0x000000007fe6d1e0)  ---  0x0004d100 0x000018a0 0xc0000040 DxeCore.debug...data
>   0x00000004 regular          [0x000000007fe6d1e0-0x000000007fe6d424)  ---  0x0004e9a0 0x00000260 0x42000040 DxeCore.debug...reloc
>   0x00000005 dwarf-abbrev     [0x000000007fe6d440-0x000000007fe74059)  ---  0x0004ec00 0x00006c20 0x42000040 DxeCore.debug...debug_abbrev
>   0x00000006 dwarf-info       [0x000000007fe74060-0x000000007fec1355)  ---  0x00055820 0x0004d300 0x42000040 DxeCore.debug...debug_info
>   0x00000007 dwarf-line       [0x000000007fec1360-0x000000007fef9fd4)  ---  0x000a2b20 0x00038c80 0x42000040 DxeCore.debug...debug_line
>   0x00000008 dwarf-macinfo    [0x000000007fef9fe0-0x000000007fefa056)  ---  0x000db7a0 0x00000080 0x42000040 DxeCore.debug...debug_macinfo
>   0x00000009 dwarf-ranges     [0x000000007fefa060-0x000000007feff610)  ---  0x000db820 0x000055c0 0x42000040 DxeCore.debug...debug_ranges
>   0x0000000a dwarf-str        [0x000000007feff620-0x000000007ff38abd)  ---  0x000e0de0 0x000394a0 0x42000040 DxeCore.debug...debug_str
> Sections for '/Users/user/Documents/UefiWorkspace/Build/OvmfX64/NOOPT_CLANGPDB/X64/DevicePathDxe.debug' (x86_64):
>   SectID     Type             Load Address                             Perm File Off.  File Size  Flags      Section Name
>   ---------- ---------------- ---------------------------------------  ---- ---------- ---------- ---------- ----------------------------
>   0xffffffffffffffff container        [0x000000007f986000-0x000000007f9cb0a0)  ---  0x00000000 0x00000000 0x00000000 DevicePathDxe.debug.
>   0x00000001 code             [0x000000007f986320-0x000000007f996dc6)  ---  0x00000320 0x00010ac0 0x60000020 DevicePathDxe.debug...text
>   0x00000002 data             [0x000000007f996de0-0x000000007f999b75)  ---  0x00010de0 0x00002da0 0x40000040 DevicePathDxe.debug...rdata
>   0x00000003 data             [0x000000007f999b80-0x000000007f99a202)  ---  0x00013b80 0x00000660 0xc0000040 DevicePathDxe.debug...data
>   0x00000004 regular          [0x000000007f99a220-0x000000007f99a464)  ---  0x000141e0 0x00000260 0x42000040 DevicePathDxe.debug...reloc
>   0x00000005 dwarf-abbrev     [0x000000007f99a480-0x000000007f99bd09)  ---  0x00014440 0x000018a0 0x42000040 DevicePathDxe.debug...debug_abbrev
>   0x00000006 dwarf-info       [0x000000007f99bd20-0x000000007f9ad8b7)  ---  0x00015ce0 0x00011ba0 0x42000040 DevicePathDxe.debug...debug_info
>   0x00000007 dwarf-line       [0x000000007f9ad8c0-0x000000007f9bc1f7)  ---  0x00027880 0x0000e940 0x42000040 DevicePathDxe.debug...debug_line
>   0x00000008 dwarf-macinfo    [0x000000007f9bc200-0x000000007f9bc223)  ---  0x000361c0 0x00000040 0x42000040 DevicePathDxe.debug...debug_macinfo
>   0x00000009 dwarf-ranges     [0x000000007f9bc240-0x000000007f9be020)  ---  0x00036200 0x00001de0 0x42000040 DevicePathDxe.debug...debug_ranges
>   0x0000000a dwarf-str        [0x000000007f9be020-0x000000007f9cb099)  ---  0x00037fe0 0x0000d080 0x42000040 DevicePathDxe.debug...debug_str
> …
> (lldb) image dump symtab
> Dumping symbol table for 90 modules.
> Symtab, file = /Users/user/Documents/UefiWorkspace/Build/OvmfX64/NOOPT_CLANGPDB/X64/DxeCore.debug, num_symbols = 4343:
>                Debug symbol
>                |Synthetic symbol
>                ||Externally Visible
>                |||
> Index   UserID DSX Type            File Address/Value Load Address       Size               Flags      Name
> ------- ------ --- --------------- ------------------ ------------------ ------------------ ---------- ----------------------------------
> [    0] 4294967295     Invalid         0x0000000000000000 0x000000007fe1b000 0x000000000000003a 0x00000000 .text
> [    1] 4294967295     Invalid         0x0000000000000000                    0x0000000000000000 0x00000000 .data
> [    2] 4294967295     Invalid         0x0000000000001898                    0x0000000000000000 0x00000000 .bss
> [    3] 4294967295     Invalid         0x0000000000046bc0 0x000000007fe61bc0 0x0000000000000374 0x00000000 .text$_ModuleEntryPoint
> [    4] 4294967295     Code            0x0000000000046bc0 0x000000007fe61bc0 0x0000000000000374 0x00000000 _ModuleEntryPoint
> [    5] 4294967295     Invalid         0x0000000000001b08                    0x0000000000000000 0x00000000 .bss$gHobList
> [    6] 4294967295     Invalid         0x0000000000001b08                    0x0000000000000000 0x00000000 gHobList
> [    7] 4294967295     Invalid         0x0000000000000000                    0x0000000000000000 0x00000000 .rdata
> [    8] 4294967295     Invalid         0x0000000000000000                    0x0000000000000000 0x00000000 .str
> [    9] 4294967295     Invalid         0x0000000000000058                    0x0000000000000000 0x00000000 .rdata
> [   10] 4294967295     Invalid         0x0000000000000058                    0x0000000000000000 0x00000000 .str.1
> [   11] 4294967295     Invalid         0x0000000000000000                    0x0000000000000000 0x00000000 .debug_str
> [   12] 4294967295     Invalid         0x0000000000000000                    0x0000000000000000 0x00000000 .debug_abbrev
> [   13] 4294967295     Invalid         0x0000000000000000                    0x0000000000000000 0x00000000 .debug_info
> [   14] 4294967295     Invalid         0x0000000000000000                    0x0000000000000000 0x00000000 .debug_ranges
> [   15] 4294967295     Invalid         0x0000000000000000                    0x0000000000000000 0x00000000 .debug_macinfo
> [   16] 4294967295     Invalid         0x0000000000000000                    0x0000000000000000 0x00000000 .debug_line
> [   17] 4294967295     Code            0x000000000003f680 0x000000007fe5a680 0x0000000000000020 0x00000000 ProcessModuleEntryPointList
> [   18] 4294967295     Code            0x0000000000023130 0x000000007fe3e130 0x0000000000000020 0x00000000 DebugAssertEnabled
> [   19] 4294967295     Code            0x0000000000023060 0x000000007fe3e060 0x00000000000000d0 0x00000000 DebugAssert
> [   20] 4294967295     Code            0x0000000000022150 0x000000007fe3d150 0x0000000000000020 0x00000000 CpuDeadLoop
> [   21] 4294967295     Invalid         0x0000000000000000 0x000000007fe1b000 0x000000000000003a 0x00000000 .text
> [   22] 4294967295     Invalid         0x0000000000000000                    0x0000000000000000 0x00000000 .data
> [   23] 4294967295     Invalid         0x0000000000001898                    0x0000000000000000 0x00000000 .bss
> [   24] 4294967295     Invalid         0x000000000003f240 0x000000007fe5a240 0x0000000000000440 0x00000000 .text$ProcessLibraryConstructorList
> [   25] 4294967295     Code            0x000000000003f240 0x000000007fe5a240 0x0000000000000440 0x00000000 ProcessLibraryConstructorList
> [   26] 4294967295     Invalid         0x000000000003f680 0x000000007fe5a680 0x0000000000000020 0x00000000 .text$ProcessModuleEntryPointList
> [   27] 4294967295     Invalid         0x0000000000000070                    0x0000000000000000 0x00000000 .data$gEfiCallerIdGuid
> [   28] 4294967295     Invalid         0x0000000000000070                    0x0000000000000000 0x00000000 gEfiCallerIdGuid
> [   29] 4294967295     Invalid         0x0000000000000148                    0x0000000000000000 0x00000000 .data$gEfiEventMemoryMapChangeGuid
> [   30] 4294967295     Invalid         0x0000000000000148                    0x0000000000000000 0x00000000 gEfiEventMemoryMapChangeGuid
> [   31] 4294967295     Invalid         0x0000000000000168                    0x0000000000000000 0x00000000 .data$gEfiEventVirtualAddressChangeGuid
> [   32] 4294967295     Invalid         0x0000000000000168                    0x0000000000000000 0x00000000 gEfiEventVirtualAddressChangeGuid
> [   33] 4294967295     Invalid         0x0000000000000138                    0x0000000000000000 0x00000000 .data$gEfiEventExitBootServicesGuid
> [   34] 4294967295     Invalid         0x0000000000000138                    0x0000000000000000 0x00000000 gEfiEventExitBootServicesGuid
> ….
> [  303] 4294967295     Invalid         0x0000000000001b60                    0x0000000000000000 0x00000000 .bss$gST
> [  304] 4294967295     Invalid         0x0000000000001b60                    0x0000000000000000 0x00000000 gST
> [  305] 4294967295     Invalid         0x00000000000018a0                    0x0000000000000000 0x00000000 .bss$gBS
> [  306] 4294967295     Invalid         0x00000000000018a0                    0x0000000000000000 0x00000000 gBS
> …
> 
> ———————————————
> 
> (lldb) p gST
> (EFI_SYSTEM_TABLE *) $2 = 0x000000007fbee018
> (lldb) p &gST
> (EFI_SYSTEM_TABLE **) $4 = 0x000000007fe6d788
> (lldb) p *gST
> (EFI_SYSTEM_TABLE) $3 = {
>   Hdr = (Signature = 0x5453595320494249, Revision = 0x00020046, HeaderSize = 0x00000078, CRC32 = 0x5355dbe2, Reserved = 0x00000000)
>   FirmwareVendor = 0x000000007fbe9c98
>   FirmwareRevision = 0x00010000
>   ConsoleInHandle = 0x000000007ede5a98
>   ConIn = 0x000000007e95f640
>   ConsoleOutHandle = 0x000000007ede4e98
>   ConOut = 0x000000007e95f800
>   StandardErrorHandle = 0x000000007ede4b98
>   StdErr = 0x000000007e95f910
>   RuntimeServices = 0x000000007fbeeb98
>   BootServices = 0x000000007fe6cde8
>   NumberOfTableEntries = 0x0000000000000009
>   ConfigurationTable = 0x000000007fbeec98
> }
> (lldb) image lookup -Av --name gST
> (lldb) image dump sections
> Dumping sections for 88 modules.
> Sections for '/Users/user/Documents/UefiWorkspace/Build/OvmfX64/NOOPT_XCODE5/X64/MdeModulePkg/Core/Dxe/DxeMain/DEBUG/DxeCore.dll' (x86_64):
>   SectID     Type             Load Address                             Perm File Off.  File Size  Flags      Section Name
>   ---------- ---------------- ---------------------------------------  ---- ---------- ---------- ---------- ----------------------------
>   0x00000100 container        [0x000000007fe26240-0x000000007fe6bb40)  r-x  0x000004a0 0x00045900 0x00000000 DxeCore.dll.__TEXT
>   0x00000001 code             [0x000000007fe26240-0x000000007fe65bd8)  r-x  0x000004a0 0x0003f998 0x80000400 DxeCore.dll.__TEXT.__text
>   0x00000002 data-cstr        [0x000000007fe65bd8-0x000000007fe6ba23)  r-x  0x0003fe38 0x00005e4b 0x00000002 DxeCore.dll.__TEXT.__cstring
>   0x00000003 regular          [0x000000007fe6ba24-0x000000007fe6ba6c)  r-x  0x00045c84 0x00000048 0x00000000 DxeCore.dll.__TEXT.__ustring
>   0x00000004 regular          [0x000000007fe6ba70-0x000000007fe6bb40)  r-x  0x00045cd0 0x000000d0 0x00000000 DxeCore.dll.__TEXT.__const
>   0x00000200 container        [0x000000007fe6bb40-0x000000007fe70e60)  rw-  0x00045da0 0x00001ae0 0x00000000 DxeCore.dll.__DATA
>   0x00000005 regular          [0x000000007fe6bb40-0x000000007fe6bd58)  rw-  0x00045da0 0x00000218 0x00000000 DxeCore.dll.__DATA.__const
>   0x00000006 data             [0x000000007fe6bd60-0x000000007fe6d610)  rw-  0x00045fc0 0x000018b0 0x00000000 DxeCore.dll.__DATA.__data
>   0x00000007 zero-fill        [0x000000007fe6d610-0x000000007fe70e20)  rw-  0x00000000 0x00000000 0x00000001 DxeCore.dll.__DATA.__common
>   0x00000008 zero-fill        [0x000000007fe70e20-0x000000007fe70e58)  rw-  0x00000000 0x00000000 0x00000001 DxeCore.dll.__DATA.__bss
>   0x00000100 container        [0x000000000004b000-0x00000000000ef000)* rw-  0x00001000 0x000a3a74 0x00000000 DxeCore.dll.__DWARF
>   0x00000001 dwarf-line       [0x000000000004b000-0x00000000000723fb)* rw-  0x00001000 0x000273fb 0x00000000 DxeCore.dll.__DWARF.__debug_line
>   0x00000002 dwarf-pubnames   [0x00000000000723fb-0x0000000000078429)* rw-  0x000283fb 0x0000602e 0x00000000 DxeCore.dll.__DWARF.__debug_pubnames
>   0x00000003 dwarf-pubtypes   [0x0000000000078429-0x000000000008b915)* rw-  0x0002e429 0x000134ec 0x00000000 DxeCore.dll.__DWARF.__debug_pubtypes
>   0x00000004 dwarf-aranges    [0x000000000008b915-0x000000000008ea65)* rw-  0x00041915 0x00003150 0x00000000 DxeCore.dll.__DWARF.__debug_aranges
>   0x00000005 dwarf-info       [0x000000000008ea65-0x00000000000c78bf)* rw-  0x00044a65 0x00038e5a 0x00000000 DxeCore.dll.__DWARF.__debug_info
>   0x00000006 dwarf-frame      [0x00000000000c78bf-0x00000000000cc697)* rw-  0x0007d8bf 0x00004dd8 0x00000000 DxeCore.dll.__DWARF.__debug_frame
>   0x00000007 dwarf-ranges     [0x00000000000cc697-0x00000000000cc757)* rw-  0x00082697 0x000000c0 0x00000000 DxeCore.dll.__DWARF.__debug_ranges
>   0x00000008 dwarf-abbrev     [0x00000000000cc757-0x00000000000cca9b)* rw-  0x00082757 0x00000344 0x00000000 DxeCore.dll.__DWARF.__debug_abbrev
>   0x00000009 dwarf-str        [0x00000000000cca9b-0x00000000000dcc05)* rw-  0x00082a9b 0x0001016a 0x00000000 DxeCore.dll.__DWARF.__debug_str
>   0x0000000a apple-names      [0x00000000000dcc05-0x00000000000e2439)* rw-  0x00092c05 0x00005834 0x00000000 DxeCore.dll.__DWARF.__apple_names
>   0x0000000b apple-namespaces [0x00000000000e2439-0x00000000000e245d)* rw-  0x00098439 0x00000024 0x00000000 DxeCore.dll.__DWARF.__apple_namespac
>   0x0000000c apple-types      [0x00000000000e245d-0x00000000000eea50)* rw-  0x0009845d 0x0000c5f3 0x00000000 DxeCore.dll.__DWARF.__apple_types
>   0x0000000d apple-objc       [0x00000000000eea50-0x00000000000eea74)* rw-  0x000a4a50 0x00000024 0x00000000 DxeCore.dll.__DWARF.__apple_objc
> Sections for '/Users/user/Documents/UefiWorkspace/Build/OvmfX64/NOOPT_XCODE5/X64/MdeModulePkg/Universal/DevicePathDxe/DevicePathDxe/DEBUG/DevicePathDxe.dll' (x86_64):
> …
> (lldb) image dump symtab
> Dumping symbol table for 88 modules.
> Symtab, file = /Users/user/Documents/UefiWorkspace/Build/OvmfX64/NOOPT_XCODE5/X64/MdeModulePkg/Core/Dxe/DxeMain/DEBUG/DxeCore.dll, num_symbols = 1203:
>                Debug symbol
>                |Synthetic symbol
>                ||Externally Visible
>                |||
> Index   UserID DSX Type            File Address/Value Load Address       Size               Flags      Name
> ------- ------ --- --------------- ------------------ ------------------ ------------------ ---------- ----------------------------------
> [    0]    118 D   SourceFile      0x0000000000000000                    Sibling -> [   20] 0x00640000 /Users/user/Documents/UefiPackages/MdeModulePkg/Core/Dxe/SectionExtraction/CoreSectionExtraction.c
> [    1]    120 D   ObjectFile      0x000000005e78588d                    0x0000000000000000 0x00660001 /Users/user/Documents/UefiWorkspace/Build/OvmfX64/NOOPT_XCODE5/X64/MdeModulePkg/Core/Dxe/DxeMain/OUTPUT/DxeCore.lib(CoreSectionExtraction.obj)
> [    2]    122 D X Code            0x0000000000000240 0x000000007fe26240 0x0000000000000260 0x000f0000 CustomGuidedSectionExtract
> [    3]    126 D X Code            0x00000000000004a0 0x000000007fe264a0 0x0000000000000110 0x000f0000 InitializeSectionExtraction
> [    4]    130 D X Code            0x00000000000005b0 0x000000007fe265b0 0x0000000000000140 0x000f0000 IsValidSectionStream
> [    5]    134 D X Code            0x00000000000006f0 0x000000007fe266f0 0x0000000000000180 0x000f0000 OpenSectionStreamEx
> [    6]    138 D X Code            0x0000000000000870 0x000000007fe26870 0x0000000000000070 0x000f0000 OpenSectionStream
> [    7]    142 D X Code            0x00000000000008e0 0x000000007fe268e0 0x0000000000000100 0x000f0000 ChildIsType
> …
> [  831]   2470 D X Code            0x00000000000306b0 0x000000007fe566b0 0x00000000000000f4 0x000f0000 UefiBootServicesTableLibConstructor
> [  832]   2473 D X Data            0x0000000000047780 0x000000007fe6d780 0x0000000000000008 0x000f0000 gImageHandle
> [  833]   2474 D X Data            0x0000000000047788 0x000000007fe6d788 0x0000000000000008 0x000f0000 gST
> [  834]   2475 D X Data            0x0000000000047790 0x000000007fe6d790 0x0000000000000008 0x000f0000 gBS
> …
> 
> This is easy enough to check as %rpb is the frame pointer so it will get saved/restored on function entry/exit.
> 
> Actually it is fine. I happened to break in the interrupt routine, which was not mapped for a particular image. After I stepped a little, it soon reached the debug symbols:
> 
> (lldb) bt all
> * thread #1, stop reason = instruction step over
>   * frame #0: 0x000000007fe49f3c DxeCore.dll`CoreRestoreTpl(NewTpl=0x0000000000000008) at Tpl.c:102:12
>     frame #1: 0x000000007e518c95
>   thread #2
>     frame #0: 0x000000007fb3ff88 CpuDxe.dll`CpuSleep + 1
>     frame #1: 0x000000007fb4d719 CpuDxe.dll`ApWakeupFunction(ExchangeInfo=0x0000000000087198, ApIndex=0x0000000000000001) at MpLib.c:768:9
>     frame #2: 0x000000007fb2d151
>   thread #3
>     frame #0: 0x000000007fb3ff88 CpuDxe.dll`CpuSleep + 1
>     frame #1: 0x000000007fb4d719 CpuDxe.dll`ApWakeupFunction(ExchangeInfo=0x0000000000087198, ApIndex=0x0000000000000002) at MpLib.c:768:9
>     frame #2: 0x000000007fb2d151
>   thread #4
>     frame #0: 0x000000007fb3ff88 CpuDxe.dll`CpuSleep + 1
>     frame #1: 0x000000007fb4d719 CpuDxe.dll`ApWakeupFunction(ExchangeInfo=0x0000000000087198, ApIndex=0x0000000000000003) at MpLib.c:768:9
>     frame #2: 0x000000007fb2d151
> 
> Best wishes,
> Vitaly
> 
> [1] https://github.com/acidanthera/OpenCorePkg/blob/7684a9f/Debug/ClangDwarf.patch <https://github.com/acidanthera/OpenCorePkg/blob/7684a9f/Debug/ClangDwarf.patch>
> [2] https://bugs.llvm.org/show_bug.cgi?id=45273 <https://bugs.llvm.org/show_bug.cgi?id=45273>
> [3] https://bugs.llvm.org/show_bug.cgi?id=45277 <https://bugs.llvm.org/show_bug.cgi?id=45277>
> [4] https://github.com/acidanthera/OpenCorePkg/commit/3e5f3f3f9a8d3f197834bbfcf5029f771df89754 <https://github.com/acidanthera/OpenCorePkg/commit/3e5f3f3f9a8d3f197834bbfcf5029f771df89754>
> 
> 
> 22 марта 2020 г., в 00:06, Andrew Fish <afish@apple.com <mailto:afish@apple.com>> написал(а):
> 
> 
> 
> 
> On Mar 21, 2020, at 11:36 AM, Vitaly Cheptsov <cheptsov@ispras.ru <mailto:cheptsov@ispras.ru>> wrote:
> 
> Andrew,
> 
> Thanks once again, but unfortunately it is not that simple. Below I answered inline explaining the particular issues, which mostly seem to be specific to CLANGPDB. LLVM stack emits PDB debug files, and even though LLDB does support them to some level, it is unlikely that this will be working well enough soon. We should really stick to more or less native debug formats, ideally those that have proper open specifications, on all platforms, and for Unix that’s DWARF.
> 
> 
> Vitaly,
> 
> I understand and I use the Xcode clang and not the CLANGPDB, but I use lldb a lot I was just trying to point out what works with Xcode.
> 
> 
> 
> I am pretty sure LLVM can be taught to emit DWARF debug information even for PE files. Perhaps we can either make some option or provide a separate toolchain for this? Another way would be recovering CLANGELF as originally suggested.
> 
> 
> There was a bug recently in the  x86_64-pc-win32-macho triple and we had to add -gdwarf to force it emit dwarf. Not sure what that compiler flag would do to CLANGPDB but it is worth a try? Last flag wins for the compiler.
> 
> 
> 
> You can teach lldb about types. There is some example code here: https://github.com/tianocore/edk2/blob/master/EmulatorPkg/Unix/lldbefi.py <https://github.com/tianocore/edk2/blob/master/EmulatorPkg/Unix/lldbefi.py>
> 
> This code works just fine with LLDB and DWARF (e.g. XCODE5), though I have not yet completed these changes for my scripts for LLDB, only for GDB. However, with CLANGPDB generated files it is not functional. The reason for this is because LLDB is unaware of the underlying type, i.e. it does not know what is EFI_STATUS or UINT32. I can implement pretty-printing when LLDB knows about a typedef, but it is not possible to do this when the debug information is already gone or not parsed:
> 
> (lldb) p Status
> (unsigned long long) $1 = 0
> (lldb) p &Status
> (unsigned long long *) $2 = 0x000000007fe19ad8
> (lldb) p (EFI_STATUS)Status
> error: use of undeclared identifier 'EFI_STATUS'
> 
> Just in case I tried using exactly your code, and other stuff like source level debugging works just fine and symbolication works fine, so it should be some bug with PDB in particular.
> 
> 
> That is strange as globals usually work best? The common issue I've seen is getting the slide wrong. The EFI modules are linked at a value near zero and relocated into memory, so the slide represents that adjustment.
> 
> You can use `image dump sections` and ` image dump symtab` to see lldb's view of symbols. More info here [1].
> 
> Yes, this one is a bit complicated, once again due to PDB most likely. It knows about global symbols, but does not list them in symtab:
> 
> (lldb) image dump symtab
> Dumping symbol table for 91 modules.
> Symtab, file = GdbSyms/Bin/X64_CLANGPDB/GdbSyms.dll, num_symbols = 0
> Symtab, file = /Users/user/Documents/UefiWorkspace/Build/OvmfX64/NOOPT_CLANGPDB/X64/MdeModulePkg/Core/Dxe/DxeMain/DEBUG/DxeCore.dll, num_symbols = 0
> Symtab, file = /Users/user/Documents/UefiWorkspace/Build/OvmfX64/NOOPT_CLANGPDB/X64/MdeModulePkg/Universal/DevicePathDxe/DevicePathDxe/DEBUG/DevicePathDxe.dll, num_symbols = 0
> …
> 
> The slides are correct, but there are two nuances that collide with it.
> 
> 1. There are multiple instances of the globals with the same name (e.g. gBS), but for some reason LLDB always tries to print the globals from the first module. This happens even when I am source-level debugging, and I see a gBS symbol from another module (e.g. DxeCore) used right at the same line. With GDB the closest symbol is used, but with LLDB it is always coming from the first module. I tried checking expr help to find whether I can pass it a module explicitly, but also failed.
> 
> 
> Usually what happens with lldb is you get the global that is in scope for the current frame.
> 
> 
> 2.  To be able to get EFI types to locate the EFI_SYSTEM_TABLE_POINTER I add a dummy GdbSyms image, which is not loaded to the firmware. So basically I cannot slide what is not in the memory, and this is also my first image. I tried deleting it anyhow, but it failed for me.
> 
> 
> I've not used the fake image to get things done so I can't speak to that. I have used a fake target so I could have XIP PEIM and shadowed PEIM address available at the same time. You can't have a module loaded at 2 addresses at the same time in llldb. But you might be able to use a fake target for your fake stuff?
> 
> Just in case:
> 
>           # create a faka target to store info about symbols
>           PeiXipTarget = target.debugger.CreateTarget (None, "i386-apple-macosx", "remote-macosx", True, error)
> 
>           # make sure the gdb-remote  connection target is the active target
>           target.debugger.SetSelectedTarget (target)
> 
> 
> 
> (lldb) image dump sections
> Dumping sections for 91 modules.
> Sections for 'GdbSyms/Bin/X64_CLANGPDB/GdbSyms.dll' (x86_64):
>   SectID     Type             Load Address                             Perm File Off.  File Size  Flags      Section Name
>   ---------- ---------------- ---------------------------------------  ---- ---------- ---------- ---------- ----------------------------
>   0xffffffffffffffff container        [0x0000000000000000-0x0000000000006ec0)* ---  0x00000000 0x00000000 0x00000000 GdbSyms.dll.
>   0x00000001 code             [0x0000000000000220-0x0000000000005bd6)* ---  0x00000220 0x000059c0 0x60000020 GdbSyms.dll...text
>   0x00000002 data             [0x0000000000005be0-0x0000000000006d79)* ---  0x00005be0 0x000011a0 0x40000040 GdbSyms.dll...rdata
>   0x00000003 data             [0x0000000000006d80-0x0000000000006e30)* ---  0x00006d80 0x00000060 0xc0000040 GdbSyms.dll...data
>   0x00000004 regular          [0x0000000000006e40-0x0000000000006ea4)* ---  0x00006de0 0x00000080 0x42000040 GdbSyms.dll...reloc
> Sections for '/Users/user/Documents/UefiWorkspace/Build/OvmfX64/NOOPT_CLANGPDB/X64/MdeModulePkg/Core/Dxe/DxeMain/DEBUG/DxeCore.dll' (x86_64):
>   SectID     Type             Load Address                             Perm File Off.  File Size  Flags      Section Name
>   ---------- ---------------- ---------------------------------------  ---- ---------- ---------- ---------- ----------------------------
>   0xffffffffffffffff container        [0x0000000000000000-0x00000000000523a0)* ---  0x00000000 0x00000000 0x00000000 DxeCore.dll.
>   0x00000001 code             [0x000000007fe1b220-0x000000007fe61e34)  ---  0x00000220 0x00046c20 0x60000020 DxeCore.dll...text
>   0x00000002 data             [0x000000007fe61e40-0x000000007fe68065)  ---  0x00046e40 0x00006240 0x40000040 DxeCore.dll...rdata
>   0x00000003 data             [0x000000007fe68080-0x000000007fe6d160)  ---  0x0004d080 0x000018a0 0xc0000040 DxeCore.dll...data
>   0x00000004 regular          [0x000000007fe6d160-0x000000007fe6d398)  ---  0x0004e920 0x00000240 0x42000040 DxeCore.dll...reloc
> Sections for '/Users/user/Documents/UefiWorkspace/Build/OvmfX64/NOOPT_CLANGPDB/X64/MdeModulePkg/Universal/DevicePathDxe/DevicePathDxe/DEBUG/DevicePathDxe.dll' (x86_64):
>   SectID     Type             Load Address                             Perm File Off.  File Size  Flags      Section Name
>   ---------- ---------------- ---------------------------------------  ---- ---------- ---------- ---------- ----------------------------
>   0xffffffffffffffff container        [0x0000000000000000-0x0000000000014420)* ---  0x00000000 0x00000000 0x00000000 DevicePathDxe.dll.
>   0x00000001 code             [0x000000007f986220-0x000000007f996cc6)  ---  0x00000220 0x00010ac0 0x60000020 DevicePathDxe.dll...text
>   0x00000002 data             [0x000000007f996ce0-0x000000007f999b04)  ---  0x00010ce0 0x00002e40 0x40000040 DevicePathDxe.dll...rdata
>   0x00000003 data             [0x000000007f999b20-0x000000007f99a1a2)  ---  0x00013b20 0x00000660 0xc0000040 DevicePathDxe.dll...data
>   0x00000004 regular          [0x000000007f99a1c0-0x000000007f99a404)  ---  0x00014180 0x00000260 0x42000040 DevicePathDxe.dll…reloc
> …
> 
> So, all in all, unique global variables work, but there is no way to access duplicating variables. They either resolve to GdbSyms or just cause a crash:
> 
> (lldb) p mDebugInfoTableHeader
> (EFI_DEBUG_IMAGE_INFO_TABLE_HEADER) $0 = {
>   UpdateStatus = 2
>   TableSize = 92
>   EfiDebugImageInfoTable = 0x000000007f814018
> }
> (lldb) p gBS
> error: Couldn't materialize: couldn't get the value of variable ::gBS: read memory from 0x6df8 failed
> error: errored out in DoExecute, couldn't PrepareToExecuteJITExpression
> (lldb) p gEfiGlobalVariableGuid
> 0  libLLVM.dylib            0x000000010e52ee68 llvm::sys::PrintStackTrace(llvm::raw_ostream&) + 40
> 1  libLLVM.dylib            0x000000010e52f262 SignalHandler(int) + 188
> 2  libsystem_platform.dylib 0x00007fff6ca5642d _sigtramp + 29
> ...
> 
> 
> If you want to inspect globals I think this logic works to get you data, you would need to print it out etc.
> 
> SBValueList = lldb.target.FindGlobalVariables ("gST", 1024)
> for SBValue in SBValueList:
>    Module = SBValue.GetAddress().GetModule()
>         ModuleStr = SBValue.GetAddress().GetModule().GetFileSpec().GetFilename()
>         Start = int (SBValue.GetLocation(), 0)
>         End = Start + SBValue.GetByteSize() - 1
>         SBDeclaration = SBValue.GetDeclaration()
>         Column = SBDeclaration.GetColumn()
> 
> I wrote a command in the early days to dump out all the instances of a global.
> 
> You can also try (lldb) image lookup -Av --name gST
> 
> 
> You can tell lldb to use the older Python like this (from the Terminal.app):
> $ defaults write com.apple.dt.lldb DefaultPythonVersion 2
> 
> Thanks, that helped quite a bit, but for some reason Xcode version still crashes more for me. I attached a couple of stack traces if you feel like having a look, but once again it seems that it is all about the PDB plugin.
> 
> For the macOS API clang emits frame pointers, so you can walk the stack without symbols. You could try adding the compiler flag to emit the frame pointers.
> 
> 
> This is easy enough to check as %rpb is the frame pointer so it will get saved/restored on function entry/exit.
> 
> 
> I am pretty sure stack frames are not disabled with UEFI, as sometimes backtracing works just fine. To me it looks like debug information parsing randomly breaks in LLDB, and once it happens it forgets about other images:
> 
> (lldb) b CoreLocateHandleBuffer
> Breakpoint 2: where = DxeCore.dll`CoreLocateHandleBuffer + 31 at Locate.c:649, address = 0x000000007fe36e4f
> (lldb) c
> Process 1 resuming
> Process 1 stopped
> * thread #1, stop reason = breakpoint 2.1
>     frame #0: 0x000000007fe36e4f DxeCore.dll`CoreLocateHandleBuffer(SearchType=ByProtocol, Protocol=0x000000007f978160, SearchKey=0x0000000000000000, NumberHandles=0x000000007fe19fd8, Buffer=0x000000007fe19fc0) at Locate.c:649
>    646      EFI_STATUS          Status;
>    647      UINTN               BufferSize;
>    648
> -> 649     if (NumberHandles == NULL) {
>    650        return EFI_INVALID_PARAMETER;
>    651      }
>    652
> (lldb) bt
> * thread #1, stop reason = breakpoint 2.1
>   * frame #0: 0x000000007fe36e4f DxeCore.dll`CoreLocateHandleBuffer(SearchType=ByProtocol, Protocol=0x000000007f978160, SearchKey=0x0000000000000000, NumberHandles=0x000000007fe19fd8, Buffer=0x000000007fe19fc0) at Locate.c:649
>     frame #1: 0x000000007fe36816 DxeCore.dll`CoreLocateDevicePath(Protocol=0x000000007f978160, DevicePath=0x000000007fe1a060, Device=0x000000007fe1a068) at Locate.c:466
>     frame #2: 0x000000007f97479a SecurityStubDxe.dll
> 
> ———
> 
> (lldb) b CopyMem
> Breakpoint 3: 70 locations.
> (lldb) c
> Process 1 resuming
> Process 1 stopped
> * thread #1, stop reason = breakpoint 2.53 3.53
>     frame #0: 0x000000007e5c13b3 MnpDxe.dll`CopyMem(DestinationBuffer=0x000000007fe19b50, SourceBuffer=0x000000007e2aa470, Length=656) at CopyMemWrapper.c:47
>    44        IN UINTN       Length
>    45        )
>    46      {
> -> 47       if (Length == 0) {
>    48          return DestinationBuffer;
>    49        }
>    50        ASSERT ((Length - 1) <= (MAX_ADDRESS - (UINTN)DestinationBuffer));
> (lldb) bt
> * thread #1, stop reason = breakpoint 2.53 3.53
>   * frame #0: 0x000000007e5c13b3 MnpDxe.dll`CopyMem(DestinationBuffer=0x000000007fe19b50, SourceBuffer=0x000000007e2aa470, Length=656) at CopyMemWrapper.c:47
> (lldb) finish
> error: Could not create return address breakpoint.
> (lldb) n
> Process 1 stopped
> * thread #1, stop reason = step over
>     frame #0: 0x000000007e5c13ce MnpDxe.dll`CopyMem(DestinationBuffer=0x000000007fe19b50, SourceBuffer=0x000000007e2aa470, Length=656) at CopyMemWrapper.c:50
>    47        if (Length == 0) {
>    48          return DestinationBuffer;
>    49        }
> -> 50       ASSERT ((Length - 1) <= (MAX_ADDRESS - (UINTN)DestinationBuffer));
>    51        ASSERT ((Length - 1) <= (MAX_ADDRESS - (UINTN)SourceBuffer));
>    52
>    53        if (DestinationBuffer == SourceBuffer) {
> (lldb)
> ...
> Process 1 stopped
> * thread #1, stop reason = step over
>     frame #0: 0x000000007e5c14b4 MnpDxe.dll`CopyMem(DestinationBuffer=0x000000007fe19b50, SourceBuffer=0x000000007e2aa470, Length=656) at CopyMemWrapper.c:57
>    54          return DestinationBuffer;
>    55        }
>    56        return InternalMemCopyMem (DestinationBuffer, SourceBuffer, Length);
> -> 57     }
> (lldb)
> Process 1 stopped
> * thread #1, stop reason = step over
>     frame #0: 0x000000007e5c726e MnpDxe.dll
> ->  0x7e5c726e: mov    rax, qword ptr [rsp + 0x60]
>     0x7e5c7273: cmp    byte ptr [rax + 0x68], 0x0
>     0x7e5c7277: jne    0x7e5c7291
>     0x7e5c727d: movabs rax, -0x7fffffffffffffed
> (lldb) bt
> * thread #1, stop reason = step over
>   * frame #0: 0x000000007e5c726e MnpDxe.dll
> 
> ———
> 
> (lldb) c
> Process 1 resuming
> Process 1 stopped
> * thread #1, stop reason = signal SIGINT
>     frame #0: 0x000000007fe4d72e DxeCore.dll
> ->  0x7fe4d72e: cmp    al, 0x0
>     0x7fe4d730: je     0x7fe4d765
>     0x7fe4d736: mov    rcx, qword ptr [rsp + 0x20]
>     0x7fe4d73b: call   0x7fe4c4b0
> (lldb) bt
> * thread #1, stop reason = signal SIGINT
>   * frame #0: 0x000000007fe4d72e DxeCore.dll
> 
> On macOS the Mach-O and dSYM have a UUID (dwarfdump -u) that is indexed by Spotlight (mdfind "com_apple_xcode_dsym_uuids == *") [2]
> This should be the UUID in the debug directory entry and you can use that to lookup the symbols like this:
> 
> module = target.AddModule (None, None, uuid)
> SBError = target.SetModuleLoadAddress (module, LoadAddress + TeAdjust)
> 
> Also lldb has built in help for commands, but it is kind of terse since it is autogenerated from the C++ swig.
> (lldb) script help (lldb.target.AddModule)
> Help on method AddModule in module lldb:
> 
> AddModule(self, *args) method of lldb.SBTarget instance
>     AddModule(SBTarget self, SBModule module) -> bool
>     AddModule(SBTarget self, char const * path, char const * triple, char const * uuid) -> SBModule
>     AddModule(SBTarget self, char const * path, char const * triple, char const * uuid_cstr, char const * symfile) -> SBModule
>     AddModule(SBTarget self, SBModuleSpec module_spec) -> SBModule
> 
> 
> The minimum  you need to symbolicate a frame is uuid, LoadAddress, and PC.
> 
> [1] http://lldb.llvm.org/use/map.html <http://lldb.llvm.org/use/map.html>
> [2] http://lldb.llvm.org/use/symbols.html <http://lldb.llvm.org/use/symbols.html>
> 
> Thanks for the links again. Yes, I am using some of these, and in fact for GDB that’s pretty much what I did when I worked with XCODE5. It is very likely that when I get to complete LLDB support for XCODE5 it will work quite fine too. But I am already happy with XCODE5 here, and making it even better will only help myself, but not other people with e.g. Linux or people that want me to use the same compiler with them.
> 
> 
> Thanks for looking out for others.
> 
> Thanks,
> 
> Andrew Fish
> 
> 
> Best regards,
> Vitaly
> 
> 
> 21 марта 2020 г., в 20:13, Andrew Fish <afish@apple.com <mailto:afish@apple.com>> написал(а):
> 
> 
> 
> 
> On Mar 21, 2020, at 3:28 AM, Vitaly Cheptsov <cheptsov@ispras.ru <mailto:cheptsov@ispras.ru>> wrote:
> 
> Hello,
> 
> Andrey, thanks for the hint, it was very helpful. I rewrote the GDB scripts to work with LLDB[1] and was able to debug OVMF built with CLANGPDB. While it is still quite dirty, at the very least it works.
> 
> Unfortunately the experience was close to terrible. I may certainly do something wrong, but it is clear that PDB and LLDB do not support each other well enough. After spending several hours on playing with the tools my conclusion is that LLDB is simply not suited for UEFI PDB debugging, and we really want DWARF  as there is no other opensource debugger that supports PDB on macOS and Linux
> 
> In case somebody knows workarounds here are the issues I faced:
> 
> 1. All integer alias typedefs are discarded in favour of underlying types. This way EFI_STATUS and EFI_TPL become unsigned long long, CHAR8 becomes char, and CHAR16 becomes unsigned short. It does not look like LLDB has the original types anywhere at all, and it also does not have them registered.
> 
>     frame #0: 0x000000007fe242aa DxeCore.dll`CoreAllocatePoolPagesI(PoolType=EfiBootServicesData, NoPages=1, Granularity=4096, NeedGuard='\0') at Pool.c:322
>    319             return NULL;
>    320           }
>    321
> -> 322          Buffer = CoreAllocatePoolPages (PoolType, NoPages, Granularity, NeedGuard);
>    323    CoreReleaseMemoryLock ();
>    324
>    325           if (Buffer != NULL) {
> (lldb) p Status
> (unsigned long long) $3 = 0
> 
> Structures work more or less fine, but for simpler types like strings we are out of even potential pretty-printing.
> 
> 
> Vitaly,
> 
> You can teach lldb about types. There is some example code here: https://github.com/tianocore/edk2/blob/master/EmulatorPkg/Unix/lldbefi.py <https://github.com/tianocore/edk2/blob/master/EmulatorPkg/Unix/lldbefi.py>
> 
> 
> 2. Global variables are not accessible. I am not sure what happens, but they either seem to not relocate or conflict with the other names:
> 
> (lldb) p gST
> error: Couldn't materialize: couldn't get the value of variable ::gST: read memory from 0x6e18 failed
> error: errored out in DoExecute, couldn't PrepareToExecuteJITExpression
> (lldb) p &gST
> error: Couldn't materialize: couldn't get the value of variable ::gST: read memory from 0x6e18 failed
> error: errored out in DoExecute, couldn't PrepareToExecuteJITExpression
> 
> 
> That is strange as globals usually work best? The common issue I've seen is getting the slide wrong. The EFI modules are linked at a value near zero and relocated into memory, so the slide represents that adjustment.
> 
> You can use `image dump sections` and ` image dump symtab` to see lldb's view of symbols. More info here [1].
> 
> 
> 3. Quite a number of crashes.
> 
> In most cases autocompletion by tab press causes a crash. E.g.
> 
> b I<TAB>
> 
> So will do printing of a GUID, e.g. p gEfiGlobalVariableGuid.
> 
> This may have to do with Python compatibility as Xcode 11 LLDB that uses Python 3 generally crashes more often than MacPorts LLDB 9.0. Surprisingly structures work more or less fine.
> 
> 
> You can tell lldb to use the older Python like this (from the Terminal.app):
> $ defaults write com.apple.dt.lldb DefaultPythonVersion 2
> 
> 
> 4. Ctrl+C does not produce a valid backtrace. When I break with a breakpoint, I see a proper stacktrace with more than one entry, with function prototypes and values. When I break with Ctrl+C I only see some weird backtrace with most of the entries missing regardless of frame position:
> 
> (lldb) bt
> * thread #1, stop reason = signal SIGTRAP
>   * frame #0: 0x000000007fe4c5f3 DxeCore.dll
> 
> Probably more and all the unintuitive stuff like the lack of more functional TUI, but it is hard to remember all the trials.
> 
> 
> For the macOS API clang emits frame pointers, so you can walk the stack without symbols. You could try adding the compiler flag to emit the frame pointers.
> 
> 
> [1] https://github.com/acidanthera/OpenCorePkg/blob/master/Debug/Scripts/lldb_uefi.py <https://github.com/acidanthera/OpenCorePkg/blob/master/Debug/Scripts/lldb_uefi.py>
> 
> 
> On macOS the Mach-O and dSYM have a UUID (dwarfdump -u) that is indexed by Spotlight (mdfind "com_apple_xcode_dsym_uuids == *") [2]
> This should be the UUID in the debug directory entry and you can use that to lookup the symbols like this:
> 
> module = target.AddModule (None, None, uuid)
> SBError = target.SetModuleLoadAddress (module, LoadAddress + TeAdjust)
> 
> Also lldb has built in help for commands, but it is kind of terse since it is autogenerated from the C++ swig.
> (lldb) script help (lldb.target.AddModule)
> Help on method AddModule in module lldb:
> 
> AddModule(self, *args) method of lldb.SBTarget instance
>     AddModule(SBTarget self, SBModule module) -> bool
>     AddModule(SBTarget self, char const * path, char const * triple, char const * uuid) -> SBModule
>     AddModule(SBTarget self, char const * path, char const * triple, char const * uuid_cstr, char const * symfile) -> SBModule
>     AddModule(SBTarget self, SBModuleSpec module_spec) -> SBModule
> 
> 
> The minimum  you need to symbolicate a frame is uuid, LoadAddress, and PC.
> 
> [1] http://lldb.llvm.org/use/map.html <http://lldb.llvm.org/use/map.html>
> [2] http://lldb.llvm.org/use/symbols.html <http://lldb.llvm.org/use/symbols.html>
> 
> Thanks,
> 
> Andrew Fish
> 
> 
> 
> Best wishes,
> Vitaly
> 
> 20 марта 2020 г., в 22:14, Andrew Fish <afish@apple.com <mailto:afish@apple.com>> написал(а):
> 
> 
> 
> 
> On Mar 20, 2020, at 8:13 AM, Vitaly Cheptsov <cheptsov@ispras.ru <mailto:cheptsov@ispras.ru>> wrote:
> 
> Hello,
> 
> We noticed that the original bugzilla, which intended to add new LLVM toolchain support[1], also wanted to bring ELF format support with DWARF debugging information. For some reason this did not make its way into EDK II, and we are currently wondering, how can one debug binaries built with LLVM 9.0.
> 
> 
> For macOS and XCODE5 toolchain we use GDB scripts based on Andrei Warkentin’s work, which allow us to integrate with QEMU and VMware[2]. It is likely that they should work with little to no work on Linux with CLANG38/GCC5 with GDB once again. However, CLANGPDB apparently is using PDB debugging information, which I believe is not handled with GDB.
> 
> Could you please provide the details on the matter and let us know about the recommended route?
> — Is dropping CLANGELF just a temporary measure and it should be resubmitted again?
> — Should LLDB, which seems to be aware of PDB, be used instead of GDB, when building with CLANGPDB? If so, did anybody try that?
> 
> 
> 
> Vitaly,
> 
> I've not tried the CLANGPDB path, but if you want to connect lldb to QEMU you need to set  plugin.process.gdb-remote.target-definition-file [1] to [2].
> 
> [1]  lldb -o "settings set plugin.process.gdb-remote.target-definition-file x86_64_target_definition.py" -o "gdb-remote 9000"
> [2] https://github.com/llvm-mirror/lldb/blob/master/examples/python/x86_64_target_definition.py <https://github.com/llvm-mirror/lldb/blob/master/examples/python/x86_64_target_definition.py>
> 
> Thanks,
> 
> Andrew Fish
> 
> 
> Thanks!
> 
> 
> Best regards,
> Vitaly
> 
> [1] https://bugzilla.tianocore.org/show_bug.cgi?id=1603 <https://bugzilla.tianocore.org/show_bug.cgi?id=1603>
> [2] https://github.com/acidanthera/OpenCorePkg/blob/master/Debug/Scripts/gdb_uefi.py <https://github.com/acidanthera/OpenCorePkg/blob/master/Debug/Scripts/gdb_uefi.py>
> 
> 
> 
> <crashes.txt>
> 
> 


[-- Attachment #1.2: Type: text/html, Size: 152213 bytes --]

[-- Attachment #2: Message signed with OpenPGP --]
[-- Type: application/pgp-signature, Size: 833 bytes --]