From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail.ispras.ru (mail.ispras.ru [83.149.199.45]) by mx.groups.io with SMTP id smtpd.web10.5268.1584786541672772038 for ; Sat, 21 Mar 2020 03:29:02 -0700 Authentication-Results: mx.groups.io; dkim=missing; spf=pass (domain: ispras.ru, ip: 83.149.199.45, mailfrom: cheptsov@ispras.ru) Received: from [127.0.0.1] (unknown [77.232.9.83]) by mail.ispras.ru (Postfix) with ESMTPSA id 724C9650; Sat, 21 Mar 2020 13:28:59 +0300 (MSK) From: "Vitaly Cheptsov" Message-Id: Mime-Version: 1.0 (Mac OS X Mail 13.0 \(3608.60.0.2.5\)) Subject: Re: [edk2-devel] CLANGPDB binary debugging Date: Sat, 21 Mar 2020 13:28:57 +0300 In-Reply-To: <7E18AD8F-9A44-45FE-A8C8-CE06A6328930@apple.com> Cc: devel@edk2.groups.io, "Gao, Liming" , =?utf-8?Q?Marvin_H=C3=A4user?= To: Andrew Fish References: <9804C565-0C9E-4778-92A7-06EA6AD8D694@ispras.ru> <7E18AD8F-9A44-45FE-A8C8-CE06A6328930@apple.com> X-Mailer: Apple Mail (2.3608.60.0.2.5) X-Groupsio-MsgNum: 56065 Content-Type: multipart/signed; boundary="Apple-Mail=_5AF7E233-F6CF-47F6-B06D-7DE6A308E026"; protocol="application/pgp-signature"; micalg=pgp-sha512 --Apple-Mail=_5AF7E233-F6CF-47F6-B06D-7DE6A308E026 Content-Type: multipart/alternative; boundary="Apple-Mail=_F6945D30-1C87-4A7C-9B20-D6B37465B29B" --Apple-Mail=_F6945D30-1C87-4A7C-9B20-D6B37465B29B Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=utf-8 Hello, Andrey, thanks for the hint, it was very helpful. I rewrote the GDB script= s to work with LLDB[1] and was able to debug OVMF built with CLANGPDB. Whil= e it is still quite dirty, at the very least it works. Unfortunately the experience was close to terrible. I may certainly do som= ething wrong, but it is clear that PDB and LLDB do not support each other w= ell enough. After spending several hours on playing with the tools my concl= usion is that LLDB is simply not suited for UEFI PDB debugging, and we real= ly want DWARF as there is no other opensource debugger that supports PDB o= n macOS and Linux In case somebody knows workarounds here are the issues I faced: 1. All integer alias typedefs are discarded in favour of underlying types.= This way EFI_STATUS and EFI_TPL become unsigned long long, CHAR8 becomes c= har, and CHAR16 becomes unsigned short. It does not look like LLDB has the = original types anywhere at all, and it also does not have them registered. frame #0: 0x000000007fe242aa DxeCore.dll`CoreAllocatePoolPagesI(PoolTy= pe=3DEfiBootServicesData, NoPages=3D1, Granularity=3D4096, NeedGuard=3D'\0'= ) at Pool.c:322 319 =09 return NULL; 320 =09 } 321 -> 322 =09 Buffer =3D CoreAllocatePoolPages (PoolType, NoPages, Granulari= ty, NeedGuard); 323 =09 CoreReleaseMemoryLock (); 324 325 =09 if (Buffer !=3D NULL) { (lldb) p Status (unsigned long long) $3 =3D 0 Structures work more or less fine, but for simpler types like strings we a= re out of even potential pretty-printing. 2. Global variables are not accessible. I am not sure what happens, but th= ey either seem to not relocate or conflict with the other names: (lldb) p gST error: Couldn't materialize: couldn't get the value of variable ::gST: rea= d memory from 0x6e18 failed error: errored out in DoExecute, couldn't PrepareToExecuteJITExpression (lldb) p &gST error: Couldn't materialize: couldn't get the value of variable ::gST: rea= d memory from 0x6e18 failed error: errored out in DoExecute, couldn't PrepareToExecuteJITExpression 3. Quite a number of crashes. In most cases autocompletion by tab press causes a crash. E.g. b I So will do printing of a GUID, e.g. p gEfiGlobalVariableGuid. This may have to do with Python compatibility as Xcode 11 LLDB that uses P= ython 3 generally crashes more often than MacPorts LLDB 9.0. Surprisingly s= tructures work more or less fine. 4. Ctrl+C does not produce a valid backtrace. When I break with a breakpoi= nt, I see a proper stacktrace with more than one entry, with function proto= types and values. When I break with Ctrl+C I only see some weird backtrace = with most of the entries missing regardless of frame position: (lldb) bt * thread #1, stop reason =3D signal SIGTRAP * frame #0: 0x000000007fe4c5f3 DxeCore.dll Probably more and all the unintuitive stuff like the lack of more function= al TUI, but it is hard to remember all the trials. [1] https://github.com/acidanthera/OpenCorePkg/blob/master/Debug/Scripts/l= ldb_uefi.py Best wishes, Vitaly > 20 =D0=BC=D0=B0=D1=80=D1=82=D0=B0 2020 =D0=B3., =D0=B2 22:14, Andrew Fis= h =D0=BD=D0=B0=D0=BF=D0=B8=D1=81=D0=B0=D0=BB(=D0=B0): >=20 >=20 >=20 >> On Mar 20, 2020, at 8:13 AM, Vitaly Cheptsov > wrote: >>=20 >> Hello, >>=20 >> We noticed that the original bugzilla, which intended to add new LLVM t= oolchain support[1], also wanted to bring ELF format support with DWARF deb= ugging information. For some reason this did not make its way into EDK II, = and we are currently wondering, how can one debug binaries built with LLVM = 9.0. >>=20 >> For macOS and XCODE5 toolchain we use GDB scripts based on Andrei Warke= ntin=E2=80=99s work, which allow us to integrate with QEMU and VMware[2]. I= t is likely that they should work with little to no work on Linux with CLAN= G38/GCC5 with GDB once again. However, CLANGPDB apparently is using PDB deb= ugging information, which I believe is not handled with GDB. >>=20 >> Could you please provide the details on the matter and let us know abou= t the recommended route? >> =E2=80=94 Is dropping CLANGELF just a temporary measure and it should b= e resubmitted again? >> =E2=80=94 Should LLDB, which seems to be aware of PDB, be used instead = of GDB, when building with CLANGPDB? If so, did anybody try that? >>=20 >=20 > Vitaly, >=20 > I've not tried the CLANGPDB path, but if you want to connect lldb to QEM= U you need to set plugin.process.gdb-remote.target-definition-file [1] to = [2]. >=20 > [1] lldb -o "settings set plugin.process.gdb-remote.target-definition-f= ile x86_64_target_definition.py" -o "gdb-remote 9000" > [2] https://github.com/llvm-mirror/lldb/blob/master/examples/python/x86_= 64_target_definition.py >=20 > Thanks, >=20 > Andrew Fish >=20 >> Thanks! >>=20 >> Best regards, >> Vitaly >>=20 >> [1] https://bugzilla.tianocore.org/show_bug.cgi?id=3D1603 >> [2] https://github.com/acidanthera/OpenCorePkg/blob/master/Debug/Script= s/gdb_uefi.py >>=20 >>=20 --Apple-Mail=_F6945D30-1C87-4A7C-9B20-D6B37465B29B Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=utf-8 Hello,
Andrey, thanks for the hint, it was very = helpful. I rewrote the GDB scripts to work with LLDB[1] and was able to deb= ug OVMF built with CLANGPDB. While it is still quite dirty, at the very lea= st it works.

Unfo= rtunately the experience was close to terrible. I may certainly do somethin= g wrong, but it is clear that PDB and LLDB do not support each other well e= nough. After spending several hours on playing with the tools my conclusion= is that LLDB is simply not suited for UEFI PDB debugging, and we really wa= nt DWARF  as there is no other opensource debugger that supports PDB o= n macOS and Linux

In case somebody knows workarounds here are the issues I faced:

1. All integer alias typede= fs are discarded in favour of underlying types. This way EFI_STATUS and EFI= _TPL become unsigned long long, CHAR8 becomes char, and CHAR16 becomes unsi= gned short. It does not look like LLDB has the original types anywhere at a= ll, and it also does not have them registered.

    frame #0: 0x000000007= fe242aa DxeCore.dll`CoreAllocatePoolPagesI(PoolType=3DEfiBootServicesD= ata, NoPages=3D1, Granularity=3D4096, NeedGuard=3D'\0') at Pool.c:322<= br class=3D"">   319      return NULL;
   320    }
   321 
->= 322  =   Buffer =3D CoreAllocatePoolPages (PoolType, NoPages, Granularit= y, NeedGuard);
   323    CoreReleaseMemoryLock = ();
   324 
   325    if&nbs= p;(Buffer !=3D NULL) {
(lldb) p Status
(un= signed long long) $3 =3D 0

Structures work more or less fine, but for simpler types like str= ings we are out of even potential pretty-printing.

2. Global variables are not acc= essible. I am not sure what happens, but they either seem to not relocate o= r conflict with the other names:

=
(lldb) p gST
error: Couldn't materialize= : couldn't get the value of variable ::gST: read memory from 0x6e18 failed<= br class=3D"">error: errored out in DoExecute, couldn't PrepareToExecuteJIT= Expression
(lldb) p &gST
error: Couldn= 't materialize: couldn't get the value of variable ::gST: read memory from = 0x6e18 failed
error: errored out in DoExecute, couldn't Prepa= reToExecuteJITExpression

3. Quite a number of crashes.

<= /div>
In most cases autocompletion by tab press causes a cra= sh. E.g.

b I<T= AB>

So will do= printing of a GUID, e.g. p gEfiGlobalVariableGuid.

This may have to do with Python compatibility as Xcode 11 LLDB that = uses Python 3 generally crashes more often than MacPorts LLDB 9.0. Sur= prisingly structures work more or less fine.

4. Ctrl+C does not produce a valid = backtrace. When I break with a breakpoint, I see a proper stacktrace with m= ore than one entry, with function prototypes and values. When I break with = Ctrl+C I only see some weird backtrace with most of the entries missing reg= ardless of frame position:

(lldb) bt* thread #1, stop reason =3D signal SIGTRAP
=   * frame #0: 0x000000007fe4c5f3 DxeCore.dll

<= /font>
Probab= ly more and all the unintuitive stuff like the lack of more functional TUI,= but it is hard to remember all the trials.
[1] ht= tps://github.com/acidanthera/OpenCorePkg/blob/master/Debug/Scripts/lldb_uef= i.py

Best wis= hes,
Vitaly

<= blockquote type=3D"cite" class=3D"">
20 =D0=BC=D0=B0=D1=80= =D1=82=D0=B0 2020 =D0=B3., =D0=B2 22:14, Andrew Fish <afish@apple.com> =D0=BD=D0=B0=D0=BF=D0= =B8=D1=81=D0=B0=D0=BB(=D0=B0):



On Mar 20, 2020, = at 8:13 AM, Vitaly Cheptsov <cheptsov@ispras.ru> wrote:

Hello,

We noticed that the original bu= gzilla, which intended to add new LLVM toolchain support[1], also wanted to= bring ELF format support with DWARF debugging information. For some reason this did not ma= ke its way into EDK II, and we are currently wondering, how can one debug b= inaries built with LLVM 9.0.

For macOS and = XCODE5 toolchain we use GDB scripts based on A= ndrei Warkentin=E2=80=99s work, which allow us to integrate with QEMU and V= Mware[2]. It is likely that they should work with little to no work on Linu= x with CLANG38/GCC5 with GDB once again. However, CLANGPDB apparently is us= ing PDB debugging information, which I believe is not handled with GDB.

Could you ple= ase provide the details on the matter and let us know about the recommended= route?
=E2=80=94 Is dropping CLANGELF just a temporar= y measure and it should be resubmitted again?
=E2=80= =94 Should LLDB= , which seems to be aware of PDB, be used instead of GDB, when building wit= h CLANGPDB? If so, did anybody try that?


Vitaly,

I've not tried the CLANGPDB path, but if you want to conne= ct lldb to QEMU you need to set  plugin.process.gdb-remote.target-defi= nition-file [1] to [2]. 
=
[1]  = ;lldb -o "settings set plugin.process.gdb-remote.target-definition-file x86= _64_target_definition.py" -o "gdb-remote 9000"
[2] https://github.com/llvm-mirror/lldb/blo= b/master/examples/python/x86_64_target_definition.py

Thanks,

Andrew Fish

Thanks!

Best regards,
Vitaly
<= div class=3D"">
[1] https://bugz= illa.tianocore.org/show_bug.cgi?id=3D1603
[2] = ;https://github.com/acidanthera/OpenCorePkg/bl= ob/master/Debug/Scripts/gdb_uefi.py


--Apple-Mail=_F6945D30-1C87-4A7C-9B20-D6B37465B29B-- --Apple-Mail=_5AF7E233-F6CF-47F6-B06D-7DE6A308E026 Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename=signature.asc Content-Type: application/pgp-signature; name=signature.asc Content-Description: Message signed with OpenPGP -----BEGIN PGP SIGNATURE----- iQIzBAEBCgAdFiEEsLABAI5Y5VbvBdmpL8K2O86Eyz4FAl517GkACgkQL8K2O86E yz7fWhAAwTpjTEQ3YbDGCuZp4u8nY7IzOQa7XxFU+QgcIr3EhjDQ6LfvYemEOqwH FB12YldCNGk1OPp8xbomcsrtKkY8YzVDLShLxBVOVaus+amMbuW3I3VBd52vQ69q RvVY0+8SxaAaosYDBS5dx5erOlAHOfrMDBiSWjAbFZOYAi1hesGVdl7Dkz86YJrK 0vBbdmOxXKobjFWTIy7bDPlH6m7KomJq9GzYjIbfrPS5c6VXOanYm3Zl8ELgOUAO EebAzTwSx+yGqIfhxD2nLX3NchS8ICQdccfLRH95bl9f+nSrUoQ4x9i40v37ulNz zKySZP+NvmfcAim0PGBnolRUV7OdXchnNl2HCFQRSRKVunT4CejwKU9vb7eACh3z 1iRkcwpKZpuXnpjnaVtJ6Z93lQnRCqWf9IF8O9e0/OvDf3Mkv7z+5YlUI9A1sPWH 1HUQ0R0uOV99lp8iecxl/6hFxKXSmWU0TN0MrtTXPGy+WiOu3wmzyL77JqktzorT R826Y8OwiR5zT02hGTuhmqMmjLTjEbBors6YbyAdJZTtbE/IHOYIGVdPEJXZqGLI MExlPZaDnnvAhRXFzQZAEfRKuJfZnrBEKKpcA8Dfq3d699vj+Xptixud/fTOEtfV 5HdJFMQ7dnwa0f6cEKWOvWcYhPqVwgvbbXgP4Aa0t66wue6DFBo= =GP2/ -----END PGP SIGNATURE----- --Apple-Mail=_5AF7E233-F6CF-47F6-B06D-7DE6A308E026--