Source code debugging of OVMF

Source code debugging of OVMF

[Johannes Swoboda]

Hello everyone,

I'm doing a research project on efi security. I would like to do source 
code debugging of OVMF. I want to step through the OVMF source code, set 
break points, et cetera, preferably with gdb. I want to debug the 
overall boot process. Is that possible?

I understand I can start qemu with the options -s -S. This gives me a 
virtual machine that awaits connection of a gdb debugger and further 
instructions.

After connecting with gdb, I can instruct the machine to continue 
execution. Unsurprisingly, there is nothing else I can do, because gdb 
lacks the relevant symbols.
I tried to read in the OVMF.fd file, as i would do with a binary that I 
would debug, but gdb can't handle the file.
It seems to be possible to load an efi app with the file command, but 
not this one.

I found one other person trying to achieve the same five years ago. [3] 
suggests that something like this
> (gdb) add-symbol-file ../edk2/Build/OvmfX64/DEBUG_GCC5/X64/Shell.debug
> The address where ../edk2/Build/OvmfX64/DEBUG_GCC5/X64/Shell.debug has 
> been loaded is missing
might be possible. However, it appears to me this is may contain 
information regarding Shell.efi, an efi-app; but I don't want to debug 
an app. I want to debug the overall boot process.

I'm quoting some terminal output to clarify what I'm trying to achieve:
[johannes@johannes-laptop OVMF_efi_hello_world]$ gdb
[...]
(gdb) file OVMF.fd
"/home/johannes/18S/bakk/uefi_virtual/OVMF_efi_hello_world/OVMF.fd": not 
in executable format: File format not recognized
(gdb) target remote localhost:1234
Remote debugging using localhost:1234
warning: No executable has been specified and target does not support
determining executable automatically.  Try using the "file" command.
0x000000000000fff0 in ?? ()
(gdb) step
Cannot find bounds of current function
(gdb) list
No symbol table is loaded.  Use the "file" command.
(gdb) continue
Continuing.
[ovmf loads in qemu window]

I can redirect ovmf debug messages into a text file[1].
As far as I understand, there is a way to do source code level debugging 
*with* ovmf, to debug efi-apps that are e.g. run via the efi shell[2], 
with the help of SourceLevelDebugPkg[4], but this package cannot be used 
to source level debug the overall boot process, right?

Is it possible to do source level debugging of ovmf?
Is it possible to step through ovmf, one instruction or function call at 
a time?
How are you ovmf developer debugging it?

If you could point me to the right direction, that would be great.

Kind regards,
Johannes

[1]: I managed to do so via the qemu options -global 
isa-debugcon.iobase=0x402 -debugcon file:qemu.ovmf.log" as demonstrated 
here
https://www.linux-kvm.org/downloads/lersek/ovmf-whitepaper-c770f8c.txt
[2]:https://github.com/tianocore/tianocore.github.io/wiki/OVMF-FAQ#how-do-i-enable-source-level-debugging-with-ovmf
[3]: 
http://edk2-devel.narkive.com/LRWe2mSQ/using-gdb-on-ovmf-with-symbols
[4]: 
https://github.com/tianocore/tianocore.github.io/wiki/SourceLevelDebugPkg

Re: Source code debugging of OVMF

[Richardson, Brian]

This page was recently added to the TianoCore wiki. Please let me know if this is useful info:
https://github.com/tianocore/tianocore.github.io/wiki/How-to-debug-OVMF-with-QEMU-using-GDB 

Thanks ... br
---
Brian Richardson, Senior Technical Marketing Engineer, Intel Software
brian.richardson@intel.com -- @intel_brian (Twitter & WeChat)
https://software.intel.com/en-us/meet-the-developers/evangelists/team/brian-richardson 

-----Original Message-----
From: edk2-devel [mailto:edk2-devel-bounces@lists.01.org] On Behalf Of Johannes Swoboda
Sent: Monday, April 16, 2018 7:09 AM
To: edk2-devel@lists.01.org
Cc: clemens.hlauschek@inso.tuwien.ac.at
Subject: [edk2] Source code debugging of OVMF

Hello everyone,

I'm doing a research project on efi security. I would like to do source code debugging of OVMF. I want to step through the OVMF source code, set break points, et cetera, preferably with gdb. I want to debug the overall boot process. Is that possible?

I understand I can start qemu with the options -s -S. This gives me a virtual machine that awaits connection of a gdb debugger and further instructions.

After connecting with gdb, I can instruct the machine to continue execution. Unsurprisingly, there is nothing else I can do, because gdb lacks the relevant symbols.
I tried to read in the OVMF.fd file, as i would do with a binary that I would debug, but gdb can't handle the file.
It seems to be possible to load an efi app with the file command, but not this one.

I found one other person trying to achieve the same five years ago. [3] suggests that something like this
> (gdb) add-symbol-file ../edk2/Build/OvmfX64/DEBUG_GCC5/X64/Shell.debug
> The address where ../edk2/Build/OvmfX64/DEBUG_GCC5/X64/Shell.debug has 
> been loaded is missing
might be possible. However, it appears to me this is may contain information regarding Shell.efi, an efi-app; but I don't want to debug an app. I want to debug the overall boot process.

I'm quoting some terminal output to clarify what I'm trying to achieve:
[johannes@johannes-laptop OVMF_efi_hello_world]$ gdb [...]
(gdb) file OVMF.fd
"/home/johannes/18S/bakk/uefi_virtual/OVMF_efi_hello_world/OVMF.fd": not in executable format: File format not recognized
(gdb) target remote localhost:1234
Remote debugging using localhost:1234
warning: No executable has been specified and target does not support determining executable automatically.  Try using the "file" command.
0x000000000000fff0 in ?? ()
(gdb) step
Cannot find bounds of current function
(gdb) list
No symbol table is loaded.  Use the "file" command.
(gdb) continue
Continuing.
[ovmf loads in qemu window]

I can redirect ovmf debug messages into a text file[1].
As far as I understand, there is a way to do source code level debugging
*with* ovmf, to debug efi-apps that are e.g. run via the efi shell[2], with the help of SourceLevelDebugPkg[4], but this package cannot be used to source level debug the overall boot process, right?

Is it possible to do source level debugging of ovmf?
Is it possible to step through ovmf, one instruction or function call at a time?
How are you ovmf developer debugging it?

If you could point me to the right direction, that would be great.

Kind regards,
Johannes

[1]: I managed to do so via the qemu options -global
isa-debugcon.iobase=0x402 -debugcon file:qemu.ovmf.log" as demonstrated here https://www.linux-kvm.org/downloads/lersek/ovmf-whitepaper-c770f8c.txt
[2]:https://github.com/tianocore/tianocore.github.io/wiki/OVMF-FAQ#how-do-i-enable-source-level-debugging-with-ovmf
[3]: 
http://edk2-devel.narkive.com/LRWe2mSQ/using-gdb-on-ovmf-with-symbols
[4]: 
https://github.com/tianocore/tianocore.github.io/wiki/SourceLevelDebugPkg
_______________________________________________
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel

Re: Source code debugging of OVMF

[Blibbet]

On 04/16/2018 08:26 AM, Richardson, Brian wrote:
>
https://github.com/tianocore/tianocore.github.io/wiki/How-to-debug-OVMF-with-QEMU-using-GDB


Also useful for this topic:

https://github.com/tianocore/tianocore.github.io/wiki/Testing-SMM-with-QEMU,-KVM-and-libvirt

HTH,
Lee

Re: Source code debugging of OVMF

[Johannes Swoboda]

Hello Brian,

Unfortunately, if I understand it correctly, this page explains how to 
debug efi-apps (where gdb can simply load the efi-file to load the 
symbols), but this isn't applicable for the ovmf.fd-file itself; gdb 
refuses to open it.

However, this will be of help when I'll look at an efi app.

Thank you and kind regards, Johannes

On 2018-04-16 17:26, Richardson, Brian wrote:
> This page was recently added to the TianoCore wiki. Please let me know
> if this is useful info:
> https://github.com/tianocore/tianocore.github.io/wiki/How-to-debug-OVMF-with-QEMU-using-GDB
> 
> Thanks ... br
> ---
> Brian Richardson, Senior Technical Marketing Engineer, Intel Software
> brian.richardson@intel.com -- @intel_brian (Twitter & WeChat)
> https://software.intel.com/en-us/meet-the-developers/evangelists/team/brian-richardson
> 
> -----Original Message-----
> From: edk2-devel [mailto:edk2-devel-bounces@lists.01.org] On Behalf Of
> Johannes Swoboda
> Sent: Monday, April 16, 2018 7:09 AM
> To: edk2-devel@lists.01.org
> Cc: clemens.hlauschek@inso.tuwien.ac.at
> Subject: [edk2] Source code debugging of OVMF
> 
> Hello everyone,
> 
> I'm doing a research project on efi security. I would like to do
> source code debugging of OVMF. I want to step through the OVMF source
> code, set break points, et cetera, preferably with gdb. I want to
> debug the overall boot process. Is that possible?
> 
> I understand I can start qemu with the options -s -S. This gives me a
> virtual machine that awaits connection of a gdb debugger and further
> instructions.
> 
> After connecting with gdb, I can instruct the machine to continue
> execution. Unsurprisingly, there is nothing else I can do, because gdb
> lacks the relevant symbols.
> I tried to read in the OVMF.fd file, as i would do with a binary that
> I would debug, but gdb can't handle the file.
> It seems to be possible to load an efi app with the file command, but
> not this one.
> 
> I found one other person trying to achieve the same five years ago.
> [3] suggests that something like this
>> (gdb) add-symbol-file ../edk2/Build/OvmfX64/DEBUG_GCC5/X64/Shell.debug
>> The address where ../edk2/Build/OvmfX64/DEBUG_GCC5/X64/Shell.debug has
>> been loaded is missing
> might be possible. However, it appears to me this is may contain
> information regarding Shell.efi, an efi-app; but I don't want to debug
> an app. I want to debug the overall boot process.
> 
> I'm quoting some terminal output to clarify what I'm trying to achieve:
> [johannes@johannes-laptop OVMF_efi_hello_world]$ gdb [...]
> (gdb) file OVMF.fd
> "/home/johannes/18S/bakk/uefi_virtual/OVMF_efi_hello_world/OVMF.fd":
> not in executable format: File format not recognized
> (gdb) target remote localhost:1234
> Remote debugging using localhost:1234
> warning: No executable has been specified and target does not support
> determining executable automatically.  Try using the "file" command.
> 0x000000000000fff0 in ?? ()
> (gdb) step
> Cannot find bounds of current function
> (gdb) list
> No symbol table is loaded.  Use the "file" command.
> (gdb) continue
> Continuing.
> [ovmf loads in qemu window]
> 
> I can redirect ovmf debug messages into a text file[1].
> As far as I understand, there is a way to do source code level 
> debugging
> *with* ovmf, to debug efi-apps that are e.g. run via the efi shell[2],
> with the help of SourceLevelDebugPkg[4], but this package cannot be
> used to source level debug the overall boot process, right?
> 
> Is it possible to do source level debugging of ovmf?
> Is it possible to step through ovmf, one instruction or function call 
> at a time?
> How are you ovmf developer debugging it?
> 
> If you could point me to the right direction, that would be great.
> 
> Kind regards,
> Johannes
> 
> [1]: I managed to do so via the qemu options -global
> isa-debugcon.iobase=0x402 -debugcon file:qemu.ovmf.log" as
> demonstrated here
> https://www.linux-kvm.org/downloads/lersek/ovmf-whitepaper-c770f8c.txt
> [2]:https://github.com/tianocore/tianocore.github.io/wiki/OVMF-FAQ#how-do-i-enable-source-level-debugging-with-ovmf
> [3]:
> http://edk2-devel.narkive.com/LRWe2mSQ/using-gdb-on-ovmf-with-symbols
> [4]:
> https://github.com/tianocore/tianocore.github.io/wiki/SourceLevelDebugPkg

Re: Source code debugging of OVMF

[Laszlo Ersek]

Hi Johannes,

On 04/16/18 13:09, Johannes Swoboda wrote:

> How are you ovmf developer debugging it?

In general I add DEBUG statements, grep the tree for protocol / PPI
GUIDs, and use an editor with good ctags support.

Here's another thread that you might find useful:

http://edk2-devel.narkive.com/6BRVus92/qestion-about-how-to-debug-ovmf-on-qemu

Occasionally I do use gdb with QEMU, but the solution I use is not
suitable for debugging modules in the SEC and PEI phases, only in DXE. I
have some terribly rough patches in my local tree that are based on
<https://github.com/andreiw/andreiw-wip/tree/master/uefi/DebugPkg>.

I don't recall having any luck with SOURCE_DEBUG_ENABLE and the UDK
debugger
<https://firmware.intel.com/develop/intel-uefi-tools-and-utilities/intel-uefi-development-kit-debugger-tool>,
but admittedly it's been a while (= years?) since I last tried to
connect debugger VM with debugge VM over a virtual serial port. Others
have more recently confirmed it works for them. I think Mike uses it
successfully, from a Windows debugger machine maybe?

Thanks,
Laszlo

Re: Source code debugging of OVMF

[Palmer, Thomas]

I use Andrew WIP's stuff often as well.  Just put the GdbSyms.inf in your DSC (not FDF) and build.  My .gdbinit script looks like this:

set architecture i386:x86-64:intel
target remote localhost:1234
source DebugPkg/Scripts/gdb_uefi.py
reload-uefi -o Build/OvmfX64/DEBUG_GCC5/X64/DebugPkg/GdbSyms/GdbSyms/DEBUG/GdbSyms.dll


Andrew, I owe you a couple of beers if we ever meet.



Regards,

Thomas Palmer

"I have only made this letter longer because I have not had the time to make it shorter" - Blaise Pascal


-----Original Message-----
From: edk2-devel [mailto:edk2-devel-bounces@lists.01.org] On Behalf Of Laszlo Ersek
Sent: Monday, April 16, 2018 11:14 AM
To: Johannes Swoboda <johannes.swoboda@student.tuwien.ac.at>
Cc: Michael Kinney <michael.d.kinney@intel.com>; edk2-devel@lists.01.org; clemens.hlauschek@inso.tuwien.ac.at
Subject: Re: [edk2] Source code debugging of OVMF

Hi Johannes,

On 04/16/18 13:09, Johannes Swoboda wrote:

> How are you ovmf developer debugging it?

In general I add DEBUG statements, grep the tree for protocol / PPI GUIDs, and use an editor with good ctags support.

Here's another thread that you might find useful:

http://edk2-devel.narkive.com/6BRVus92/qestion-about-how-to-debug-ovmf-on-qemu

Occasionally I do use gdb with QEMU, but the solution I use is not suitable for debugging modules in the SEC and PEI phases, only in DXE. I have some terribly rough patches in my local tree that are based on <https://github.com/andreiw/andreiw-wip/tree/master/uefi/DebugPkg>.

I don't recall having any luck with SOURCE_DEBUG_ENABLE and the UDK debugger <https://firmware.intel.com/develop/intel-uefi-tools-and-utilities/intel-uefi-development-kit-debugger-tool>,
but admittedly it's been a while (= years?) since I last tried to connect debugger VM with debugge VM over a virtual serial port. Others have more recently confirmed it works for them. I think Mike uses it successfully, from a Windows debugger machine maybe?

Thanks,
Laszlo
_______________________________________________
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel

Re: Source code debugging of OVMF

[Rebecca Cran]

On 04/16/18 10:13, Laszlo Ersek wrote:

> Here's another thread that you might find useful:
> 
> http://edk2-devel.narkive.com/6BRVus92/qestion-about-how-to-debug-ovmf-on-qemu

I should get my Phabricator wiki running again, which has a
nicely-formatted version of that - I haven't set it up again after
moving systems.

-- 
Rebecca

Re: Source code debugging of OVMF

[Laszlo Ersek]

On 04/16/18 20:25, Rebecca Cran wrote:
> On 04/16/18 10:13, Laszlo Ersek wrote:
> 
>> Here's another thread that you might find useful:
>>
>> http://edk2-devel.narkive.com/6BRVus92/qestion-about-how-to-debug-ovmf-on-qemu
> 
> I should get my Phabricator wiki running again, which has a
> nicely-formatted version of that - I haven't set it up again after
> moving systems.
> 

Yes please! :)

Thanks!
Laszlo

Re: Source code debugging of OVMF

[Rebecca Cran]

On 04/16/18 12:25, Rebecca Cran wrote:
> On 04/16/18 10:13, Laszlo Ersek wrote:
> 
>> Here's another thread that you might find useful:
>>
>> http://edk2-devel.narkive.com/6BRVus92/qestion-about-how-to-debug-ovmf-on-qemu
> 
> I should get my Phabricator wiki running again, which has a
> nicely-formatted version of that - I haven't set it up again after
> moving systems.
> 

https://code.bluestop.org/w/tianocore/debugging-with-gdb/ is now working
again.

-- 
Rebecca

Re: Source code debugging of OVMF

[Laszlo Ersek]

On 04/19/18 04:46, Rebecca Cran wrote:
> On 04/16/18 12:25, Rebecca Cran wrote:
>> On 04/16/18 10:13, Laszlo Ersek wrote:
>>
>>> Here's another thread that you might find useful:
>>>
>>> http://edk2-devel.narkive.com/6BRVus92/qestion-about-how-to-debug-ovmf-on-qemu
>>
>> I should get my Phabricator wiki running again, which has a
>> nicely-formatted version of that - I haven't set it up again after
>> moving systems.
>>
> 
> https://code.bluestop.org/w/tianocore/debugging-with-gdb/ is now working
> again.
> 

Thank you!
Laszlo

gdb reload-uefi missing EFI_SYSTEM_TABLE_POINTER [was: Source code debugging of OVMF]

[Johannes Swoboda]

Hello,

Thank you all for your insights.

You advised me to use the scripts from [1].
I did so following this guide [2] (thank you also for putting it back 
up!), with the minor adaptions that I made the script 
python3-compatible[3], and applied the patches mentioned in [2] 
manually. I assume they failed because they could only be applied to 
older versions of the files to be patched.

In particular, I added the line DebugPkg/GdbSyms/GdbSyms.inf right below 
line OvmfPkg/PlatformDxe/Platform.inf in line 830 of the vUDK2018 
release. That makes sense, right?

The guide looks promising; I think this would give me everything I need.
However I ran into a problem when executing
> (gdb) reload-uefi -o 
> ./Build/OvmfX64/DEBUG_GCC5/X64/DebugPkg/GdbSyms/GdbSyms/DEBUG/GdbSyms.dll

gdb complained:
> Python Exception <class 'gdb.error'> No type named 
> EFI_SYSTEM_TABLE_POINTER.:
> Error occurred in Python command: No type named 
> EFI_SYSTEM_TABLE_POINTER.

The EFI_SYSTEM_TABLE_POINTER is defined in 
[WORKSPACE]/MdeModulePkg/Core/Dxe/Misc/DebugImageInfo.c, so I'm guessing 
I lack that file respectively its corresponding module.

I compiled OVMF with nice OvmfPkg/build.sh -a X64 -n $(getconf 
_NPROCESSORS_ONLN).

If you could tell me whether my guess make sense and how I would include 
[WORKSPACE]/MdeModulePkg/Core/Dxe/Misc/DebugImageInfo.c, that would be 
great.

Kind regards,
Johannes

[1]: https://github.com/andreiw/andreiw-wip.git
[2]: https://code.bluestop.org/w/tianocore/debugging-with-gdb/
[3]: 
https://github.com/johannesswoboda/andreiw-wip/blob/master/uefi/DebugPkg/Scripts/gdb_uefi.py 
- in case someone else needs it one day

Re: gdb reload-uefi missing EFI_SYSTEM_TABLE_POINTER [was: Source code debugging of OVMF]

[Laszlo Ersek]

On 04/20/18 16:54, Johannes Swoboda wrote:
> Hello,
> 
> Thank you all for your insights.
> 
> You advised me to use the scripts from [1].
> I did so following this guide [2] (thank you also for putting it back
> up!), with the minor adaptions that I made the script
> python3-compatible[3], and applied the patches mentioned in [2]
> manually. I assume they failed because they could only be applied to
> older versions of the files to be patched.
> 
> In particular, I added the line DebugPkg/GdbSyms/GdbSyms.inf right below
> line OvmfPkg/PlatformDxe/Platform.inf in line 830 of the vUDK2018
> release. That makes sense, right?

You can add GdbSyms.inf anywhere near the other DXE phase modules. Just
make sure that you patch the DSC file(s) that you build and run
afterwards. (I.e., avoid the case where you patch e.g. the X64 DSC but
then build & boot the IA32 binary.)

> 
> The guide looks promising; I think this would give me everything I need.
> However I ran into a problem when executing
>> (gdb) reload-uefi -o
>> ./Build/OvmfX64/DEBUG_GCC5/X64/DebugPkg/GdbSyms/GdbSyms/DEBUG/GdbSyms.dll
> 
> gdb complained:
>> Python Exception <class 'gdb.error'> No type named
>> EFI_SYSTEM_TABLE_POINTER.:
>> Error occurred in Python command: No type named EFI_SYSTEM_TABLE_POINTER.
> 
> The EFI_SYSTEM_TABLE_POINTER is defined in
> [WORKSPACE]/MdeModulePkg/Core/Dxe/Misc/DebugImageInfo.c, so I'm guessing
> I lack that file respectively its corresponding module.

This command will only work after you attach gdb to qemu. In general,
python extensions for gdb can only look up non-standard typenames etc.
if gdb is connected to a live image that uses those types.

People generally create gdb scripts as follows:

set height 0
set architecture i386:intel
target remote localhost:1234
source /.../edk2/DebugPkg/Scripts/gdb_uefi.py
reload-uefi -o
/.../edk2/Build/Ovmf.../.../.../DebugPkg/GdbSyms/GdbSyms/DEBUG/GdbSyms.dll

And then pass this command script to gdb with the "-x" option.

Thanks,
Laszlo

Re: gdb reload-uefi missing EFI_SYSTEM_TABLE_POINTER [was: Source code debugging of OVMF]

[Palmer, Thomas]

Also, did you start qemu with the "-s" option before you started gdb?


Regards,

Thomas Palmer

“I have only made this letter longer because I have not had the time to make it shorter” - Blaise Pascal


-----Original Message-----
From: Laszlo Ersek [mailto:lersek@redhat.com] 
Sent: Friday, April 20, 2018 10:20 AM
To: Johannes Swoboda <johannes.swoboda@student.tuwien.ac.at>; Rebecca Cran <rebecca@bluestop.org>; Palmer, Thomas <thomas.palmer@hpe.com>
Cc: Michael Kinney <michael.d.kinney@intel.com>; edk2-devel@lists.01.org; clemens.hlauschek@inso.tuwien.ac.at
Subject: Re: gdb reload-uefi missing EFI_SYSTEM_TABLE_POINTER [was: Source code debugging of OVMF]

On 04/20/18 16:54, Johannes Swoboda wrote:
> Hello,
> 
> Thank you all for your insights.
> 
> You advised me to use the scripts from [1].
> I did so following this guide [2] (thank you also for putting it back 
> up!), with the minor adaptions that I made the script 
> python3-compatible[3], and applied the patches mentioned in [2] 
> manually. I assume they failed because they could only be applied to 
> older versions of the files to be patched.
> 
> In particular, I added the line DebugPkg/GdbSyms/GdbSyms.inf right 
> below line OvmfPkg/PlatformDxe/Platform.inf in line 830 of the 
> vUDK2018 release. That makes sense, right?

You can add GdbSyms.inf anywhere near the other DXE phase modules. Just make sure that you patch the DSC file(s) that you build and run afterwards. (I.e., avoid the case where you patch e.g. the X64 DSC but then build & boot the IA32 binary.)

> 
> The guide looks promising; I think this would give me everything I need.
> However I ran into a problem when executing
>> (gdb) reload-uefi -o
>> ./Build/OvmfX64/DEBUG_GCC5/X64/DebugPkg/GdbSyms/GdbSyms/DEBUG/GdbSyms
>> .dll
> 
> gdb complained:
>> Python Exception <class 'gdb.error'> No type named
>> EFI_SYSTEM_TABLE_POINTER.:
>> Error occurred in Python command: No type named EFI_SYSTEM_TABLE_POINTER.
> 
> The EFI_SYSTEM_TABLE_POINTER is defined in 
> [WORKSPACE]/MdeModulePkg/Core/Dxe/Misc/DebugImageInfo.c, so I'm 
> guessing I lack that file respectively its corresponding module.

This command will only work after you attach gdb to qemu. In general, python extensions for gdb can only look up non-standard typenames etc.
if gdb is connected to a live image that uses those types.

People generally create gdb scripts as follows:

set height 0
set architecture i386:intel
target remote localhost:1234
source /.../edk2/DebugPkg/Scripts/gdb_uefi.py
reload-uefi -o
/.../edk2/Build/Ovmf.../.../.../DebugPkg/GdbSyms/GdbSyms/DEBUG/GdbSyms.dll

And then pass this command script to gdb with the "-x" option.

Thanks,
Laszlo

Re: gdb reload-uefi missing EFI_SYSTEM_TABLE_POINTER [was: Source code debugging of OVMF]

[Johannes Swoboda]

Hello again,

Thank you again for taking the time to help me with this.

On 2018-04-20 17:58, Palmer, Thomas wrote:
> Also, did you start qemu with the "-s" option before you started gdb?

Unfortunately, I start qemu with -s -S; then, I attach gdb. I know the 
attaching works because I gdb accepts the continue command which 
instructs qemu to begin execution of the virtual machine.
The source [...]/gdb_uefi.py command seems to work find, but reload-efi 
-o [...]/DEBUG/GdbSyms.dll throws the error "No type named 
EFI_SYSTEM_TABLE_POINTER".

Here is my .gdbinit
set height 0
set architecture i386:x86-64:intel
target remote localhost:1234
source ~/18S/bakk/work/debug_w_andreiw/edk2/DebugPkg/Scripts/gdb_uefi.py
reload-uefi -o 
~/18S/bakk/work/debug_w_andreiw/edk2/Build/OvmfX64/DEBUG_GCC5/X64/DebugPkg/GdbSyms/GdbSyms/DEBUG/GdbSyms.dll

which is executed on gdb startup:
[johannes@johannes-laptop efi_app_only]$ gdb
[...]
The target architecture is assumed to be i386:x86-64:intel
warning: No executable has been specified and target does not support
determining executable automatically.  Try using the "file" command.
0x000000000000fff0 in ?? ()
Python Exception <class 'gdb.error'> No type named 
EFI_SYSTEM_TABLE_POINTER.:
.gdbinit:5: Error in sourced command file:
Error occurred in Python command: No type named 
EFI_SYSTEM_TABLE_POINTER.
(gdb)

Do you have any other suggestions?

Kind regards,
Johannes

> -----Original Message-----
> From: Laszlo Ersek [mailto:lersek@redhat.com]
> On 04/20/18 16:54, Johannes Swoboda wrote:
>> 
>> You advised me to use the scripts from [1].
>> I did so following this guide [2] (thank you also for putting it back
>> up!), with the minor adaptions that I made the script
>> python3-compatible[3], and applied the patches mentioned in [2]
>> manually. I assume they failed because they could only be applied to
>> older versions of the files to be patched.
>> 
>> In particular, I added the line DebugPkg/GdbSyms/GdbSyms.inf right
>> below line OvmfPkg/PlatformDxe/Platform.inf in line 830 of the
>> vUDK2018 release. That makes sense, right?
> 
> You can add GdbSyms.inf anywhere near the other DXE phase modules.
> Just make sure that you patch the DSC file(s) that you build and run
> afterwards. (I.e., avoid the case where you patch e.g. the X64 DSC but
> then build & boot the IA32 binary.)
Allright, I should be safe there.

>> However I ran into a problem when executing
>>> (gdb) reload-uefi -o
>>> ./Build/OvmfX64/DEBUG_GCC5/X64/DebugPkg/GdbSyms/GdbSyms/DEBUG/GdbSyms
>>> .dll
>> 
>> gdb complained:
>>> Python Exception <class 'gdb.error'> No type named
>>> EFI_SYSTEM_TABLE_POINTER.:
>>> Error occurred in Python command: No type named 
>>> EFI_SYSTEM_TABLE_POINTER.
>> 
>> The EFI_SYSTEM_TABLE_POINTER is defined in
>> [WORKSPACE]/MdeModulePkg/Core/Dxe/Misc/DebugImageInfo.c, so I'm
>> guessing I lack that file respectively its corresponding module.
> 
> This command will only work after you attach gdb to qemu. In general,
> python extensions for gdb can only look up non-standard typenames etc.
> if gdb is connected to a live image that uses those types.

Re: gdb reload-uefi missing EFI_SYSTEM_TABLE_POINTER [was: Source code debugging of OVMF]

[Palmer, Thomas]

Can you check that the arguments for the source and reload-uefi arguments are real files and resolve correctly? 

Also, remove the "-S" part and try connecting gdb when EDK2 transistions to DXE or BDS  


Regards,

Thomas Palmer

"I have only made this letter longer because I have not had the time to make it shorter" - Blaise Pascal


-----Original Message-----
From: Johannes Swoboda [mailto:johannes.swoboda@student.tuwien.ac.at] 
Sent: Friday, April 20, 2018 11:39 AM
To: Palmer, Thomas <thomas.palmer@hpe.com>; Laszlo Ersek <lersek@redhat.com>
Cc: Rebecca Cran <rebecca@bluestop.org>; Michael Kinney <michael.d.kinney@intel.com>; edk2-devel@lists.01.org; clemens.hlauschek@inso.tuwien.ac.at
Subject: RE: gdb reload-uefi missing EFI_SYSTEM_TABLE_POINTER [was: Source code debugging of OVMF]

Hello again,

Thank you again for taking the time to help me with this.

On 2018-04-20 17:58, Palmer, Thomas wrote:
> Also, did you start qemu with the "-s" option before you started gdb?

Unfortunately, I start qemu with -s -S; then, I attach gdb. I know the attaching works because I gdb accepts the continue command which instructs qemu to begin execution of the virtual machine.
The source [...]/gdb_uefi.py command seems to work find, but reload-efi -o [...]/DEBUG/GdbSyms.dll throws the error "No type named EFI_SYSTEM_TABLE_POINTER".

Here is my .gdbinit
set height 0
set architecture i386:x86-64:intel
target remote localhost:1234
source ~/18S/bakk/work/debug_w_andreiw/edk2/DebugPkg/Scripts/gdb_uefi.py
reload-uefi -o
~/18S/bakk/work/debug_w_andreiw/edk2/Build/OvmfX64/DEBUG_GCC5/X64/DebugPkg/GdbSyms/GdbSyms/DEBUG/GdbSyms.dll

which is executed on gdb startup:
[johannes@johannes-laptop efi_app_only]$ gdb [...] The target architecture is assumed to be i386:x86-64:intel
warning: No executable has been specified and target does not support determining executable automatically.  Try using the "file" command.
0x000000000000fff0 in ?? ()
Python Exception <class 'gdb.error'> No type named
EFI_SYSTEM_TABLE_POINTER.:
.gdbinit:5: Error in sourced command file:
Error occurred in Python command: No type named EFI_SYSTEM_TABLE_POINTER.
(gdb)

Do you have any other suggestions?

Kind regards,
Johannes

> -----Original Message-----
> From: Laszlo Ersek [mailto:lersek@redhat.com] On 04/20/18 16:54, 
> Johannes Swoboda wrote:
>> 
>> You advised me to use the scripts from [1].
>> I did so following this guide [2] (thank you also for putting it back 
>> up!), with the minor adaptions that I made the script 
>> python3-compatible[3], and applied the patches mentioned in [2] 
>> manually. I assume they failed because they could only be applied to 
>> older versions of the files to be patched.
>> 
>> In particular, I added the line DebugPkg/GdbSyms/GdbSyms.inf right 
>> below line OvmfPkg/PlatformDxe/Platform.inf in line 830 of the
>> vUDK2018 release. That makes sense, right?
> 
> You can add GdbSyms.inf anywhere near the other DXE phase modules.
> Just make sure that you patch the DSC file(s) that you build and run 
> afterwards. (I.e., avoid the case where you patch e.g. the X64 DSC but 
> then build & boot the IA32 binary.)
Allright, I should be safe there.

>> However I ran into a problem when executing
>>> (gdb) reload-uefi -o
>>> ./Build/OvmfX64/DEBUG_GCC5/X64/DebugPkg/GdbSyms/GdbSyms/DEBUG/GdbSym
>>> s
>>> .dll
>> 
>> gdb complained:
>>> Python Exception <class 'gdb.error'> No type named
>>> EFI_SYSTEM_TABLE_POINTER.:
>>> Error occurred in Python command: No type named 
>>> EFI_SYSTEM_TABLE_POINTER.
>> 
>> The EFI_SYSTEM_TABLE_POINTER is defined in 
>> [WORKSPACE]/MdeModulePkg/Core/Dxe/Misc/DebugImageInfo.c, so I'm 
>> guessing I lack that file respectively its corresponding module.
> 
> This command will only work after you attach gdb to qemu. In general, 
> python extensions for gdb can only look up non-standard typenames etc.
> if gdb is connected to a live image that uses those types.

Re: gdb reload-uefi missing EFI_SYSTEM_TABLE_POINTER [was: Source code debugging of OVMF]

[Rebecca Cran]

On 04/20/18 10:38, Johannes Swoboda wrote:
> Hello again,
> 
> Thank you again for taking the time to help me with this.
> 
> On 2018-04-20 17:58, Palmer, Thomas wrote:
>> Also, did you start qemu with the "-s" option before you started gdb?
> 
> Unfortunately, I start qemu with -s -S; then, I attach gdb. I know the
> attaching works because I gdb accepts the continue command which
> instructs qemu to begin execution of the virtual machine.
> The source [...]/gdb_uefi.py command seems to work find, but reload-efi
> -o [...]/DEBUG/GdbSyms.dll throws the error "No type named
> EFI_SYSTEM_TABLE_POINTER".
...
> Do you have any other suggestions?

I'm not sure things have loaded to the point necessary by then. I'd
suggest putting a call to CpuDeadLoop() someplace maybe near where you
want to start debugging. Then once attached to dbg, set the `Index`
variable to 1 to continue.

-- 
Rebecca

Re: gdb reload-uefi missing EFI_SYSTEM_TABLE_POINTER [was: Source code debugging of OVMF]

[Gary Lin]

On Fri, Apr 20, 2018 at 04:54:06PM +0200, Johannes Swoboda wrote:
> Hello,
> 
> Thank you all for your insights.
> 
> You advised me to use the scripts from [1].
> I did so following this guide [2] (thank you also for putting it back up!),
> with the minor adaptions that I made the script python3-compatible[3], and
> applied the patches mentioned in [2] manually. I assume they failed because
> they could only be applied to older versions of the files to be patched.
> 
> In particular, I added the line DebugPkg/GdbSyms/GdbSyms.inf right below
> line OvmfPkg/PlatformDxe/Platform.inf in line 830 of the vUDK2018 release.
> That makes sense, right?
> 
> The guide looks promising; I think this would give me everything I need.
> However I ran into a problem when executing
> > (gdb) reload-uefi -o
> > ./Build/OvmfX64/DEBUG_GCC5/X64/DebugPkg/GdbSyms/GdbSyms/DEBUG/GdbSyms.dll
> 
> gdb complained:
> > Python Exception <class 'gdb.error'> No type named
> > EFI_SYSTEM_TABLE_POINTER.:
> > Error occurred in Python command: No type named
> > EFI_SYSTEM_TABLE_POINTER.
> 
> The EFI_SYSTEM_TABLE_POINTER is defined in
> [WORKSPACE]/MdeModulePkg/Core/Dxe/Misc/DebugImageInfo.c, so I'm guessing I
> lack that file respectively its corresponding module.
> 
I encountered the issue before and it seems caused by the linker option,
"--whole-archive", which drops the symbols not used.

As a workaround, I add the following lines to Initialize():

  DEBUG ((DEBUG_VERBOSE, "%a: %llx\n", __FUNCTION__, &ESTP));
  DEBUG ((DEBUG_VERBOSE, "%a: %llx\n", __FUNCTION__, &EDIITH));
  DEBUG ((DEBUG_VERBOSE, "%a: %llx\n", __FUNCTION__, &EIDH));
  DEBUG ((DEBUG_VERBOSE, "%a: %llx\n", __FUNCTION__, &EIOHU));
  DEBUG ((DEBUG_VERBOSE, "%a: %llx\n", __FUNCTION__, &EIDDE));
  DEBUG ((DEBUG_VERBOSE, "%a: %llx\n", __FUNCTION__, &EIDCNE));
  DEBUG ((DEBUG_VERBOSE, "%a: %llx\n", __FUNCTION__, &EIDCRE));
  DEBUG ((DEBUG_VERBOSE, "%a: %llx\n", __FUNCTION__, &EIDCME));

It at least makes the linker believe the symbols are important.
Maybe you can try it.

Cheers,

Gary Lin

> I compiled OVMF with nice OvmfPkg/build.sh -a X64 -n $(getconf
> _NPROCESSORS_ONLN).
> 
> If you could tell me whether my guess make sense and how I would include
> [WORKSPACE]/MdeModulePkg/Core/Dxe/Misc/DebugImageInfo.c, that would be
> great.
> 
> Kind regards,
> Johannes
> 
> [1]: https://github.com/andreiw/andreiw-wip.git
> [2]: https://code.bluestop.org/w/tianocore/debugging-with-gdb/
> [3]: https://github.com/johannesswoboda/andreiw-wip/blob/master/uefi/DebugPkg/Scripts/gdb_uefi.py
> - in case someone else needs it one day
> _______________________________________________
> edk2-devel mailing list
> edk2-devel@lists.01.org
> https://lists.01.org/mailman/listinfo/edk2-devel
> 

Re: gdb reload-uefi missing EFI_SYSTEM_TABLE_POINTER [was: Source code debugging of OVMF]

[Johannes Swoboda]

Hello,

I asked you for help for source level debugging of ovmf some time ago.
Things got in the way after that, but now I was able to follow your
advice and achieved my goal.

In particular,

1. I added the debug-lines below to GdbSyms.c
2. I started debugging later in the BDS phase (as suggested by several
of you)
3. I got rid of the qemu flag -enable-kvm (it screwed up the stepping in
gdb)

Thank you very much!

Kind regards,
Johannes


On 2018-04-24 04:13, Gary Lin wrote:
> On Fri, Apr 20, 2018 at 04:54:06PM +0200, Johannes Swoboda wrote:
>> gdb complained:
>>> Python Exception <class 'gdb.error'> No type named
>>> EFI_SYSTEM_TABLE_POINTER.:
>>> [...]
> I encountered the issue before and it seems caused by the linker option,
> "--whole-archive", which drops the symbols not used.
> 
> As a workaround, I add the following lines to Initialize():
> 
>   DEBUG ((DEBUG_VERBOSE, "%a: %llx\n", __FUNCTION__, &ESTP));
>   DEBUG ((DEBUG_VERBOSE, "%a: %llx\n", __FUNCTION__, &EDIITH));
>   DEBUG ((DEBUG_VERBOSE, "%a: %llx\n", __FUNCTION__, &EIDH));
>   DEBUG ((DEBUG_VERBOSE, "%a: %llx\n", __FUNCTION__, &EIOHU));
>   DEBUG ((DEBUG_VERBOSE, "%a: %llx\n", __FUNCTION__, &EIDDE));
>   DEBUG ((DEBUG_VERBOSE, "%a: %llx\n", __FUNCTION__, &EIDCNE));
>   DEBUG ((DEBUG_VERBOSE, "%a: %llx\n", __FUNCTION__, &EIDCRE));
>   DEBUG ((DEBUG_VERBOSE, "%a: %llx\n", __FUNCTION__, &EIDCME));
> 
> It at least makes the linker believe the symbols are important.
> Maybe you can try it.
> 
> Cheers,
> 
> Gary Lin