EmulatorPkg and the state of DlLoadImage()

EmulatorPkg and the state of DlLoadImage()

[Marvin Häuser]

Good day everyone,

I'm currently refining the port of EmulatorPkg to my new PE/COFF loader 
library instance.
In the process, I found the function DlOpenImage() [1], which loads UEFI 
Images via the OS loader to utilise its symbol loading capability. 
Theoretically, this should e.g. allow arbitrary debuggers using the OS 
APIs to symbolise the backtrace.

macOS: The function seems to be unused entirely. [2]

Linux: On my system running Fedora 34, the function neither works 
out-of-the-box, nor after significant time of trying to fix it. The 
first issue is that it only proceeds if the Image has a PDB path with 
".pdb" extension [3], while the GCC5 toolchain generates Images with 
".dll" files for PDB paths (see errors below). Once this is resolved, 
there is an error message indicating insufficient Image section alignment:

[...]/Build/EmulatorX64/DEBUG_GCC5/X64/MdeModulePkg/Universal/EbcDxe/EbcDxe/DEBUG/EbcDxe.dll: 
ELF load command alignment not page-aligned

Resolving this yields an error that executable files cannot be loaded 
dynamically:

[...]/Build/EmulatorX64/DEBUG_GCC5/X64/MdeModulePkg/Core/Pei/PeiMain/DEBUG/PeiCore.dll: 
cannot dynamically load executable

With my very limited knowledge about Linux and ELF I tried the naive 
approach of building the Images as shared (hoping it would be similar to 
DLLs, which are built on Windows), but this just silently crashes.

So my questions are:
1) Does this code currently work for anyone?
2) Does anyone use a debugging setup that is incompatible with Images 
loaded by EDK II rather than the OS?
3) Are the issues above known and planned to be fixed?

Thank you for your time!

Best regards,
Marvin


[1]
https://github.com/tianocore/edk2/blob/be282b14938846960cce30825a9fe762e14ca8c9/EmulatorPkg/Unix/Host/Host.c#L1065-L1113

[2]
https://github.com/tianocore/edk2/blob/be282b14938846960cce30825a9fe762e14ca8c9/EmulatorPkg/Unix/Host/Host.c#L1071-L1073

[3]
https://github.com/tianocore/edk2/blob/be282b14938846960cce30825a9fe762e14ca8c9/EmulatorPkg/Unix/Host/Host.c#L1084-L1086
https://github.com/tianocore/edk2/blob/be282b14938846960cce30825a9fe762e14ca8c9/EmulatorPkg/Unix/Host/Host.c#L1003-L1026

Re: EmulatorPkg and the state of DlLoadImage()

[Andrew Fish]

[-- Attachment #1: Type: text/plain, Size: 4324 bytes --]



> On Jul 30, 2021, at 3:37 AM, Marvin Häuser <mhaeuser@posteo.de> wrote:
> 
> Good day everyone,
> 
> I'm currently refining the port of EmulatorPkg to my new PE/COFF loader library instance.
> In the process, I found the function DlOpenImage() [1], which loads UEFI Images via the OS loader to utilise its symbol loading capability. Theoretically, this should e.g. allow arbitrary debuggers using the OS APIs to symbolise the backtrace.
> 
> macOS: The function seems to be unused entirely. [2]
> 
> Linux: On my system running Fedora 34, the function neither works out-of-the-box, nor after significant time of trying to fix it. The first issue is that it only proceeds if the Image has a PDB path with ".pdb" extension [3], while the GCC5 toolchain generates Images with ".dll" files for PDB paths (see errors below). Once this is resolved, there is an error message indicating insufficient Image section alignment:
> 
> [...]/Build/EmulatorX64/DEBUG_GCC5/X64/MdeModulePkg/Universal/EbcDxe/EbcDxe/DEBUG/EbcDxe.dll: ELF load command alignment not page-aligned
> 

The requiring *.pdb seems like something that rotted out and could be fixed. 

> Resolving this yields an error that executable files cannot be loaded dynamically:
> 
> [...]/Build/EmulatorX64/DEBUG_GCC5/X64/MdeModulePkg/Core/Pei/PeiMain/DEBUG/PeiCore.dll: cannot dynamically load executable
> 
> With my very limited knowledge about Linux and ELF I tried the naive approach of building the Images as shared (hoping it would be similar to DLLs, which are built on Windows), but this just silently crashes.
> 

This code is very very old. Notice the comment about gdb predates gdb Python support [1].

What happens if you comment out the DlLoadImage path? There seems to be some gdb scripts? The macOS path sets breakpoints on SecGdbScriptBreak() in an lldb script and loads symbols via that path. That his probably the best path forward for gdb too? 

It looks like if you `build.sh run` you should launch the emulator under gdb and source the symbol loading file.
EmulatorPkg/build.sh:221:  /usr/bin/gdb $BUILD_ROOT_ARCH/Host -q -cd=$BUILD_ROOT_ARCH -x $WORKSPACE/EmulatorPkg/Unix/GdbRun.sh

If you comment out the dlopen() path does it start working? Looks like breaking in with gdb should get symbols loaded? 

> So my questions are:
> 1) Does this code currently work for anyone?
> 2) Does anyone use a debugging setup that is incompatible with Images loaded by EDK II rather than the OS?

Not a 100% sure what you are asking? In a lot of cases you are debugging what is compatible with the OS? For example on macOS we build a mach-O and convert that to PE/COFF. We point the PDB entry at the mach-O file and that is what the debugger sees. As long as the PE/COFF lines up with the mach-O it does not really matter, as at the end of the day the debugger is just processing the dwarf debug info associated with addresses in system memory. 

> 3) Are the issues above known and planned to be fixed?
> 

Not likely please file a BZ. 

Note I’m working on getting a generic gdb debugging script into the edk2 [2] and that should also work with the Emulator. I think you could replace the ` -x $WORKSPACE/EmulatorPkg/Unix/GdbRun.sh` with `-ex efi_gdb.py’. There is not a break hook in those scripts so you would have to run the `efi` command the 1st time you attach to load symbols. The efi_gdb.py script works on stock EFI so it does not depend on any of the hooks in the EmulatorPkg to work. 

> Thank you for your time!
> 
> Best regards,
> Marvin
> 
> 
> [1]
> https://github.com/tianocore/edk2/blob/be282b14938846960cce30825a9fe762e14ca8c9/EmulatorPkg/Unix/Host/Host.c#L1065-L1113
> 
> [2]
> https://github.com/tianocore/edk2/blob/be282b14938846960cce30825a9fe762e14ca8c9/EmulatorPkg/Unix/Host/Host.c#L1071-L1073
> 
> [3]
> https://github.com/tianocore/edk2/blob/be282b14938846960cce30825a9fe762e14ca8c9/EmulatorPkg/Unix/Host/Host.c#L1084-L1086
> https://github.com/tianocore/edk2/blob/be282b14938846960cce30825a9fe762e14ca8c9/EmulatorPkg/Unix/Host/Host.c#L1003-L1026
> 

[1] https://github.com/tianocore/edk2/blob/be282b14938846960cce30825a9fe762e14ca8c9/EmulatorPkg/Unix/Host/Host.c#L1179
[2] https://github.com/ajfish/edk2/blob/BZ3500-gdb/efi_gdb.py

Thanks,

Andrew Fish


[-- Attachment #2: Type: text/html, Size: 6975 bytes --]

Re: [edk2-devel] EmulatorPkg and the state of DlLoadImage()

[Marvin Häuser]

30.07.2021 17:58:05 Andrew Fish via groups.io <afish=apple.com@groups.io>:

>
>
>> On Jul 30, 2021, at 3:37 AM, Marvin Häuser <mhaeuser@posteo.de> wrote:
>>
>> Good day everyone,
>>
>> I'm currently refining the port of EmulatorPkg to my new PE/COFF loader library instance.
>> In the process, I found the function DlOpenImage() [1], which loads UEFI Images via the OS loader to utilise its symbol loading capability. Theoretically, this should e.g. allow arbitrary debuggers using the OS APIs to symbolise the backtrace.
>>
>> macOS: The function seems to be unused entirely. [2]
>>
>> Linux: On my system running Fedora 34, the function neither works out-of-the-box, nor after significant time of trying to fix it. The first issue is that it only proceeds if the Image has a PDB path with ".pdb" extension [3], while the GCC5 toolchain generates Images with ".dll" files for PDB paths (see errors below). Once this is resolved, there is an error message indicating insufficient Image section alignment:
>>
>> [...]/Build/EmulatorX64/DEBUG_GCC5/X64/MdeModulePkg/Universal/EbcDxe/EbcDxe/DEBUG/EbcDxe.dll: ELF load command alignment not page-aligned
>>
>
> The requiring *.pdb seems like something that rotted out and could be fixed. 

OK, yes.

>> Resolving this yields an error that executable files cannot be loaded dynamically:
>>
>> [...]/Build/EmulatorX64/DEBUG_GCC5/X64/MdeModulePkg/Core/Pei/PeiMain/DEBUG/PeiCore.dll: cannot dynamically load executable
>>
>> With my very limited knowledge about Linux and ELF I tried the naive approach of building the Images as shared (hoping it would be similar to DLLs, which are built on Windows), but this just silently crashes.
>>
>
> This code is very very old. Notice the comment about gdb predates gdb Python support [1].

Right, that is why I wondered, whether it is still needed.
> What happens if you comment out the DlLoadImage path? There seems to be some gdb scripts?

It already is pretty much no-op on my system. On break, the debug code 
writes the symbol file paths into a file (Host.gdb) and a gdb script 
loads them. With gdb, this code seems to not be needed at all.

> The macOS path sets breakpoints on SecGdbScriptBreak() in an lldb script and loads symbols via that path. That his probably the best path forward for gdb too? 
>
> It looks like if you `build.sh run` you should launch the emulator under gdb and source the symbol loading file.
> EmulatorPkg/build.sh:221:  /usr/bin/gdb $BUILD_ROOT_ARCH/Host -q -cd=$BUILD_ROOT_ARCH -x $WORKSPACE/EmulatorPkg/Unix/*GdbRun.sh*
>
> If you comment out the dlopen() path does it start working? Looks like breaking in with gdb should get symbols loaded? 

Yes, it works, even without uncommenting. I'm not experienced with gdb 
enough to know how it should work, but I could help with a better 
solution if such is needed later. For now, I'd like to figure out 
whether dlopen() is legacy code nobody uses or needs first.

>
>> So my questions are:
>> 1) Does this code currently work for anyone?
>> 2) Does anyone use a debugging setup that is incompatible with Images loaded by EDK II rather than the OS?
>
> Not a 100% sure what you are asking?

Sorry if it was unclear, "loaded by EDK II" refers to Images loaded 
without dlopen() and "loaded by the OS" to Images loaded with dlopen().

> In a lot of cases you are debugging what is compatible with the OS? For example on macOS we build a mach-O and convert that to PE/COFF. We point the PDB entry at the mach-O file and that is what the debugger sees. As long as the PE/COFF lines up with the mach-O it does not really matter, as at the end of the day the debugger is just processing the dwarf debug info associated with addresses in system memory. 
>
>> 3) Are the issues above known and planned to be fixed?
>>
>
> Not likely please file a BZ. 

OK, will do soon.

>
> Note I’m working on getting a generic gdb debugging script into the edk2 [2] and that should also work with the Emulator. I think you could replace the ` -x $WORKSPACE/EmulatorPkg/Unix/GdbRun.sh` with `-ex efi_gdb.py’. There is not a break hook in those scripts so you would have to run the `efi` command the 1st time you attach to load symbols. The efi_gdb.py script works on stock EFI so it does not depend on any of the hooks in the EmulatorPkg to work. 

I'm not good with Python, so will have to check. :)

Best regards,
Marvin

>
>> Thank you for your time!
>>
>> Best regards,
>> Marvin
>>
>>
>> [1]
>> https://github.com/tianocore/edk2/blob/be282b14938846960cce30825a9fe762e14ca8c9/EmulatorPkg/Unix/Host/Host.c#L1065-L1113
>>
>> [2]
>> https://github.com/tianocore/edk2/blob/be282b14938846960cce30825a9fe762e14ca8c9/EmulatorPkg/Unix/Host/Host.c#L1071-L1073
>>
>> [3]
>> https://github.com/tianocore/edk2/blob/be282b14938846960cce30825a9fe762e14ca8c9/EmulatorPkg/Unix/Host/Host.c#L1084-L1086
>> https://github.com/tianocore/edk2/blob/be282b14938846960cce30825a9fe762e14ca8c9/EmulatorPkg/Unix/Host/Host.c#L1003-L1026
>>
>
> [1] https://github.com/tianocore/edk2/blob/be282b14938846960cce30825a9fe762e14ca8c9/EmulatorPkg/Unix/Host/Host.c#L1179
> [2] https://github.com/ajfish/edk2/blob/BZ3500-gdb/efi_gdb.py
>
> Thanks,
>
> Andrew Fish
>
> 

Re: [edk2-devel] EmulatorPkg and the state of DlLoadImage()

[Andrew Fish]

[-- Attachment #1: Type: text/plain, Size: 6487 bytes --]



> On Jul 30, 2021, at 1:34 PM, Marvin Häuser <mhaeuser@posteo.de> wrote:
> 
> 30.07.2021 17:58:05 Andrew Fish via groups.io <http://groups.io/> <afish=apple.com@groups.io <mailto:afish=apple.com@groups.io>>:
> 
>> 
>> 
>>> On Jul 30, 2021, at 3:37 AM, Marvin Häuser <mhaeuser@posteo.de> wrote:
>>> 
>>> Good day everyone,
>>> 
>>> I'm currently refining the port of EmulatorPkg to my new PE/COFF loader library instance.
>>> In the process, I found the function DlOpenImage() [1], which loads UEFI Images via the OS loader to utilise its symbol loading capability. Theoretically, this should e.g. allow arbitrary debuggers using the OS APIs to symbolise the backtrace.
>>> 
>>> macOS: The function seems to be unused entirely. [2]
>>> 
>>> Linux: On my system running Fedora 34, the function neither works out-of-the-box, nor after significant time of trying to fix it. The first issue is that it only proceeds if the Image has a PDB path with ".pdb" extension [3], while the GCC5 toolchain generates Images with ".dll" files for PDB paths (see errors below). Once this is resolved, there is an error message indicating insufficient Image section alignment:
>>> 
>>> [...]/Build/EmulatorX64/DEBUG_GCC5/X64/MdeModulePkg/Universal/EbcDxe/EbcDxe/DEBUG/EbcDxe.dll: ELF load command alignment not page-aligned
>>> 
>> 
>> The requiring *.pdb seems like something that rotted out and could be fixed. 
> 
> OK, yes.
> 
>>> Resolving this yields an error that executable files cannot be loaded dynamically:
>>> 
>>> [...]/Build/EmulatorX64/DEBUG_GCC5/X64/MdeModulePkg/Core/Pei/PeiMain/DEBUG/PeiCore.dll: cannot dynamically load executable
>>> 
>>> With my very limited knowledge about Linux and ELF I tried the naive approach of building the Images as shared (hoping it would be similar to DLLs, which are built on Windows), but this just silently crashes.
>>> 
>> 
>> This code is very very old. Notice the comment about gdb predates gdb Python support [1].
> 
> Right, that is why I wondered, whether it is still needed.
>> What happens if you comment out the DlLoadImage path? There seems to be some gdb scripts?
> 
> It already is pretty much no-op on my system. On break, the debug code writes the symbol file paths into a file (Host.gdb) and a gdb script loads them. With gdb, this code seems to not be needed at all.
> 
>> The macOS path sets breakpoints on SecGdbScriptBreak() in an lldb script and loads symbols via that path. That his probably the best path forward for gdb too? 
>> It looks like if you `build.sh run` you should launch the emulator under gdb and source the symbol loading file.
>> EmulatorPkg/build.sh:221:  /usr/bin/gdb $BUILD_ROOT_ARCH/Host -q -cd=$BUILD_ROOT_ARCH -x $WORKSPACE/EmulatorPkg/Unix/*GdbRun.sh*
>> 
>> If you comment out the dlopen() path does it start working? Looks like breaking in with gdb should get symbols loaded? 
> 
> Yes, it works, even without uncommenting. I'm not experienced with gdb enough to know how it should work, but I could help with a better solution if such is needed later. For now, I'd like to figure out whether dlopen() is legacy code nobody uses or needs first.
> 

Marvin,

Sorry I started looking at the code and forgot to give the history. The dlopen() was a trick copied from the Windows port. Basically an EFI image is loaded into EFI memory and it is also loaded into the application memory outside of EFI vis the dlopen() and the entry point is returned to be the dlopen() one. This fakes EFI out enough to be happy, and since the EFI code was loaded via dlopen() symbols get loaded automatically. So the only purpose of the dlopen is automatic source level debugging without requiring any debugger scripting. 

The gdb/lldb scripts have magic to tell the debugger the location things EFI got loaded, so you don’t need the dlopen() to load symbols. Basically not using the dlopen is a more realistic simulation of EFI. 

>> 
>>> So my questions are:
>>> 1) Does this code currently work for anyone?
>>> 2) Does anyone use a debugging setup that is incompatible with Images loaded by EDK II rather than the OS?
>> 
>> Not a 100% sure what you are asking?
> 
> Sorry if it was unclear, "loaded by EDK II" refers to Images loaded without dlopen() and "loaded by the OS" to Images loaded with dlopen().
> 

Hopefully the above history clears that up.

>> In a lot of cases you are debugging what is compatible with the OS? For example on macOS we build a mach-O and convert that to PE/COFF. We point the PDB entry at the mach-O file and that is what the debugger sees. As long as the PE/COFF lines up with the mach-O it does not really matter, as at the end of the day the debugger is just processing the dwarf debug info associated with addresses in system memory. 
>>> 3) Are the issues above known and planned to be fixed?
>>> 
>> 
>> Not likely please file a BZ. 
> 
> OK, will do soon.
> 
>> 
>> Note I’m working on getting a generic gdb debugging script into the edk2 [2] and that should also work with the Emulator. I think you could replace the ` -x $WORKSPACE/EmulatorPkg/Unix/GdbRun.sh` with `-ex efi_gdb.py’. There is not a break hook in those scripts so you would have to run the `efi` command the 1st time you attach to load symbols. The efi_gdb.py script works on stock EFI so it does not depend on any of the hooks in the EmulatorPkg to work. 
> 
> I'm not good with Python, so will have to check. :)

I was forced to learn Python to figure about debugging scripts :)

Thanks,

Andrew Fish

> 
> Best regards,
> Marvin
> 
>> 
>>> Thank you for your time!
>>> 
>>> Best regards,
>>> Marvin
>>> 
>>> 
>>> [1]
>>> https://github.com/tianocore/edk2/blob/be282b14938846960cce30825a9fe762e14ca8c9/EmulatorPkg/Unix/Host/Host.c#L1065-L1113
>>> 
>>> [2]
>>> https://github.com/tianocore/edk2/blob/be282b14938846960cce30825a9fe762e14ca8c9/EmulatorPkg/Unix/Host/Host.c#L1071-L1073
>>> 
>>> [3]
>>> https://github.com/tianocore/edk2/blob/be282b14938846960cce30825a9fe762e14ca8c9/EmulatorPkg/Unix/Host/Host.c#L1084-L1086
>>> https://github.com/tianocore/edk2/blob/be282b14938846960cce30825a9fe762e14ca8c9/EmulatorPkg/Unix/Host/Host.c#L1003-L1026
>>> 
>> 
>> [1] https://github.com/tianocore/edk2/blob/be282b14938846960cce30825a9fe762e14ca8c9/EmulatorPkg/Unix/Host/Host.c#L1179
>> [2] https://github.com/ajfish/edk2/blob/BZ3500-gdb/efi_gdb.py
>> 
>> Thanks,
>> 
>> Andrew Fish
>> 
> 
> 
> 


[-- Attachment #2: Type: text/html, Size: 26838 bytes --]