Re: [edk2-discuss] GSoC Proposal

Re: [edk2-discuss] GSoC Proposal

[Marvin Häuser]

CC Nate (GSoC admin)
CC Steven (task mentor)
CC edk2-devel (you picked the logically correct list, but it’s pretty dead and barely anyone reads it)

Hey Ada,

Out of mere curiosity, why did you pick this item? :)

Hey Steven,

I feel like there is more to your proposal than is given on the task page. Why is it “ELF first”, is it something useful for UefiPayloadPkg or Linux somehow?

As for supporting it in the EDK II core, I personally feel like this is much too late. The entire ecosystem is centred around protocols (and the services tables) already. “Loading only when necessary” doesn’t sound very important to me personally, as the firmware image is already supposed to be fairly minimal. I’d rather like to see the introduction of “lazy protocols” (which do not require any new fundamental concepts), e.g., for network and HID stuff like mice and touch, which go through the driver connection procedure only when a protocol function is called for the first time. A big issue with this of course are non-function pointers in the protocol structure.

This will not only require a dynamic linker in the firmware to maybe double the size of the already disgusting and vastly unmaintained PE loader, it will also require further format conversion from ELF and Mach-O, both of which already are buggy (the former much more so than the latter). This is a tremendous effort in my opinion and introducing partial support will cause more awkward toolchain limitations.

Can you please outline why this (in my opinion, big) tradeoff is worth it? Just curious. :)

Best regards,
Marvin

> On 13. Apr 2022, at 03:05, Ada Christine <adachristine18@gmail.com> wrote:
> Hello, edk2 developers!
> 
> I've registered as a contributor candidate for GSoC 2022 and am interested in working on one of the items from the Tasks list here https://github.com/tianocore/tianocore.github.io/wiki/Tasks. Specifically, adding dynamic linking support caught my attention as this is something i've been investigating and learning more about in one of my own personal projects. As a little background, my personal project is an experiment in OS development and I use a very small subset of the boot services to get started and loaded. It can be found here: https://github.com/adachristine/sophia. Recently I've started investigating (and begun to implement) using ELF's dynamic facilities to dynamically load kernel modules. I know PE is slightly different to ELF, but the principles seem similar enough.
> 
> I've had a few glances at the EDKII source code in the past and have a general idea of how it all fits together. What I have in mind to implement this would be the following:
> - create a dynamic linker as a module package to be compiled into the main application
>  (alternatively, implement dynamic linking as a runtime service driver?)
> - adjust the build system to enable building as DLLs and dynamic linking of module packages to the main application
>  (module packages could be per-application and optionally site packages in a subdirectory of the ESP?)
> 
> I know the details of how this would all fit together are a little more involved, but this is just the rough first idea that came to my mind. Happy to hear feedback, and if my idea seems feasible I can get to work on a more in-depth plan to put this together.
> 
> Thanks!
> 
> - Ada
> 
> 
> 

Re: [edk2-discuss] GSoC Proposal

[Rebecca Cran]

On 4/13/22 00:54, Marvin Häuser wrote:
> CC edk2-devel (you picked the logically correct list, but it’s pretty dead and barely anyone reads it)
Just a minor correction - it's edk2-discuss (discuss@edk2.groups.io) 
that was cc'd but that's pretty dead.
edk2-devel (devel@edk2.groups.io) is much more active!

-- 
Rebecca Cran

Re: [edk2-devel] [edk2-discuss] GSoC Proposal

[Rebecca Cran]

Oh I see what was meant - ignore me!
I do think it would be good to have more discussions on edk2-discuss 
though :)

-- 
Rebecca Cran

On 4/13/22 08:44, Rebecca Cran wrote:
> On 4/13/22 00:54, Marvin Häuser wrote:
>> CC edk2-devel (you picked the logically correct list, but it’s pretty 
>> dead and barely anyone reads it)
> Just a minor correction - it's edk2-discuss (discuss@edk2.groups.io) 
> that was cc'd but that's pretty dead.
> edk2-devel (devel@edk2.groups.io) is much more active!
>

FW: [edk2-discuss] GSoC Proposal

[Nate DeSimone]

Hi Ada,

Great to meet you and welcome to the TianoCore project! Great to hear you are interested! Despite Marvin's misgivings, I think dynamic linking would be an excellent addition to EDK II! Marvin is right that we would not want to use it in UEFI spec compliant applications or OpROMs at least for now, but even if you make the assumption that this feature would only be used with EDK II it is still valuable. The primary value I see is reducing the size of BIOS images. Every DXE driver and PEIM includes a lot of statically linked libraries from MdePkg and/or MdeModulePkg depending on the exact driver and build configuration, and it adds up very quickly. On debug builds, the infrastructure for debug messages becomes particularly large. Every PEIM/DXE driver needs to have a copy of the parser for printf() style format strings. The overhead works out to ~12KB for every PEIM and DXE driver. Most BIOS images have ~250 DXE drivers and ~30 PEIMs these days. Uncompressed that works out to >3MB of overhead. The DXE drivers are typically compressed, but LZMA isn't perfect at deduplication and as optimizing compliers have gotten better the deduplication has gotten worse, which is causing this to become more important over time. I did some experiments with the Intel reference BIOS and I found that we can save roughly 1.5-2MB of space with dynamic linking, which is >10% of the flash budget. For PEI where there is no compression, the flash budget savings is >20%, and it has the extra benefit of reducing usage of precious NEM memory. Another thing to consider is that linking OpenSSL for doing code signing checks add ~65KB to the size of the PEIM doing the signature check. When you consider that moving PEI from 32-bit to 64-bit is going to increase code size by ~20%, this feature is extremely valuable for many reasons.

More forward looking, if the project to add Rust is successful then suddenly the rich set of libraries available in Rust crates becomes available to EDK II. Those are going to be big and the only way we will be able to use them is with some deduplication facility like dynamic linking. That will require having some way of using type safe Rust objects across driver boundaries... which is completely out of scope for this GSoC project, but at the same time this GSoC project would be a necessary prerequisite before we could even start thinking about that 😊.

I agree with Marvin that PE/COFF should be the preferred starting point. I believe ELF to be a security risk. It looks easy to parse on the surface but it has been well documented that the devil is in the details and as the CVE history shows it is very easy to build a ELF parser with unintended overflows and other security vulnerabilities. Perhaps most worrying to me is that I don't believe the security community has done enough research on ELF yet, whereas PE/COFF has been very heavily researched and has a mature literature.

Let me know if any of this piques your interest, I would be happy to answer any questions you have!

Hope this helps and welcome to the project!

With Best Regards,
Nate

> -----Original Message-----
> From: discuss@edk2.groups.io <discuss@edk2.groups.io> On Behalf Of Ada 
> Christine
> Sent: Tuesday, April 12, 2022 11:09 AM
> To: discuss@edk2.groups.io
> Subject: [edk2-discuss] GSoC Proposal
> 
> Hello, edk2 developers!
> 
> I've registered as a contributor candidate for GSoC 2022 and am 
> interested in working on one of the items from the Tasks list here 
> https://github.com/tianocore/tianocore.github.io/wiki/Tasks. 
> Specifically, adding dynamic linking support caught my attention as 
> this is something i've been investigating and learning more about in 
> one of my own personal projects. As a little background, my personal 
> project is an experiment in OS development and I use a very small 
> subset of the boot services to get started and loaded. It can be found here: https://github.com/adachristine/sophia.
> Recently I've started investigating (and begun to implement) using 
> ELF's dynamic facilities to dynamically load kernel modules. I know PE 
> is slightly different to ELF, but the principles seem similar enough.
> 
> I've had a few glances at the EDKII source code in the past and have a 
> general idea of how it all fits together. What I have in mind to 
> implement this would be the following:
> - create a dynamic linker as a module package to be compiled into the 
> main application
>   (alternatively, implement dynamic linking as a runtime service 
> driver?)
> - adjust the build system to enable building as DLLs and dynamic 
> linking of module packages to the main application
>   (module packages could be per-application and optionally site 
> packages in a subdirectory of the ESP?)
> 
> I know the details of how this would all fit together are a little 
> more involved, but this is just the rough first idea that came to my 
> mind. Happy to hear feedback, and if my idea seems feasible I can get 
> to work on a more in-depth plan to put this together.
> 
> Thanks!
> 
> - Ada
> 
> 
> 
>