Re: [edk2-devel] [edk2-discuss] GSoC Proposal

["Marvin Häuser" <mhaeuser@posteo.de>]

> On 14. Apr 2022, at 00:57, Nate DeSimone <nathaniel.l.desimone@intel.com> wrote:
> 
> Hi All,
> 
> Pedro is 100% correct. The primary use case and the reason I added this is to remove library duplication across all the .efi files. This is actually super valuable because LZMA compression is becoming ineffective because compiler optimization is getting so good that the patterns for a library across binaries don’t match very well anymore.

I feel like there are so many much easier solutions to this problem that are at most limited by the clear specification. The UEFI specification with regards to booting and all of that obviously is of utmost importance. The UEFI PI specification parts that deal about internal structure, as far as I know, are only in place to make it easy to integrate Intel IP. In fact, I don’t *know*, but I’m pretty sure the very strict separation between PEI and DXE was preserved mostly because MRC was 32-bit-only for a long time. Glad that seems to have been resolved, AMD does memory init by PSP nowadays.

For many good reasons, Linux does not provide a stable kernel API. This allows to easily deploy breaking changes across the entire codebase. Obviously, this is infeasible at a large scale when you need to integrate IP blobs, but it would already help to break the expectation that UEFI PI is a perfectly forwards- and backwards-compatible system. DXE has SetMem and CopyMem as part of gBS. Maybe I don’t like it that much as part of the spec itself, but it’s a good idea technically. I’d probably opt to have a quickly accessible implementation detail of important function pointers appended to PEI and DXE services, used by in-tree modules. This may break both forwards- and backwards-compatibility, but as it only applies to in-tree modules, that is fine so long as we let go of the compatibility notions. PPIs and protocols are an option too of course, but they have a lookup performance penalty. Compared to dynamic linking, that should hopefully be negligible however.

Absolutely optional to read, I don’t intend to waste anyone’s time much, some philosophical stuff about my rationale:

If you started UEFI from scratch right now, would it have strictly separated PEI and DXE? The duplication between PEI and DXE core, and by extension MM core, would be my most obvious place to start reducing size. I would probably opt for a PEI/DXE hybrid where it starts in „minimal mode“ (maybe think of PEI more like a microkernel here) and after memory is initialised, the rest of DXE is loaded. And MM core would get no loading at all, this requirement has gladly been dropped ages ago. Just one prelinked snapshot of the address space with a relocation table and a safe self-relocator on entry (this is needed at the very least for ARM).

Ironically, with my idea for MM, dynamic loading would be free as everything is prelinked anyway. The same is true for PEI XIP, it is prelinked by nature. What I do not like is the additional dynamic linking code at load-time for non-XIP modules. Though, the more I think about it, the more I wonder whether not the entirety of UEFI could be composed of prelinked, relocatable memory snapshots without traditional image loading entirely (for in-FW stuff). macOS has a similar concept with its “Kernel Collections”. Well, way too much off-topic now. :)

Why am I explaining all this despite the fact everyone knows this will never happen? Because I don’t like the notion of fixing issues of an already overcomplicated system by adding even more complicated stuff. Especially when the existing overcomplicated stuff is already uncomfortably broken.

Best regards,
Marvin

> For XIP PEI code… it will really help and would be very timely since the transition of PEI from 32-bit to 64-bit is going to increase the size of PEI by ~20%.
> 
> Thanks,
> Nate
> 
> From: Pedro Falcato <pedro.falcato@gmail.com>
> Sent: Wednesday, April 13, 2022 11:43 AM
> To: edk2-devel-groups-io <devel@edk2.groups.io>; Marvin Häuser <mhaeuser@posteo.de>
> Cc: discuss@edk2.groups.io; adachristine18@gmail.com; Desimone, Nathaniel L <nathaniel.l.desimone@intel.com>; Shi, Steven <steven.shi@intel.com>
> Subject: Re: [edk2-devel] [edk2-discuss] GSoC Proposal
> 
> Hi Marvin, Ada,
> 
> Some comments:
> 
> I don't think the purpose of the dynamic linker is to treat EFI as a complete operating system, but to try to eliminate the static linking that may be needlessly duplicating
> code that could instead be put in a single dynamic library. For instance, MdePkg and MdeModulePkg are linked into a *lot* of .efi, instead of being just a library. It'd be nice to see some
> numbers on this (something like Google's bloaty could be run on every .efi file, in order to understand how much file space we would actually save).
> 
> Other comments inline.
> 
> On Wed, Apr 13, 2022 at 4:15 PM Marvin Häuser <mhaeuser@posteo.de<mailto:mhaeuser@posteo.de>> wrote:
> 
> On 13. Apr 2022, at 16:38, Ada Christine <adachristine18@gmail.com<mailto:adachristine18@gmail.com>> wrote:
> i was replying via the groups.io<http://groups.io> web interface, I'm guessing that messed up
> the thread? i haven't used mailing lists before and don't know how they
> work. I'll use my mail client from here on.
> 
> I'm on board with not treating EFI as an operating system. the more i think
> about it the more it looks like scope creep.
> 
> Agreed.
> 
> 
> I'm not quite as enthusiastic
> about it as i was at first glance.
> 
> I'm still keen on doing my gsoc proposal for edk, though, and even if this
> task and the acpica application are decided to be out of scope unit
> testing,
> 
> How about fuzz-testing? This is also something edk2 needs quite badly. At Acidanthera, we compile edk2 code in userspace outside the edk2 build system and fuzz with dummy applications.
> 
> Note: fuzzing is also part of the LLVM instrumentation suite (see https://llvm.org/docs/LibFuzzer.html) and is something I could happily mentor.
> clang integration
> 
> Pedro and Vitaly are looking for someone to finish ASan: https://edk2.groups.io/g/devel/topic/90010978#87991
> There are working UBSan concepts, but they also need to be mainlined.
> 
> Is Vitaly going to be a mentor? I was assuming it was going to be me and some other, more senior, mentor (possibly Steven Shi, which I included in the task).
> Anyway, re: ASAN, if the project includes ASAN, UBSAN and possibly some other sanitizer it's quite possible that it could be considered a large project (which means more hours but a larger stipend too). Fuzzing + coverage could
> be very nice additions to this project idea.
> Also, is stress-testing a decent idea?
> 
> 
> and source-level debugging are all relevant to
> my interests.
> 
> how about your ideas for security stuff?
> 
> I want the entirety of MM to leverage SmmMemLib and to support SMAP. SmmMemLib would then handle UEFI->MMRAM and BaseMemoryLib would only work on MMRAM. Also evaluation of how to best avoid pointers in MM communication buffers would be nice.
> 
> There also is a bunch of other stuff, like working out moving a part of CpuDxe into DxeCore to have memory protection live immediately, memory protection in PEI, a replacement for the TE format (it’s buggy and most platforms mostly abandoned it over various issues), and alternatives to guarding critical code with SMM (like allowing NVRAM commits only as part of a reboot).
> 
> I personally find all of those projects very important, but I cannot promise many people agree. Especially those that impose global changes (most notably the TE replacement) may be very tedious to submit. Gladly, I believe you can submit multiple proposals (?)
> 
> Best regards,
> Marvin
> 
> 
> I'm not very knowledgeable about
> trusted platform or secure boot but I'm willing to learn whatever is
> necessary to get something spun up for my proposal.
> 
> On Wed, Apr 13, 2022, 12:05 Marvin Häuser <mhaeuser@posteo.de<mailto:mhaeuser@posteo.de>> wrote:
> 
> 
> Do you use the “reply all” option in your mail client? Looks like my CCs
> have been dropped again. Comments inline.
> 
> On 13. Apr 2022, at 12:54, Ada Christine <adachristine18@gmail.com<mailto:adachristine18@gmail.com>>
> wrote:
> Hi, Marvin
> 
> Its similarity to my own latest experiment is the key to what grabbed my
> attention. I have no particular use case in mind for it, but I see its
> potential for anybody developing larger applications in that when a library
> is changed there's no need to distribute a new version of the whole binary,
> just the relevant library module.
> 
> I really do not like the trend of treating UEFI as a full-fledged OS - it
> is not. The most used UEFI applications, OS loaders, are really not that
> huge and are distributed as part of the OS image anyway. Even for less used
> applications, you will always get a full snapshot anyhow. Gladly we don’t
> have auto-update and package management yet. :)
> 
> 
> I slept on it and it occurred to me that the whole thing could operate
> similarly to the shell protocol in that the linker/loader is itself an
> application that does a LoadImage() on the application needing dynamic
> linking facilities.
> 
> That would mean the linker itself is shipped with every application that
> requires it? Otherwise it doesn’t make much sense for it to be an app and
> below’s problems apply.
> 
> If however the whole plan is making the linker as a DXE and including it
> with the firmware, that I'm not quite as sure about. That would necessarily
> tie any applications using dynamic linking to TianoCore or any firmware
> distribution that derives from it.
> 
> I think that was the idea referred to as “edk2 core” by Steven, but I’d
> like to hear his proposal to be sure. Virtually everyone uses edk2, so that
> itself is not the problem, but versioning is. Vendors are slow to update
> their snapshots or have just given up doing that entirely. Distributing it
> for external applications like OS loaders would mean this can be leveraged
> probably no earlier than 10 years from now. And for in-firmware things, I
> have a hard time thinking about a use-case that outweighs the drawbacks.
> 
> 
> To shift the topic slightly back to GSoC, however, I'm willing to work
> on other items on the task list. Unit testing and an ACPICA application are
> the alternative projects I had thought about. I need to choose fairly soon
> as the proposal deadline is next Tuesday. I know a tiny bit about porting
> ACPICA as I also have plans to incorporate it into my own project.
> 
> I have a few more ideas for security stuff, but Nate did not confirm them
> as appropriate yet and I’m not here to drive you away from this specific
> task (or the others). However, I’m still curious and concerned. :)
> 
> Best regards,
> Marvin
> 
> 
> 
> 
> 
> 
> 
> 
> --
> Pedro Falcato
> 
> 
> 
> 
>