From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from ma1-aaemail-dr-lapp02.apple.com (ma1-aaemail-dr-lapp02.apple.com [17.171.2.68]) by mx.groups.io with SMTP id smtpd.web09.3125.1649990555201236874 for ; Thu, 14 Apr 2022 19:42:35 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=@apple.com header.s=20180706 header.b=rNH3VyVm; spf=pass (domain: apple.com, ip: 17.171.2.68, mailfrom: afish@apple.com) Received: from pps.filterd (ma1-aaemail-dr-lapp02.apple.com [127.0.0.1]) by ma1-aaemail-dr-lapp02.apple.com (8.16.0.42/8.16.0.42) with SMTP id 23F2dr0V058237; Thu, 14 Apr 2022 19:42:34 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=apple.com; h=content-type : mime-version : subject : from : in-reply-to : date : cc : content-transfer-encoding : message-id : references : to; s=20180706; bh=kZT+a4+k4MAapoqtay/epsCakshEOjnCeHm2LpabUfU=; b=rNH3VyVma0qzFoks4NtmTbTeRmtJSK6v3c9oRKOorsUgJxZWHb68QA9sZb2ey9R+gR+E XY7Xc9SNQ2mkFnWM+QVOtGMe7KHJm8l8B8jgpMkKveTcz2QpbkccjWzzPLeGmwiL16G1 tgP+ta3B4rlJ3iZZBYmdUMskhb8KfUBXnyClLBL6TB5SkpAH/5dF7RLFwW/C8QcIJkWb clKyHJORM3kCWHTeU91/Wp1el2BlMDzvVe/HVtwY65X0tEoytc/MR1VaPDkec2YbQmpW 41Mb+DQjB9fPjSfWlY6yuMn4MHdYL+QaUSOyAcvvqLnOu6bwpGNLsFhe5fdJ0HINDYE5 YQ== Received: from rn-mailsvcp-mta-lapp01.rno.apple.com (rn-mailsvcp-mta-lapp01.rno.apple.com [10.225.203.149]) by ma1-aaemail-dr-lapp02.apple.com with ESMTP id 3fb6tuv1s2-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NO); Thu, 14 Apr 2022 19:42:34 -0700 Received: from rn-mailsvcp-mmp-lapp03.rno.apple.com (rn-mailsvcp-mmp-lapp03.rno.apple.com [17.179.253.16]) by rn-mailsvcp-mta-lapp01.rno.apple.com (Oracle Communications Messaging Server 8.1.0.16.20220118 64bit (built Jan 18 2022)) with ESMTPS id <0RAD00J3A0UX3N20@rn-mailsvcp-mta-lapp01.rno.apple.com>; Thu, 14 Apr 2022 19:42:33 -0700 (PDT) Received: from process_milters-daemon.rn-mailsvcp-mmp-lapp03.rno.apple.com by rn-mailsvcp-mmp-lapp03.rno.apple.com (Oracle Communications Messaging Server 8.1.0.16.20220118 64bit (built Jan 18 2022)) id <0RAD009000QRGK00@rn-mailsvcp-mmp-lapp03.rno.apple.com>; Thu, 14 Apr 2022 19:42:33 -0700 (PDT) X-Va-A: X-Va-T-CD: 2c15df6d2d4c3f10bb2d6d67d134c706 X-Va-E-CD: bce8c484736e3f12f4d3e2df5e356d68 X-Va-R-CD: 7f0183fb7246e1f2662e7dae7f1bb48e X-Va-CD: 0 X-Va-ID: 756f4fc5-1dd7-4060-aef1-b98a783addec X-V-A: X-V-T-CD: 2c15df6d2d4c3f10bb2d6d67d134c706 X-V-E-CD: bce8c484736e3f12f4d3e2df5e356d68 X-V-R-CD: 7f0183fb7246e1f2662e7dae7f1bb48e X-V-CD: 0 X-V-ID: 8d12e92d-306a-4d3d-9b5e-8bccc79b5952 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.486,18.0.858 definitions=2022-04-14_07:2022-04-14,2022-04-14 signatures=0 Received: from smtpclient.apple (unknown [17.235.1.170]) by rn-mailsvcp-mmp-lapp03.rno.apple.com (Oracle Communications Messaging Server 8.1.0.16.20220118 64bit (built Jan 18 2022)) with ESMTPSA id <0RAD0095S0UWKK00@rn-mailsvcp-mmp-lapp03.rno.apple.com>; Thu, 14 Apr 2022 19:42:33 -0700 (PDT) MIME-version: 1.0 (Mac OS X Mail 15.0 \(3693.20.0.1.32\)) Subject: Re: [edk2-devel] [edk2-discuss] GSoC Proposal From: "Andrew Fish" In-reply-to: Date: Thu, 14 Apr 2022 19:42:31 -0700 Cc: "mhaeuser@posteo.de" , "discuss@edk2.groups.io" , Pedro Falcato , "adachristine18@gmail.com" , "Shi, Steven" Message-id: <47C4D916-19E4-48E8-BB81-982F9B70B5DC@apple.com> References: <865CD9EA-0EB4-4DE9-AFC6-DCB505A067EE@posteo.de> To: devel@edk2.groups.io, nathaniel.l.desimone@intel.com X-Mailer: Apple Mail (2.3693.20.0.1.32) X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.486,18.0.858 definitions=2022-04-14_07:2022-04-14,2022-04-14 signatures=0 Content-type: text/plain; charset=utf-8 Content-transfer-encoding: quoted-printable > On Apr 14, 2022, at 6:06 PM, Nate DeSimone wrote: >=20 > Hi Marvin, >=20 >> -----Original Message----- >> From: devel@edk2.groups.io On Behalf Of Marvin >> H=C3=A4user >> Sent: Thursday, April 14, 2022 12:56 AM >> To: discuss@edk2.groups.io; Desimone, Nathaniel L >> >> Cc: Pedro Falcato ; edk2-devel-groups-io >> ; adachristine18@gmail.com; Shi, Steven >> >> Subject: Re: [edk2-devel] [edk2-discuss] GSoC Proposal >>=20 >> I feel like there are so many much easier solutions to this problem that= are at >> most limited by the clear specification. The UEFI specification with reg= ards to >> booting and all of that obviously is of utmost importance. >=20 > If you have a better idea that retains compatibility with the existing UE= FI PI then I would be happy to hear it. Ultimately anything we do needs to = be a pure extension that retains compatibility with old code. Given that re= striction having the ability to coalesce all the LibraryClasses into a sing= le module and have cross-module linking seems like the best way to handle i= t to me. >=20 >> The UEFI PI specification parts that deal about internal structure, as f= ar as I know, are >> only in place to make it easy to integrate Intel IP. >=20 > Its not just for Intel. The biggest reason for it to increase the standar= dization of the boot flow across the PC ecosystem. We have learned from exp= erience that firmware is super critical to get a product out the door but i= t is also difficult to write. So we try to make it as reusable as humanly p= ossible. >=20 >> In fact, I don=E2=80=99t *know*, but I=E2=80=99m pretty sure the very st= rict separation between PEI and DXE was >> preserved mostly because MRC was 32-bit-only for a long time. Glad that >> seems to have been resolved, AMD does memory init by PSP nowadays. >=20 > Having less complex early stages chain load more complex later stages is = a common design pattern in firmware, not just UEFI. For example, your typic= al ARM system loads kinda like this: >=20 > PBL (SoC ROM) --> SBL (RAM Init) --> ARM Trust Zone --> Hypervisor --> ED= K II or U-Boot or LittleKernel (which runs android fastboot) >=20 > Comparing relative complexity I believe the Intel UEFI PI design is actua= lly pretty simple when you consider how much it gets done: >=20 > Ucode ROM --> SEC + PEI --> DXE + SMM + BDS >=20 > My biggest criticism of the design is that the strict separation between = PEI and DXE doesn't exist between DXE, SMM, and BDS =F0=9F=98=8A >=20 > There are a few reasons why PEI was 32-bit for quite some time. The bigge= st one is the code size increase, 64-bit x86 code is 20-30% larger than 32-= bit x86 code. Since the only RAM Pre-Memory code has access to is the cache= onboard the processor and for security reasons all that code has to fit wi= thin that RAM we generally do everything we can to make that image as small= as possible. Second, 64-bit requires a page table and since we desired to = keep PEI simple we tried to avoid that. Finally, the PI spec didn't allow a= 64-bit PEI until recently. MRC is 32-bit code just because that is what PE= I happens to be. Porting it to 64-bit is not terribly difficult. >=20 > Ultimately the mix of 32/64-bit does cause some difficulties. Any data st= ructures that get shared between PEI and DXE (HOBs, PCDs, etc.) need to res= olve to the same size and packing. LibraryClasses need to be written to com= pile properly in both modes. In the case of FSP API mode you need to resort= to thunking between 32 and 64-bit modes during DXE. More or less we decide= d that the costs are starting to outweigh the advantages. >=20 I=E2=80=99d also point out that x86 VMs use X64 (x86-64) PEI today and it w= orks so the 32-bit/64-bit mix has nothing to do with UEFI/PI/edk2.=20 In the PC ecosystem a single chipset family can power thousands of unique d= esigns. So the DRAM memory needs to be external, support lots of different= chipset packages(signal integrity...), support the lowest cost through the= highest cost DRAM and thousands of different board layouts. So programing = DRAM takes a masters degree in antenna design. I=E2=80=99ve seen MRC (Memor= y Reference Code) with over a MiB of DEBUG prints in it, and it literally i= s printing histograms of what it is tuning. So all this code has to run bef= ore the system has any DRAM, thus it is running using the cache as RAM. I= =E2=80=99ve not looked at the x86 architecture specs form the vendors in a = while, but back in the day they did not support page tables in ROM or pinne= d cached. Now it might work, but if it breaks your CPU vendor blames you so= you don=E2=80=99t code PEI in X64=E2=80=A6. We contributed the 1st edk2 ARM platform, Beagle Board, and It was a long t= ime ago but I seem to remember the mask ROM used a table in NOR FLASH to in= it memory and then copied an image from NOR FLASH into DRAM and jumped to i= t. So PEI is kind of not really needed and we implemented a PrePEI and jump= ed directly to DXE.=20 Given I was around back in the day when all this stuff was designed I can s= ay SEC was always a place holder for security code, as security code always= has to run 1st. PEI (Pre EFI) was designed to get DRAM programmed and then= jump to DXE. It kind of also fell in naturally to ACPI S3 flow since that = was turning memory back on. When we designed PEI we kind of though of it mo= re like a boot loader stage for the firmware that turned on memory and all = the work would happen in DXE. Then reality strikes and the existing BIOS as= sembly programmers start learning C (lots of cranky people) and they start= having to learn all about PEI to turn on memory. They had to write a big c= hunk of code for the memory init in PEI. These folks had never written any = EFI code, so to them it was easier to move a lot of the chipset init code i= nto PEI as that is the world they had to figure out to get memory turned on= . I mean why learn EFI if you don=E2=80=99t have to? So that is how we get = so much code in IA32 (i386) on some platforms. This start kind of biased fu= ture choices and how to enable non edk2 code bases=E2=80=A6. Thanks, Andrew Fish >>=20 >> For many good reasons, Linux does not provide a stable kernel API. This >> allows to easily deploy breaking changes across the entire codebase. >> Obviously, this is infeasible at a large scale when you need to integrat= e IP >> blobs, but it would already help to break the expectation that UEFI PI i= s a >> perfectly forwards- and backwards-compatible system. DXE has SetMem and >> CopyMem as part of gBS. Maybe I don=E2=80=99t like it that much as part = of the spec >> itself, but it=E2=80=99s a good idea technically. I=E2=80=99d probably o= pt to have a quickly >> accessible implementation detail of important function pointers appended= to >> PEI and DXE services, used by in-tree modules. This may break both >> forwards- and backwards-compatibility, but as it only applies to in-tree >> modules, that is fine so long as we let go of the compatibility notions.= PPIs >> and protocols are an option too of course, but they have a lookup >> performance penalty. Compared to dynamic linking, that should hopefully = be >> negligible however. >>=20 >> Absolutely optional to read, I don=E2=80=99t intend to waste anyone=E2= =80=99s time much, >> some philosophical stuff about my rationale: >>=20 >> If you started UEFI from scratch right now, would it have strictly separ= ated >> PEI and DXE? >=20 > For sure a clean slate design started today would look a lot different th= an PEI/DXE... which was a clean slate design circa 1998 =F0=9F=98=8A. In my= opinion, if we were to suddenly go back to the drawing board we would buil= d something that is much closer to a full OS now than we did back then. The= re have been cases where not being able to use interrupt handlers and not h= aving thread scheduling has prevented implementation of desired features. T= he ARM guys built LittleKernel (https://github.com/littlekernel/lk) for a l= ot of these reasons. In the data center world some have decided to go to th= e extreme of putting an entire copy of Linux in SPI so they can do a networ= k boot that downloads the OS image using BitTorrent! >=20 >> The duplication between PEI and DXE core, and by extension >> MM core, would be my most obvious place to start reducing size. I would >> probably opt for a PEI/DXE hybrid where it starts in =E2=80=9Eminimal mo= de=E2=80=9C (maybe >> think of PEI more like a microkernel here) and after memory is initialis= ed, the >> rest of DXE is loaded. And MM core would get no loading at all, this >> requirement has gladly been dropped ages ago. Just one prelinked snapsho= t >> of the address space with a relocation table and a safe self-relocator o= n entry >> (this is needed at the very least for ARM). >>=20 >> Ironically, with my idea for MM, dynamic loading would be free as everyt= hing >> is prelinked anyway. The same is true for PEI XIP, it is prelinked by na= ture. >=20 > Actually Post-Memory PEI can have non-prelinked PEIMs. And that does get = used for the PEI GOP driver. >=20 >> What I do not like is the additional dynamic linking code at load-time f= or non- >> XIP modules. Though, the more I think about it, the more I wonder whethe= r >> not the entirety of UEFI could be composed of prelinked, relocatable >> memory snapshots without traditional image loading entirely (for in-FW >> stuff). macOS has a similar concept with its =E2=80=9CKernel Collections= =E2=80=9D. Well, way >> too much off-topic now. :) >=20 > If you make the assumption that 100% of the code is compiled all at once = then yes that works. UEFI was designed so that assumption does not need to = be true. There are good use cases for it: OpROMs, generic OS loaders, netwo= rk boot, etc.=20 >=20 > Thanks, > Nate >=20 >>=20 >> Why am I explaining all this despite the fact everyone knows this will n= ever >> happen? Because I don=E2=80=99t like the notion of fixing issues of an a= lready >> overcomplicated system by adding even more complicated stuff. Especially >> when the existing overcomplicated stuff is already uncomfortably broken. >>=20 >> Best regards, >> Marvin >>=20 >>> For XIP PEI code=E2=80=A6 it will really help and would be very timely = since the >> transition of PEI from 32-bit to 64-bit is going to increase the size of= PEI by >> ~20%. >>>=20 >>> Thanks, >>> Nate >>>=20 >>> From: Pedro Falcato >>> Sent: Wednesday, April 13, 2022 11:43 AM >>> To: edk2-devel-groups-io ; Marvin H=C3=A4user >>> >>> Cc: discuss@edk2.groups.io; adachristine18@gmail.com; Desimone, >>> Nathaniel L ; Shi, Steven >>> >>> Subject: Re: [edk2-devel] [edk2-discuss] GSoC Proposal >>>=20 >>> Hi Marvin, Ada, >>>=20 >>> Some comments: >>>=20 >>> I don't think the purpose of the dynamic linker is to treat EFI as a >>> complete operating system, but to try to eliminate the static linking >>> that may be needlessly duplicating code that could instead be put in a >> single dynamic library. For instance, MdePkg and MdeModulePkg are linked >> into a *lot* of .efi, instead of being just a library. It'd be nice to s= ee some >> numbers on this (something like Google's bloaty could be run on every .e= fi >> file, in order to understand how much file space we would actually save)= . >>>=20 >>> Other comments inline. >>>=20 >>> On Wed, Apr 13, 2022 at 4:15 PM Marvin H=C3=A4user >> > wrote: >>>=20 >>> On 13. Apr 2022, at 16:38, Ada Christine >> > wrote: >>> =EF=BB=BFi was replying via the groups.io web interfa= ce, I'm >>> guessing that messed up the thread? i haven't used mailing lists >>> before and don't know how they work. I'll use my mail client from here = on. >>>=20 >>> I'm on board with not treating EFI as an operating system. the more i >>> think about it the more it looks like scope creep. >>>=20 >>> Agreed. >>>=20 >>>=20 >>> I'm not quite as enthusiastic >>> about it as i was at first glance. >>>=20 >>> I'm still keen on doing my gsoc proposal for edk, though, and even if >>> this task and the acpica application are decided to be out of scope >>> unit testing, >>>=20 >>> How about fuzz-testing? This is also something edk2 needs quite badly. = At >> Acidanthera, we compile edk2 code in userspace outside the edk2 build >> system and fuzz with dummy applications. >>>=20 >>> Note: fuzzing is also part of the LLVM instrumentation suite (see >> https://llvm.org/docs/LibFuzzer.html) and is something I could happily >> mentor. >>> clang integration >>>=20 >>> Pedro and Vitaly are looking for someone to finish ASan: >>> https://edk2.groups.io/g/devel/topic/90010978#87991 >>> There are working UBSan concepts, but they also need to be mainlined. >>>=20 >>> Is Vitaly going to be a mentor? I was assuming it was going to be me an= d >> some other, more senior, mentor (possibly Steven Shi, which I included i= n >> the task). >>> Anyway, re: ASAN, if the project includes ASAN, UBSAN and possibly >>> some other sanitizer it's quite possible that it could be considered a = large >> project (which means more hours but a larger stipend too). Fuzzing + >> coverage could be very nice additions to this project idea. >>> Also, is stress-testing a decent idea? >>>=20 >>>=20 >>> and source-level debugging are all relevant to my interests. >>>=20 >>> how about your ideas for security stuff? >>>=20 >>> I want the entirety of MM to leverage SmmMemLib and to support SMAP. >> SmmMemLib would then handle UEFI->MMRAM and BaseMemoryLib would >> only work on MMRAM. Also evaluation of how to best avoid pointers in MM >> communication buffers would be nice. >>>=20 >>> There also is a bunch of other stuff, like working out moving a part of >> CpuDxe into DxeCore to have memory protection live immediately, memory >> protection in PEI, a replacement for the TE format (it=E2=80=99s buggy a= nd most >> platforms mostly abandoned it over various issues), and alternatives to >> guarding critical code with SMM (like allowing NVRAM commits only as par= t >> of a reboot). >>>=20 >>> I personally find all of those projects very important, but I cannot >>> promise many people agree. Especially those that impose global changes >>> (most notably the TE replacement) may be very tedious to submit. >>> Gladly, I believe you can submit multiple proposals (?) >>>=20 >>> Best regards, >>> Marvin >>>=20 >>>=20 >>> I'm not very knowledgeable about >>> trusted platform or secure boot but I'm willing to learn whatever is >>> necessary to get something spun up for my proposal. >>>=20 >>> On Wed, Apr 13, 2022, 12:05 Marvin H=C3=A4user >> > wrote: >>>=20 >>>=20 >>> Do you use the =E2=80=9Creply all=E2=80=9D option in your mail client? = Looks like my >>> CCs have been dropped again. Comments inline. >>>=20 >>> On 13. Apr 2022, at 12:54, Ada Christine >>> > >>> wrote: >>> =EF=BB=BFHi, Marvin >>>=20 >>> Its similarity to my own latest experiment is the key to what grabbed >>> my attention. I have no particular use case in mind for it, but I see >>> its potential for anybody developing larger applications in that when >>> a library is changed there's no need to distribute a new version of >>> the whole binary, just the relevant library module. >>>=20 >>> I really do not like the trend of treating UEFI as a full-fledged OS - >>> it is not. The most used UEFI applications, OS loaders, are really not >>> that huge and are distributed as part of the OS image anyway. Even for >>> less used applications, you will always get a full snapshot anyhow. >>> Gladly we don=E2=80=99t have auto-update and package management yet. :) >>>=20 >>>=20 >>> I slept on it and it occurred to me that the whole thing could operate >>> similarly to the shell protocol in that the linker/loader is itself an >>> application that does a LoadImage() on the application needing dynamic >>> linking facilities. >>>=20 >>> That would mean the linker itself is shipped with every application >>> that requires it? Otherwise it doesn=E2=80=99t make much sense for it t= o be an >>> app and below=E2=80=99s problems apply. >>>=20 >>> If however the whole plan is making the linker as a DXE and including >>> it with the firmware, that I'm not quite as sure about. That would >>> necessarily tie any applications using dynamic linking to TianoCore or >>> any firmware distribution that derives from it. >>>=20 >>> I think that was the idea referred to as =E2=80=9Cedk2 core=E2=80=9D by= Steven, but >>> I=E2=80=99d like to hear his proposal to be sure. Virtually everyone us= es >>> edk2, so that itself is not the problem, but versioning is. Vendors >>> are slow to update their snapshots or have just given up doing that >>> entirely. Distributing it for external applications like OS loaders >>> would mean this can be leveraged probably no earlier than 10 years >>> from now. And for in-firmware things, I have a hard time thinking about= a >> use-case that outweighs the drawbacks. >>>=20 >>>=20 >>> To shift the topic slightly back to GSoC, however, I'm willing to work >>> on other items on the task list. Unit testing and an ACPICA >>> application are the alternative projects I had thought about. I need >>> to choose fairly soon as the proposal deadline is next Tuesday. I know >>> a tiny bit about porting ACPICA as I also have plans to incorporate it = into my >> own project. >>>=20 >>> I have a few more ideas for security stuff, but Nate did not confirm >>> them as appropriate yet and I=E2=80=99m not here to drive you away from= this >>> specific task (or the others). However, I=E2=80=99m still curious and >>> concerned. :) >>>=20 >>> Best regards, >>> Marvin >>>=20 >>>=20 >>>=20 >>>=20 >>>=20 >>>=20 >>>=20 >>>=20 >>> -- >>> Pedro Falcato >>>=20 >>>=20 >>>=20 >>>=20 >>>=20 >>=20 >>=20 >>=20 >>=20 >>=20 >=20 >=20 >=20 >=20 >=20 >=20