[edk2-devel] GSoC 2021 (MinPlatform, Ext2, ACPICA, etc)

[edk2-devel] GSoC 2021 (MinPlatform, Ext2, ACPICA, etc)

[pedro.falcato]

[-- Attachment #1: Type: text/plain, Size: 1483 bytes --]

Hi everyone!

I'm Pedro Falcato, a student from FCT Nova in Lisbon, Portugal. I've gotten
a bunch of experience over the years with C/C++, x86 in general and
UEFI/ACPI with my hobby OS/kernel development, and I've got to say, I'm
quite interested in some of the projects you've got here.

So, a few questions:

1) What entails building a MinPlatform board port for any board whatsoever?
I've seen Kaaira wants to do the Qemu port, I would love to do something
like that but for the RPi or some real motherboard, but I fear it might be
too difficult?

2) How much knowledge of EFI firmware internals do you need? With my EFI
bootloader development over the years I already have a firm hand on how the
external-facing API looks like, but I have to say I haven't really read the
parts of the spec that describe the driver and internal APIs, so to speak.

3) Isn't there already an ACPICA port for UEFI environments? What stops us
from going one step further and also build the rest of the "user-space"
utilities?

4) How's the status of the ext2 driver? How different do Tianocore
filesystem implementations look from the standard-ish kernel interfaces you
can see in Linux, *BSD, etc? I'm also quite interested in this one because
I've written a read/write ext2 driver before, so the concepts are kind-of
fresh in my head.

I hope you folks can answer my questions so I can figure out what project I
want to work on! :)

Looking forward to working in Tianocore!

Thanks,
Pedro Falcato

[-- Attachment #2: Type: text/html, Size: 1879 bytes --]

Re: [edk2-devel] GSoC 2021 (MinPlatform, Ext2, ACPICA, etc)

[Andrew Fish]

[-- Attachment #1: Type: text/plain, Size: 2090 bytes --]



> On Mar 12, 2021, at 5:07 AM, Pedro Falcato <pedro.falcato@gmail.com> wrote:
> 
> Hi everyone!
> 
> I'm Pedro Falcato, a student from FCT Nova in Lisbon, Portugal. I've gotten a bunch of experience over the years with C/C++, x86 in general and UEFI/ACPI with my hobby OS/kernel development, and I've got to say, I'm quite interested in some of the projects you've got here.
> 
> So, a few questions:
> 
> 1) What entails building a MinPlatform board port for any board whatsoever? I've seen Kaaira wants to do the Qemu port, I would love to do something like that but for the RPi or some real motherboard, but I fear it might be too difficult?
> 
> 2) How much knowledge of EFI firmware internals do you need? With my EFI bootloader development over the years I already have a firm hand on how the external-facing API looks like, but I have to say I haven't really read the parts of the spec that describe the driver and internal APIs, so to speak.
> 

Pedro,

Knowing UEFI is a really good baseline. Tianocore is UEFI + edk2 Build System + UEFI Platform initialization Specification [1],

There is some training here: [2]

Some of us also sit on the spec committees so you can usually get good quality answers to questions on the mailing list. 

[1] https://uefi.org/specifications

[2] https://github.com/tianocore/tianocore.github.io/wiki/UEFI-EDKII-Learning-Dev

Thanks,

Andrew Fish

> 3) Isn't there already an ACPICA port for UEFI environments? What stops us from going one step further and also build the rest of the "user-space" utilities?
> 
> 4) How's the status of the ext2 driver? How different do Tianocore filesystem implementations look from the standard-ish kernel interfaces you can see in Linux, *BSD, etc? I'm also quite interested in this one because I've written a read/write ext2 driver before, so the concepts are kind-of fresh in my head.
> 
> I hope you folks can answer my questions so I can figure out what project I want to work on! :)
> 
> Looking forward to working in Tianocore!
> 
> Thanks,
> Pedro Falcato
> 


[-- Attachment #2: Type: text/html, Size: 3599 bytes --]

Re: [edk2-devel] GSoC 2021 (MinPlatform, Ext2, ACPICA, etc)

[Pedro Falcato]

[-- Attachment #1: Type: text/plain, Size: 232 bytes --]

Hi Andrew,

Thanks for the quick response! I'll be sure to check out the training, it looks very interesting and useful. I'll also keep reading the specs, the details might be important to know.

Thanks again,

Pedro Falcato

[-- Attachment #2: Type: text/html, Size: 924 bytes --]

Re: [edk2-devel] GSoC 2021 (MinPlatform, Ext2, ACPICA, etc)

[Nate DeSimone]

[-- Attachment #1: Type: text/plain, Size: 9640 bytes --]

Hi Pedro,

Great to meet you and welcome to the TianoCore project! Glad you hear you are interested! A MinPlatform board port can go in two wildly different directions, depending on the answer to one question: Does MinPlatform currently have support for the Processor/Chipset/SoC used by the motherboard?

If yes, then building the new board port only needs to focus on the things that change between each motherboard. For a good example of that of what to expect to be different between motherboards, take a look at the differences between https://github.com/tianocore/edk2-platforms/tree/master/Platform/Intel/WhiskeylakeOpenBoardPkg/WhiskeylakeURvp and https://github.com/tianocore/edk2-platforms/tree/master/Platform/Intel/WhiskeylakeOpenBoardPkg/UpXtreme.

A few things stick out:


  1.  The WhiskeyLakeURvp has DIMM slots for its DRAM, whereas the UpXtreme board has soldered down DRAM. This changes the methodology for getting the SPD Data (https://en.wikipedia.org/wiki/Serial_presence_detect). For DRAM in DIMM slots, one must read the SPD data by sending read commands over the SMBus (https://en.wikipedia.org/wiki/System_Management_Bus). Each physical DIMM slot will have a different SMBus address where the SPD chip is located, and different motherboards use different addresses. Building the new board port requires the implementer to know what SMBus addresses to use. For an example of this, see the following snippet from https://github.com/tianocore/edk2-platforms/blob/master/Platform/Intel/WhiskeylakeOpenBoardPkg/WhiskeylakeURvp/Library/BoardInitLib/BoardSaInitPreMemLib.c:

PcdSet32S (PcdMrcSpdData, 0);
PcdSet16S (PcdMrcSpdDataSize, 0);
PcdSet8S (PcdMrcSpdAddressTable0, 0xA0);
PcdSet8S (PcdMrcSpdAddressTable1, 0xA2);
PcdSet8S (PcdMrcSpdAddressTable2, 0xA4);
PcdSet8S (PcdMrcSpdAddressTable3, 0xA6);

For soldered down DRAM, typically there is no SPD chip. So the MinPlatform board code needs to carry a copy of the SPD data for the DRAM. That creates the problem that different DRAM vendors need different SPD data, and the motherboard manufacturer will often need to purchase DRAM from more than one memory vendor. To address this, the board code needs some way to detect which memory vendor was used on the current board and choose the correct SPD data. The UpXtreme board port gives a good example of this. Here is the same snippet of code from https://github.com/tianocore/edk2-platforms/blob/master/Platform/Intel/WhiskeylakeOpenBoardPkg/UpXtreme/Library/BoardInitLib/BoardSaInitPreMemLib.c:

BomId = PcdGet8(PcdBoardBomId);
DEBUG ((DEBUG_INFO, "Up Xtreme Bom ID 0x%x\n",BomId));
if ((BomId & BIT1) == BIT1) {
  PcdSet32S (PcdMrcSpdData, (UINTN) mUpXtremeSamsungDdr4Spd);
  PcdSet16S (PcdMrcSpdDataSize, mUpXtremeSamsungDdr4SpdSize);
  DEBUG ((DEBUG_INFO, "Using Xtreme SPD Samsung Ddr4\n"));
} else {
  PcdSet32S (PcdMrcSpdData, (UINTN) mUpXtremeSkhynixDdr4Spd);
  PcdSet16S (PcdMrcSpdDataSize, mUpXtremeSkhynixDdr4SpdSize);
  DEBUG ((DEBUG_INFO, "Using Xtreme SPD Skhynix Ddr4\n"));
}
PcdSet8S (PcdMrcSpdAddressTable0, 0);
PcdSet8S (PcdMrcSpdAddressTable1, 0);
PcdSet8S (PcdMrcSpdAddressTable2, 0);
PcdSet8S (PcdMrcSpdAddressTable3, 0);

In this case, the SMBus addresses are set to zero and the SPD data is provided directly. The board code chooses between the different SPD data payloads based on a BOM (Bill of Materials) identifier. For the UpXtreme, the BomId is initialized by reading the value of 6 GPIO pins and constructing an integer, see https://github.com/tianocore/edk2-platforms/blob/master/Platform/Intel/WhiskeylakeOpenBoardPkg/UpXtreme/Library/BoardInitLib/PeiUpXtremeDetect.c:

// Sample the GPIO pin level
for (Index = 0; Index < NumberOfGpios; Index++) {
  Status = GpioGetInputValue (mUpxGpioBomPad[Index], &GpioData);
  if (EFI_ERROR(Status)) {
    break;
  }
  BomId = (BomId << 1) + (GpioData & 1);
}
if (Index == NumberOfGpios) {
  PcdSet8S(PcdBoardBomId, BomId);
}

  2.  The usage of the GPIO pins provided by the chipset will often vary between motherboard designs. This requires board code to initialize the pins differently. Usually this is done by writing a table describing the GPIO usage for the board. For an example of different GPIO tables for each board, see https://github.com/tianocore/edk2-platforms/blob/master/Platform/Intel/WhiskeylakeOpenBoardPkg/WhiskeylakeURvp/Library/BoardInitLib/GpioTableWhiskeylakeUDdr4Rvp.c and https://github.com/tianocore/edk2-platforms/blob/master/Platform/Intel/WhiskeylakeOpenBoardPkg/UpXtreme/Library/BoardInitLib/GpioTableUpXtreme.c.

  3.  One of the uses of GPIO pins is to provide interrupt signaling, the different GPIO tables will also result in the need for different interrupt routing. This requires the GPIO pins that are used for interrupts to be programmed correctly. And the interrupt routing needs to be reflected in the FADT and MADT ACPI tables. MinPlatformPkg contains the code to generate these ACPI tables based on PCD values: https://github.com/tianocore/edk2-platforms/blob/master/Platform/Intel/MinPlatformPkg/Acpi/AcpiTables/AcpiPlatform.c. But the PCD values may need to be adjusted.

  4.  The policy data provided to the FSP needs to be adjusted according to the board design. Motherboards will have different configurations for the number of PCIe/USB/SATA ports provided. Intel chipsets have a design called “Flex I/O Architecture” (FIA). FIA allows a single HSIO (High Speed I/O) lane to behave as either a USB, PCIe, or SATA lane. Note that a lane in a combination of two pins; two pins are needed for differential voltage signaling. The FIA configuration varies between motherboards, so the configuration needs to be reflected in the FSP input values.

The following Processor/Chipset/SoC are currently supported by MinPlatform, so this type of project would need to choose a motherboard designed for one of these:


  *   Kaby Lake
  *   Whiskey Lake
  *   Comet Lake
  *   Tiger Lake

If MinPlatform currently does NOT support for the Processor/Chipset/SoC used by the motherboard, then a new *OpenBoardPkg will need to be constructed. For the purposes of a GSoC project, I would not recommend attempting this unless the code for the Processor/Chipset/SoC and motherboard already exists somewhere else in TianoCore. Raspberry Pi and Qemu are good examples where this scenario exists, as we already have code to support both of those targets. In that case, Creating the OpenBoardPkg would be an exercise in adapting that existing code to the MinPlatform architecture: https://tianocore-docs.github.io/edk2-MinimumPlatformSpecification/draft/2_architecture/#2-architecture.

MinPlatform is designed to limit the amount of EFI firmware internals you need in order to implement a MinPlatform board port. In general, we try to create generic code that provides those details whenever possible and limit the implementation work to purely the changes needed to get a new motherboard working.

The ACPICA tools are currently only used during the build process to compile the firmware image. They are not executable as UEFI pre-OS applications, which could be useful in some cases. For example, dumping a disassembly of the ACPI code to the UEFI shell might be a useful debugging tool in some cases.

To my knowledge, I don’t believe anyone has worked on the ext2 driver since GSoC 2011. While some progress was made during GSoC 2011, it was not completed. The code from that project is available here: https://github.com/GunioRobot/Ext2Pkg At this point, the filesystem driver would probably need to support ext4 to be really useful. I haven’t done a detailed investigation on how much work that would be. Honestly it should like you might have a better idea of it than I do 😊.

Sorry for the long email, but I hope it helps. Finally I'd like to reiterate... Welcome to the project!

With Best Regards,
Nate

From: devel@edk2.groups.io <devel@edk2.groups.io> On Behalf Of Pedro Falcato
Sent: Friday, March 12, 2021 5:08 AM
To: devel@edk2.groups.io
Subject: [edk2-devel] GSoC 2021 (MinPlatform, Ext2, ACPICA, etc)

Hi everyone!

I'm Pedro Falcato, a student from FCT Nova in Lisbon, Portugal. I've gotten a bunch of experience over the years with C/C++, x86 in general and UEFI/ACPI with my hobby OS/kernel development, and I've got to say, I'm quite interested in some of the projects you've got here.

So, a few questions:

1) What entails building a MinPlatform board port for any board whatsoever? I've seen Kaaira wants to do the Qemu port, I would love to do something like that but for the RPi or some real motherboard, but I fear it might be too difficult?

2) How much knowledge of EFI firmware internals do you need? With my EFI bootloader development over the years I already have a firm hand on how the external-facing API looks like, but I have to say I haven't really read the parts of the spec that describe the driver and internal APIs, so to speak.

3) Isn't there already an ACPICA port for UEFI environments? What stops us from going one step further and also build the rest of the "user-space" utilities?

4) How's the status of the ext2 driver? How different do Tianocore filesystem implementations look from the standard-ish kernel interfaces you can see in Linux, *BSD, etc? I'm also quite interested in this one because I've written a read/write ext2 driver before, so the concepts are kind-of fresh in my head.

I hope you folks can answer my questions so I can figure out what project I want to work on! :)

Looking forward to working in Tianocore!

Thanks,
Pedro Falcato


[-- Attachment #2: Type: text/html, Size: 38760 bytes --]

Re: [edk2-devel] GSoC 2021 (MinPlatform, Ext2, ACPICA, etc)

[Pedro Falcato]

[-- Attachment #1: Type: text/plain, Size: 1471 bytes --]

Hi Nate!

Sorry for taking so long to get back to you, I've been a bit busy and I'm not too used to so many emails in my inbox :)

So, if I'm getting this right, essentially what is proposed in the MinPlatform board ports is to refactor the existing board code into an OpenBoardPkg that uses MinPlatform to reuse more generic code? I was thinking about getting a Raspberry Pi and doing the MinPlatform port for that, although honestly I'm not too inclined for that option anymore.

Honestly, I'm looking more towards the ext2/4 drivers now. I've been poking a little at the build system and how the driver model is supposed to work and I think I more or less got the idea. Here's the link to my ext2 test repo, if you're curious: https://github.com/heatd/edk2-ext. Note that it doesn't quite do anything right now, it's missing all sorts of features and testing and what's there is mostly driver model and build system boilerplate that was pieced together by looking at the UEFI spec and the FatPkg code, plus some of my own ext2 headers.

With regards to ext4, yes, it sounds like the better option at the moment, although I'm not terribly familiar with it. I do have some questions though:

* What are the standards for filesystem driver performance? Is a page/disk cache a necessity for the driver? I would assume the FS driver has some substancial footprint in the overall boot time.
* Is the read-only behaviour still the target?
Thanks,
Pedro Falcato

[-- Attachment #2: Type: text/html, Size: 1623 bytes --]

Re: [edk2-devel] GSoC 2021 (MinPlatform, Ext2, ACPICA, etc)

[Nate DeSimone]

[-- Attachment #1: Type: text/plain, Size: 6428 bytes --]

Hi Pedro,

No problem, I’ve been really busy recently too 😊. I definitely recommend setting up filtering rules for the edk2 mailing list(s) because there is a LOT of volume. Personally, set up my filter rules so that PATCH emails go in a separate folder, that makes the list much more readable.

For the Raspberry Pi… yeah it is more of a refactoring activity. For some of the other board ports it is more interesting. We already went over the x86 board porting, which I think would be an interesting and unique project, the problem is finding a suitable board… and most x86 boards are expensive. The Qemu OpenBoard also has a bit more meat to it I think. Since it involves figuring out how to merge ArmVirtPkg and OvmfPkg in a way that makes sense, it probably involves building some new advanced features. I think it is possible to spice up the Raspberry Pi project a bit though. Maybe add support for the Raspberry Pi Zero? Right now we only support the Raspberry Pi 3 & 4.

The ext2/4 disk driver would certainly also be interesting! My understanding is that ext4 is pretty much the same filesystem as ext2, they just added some features on top of ext2. I took a quick peek and your boilerplate and it looks like a good start. The one thing you will have to be VERY careful about is keeping TianoCore GPL-free. You need to be very careful to not read a single line of Linux kernel source code. You can read the kernel’s ext2/3/4 documentation but do NOT read the kernel’s source code. I think that should be doable since FreeBSD 12 (circa 2018) now has full read/write support for ext4 in their ext2 driver and has had a GPL-free ext2 driver for quite some time (FreeBSD 9 circa 2012.) You are welcome to read (and even use) the FreeBSD source code (as long as it is newer than 2012 of course.) In your proposal, I would recommend you set an achievable target. Maybe promise read-only as a baseline success criteria and then make writing a stretch goal. I strongly prefer that everyone ends up with a successful project.

I do this this project would be useful. If full write support is achieved GPL-free, this driver could be integrated into the firmware distributed in OEM motherboards. This would make it really easy for us to boot Linux kernels directly from the UEFI boot manager, eliminating the need for GRUB and a separate EFI system partition for loading Linux.

Our disk I/O subsystem does not need to reach the performance level of an operating system. We usually only need to load a few MBs from disk before we handoff to the OS kernel. In general, UEFI is designed to favor simplicity over performance. The biggest bottleneck is the fact that UEFI uses exclusively polling I/O and does not have any interrupt handlers except for the timer interrupt. My guess is a disk cache is probably overkill because the filesystem driver is unlikely to be the bottleneck.

For comparison, currently our only filesystem driver is the FAT32 driver, and the FAT32 driver is definitely fast enough for our purposes. On an Intel Tiger Lake platform with the EFI System Partition stored on a NVMe SSD, it takes 60ms (milliseconds) to initialize the NVMe driver, enumerate the partition table, and mount the FAT32 filesystem. Out of that 60ms, only 1.4ms is directly consumed by the FAT32 driver, the rest of it is initialization of the lower level drivers. After initialization is complete, it takes the FAT32 driver 3.5ms to load bootmgfw.efi from disk to memory. To put that into perspective, after bootmgfw.efi is loaded into memory it takes the DXE core 13.5ms to verify that the signature on the file is valid and was signed by a code signing certificate that is in the UEFI secure boot trusted certificate list as well as processing PE/COFF relocation fixups needed to execute the file.

Thinking about performance, bootmgfw.efi (the Windows boot manager) is a little over 1.5MB, so that works out to a disk I/O subsystem performance of roughly 450MB/sec… while this is not the ~1000MB/sec you would expect from a high performance NVMe driver, it is still not bad considering it is being done using polling I/O and the EDK II NVMe driver doesn’t build the largest possible scatter gather list that the NVMe controller can handle.

If you look at all of this at the high level, we ended up spending 64ms on disk I/O, when the entire Tiger Lake UEFI firmware takes 1.74 sec to run. So, only ~4% of the system boot time was spent on disk I/O, so right now the filesystem drivers are a small drop in the bucket of the overall boot time. As long as your driver reads and writes at the same speed or better than FatPkg, there would be no concerns.

Hope that helps,
Nate

From: Pedro Falcato <pedro.falcato@gmail.com>
Sent: Tuesday, March 23, 2021 7:17 AM
To: Desimone, Nathaniel L <nathaniel.l.desimone@intel.com>; devel@edk2.groups.io
Subject: Re: [edk2-devel] GSoC 2021 (MinPlatform, Ext2, ACPICA, etc)

Hi Nate!

Sorry for taking so long to get back to you, I've been a bit busy and I'm not too used to so many emails in my inbox :)

So, if I'm getting this right, essentially what is proposed in the MinPlatform board ports is to refactor the existing board code into an OpenBoardPkg that uses MinPlatform to reuse more generic code? I was thinking about getting a Raspberry Pi and doing the MinPlatform port for that, although honestly I'm not too inclined for that option anymore.

Honestly, I'm looking more towards the ext2/4 drivers now. I've been poking a little at the build system and how the driver model is supposed to work and I think I more or less got the idea. Here's the link to my ext2 test repo, if you're curious: https://github.com/heatd/edk2-ext. Note that it doesn't quite do anything right now, it's missing all sorts of features and testing and what's there is mostly driver model and build system boilerplate that was pieced together by looking at the UEFI spec and the FatPkg code, plus some of my own ext2 headers.

With regards to ext4, yes, it sounds like the better option at the moment, although I'm not terribly familiar with it. I do have some questions though:

  1.  What are the standards for filesystem driver performance? Is a page/disk cache a necessity for the driver? I would assume the FS driver has some substancial footprint in the overall boot time.
  2.  Is the read-only behaviour still the target?
Thanks,
Pedro Falcato

[-- Attachment #2: Type: text/html, Size: 9760 bytes --]

Re: [edk2-devel] GSoC 2021 (MinPlatform, Ext2, ACPICA, etc)

[Bret Barkelew]

[-- Attachment #1.1: Type: text/plain, Size: 7258 bytes --]

I agree that this does seem like a fun and useful project. Great to have you working with us, Pedro!

- Bret

From: Nate DeSimone via groups.io<mailto:nathaniel.l.desimone=intel.com@groups.io>
Sent: Sunday, March 28, 2021 3:31 PM
To: Pedro Falcato<mailto:pedro.falcato@gmail.com>; devel@edk2.groups.io<mailto:devel@edk2.groups.io>
Subject: [EXTERNAL] Re: [edk2-devel] GSoC 2021 (MinPlatform, Ext2, ACPICA, etc)

Hi Pedro,

No problem, I’ve been really busy recently too 😊. I definitely recommend setting up filtering rules for the edk2 mailing list(s) because there is a LOT of volume. Personally, set up my filter rules so that PATCH emails go in a separate folder, that makes the list much more readable.

For the Raspberry Pi… yeah it is more of a refactoring activity. For some of the other board ports it is more interesting. We already went over the x86 board porting, which I think would be an interesting and unique project, the problem is finding a suitable board… and most x86 boards are expensive. The Qemu OpenBoard also has a bit more meat to it I think. Since it involves figuring out how to merge ArmVirtPkg and OvmfPkg in a way that makes sense, it probably involves building some new advanced features. I think it is possible to spice up the Raspberry Pi project a bit though. Maybe add support for the Raspberry Pi Zero? Right now we only support the Raspberry Pi 3 & 4.

The ext2/4 disk driver would certainly also be interesting! My understanding is that ext4 is pretty much the same filesystem as ext2, they just added some features on top of ext2. I took a quick peek and your boilerplate and it looks like a good start. The one thing you will have to be VERY careful about is keeping TianoCore GPL-free. You need to be very careful to not read a single line of Linux kernel source code. You can read the kernel’s ext2/3/4 documentation but do NOT read the kernel’s source code. I think that should be doable since FreeBSD 12 (circa 2018) now has full read/write support for ext4 in their ext2 driver and has had a GPL-free ext2 driver for quite some time (FreeBSD 9 circa 2012.) You are welcome to read (and even use) the FreeBSD source code (as long as it is newer than 2012 of course.) In your proposal, I would recommend you set an achievable target. Maybe promise read-only as a baseline success criteria and then make writing a stretch goal. I strongly prefer that everyone ends up with a successful project.

I do this this project would be useful. If full write support is achieved GPL-free, this driver could be integrated into the firmware distributed in OEM motherboards. This would make it really easy for us to boot Linux kernels directly from the UEFI boot manager, eliminating the need for GRUB and a separate EFI system partition for loading Linux.

Our disk I/O subsystem does not need to reach the performance level of an operating system. We usually only need to load a few MBs from disk before we handoff to the OS kernel. In general, UEFI is designed to favor simplicity over performance. The biggest bottleneck is the fact that UEFI uses exclusively polling I/O and does not have any interrupt handlers except for the timer interrupt. My guess is a disk cache is probably overkill because the filesystem driver is unlikely to be the bottleneck.

For comparison, currently our only filesystem driver is the FAT32 driver, and the FAT32 driver is definitely fast enough for our purposes. On an Intel Tiger Lake platform with the EFI System Partition stored on a NVMe SSD, it takes 60ms (milliseconds) to initialize the NVMe driver, enumerate the partition table, and mount the FAT32 filesystem. Out of that 60ms, only 1.4ms is directly consumed by the FAT32 driver, the rest of it is initialization of the lower level drivers. After initialization is complete, it takes the FAT32 driver 3.5ms to load bootmgfw.efi from disk to memory. To put that into perspective, after bootmgfw.efi is loaded into memory it takes the DXE core 13.5ms to verify that the signature on the file is valid and was signed by a code signing certificate that is in the UEFI secure boot trusted certificate list as well as processing PE/COFF relocation fixups needed to execute the file.

Thinking about performance, bootmgfw.efi (the Windows boot manager) is a little over 1.5MB, so that works out to a disk I/O subsystem performance of roughly 450MB/sec… while this is not the ~1000MB/sec you would expect from a high performance NVMe driver, it is still not bad considering it is being done using polling I/O and the EDK II NVMe driver doesn’t build the largest possible scatter gather list that the NVMe controller can handle.

If you look at all of this at the high level, we ended up spending 64ms on disk I/O, when the entire Tiger Lake UEFI firmware takes 1.74 sec to run. So, only ~4% of the system boot time was spent on disk I/O, so right now the filesystem drivers are a small drop in the bucket of the overall boot time. As long as your driver reads and writes at the same speed or better than FatPkg, there would be no concerns.

Hope that helps,
Nate

From: Pedro Falcato <pedro.falcato@gmail.com>
Sent: Tuesday, March 23, 2021 7:17 AM
To: Desimone, Nathaniel L <nathaniel.l.desimone@intel.com>; devel@edk2.groups.io
Subject: Re: [edk2-devel] GSoC 2021 (MinPlatform, Ext2, ACPICA, etc)

Hi Nate!

Sorry for taking so long to get back to you, I've been a bit busy and I'm not too used to so many emails in my inbox :)

So, if I'm getting this right, essentially what is proposed in the MinPlatform board ports is to refactor the existing board code into an OpenBoardPkg that uses MinPlatform to reuse more generic code? I was thinking about getting a Raspberry Pi and doing the MinPlatform port for that, although honestly I'm not too inclined for that option anymore.

Honestly, I'm looking more towards the ext2/4 drivers now. I've been poking a little at the build system and how the driver model is supposed to work and I think I more or less got the idea. Here's the link to my ext2 test repo, if you're curious: https://github.com/heatd/edk2-ext<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fheatd%2Fedk2-ext&data=04%7C01%7CBret.Barkelew%40microsoft.com%7Cffc5fb1e97b14ea49f1408d8f2393000%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637525674730243936%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=lfDmN9bXAXqoAHaSM3xHP%2BJGpa9BOoF7dQ5HWTSNEPI%3D&reserved=0>. Note that it doesn't quite do anything right now, it's missing all sorts of features and testing and what's there is mostly driver model and build system boilerplate that was pieced together by looking at the UEFI spec and the FatPkg code, plus some of my own ext2 headers.

With regards to ext4, yes, it sounds like the better option at the moment, although I'm not terribly familiar with it. I do have some questions though:

  1.  What are the standards for filesystem driver performance? Is a page/disk cache a necessity for the driver? I would assume the FS driver has some substancial footprint in the overall boot time.
  2.  Is the read-only behaviour still the target?
Thanks,
Pedro Falcato



[-- Attachment #1.2: Type: text/html, Size: 12109 bytes --]

[-- Attachment #2: F84348EFA8CF40C79178A6087D604B5E.png --]
[-- Type: image/png, Size: 140 bytes --]

Re: [edk2-devel] GSoC 2021 (MinPlatform, Ext2, ACPICA, etc)

[Pedro Falcato]

[-- Attachment #1: Type: text/plain, Size: 391 bytes --]

Hi Nate,

Everything you said sounds great :))
I did end up going with ext4 as a target, as I find it's the more useful and more interesting project, the only thing I'm not too sure of is whether or not to add write support as one of the project's targets.
Anyway, I've just shared my draft with Tianocore, feel free give out some much wanted(and needed!) feedback!

Thanks,

Pedro

[-- Attachment #2: Type: text/html, Size: 423 bytes --]

Re: [edk2-devel] GSoC 2021 (MinPlatform, Ext2, ACPICA, etc)

[Pedro Falcato]

[-- Attachment #1: Type: text/plain, Size: 175 bytes --]

Hi Bret,

I've shared the project's draft with Tianocore, take a look and give out some feedback if you want to :)
Thanks, it's great to work with you folks too!

Pedro

[-- Attachment #2: Type: text/html, Size: 195 bytes --]

Re: [edk2-devel] GSoC 2021 (MinPlatform, Ext2, ACPICA, etc)

[Nate DeSimone]

[-- Attachment #1: Type: text/plain, Size: 1139 bytes --]

Hi Pedro,

Happy to hear our conversations here have helped 😊.

The ext4 project sounds good! I’d recommend putting write support on as a “if I have time at the end” stretch goal. Just getting read working and stable would be a great accomplishment. GSoC does end in August… but you can continue working on your driver afterwards. I don’t want you to find yourself in a situation in August were you have more work to do than time remaining and end up not getting paid!

Thanks,
Nate

From: Pedro Falcato <pedro.falcato@gmail.com>
Sent: Wednesday, March 31, 2021 1:43 PM
To: Desimone, Nathaniel L <nathaniel.l.desimone@intel.com>; devel@edk2.groups.io
Subject: Re: [edk2-devel] GSoC 2021 (MinPlatform, Ext2, ACPICA, etc)

Hi Nate,

Everything you said sounds great :))
I did end up going with ext4 as a target, as I find it's the more useful and more interesting project, the only thing I'm not too sure of is whether or not to add write support as one of the project's targets.
Anyway, I've just shared my draft with Tianocore, feel free give out some much wanted(and needed!) feedback!

Thanks,

Pedro

[-- Attachment #2: Type: text/html, Size: 3368 bytes --]

Re: [edk2-devel] GSoC 2021 (MinPlatform, Ext2, ACPICA, etc)

[Pedro Falcato]

[-- Attachment #1: Type: text/plain, Size: 368 bytes --]

Hi,

Sounds great! You may be right about that, although I believe I read somewhere that you need to set a milestone for each evaluation? If so, I'm out of ideas as to what can be tangible enough to be a milestone; if not, specifying a read-only driver as project's objective and having write support as a stretch goal is certainly the way to go.

Thanks,
Pedro

[-- Attachment #2: Type: text/html, Size: 388 bytes --]

Re: [edk2-devel] GSoC 2021 (MinPlatform, Ext2, ACPICA, etc)

[Andrew Fish]

[-- Attachment #1: Type: text/plain, Size: 1925 bytes --]



> On Apr 4, 2021, at 3:50 AM, Pedro Falcato <pedro.falcato@gmail.com> wrote:
> 
> Hi,
> 
> Sounds great! You may be right about that, although I believe I read somewhere that you need to set a milestone for each evaluation? If so, I'm out of ideas as to what can be tangible enough to be a milestone; if not, specifying a read-only driver as project's objective and having write support as a stretch goal is certainly the way to go.
> 

Pedro,

Just an idea but but you could have a milestone to feature complete  of the driver, and have a milestone to develop a test strategy, and test suite to validate the driver. Documentation could also be a milestone. For example I’ve worked on some proprietary EFI File System drivers (HFS+, apfs) and we chose not to implement certain features like compressed files. So you would want to have good documentation about that and tests to make sure the driver gracefully fails in these cases. The test cases don’t only need to be tests as you could create a file system disk image that you could mount in OVMF that contains a lot of file system end case constructs (max size file, links, etc.) to make testing easier. It is not just about leaving a driver behind, but a driver that is easy to maintain and modify over the years. For bonus points that would probably look good on your CV. 

Historical note: EFI did not start with the FAT32 file system as it required a license. EFI started with a made-up file system. The Microsoft file system team rejected this as it turns out testing file systems for all the possible end cases, across the range of produces and consumers is very very complicated. Microsoft ended up contributing FAT32 to EFI so that another new file system would not end up in the world that needed to be validated. So that is probably a good argument to invest in testing this new driver. 

Thanks,

Andrew Fish

> Thanks,
> Pedro
> 


[-- Attachment #2: Type: text/html, Size: 2644 bytes --]

Re: [edk2-devel] GSoC 2021 (MinPlatform, Ext2, ACPICA, etc)

[Nate DeSimone]

Hi Pedro,

I agree with Andrew Fish. I’d say for the first half try to have an initial (but probably broken) implementation of the driver done. Then your second half can be fixing bugs, writing tests, and documenting your work.

Thanks,
Nate

From: Pedro Falcato <pedro.falcato@gmail.com>
Date: Sunday, April 4, 2021 at 3:50 AM
To: "Desimone, Nathaniel L" <nathaniel.l.desimone@intel.com>, "devel@edk2.groups.io" <devel@edk2.groups.io>
Subject: Re: [edk2-devel] GSoC 2021 (MinPlatform, Ext2, ACPICA, etc)

Hi,

Sounds great! You may be right about that, although I believe I read somewhere that you need to set a milestone for each evaluation? If so, I'm out of ideas as to what can be tangible enough to be a milestone; if not, specifying a read-only driver as project's objective and having write support as a stretch goal is certainly the way to go.

Thanks,
Pedro 

Re: [edk2-devel] GSoC 2021 (MinPlatform, Ext2, ACPICA, etc)

[Pedro Falcato]

[-- Attachment #1: Type: text/plain, Size: 194 bytes --]

Hi Nate, Andrew,

Sounds great! I'll finish up my proposal and submit it tonight.
Thank you all for your advice!

Hopefully we'll see each other again in the summer :D

Thanks,

Pedro

[-- Attachment #2: Type: text/html, Size: 230 bytes --]