EfiRom vs. iPXE usability note

EfiRom vs. iPXE usability note

[Laszlo Ersek]

Hi All,

this is just a small note on the usability of the EfiRom utility from
BaseTools, in particular in combination with iPXE option ROMs.

The PCI Firmware Spec does not seem to specify a particular "checksum
byte" in the option ROM format, it only seems to state that the bytes in
the option ROM must sum to zero.

This (apparently) allows option ROM providers to implement different
schemes for placing the checksum byte.

When talking about traditional BIOS ROMs, EfiRom considers the last byte
in the image the checksum byte. The assumption is that the last byte is
padding anyway, so it can be repurposed as a checksum byte.

On the other hand, iPXE's "util/catrom.pl", or more precisely,
"util/Option/ROM.pm", considers byte#6 -- a reserved byte -- in the PCI
Expansion ROM Header the checksum byte.

iPXE's choice is arguably safer, because it does not assume any
particular padding at the end of the traditional ROM BIOS image that
could be stolen as checksum byte.

We've recently had a weird (difficult) bug [*] where, in the iPXE build,
the virtio-net BIOS option ROM ended up a whole multiple of 512 bytes,
*without* padding. Therefore, the "util/padimg.pl" utility, invoked by
the iPXE build process, would not append any padding to the end of the
BIOS ROM file.

Subsequently, when we combined the BIOS option ROM and the EFI option
ROM into a combined image, as follows:

  EfiRom -f "$vid" -i "$did" --pci23 \
         -b  bin/${rom}.rom \
         -ec bin-x86_64-efi/${rom}.efidrv \
         -o  bin-combined/${rom}.rom

EfiRom would *corrupt* the BIOS ROM, by changing its last byte (which
was *not* a padding byte, in this rare case). EfiRom justifiedly needed
to update the "last image indicator" in the BIOS ROM, but the checksum
compensation for that happened in the wrong place (the last byte was not
a padding byte).

Consequently, iPXE's self-decompression (LZMA) and/or CRC32 check would
fail during traditional BIOS boot, in the following call tree:

install_prealloc                 [arch/x86/prefix/libprefix.S]
  install_block (...)
  install_block (...)
  install_block (...)
  install_block (.textdata)
    process_bytes (decompress16)
    install_block_death

This was difficult to debug because even small changes to the assembly
code (let alone enabling the existent, feature-ful, debugging
infrastructure) would increase the LZMA-compressed stream's size,
trigger padding during the build, and hide the corruption.

[*] https://bugzilla.redhat.com/show_bug.cgi?id=1926561

We found the following solution:

  EfiRom -f "$vid" -i "$did" --pci23 \
         -ec bin-x86_64-efi/${rom}.efidrv \
         -o  bin-combined/${rom}.eficrom
  util/catrom.pl \
      bin/${rom}.rom \
      bin-combined/${rom}.eficrom \
      > bin-combined/${rom}.rom

I.e., continue invoking EfiRom, but only for compressing the EFI driver
image, stand-alone. And the combining of the BIOS and EFI ROMs is now
performed with iPXE's "util/catrom.pl" program, which has the safer
checksum byte placement.

(

Like Michael has recommended before, it's also possible to eliminate
EfiRom from the equation completely, and just build "*.efirom" in place
of "*.efidrv". That would give us an EFI ROM image, passable to
"catrom.pl", unlike a "naked" EFI driver executable (*.efidrv).

However, iPXE's "util/efirom" tool, which converts *.efidrv to *.efirom,
doesn't seem to offer "EFI compression", while EfiRom does (-ec option).
For QEMU live-migration compatibility, we further pad the *combined* ROM
images, currently to 256 KB. Abandoning EFI compression would eat up
approx. 80KB suddenly, and nearly exhaust our current padding. Hence the
above "hybrid" approach, where we retain EfiRom for the EFI
compression's sake, but use "util/catrom.pl" for combining the images.

)

Assuming my reading of the PCI Firmware Spec is correct, I think that
not specifying a particular checksum byte, in the various headers, was a
mistake in the spec. It's difficult to combine ROMs of different origins
into a multi-ROM image, like this.

I'm sending this note because it could prove useful to other distros
that use EfiRom similarly.

Thanks
Laszlo

Re: [edk2-devel] EfiRom vs. iPXE usability note

[Michael Brown]

On 19/02/2021 17:33, Laszlo Ersek wrote:
> The PCI Firmware Spec does not seem to specify a particular "checksum
> byte" in the option ROM format, it only seems to state that the bytes in
> the option ROM must sum to zero.
> 
> This (apparently) allows option ROM providers to implement different
> schemes for placing the checksum byte.
> 
> When talking about traditional BIOS ROMs, EfiRom considers the last byte
> in the image the checksum byte. The assumption is that the last byte is
> padding anyway, so it can be repurposed as a checksum byte.
> 
> On the other hand, iPXE's "util/catrom.pl", or more precisely,
> "util/Option/ROM.pm", considers byte#6 -- a reserved byte -- in the PCI
> Expansion ROM Header the checksum byte.
> 
> iPXE's choice is arguably safer, because it does not assume any
> particular padding at the end of the traditional ROM BIOS image that
> could be stolen as checksum byte.

Thank you for sharing this.  It made me curious as to the reason why we 
use that byte for the checksum.

As far as I can tell, it dates back to at least the ISA-era Plug and 
Play BIOS Specification v1.0a, which defines the option ROM header as 
including a 4-byte "initialization vector" occupying bytes 3-6 
inclusive, with the comment:

   The field is four bytes wide even though most implementations may
   adhere to the custom of defining a simple three byte NEAR JMP.
   The definition of the fourth byte may be OEM specific.

So, iPXE is safe to choose to use offset 6 as the checksum byte for any 
iPXE ROM images, knowing that future specification versions could not 
define an alternative use for this byte.

> However, iPXE's "util/efirom" tool, which converts *.efidrv to *.efirom,
> doesn't seem to offer "EFI compression", while EfiRom does (-ec option).
> For QEMU live-migration compatibility, we further pad the *combined* ROM
> images, currently to 256 KB. Abandoning EFI compression would eat up
> approx. 80KB suddenly, and nearly exhaust our current padding. Hence the
> above "hybrid" approach, where we retain EfiRom for the EFI
> compression's sake, but use "util/catrom.pl" for combining the images.

That part, at least, I can fix:

   https://github.com/ipxe/ipxe/pull/268

iPXE now produces compressed EFI ROM images by default.  Thank you for 
pushing me to do this!

> Assuming my reading of the PCI Firmware Spec is correct, I think that
> not specifying a particular checksum byte, in the various headers, was a
> mistake in the spec. It's difficult to combine ROMs of different origins
> into a multi-ROM image, like this.

I concur with this interpretation.  As far as I can tell, there is no 
general solution for updating the checksum that is guaranteed to work on 
arbitrary BIOS ROM images.

As the closest thing to the OEM for iPXE: please consider this email to 
be the PnP "OEM specific" definition of the byte at offset 6 of the 
expansion ROM header as being the checksum byte for any iPXE ROMs. 
Tools working on _iPXE_ BIOS ROM images may update this byte as needed.

Thanks,

Michael

Re: [edk2-devel] EfiRom vs. iPXE usability note

[Laszlo Ersek]

On 02/19/21 21:12, Michael Brown wrote:
> On 19/02/2021 17:33, Laszlo Ersek wrote:
>> The PCI Firmware Spec does not seem to specify a particular "checksum
>> byte" in the option ROM format, it only seems to state that the bytes in
>> the option ROM must sum to zero.
>>
>> This (apparently) allows option ROM providers to implement different
>> schemes for placing the checksum byte.
>>
>> When talking about traditional BIOS ROMs, EfiRom considers the last byte
>> in the image the checksum byte. The assumption is that the last byte is
>> padding anyway, so it can be repurposed as a checksum byte.
>>
>> On the other hand, iPXE's "util/catrom.pl", or more precisely,
>> "util/Option/ROM.pm", considers byte#6 -- a reserved byte -- in the PCI
>> Expansion ROM Header the checksum byte.
>>
>> iPXE's choice is arguably safer, because it does not assume any
>> particular padding at the end of the traditional ROM BIOS image that
>> could be stolen as checksum byte.
> 
> Thank you for sharing this.  It made me curious as to the reason why we
> use that byte for the checksum.
> 
> As far as I can tell, it dates back to at least the ISA-era Plug and
> Play BIOS Specification v1.0a, which defines the option ROM header as
> including a 4-byte "initialization vector" occupying bytes 3-6
> inclusive, with the comment:
> 
>   The field is four bytes wide even though most implementations may
>   adhere to the custom of defining a simple three byte NEAR JMP.
>   The definition of the fourth byte may be OEM specific.
> 
> So, iPXE is safe to choose to use offset 6 as the checksum byte for any
> iPXE ROM images, knowing that future specification versions could not
> define an alternative use for this byte.
> 
>> However, iPXE's "util/efirom" tool, which converts *.efidrv to *.efirom,
>> doesn't seem to offer "EFI compression", while EfiRom does (-ec option).
>> For QEMU live-migration compatibility, we further pad the *combined* ROM
>> images, currently to 256 KB. Abandoning EFI compression would eat up
>> approx. 80KB suddenly, and nearly exhaust our current padding. Hence the
>> above "hybrid" approach, where we retain EfiRom for the EFI
>> compression's sake, but use "util/catrom.pl" for combining the images.
> 
> That part, at least, I can fix:
> 
>   https://github.com/ipxe/ipxe/pull/268
> 
> iPXE now produces compressed EFI ROM images by default.  Thank you for
> pushing me to do this!

:)

I was 99% sure you'd just go ahead and implement it, in response to my
email! :)

Having browsed the iPXE commit history on-and-off for a while now, one
gets the impression that such "events" are not "out of the ordinary" :)

Thank you, Michael!
Laszlo

> 
>> Assuming my reading of the PCI Firmware Spec is correct, I think that
>> not specifying a particular checksum byte, in the various headers, was a
>> mistake in the spec. It's difficult to combine ROMs of different origins
>> into a multi-ROM image, like this.
> 
> I concur with this interpretation.  As far as I can tell, there is no
> general solution for updating the checksum that is guaranteed to work on
> arbitrary BIOS ROM images.
> 
> As the closest thing to the OEM for iPXE: please consider this email to
> be the PnP "OEM specific" definition of the byte at offset 6 of the
> expansion ROM header as being the checksum byte for any iPXE ROMs. Tools
> working on _iPXE_ BIOS ROM images may update this byte as needed.
> 
> Thanks,
> 
> Michael
>