* Creating EFI System Partition @ 2018-06-15 0:35 Robinson, Herbie 2018-06-15 0:41 ` Andrew Fish 2018-06-15 13:17 ` Rod Smith 0 siblings, 2 replies; 6+ messages in thread From: Robinson, Herbie @ 2018-06-15 0:35 UTC (permalink / raw) To: edk2-devel@lists.01.org I have been tasked with implementing EFI boot in our VOS operating system (in a panic, nobody bothered to tell us ahead of time that legacy boot was being dropped from the BIOS on our latest hardware). I think we have a pretty good handle on writing the bootloader, but I can't find any solid information on creating an empty EFI System Partition bearing the correct flavor of FAT32 to put the bootloader in. I need to end up with a binary image file (which I can process into an object module and embed into our OS code for initializing disks). I found a thread on submitting a "GPT" EFI shell application on this list, but it seems to have trailed off to nowhere about two years ago. Did that end up somewhere that I haven't found? Herbie Robinson Software Architect Stratus Technologies | www.stratus.com 5 Mill and Main Place, Suite 500 | Maynard, MA 01754 T: +1-978-461-7531 | E: Herbie.Robinson@stratus.com [Stratus Technologies]<http://go.stratus.com/US> ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: Creating EFI System Partition 2018-06-15 0:35 Creating EFI System Partition Robinson, Herbie @ 2018-06-15 0:41 ` Andrew Fish 2018-06-15 13:17 ` Rod Smith 1 sibling, 0 replies; 6+ messages in thread From: Andrew Fish @ 2018-06-15 0:41 UTC (permalink / raw) To: Robinson, Herbie; +Cc: edk2-devel@lists.01.org > On Jun 14, 2018, at 5:35 PM, Robinson, Herbie <Herbie.Robinson@stratus.com> wrote: > > I have been tasked with implementing EFI boot in our VOS operating system (in a panic, nobody bothered to tell us ahead of time that legacy boot was being dropped from the BIOS on our latest hardware). I think we have a pretty good handle on writing the bootloader, but I can't find any solid information on creating an empty EFI System Partition bearing the correct flavor of FAT32 to put the bootloader in. I need to end up with a binary image file (which I can process into an object module and embed into our OS code for initializing disks). > Herbie, It is "bog standard" FAT3", see Microsoft Extensible Firmware Initiative FAT32 File System Specification <https://www.google.com/url?sa=t&rct=j&q=&esrc=s&source=web&cd=1&ved=0ahUKEwi_g6P8ttTbAhUCE3wKHQMHC24QFggpMAA&url=https%3A%2F%2Fdownload.microsoft.com%2Fdownload%2F1%2F6%2F1%2F161ba512-40e2-4cc9-843a-923143f3456c%2Ffatgen103.doc&usg=AOvVaw2NwPT7yLI0sIcY1mEmzrbe> The OS loader is just an EFI Executable in file. So any old FAT32 tools will do. So are you looking for EFI tools to do GPT partitioning and FAT32 file system creation? Thanks, Andrew Fish > I found a thread on submitting a "GPT" EFI shell application on this list, but it seems to have trailed off to nowhere about two years ago. > > Did that end up somewhere that I haven't found? > > Herbie Robinson > Software Architect > Stratus Technologies | www.stratus.com > 5 Mill and Main Place, Suite 500 | Maynard, MA 01754 > T: +1-978-461-7531 | E: Herbie.Robinson@stratus.com > [Stratus Technologies]<http://go.stratus.com/US> > > _______________________________________________ > edk2-devel mailing list > edk2-devel@lists.01.org > https://lists.01.org/mailman/listinfo/edk2-devel ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: Creating EFI System Partition 2018-06-15 0:35 Creating EFI System Partition Robinson, Herbie 2018-06-15 0:41 ` Andrew Fish @ 2018-06-15 13:17 ` Rod Smith 2018-06-15 15:01 ` Andrew Fish 1 sibling, 1 reply; 6+ messages in thread From: Rod Smith @ 2018-06-15 13:17 UTC (permalink / raw) To: edk2-devel On 06/14/2018 08:35 PM, Robinson, Herbie wrote: > I have been tasked with implementing EFI boot in our VOS operating > system (in a panic, nobody bothered to tell us ahead of time that > legacy boot was being dropped from the BIOS on our latest hardware). > I think we have a pretty good handle on writing the bootloader, but I > can't find any solid information on creating an empty EFI System > Partition bearing the correct flavor of FAT32 to put the bootloader > in. The EFI spec has some quirky CYA-type wording about its filesystem, but AFAIK, any common tool for creating a FAT32 filesystem should work. I generally do it with mkdosfs in Linux, but equivalent tools in macOS, Windows, or the BSDs also work. In practice, FAT16 and FAT12 usually work, too, although the EFI spec does explicitly say at one point that the filesystem should be FAT32, and I know of at least one implementation that gets a little flaky with FAT16, so I'd stick with FAT32. An exception is if you need a very small filesystem, as on a bootable optical disc, in which case FAT16 or FAT12 might be required. Also, I've seen reports of problems with filesystems smaller than 512MiB on some EFIs, so I recommend making it at least that large, at least if it will be on a hard disk. Ideally, the EFI System Partition (ESP) should be identified by the correct partition table type code. On an MBR disk, this is 0xEF; and on a GPT disk, it's C12A7328-F81F-11D2-BA4B-00A0C93EC93B, which is identified in various ways depending on the partitioning software (type EF00 in gdisk; a "boot flag" or "esp flag" set in parted; etc.). GPT is preferable, particularly for installations on hard disks, since some OSes tie the boot mode to the partition table type. (Of course, this might not be an issue for you -- it depends on what you're preparing.) In practice, I'm not aware of any EFI that actually requires the partition type code to be set correctly; every one I've tried will boot fine even if the type code is set to something other than an official ESP type code. That said, I'd set the type code correctly just to be sure it works on an oddball system, and to avoid confusion if/when users look at the disk. If the boot medium is an optical disc, you'll need a particular variant of an El Torito boot image. If you need help with this, please say so; I can dig up the details, or at least point you to a sample mkisofs command. > I need to end up with a binary image file (which I can process > into an object module and embed into our OS code for initializing > disks). GPT places data structures at both the beginning and the end of the disk, which might create some complications, depending on your exact needs. If you need to write an image to a blank disk of unknown size, several options occur to me: * You can use MBR, which does not rely on data structures at the end of the disk. As noted above, though, that's a little iffy in some cases -- but it might be fine for your case (it's hard to tell without more details). * You can prepare a virtual image of a small disk and write it out to a larger disk, leaving the backup GPT data structures before the end of the disk; then either: * Leave it that way, which will effectively limit the size of the disk. This might be OK for a USB flash drive. * Use a disk partitioning tool to relocate the backup GPT data structures to the end of the disk. The sgdisk "-e" option will do this, for instance. IIRC, parted will do it automatically, or at least prompt that it be done, if any operation is performed on the disk. * You can create an image of the FAT32 ESP filesystem ONLY (without partition table), then have your wrapper tool use sgdisk, parted, or some other tool to prepare a disk with GPT and an ESP. You'd then write out the image to the ESP on the disk you've just partitioned. Caveat/full disclosure: I mostly use Linux, so I've referenced Linux commands. I'm also the author of GPT fdisk (gdisk, sgdisk, and cgdisk); but of course, other tools on many platforms can do what you need. > I found a thread on submitting a "GPT" EFI shell application on this > list, but it seems to have trailed off to nowhere about two years > ago. > > Did that end up somewhere that I haven't found? > > Herbie Robinson Software Architect Stratus Technologies | > www.stratus.com 5 Mill and Main Place, Suite 500 | Maynard, MA 01754 > T: +1-978-461-7531 | E: Herbie.Robinson@stratus.com [Stratus > Technologies]<http://go.stratus.com/US> > > _______________________________________________ edk2-devel mailing > list edk2-devel@lists.01.org > https://lists.01.org/mailman/listinfo/edk2-devel > -- Rod Smith rodsmith@rodsbooks.com http://www.rodsbooks.com ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: Creating EFI System Partition 2018-06-15 13:17 ` Rod Smith @ 2018-06-15 15:01 ` Andrew Fish 2018-06-15 15:09 ` Andrew Fish 2018-06-15 20:11 ` Rod Smith 0 siblings, 2 replies; 6+ messages in thread From: Andrew Fish @ 2018-06-15 15:01 UTC (permalink / raw) To: Rod Smith; +Cc: edk2-devel > On Jun 15, 2018, at 6:17 AM, Rod Smith <rodsmith@rodsbooks.com> wrote: > > On 06/14/2018 08:35 PM, Robinson, Herbie wrote: >> I have been tasked with implementing EFI boot in our VOS operating >> system (in a panic, nobody bothered to tell us ahead of time that >> legacy boot was being dropped from the BIOS on our latest hardware). >> I think we have a pretty good handle on writing the bootloader, but I >> can't find any solid information on creating an empty EFI System >> Partition bearing the correct flavor of FAT32 to put the bootloader >> in. > > The EFI spec has some quirky CYA-type wording about its filesystem, Rob, Thanks for all this info. The CYA is the UEFI Spec referencing the FAT spec I cross linked. Given a specification needs to be the arbiter of truth some times it is required to be pedantic. In this case the UEFI Spec needed a definition of FAT32, and that is the one and only FAT spec. contributed by Microsoft. > but > AFAIK, any common tool for creating a FAT32 filesystem should work. I > generally do it with mkdosfs in Linux, but equivalent tools in macOS, > Windows, or the BSDs also work. In practice, FAT16 and FAT12 usually > work, too, although the EFI spec does explicitly say at one point that > the filesystem should be FAT32, and I know of at least one > implementation that gets a little flaky with FAT16, so I'd stick with > FAT32. An exception is if you need a very small filesystem, as on a > bootable optical disc, in which case FAT16 or FAT12 might be required. > Also, I've seen reports of problems with filesystems smaller than 512MiB > on some EFIs, so I recommend making it at least that large, at least if > it will be on a hard disk. > I seem to remember that the FAT32 spec also defined FAT16 and FAT12. it also defines when FAT32, FAT16, or FAT12 should be used for media. If the edk2 FAT driver has an issue with media that conform to the FAT32 spec we should fix that. If the issue is non conferment, then we need to decide if the fix can be made that will follow the FAT Spec. See that CYA was useful after all :). Thanks, Andrew Fish > Ideally, the EFI System Partition (ESP) should be identified by the > correct partition table type code. On an MBR disk, this is 0xEF; and on > a GPT disk, it's C12A7328-F81F-11D2-BA4B-00A0C93EC93B, which is > identified in various ways depending on the partitioning software (type > EF00 in gdisk; a "boot flag" or "esp flag" set in parted; etc.). GPT is > preferable, particularly for installations on hard disks, since some > OSes tie the boot mode to the partition table type. (Of course, this > might not be an issue for you -- it depends on what you're preparing.) > In practice, I'm not aware of any EFI that actually requires the > partition type code to be set correctly; every one I've tried will boot > fine even if the type code is set to something other than an official > ESP type code. That said, I'd set the type code correctly just to be > sure it works on an oddball system, and to avoid confusion if/when users > look at the disk. > > If the boot medium is an optical disc, you'll need a particular variant > of an El Torito boot image. If you need help with this, please say so; I > can dig up the details, or at least point you to a sample mkisofs command. > >> I need to end up with a binary image file (which I can process >> into an object module and embed into our OS code for initializing >> disks). > > GPT places data structures at both the beginning and the end of the > disk, which might create some complications, depending on your exact > needs. If you need to write an image to a blank disk of unknown size, > several options occur to me: > > * You can use MBR, which does not rely on data structures at the > end of the disk. As noted above, though, that's a little iffy in > some cases -- but it might be fine for your case (it's hard to > tell without more details). > * You can prepare a virtual image of a small disk and write it out > to a larger disk, leaving the backup GPT data structures before > the end of the disk; then either: > * Leave it that way, which will effectively limit the size of the > disk. This might be OK for a USB flash drive. > * Use a disk partitioning tool to relocate the backup GPT data > structures to the end of the disk. The sgdisk "-e" option will > do this, for instance. IIRC, parted will do it automatically, > or at least prompt that it be done, if any operation is performed > on the disk. > * You can create an image of the FAT32 ESP filesystem ONLY (without > partition table), then have your wrapper tool use sgdisk, parted, > or some other tool to prepare a disk with GPT and an ESP. You'd > then write out the image to the ESP on the disk you've just > partitioned. > > Caveat/full disclosure: I mostly use Linux, so I've referenced Linux > commands. I'm also the author of GPT fdisk (gdisk, sgdisk, and cgdisk); > but of course, other tools on many platforms can do what you need. > >> I found a thread on submitting a "GPT" EFI shell application on this >> list, but it seems to have trailed off to nowhere about two years >> ago. >> >> Did that end up somewhere that I haven't found? >> >> Herbie Robinson Software Architect Stratus Technologies | >> www.stratus.com 5 Mill and Main Place, Suite 500 | Maynard, MA 01754 >> T: +1-978-461-7531 | E: Herbie.Robinson@stratus.com [Stratus >> Technologies]<http://go.stratus.com/US> >> >> _______________________________________________ edk2-devel mailing >> list edk2-devel@lists.01.org >> https://lists.01.org/mailman/listinfo/edk2-devel >> > > > -- > Rod Smith > rodsmith@rodsbooks.com <mailto:rodsmith@rodsbooks.com> > http://www.rodsbooks.com <http://www.rodsbooks.com/> > _______________________________________________ > edk2-devel mailing list > edk2-devel@lists.01.org <mailto:edk2-devel@lists.01.org> > https://lists.01.org/mailman/listinfo/edk2-devel <https://lists.01.org/mailman/listinfo/edk2-devel> ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: Creating EFI System Partition 2018-06-15 15:01 ` Andrew Fish @ 2018-06-15 15:09 ` Andrew Fish 2018-06-15 20:11 ` Rod Smith 1 sibling, 0 replies; 6+ messages in thread From: Andrew Fish @ 2018-06-15 15:09 UTC (permalink / raw) To: Rod Smith; +Cc: edk2-devel Sorry Rod, called you Rob. I blame pre coffee emailing. My bad. Thanks, Andrew Fish > On Jun 15, 2018, at 8:01 AM, Andrew Fish <afish@apple.com> wrote: > > > >> On Jun 15, 2018, at 6:17 AM, Rod Smith <rodsmith@rodsbooks.com <mailto:rodsmith@rodsbooks.com>> wrote: >> >> On 06/14/2018 08:35 PM, Robinson, Herbie wrote: >>> I have been tasked with implementing EFI boot in our VOS operating >>> system (in a panic, nobody bothered to tell us ahead of time that >>> legacy boot was being dropped from the BIOS on our latest hardware). >>> I think we have a pretty good handle on writing the bootloader, but I >>> can't find any solid information on creating an empty EFI System >>> Partition bearing the correct flavor of FAT32 to put the bootloader >>> in. >> >> The EFI spec has some quirky CYA-type wording about its filesystem, > > Rob, > > Thanks for all this info. > > The CYA is the UEFI Spec referencing the FAT spec I cross linked. Given a specification needs to be the arbiter of truth some times it is required to be pedantic. In this case the UEFI Spec needed a definition of FAT32, and that is the one and only FAT spec. contributed by Microsoft. > >> but >> AFAIK, any common tool for creating a FAT32 filesystem should work. I >> generally do it with mkdosfs in Linux, but equivalent tools in macOS, >> Windows, or the BSDs also work. In practice, FAT16 and FAT12 usually >> work, too, although the EFI spec does explicitly say at one point that >> the filesystem should be FAT32, and I know of at least one >> implementation that gets a little flaky with FAT16, so I'd stick with >> FAT32. An exception is if you need a very small filesystem, as on a >> bootable optical disc, in which case FAT16 or FAT12 might be required. >> Also, I've seen reports of problems with filesystems smaller than 512MiB >> on some EFIs, so I recommend making it at least that large, at least if >> it will be on a hard disk. >> > > I seem to remember that the FAT32 spec also defined FAT16 and FAT12. it also defines when FAT32, FAT16, or FAT12 should be used for media. > > If the edk2 FAT driver has an issue with media that conform to the FAT32 spec we should fix that. If the issue is non conferment, then we need to decide if the fix can be made that will follow the FAT Spec. See that CYA was useful after all :). > > Thanks, > > Andrew Fish > >> Ideally, the EFI System Partition (ESP) should be identified by the >> correct partition table type code. On an MBR disk, this is 0xEF; and on >> a GPT disk, it's C12A7328-F81F-11D2-BA4B-00A0C93EC93B, which is >> identified in various ways depending on the partitioning software (type >> EF00 in gdisk; a "boot flag" or "esp flag" set in parted; etc.). GPT is >> preferable, particularly for installations on hard disks, since some >> OSes tie the boot mode to the partition table type. (Of course, this >> might not be an issue for you -- it depends on what you're preparing.) >> In practice, I'm not aware of any EFI that actually requires the >> partition type code to be set correctly; every one I've tried will boot >> fine even if the type code is set to something other than an official >> ESP type code. That said, I'd set the type code correctly just to be >> sure it works on an oddball system, and to avoid confusion if/when users >> look at the disk. >> >> If the boot medium is an optical disc, you'll need a particular variant >> of an El Torito boot image. If you need help with this, please say so; I >> can dig up the details, or at least point you to a sample mkisofs command. >> >>> I need to end up with a binary image file (which I can process >>> into an object module and embed into our OS code for initializing >>> disks). >> >> GPT places data structures at both the beginning and the end of the >> disk, which might create some complications, depending on your exact >> needs. If you need to write an image to a blank disk of unknown size, >> several options occur to me: >> >> * You can use MBR, which does not rely on data structures at the >> end of the disk. As noted above, though, that's a little iffy in >> some cases -- but it might be fine for your case (it's hard to >> tell without more details). >> * You can prepare a virtual image of a small disk and write it out >> to a larger disk, leaving the backup GPT data structures before >> the end of the disk; then either: >> * Leave it that way, which will effectively limit the size of the >> disk. This might be OK for a USB flash drive. >> * Use a disk partitioning tool to relocate the backup GPT data >> structures to the end of the disk. The sgdisk "-e" option will >> do this, for instance. IIRC, parted will do it automatically, >> or at least prompt that it be done, if any operation is performed >> on the disk. >> * You can create an image of the FAT32 ESP filesystem ONLY (without >> partition table), then have your wrapper tool use sgdisk, parted, >> or some other tool to prepare a disk with GPT and an ESP. You'd >> then write out the image to the ESP on the disk you've just >> partitioned. >> >> Caveat/full disclosure: I mostly use Linux, so I've referenced Linux >> commands. I'm also the author of GPT fdisk (gdisk, sgdisk, and cgdisk); >> but of course, other tools on many platforms can do what you need. >> >>> I found a thread on submitting a "GPT" EFI shell application on this >>> list, but it seems to have trailed off to nowhere about two years >>> ago. >>> >>> Did that end up somewhere that I haven't found? >>> >>> Herbie Robinson Software Architect Stratus Technologies | >>> www.stratus.com 5 Mill and Main Place, Suite 500 | Maynard, MA 01754 >>> T: +1-978-461-7531 | E: Herbie.Robinson@stratus.com [Stratus >>> Technologies]<http://go.stratus.com/US> >>> >>> _______________________________________________ edk2-devel mailing >>> list edk2-devel@lists.01.org >>> https://lists.01.org/mailman/listinfo/edk2-devel >>> >> >> >> -- >> Rod Smith >> rodsmith@rodsbooks.com <mailto:rodsmith@rodsbooks.com> <mailto:rodsmith@rodsbooks.com <mailto:rodsmith@rodsbooks.com>> >> http://www.rodsbooks.com <http://www.rodsbooks.com/> <http://www.rodsbooks.com/ <http://www.rodsbooks.com/>> >> _______________________________________________ >> edk2-devel mailing list >> edk2-devel@lists.01.org <mailto:edk2-devel@lists.01.org> <mailto:edk2-devel@lists.01.org <mailto:edk2-devel@lists.01.org>> >> https://lists.01.org/mailman/listinfo/edk2-devel <https://lists.01.org/mailman/listinfo/edk2-devel> <https://lists.01.org/mailman/listinfo/edk2-devel <https://lists.01.org/mailman/listinfo/edk2-devel>> > _______________________________________________ > edk2-devel mailing list > edk2-devel@lists.01.org <mailto:edk2-devel@lists.01.org> > https://lists.01.org/mailman/listinfo/edk2-devel <https://lists.01.org/mailman/listinfo/edk2-devel> ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: Creating EFI System Partition 2018-06-15 15:01 ` Andrew Fish 2018-06-15 15:09 ` Andrew Fish @ 2018-06-15 20:11 ` Rod Smith 1 sibling, 0 replies; 6+ messages in thread From: Rod Smith @ 2018-06-15 20:11 UTC (permalink / raw) To: edk2-devel On 06/15/2018 11:01 AM, Andrew Fish wrote: > >> On Jun 15, 2018, at 6:17 AM, Rod Smith <rodsmith@rodsbooks.com >> <mailto:rodsmith@rodsbooks.com>> wrote: >> >> but AFAIK, any common tool for creating a FAT32 filesystem should >> work. I generally do it with mkdosfs in Linux, but equivalent tools >> in macOS, Windows, or the BSDs also work. In practice, FAT16 and >> FAT12 usually work, too, although the EFI spec does explicitly say >> at one point that the filesystem should be FAT32, and I know of at >> least one implementation that gets a little flaky with FAT16, so >> I'd stick with FAT32. >> > > I seem to remember that the FAT32 spec also defined FAT16 and FAT12. > it also defines when FAT32, FAT16, or FAT12 should be used for > media. > > If the edk2 FAT driver has an issue with media that conform to the > FAT32 spec we should fix that. If the issue is non conferment, then > we need to decide if the fix can be made that will follow the FAT > Spec. See that CYA was useful after all :). No, I didn't mean to imply that the EDK2 FAT driver gets flaky with 12- or 16-bit FAT filesystems. The EFI in question was an early EFI-over-BIOS implementation on a Gigabyte motherboard from 2011 or 2012. That thing was a horror, and I wrote up my experiences at the time: http://www.rodsbooks.com/gb-hybrid-efi/ I no longer have that motherboard, so I can't do any more tests with it or double-check my findings from 2012. From that page, though: : A FAT-16 ESP, on the other hand, seems problematic. Ubuntu 11.04 (and : 11.10) in EFI mode creates a dinky FAT-16 ESP, and after my test : install of Ubuntu 11.04, the board hung on reboot until I reworked : the ESP as FAT-32. Thus, if you plan to install Ubuntu, or any other : OS that creates a FAT-16 ESP, be prepared to fix it, preferably : before the system reboots! Note that Ubuntu no longer creates a "dinky FAT-16 ESP;" it now creates a 512MiB FAT-32 ESP. The experience remains a relevant cautionary tale, though, for anybody who's trying to write an OS installer, particularly if the system must be installable on some random computer -- systems from that period are still in use today, so a FAT-16 ESP could cause problems in the real world. That said, I've not encountered this problem on any modern (say, 2014 or later) EFI. -- Rod Smith rodsmith@rodsbooks.com http://www.rodsbooks.com ^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2018-06-15 20:11 UTC | newest] Thread overview: 6+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2018-06-15 0:35 Creating EFI System Partition Robinson, Herbie 2018-06-15 0:41 ` Andrew Fish 2018-06-15 13:17 ` Rod Smith 2018-06-15 15:01 ` Andrew Fish 2018-06-15 15:09 ` Andrew Fish 2018-06-15 20:11 ` Rod Smith
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox