From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from nwk-aaemail-lapp03.apple.com (nwk-aaemail-lapp03.apple.com [17.151.62.68]) by mx.groups.io with SMTP id smtpd.web10.32332.1597007680336955743 for ; Sun, 09 Aug 2020 14:14:40 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=@apple.com header.s=20180706 header.b=roZ3nN15; spf=pass (domain: apple.com, ip: 17.151.62.68, mailfrom: afish@apple.com) Received: from pps.filterd (nwk-aaemail-lapp03.apple.com [127.0.0.1]) by nwk-aaemail-lapp03.apple.com (8.16.0.42/8.16.0.42) with SMTP id 079LAWbE040359; Sun, 9 Aug 2020 14:14:38 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=apple.com; h=from : message-id : content-type : mime-version : subject : date : in-reply-to : cc : to : references; s=20180706; bh=kev/BIjiDqu7M6epuFPaKEm0dhyv2guIc8f1mZ1r9Q8=; b=roZ3nN15DWIGoCCXR/hbLaQYWhnWSnz4fYZpiNEl/gsGCSUoo+h1tICCDGf7Qa8fWJLM pk1MkjRya/+Y21tk8wH8+prIYtaTkYqWLeh8cCjUDzsASoFnc7hPbCk58/0u7v8bFsET cn4Zq+O5FmoLWWw8rC5aA3pfNMPMVO0Mlzhp4oyunfEWh13T7lfuytVR74vV0M4PoLWx dlkLY4XvgnB98YUP+De4voHPWwj+5wGRH9uLSrpebYSaCwgsLkioKgoRGcZRwEnYxftW 0oNP94KTe+mSYoPU4tP4ZdWfD7A/MScvPxsldpZotbZg7N6tIEaIF1CnxQ+iFlU1eDny rg== Received: from rn-mailsvcp-mta-lapp01.rno.apple.com (rn-mailsvcp-mta-lapp01.rno.apple.com [10.225.203.149]) by nwk-aaemail-lapp03.apple.com with ESMTP id 32tbrmrqyy-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NO); Sun, 09 Aug 2020 14:14:38 -0700 Received: from rn-mailsvcp-mmp-lapp04.rno.apple.com (rn-mailsvcp-mmp-lapp04.rno.apple.com [17.179.253.17]) by rn-mailsvcp-mta-lapp01.rno.apple.com (Oracle Communications Messaging Server 8.1.0.6.20200729 64bit (built Jul 29 2020)) with ESMTPS id <0QET002FHF0EO0B0@rn-mailsvcp-mta-lapp01.rno.apple.com>; Sun, 09 Aug 2020 14:14:38 -0700 (PDT) Received: from process_milters-daemon.rn-mailsvcp-mmp-lapp04.rno.apple.com by rn-mailsvcp-mmp-lapp04.rno.apple.com (Oracle Communications Messaging Server 8.1.0.6.20200729 64bit (built Jul 29 2020)) id <0QET00Z00EBZ6W00@rn-mailsvcp-mmp-lapp04.rno.apple.com>; Sun, 09 Aug 2020 14:14:38 -0700 (PDT) X-Va-A: X-Va-T-CD: 8872bb1b158984713a555814cdbd17e8 X-Va-E-CD: 2ae8cfcd3550c6eec2a73d12feb49f33 X-Va-R-CD: b6caeb52d7bc8ab929dbce3e7b3a9d6f X-Va-CD: 0 X-Va-ID: f567f3f5-30b3-41f7-a13a-62fc7eb40e4b X-V-A: X-V-T-CD: 8872bb1b158984713a555814cdbd17e8 X-V-E-CD: 2ae8cfcd3550c6eec2a73d12feb49f33 X-V-R-CD: b6caeb52d7bc8ab929dbce3e7b3a9d6f X-V-CD: 0 X-V-ID: 82264fc9-c1c4-421f-a338-fc244a3fcde4 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.235,18.0.687 definitions=2020-08-09_12:2020-08-06,2020-08-09 signatures=0 Received: from [17.235.5.162] (unknown [17.235.5.162]) by rn-mailsvcp-mmp-lapp04.rno.apple.com (Oracle Communications Messaging Server 8.1.0.6.20200729 64bit (built Jul 29 2020)) with ESMTPSA id <0QET00F62F0C3N00@rn-mailsvcp-mmp-lapp04.rno.apple.com>; Sun, 09 Aug 2020 14:14:37 -0700 (PDT) From: "Andrew Fish" Message-id: <14F1E0BE-0278-4FE7-B2C8-A9727DCD31D9@apple.com> MIME-version: 1.0 (Mac OS X Mail 13.4 \(3608.80.23.2.2\)) Subject: Re: [edk2-devel] Partition issue with Linux Distribution iso image Date: Sun, 09 Aug 2020 14:14:35 -0700 In-reply-to: Cc: "Ni, Ray" , Laszlo Ersek , Gary Lin , "Wu, Hao A" To: devel@edk2.groups.io, zhichao.gao@intel.com References: X-Mailer: Apple Mail (2.3608.80.23.2.2) X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.235,18.0.687 definitions=2020-08-09_12:2020-08-06,2020-08-09 signatures=0 Content-type: multipart/alternative; boundary="Apple-Mail=_FDC2E44A-1DA8-4798-BFD3-19D7B774F9E9" --Apple-Mail=_FDC2E44A-1DA8-4798-BFD3-19D7B774F9E9 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=utf-8 Zhichao, The discovery algorithm for partitions is defined in the UEFI Spec.=20 13.3.2 Partition Discovery ... The following is the order in which a block device must be scanned to dete= rmine if it contains partitions. When a check for a valid partitioning scheme succeeds, the search terminat= es. 1) Check for GUID Partition Table Headers. 2) Follow ISO-9660 specification to search for ISO-9660 volume structures = on the magic LBA.=20 3) Check for an =E2=80=9CEl Torito=E2=80=9D volume extension and follow th= e =E2=80=9CEl Torito=E2=80=9D CD-ROM specification.=20 4) If none of the above, check LBA 0 for a legacy MBR partition table. 5) No partition found on device. If a disk contains a recognized RAID structure (e.g. DDF structure as defi= ned in The Storage Networking Industry Association Common RAID Disk Data Fo= rmat Specification--see Glossary), the data on the disk must be ignored, un= less the driver is using the RAID structure to produce a logical RAID volum= e. EFI supports the nesting of legacy MBR partitions, by allowing any legacy = MBR partition to contain more legacy MBR partitions. This is accomplished b= y supporting the same partition discovery algorithm on every logical block = device. It should be noted that the GUID Partition Table does not allow nes= ting of GUID Partition Table Headers. Nesting is not needed since a GUID Pa= rtition Table Header can support an arbitrary number of partitions (the add= ressability limits of a 64-bit LBA are the limiting factor). 13.3.2.1 ISO-9660 and El Torito IS0-9660 is the industry standard low level format used on CD-ROM and DVD-= ROM. The CD-ROM format is completely described by the =E2=80=9CEl Torito=E2= = =80=9D Bootable CD-ROM Format Specification Version 1.0. To boot from a CD= -ROM or DVD-ROM in the boot services environment, an EFI System partition i= s stored in a =E2=80=9Cno emulation=E2=80=9D mode as defined by the =E2=80= =9CEl Torito=E2=80=9D specification. A Platform ID of 0xEF indicates an EF= I System Partition. The Platform ID is in either the Section Header Entry o= r the Validation Entry of the Booting Catalog as defined by the =E2=80=9CEl= Torito=E2=80=9D specification. EFI differs from =E2=80=9CEl Torito=E2=80= =9D =E2=80=9Cno emulation=E2=80=9D mode in that it does not load the =E2= =80=9Cno emulation=E2=80=9D image into memory and jump to it. EFI interpre= ts the =E2=80=9Cno emulation=E2=80=9D image as an EFI system partition. EFI= interprets the Sector Count in the Initial/Default Entry or the Section He= ader Entry to be the size of the EFI system partition. If the value of Sect= or Count is set to 0 or 1, EFI will assume the system partition consumes th= e space from the beginning of the =E2=80=9Cno emulation=E2=80=9D image to t= he end of the CD-ROM. A DVD-ROM image formatted as required by the UDF 2.0 specification (OSTA U= niversal Disk Format Specification, Revision 2.0) shall be booted by UEFI i= f: =E2=80=A2 the DVD-ROM image conforms to the "UDF Bridge" format defined in= the UDF 2.0 specification, and =E2=80=A2 the DVD-ROM image contains exactly one ISO-9660 file system, and =E2=80=A2 the ISO-9660 file system conforms to the "El Torito" Bootable CD= -ROM Format Specification. Booting from a DVD-ROM that satisfies the above requirements is accomplish= ed using the same methods as booting from a CD-ROM: the ISO-9660 file syste= m shall be booted. Since the EFI file system definition does not use the same Initial/Default= entry as a legacy CD-ROM it is possible to boot personal computers using a= n EFI CD-ROM or DVD-ROM. The inclusion of boot code for personal computers = is optional and not required by EFI. Thanks, Andrew Fish > On Aug 9, 2020, at 6:51 AM, Gao, Zhichao wrote: >=20 > Hi All, > > Some Linux Distribution ISO image would contain both Eltorito(iso 9660) = volume info and MBR table at the same time. That would cause the partition = check pass the MBR and ignore the UDF (Eltorito compatible) check. > There is a bug in the partition child handler: https://bugzilla.tianocor= e.org/show_bug.cgi?id=3D2843 . It would make the MBR transfer the wrong device size. That cause= the FAT driver fail to install the FAT protocol onto the device. > But when I solve the issue, the EFI partition can be found as MBR FAT de= vice. That would cause another issue. Boot from the MBR FAT from ISO image = is different with boot from CD FAT. > Here is the difference: > Boot from MBR FAT: go to the grub terminal window. > Boot from CD FAT: go to the installation window. > I don=E2=80=99t know why the same grub image act different behaviors. Bu= t I believe =E2=80=9CBoot from CD FAT=E2=80=9D is the right behavior. That = means the ISO 9660 should not be treated as MBR device in UEFI BIOS. > > So I would like to add a logic: the ISO image that contain both ISO 9660= volume info and MBR partition, the partition driver would skip the MBR che= ck and continue to do the ISO 9660 check. There is a easy way to do this, i= .e. put the UDF (ISO 9660 compatible) before the MBR check. It means the pa= rtition routine child handle would have a priority and the UDF (ISO 9660) h= as high priority than the MBR. That would solve all the Linux Distribution = ISO image issue in my thought. > > I want to get some suggestion from the community before I sent the patch= . > > Thanks, > Zhichao >=20 --Apple-Mail=_FDC2E44A-1DA8-4798-BFD3-19D7B774F9E9 Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=utf-8 Zhichao,
<= br class=3D"">
The discovery algorithm for partitions = is defined in the UEFI Spec. 

13.3.2 Partition Discovery
...
=

The fo= llowing is the order in which a block device must be scanned to determine i= f it contains partitions.
When a check for a valid par= titioning scheme succeeds, the search terminates.

1) Check for GUID Partition Table Headers.=
2) Follow ISO-9660 specification to search for ISO-96= 60 volume structures on the magic LBA. 
3) Check = for an =E2=80=9CEl Torito=E2=80=9D volume extension and follow the =E2=80= =9CEl Torito=E2=80=9D CD-ROM specification. 
4) = If none of the above, check LBA 0 for a legacy MBR partition table.
5) No partition found on device.

If a disk contains a recognized RAID structure (e.g. DDF struc= ture as defined in The Storage Networking Industry Association Common RAID = Disk Data Format Specification--see Glossary), the data on the disk must be= ignored, unless the driver is using the RAID structure to produce a logica= l RAID volume.
EFI supports the nesting of legacy MBR partitions,= by allowing any legacy MBR partition to contain more legacy MBR partitions= . This is accomplished by supporting the same partition discovery algorithm= on every logical block device. It should be noted that the GUID Partition = Table does not allow nesting of GUID Partition Table Headers. Nesting is no= t needed since a GUID Partition Table Header can support an arbitrary numbe= r of partitions (the addressability limits of a 64-bit LBA are the limiting= factor).

13.3.2.1 ISO-9660 and El Tori= to
IS0-9660 is the industry standard low level format used on CD-= ROM and DVD-ROM. The CD-ROM format is completely described by the =E2=80=9C= El Torito=E2=80=9D Bootable CD-ROM Format Specification Version 1.0. To boo= t from a CD-ROM or DVD-ROM in the boot services environment, an EFI System = partition is stored in a =E2=80=9Cno emulation=E2=80=9D mode as defined by = the =E2=80=9CEl Torito=E2=80=9D specification. A Platform ID of 0xEF indica= tes an EFI System Partition. The Platform ID is in either the Section Heade= r Entry or the Validation Entry of the Booting Catalog as defined by the = =E2=80=9CEl Torito=E2=80=9D specification. EFI differs from =E2=80=9CEl To= rito=E2=80=9D =E2=80=9Cno emulation=E2=80=9D mode in that it does not load = the =E2=80=9Cno emulation=E2=80=9D image into memory and jump to it. EFI in= terprets the =E2=80=9Cno emulation=E2=80=9D image as an EFI system partitio= n. EFI interprets the Sector Count in the Initial/Default Entry or the Sect= ion Header Entry to be the size of the EFI system partition. If the value o= f Sector Count is set to 0 or 1, EFI will assume the system partition consu= mes the space from the beginning of the =E2=80=9Cno emulation=E2=80=9D imag= e to the end of the CD-ROM.
A DVD-ROM image formatted as required= by the UDF 2.0 specification (OSTA Universal Disk Format Specification, Re= vision 2.0) shall be booted by UEFI if:
=E2=80=A2 the DVD-ROM ima= ge conforms to the "UDF Bridge" format defined in the UDF 2.0 specification= , and
=E2=80=A2 the DVD-ROM image contains exactly one ISO-9660 f= ile system, and
=E2=80=A2 the ISO-9660 file system conforms to th= e "El Torito" Bootable CD-ROM Format Specification.
Booting from = a DVD-ROM that satisfies the above requirements is accomplished using the s= ame methods as booting from a CD-ROM: the ISO-9660 file system shall be boo= ted.
Since the EFI file system definition does not use the same I= nitial/Default entry as a legacy CD-ROM it is possible to boot personal com= puters using an EFI CD-ROM or DVD-ROM. The inclusion of boot code for perso= nal computers is optional and not required by EFI.

Thanks,

Andrew Fish

On Aug 9, 2020, at 6:51 AM, Gao, Zhichao <zhichao.gao@intel.com> wrote:

Hi= All,
 
Some Linux Distribution ISO= image would contain both Eltorito(iso 9660) volume info and MBR table at t= he same time. That would cause the partition check pass the MBR and ignore = the UDF (Eltorito compatible) check.
There is a bug in the partition child handler: https://bugzilla.tianocore.org/show_bug.cgi?i= d=3D2843. It would make the MBR transfer the wrong device size. That ca= use the FAT driver fail to install the FAT     &nb= sp; protocol onto the device.
But when I solve the issue, the EFI partition can be found as MBR= FAT device. That would cause another issue. Boot from the MBR FAT from ISO= image is different with boot from CD FAT.
Here is the difference:
<= div style=3D"margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibr= i, sans-serif;" class=3D"">Boot from MBR FAT: go to the grub terminal windo= w.
Boot from CD FAT:= go to the installation window.
I don=E2=80=99t know why the same grub image act different beha= viors. But I believe =E2=80=9CBoot from CD FAT=E2=80=9D is the right behavi= or. That means the ISO 9660 should not be treated as MBR device in UEFI BIO= S.
&= nbsp;
So I would like to add a logic= : the ISO image that contain both ISO 9660 volume info and MBR partition, t= he partition driver would skip the MBR check and continue to do the ISO 966= 0 check. There is a easy way to do this, i.e. put the UDF (ISO 9660 compati= ble) before the MBR check. It means the partition routine child handle woul= d have a priority and the UDF (ISO 9660) has high priority than the MBR. Th= at would solve all the Linux Distribution ISO image issue in my thought.
 =
I want to get some suggestion from = the community before I sent the patch.
 
Thanks,
Zhich= ao

--Apple-Mail=_FDC2E44A-1DA8-4798-BFD3-19D7B774F9E9--