From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail.byosoft.com.cn (mail.byosoft.com.cn [58.240.74.242]) by mx.groups.io with SMTP id smtpd.web11.2116.1626916855146502171 for ; Wed, 21 Jul 2021 18:20:56 -0700 Authentication-Results: mx.groups.io; dkim=missing; spf=none, err=permanent DNS error (domain: byosoft.com.cn, ip: 58.240.74.242, mailfrom: gaoliming@byosoft.com.cn) Received: from DESKTOPS6D0PVI ([58.246.60.130]) (envelope-sender ) by 192.168.6.13 with ESMTP for ; Thu, 22 Jul 2021 09:20:50 +0800 X-WM-Sender: gaoliming@byosoft.com.cn X-Originating-IP: 58.246.60.130 X-WM-AuthFlag: YES X-WM-AuthUser: gaoliming@byosoft.com.cn From: "gaoliming" To: , , References: In-Reply-To: Subject: =?UTF-8?B?5Zue5aSNOiBbZWRrMi1yZmNdIFJGQzogRVhUNCBmaWxlc3lzdGVtIGRyaXZlcg==?= Date: Thu, 22 Jul 2021 09:20:54 +0800 Message-ID: <000001d77e97$d1089100$7319b300$@byosoft.com.cn> MIME-Version: 1.0 X-Mailer: Microsoft Outlook 16.0 Thread-Index: AQJUjZGJHgvZMLaT7tUYgGKSroFJtapUE6mQ Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Content-Language: zh-cn > -----=E9=82=AE=E4=BB=B6=E5=8E=9F=E4=BB=B6----- > =E5=8F=91=E4=BB=B6=E4=BA=BA: rfc@edk2.groups.io =E4= = =BB=A3=E8=A1=A8 Pedro Falcato > =E5=8F=91=E9=80=81=E6=97=B6=E9=97=B4: 2021=E5=B9=B47=E6=9C=8822=E6=97=A5= 7:12 > =E6=94=B6=E4=BB=B6=E4=BA=BA: devel@edk2.groups.io > =E6=8A=84=E9=80=81: rfc@edk2.groups.io > =E4=B8=BB=E9=A2=98: [edk2-rfc] RFC: EXT4 filesystem driver >=20 > EXT4 (fourth extended filesystem) is a filesystem developed for Linux > that has been in wide use (desktops, servers, smartphones) since 2008. >=20 > The Ext4Pkg implements the Simple File System Protocol for a partition > that is formatted with the EXT4 file system. This allows UEFI Drivers, > UEFI Applications, UEFI OS Loaders, and the UEFI Shell to access files > on an EXT4 partition and supports booting a UEFI OS Loader from an > EXT4 partition. > This project is one of the TianoCore Google Summer of Code projects. >=20 > Right now, Ext4Pkg only contains a single member, Ext4Dxe, which is a > UEFI driver that consumes Block I/O, Disk I/O and (optionally) Disk > I/O 2 Protocols, and produces the Simple File System protocol. It > allows mounting ext4 filesystems exclusively. >=20 > Brief overhead of EXT4: > Layout of an EXT2/3/4 filesystem: > (note: this driver has been developed using > https://www.kernel.org/doc/html/latest/filesystems/ext4/index.html as > documentation). >=20 > An ext2/3/4 filesystem (here on out referred to as simply an ext4 filesy= stem, > due to the similarities) is composed of various concepts: >=20 > 1) Superblock > The superblock is the structure near (1024 bytes offset from the start) > the start of the partition, and describes the filesystem in general. > Here, we get to know the size of the filesystem's blocks, which features > it supports or not, whether it's been cleanly unmounted, how many blocks > we have, etc. >=20 > 2) Block groups > EXT4 filesystems are divided into block groups, and each block group cov= ers > s_blocks_per_group(8 * Block Size) blocks. Each block group has an > associated block group descriptor; these are present directly after the > superblock. Each block group descriptor contains the location of the > inode table, and the inode and block bitmaps (note these bitmaps are onl= y > a block long, which gets us the 8 * Block Size formula covered previousl= y). >=20 > 3) Blocks > The ext4 filesystem is divided into blocks, of size s_log_block_size ^ 1= 024. > Blocks can be allocated using individual block groups's bitmaps. Note > that block 0 is invalid and its presence on extents/block tables means > it's part of a file hole, and that particular location must be read as > a block full of zeros. >=20 > 4) Inodes > The ext4 filesystem divides files/directories into inodes (originally > index nodes). Each file/socket/symlink/directory/etc (here on out referr= ed > to as a file, since there is no distinction under the ext4 filesystem) i= s > stored as a /nameless/ inode, that is stored in some block group's inode > table. Each inode has s_inode_size size (or GOOD_OLD_INODE_SIZE if it's > an old filesystem), and holds various metadata about the file. Since the > largest inode structure right now is ~160 bytes, the rest of the inode > contains inline extended attributes. Inodes' data is stored using either > data blocks (under ext2/3) or extents (under ext4). >=20 > 5) Extents > Ext4 inodes store data in extents. These let N contiguous logical blocks > that are represented by N contiguous physical blocks be represented by a > single extent structure, which minimizes filesystem metadata bloat and > speeds up block mapping (particularly due to the fact that high-quality > ext4 implementations like linux's try /really/ hard to make the file > contiguous, so it's common to have files with almost 0 fragmentation). > Inodes that use extents store them in a tree, and the top of the tree > is stored on i_data. The tree's leaves always start with an > EXT4_EXTENT_HEADER and contain EXT4_EXTENT_INDEX on eh_depth !=3D 0 > and > EXT4_EXTENT on eh_depth =3D 0; these entries are always sorted by logica= l > block. >=20 > 6) Directories > Ext4 directories are files that store name -> inode mappings for the > logical directory; this is where files get their names, which means ext4 > inodes do not themselves have names, since they can be linked (present) > multiple times with different names. Directories can store entries in tw= o > different ways: > 1) Classical linear directories: They store entries as a mostly-linked > mostly-list of EXT4_DIR_ENTRY. > 2) Hash tree directories: These are used for larger directories, with > hundreds of entries, and are designed in a backwards-compatible way. > These are not yet implemented in the Ext4Dxe driver. >=20 > 7) Journal > Ext3/4 filesystems have a journal to help protect the filesystem against > system crashes. This is not yet implemented in Ext4Dxe but is described > in detail in the Linux kernel's documentation. >=20 > The EDK2 implementation of ext4 is based only on the public documentatio= n > available at > https://www.kernel.org/doc/html/latest/filesystems/ext4/index.html > and > the FreeBSD ext2fs driver (available at > https://github.com/freebsd/freebsd-src/tree/main/sys/fs/ext2fs, > BSD-2-Clause-FreeBSD licensed). It is licensed as > SPDX-License-Identifier: BSD-2-Clause-Patent. >=20 > After a brief discussion with the community, the proposed package > location is edk2-platform/Features/Ext4Pkg > (relevant discussion: https://edk2.groups.io/g/devel/topic/83060185). >=20 > I was the main contributor and I would like to maintain the package in > the future, if possible. >=20 > Current limitations: > 1) The Ext4Dxe driver is, at the moment, read-only. > 2) The Ext4Dxe driver at the moment cannot mount older (ext2/3) > filesystems. Ensuring compatibility with > those may not be a bad idea. >=20 > I intend to test the package using the UEFI SCTs present in edk2-test, > and implement any other needed unit tests myself using the already > available unit test framework. I also intend to (privately) fuzz the > UEFI driver with bad/unusual disk images, to improve the security and > reliability of the driver. >=20 > In the future, ext4 write support should be added so edk2 has a > fully-featured RW ext4 driver. There could also be a focus on > supporting the older ext4-like filesystems, as I mentioned in the > limitations, but that is open for discussion. >=20 > The driver's handling of unclean unmounting through forced shutdown is > unclear. > Is there a position in edk2 on how to handle such cases? I don't think > FAT32 has a "this filesystem is/was dirty" and even though it seems to > me that stopping a system from booting/opening the partition because > "we may find some tiny irregularities" is not the best course of > action, I can't find a clear answer. >=20 > The driver also had to add implementations of CRC32C and CRC16, and > after talking with my mentor we quickly reached the conclusion that > these may be good candidates for inclusion in MdePkg. We also > discussed moving the Ucs2 <-> Utf8 conversion library in RedfishPkg > (BaseUcs2Utf8Lib) into MdePkg as well. Any comments? Current MdePkg BaseLib has CalculateCrc32(). So, CRC32C and CRC16 can be a= dded into BaseLib.=20 If more modules need to consume Ucs2 <-> Utf8 conversion library, BaseUcs2= Utf8Lib is generic enough to be placed in MdePkg.=20 Thanks Liming >=20 > Feel free to ask any questions you may find relevant. >=20 > Best Regards, >=20 > Pedro Falcato >=20 >=20 >=20 >=20