From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail.byosoft.com.cn (mail.byosoft.com.cn [58.240.74.242]) by mx.groups.io with SMTP id smtpd.web08.4814.1607407271663540872 for ; Mon, 07 Dec 2020 22:01:13 -0800 Authentication-Results: mx.groups.io; dkim=missing; spf=none, err=permanent DNS error (domain: byosoft.com.cn, ip: 58.240.74.242, mailfrom: gaoliming@byosoft.com.cn) Received: from DESKTOPS6D0PVI ([58.246.60.130]) (envelope-sender ) by 192.168.6.13 with ESMTP for ; Tue, 08 Dec 2020 14:01:00 +0800 X-WM-Sender: gaoliming@byosoft.com.cn X-WM-AuthFlag: YES X-WM-AuthUser: gaoliming@byosoft.com.cn From: "gaoliming" To: , Cc: "'Laszlo Ersek'" , "'Lin, Derek \(HPS SW\)'" , , References: <9ebc277a-4acd-f567-cc08-fa076e4a5766@redhat.com> <16c68ee6-19a7-1446-719d-3553a2df842e@redhat.com> <00a201d6c9e5$1c2949d0$547bdd70$@byosoft.com.cn> <61a0f744-9167-33cf-b4ed-b39b6456e9e2@hpe.com> In-Reply-To: <61a0f744-9167-33cf-b4ed-b39b6456e9e2@hpe.com> Subject: =?UTF-8?B?5Zue5aSNOiDlm57lpI06IFtlZGsyLWRldmVsXSBNdWx0aXRocmVhZGVkIGNvbXByZXNzaW9uIHdpdGggTFpNQTI=?= Date: Tue, 8 Dec 2020 14:01:01 +0800 Message-ID: <008601d6cd27$81b28fb0$8517af10$@byosoft.com.cn> MIME-Version: 1.0 X-Mailer: Microsoft Outlook 16.0 Thread-Index: AQHflGFSZn2WPqT5OCaPh7KljJ49mwHedulOAlE21gQCBuBYNAEdHaMhAgKlBqypkG2loA== Content-Type: text/plain; charset="gb2312" Content-Transfer-Encoding: quoted-printable Content-Language: zh-cn Daniel: > -----=D3=CA=BC=FE=D4=AD=BC=FE----- > =B7=A2=BC=FE=C8=CB: bounce+27952+68335+4905953+8761045@groups.io > =B4=FA=B1=ED Daniel > Schaefer > =B7=A2=CB=CD=CA=B1=BC=E4: 2020=C4=EA12=D4=C24=C8=D5 17:03 > =CA=D5=BC=FE=C8=CB: devel@edk2.groups.io; gaoliming@byosoft.com.cn > =B3=AD=CB=CD: Laszlo Ersek ; Lin, Derek (HPS SW) > ; Bret.Barkelew@microsoft.com; afish@apple.com > =D6=F7=CC=E2: Re: =BB=D8=B8=B4: [edk2-devel] Multithreaded compression w= ith LZMA2 >=20 > On 12/4/20 10:28 AM, gaoliming wrote: > > Daniel=A3=BA > > Yes. New guided section extractor matches new compression > algorithm. >=20 > Good. I see that we use version 18.05 of the LZMA SDK, while there's already > 19.00: > https://www.7-zip.org/sdk.html >=20 > Should we use this opportunity to update? >=20 I suggest to separate them. The update doesn't block this change.=20 > > For the compression algorithm, its compression ratio, compression > performance, > > the decompression performance, the decompression taken memory are all > > required to be considered. >=20 > For compression ratio and performance, see my earlier emails. Summary: > Compression ratio is basically the same. > Performance is also the same, except when using 4 threads it compresses our > main image in just 40% of the time. Some images don't compress as well, they > take the same time to compress. >=20 > For decompression memory usage I used the xz commands on Linux again: >=20 > # LZMA1 > $ /usr/bin/time -v unxz testfile.lzma > Maximum resident set size (kbytes): 10228, 10492, 10460, 10200, 10244 = =3D> > 10324.8 >=20 > # LZMA2 > $ /usr/bin/time -v unxz testfile.xz > Maximum resident set size (kbytes): 10456, 10460, 10224, 10212, 10548 = =3D> > 10380.0 >=20 > Result: Basically the same. >=20 > I don't know how I would measure this in EDK2. Any ideas? >=20 You need to enable it in Edk2 like current LzmaCompression tool and LzmaDecompress library,=20 and apply them in the platform DSC/FDF, then measure its build and decompression. Thanks Liming >=20 > From the manpage of xz and LZMA authors: >=20 > > LZMA2 is an updated version of LZMA1 to fix some practical issues of > LZMA1. > > Compression speed and ratios of LZMA1 and LZMA2 are practically > the same. > > LZMA2 is better than LZMA, if you compress already compressed data. >=20 > Here's a benchmark which compares both of them, among others: > https://stephane.lesimple.fr/blog/lzop-vs-compress-vs-gzip-vs-bzip2-vs-lzm= a- > vs-lzma2xz-benchmark-reloaded/ > Same result: They're basically the same. >=20 >=20 > > Thanks > > Liming > >> -----=D3=CA=BC=FE=D4=AD=BC=FE----- > >> =B7=A2=BC=FE=C8=CB: bounce+27952+68292+4905953+8761045@groups.io > >> =B4=FA=B1=ED Laszlo > Ersek > >> =B7=A2=CB=CD=CA=B1=BC=E4: 2020=C4=EA12=D4=C24=C8=D5 7:35 > >> =CA=D5=BC=FE=C8=CB: Schaefer, Daniel ; > devel@edk2.groups.io > >> =B3=AD=CB=CD: Lin, Derek (HPS SW) > >> =D6=F7=CC=E2: Re: [edk2-devel] Multithreaded compression with LZMA2 > >> > >> On 12/03/20 13:11, Schaefer, Daniel wrote: > >>> > >>> From: Laszlo Ersek > >>> Sent: Thursday, December 3, 2020 18:24 > >>> To: devel@edk2.groups.io ; Schaefer, Daniel > >> > >>> Cc: Lin, Derek (HPS SW) > >>> Subject: Re: [edk2-devel] Multithreaded compression with LZMA2 > >>> > >> > >>> "xz -T" works by splitting the input into blocks, and it generates a > >>> multi-block compressed output. > >>> > >>> Yes, that's correct. > >>> > >>>> I'm unsure if the current LZMA > >>> decompressor that runs inside the firmware (=3D guided section extractor) > >>> copes with multi-block input. > >>> > >>> I think you're right that it doesn't. But we can make the guided section > >> extractor use that same algorithm(LZMA2) and assign it a different GUID, > >> right? > >> > >> I guess so... > >> > >> Thanks > >> Laszlo >=20 >=20 >=20 >=20