回复: 回复: [edk2-devel] Multithreaded compression with LZMA2

["gaoliming" <gaoliming@byosoft.com.cn>]

Daniel:

> -----邮件原件-----
> 发件人: bounce+27952+68335+4905953+8761045@groups.io
> <bounce+27952+68335+4905953+8761045@groups.io> 代表 Daniel
> Schaefer
> 发送时间: 2020年12月4日 17:03
> 收件人: devel@edk2.groups.io; gaoliming@byosoft.com.cn
> 抄送: Laszlo Ersek <lersek@redhat.com>; Lin, Derek (HPS SW)
> <derek.lin2@hpe.com>; Bret.Barkelew@microsoft.com; afish@apple.com
> 主题: Re: 回复: [edk2-devel] Multithreaded compression with LZMA2
> 
> On 12/4/20 10:28 AM, gaoliming wrote:
> > Daniel：
> >    Yes. New guided section extractor matches new compression
> algorithm.
> 
> Good. I see that we use version 18.05 of the LZMA SDK, while there's
already
> 19.00:
> https://www.7-zip.org/sdk.html
> 
> Should we use this opportunity to update?
> 
I suggest to separate them. The update doesn't block this change. 

> > For the compression algorithm, its compression ratio, compression
> performance,
> > the decompression performance, the decompression taken memory are all
> > required to be considered.
> 
> For compression ratio and performance, see my earlier emails. Summary:
> Compression ratio is basically the same.
> Performance is also the same, except when using 4 threads it compresses
our
> main image in just 40% of the time. Some images don't compress as well,
they
> take the same time to compress.
> 
> For decompression memory usage I used the xz commands on Linux again:
> 
> # LZMA1
> $ /usr/bin/time -v unxz testfile.lzma
> Maximum resident set size (kbytes): 10228, 10492, 10460, 10200, 10244 =>
> 10324.8
> 
> # LZMA2
> $ /usr/bin/time -v unxz testfile.xz
> Maximum resident set size (kbytes): 10456, 10460, 10224, 10212, 10548 =>
> 10380.0
> 
> Result: Basically the same.
> 
> I don't know how I would measure this in EDK2. Any ideas?
> 
You need to enable it in Edk2 like current LzmaCompression tool and
LzmaDecompress library, 
and apply them in the platform DSC/FDF, then measure its build and
decompression.

Thanks
Liming
> 
>  From the manpage of xz and LZMA authors:
> 
>  > LZMA2  is an updated version of LZMA1 to fix some practical issues of
> LZMA1.
>  > Compression speed and  ratios  of  LZMA1 and LZMA2 are practically
> the same.
>  > LZMA2 is better than LZMA, if you compress already compressed data.
> 
> Here's a benchmark which compares both of them, among others:
>
https://stephane.lesimple.fr/blog/lzop-vs-compress-vs-gzip-vs-bzip2-vs-lzma-
> vs-lzma2xz-benchmark-reloaded/
> Same result: They're basically the same.
> 
> 
> > Thanks
> > Liming
> >> -----邮件原件-----
> >> 发件人: bounce+27952+68292+4905953+8761045@groups.io
> >> <bounce+27952+68292+4905953+8761045@groups.io> 代表 Laszlo
> Ersek
> >> 发送时间: 2020年12月4日 7:35
> >> 收件人: Schaefer, Daniel <daniel.schaefer@hpe.com>;
> devel@edk2.groups.io
> >> 抄送: Lin, Derek (HPS SW) <derek.lin2@hpe.com>
> >> 主题: Re: [edk2-devel] Multithreaded compression with LZMA2
> >>
> >> On 12/03/20 13:11, Schaefer, Daniel wrote:
> >>>
> >>> From: Laszlo Ersek <lersek@redhat.com>
> >>> Sent: Thursday, December 3, 2020 18:24
> >>> To: devel@edk2.groups.io <devel@edk2.groups.io>; Schaefer, Daniel
> >> <daniel.schaefer@hpe.com>
> >>> Cc: Lin, Derek (HPS SW) <derek.lin2@hpe.com>
> >>> Subject: Re: [edk2-devel] Multithreaded compression with LZMA2
> >>>
> >>
> >>> "xz -T" works by splitting the input into blocks, and it generates a
> >>> multi-block compressed output.
> >>>
> >>> Yes, that's correct.
> >>>
> >>>> I'm unsure if the current LZMA
> >>> decompressor that runs inside the firmware (= guided section
extractor)
> >>> copes with multi-block input.
> >>>
> >>> I think you're right that it doesn't. But we can make the guided
section
> >> extractor use that same algorithm(LZMA2) and assign it a different
GUID,
> >> right?
> >>
> >> I guess so...
> >>
> >> Thanks
> >> Laszlo
> 
> 
> 
>