Problem with decompression on EDK2

Problem with decompression on EDK2

[Loh, Tien Hock]

Hi

I have an issue while porting a new platform to EDK2. After porting the Stratix 10 platform (ARM) to EDK2 it works correctly. However, when I tried to add more INF to the FDF file, it failed to decompress the image during boot time, log as below
INFO:    DDR: DRAM calibration success.
INFO:    Scrubbing ECC
INFO:    Init HPS NOC's DDR Scheduler.
NOTICE:  BL2: v2.1(debug):v2.1-15-g5880144-dirty
NOTICE:  BL2: Built : 14:02:01, Aug 22 2019
INFO:    BL2: Doing platform setup
INFO:    BL2: Loading image id 3
INFO:    Loading image id=3 at address 0xffe1c000
INFO:    Image id=3 loaded: 0xffe1c000 - 0xffe22019
INFO:    BL2: Loading image id 5
INFO:    Loading image id=5 at address 0x50000
INFO:    Image id=5 loaded: 0x50000 - 0x150000
NOTICE:  BL2: Booting BL31
INFO:    Entry point address = 0xffe1c000
INFO:    SPSR = 0x3cd
NOTICE:  BL31: v2.1(release):v2.1-604-g3441952
NOTICE:  BL31: Built : 15:32:55, Sep 25 2019
UEFI firmware (version 1.0 built at 15:15:26 on Sep 26 2019)
Decompress Failed - Invalid Parameter

ASSERT_EFI_ERROR (Status = Not Found)
ASSERT [ArmPlatformPrePiUniCore] /nfs/png/disks/swuser_work_thloh/push/build/edk2/ArmPlatformPkg/PrePi/PrePi.c(151): !EFI_ERROR (Status)

This happens if I add more items to the image, and if I reduce the size of the image, it works correctly again. This could be a clue to what could've gone wrong. Does anyone have any ideas what might've gone wrong?
I traced the decompression to LzmaDecode, and got lost in the code there.

Thanks!

Re: [edk2-devel] Problem with decompression on EDK2

[Laszlo Ersek]

On 09/26/19 11:27, Loh, Tien Hock wrote:
> Hi
> 
> I have an issue while porting a new platform to EDK2. After porting the Stratix 10 platform (ARM) to EDK2 it works correctly. However, when I tried to add more INF to the FDF file, it failed to decompress the image during boot time, log as below
> INFO:    DDR: DRAM calibration success.
> INFO:    Scrubbing ECC
> INFO:    Init HPS NOC's DDR Scheduler.
> NOTICE:  BL2: v2.1(debug):v2.1-15-g5880144-dirty
> NOTICE:  BL2: Built : 14:02:01, Aug 22 2019
> INFO:    BL2: Doing platform setup
> INFO:    BL2: Loading image id 3
> INFO:    Loading image id=3 at address 0xffe1c000
> INFO:    Image id=3 loaded: 0xffe1c000 - 0xffe22019
> INFO:    BL2: Loading image id 5
> INFO:    Loading image id=5 at address 0x50000
> INFO:    Image id=5 loaded: 0x50000 - 0x150000
> NOTICE:  BL2: Booting BL31
> INFO:    Entry point address = 0xffe1c000
> INFO:    SPSR = 0x3cd
> NOTICE:  BL31: v2.1(release):v2.1-604-g3441952
> NOTICE:  BL31: Built : 15:32:55, Sep 25 2019
> UEFI firmware (version 1.0 built at 15:15:26 on Sep 26 2019)
> Decompress Failed - Invalid Parameter
> 
> ASSERT_EFI_ERROR (Status = Not Found)
> ASSERT [ArmPlatformPrePiUniCore] /nfs/png/disks/swuser_work_thloh/push/build/edk2/ArmPlatformPkg/PrePi/PrePi.c(151): !EFI_ERROR (Status)
> 
> This happens if I add more items to the image, and if I reduce the size of the image, it works correctly again. This could be a clue to what could've gone wrong. Does anyone have any ideas what might've gone wrong?
> I traced the decompression to LzmaDecode, and got lost in the code there.

The compressed section in a firmware volume is likely truncated at build
time, and does not decompress at boot time. I would expect the "build"
utility to report this issue, and stop with an error.

When you run "build" in the successful and in the failing cases, what
are the stats, respectively, that are printed at the end of the build logs?

Also, you could check your FDF file(s) near
"EE4E5898-3914-4259-9D6E-DC7BD79403CF"; perhaps you find interesting bits.

Furthermore, if the boundary seems to be 16MB, then it could be an issue
in the processing code. See "ExtendedSize" in
"MdePkg/Include/Pi/PiFirmwareFile.h". I vaguely recall issues around
some code trying to parse EFI_FFS_FILE_HEADER2 as EFI_FFS_FILE_HEADER,
or EFI_COMMON_SECTION_HEADER2 as EFI_COMMON_SECTION_HEADER.

Thanks
Laszlo

Re: [edk2-devel] Problem with decompression on EDK2

[Liming Gao]

Can you share the generated FD image? I can help check whether it is generated correctly with compression. 

Thanks
Liming

>-----Original Message-----
>From: devel@edk2.groups.io [mailto:devel@edk2.groups.io] On Behalf Of
>Laszlo Ersek
>Sent: Friday, September 27, 2019 3:22 AM
>To: devel@edk2.groups.io; Loh, Tien Hock <tien.hock.loh@intel.com>
>Cc: leif.lindholm@linaro.org; ard.biesheuvel@linaro.org
>Subject: Re: [edk2-devel] Problem with decompression on EDK2
>
>On 09/26/19 11:27, Loh, Tien Hock wrote:
>> Hi
>>
>> I have an issue while porting a new platform to EDK2. After porting the
>Stratix 10 platform (ARM) to EDK2 it works correctly. However, when I tried to
>add more INF to the FDF file, it failed to decompress the image during boot
>time, log as below
>> INFO:    DDR: DRAM calibration success.
>> INFO:    Scrubbing ECC
>> INFO:    Init HPS NOC's DDR Scheduler.
>> NOTICE:  BL2: v2.1(debug):v2.1-15-g5880144-dirty
>> NOTICE:  BL2: Built : 14:02:01, Aug 22 2019
>> INFO:    BL2: Doing platform setup
>> INFO:    BL2: Loading image id 3
>> INFO:    Loading image id=3 at address 0xffe1c000
>> INFO:    Image id=3 loaded: 0xffe1c000 - 0xffe22019
>> INFO:    BL2: Loading image id 5
>> INFO:    Loading image id=5 at address 0x50000
>> INFO:    Image id=5 loaded: 0x50000 - 0x150000
>> NOTICE:  BL2: Booting BL31
>> INFO:    Entry point address = 0xffe1c000
>> INFO:    SPSR = 0x3cd
>> NOTICE:  BL31: v2.1(release):v2.1-604-g3441952
>> NOTICE:  BL31: Built : 15:32:55, Sep 25 2019
>> UEFI firmware (version 1.0 built at 15:15:26 on Sep 26 2019)
>> Decompress Failed - Invalid Parameter
>>
>> ASSERT_EFI_ERROR (Status = Not Found)
>> ASSERT [ArmPlatformPrePiUniCore]
>/nfs/png/disks/swuser_work_thloh/push/build/edk2/ArmPlatformPkg/PrePi
>/PrePi.c(151): !EFI_ERROR (Status)
>>
>> This happens if I add more items to the image, and if I reduce the size of the
>image, it works correctly again. This could be a clue to what could've gone
>wrong. Does anyone have any ideas what might've gone wrong?
>> I traced the decompression to LzmaDecode, and got lost in the code there.
>
>The compressed section in a firmware volume is likely truncated at build
>time, and does not decompress at boot time. I would expect the "build"
>utility to report this issue, and stop with an error.
>
>When you run "build" in the successful and in the failing cases, what
>are the stats, respectively, that are printed at the end of the build logs?
>
>Also, you could check your FDF file(s) near
>"EE4E5898-3914-4259-9D6E-DC7BD79403CF"; perhaps you find interesting bits.
>
>Furthermore, if the boundary seems to be 16MB, then it could be an issue
>in the processing code. See "ExtendedSize" in
>"MdePkg/Include/Pi/PiFirmwareFile.h". I vaguely recall issues around
>some code trying to parse EFI_FFS_FILE_HEADER2 as EFI_FFS_FILE_HEADER,
>or EFI_COMMON_SECTION_HEADER2 as EFI_COMMON_SECTION_HEADER.
>
>Thanks
>Laszlo
>
>

Re: [edk2-devel] Problem with decompression on EDK2

[Loh, Tien Hock]

[-- Attachment #1: Type: text/plain, Size: 3803 bytes --]

Hi Liming,

I've tested another workaround - Move the FD file to a higher RAM location (originally, it is loaded into 0x50000, now it is at 0x10000000)
With the FD file moved to 0x10000000, the decompression works every time, even with bigger FD. I've attached the my FD file in the mail. 

Thanks
Tien Hock

> -----Original Message-----
> From: Gao, Liming <liming.gao@intel.com>
> Sent: Friday, September 27, 2019 8:13 AM
> To: devel@edk2.groups.io; lersek@redhat.com; Loh, Tien Hock
> <tien.hock.loh@intel.com>
> Cc: leif.lindholm@linaro.org; ard.biesheuvel@linaro.org
> Subject: RE: [edk2-devel] Problem with decompression on EDK2
> 
> Can you share the generated FD image? I can help check whether it is
> generated correctly with compression.
> 
> Thanks
> Liming
> 
> >-----Original Message-----
> >From: devel@edk2.groups.io [mailto:devel@edk2.groups.io] On Behalf Of
> >Laszlo Ersek
> >Sent: Friday, September 27, 2019 3:22 AM
> >To: devel@edk2.groups.io; Loh, Tien Hock <tien.hock.loh@intel.com>
> >Cc: leif.lindholm@linaro.org; ard.biesheuvel@linaro.org
> >Subject: Re: [edk2-devel] Problem with decompression on EDK2
> >
> >On 09/26/19 11:27, Loh, Tien Hock wrote:
> >> Hi
> >>
> >> I have an issue while porting a new platform to EDK2. After porting
> >> the
> >Stratix 10 platform (ARM) to EDK2 it works correctly. However, when I
> >tried to add more INF to the FDF file, it failed to decompress the
> >image during boot time, log as below
> >> INFO:    DDR: DRAM calibration success.
> >> INFO:    Scrubbing ECC
> >> INFO:    Init HPS NOC's DDR Scheduler.
> >> NOTICE:  BL2: v2.1(debug):v2.1-15-g5880144-dirty
> >> NOTICE:  BL2: Built : 14:02:01, Aug 22 2019
> >> INFO:    BL2: Doing platform setup
> >> INFO:    BL2: Loading image id 3
> >> INFO:    Loading image id=3 at address 0xffe1c000
> >> INFO:    Image id=3 loaded: 0xffe1c000 - 0xffe22019
> >> INFO:    BL2: Loading image id 5
> >> INFO:    Loading image id=5 at address 0x50000
> >> INFO:    Image id=5 loaded: 0x50000 - 0x150000
> >> NOTICE:  BL2: Booting BL31
> >> INFO:    Entry point address = 0xffe1c000
> >> INFO:    SPSR = 0x3cd
> >> NOTICE:  BL31: v2.1(release):v2.1-604-g3441952
> >> NOTICE:  BL31: Built : 15:32:55, Sep 25 2019 UEFI firmware (version
> >> 1.0 built at 15:15:26 on Sep 26 2019) Decompress Failed - Invalid
> >> Parameter
> >>
> >> ASSERT_EFI_ERROR (Status = Not Found) ASSERT
> >> [ArmPlatformPrePiUniCore]
> >/nfs/png/disks/swuser_work_thloh/push/build/edk2/ArmPlatformPkg/PreP
> i
> >/PrePi.c(151): !EFI_ERROR (Status)
> >>
> >> This happens if I add more items to the image, and if I reduce the
> >> size of the
> >image, it works correctly again. This could be a clue to what could've
> >gone wrong. Does anyone have any ideas what might've gone wrong?
> >> I traced the decompression to LzmaDecode, and got lost in the code there.
> >
> >The compressed section in a firmware volume is likely truncated at
> >build time, and does not decompress at boot time. I would expect the
> "build"
> >utility to report this issue, and stop with an error.
> >
> >When you run "build" in the successful and in the failing cases, what
> >are the stats, respectively, that are printed at the end of the build logs?
> >
> >Also, you could check your FDF file(s) near
> >"EE4E5898-3914-4259-9D6E-DC7BD79403CF"; perhaps you find interesting
> bits.
> >
> >Furthermore, if the boundary seems to be 16MB, then it could be an
> >issue in the processing code. See "ExtendedSize" in
> >"MdePkg/Include/Pi/PiFirmwareFile.h". I vaguely recall issues around
> >some code trying to parse EFI_FFS_FILE_HEADER2 as EFI_FFS_FILE_HEADER,
> >or EFI_COMMON_SECTION_HEADER2 as EFI_COMMON_SECTION_HEADER.
> >
> >Thanks
> >Laszlo
> >
> >


[-- Attachment #2: INTELSTRATIX10_EFI.fd --]
[-- Type: application/octet-stream, Size: 1048576 bytes --]

Re: [edk2-devel] Problem with decompression on EDK2

[Liming Gao]

Tien Hock:
  The image is correct. It can be decompressed by lzma. 
  When FD is loaded into 0x50000, I suspect that the image data can't fully read. Then, the decompress will failure. 

  FD image last valid byte offset is 0xC1B8F. Its value is 0x83. When FD base address is 0x50000, this position is 0x111B8F. 
  You can read this place and check its value. 

Thanks
Liming
>-----Original Message-----
>From: Loh, Tien Hock
>Sent: Friday, September 27, 2019 3:05 PM
>To: Gao, Liming <liming.gao@intel.com>; devel@edk2.groups.io;
>lersek@redhat.com
>Cc: leif.lindholm@linaro.org; ard.biesheuvel@linaro.org
>Subject: RE: [edk2-devel] Problem with decompression on EDK2
>
>Hi Liming,
>
>I've tested another workaround - Move the FD file to a higher RAM location
>(originally, it is loaded into 0x50000, now it is at 0x10000000)
>With the FD file moved to 0x10000000, the decompression works every time,
>even with bigger FD. I've attached the my FD file in the mail.
>
>Thanks
>Tien Hock
>
>> -----Original Message-----
>> From: Gao, Liming <liming.gao@intel.com>
>> Sent: Friday, September 27, 2019 8:13 AM
>> To: devel@edk2.groups.io; lersek@redhat.com; Loh, Tien Hock
>> <tien.hock.loh@intel.com>
>> Cc: leif.lindholm@linaro.org; ard.biesheuvel@linaro.org
>> Subject: RE: [edk2-devel] Problem with decompression on EDK2
>>
>> Can you share the generated FD image? I can help check whether it is
>> generated correctly with compression.
>>
>> Thanks
>> Liming
>>
>> >-----Original Message-----
>> >From: devel@edk2.groups.io [mailto:devel@edk2.groups.io] On Behalf Of
>> >Laszlo Ersek
>> >Sent: Friday, September 27, 2019 3:22 AM
>> >To: devel@edk2.groups.io; Loh, Tien Hock <tien.hock.loh@intel.com>
>> >Cc: leif.lindholm@linaro.org; ard.biesheuvel@linaro.org
>> >Subject: Re: [edk2-devel] Problem with decompression on EDK2
>> >
>> >On 09/26/19 11:27, Loh, Tien Hock wrote:
>> >> Hi
>> >>
>> >> I have an issue while porting a new platform to EDK2. After porting
>> >> the
>> >Stratix 10 platform (ARM) to EDK2 it works correctly. However, when I
>> >tried to add more INF to the FDF file, it failed to decompress the
>> >image during boot time, log as below
>> >> INFO:    DDR: DRAM calibration success.
>> >> INFO:    Scrubbing ECC
>> >> INFO:    Init HPS NOC's DDR Scheduler.
>> >> NOTICE:  BL2: v2.1(debug):v2.1-15-g5880144-dirty
>> >> NOTICE:  BL2: Built : 14:02:01, Aug 22 2019
>> >> INFO:    BL2: Doing platform setup
>> >> INFO:    BL2: Loading image id 3
>> >> INFO:    Loading image id=3 at address 0xffe1c000
>> >> INFO:    Image id=3 loaded: 0xffe1c000 - 0xffe22019
>> >> INFO:    BL2: Loading image id 5
>> >> INFO:    Loading image id=5 at address 0x50000
>> >> INFO:    Image id=5 loaded: 0x50000 - 0x150000
>> >> NOTICE:  BL2: Booting BL31
>> >> INFO:    Entry point address = 0xffe1c000
>> >> INFO:    SPSR = 0x3cd
>> >> NOTICE:  BL31: v2.1(release):v2.1-604-g3441952
>> >> NOTICE:  BL31: Built : 15:32:55, Sep 25 2019 UEFI firmware (version
>> >> 1.0 built at 15:15:26 on Sep 26 2019) Decompress Failed - Invalid
>> >> Parameter
>> >>
>> >> ASSERT_EFI_ERROR (Status = Not Found) ASSERT
>> >> [ArmPlatformPrePiUniCore]
>> >/nfs/png/disks/swuser_work_thloh/push/build/edk2/ArmPlatformPkg/Pr
>eP
>> i
>> >/PrePi.c(151): !EFI_ERROR (Status)
>> >>
>> >> This happens if I add more items to the image, and if I reduce the
>> >> size of the
>> >image, it works correctly again. This could be a clue to what could've
>> >gone wrong. Does anyone have any ideas what might've gone wrong?
>> >> I traced the decompression to LzmaDecode, and got lost in the code
>there.
>> >
>> >The compressed section in a firmware volume is likely truncated at
>> >build time, and does not decompress at boot time. I would expect the
>> "build"
>> >utility to report this issue, and stop with an error.
>> >
>> >When you run "build" in the successful and in the failing cases, what
>> >are the stats, respectively, that are printed at the end of the build logs?
>> >
>> >Also, you could check your FDF file(s) near
>> >"EE4E5898-3914-4259-9D6E-DC7BD79403CF"; perhaps you find interesting
>> bits.
>> >
>> >Furthermore, if the boundary seems to be 16MB, then it could be an
>> >issue in the processing code. See "ExtendedSize" in
>> >"MdePkg/Include/Pi/PiFirmwareFile.h". I vaguely recall issues around
>> >some code trying to parse EFI_FFS_FILE_HEADER2 as
>EFI_FFS_FILE_HEADER,
>> >or EFI_COMMON_SECTION_HEADER2 as
>EFI_COMMON_SECTION_HEADER.
>> >
>> >Thanks
>> >Laszlo
>> >
>> >

Re: [edk2-devel] Problem with decompression on EDK2

[Loh, Tien Hock]

Hi Liming,

Thanks for the quick response.
Can you help show me where I can find more information on the decompression (ie. how to test the image, etc.)?

Thanks
Tien Hock

> -----Original Message-----
> From: Gao, Liming <liming.gao@intel.com>
> Sent: Sunday, September 29, 2019 3:06 PM
> To: Loh, Tien Hock <tien.hock.loh@intel.com>; devel@edk2.groups.io;
> lersek@redhat.com
> Cc: leif.lindholm@linaro.org; ard.biesheuvel@linaro.org
> Subject: RE: [edk2-devel] Problem with decompression on EDK2
> 
> Tien Hock:
>   The image is correct. It can be decompressed by lzma.
>   When FD is loaded into 0x50000, I suspect that the image data can't fully
> read. Then, the decompress will failure.
> 
>   FD image last valid byte offset is 0xC1B8F. Its value is 0x83. When FD base
> address is 0x50000, this position is 0x111B8F.
>   You can read this place and check its value.
> 
> Thanks
> Liming
> >-----Original Message-----
> >From: Loh, Tien Hock
> >Sent: Friday, September 27, 2019 3:05 PM
> >To: Gao, Liming <liming.gao@intel.com>; devel@edk2.groups.io;
> >lersek@redhat.com
> >Cc: leif.lindholm@linaro.org; ard.biesheuvel@linaro.org
> >Subject: RE: [edk2-devel] Problem with decompression on EDK2
> >
> >Hi Liming,
> >
> >I've tested another workaround - Move the FD file to a higher RAM
> >location (originally, it is loaded into 0x50000, now it is at
> >0x10000000) With the FD file moved to 0x10000000, the decompression
> >works every time, even with bigger FD. I've attached the my FD file in the
> mail.
> >
> >Thanks
> >Tien Hock
> >
> >> -----Original Message-----
> >> From: Gao, Liming <liming.gao@intel.com>
> >> Sent: Friday, September 27, 2019 8:13 AM
> >> To: devel@edk2.groups.io; lersek@redhat.com; Loh, Tien Hock
> >> <tien.hock.loh@intel.com>
> >> Cc: leif.lindholm@linaro.org; ard.biesheuvel@linaro.org
> >> Subject: RE: [edk2-devel] Problem with decompression on EDK2
> >>
> >> Can you share the generated FD image? I can help check whether it is
> >> generated correctly with compression.
> >>
> >> Thanks
> >> Liming
> >>
> >> >-----Original Message-----
> >> >From: devel@edk2.groups.io [mailto:devel@edk2.groups.io] On Behalf
> >> >Of Laszlo Ersek
> >> >Sent: Friday, September 27, 2019 3:22 AM
> >> >To: devel@edk2.groups.io; Loh, Tien Hock <tien.hock.loh@intel.com>
> >> >Cc: leif.lindholm@linaro.org; ard.biesheuvel@linaro.org
> >> >Subject: Re: [edk2-devel] Problem with decompression on EDK2
> >> >
> >> >On 09/26/19 11:27, Loh, Tien Hock wrote:
> >> >> Hi
> >> >>
> >> >> I have an issue while porting a new platform to EDK2. After
> >> >> porting the
> >> >Stratix 10 platform (ARM) to EDK2 it works correctly. However, when
> >> >I tried to add more INF to the FDF file, it failed to decompress the
> >> >image during boot time, log as below
> >> >> INFO:    DDR: DRAM calibration success.
> >> >> INFO:    Scrubbing ECC
> >> >> INFO:    Init HPS NOC's DDR Scheduler.
> >> >> NOTICE:  BL2: v2.1(debug):v2.1-15-g5880144-dirty
> >> >> NOTICE:  BL2: Built : 14:02:01, Aug 22 2019
> >> >> INFO:    BL2: Doing platform setup
> >> >> INFO:    BL2: Loading image id 3
> >> >> INFO:    Loading image id=3 at address 0xffe1c000
> >> >> INFO:    Image id=3 loaded: 0xffe1c000 - 0xffe22019
> >> >> INFO:    BL2: Loading image id 5
> >> >> INFO:    Loading image id=5 at address 0x50000
> >> >> INFO:    Image id=5 loaded: 0x50000 - 0x150000
> >> >> NOTICE:  BL2: Booting BL31
> >> >> INFO:    Entry point address = 0xffe1c000
> >> >> INFO:    SPSR = 0x3cd
> >> >> NOTICE:  BL31: v2.1(release):v2.1-604-g3441952
> >> >> NOTICE:  BL31: Built : 15:32:55, Sep 25 2019 UEFI firmware
> >> >> (version
> >> >> 1.0 built at 15:15:26 on Sep 26 2019) Decompress Failed - Invalid
> >> >> Parameter
> >> >>
> >> >> ASSERT_EFI_ERROR (Status = Not Found) ASSERT
> >> >> [ArmPlatformPrePiUniCore]
> >>
> >/nfs/png/disks/swuser_work_thloh/push/build/edk2/ArmPlatformPkg/Pr
> >eP
> >> i
> >> >/PrePi.c(151): !EFI_ERROR (Status)
> >> >>
> >> >> This happens if I add more items to the image, and if I reduce the
> >> >> size of the
> >> >image, it works correctly again. This could be a clue to what
> >> >could've gone wrong. Does anyone have any ideas what might've gone
> wrong?
> >> >> I traced the decompression to LzmaDecode, and got lost in the code
> >there.
> >> >
> >> >The compressed section in a firmware volume is likely truncated at
> >> >build time, and does not decompress at boot time. I would expect the
> >> "build"
> >> >utility to report this issue, and stop with an error.
> >> >
> >> >When you run "build" in the successful and in the failing cases,
> >> >what are the stats, respectively, that are printed at the end of the build
> logs?
> >> >
> >> >Also, you could check your FDF file(s) near
> >> >"EE4E5898-3914-4259-9D6E-DC7BD79403CF"; perhaps you find
> interesting
> >> bits.
> >> >
> >> >Furthermore, if the boundary seems to be 16MB, then it could be an
> >> >issue in the processing code. See "ExtendedSize" in
> >> >"MdePkg/Include/Pi/PiFirmwareFile.h". I vaguely recall issues around
> >> >some code trying to parse EFI_FFS_FILE_HEADER2 as
> >EFI_FFS_FILE_HEADER,
> >> >or EFI_COMMON_SECTION_HEADER2 as
> >EFI_COMMON_SECTION_HEADER.
> >> >
> >> >Thanks
> >> >Laszlo
> >> >
> >> >

Re: [edk2-devel] Problem with decompression on EDK2

[Liming Gao]

BaseTools VolInfo tool. Its input is the binary FvImage file. In the same directory, GuidedSectionTools.txt (built in the FV directory) should be here. 

Thanks
Liming
> -----Original Message-----
> From: Loh, Tien Hock <tien.hock.loh@intel.com>
> Sent: Tuesday, October 1, 2019 10:34 AM
> To: Gao, Liming <liming.gao@intel.com>; devel@edk2.groups.io; lersek@redhat.com
> Cc: leif.lindholm@linaro.org; ard.biesheuvel@linaro.org
> Subject: RE: [edk2-devel] Problem with decompression on EDK2
> 
> Hi Liming,
> 
> Thanks for the quick response.
> Can you help show me where I can find more information on the decompression (ie. how to test the image, etc.)?
> 
> Thanks
> Tien Hock
> 
> > -----Original Message-----
> > From: Gao, Liming <liming.gao@intel.com>
> > Sent: Sunday, September 29, 2019 3:06 PM
> > To: Loh, Tien Hock <tien.hock.loh@intel.com>; devel@edk2.groups.io;
> > lersek@redhat.com
> > Cc: leif.lindholm@linaro.org; ard.biesheuvel@linaro.org
> > Subject: RE: [edk2-devel] Problem with decompression on EDK2
> >
> > Tien Hock:
> >   The image is correct. It can be decompressed by lzma.
> >   When FD is loaded into 0x50000, I suspect that the image data can't fully
> > read. Then, the decompress will failure.
> >
> >   FD image last valid byte offset is 0xC1B8F. Its value is 0x83. When FD base
> > address is 0x50000, this position is 0x111B8F.
> >   You can read this place and check its value.
> >
> > Thanks
> > Liming
> > >-----Original Message-----
> > >From: Loh, Tien Hock
> > >Sent: Friday, September 27, 2019 3:05 PM
> > >To: Gao, Liming <liming.gao@intel.com>; devel@edk2.groups.io;
> > >lersek@redhat.com
> > >Cc: leif.lindholm@linaro.org; ard.biesheuvel@linaro.org
> > >Subject: RE: [edk2-devel] Problem with decompression on EDK2
> > >
> > >Hi Liming,
> > >
> > >I've tested another workaround - Move the FD file to a higher RAM
> > >location (originally, it is loaded into 0x50000, now it is at
> > >0x10000000) With the FD file moved to 0x10000000, the decompression
> > >works every time, even with bigger FD. I've attached the my FD file in the
> > mail.
> > >
> > >Thanks
> > >Tien Hock
> > >
> > >> -----Original Message-----
> > >> From: Gao, Liming <liming.gao@intel.com>
> > >> Sent: Friday, September 27, 2019 8:13 AM
> > >> To: devel@edk2.groups.io; lersek@redhat.com; Loh, Tien Hock
> > >> <tien.hock.loh@intel.com>
> > >> Cc: leif.lindholm@linaro.org; ard.biesheuvel@linaro.org
> > >> Subject: RE: [edk2-devel] Problem with decompression on EDK2
> > >>
> > >> Can you share the generated FD image? I can help check whether it is
> > >> generated correctly with compression.
> > >>
> > >> Thanks
> > >> Liming
> > >>
> > >> >-----Original Message-----
> > >> >From: devel@edk2.groups.io [mailto:devel@edk2.groups.io] On Behalf
> > >> >Of Laszlo Ersek
> > >> >Sent: Friday, September 27, 2019 3:22 AM
> > >> >To: devel@edk2.groups.io; Loh, Tien Hock <tien.hock.loh@intel.com>
> > >> >Cc: leif.lindholm@linaro.org; ard.biesheuvel@linaro.org
> > >> >Subject: Re: [edk2-devel] Problem with decompression on EDK2
> > >> >
> > >> >On 09/26/19 11:27, Loh, Tien Hock wrote:
> > >> >> Hi
> > >> >>
> > >> >> I have an issue while porting a new platform to EDK2. After
> > >> >> porting the
> > >> >Stratix 10 platform (ARM) to EDK2 it works correctly. However, when
> > >> >I tried to add more INF to the FDF file, it failed to decompress the
> > >> >image during boot time, log as below
> > >> >> INFO:    DDR: DRAM calibration success.
> > >> >> INFO:    Scrubbing ECC
> > >> >> INFO:    Init HPS NOC's DDR Scheduler.
> > >> >> NOTICE:  BL2: v2.1(debug):v2.1-15-g5880144-dirty
> > >> >> NOTICE:  BL2: Built : 14:02:01, Aug 22 2019
> > >> >> INFO:    BL2: Doing platform setup
> > >> >> INFO:    BL2: Loading image id 3
> > >> >> INFO:    Loading image id=3 at address 0xffe1c000
> > >> >> INFO:    Image id=3 loaded: 0xffe1c000 - 0xffe22019
> > >> >> INFO:    BL2: Loading image id 5
> > >> >> INFO:    Loading image id=5 at address 0x50000
> > >> >> INFO:    Image id=5 loaded: 0x50000 - 0x150000
> > >> >> NOTICE:  BL2: Booting BL31
> > >> >> INFO:    Entry point address = 0xffe1c000
> > >> >> INFO:    SPSR = 0x3cd
> > >> >> NOTICE:  BL31: v2.1(release):v2.1-604-g3441952
> > >> >> NOTICE:  BL31: Built : 15:32:55, Sep 25 2019 UEFI firmware
> > >> >> (version
> > >> >> 1.0 built at 15:15:26 on Sep 26 2019) Decompress Failed - Invalid
> > >> >> Parameter
> > >> >>
> > >> >> ASSERT_EFI_ERROR (Status = Not Found) ASSERT
> > >> >> [ArmPlatformPrePiUniCore]
> > >>
> > >/nfs/png/disks/swuser_work_thloh/push/build/edk2/ArmPlatformPkg/Pr
> > >eP
> > >> i
> > >> >/PrePi.c(151): !EFI_ERROR (Status)
> > >> >>
> > >> >> This happens if I add more items to the image, and if I reduce the
> > >> >> size of the
> > >> >image, it works correctly again. This could be a clue to what
> > >> >could've gone wrong. Does anyone have any ideas what might've gone
> > wrong?
> > >> >> I traced the decompression to LzmaDecode, and got lost in the code
> > >there.
> > >> >
> > >> >The compressed section in a firmware volume is likely truncated at
> > >> >build time, and does not decompress at boot time. I would expect the
> > >> "build"
> > >> >utility to report this issue, and stop with an error.
> > >> >
> > >> >When you run "build" in the successful and in the failing cases,
> > >> >what are the stats, respectively, that are printed at the end of the build
> > logs?
> > >> >
> > >> >Also, you could check your FDF file(s) near
> > >> >"EE4E5898-3914-4259-9D6E-DC7BD79403CF"; perhaps you find
> > interesting
> > >> bits.
> > >> >
> > >> >Furthermore, if the boundary seems to be 16MB, then it could be an
> > >> >issue in the processing code. See "ExtendedSize" in
> > >> >"MdePkg/Include/Pi/PiFirmwareFile.h". I vaguely recall issues around
> > >> >some code trying to parse EFI_FFS_FILE_HEADER2 as
> > >EFI_FFS_FILE_HEADER,
> > >> >or EFI_COMMON_SECTION_HEADER2 as
> > >EFI_COMMON_SECTION_HEADER.
> > >> >
> > >> >Thanks
> > >> >Laszlo
> > >> >
> > >> >

Re: [edk2-devel] Problem with decompression on EDK2

[Loh, Tien Hock]

Ok, noted with thanks Liming. I'll try the tools out. 

Thanks
Tien Hock

> -----Original Message-----
> From: Gao, Liming <liming.gao@intel.com>
> Sent: Tuesday, October 8, 2019 10:04 PM
> To: Loh, Tien Hock <tien.hock.loh@intel.com>; devel@edk2.groups.io;
> lersek@redhat.com
> Cc: leif.lindholm@linaro.org; ard.biesheuvel@linaro.org
> Subject: RE: [edk2-devel] Problem with decompression on EDK2
> 
> BaseTools VolInfo tool. Its input is the binary FvImage file. In the same
> directory, GuidedSectionTools.txt (built in the FV directory) should be here.
> 
> Thanks
> Liming
> > -----Original Message-----
> > From: Loh, Tien Hock <tien.hock.loh@intel.com>
> > Sent: Tuesday, October 1, 2019 10:34 AM
> > To: Gao, Liming <liming.gao@intel.com>; devel@edk2.groups.io;
> > lersek@redhat.com
> > Cc: leif.lindholm@linaro.org; ard.biesheuvel@linaro.org
> > Subject: RE: [edk2-devel] Problem with decompression on EDK2
> >
> > Hi Liming,
> >
> > Thanks for the quick response.
> > Can you help show me where I can find more information on the
> decompression (ie. how to test the image, etc.)?
> >
> > Thanks
> > Tien Hock
> >
> > > -----Original Message-----
> > > From: Gao, Liming <liming.gao@intel.com>
> > > Sent: Sunday, September 29, 2019 3:06 PM
> > > To: Loh, Tien Hock <tien.hock.loh@intel.com>; devel@edk2.groups.io;
> > > lersek@redhat.com
> > > Cc: leif.lindholm@linaro.org; ard.biesheuvel@linaro.org
> > > Subject: RE: [edk2-devel] Problem with decompression on EDK2
> > >
> > > Tien Hock:
> > >   The image is correct. It can be decompressed by lzma.
> > >   When FD is loaded into 0x50000, I suspect that the image data
> > > can't fully read. Then, the decompress will failure.
> > >
> > >   FD image last valid byte offset is 0xC1B8F. Its value is 0x83.
> > > When FD base address is 0x50000, this position is 0x111B8F.
> > >   You can read this place and check its value.
> > >
> > > Thanks
> > > Liming
> > > >-----Original Message-----
> > > >From: Loh, Tien Hock
> > > >Sent: Friday, September 27, 2019 3:05 PM
> > > >To: Gao, Liming <liming.gao@intel.com>; devel@edk2.groups.io;
> > > >lersek@redhat.com
> > > >Cc: leif.lindholm@linaro.org; ard.biesheuvel@linaro.org
> > > >Subject: RE: [edk2-devel] Problem with decompression on EDK2
> > > >
> > > >Hi Liming,
> > > >
> > > >I've tested another workaround - Move the FD file to a higher RAM
> > > >location (originally, it is loaded into 0x50000, now it is at
> > > >0x10000000) With the FD file moved to 0x10000000, the decompression
> > > >works every time, even with bigger FD. I've attached the my FD file
> > > >in the
> > > mail.
> > > >
> > > >Thanks
> > > >Tien Hock
> > > >
> > > >> -----Original Message-----
> > > >> From: Gao, Liming <liming.gao@intel.com>
> > > >> Sent: Friday, September 27, 2019 8:13 AM
> > > >> To: devel@edk2.groups.io; lersek@redhat.com; Loh, Tien Hock
> > > >> <tien.hock.loh@intel.com>
> > > >> Cc: leif.lindholm@linaro.org; ard.biesheuvel@linaro.org
> > > >> Subject: RE: [edk2-devel] Problem with decompression on EDK2
> > > >>
> > > >> Can you share the generated FD image? I can help check whether it
> > > >> is generated correctly with compression.
> > > >>
> > > >> Thanks
> > > >> Liming
> > > >>
> > > >> >-----Original Message-----
> > > >> >From: devel@edk2.groups.io [mailto:devel@edk2.groups.io] On
> > > >> >Behalf Of Laszlo Ersek
> > > >> >Sent: Friday, September 27, 2019 3:22 AM
> > > >> >To: devel@edk2.groups.io; Loh, Tien Hock
> > > >> ><tien.hock.loh@intel.com>
> > > >> >Cc: leif.lindholm@linaro.org; ard.biesheuvel@linaro.org
> > > >> >Subject: Re: [edk2-devel] Problem with decompression on EDK2
> > > >> >
> > > >> >On 09/26/19 11:27, Loh, Tien Hock wrote:
> > > >> >> Hi
> > > >> >>
> > > >> >> I have an issue while porting a new platform to EDK2. After
> > > >> >> porting the
> > > >> >Stratix 10 platform (ARM) to EDK2 it works correctly. However,
> > > >> >when I tried to add more INF to the FDF file, it failed to
> > > >> >decompress the image during boot time, log as below
> > > >> >> INFO:    DDR: DRAM calibration success.
> > > >> >> INFO:    Scrubbing ECC
> > > >> >> INFO:    Init HPS NOC's DDR Scheduler.
> > > >> >> NOTICE:  BL2: v2.1(debug):v2.1-15-g5880144-dirty
> > > >> >> NOTICE:  BL2: Built : 14:02:01, Aug 22 2019
> > > >> >> INFO:    BL2: Doing platform setup
> > > >> >> INFO:    BL2: Loading image id 3
> > > >> >> INFO:    Loading image id=3 at address 0xffe1c000
> > > >> >> INFO:    Image id=3 loaded: 0xffe1c000 - 0xffe22019
> > > >> >> INFO:    BL2: Loading image id 5
> > > >> >> INFO:    Loading image id=5 at address 0x50000
> > > >> >> INFO:    Image id=5 loaded: 0x50000 - 0x150000
> > > >> >> NOTICE:  BL2: Booting BL31
> > > >> >> INFO:    Entry point address = 0xffe1c000
> > > >> >> INFO:    SPSR = 0x3cd
> > > >> >> NOTICE:  BL31: v2.1(release):v2.1-604-g3441952
> > > >> >> NOTICE:  BL31: Built : 15:32:55, Sep 25 2019 UEFI firmware
> > > >> >> (version
> > > >> >> 1.0 built at 15:15:26 on Sep 26 2019) Decompress Failed -
> > > >> >> Invalid Parameter
> > > >> >>
> > > >> >> ASSERT_EFI_ERROR (Status = Not Found) ASSERT
> > > >> >> [ArmPlatformPrePiUniCore]
> > > >>
> > >
> >/nfs/png/disks/swuser_work_thloh/push/build/edk2/ArmPlatformPkg/Pr
> > > >eP
> > > >> i
> > > >> >/PrePi.c(151): !EFI_ERROR (Status)
> > > >> >>
> > > >> >> This happens if I add more items to the image, and if I reduce
> > > >> >> the size of the
> > > >> >image, it works correctly again. This could be a clue to what
> > > >> >could've gone wrong. Does anyone have any ideas what might've
> > > >> >gone
> > > wrong?
> > > >> >> I traced the decompression to LzmaDecode, and got lost in the
> > > >> >> code
> > > >there.
> > > >> >
> > > >> >The compressed section in a firmware volume is likely truncated
> > > >> >at build time, and does not decompress at boot time. I would
> > > >> >expect the
> > > >> "build"
> > > >> >utility to report this issue, and stop with an error.
> > > >> >
> > > >> >When you run "build" in the successful and in the failing cases,
> > > >> >what are the stats, respectively, that are printed at the end of
> > > >> >the build
> > > logs?
> > > >> >
> > > >> >Also, you could check your FDF file(s) near
> > > >> >"EE4E5898-3914-4259-9D6E-DC7BD79403CF"; perhaps you find
> > > interesting
> > > >> bits.
> > > >> >
> > > >> >Furthermore, if the boundary seems to be 16MB, then it could be
> > > >> >an issue in the processing code. See "ExtendedSize" in
> > > >> >"MdePkg/Include/Pi/PiFirmwareFile.h". I vaguely recall issues
> > > >> >around some code trying to parse EFI_FFS_FILE_HEADER2 as
> > > >EFI_FFS_FILE_HEADER,
> > > >> >or EFI_COMMON_SECTION_HEADER2 as
> > > >EFI_COMMON_SECTION_HEADER.
> > > >> >
> > > >> >Thanks
> > > >> >Laszlo
> > > >> >
> > > >> >