From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <sigmaepsilon92@gmail.com>
Received-SPF: Pass (sender SPF authorized) identity=mailfrom;
 client-ip=2607:f8b0:4003:c06::234; helo=mail-oi0-x234.google.com;
 envelope-from=sigmaepsilon92@gmail.com; receiver=edk2-devel@lists.01.org 
Received: from mail-oi0-x234.google.com (mail-oi0-x234.google.com
 [IPv6:2607:f8b0:4003:c06::234])
 (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits))
 (No client certificate requested)
 by ml01.01.org (Postfix) with ESMTPS id A7D20220FB322
 for <edk2-devel@lists.01.org>; Sun, 10 Dec 2017 23:25:49 -0800 (PST)
Received: by mail-oi0-x234.google.com with SMTP id r63so10962584oia.6
 for <edk2-devel@lists.01.org>; Sun, 10 Dec 2017 23:30:27 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025;
 h=mime-version:in-reply-to:references:from:date:message-id:subject:to
 :cc; bh=yzZKrP5hzw0I4LDDIgg3n+lw6XmDzlxIdMrcjBqwXTA=;
 b=PC2YSic6t1rzPeVmW6vY0S0h8N5mWi7EGBuN+uXR15m7BdOKa95zHXn8fYqPUXtDrs
 qDSEyC8asrn1+Q2h0+6DVKQP9hbq2za3HDmEKe8d4HL9aQhrX40SxqKbgvWUWdae03nM
 Nmrie4753jS1TVcRi0BvG5CruGSl/pwi+Tw+QNRqSPdCQsxi/iIJxkkGPV3VKK9kJNig
 5r/6EMFhv39qbJL3+t9EgmxOaz5jJ5n1XYKyFgAsgKszQtOMGHUDVb8cb1h6SjkXrqG+
 tUmwTTj/tbla5Cz9qosuQwXKQ2E2cKAJBCDSprPkv3ceACrr52bWsczY+NmGlWnRG3+f
 L1Bw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=1e100.net; s=20161025;
 h=x-gm-message-state:mime-version:in-reply-to:references:from:date
 :message-id:subject:to:cc;
 bh=yzZKrP5hzw0I4LDDIgg3n+lw6XmDzlxIdMrcjBqwXTA=;
 b=NajScsubCjSsvuGP6ASLBI/6GQmgw2DH56DdEpa8CSlmrF1D1ss/eO8zAlpflspT7P
 fyoIuOlyn1N39zctnFye6/rvS/kYHvncU8rJqBTp3/9PBgc4Wk4qKBGIvTRHbaYLomAJ
 Hf18FYcmBLC0V8WE//VjsX0F8C2VzcfKrqFQNY4qeSvIcpoVtxQ8VyygE85E5GiGvy28
 Ir1EKml1nfGwMcw/8cmPRx291BTla2eRsmCrWXRUG04MFLwCZSuHxWsS0HX2dUHUuMqE
 8xmwO2UwiK9WW8RSM8BgPtEBw60niji4e27EeFp6h8cm9/x+Bqo1iSvPfNElyWmh7P88
 GnGg==
X-Gm-Message-State: AJaThX6D7WQbRM/U0aLeUHaV3B4mYsCUtS/aTr1TrgrpCMPXUEE9Yi6O
 ltTEpWyUkOCltiDGxVIXBKdwtAGKcLV4zTfLF/s=
X-Google-Smtp-Source: AGs4zMZ4+ftOwkytjq6zjnN00PlnNO8y17glmjiE4CQ4GWXi5kiQTtAYsiqol8tcNBy4igAE8ZcDnrnAJciPFzqrOrs=
X-Received: by 10.202.237.11 with SMTP id l11mr30367942oih.249.1512977425844; 
 Sun, 10 Dec 2017 23:30:25 -0800 (PST)
MIME-Version: 1.0
Received: by 10.157.1.204 with HTTP; Sun, 10 Dec 2017 23:30:25 -0800 (PST)
In-Reply-To: <CAKv+Gu8_Gu276s2_Ufh_TjR19jtEQhuuj=wHJDr0Fmy4WXi+Zw@mail.gmail.com>
References: <CAN9vWD+O5=06gCVhxuZ=S2kxVBjkw3nrJMdo-zUKfNx0TzWPeg@mail.gmail.com>
 <CAKv+Gu9TcfRbwZK61JHwDWc5z-fo4dQhD_Aq6+pFFkdHRWDOJA@mail.gmail.com>
 <CAKv+Gu-U6kxiw46kr4O=iTusNg_BK6Vg6_snzqjCrNWU-OTVhA@mail.gmail.com>
 <CAN9vWDKZeNTdYqYFVL3YwvQfpRs1E7NDcN3QH4HDpuLCbNcLdA@mail.gmail.com>
 <CAKv+Gu9LD9N2GtCah4ogy2qbrQ-a=wz0+LE8endRy5AtWcL28g@mail.gmail.com>
 <CAN9vWDJepuOSsdsfGsuqsvNepQDiwn6=FPxFtSL=n1g3d6OkMg@mail.gmail.com>
 <CAKv+Gu-uYkWW_uPuQ5YRej3_VhdAQt-QOR91igLw_S=DxDc1bw@mail.gmail.com>
 <CAN9vWDK5MmyvinRXx5XQoNTnGi90rgDCFDqdGBk1t-QZ=NqNQg@mail.gmail.com>
 <CAN9vWDLKTRjZ88uyiD0x8__Fm9y1Zx1W1z+Si9T9-REQFSeZXQ@mail.gmail.com>
 <CAN9vWDKKKaCGL9nRnqVWR8T60ABpCP7tH723EjXaHfc2bWsa1A@mail.gmail.com>
 <CAN9vWDJWshkMbfkBSc2pgyqWqWfc8QsfQT7pUM=Zx6VLx_5qZg@mail.gmail.com>
 <CAN9vWD+yfqZxRcsMqSiKMrHXf8BAvYLYcUhbU9S+akBnF_rCOg@mail.gmail.com>
 <CAKv+Gu8_Gu276s2_Ufh_TjR19jtEQhuuj=wHJDr0Fmy4WXi+Zw@mail.gmail.com>
From: Michael Zimmermann <sigmaepsilon92@gmail.com>
Date: Mon, 11 Dec 2017 08:30:25 +0100
Message-ID: <CAN9vWDKc4r=D7n0zmYX3cjvNUTZ-ZwpPNydbdG+AZPR_LHLvHw@mail.gmail.com>
To: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Cc: "Gao, Liming" <liming.gao@intel.com>, Yonghong Zhu <yonghong.zhu@intel.com>,
 edk2-devel-01 <edk2-devel@lists.01.org>,
 Leif Lindholm <leif.lindholm@linaro.org>
Subject: Re: PrePiLib's FwVol.c can't handle padding before volume header
X-BeenThere: edk2-devel@lists.01.org
X-Mailman-Version: 2.1.22
Precedence: list
List-Id: EDK II Development  <edk2-devel.lists.01.org>
List-Unsubscribe: <https://lists.01.org/mailman/options/edk2-devel>,
 <mailto:edk2-devel-request@lists.01.org?subject=unsubscribe>
List-Archive: <http://lists.01.org/pipermail/edk2-devel/>
List-Post: <mailto:edk2-devel@lists.01.org>
List-Help: <mailto:edk2-devel-request@lists.01.org?subject=help>
List-Subscribe: <https://lists.01.org/mailman/listinfo/edk2-devel>,
 <mailto:edk2-devel-request@lists.01.org?subject=subscribe>
X-List-Received-Date: Mon, 11 Dec 2017 07:25:49 -0000
Content-Type: text/plain; charset="UTF-8"

without the big file at the end it looks very similar:
00000000  c4 cc 4a 17 00 00 00 00  00 00 00 00 00 00 00 00  |..J.............|
00000010  00 00 00 00 78 e5 8c 8c  3d 8a 1c 4f 99 35 89 61  |....x...=..O.5.a|
00000020  85 c3 2d d3 c0 cc 4a 00  00 00 00 00 5f 46 56 48  |..-...J....._FVH|
00000030  ff fe 04 00 48 00 f1 fd  60 00 00 02 33 2b 01 00  |....H...`...3+..|
00000040  40 00 00 00 00 00 00 00  00 00 00 00 ff ff ff ff  |@...............|
00000050  ff ff ff ff ff ff ff ff  ff ff ff ff f4 aa f0 00  |................|
00000060  2c 00 00 f8 bd 1d f5 8f  56 b8 cb 4a b8 59 85 36  |,.......V..J.Y.6|
00000070  d8 92 ed 3a 14 00 00 00  ff ff ff ff e7 0e 51 fc  |...:..........Q.|
00000080  dc ff d4 11 bd 41 00 80  c7 3c 88 81 e6 aa 02 00  |.....A...<......|
00000090  8c 00 00 f8 74 00 00 19  57 72 cf 80 ab 87 f9 47  |....t...Wr.....G|

with the big file (when it's broken) it looks like this:
00000000  ff ff ff 17 08 cd 03 01  00 00 00 00 00 00 00 00  |................|
00000010  00 00 00 00 00 00 00 00  78 e5 8c 8c 3d 8a 1c 4f  |........x...=..O|
00000020  99 35 89 61 85 c3 2d d3  00 cd 03 01 00 00 00 00  |.5.a..-.........|
00000030  5f 46 56 48 ff fe 04 00  48 00 f4 18 60 00 00 02  |_FVH....H...`...|
00000040  34 0f 04 00 40 00 00 00  00 00 00 00 00 00 00 00  |4...@...........|
00000050  ff ff ff ff ff ff ff ff  ff ff ff ff ff ff ff ff  |................|
00000060  f4 aa f0 00 2c 00 00 f8  bd 1d f5 8f 56 b8 cb 4a  |....,.......V..J|
00000070  b8 59 85 36 d8 92 ed 3a  14 00 00 00 ff ff ff ff  |.Y.6...:........|
00000080  e7 0e 51 fc dc ff d4 11  bd 41 00 80 c7 3c 88 81  |..Q......A...<..|
00000090  e6 aa 02 00 8c 00 00 f8  74 00 00 19 57 72 cf 80  |........t...Wr..|

While looking for the header format and checking what these ff's mean
I found that this is a EFI_COMMON_SECTION_HEADER2 which is used
because the max size of EFI_COMMON_SECTION_HEADER is 0xFFFFFF.
MdeModulePkg's FwVol.c also seems to have some code for handling v2 sections.
I'll send a patch to fix this in a bit.



On Mon, Dec 11, 2017 at 12:08 AM, Ard Biesheuvel
<ard.biesheuvel@linaro.org> wrote:
> On 10 December 2017 at 22:18, Michael Zimmermann
> <sigmaepsilon92@gmail.com> wrote:
>> Exactly. If I shift the pointer by 4 bytes from within PrePiLib the device
>> boots just fine.
>>
>> I'm not sure if the size is the root cause but right now it only happens
>> when adding a ~15mb binary efi to the end of fvmain.
>>
>
> It appears so.
>
> So it would be good to check where the disparity originates.
>
> The Ffs section containing the compressed FV looks like this for ArmVirtQemu
>
> $ hexdump -C 9E21FD93-9C72-4c15-8C4B-E77F1DB2D792SEC1.1fv.sec |head
>
> 00000000  84 a8 48 17 00 00 00 00  00 00 00 00 00 00 00 00  |..H.............|
> 00000010  00 00 00 00 78 e5 8c 8c  3d 8a 1c 4f 99 35 89 61  |....x...=..O.5.a|
> 00000020  85 c3 2d d3 80 a8 48 00  00 00 00 00 5f 46 56 48  |..-...H....._FVH|
> 00000030  ff fe 04 00 48 00 24 2b  00 00 00 02 a2 22 01 00  |....H.$+....."..|
> 00000040  40 00 00 00 00 00 00 00  00 00 00 00 7f cb a2 d6  |@...............|
> 00000050  18 6a 2f 4e b4 3b 99 20  a7 33 70 0a 4d aa 05 00  |.j/N.;. .3p.M...|
> 00000060  30 c0 01 f8 04 c0 01 10  4d 5a 00 00 00 00 00 00  |0.......MZ......|
> 00000070  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
> *
> 000000a0  00 00 00 00 58 0e 00 00  00 00 00 00 00 00 00 00  |....X...........|
>
> Could you compare with your build?