[edk2-devel] CLANGDWARF OVMF hangs on start

[edk2-devel] CLANGDWARF OVMF hangs on start

[Mike Beaton]

From commit https://github.com/tianocore/edk2/commit/140e4422b16482f0bcafdc20d42141434d450450
(and see also the preceding commit, which I believe is related) up to
the current tip of master, CLANGDWARF OVMF is broken. It builds and
starts but hangs early (more info below). I do not currently have a
patch to offer though of course it would be useful if this could be
resolved.

Note: I've cc'd those folks who have more or less recently (this year)
authored or reviewed commits which have modified CLANGDWARF OVMF build
behaviour in various interesting ways, including the above, as
described below.

### More detail:

Using the following commands

```
cd OvmfPkg
./build.sh -a X64 -b NOOPT -t CLANGDWARF -D DEBUG_ON_SERIAL_PORT=1
./build.sh -a X64 -b NOOPT -t CLANGDWARF qemu -serial stdio
```

from the mentioned commit up to the current tip of master CLANGDWARF
OVMF builds, but when started it hangs, repeatedly outputting the line
`SecCoreStartupWithStack(0xFFFCC000, 0x820000)`.

Prior to that commit, CLANGDWARF OVMF builds and starts normally
(though see the following).

As additional possibly helpful information, it may be worth noting
that if we go back further we reach
https://github.com/tianocore/edk2/commit/c6f47e678f994ac86d36955d24baae465330d356
which is the earliest commit with correct behaviour. In the commit
before that CLANGDWARF OVMF builds but when run stops with an ASSERT,
and in the commit before that again, CLANGDWARF OVMF does not build,
giving:

```
In file included from
/home/mjsbeaton/OpenSource/edk2/CryptoPkg/Library/OpensslLib/openssl/crypto/asn1/a_sign.c:22:
...
In file included from /usr/include/stdint.h:34:
/usr/include/x86_64-linux-gnu/bits/stdint-intn.h:27:19: error: typedef
redefinition with different types ('__int64_t' (aka 'long') vs 'INT64'
(aka 'long long'))
typedef __int64_t int64_t;
                  ^
/home/mjsbeaton/OpenSource/edk2/CryptoPkg/Library/OpensslLib/openssl/include/openssl/e_os2.h:238:15:
note: previous definition is here
typedef INT64 int64_t;
              ^
...
/usr/include/x86_64-linux-gnu/bits/stdint-uintn.h:27:20: error:
typedef redefinition with different types ('__uint64_t' (aka 'unsigned
long') vs 'UINT64' (aka 'unsigned long long'))
typedef __uint64_t uint64_t;
                   ^
/home/mjsbeaton/OpenSource/edk2/CryptoPkg/Library/OpensslLib/openssl/include/openssl/e_os2.h:239:16:
note: previous definition is here
typedef UINT64 uint64_t;
               ^
...
2 errors generated.
```

That failure to build persists back in the commit history for some
time. I didn't find a commit further back than that where CLANGDWARF
OVMF builds and runs correctly with current tools, though the exact
build failure changes at different points in the commit history, and I
haven't bisected everything which happens. (Note: when doing the above
bisects, it seems helpful/necessary to rebuild base tools, plus of
course `git submodule update --init`, at each step to get reliable
results.)

Mike


-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.
View/Reply Online (#112168): https://edk2.groups.io/g/devel/message/112168
Mute This Topic: https://groups.io/mt/103030855/7686176
Group Owner: devel+owner@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub [rebecca@openfw.io]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [edk2-devel] CLANGDWARF OVMF hangs on start

[Gerd Hoffmann]

On Thu, Dec 07, 2023 at 08:40:55AM +0000, Mike Beaton wrote:
> From commit https://github.com/tianocore/edk2/commit/140e4422b16482f0bcafdc20d42141434d450450
> (and see also the preceding commit, which I believe is related) up to
> the current tip of master, CLANGDWARF OVMF is broken. It builds and
> starts but hangs early (more info below). I do not currently have a
> patch to offer though of course it would be useful if this could be
> resolved.

Which clang version is this?

It works for me, fedora 39 / clang 17, and I'm pretty sure I tested it
when creating the patch on a slightly older clang version (15 or 16).

take care,
  Gerd



-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.
View/Reply Online (#112174): https://edk2.groups.io/g/devel/message/112174
Mute This Topic: https://groups.io/mt/103030855/7686176
Group Owner: devel+owner@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub [rebecca@openfw.io]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [edk2-devel] CLANGDWARF OVMF hangs on start

[Mike Beaton]

> Which clang version is this?
>
> It works for me, fedora 39 / clang 17, and I'm pretty sure I tested it
> when creating the patch on a slightly older clang version (15 or 16).

It is 14 ('Ubuntu clang version 14.0.0-1ubuntu1.1' on Ubuntu 22.04.3 LTS)

Cheers,
Mike


-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.
View/Reply Online (#112183): https://edk2.groups.io/g/devel/message/112183
Mute This Topic: https://groups.io/mt/103030855/7686176
Group Owner: devel+owner@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub [rebecca@openfw.io]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [edk2-devel] CLANGDWARF OVMF hangs on start

[Mike Beaton]

> > Which clang version is this?
> >
> > It works for me, fedora 39 / clang 17, and I'm pretty sure I tested it
> > when creating the patch on a slightly older clang version (15 or 16).
>
> It is 14 ('Ubuntu clang version 14.0.0-1ubuntu1.1' on Ubuntu 22.04.3 LTS)
>
> Cheers,
> Mike

Strangely, given what you say:

 - I have just replicated exactly the same issue from a clean, new
checkout and build on Fedora 38 and clang 16 (Fedora 16.0.6-3.fc38).
 - I then upgraded that same VM to Fedora 39 and clang 17 and ... I
still get the same issue, including that it works again when switching
to 140e4422b16482f0bcafdc20d42141434d450450~1 (then cleaning Conf/ and
Build/, then `make clean`, `make` in BaseTools/, then retry, as for
each re-test).

Mike


-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.
View/Reply Online (#112186): https://edk2.groups.io/g/devel/message/112186
Mute This Topic: https://groups.io/mt/103030855/7686176
Group Owner: devel+owner@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub [rebecca@openfw.io]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [edk2-devel] CLANGDWARF OVMF hangs on start

[Mike Beaton]

>  - I have just replicated exactly the same issue from a clean, new
> checkout and build on Fedora 38 and clang 16 (Fedora 16.0.6-3.fc38).
>  - I then upgraded that same VM to Fedora 39 and clang 17 and ... I
> still get the same issue, including that it works again when switching
> to 140e4422b16482f0bcafdc20d42141434d450450~1 (then cleaning Conf/ and
> Build/, then `make clean`, `make` in BaseTools/, then retry, as for
> each re-test).

I assume we must all be talking about clean builds at each point, more
or less by definition. But for the avoidance of doubt the script I'm
using to fully rebuild at each commit (which is reproducing these
problems, on all versions listed above, and which matches what happens
on a clean checkout - for me) is:

```
#!/bin/bash
git submodule update --init
rm -rf Build
rm Conf/*
rm -rf Conf/.cache
rm Conf/.AutoGenIdFile.txt
cd BaseTools
make clean
make
cd ../OvmfPkg
./build.sh -a X64 -b NOOPT -t CLANGDWARF -D DEBUG_ON_SERIAL_PORT=1 || exit 1
./build.sh -a X64 -b NOOPT -t CLANGDWARF qemu -serial stdio
```


-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.
View/Reply Online (#112187): https://edk2.groups.io/g/devel/message/112187
Mute This Topic: https://groups.io/mt/103030855/7686176
Group Owner: devel+owner@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub [rebecca@openfw.io]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [edk2-devel] CLANGDWARF OVMF hangs on start

[Mike Beaton]

I have raised a bug,
https://bugzilla.tianocore.org/show_bug.cgi?id=4617 - adding a little
bit of new information which I believe is correct, that I've been able
to dig out.


-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.
View/Reply Online (#112251): https://edk2.groups.io/g/devel/message/112251
Mute This Topic: https://groups.io/mt/103030855/7686176
Group Owner: devel+owner@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub [rebecca@openfw.io]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [edk2-devel] CLANGDWARF OVMF hangs on start

[Mike Beaton]

I spotted that Ard Biesheuvel authored the original commit
https://github.com/tianocore/edk2/commit/28ade7b802e which sets the
pragma in question. I hope it is okay to issue a cc in case they could
shed any additional light on what might be going wrong with the
clangdrawf build, without the (now removed) pragma?


On Sat, 9 Dec 2023 at 15:24, Mike Beaton <mjsbeaton@gmail.com> wrote:
>
> I have raised a bug,
> https://bugzilla.tianocore.org/show_bug.cgi?id=4617 - adding a little
> bit of new information which I believe is correct, that I've been able
> to dig out.


-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.
View/Reply Online (#112252): https://edk2.groups.io/g/devel/message/112252
Mute This Topic: https://groups.io/mt/103030855/7686176
Group Owner: devel+owner@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub [rebecca@openfw.io]
-=-=-=-=-=-=-=-=-=-=-=-