OvmfPkg XenPkg: X64 DEBUG GCC5 -DDEBUG_ON_SERIAL_PORT=TRUE build is broken

OvmfPkg XenPkg: X64 DEBUG GCC5 -DDEBUG_ON_SERIAL_PORT=TRUE build is broken

[Rebecca Cran]

I was trying to build OvmfPkg/XenPkg -a X64 -t GCC5 -b DEBUG 
-DDEBUG_ON_SERIAL_PORT=TRUE, but the build fails. Both plain DEBUG and 
RELEASE builds without trying to put the debug output on the serial port 
work.


[bcran@smic ~/src/tmp/edk2]$ build -p OvmfPkg/OvmfXen.dsc -a X64 -t GCC5 
-b DEBUG -DDEBUG_ON_SERIAL_PORT=TRUE
Build environment: FreeBSD-13.0-CURRENT-amd64-64bit-ELF
Build start time: 11:59:24, Apr.14 2020

WORKSPACE        = /home/bcran/src/tmp/edk2
EDK_TOOLS_PATH   = /home/bcran/src/tmp/edk2/BaseTools
CONF_PATH        = /home/bcran/src/tmp/edk2/Conf
PYTHON_COMMAND   = /usr/local/bin/python3


Processing meta-data .
Architecture(s)  = X64
Build target     = DEBUG
Toolchain        = GCC5

Active Platform          = /home/bcran/src/tmp/edk2/OvmfPkg/OvmfXen.dsc
.

build.py...
/home/bcran/src/tmp/edk2/OvmfPkg/OvmfXen.dsc(...): error F002: Library 
[/home/bcran/src/tmp/edk2/MdePkg/Library/UefiBootServicesTableLib/UefiBootServicesTableLib.inf] 
with constructors has a cycle
                 consumed by 
/home/bcran/src/tmp/edk2/MdePkg/Library/UefiLib/UefiLib.inf
         consumed by 
/home/bcran/src/tmp/edk2/MdePkg/Library/UefiDevicePathLibDevicePathProtocol/UefiDevicePathLibDevicePathProtocol.inf


- Failed -
Build end time: 11:59:25, Apr.14 2020
Build total time: 00:00:02


-- 
Rebecca Cran

Re: OvmfPkg XenPkg: X64 DEBUG GCC5 -DDEBUG_ON_SERIAL_PORT=TRUE build is broken

[Laszlo Ersek]

On 04/14/20 20:01, Rebecca Cran wrote:
> I was trying to build OvmfPkg/XenPkg -a X64 -t GCC5 -b DEBUG
> -DDEBUG_ON_SERIAL_PORT=TRUE, but the build fails. Both plain DEBUG and
> RELEASE builds without trying to put the debug output on the serial
> port work.
>
>
> [bcran@smic ~/src/tmp/edk2]$ build -p OvmfPkg/OvmfXen.dsc -a X64 -t
> GCC5 -b DEBUG -DDEBUG_ON_SERIAL_PORT=TRUE
> Build environment: FreeBSD-13.0-CURRENT-amd64-64bit-ELF
> Build start time: 11:59:24, Apr.14 2020
>
> WORKSPACE        = /home/bcran/src/tmp/edk2
> EDK_TOOLS_PATH   = /home/bcran/src/tmp/edk2/BaseTools
> CONF_PATH        = /home/bcran/src/tmp/edk2/Conf
> PYTHON_COMMAND   = /usr/local/bin/python3
>
>
> Processing meta-data .
> Architecture(s)  = X64
> Build target     = DEBUG
> Toolchain        = GCC5
>
> Active Platform          = /home/bcran/src/tmp/edk2/OvmfPkg/OvmfXen.dsc
> .
>
> build.py...
> /home/bcran/src/tmp/edk2/OvmfPkg/OvmfXen.dsc(...): error F002: Library
> [/home/bcran/src/tmp/edk2/MdePkg/Library/UefiBootServicesTableLib/UefiBootServicesTableLib.inf]
> with constructors has a cycle
>                 consumed by
> /home/bcran/src/tmp/edk2/MdePkg/Library/UefiLib/UefiLib.inf
>         consumed by
> /home/bcran/src/tmp/edk2/MdePkg/Library/UefiDevicePathLibDevicePathProtocol/UefiDevicePathLibDevicePathProtocol.inf

 From commit 05480e2fd4ff ("OvmfPkg/PlatformBootManagerLib: Use a Xen
console for ConOut/ConIn", 2019-08-21), we have:

> diff --git a/OvmfPkg/OvmfXen.dsc b/OvmfPkg/OvmfXen.dsc
> index 54ac910d8eed..e719a168f81e 100644
> --- a/OvmfPkg/OvmfXen.dsc
> +++ b/OvmfPkg/OvmfXen.dsc
> @@ -586,6 +586,10 @@ [Components]
>    OvmfPkg/XenIoPciDxe/XenIoPciDxe.inf
>    OvmfPkg/XenBusDxe/XenBusDxe.inf
>    OvmfPkg/XenPvBlkDxe/XenPvBlkDxe.inf
> +  MdeModulePkg/Universal/SerialDxe/SerialDxe.inf {
> +    <LibraryClasses>
> +      SerialPortLib|OvmfPkg/Library/XenConsoleSerialPortLib/XenConsoleSerialPortLib.inf
> +  }
>    MdeModulePkg/Universal/WatchdogTimerDxe/WatchdogTimer.inf
>    MdeModulePkg/Universal/MonotonicCounterRuntimeDxe/MonotonicCounterRuntimeDxe.inf
>    MdeModulePkg/Universal/CapsuleRuntimeDxe/CapsuleRuntimeDxe.inf

If you run the "build" command with the "-v" option added, the last part
(just preceding the error message above) is:

> Library instances of module [MdeModulePkg/Universal/SerialDxe/SerialDxe.inf] [X64]:
>         UefiDriverEntryPoint : MdePkg/Library/UefiDriverEntryPoint/UefiDriverEntryPoint.inf
>         UefiBootServicesTableLib : MdePkg/Library/UefiBootServicesTableLib/UefiBootServicesTableLib.inf
>         DebugLib : MdePkg/Library/BaseDebugLibSerialPort/BaseDebugLibSerialPort.inf
>         PcdLib : MdePkg/Library/DxePcdLib/DxePcdLib.inf
>         SerialPortLib : OvmfPkg/Library/XenConsoleSerialPortLib/XenConsoleSerialPortLib.inf
>         BaseLib : MdePkg/Library/BaseLib/BaseLib.inf
>         XenHypercallLib : OvmfPkg/Library/XenHypercallLib/XenHypercallLib.inf
>         HobLib : MdePkg/Library/DxeHobLib/DxeHobLib.inf
>         BaseMemoryLib : MdePkg/Library/BaseMemoryLibRepStr/BaseMemoryLibRepStr.inf
>         UefiLib : MdePkg/Library/UefiLib/UefiLib.inf
>         PrintLib : MdePkg/Library/BasePrintLib/BasePrintLib.inf
>         MemoryAllocationLib : MdePkg/Library/UefiMemoryAllocationLib/UefiMemoryAllocationLib.inf
>         DevicePathLib : MdePkg/Library/UefiDevicePathLibDevicePathProtocol/UefiDevicePathLibDevicePathProtocol.inf
>         UefiRuntimeServicesTableLib : MdePkg/Library/UefiRuntimeServicesTableLib/UefiRuntimeServicesTableLib.inf
>         DebugPrintErrorLevelLib : MdePkg/Library/BaseDebugPrintErrorLevelLib/BaseDebugPrintErrorLevelLib.inf

The dependency loop is formed by:

  DebugLib        : BaseDebugLibSerialPort ->
  SerialPortLib   : XenConsoleSerialPortLib ->
  XenHypercallLib : XenHypercallLib ->
  DebugLib

where all the library instances BaseDebugLibSerialPort,
XenConsoleSerialPortLib, XenHypercallLib have constructors.

Note that XenConsoleSerialPortLib *itself* is careful to avoid a
DebugLib dependency; from "XenConsoleSerialPortLib.c":

> //
> // We can't use DebugLib due to a constructor dependency cycle between DebugLib
> // and ourselves.
> //
> #define ASSERT(Expression)      \
>   do {                          \
>     if (!(Expression)) {        \
>       CpuDeadLoop ();           \
>     }                           \
>   } while (FALSE)

However, XenConsoleSerialPortLib still inherits a DebugLib dependency
through XenHypercallLib -- but only on IA32/X64, not on ARM/AARCH64! See
"XenHypercallLib.inf":

> [LibraryClasses.IA32, LibraryClasses.X64]
>   BaseLib
>   HobLib
>   DebugLib

There is no issue without specifying DEBUG_ON_SERIAL_PORT, because
OvmfXen resolves DebugLib to PlatformDebugLibIoPort then, which does not
depend on SerialPortLib (thus the cycle is severed).


Now, I think there may not be a simple solution for this, because, as
the message on commit 05480e2fd4ff says, "On a Xen PVH guest, none of
the existing serial or console interface works".

So even if we eliminated the constructor cycle for this module -- i.e.,
SerialDxe -- somehow [1], the larger DEBUG_ON_SERIAL_PORT feature might
not be feasible for *some* module types in OvmfXen. Because,
XenConsoleSerialPortLib ultimately depends on hypercalls, and I'm unsure
if those could be used as early as in SEC / PEI.

[1] For breaking up just this one constructor cycle, I guess we could
    remove the DebugLib dependency from XenHypercallLib even on
    IA32/X64.

In other words, I'm not sure if you could send any DEBUG messages *at
all* to the Xen serial port as early as in SEC or PEI. Meaning that even
if DEBUG_ON_SERIAL_PORT didn't break the build, it still wouldn't
(comprehensively) do what its name says.

I figure you attempted the DEBUG_ON_SERIAL_PORT build because without it
you get no debug output at all (due to the default
PlatformDebugLibIoPort resolution, which produces no logs on Xen).

For a start, I would recommend removing DEBUG_ON_SERIAL_PORT altogether
from OvmfXen (to decrease confusion and to eliminate the build
breakage), and then filing a feature request in the TianoCore bugzilla
for "some" kind of debug logging in OvmfXen, if the latter is possible.

Thanks
Laszlo

Re: OvmfPkg XenPkg: X64 DEBUG GCC5 -DDEBUG_ON_SERIAL_PORT=TRUE build is broken

[Rebecca Cran]

On 4/15/2020 8:04 AM, Laszlo Ersek wrote:
>
> I figure you attempted the DEBUG_ON_SERIAL_PORT build because without it
> you get no debug output at all (due to the default
> PlatformDebugLibIoPort resolution, which produces no logs on Xen).

I just attempted the DEBUG_ON_SERIAL_PORT build because that's how I do 
the other DEBUG OVMF builds and was running the following basic build 
script to make sure my bhyve changes weren't breaking things.


build -p OvmfPkg/OvmfPkgX64.dsc -a X64 -t GCC5 -b RELEASE
build -p OvmfPkg/OvmfPkgX64.dsc -a X64 -t GCC5 -b DEBUG 
-DDEBUG_ON_SERIAL_PORT=TRUE
build -p OvmfPkg/OvmfPkgX64.dsc -a X64 -t GCC5 -b NOOPT
build -p OvmfPkg/OvmfPkgIa32.dsc -a IA32 -t GCC5 -b RELEASE
build -p OvmfPkg/OvmfPkgIa32.dsc -a IA32 -t GCC5 -b DEBUG 
-DDEBUG_ON_SERIAL_PORT=TRUE
build -p OvmfPkg/OvmfPkgIa32.dsc -a IA32 -t GCC5 -b NOOPT
build -p OvmfPkg/OvmfPkgIa32X64.dsc -a IA32 -a X64 -t GCC5 -b RELEASE
build -p OvmfPkg/OvmfPkgIa32X64.dsc -a IA32 -a X64 -t GCC5 -b DEBUG 
-DDEBUG_ON_SERIAL_PORT=TRUE
build -p OvmfPkg/OvmfPkgIa32X64.dsc -a IA32 -a X64 -t GCC5 -b NOOPT
build -p OvmfPkg/OvmfXen.dsc -a X64 -t GCC5 -b RELEASE
build -p OvmfPkg/OvmfXen.dsc -a X64 -t GCC5 -b DEBUG 
-DDEBUG_ON_SERIAL_PORT=TRUE
build -p OvmfPkg/OvmfXen.dsc -a X64 -t GCC5 -b NOOPT
build -p BhyvePkg/BhyvePkgX64.dsc -a X64 -t GCC5 -b RELEASE
build -p BhyvePkg/BhyvePkgX64.dsc -a X64 -t GCC5 -b DEBUG 
-DDEBUG_ON_SERIAL_PORT=TRUE
build -p BhyvePkg/BhyvePkgX64.dsc -a X64 -t GCC5 -b NOOPT

-- 
Rebecca Cran

Re: [edk2-devel] OvmfPkg XenPkg: X64 DEBUG GCC5 -DDEBUG_ON_SERIAL_PORT=TRUE build is broken

[Laszlo Ersek]

On 04/15/20 16:20, Rebecca Cran wrote:
> On 4/15/2020 8:04 AM, Laszlo Ersek wrote:
>>
>> I figure you attempted the DEBUG_ON_SERIAL_PORT build because without it
>> you get no debug output at all (due to the default
>> PlatformDebugLibIoPort resolution, which produces no logs on Xen).
> 
> I just attempted the DEBUG_ON_SERIAL_PORT build because that's how I do
> the other DEBUG OVMF builds and was running the following basic build
> script to make sure my bhyve changes weren't breaking things.
> 
> 
> build -p OvmfPkg/OvmfPkgX64.dsc -a X64 -t GCC5 -b RELEASE
> build -p OvmfPkg/OvmfPkgX64.dsc -a X64 -t GCC5 -b DEBUG
> -DDEBUG_ON_SERIAL_PORT=TRUE
> build -p OvmfPkg/OvmfPkgX64.dsc -a X64 -t GCC5 -b NOOPT
> build -p OvmfPkg/OvmfPkgIa32.dsc -a IA32 -t GCC5 -b RELEASE
> build -p OvmfPkg/OvmfPkgIa32.dsc -a IA32 -t GCC5 -b DEBUG
> -DDEBUG_ON_SERIAL_PORT=TRUE
> build -p OvmfPkg/OvmfPkgIa32.dsc -a IA32 -t GCC5 -b NOOPT
> build -p OvmfPkg/OvmfPkgIa32X64.dsc -a IA32 -a X64 -t GCC5 -b RELEASE
> build -p OvmfPkg/OvmfPkgIa32X64.dsc -a IA32 -a X64 -t GCC5 -b DEBUG
> -DDEBUG_ON_SERIAL_PORT=TRUE
> build -p OvmfPkg/OvmfPkgIa32X64.dsc -a IA32 -a X64 -t GCC5 -b NOOPT
> build -p OvmfPkg/OvmfXen.dsc -a X64 -t GCC5 -b RELEASE
> build -p OvmfPkg/OvmfXen.dsc -a X64 -t GCC5 -b DEBUG
> -DDEBUG_ON_SERIAL_PORT=TRUE
> build -p OvmfPkg/OvmfXen.dsc -a X64 -t GCC5 -b NOOPT
> build -p BhyvePkg/BhyvePkgX64.dsc -a X64 -t GCC5 -b RELEASE
> build -p BhyvePkg/BhyvePkgX64.dsc -a X64 -t GCC5 -b DEBUG
> -DDEBUG_ON_SERIAL_PORT=TRUE
> build -p BhyvePkg/BhyvePkgX64.dsc -a X64 -t GCC5 -b NOOPT
> 

Thanks.

Then the suggestion to remove DEBUG_ON_SERIAL_PORT from OvmfXen (maybe
as a first step only?) would fit your use case too, I think.

Laszlo

Re: OvmfPkg XenPkg: X64 DEBUG GCC5 -DDEBUG_ON_SERIAL_PORT=TRUE build is broken

[Anthony PERARD]

On Wed, Apr 15, 2020 at 04:04:35PM +0200, Laszlo Ersek wrote:
> On 04/14/20 20:01, Rebecca Cran wrote:
> > I was trying to build OvmfPkg/XenPkg -a X64 -t GCC5 -b DEBUG
> > -DDEBUG_ON_SERIAL_PORT=TRUE, but the build fails. Both plain DEBUG and
> > RELEASE builds without trying to put the debug output on the serial
> > port work.
> >
> >
> > [bcran@smic ~/src/tmp/edk2]$ build -p OvmfPkg/OvmfXen.dsc -a X64 -t
> > GCC5 -b DEBUG -DDEBUG_ON_SERIAL_PORT=TRUE
> > Build environment: FreeBSD-13.0-CURRENT-amd64-64bit-ELF
> > Build start time: 11:59:24, Apr.14 2020
> >
> > WORKSPACE        = /home/bcran/src/tmp/edk2
> > EDK_TOOLS_PATH   = /home/bcran/src/tmp/edk2/BaseTools
> > CONF_PATH        = /home/bcran/src/tmp/edk2/Conf
> > PYTHON_COMMAND   = /usr/local/bin/python3
> >
> >
> > Processing meta-data .
> > Architecture(s)  = X64
> > Build target     = DEBUG
> > Toolchain        = GCC5
> >
> > Active Platform          = /home/bcran/src/tmp/edk2/OvmfPkg/OvmfXen.dsc
> > .
> >
> > build.py...
> > /home/bcran/src/tmp/edk2/OvmfPkg/OvmfXen.dsc(...): error F002: Library
> > [/home/bcran/src/tmp/edk2/MdePkg/Library/UefiBootServicesTableLib/UefiBootServicesTableLib.inf]
> > with constructors has a cycle
> >                 consumed by
> > /home/bcran/src/tmp/edk2/MdePkg/Library/UefiLib/UefiLib.inf
> >         consumed by
> > /home/bcran/src/tmp/edk2/MdePkg/Library/UefiDevicePathLibDevicePathProtocol/UefiDevicePathLibDevicePathProtocol.inf
> 
>  From commit 05480e2fd4ff ("OvmfPkg/PlatformBootManagerLib: Use a Xen
> console for ConOut/ConIn", 2019-08-21), we have:
> 
> > diff --git a/OvmfPkg/OvmfXen.dsc b/OvmfPkg/OvmfXen.dsc
> > index 54ac910d8eed..e719a168f81e 100644
> > --- a/OvmfPkg/OvmfXen.dsc
> > +++ b/OvmfPkg/OvmfXen.dsc
> > @@ -586,6 +586,10 @@ [Components]
> >    OvmfPkg/XenIoPciDxe/XenIoPciDxe.inf
> >    OvmfPkg/XenBusDxe/XenBusDxe.inf
> >    OvmfPkg/XenPvBlkDxe/XenPvBlkDxe.inf
> > +  MdeModulePkg/Universal/SerialDxe/SerialDxe.inf {
> > +    <LibraryClasses>
> > +      SerialPortLib|OvmfPkg/Library/XenConsoleSerialPortLib/XenConsoleSerialPortLib.inf
> > +  }
> >    MdeModulePkg/Universal/WatchdogTimerDxe/WatchdogTimer.inf
> >    MdeModulePkg/Universal/MonotonicCounterRuntimeDxe/MonotonicCounterRuntimeDxe.inf
> >    MdeModulePkg/Universal/CapsuleRuntimeDxe/CapsuleRuntimeDxe.inf
> 
> If you run the "build" command with the "-v" option added, the last part
> (just preceding the error message above) is:
> 
> > Library instances of module [MdeModulePkg/Universal/SerialDxe/SerialDxe.inf] [X64]:
> >         UefiDriverEntryPoint : MdePkg/Library/UefiDriverEntryPoint/UefiDriverEntryPoint.inf
> >         UefiBootServicesTableLib : MdePkg/Library/UefiBootServicesTableLib/UefiBootServicesTableLib.inf
> >         DebugLib : MdePkg/Library/BaseDebugLibSerialPort/BaseDebugLibSerialPort.inf
> >         PcdLib : MdePkg/Library/DxePcdLib/DxePcdLib.inf
> >         SerialPortLib : OvmfPkg/Library/XenConsoleSerialPortLib/XenConsoleSerialPortLib.inf
> >         BaseLib : MdePkg/Library/BaseLib/BaseLib.inf
> >         XenHypercallLib : OvmfPkg/Library/XenHypercallLib/XenHypercallLib.inf
> >         HobLib : MdePkg/Library/DxeHobLib/DxeHobLib.inf
> >         BaseMemoryLib : MdePkg/Library/BaseMemoryLibRepStr/BaseMemoryLibRepStr.inf
> >         UefiLib : MdePkg/Library/UefiLib/UefiLib.inf
> >         PrintLib : MdePkg/Library/BasePrintLib/BasePrintLib.inf
> >         MemoryAllocationLib : MdePkg/Library/UefiMemoryAllocationLib/UefiMemoryAllocationLib.inf
> >         DevicePathLib : MdePkg/Library/UefiDevicePathLibDevicePathProtocol/UefiDevicePathLibDevicePathProtocol.inf
> >         UefiRuntimeServicesTableLib : MdePkg/Library/UefiRuntimeServicesTableLib/UefiRuntimeServicesTableLib.inf
> >         DebugPrintErrorLevelLib : MdePkg/Library/BaseDebugPrintErrorLevelLib/BaseDebugPrintErrorLevelLib.inf
> 
> The dependency loop is formed by:
> 
>   DebugLib        : BaseDebugLibSerialPort ->
>   SerialPortLib   : XenConsoleSerialPortLib ->
>   XenHypercallLib : XenHypercallLib ->
>   DebugLib
> 
> where all the library instances BaseDebugLibSerialPort,
> XenConsoleSerialPortLib, XenHypercallLib have constructors.
> 
> Note that XenConsoleSerialPortLib *itself* is careful to avoid a
> DebugLib dependency; from "XenConsoleSerialPortLib.c":
> 
> > //
> > // We can't use DebugLib due to a constructor dependency cycle between DebugLib
> > // and ourselves.
> > //
> > #define ASSERT(Expression)      \
> >   do {                          \
> >     if (!(Expression)) {        \
> >       CpuDeadLoop ();           \
> >     }                           \
> >   } while (FALSE)
> 
> However, XenConsoleSerialPortLib still inherits a DebugLib dependency
> through XenHypercallLib -- but only on IA32/X64, not on ARM/AARCH64! See
> "XenHypercallLib.inf":
> 
> > [LibraryClasses.IA32, LibraryClasses.X64]
> >   BaseLib
> >   HobLib
> >   DebugLib
> 
> There is no issue without specifying DEBUG_ON_SERIAL_PORT, because
> OvmfXen resolves DebugLib to PlatformDebugLibIoPort then, which does not
> depend on SerialPortLib (thus the cycle is severed).
> 
> 
> Now, I think there may not be a simple solution for this, because, as
> the message on commit 05480e2fd4ff says, "On a Xen PVH guest, none of
> the existing serial or console interface works".
> 
> So even if we eliminated the constructor cycle for this module -- i.e.,
> SerialDxe -- somehow [1], the larger DEBUG_ON_SERIAL_PORT feature might
> not be feasible for *some* module types in OvmfXen. Because,
> XenConsoleSerialPortLib ultimately depends on hypercalls, and I'm unsure
> if those could be used as early as in SEC / PEI.
> 
> [1] For breaking up just this one constructor cycle, I guess we could
>     remove the DebugLib dependency from XenHypercallLib even on
>     IA32/X64.
> 
> In other words, I'm not sure if you could send any DEBUG messages *at
> all* to the Xen serial port as early as in SEC or PEI. Meaning that even
> if DEBUG_ON_SERIAL_PORT didn't break the build, it still wouldn't
> (comprehensively) do what its name says.
> 
> I figure you attempted the DEBUG_ON_SERIAL_PORT build because without it
> you get no debug output at all (due to the default
> PlatformDebugLibIoPort resolution, which produces no logs on Xen).
> 
> For a start, I would recommend removing DEBUG_ON_SERIAL_PORT altogether
> from OvmfXen (to decrease confusion and to eliminate the build
> breakage), and then filing a feature request in the TianoCore bugzilla
> for "some" kind of debug logging in OvmfXen, if the latter is possible.

Thanks Laszlo for the investigation. I usually use DebugLib to debug
OVMF under Xen, I just need to enable the debug port of QEMU. And for
when QEMU isn't available (when running Xen PVH guest), I have a local
patch that I haven't send upstream yet. It's a hacked version of
PlatformDebugLibIoPort which sends logs to Xen's debug port, and doesn't
check if the port is open.

I'll prepare a patch to remove DEBUG_ON_SERIAL_PORT support from OvmfXen
and send my other patch.

Thanks,

-- 
Anthony PERARD