[edk2-devel] EDK2 ArmVirtQemu behaviour with multiple UARTs

[edk2-devel] EDK2 ArmVirtQemu behaviour with multiple UARTs

[Peter Maydell]

Hi; I've been looking again at a very long standing missing feature in
the QEMU virt board, which is that we only have one UART. One of the
things that has stalled this in the past has been the odd behaviour of
EDK2 if the DTB that QEMU passes it describes two UARTs.

I'm going to describe the behaviour I see in more detail below, but to
put the summary up front:
 * EDK2 puts some debug output on one UART and some on the other
   (the exact arrangement depends on ordering of the dtb nodes)
 * EDK2 doesn't look at either stdout-path or the serial* aliases,
   so its choices about how to use the UARTs differ from those
   made by the guest kernel it is booting (and it also seems to be
   iterating through the dtb in the opposite order to the kernel)

The current proposal for adding a second UART is that it only happens
if you explicitly add one on the command line (with a second "-serial
something" option), so whatever we do won't break existing user
setups. So we have scope for saying "if you want to use a second UART,
you're going to want a newer EDK2 which handles it better". Exactly
what "better" means here is up for grabs, but honouring stdout-path
and the serial aliases would be the ideal I think. It would also be
possible to select a particular ordering for the DTB nodes to produce
"least-worst" behaviour from an existing EDK2 binary, but I'm not
sure if that's worth doing.

What do the EDK2 folks think about what the correct behaviour
should be for a 2-UART setup?

Anyway, on to the details about the setup and what I see from EDK2:

This is all with a debug ArmVirtQemu build, running at EL2 (i.e.
entirely non-secure), with some patches I've been working on to add
the extra UART to the board and the DTB. The DTB has the two UARTs:

        pl011@9000000 {
                clock-names = "uartclk\0apb_pclk";
                clocks = <0x8000 0x8000>;
                interrupts = <0x00 0x01 0x04>;
                reg = <0x00 0x9000000 0x00 0x1000>;
                compatible = "arm,pl011\0arm,primecell";
        };

        pl011@9040000 {
                clock-names = "uartclk\0apb_pclk";
                clocks = <0x8000 0x8000>;
                interrupts = <0x00 0x08 0x04>;
                reg = <0x00 0x9040000 0x00 0x1000>;
                compatible = "arm,pl011\0arm,primecell";
        };

and aliases:

        aliases {
                serial0 = "/pl011@9000000";
                serial1 = "/pl011@9040000";
        };

and in the /chosen node:
                stdout-path = "/pl011@9000000";

The ACPI table fragments generated by QEMU have entries for both
UARTs, as COM0 and COM1.

Given all this, EDK2 outputs:

uart0:
 * some UEFI output including debug output, starting:
UEFI firmware (version  built at 15:19:20 on Sep 19 2023)
add-symbol-file
/home/petmay01/linaro/edk2/Build/ArmVirtQemu-AARCH64/DEBUG_GCC5/AARCH64/ArmPlatformPkg/PrePeiCore/PrePeiCoreUniCore/DEBUG/ArmPlatformPrePeiCore.dll
0x2000
add-symbol-file
/home/petmay01/linaro/edk2/Build/ArmVirtQemu-AARCH64/DEBUG_GCC5/AARCH64/MdeModulePkg/Core/Pei/PeiMain/DEBUG/PeiCore.dll
0xD240
Register PPI Notify: DCD0BE23-9586-40F4-B643-06522CED4EDE
Install PPI: 8C8CE578-8A3D-4F1C-9935-896185C32DD3
Install PPI: 5473C07A-3DCB-4DCA-BD6F-1E9689E7349A
The 0th FV start address is 0x00000001000, size is 0x001FF000, handle is 0x1000

 * the guest Linux kernel output (on what Linux says is ttyAMA0)

uart1:
 * a lot of UEFI output including debug output, starting:
DxeMain: MemoryBaseAddress=0x48000000 MemoryLength=0x38000000
add-symbol-file
/home/petmay01/linaro/edk2/Build/ArmVirtQemu-AARCH64/DEBUG_GCC5/AARCH64/MdeModulePkg/Core/Dxe/DxeMain/DEBUG/DxeCore.dll
0x47860000
HOBLIST address in DXE = 0x7FA35018
Memory Allocation 0x00000004 0x47FFF000 - 0x47FFFFFF
Memory Allocation 0x00000004 0x47FFE000 - 0x47FFEFFF

 * the GNU GRUB OS select screen and other GRUB output

The full output dumps can be seen at:
https://people.linaro.org/~peter.maydell/uart0.txt
https://people.linaro.org/~peter.maydell/uart1.txt

With only 1 UART, all the above appears on the single UART:
https://people.linaro.org/~peter.maydell/uart-single.txt

If I change QEMU to reverse the order of the nodes in the DTB (so the
pl011@9040000 nodes is listed first in the dtc output, and
pl011@900000 is listed second), then EDK2's output changes: the debug
output previously on uart0 is now on uart1, and vice-versa. The GRUB
output also switches to uart0. The Linux kernel output remains on
uart0 (this makes sense, because Linux is looking at the ACPI tables,
which are generated independently from the dtb). Output for this
setup is here:

https://people.linaro.org/~peter.maydell/uart0-rev.txt
https://people.linaro.org/~peter.maydell/uart1-rev.txt

A direct boot of Linux doesn't care about the dtb node ordering -- it
honours the aliases node and the chosen stdout-path string.
(Without the 'aliases' node, only the "pl011@9000000 first" dtb
order works, because it assigns ttyAMA0 and ttyAMA1 in the same
order as dtc prints them in the dtb disassembly.)

I would be happier if I understood why putting the nodes in reverse
order works, given that the code in EDK2 seems to be iterating through
the dtb forwards. I know there is at least one place in QEMU where the
node ordering gets reversed in the process of writing out the dtb, so
maybe there are more depending on how exactly the dtb is read. That
would I suppose explain why some EDK2 debug output goes to one UART
and some to the other, if the dtb read process differs during different
phases of EDK2 boot.

If you want to play around with this, I have some WIP patches at
https://git.linaro.org/people/pmaydell/qemu-arm.git uart-edk-investigation
(content wise they should be fine, but I haven't cleaned them up into
a coherent set of distinct patches yet, so they're a bit messy.)
A run of QEMU with both UARTs which sends all output to files looks like:

./build/arm-clang/qemu-system-aarch64 -display none -vga none \
  -machine virt,acpi=on,virtualization=on,mte=on,gic-version=max,iommu=smmuv3 \
  -smp 2 -m 1024 -cpu max,pauth-impdef=on \
  -bios ~/linaro/edk2/QEMU_EFI_DEBUG.fd \
  -drive file=/home/petmay01/avocado/data/cache/by_location/0154b7cd3a4f5e135299060c8cabbeec10b70b6d/alpine-standard-3.17.2-aarch64.iso,format=raw
\
  -device virtio-rng-pci,rng=rng0 \
  -object rng-random,id=rng0,filename=/dev/urandom \
  -chardev file,id=chr0,path=/tmp/uart0-rev.txt \
  -chardev file,id=chr1,path=/tmp/uart1-rev.txt \
  -serial chardev:chr0 -serial chardev:chr1

(adjust -serial options to taste)

thanks
-- PMM


-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.
View/Reply Online (#108959): https://edk2.groups.io/g/devel/message/108959
Mute This Topic: https://groups.io/mt/101498371/7686176
Group Owner: devel+owner@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub [rebecca@openfw.io]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [edk2-devel] EDK2 ArmVirtQemu behaviour with multiple UARTs

[Ard Biesheuvel]

On Thu, 21 Sept 2023 at 10:50, Peter Maydell <peter.maydell@linaro.org> wrote:
>
> Hi; I've been looking again at a very long standing missing feature in
> the QEMU virt board, which is that we only have one UART. One of the
> things that has stalled this in the past has been the odd behaviour of
> EDK2 if the DTB that QEMU passes it describes two UARTs.
>
> I'm going to describe the behaviour I see in more detail below, but to
> put the summary up front:
>  * EDK2 puts some debug output on one UART and some on the other
>    (the exact arrangement depends on ordering of the dtb nodes)
>  * EDK2 doesn't look at either stdout-path or the serial* aliases,
>    so its choices about how to use the UARTs differ from those
>    made by the guest kernel it is booting (and it also seems to be
>    iterating through the dtb in the opposite order to the kernel)
>
> The current proposal for adding a second UART is that it only happens
> if you explicitly add one on the command line (with a second "-serial
> something" option), so whatever we do won't break existing user
> setups. So we have scope for saying "if you want to use a second UART,
> you're going to want a newer EDK2 which handles it better". Exactly
> what "better" means here is up for grabs, but honouring stdout-path
> and the serial aliases would be the ideal I think. It would also be
> possible to select a particular ordering for the DTB nodes to produce
> "least-worst" behaviour from an existing EDK2 binary, but I'm not
> sure if that's worth doing.
>
> What do the EDK2 folks think about what the correct behaviour
> should be for a 2-UART setup?
>

Hi Peter,

Thanks for the elaborate analysis.

EDK2's DEBUG output is extremely noisy, so being able to redirect this
output to a different UART would be very useful.

The stdout-path is the intended console, and so we should honour that.
This also means that we should parse aliases. But the console is
actually configurable [persistenly] via the UEFI menu, and so it would
be nice if we could take advantage of this flexibility. This means in
principle that the UARTs should be represented via different device
paths (which would include the base address so they are
distinguishable) with perhaps a magical alias which is the default and
is tied to whatever stdout-path points to. This way, all the logic we
introduce is spec compliant and reusable on physical platforms with
multiple UARTs.

The DEBUG output is a different matter. On physical hardware, this is
typically configured at build time, as the info is needed extremely
early and on a physical platform, the debug port generally doesn't
change. Currently, we just grab the first UART that we encounter in
the DT, but the logic used by the DEBUG code and the ordinary console
driver are mostly separate.

What we might do is use stdout-path as well, unless a certain DT alias
exist perhaps? We should probably align here with other projects,
although this a distinction of the same nature may not exist there.

-- 
Ard.


-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.
View/Reply Online (#108941): https://edk2.groups.io/g/devel/message/108941
Mute This Topic: https://groups.io/mt/101498371/7686176
Group Owner: devel+owner@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub [rebecca@openfw.io]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [edk2-devel] EDK2 ArmVirtQemu behaviour with multiple UARTs

[Laszlo Ersek]

On 9/21/23 14:02, ardb at kernel.org (Ard Biesheuvel) wrote:
> On Thu, 21 Sept 2023 at 10:50, Peter Maydell <peter.maydell at linaro.org> wrote:
>>
>> Hi; I've been looking again at a very long standing missing feature in
>> the QEMU virt board, which is that we only have one UART. One of the
>> things that has stalled this in the past has been the odd behaviour of
>> EDK2 if the DTB that QEMU passes it describes two UARTs.
>>
>> I'm going to describe the behaviour I see in more detail below, but to
>> put the summary up front:
>>  * EDK2 puts some debug output on one UART and some on the other
>>    (the exact arrangement depends on ordering of the dtb nodes)
>>  * EDK2 doesn't look at either stdout-path or the serial* aliases,
>>    so its choices about how to use the UARTs differ from those
>>    made by the guest kernel it is booting (and it also seems to be
>>    iterating through the dtb in the opposite order to the kernel)
>>
>> The current proposal for adding a second UART is that it only happens
>> if you explicitly add one on the command line (with a second "-serial
>> something" option), so whatever we do won't break existing user
>> setups. So we have scope for saying "if you want to use a second UART,
>> you're going to want a newer EDK2 which handles it better". Exactly
>> what "better" means here is up for grabs, but honouring stdout-path
>> and the serial aliases would be the ideal I think. It would also be
>> possible to select a particular ordering for the DTB nodes to produce
>> "least-worst" behaviour from an existing EDK2 binary, but I'm not
>> sure if that's worth doing.
>>
>> What do the EDK2 folks think about what the correct behaviour
>> should be for a 2-UART setup?
>>
> 
> Hi Peter,
> 
> Thanks for the elaborate analysis.
> 
> EDK2's DEBUG output is extremely noisy, so being able to redirect this
> output to a different UART would be very useful.
> 
> The stdout-path is the intended console, and so we should honour that.
> This also means that we should parse aliases. But the console is
> actually configurable [persistenly] via the UEFI menu, and so it would
> be nice if we could take advantage of this flexibility. This means in
> principle that the UARTs should be represented via different device
> paths (which would include the base address so they are
> distinguishable) with perhaps a magical alias which is the default and
> is tied to whatever stdout-path points to. This way, all the logic we
> introduce is spec compliant and reusable on physical platforms with
> multiple UARTs.
> 
> The DEBUG output is a different matter. On physical hardware, this is
> typically configured at build time, as the info is needed extremely
> early and on a physical platform, the debug port generally doesn't
> change. Currently, we just grab the first UART that we encounter in
> the DT, but the logic used by the DEBUG code and the ordinary console
> driver are mostly separate.
> 
> What we might do is use stdout-path as well, unless a certain DT alias
> exist perhaps? We should probably align here with other projects,
> although this a distinction of the same nature may not exist there.
> 

Alias parsing in edk2 would be a bit too complicated for my taste. :)

I see the following two problems with the current state (based on
Peter's captures, using the original UART order in the DTB, i.e.,
<https://people.linaro.org/~peter.maydell/uart0.txt> and
<https://people.linaro.org/~peter.maydell/uart1.txt>):

(1) The DEBUG output switches from one UART to the other when we reach
the DXE_CORE (in this case, from UART0 to UART1, but the precise numbers
aren't the problem, the switchover is),

(2) The UEFI console (which is used by the setup browser, the UEFI
shell, grub, etc) is on UART1, while the kernel stuff is on UART0.

Here's what I'd propose:

- if there is only one UART in the DTB, no change

- otherwise, direct all DEBUG messages to the UART found *second* via
forward traversal in the DTB (let's call this UART1), and include the
UART found *first* via forward traversal in the DTB (let's call this
UART0) in the UEFI console. Furthermore, do not expose UART1 in the UEFI
protocol database *at all* (don't install devpath protocol / SerialIo
protocol); make it effectively hidden hardware (similarly how the x86
QEMU debug console, IO Port 0x402, is not exposed at all). Let the
system think there is only one UART (UART0), and treat UART1 as a
"bespoke", custom debug device only. This also ensures that existent
higher level products such as libvirt, which may only handle UART0 at
the moment, will expose the interactive console (UEFI and Linux) to the
user, and at worst the firmware debug log will not be captured.

For this a few custom DebugLib / SerialPortLib instances may have to be
created, but that should be doable.

Laszlo



-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.
View/Reply Online (#109141): https://edk2.groups.io/g/devel/message/109141
Mute This Topic: https://groups.io/mt/101498371/7686176
Group Owner: devel+owner@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/leave/12367111/7686176/1913456212/xyzzy [rebecca@openfw.io]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [edk2-devel] EDK2 ArmVirtQemu behaviour with multiple UARTs

[Peter Maydell]

On Thu, 28 Sept 2023 at 08:54, Laszlo Ersek <lersek@redhat.com> wrote:
>
> On 9/21/23 14:02, ardb at kernel.org (Ard Biesheuvel) wrote:
> > EDK2's DEBUG output is extremely noisy, so being able to redirect this
> > output to a different UART would be very useful.
> >
> > The stdout-path is the intended console, and so we should honour that.
> > This also means that we should parse aliases. But the console is
> > actually configurable [persistenly] via the UEFI menu, and so it would
> > be nice if we could take advantage of this flexibility. This means in
> > principle that the UARTs should be represented via different device
> > paths (which would include the base address so they are
> > distinguishable) with perhaps a magical alias which is the default and
> > is tied to whatever stdout-path points to. This way, all the logic we
> > introduce is spec compliant and reusable on physical platforms with
> > multiple UARTs.

> > What we might do is use stdout-path as well, unless a certain DT alias
> > exist perhaps? We should probably align here with other projects,
> > although this a distinction of the same nature may not exist there.
> >
>
> Alias parsing in edk2 would be a bit too complicated for my taste. :)
>
> I see the following two problems with the current state (based on
> Peter's captures, using the original UART order in the DTB, i.e.,
> <https://people.linaro.org/~peter.maydell/uart0.txt> and
> <https://people.linaro.org/~peter.maydell/uart1.txt>):
>
> (1) The DEBUG output switches from one UART to the other when we reach
> the DXE_CORE (in this case, from UART0 to UART1, but the precise numbers
> aren't the problem, the switchover is),
>
> (2) The UEFI console (which is used by the setup browser, the UEFI
> shell, grub, etc) is on UART1, while the kernel stuff is on UART0.
>
> Here's what I'd propose:
>
> - if there is only one UART in the DTB, no change
>
> - otherwise, direct all DEBUG messages to the UART found *second* via
> forward traversal in the DTB (let's call this UART1), and include the
> UART found *first* via forward traversal in the DTB (let's call this
> UART0) in the UEFI console. Furthermore, do not expose UART1 in the UEFI
> protocol database *at all* (don't install devpath protocol / SerialIo
> protocol); make it effectively hidden hardware (similarly how the x86
> QEMU debug console, IO Port 0x402, is not exposed at all). Let the
> system think there is only one UART (UART0), and treat UART1 as a
> "bespoke", custom debug device only. This also ensures that existent
> higher level products such as libvirt, which may only handle UART0 at
> the moment, will expose the interactive console (UEFI and Linux) to the
> user, and at worst the firmware debug log will not be captured.

The 16550 version of the QEMU-specific EDK uart-location code (used when
running it under kvmtool) already honours stdout-path, so I'm not sure
why we wouldn't want to be consistent with that. I'm not really a fan of
anything that depends on ordering of nodes in the DTB -- it is pretty
fragile in my experience. The DTB spec provides a mechanism to
correctly identify which UART to use, so I think that there would
need to be a really strong reason not to do it that way.

thanks
-- PMM


-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.
View/Reply Online (#109146): https://edk2.groups.io/g/devel/message/109146
Mute This Topic: https://groups.io/mt/101498371/7686176
Group Owner: devel+owner@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub [rebecca@openfw.io]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [edk2-devel] EDK2 ArmVirtQemu behaviour with multiple UARTs

[Laszlo Ersek]

On 9/28/23 13:24, Peter Maydell wrote:
> On Thu, 28 Sept 2023 at 08:54, Laszlo Ersek <lersek@redhat.com> wrote:
>>
>> On 9/21/23 14:02, ardb at kernel.org (Ard Biesheuvel) wrote:
>>> EDK2's DEBUG output is extremely noisy, so being able to redirect this
>>> output to a different UART would be very useful.
>>>
>>> The stdout-path is the intended console, and so we should honour that.
>>> This also means that we should parse aliases. But the console is
>>> actually configurable [persistenly] via the UEFI menu, and so it would
>>> be nice if we could take advantage of this flexibility. This means in
>>> principle that the UARTs should be represented via different device
>>> paths (which would include the base address so they are
>>> distinguishable) with perhaps a magical alias which is the default and
>>> is tied to whatever stdout-path points to. This way, all the logic we
>>> introduce is spec compliant and reusable on physical platforms with
>>> multiple UARTs.
> 
>>> What we might do is use stdout-path as well, unless a certain DT alias
>>> exist perhaps? We should probably align here with other projects,
>>> although this a distinction of the same nature may not exist there.
>>>
>>
>> Alias parsing in edk2 would be a bit too complicated for my taste. :)
>>
>> I see the following two problems with the current state (based on
>> Peter's captures, using the original UART order in the DTB, i.e.,
>> <https://people.linaro.org/~peter.maydell/uart0.txt> and
>> <https://people.linaro.org/~peter.maydell/uart1.txt>):
>>
>> (1) The DEBUG output switches from one UART to the other when we reach
>> the DXE_CORE (in this case, from UART0 to UART1, but the precise numbers
>> aren't the problem, the switchover is),
>>
>> (2) The UEFI console (which is used by the setup browser, the UEFI
>> shell, grub, etc) is on UART1, while the kernel stuff is on UART0.
>>
>> Here's what I'd propose:
>>
>> - if there is only one UART in the DTB, no change
>>
>> - otherwise, direct all DEBUG messages to the UART found *second* via
>> forward traversal in the DTB (let's call this UART1), and include the
>> UART found *first* via forward traversal in the DTB (let's call this
>> UART0) in the UEFI console. Furthermore, do not expose UART1 in the UEFI
>> protocol database *at all* (don't install devpath protocol / SerialIo
>> protocol); make it effectively hidden hardware (similarly how the x86
>> QEMU debug console, IO Port 0x402, is not exposed at all). Let the
>> system think there is only one UART (UART0), and treat UART1 as a
>> "bespoke", custom debug device only. This also ensures that existent
>> higher level products such as libvirt, which may only handle UART0 at
>> the moment, will expose the interactive console (UEFI and Linux) to the
>> user, and at worst the firmware debug log will not be captured.
> 
> The 16550 version of the QEMU-specific EDK uart-location code (used when
> running it under kvmtool) already honours stdout-path,

Right -- I wasn't aware. I can see GetSerialConsolePortAddress() in
"ArmVirtPkg/Library/Fdt16550SerialPortHookLib/EarlyFdt16550SerialPortHookLib.c",
which is used in "ArmVirtPkg/ArmVirtKvmTool.dsc".

> so I'm not sure
> why we wouldn't want to be consistent with that. I'm not really a fan of
> anything that depends on ordering of nodes in the DTB -- it is pretty
> fragile in my experience. The DTB spec provides a mechanism to
> correctly identify which UART to use, so I think that there would
> need to be a really strong reason not to do it that way.

OK -- I think the proposal might still work if we replaced "UART found
second" with "first non-stdout-path UART", and "UART found first" with
"stdout-path UART".

Laszlo




-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.
View/Reply Online (#109144): https://edk2.groups.io/g/devel/message/109144
Mute This Topic: https://groups.io/mt/101498371/7686176
Group Owner: devel+owner@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/leave/12367111/7686176/1913456212/xyzzy [rebecca@openfw.io]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [edk2-devel] EDK2 ArmVirtQemu behaviour with multiple UARTs

[Laszlo Ersek]

On 9/28/23 13:50, Laszlo Ersek wrote:
> On 9/28/23 13:24, Peter Maydell wrote:
>> On Thu, 28 Sept 2023 at 08:54, Laszlo Ersek <lersek@redhat.com> wrote:
>>>
>>> On 9/21/23 14:02, ardb at kernel.org (Ard Biesheuvel) wrote:
>>>> EDK2's DEBUG output is extremely noisy, so being able to redirect this
>>>> output to a different UART would be very useful.
>>>>
>>>> The stdout-path is the intended console, and so we should honour that.
>>>> This also means that we should parse aliases. But the console is
>>>> actually configurable [persistenly] via the UEFI menu, and so it would
>>>> be nice if we could take advantage of this flexibility. This means in
>>>> principle that the UARTs should be represented via different device
>>>> paths (which would include the base address so they are
>>>> distinguishable) with perhaps a magical alias which is the default and
>>>> is tied to whatever stdout-path points to. This way, all the logic we
>>>> introduce is spec compliant and reusable on physical platforms with
>>>> multiple UARTs.
>>
>>>> What we might do is use stdout-path as well, unless a certain DT alias
>>>> exist perhaps? We should probably align here with other projects,
>>>> although this a distinction of the same nature may not exist there.
>>>>
>>>
>>> Alias parsing in edk2 would be a bit too complicated for my taste. :)
>>>
>>> I see the following two problems with the current state (based on
>>> Peter's captures, using the original UART order in the DTB, i.e.,
>>> <https://people.linaro.org/~peter.maydell/uart0.txt> and
>>> <https://people.linaro.org/~peter.maydell/uart1.txt>):
>>>
>>> (1) The DEBUG output switches from one UART to the other when we reach
>>> the DXE_CORE (in this case, from UART0 to UART1, but the precise numbers
>>> aren't the problem, the switchover is),
>>>
>>> (2) The UEFI console (which is used by the setup browser, the UEFI
>>> shell, grub, etc) is on UART1, while the kernel stuff is on UART0.
>>>
>>> Here's what I'd propose:
>>>
>>> - if there is only one UART in the DTB, no change
>>>
>>> - otherwise, direct all DEBUG messages to the UART found *second* via
>>> forward traversal in the DTB (let's call this UART1), and include the
>>> UART found *first* via forward traversal in the DTB (let's call this
>>> UART0) in the UEFI console. Furthermore, do not expose UART1 in the UEFI
>>> protocol database *at all* (don't install devpath protocol / SerialIo
>>> protocol); make it effectively hidden hardware (similarly how the x86
>>> QEMU debug console, IO Port 0x402, is not exposed at all). Let the
>>> system think there is only one UART (UART0), and treat UART1 as a
>>> "bespoke", custom debug device only. This also ensures that existent
>>> higher level products such as libvirt, which may only handle UART0 at
>>> the moment, will expose the interactive console (UEFI and Linux) to the
>>> user, and at worst the firmware debug log will not be captured.
>>
>> The 16550 version of the QEMU-specific EDK uart-location code (used when
>> running it under kvmtool) already honours stdout-path,
> 
> Right -- I wasn't aware. I can see GetSerialConsolePortAddress() in
> "ArmVirtPkg/Library/Fdt16550SerialPortHookLib/EarlyFdt16550SerialPortHookLib.c",
> which is used in "ArmVirtPkg/ArmVirtKvmTool.dsc".
> 
>> so I'm not sure
>> why we wouldn't want to be consistent with that. I'm not really a fan of
>> anything that depends on ordering of nodes in the DTB -- it is pretty
>> fragile in my experience. The DTB spec provides a mechanism to
>> correctly identify which UART to use, so I think that there would
>> need to be a really strong reason not to do it that way.
> 
> OK -- I think the proposal might still work if we replaced "UART found
> second" with "first non-stdout-path UART", and "UART found first" with
> "stdout-path UART".

I'm working on this.

Laszlo



-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.
View/Reply Online (#109239): https://edk2.groups.io/g/devel/message/109239
Mute This Topic: https://groups.io/mt/101498371/7686176
Group Owner: devel+owner@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/leave/12367111/7686176/1913456212/xyzzy [rebecca@openfw.io]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [edk2-devel] EDK2 ArmVirtQemu behaviour with multiple UARTs

[Gerd Hoffmann]

On Thu, Sep 21, 2023 at 11:50:20AM +0100, Peter Maydell wrote:
> Hi; I've been looking again at a very long standing missing feature in
> the QEMU virt board, which is that we only have one UART. One of the
> things that has stalled this in the past has been the odd behaviour of
> EDK2 if the DTB that QEMU passes it describes two UARTs.

Note that edk2 recently got support for virtio-serial, so you can use
that for the console and leave the uart for debug logging.  The prebuild
edk2 binaries in qemu have been updated days ago and these already
support for virtio-serial..

take care,
  Gerd



-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.
View/Reply Online (#108946): https://edk2.groups.io/g/devel/message/108946
Mute This Topic: https://groups.io/mt/101498371/7686176
Group Owner: devel+owner@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub [rebecca@openfw.io]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [edk2-devel] EDK2 ArmVirtQemu behaviour with multiple UARTs

[Peter Maydell]

On Thu, 21 Sept 2023 at 16:26, Gerd Hoffmann <kraxel@redhat.com> wrote:
>
> On Thu, Sep 21, 2023 at 11:50:20AM +0100, Peter Maydell wrote:
> > Hi; I've been looking again at a very long standing missing feature in
> > the QEMU virt board, which is that we only have one UART. One of the
> > things that has stalled this in the past has been the odd behaviour of
> > EDK2 if the DTB that QEMU passes it describes two UARTs.
>
> Note that edk2 recently got support for virtio-serial, so you can use
> that for the console and leave the uart for debug logging.  The prebuild
> edk2 binaries in qemu have been updated days ago and these already
> support for virtio-serial..

As long as EDK2 does something sensible when the DTB says "two
UARTs here and here" and it also finds a virtio-serial PCI
device, I don't mind what exactly it does. The problem here is
more that EDK2 currently does strange things when told that
the hardware is present, rather than that anybody specifically wants
EDK2 to use multiple serial outputs.

Though given there's no way to say in the DTB "use a PCI card
for your console" I think the virtio-serial approach is likely
to be awkward for users in practice.

-- PMM


-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.
View/Reply Online (#108960): https://edk2.groups.io/g/devel/message/108960
Mute This Topic: https://groups.io/mt/101498371/7686176
Group Owner: devel+owner@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub [rebecca@openfw.io]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [edk2-devel] EDK2 ArmVirtQemu behaviour with multiple UARTs

[Gerd Hoffmann]

On Thu, Sep 21, 2023 at 04:34:27PM +0100, Peter Maydell wrote:
> As long as EDK2 does something sensible when the DTB says "two
> UARTs here and here" and it also finds a virtio-serial PCI
> device, I don't mind what exactly it does. The problem here is
> more that EDK2 currently does strange things when told that
> the hardware is present, rather than that anybody specifically wants
> EDK2 to use multiple serial outputs.
> 
> Though given there's no way to say in the DTB "use a PCI card
> for your console" I think the virtio-serial approach is likely
> to be awkward for users in practice.

edk2 adds a virtio console to the edk2 console multiplexer if
present (for both pci and mmio virtio transports), and systemd
spawns also spawns a getty on /dev/hvc0 if present.  So this
works mostly automatic.  Only if you also want the linux boot
messages show up there too you need to add 'console=hvc0' to
your kernel command line.

take care,
  Gerd



-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.
View/Reply Online (#108958): https://edk2.groups.io/g/devel/message/108958
Mute This Topic: https://groups.io/mt/101498371/7686176
Group Owner: devel+owner@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub [rebecca@openfw.io]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [edk2-devel] EDK2 ArmVirtQemu behaviour with multiple UARTs

[Laszlo Ersek]

On 9/21/23 12:50, peter.maydell at linaro.org (Peter Maydell) wrote:

> If you want to play around with this, I have some WIP patches at
> https://git.linaro.org/people/pmaydell/qemu-arm.git uart-edk-investigation
> (content wise they should be fine, but I haven't cleaned them up into
> a coherent set of distinct patches yet, so they're a bit messy.)
> A run of QEMU with both UARTs which sends all output to files looks like:
> 
> ./build/arm-clang/qemu-system-aarch64 -display none -vga none \
>   -machine virt,acpi=on,virtualization=on,mte=on,gic-version=max,iommu=smmuv3 \
>   -smp 2 -m 1024 -cpu max,pauth-impdef=on \
>   -bios ~/linaro/edk2/QEMU_EFI_DEBUG.fd \
>   -drive file=/home/petmay01/avocado/data/cache/by_location/0154b7cd3a4f5e135299060c8cabbeec10b70b6d/alpine-standard-3.17.2-aarch64.iso,format=raw
> \
>   -device virtio-rng-pci,rng=rng0 \
>   -object rng-random,id=rng0,filename=/dev/urandom \
>   -chardev file,id=chr0,path=/tmp/uart0-rev.txt \
>   -chardev file,id=chr1,path=/tmp/uart1-rev.txt \
>   -serial chardev:chr0 -serial chardev:chr1
> 
> (adjust -serial options to taste)

Is the "uart-edk-investigation" branch still the most recent one for
this? (I've not checked it yet, but coming close.)

Thanks
Laszlo



-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.
View/Reply Online (#109241): https://edk2.groups.io/g/devel/message/109241
Mute This Topic: https://groups.io/mt/101498371/7686176
Group Owner: devel+owner@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/leave/12367111/7686176/1913456212/xyzzy [rebecca@openfw.io]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [edk2-devel] EDK2 ArmVirtQemu behaviour with multiple UARTs

[Laszlo Ersek]

On 10/2/23 03:51, Laszlo Ersek wrote:
> On 9/21/23 12:50, peter.maydell at linaro.org (Peter Maydell) wrote:
> 
>> If you want to play around with this, I have some WIP patches at
>> https://git.linaro.org/people/pmaydell/qemu-arm.git uart-edk-investigation
>> (content wise they should be fine, but I haven't cleaned them up into
>> a coherent set of distinct patches yet, so they're a bit messy.)
>> A run of QEMU with both UARTs which sends all output to files looks like:
>>
>> ./build/arm-clang/qemu-system-aarch64 -display none -vga none \
>>   -machine virt,acpi=on,virtualization=on,mte=on,gic-version=max,iommu=smmuv3 \
>>   -smp 2 -m 1024 -cpu max,pauth-impdef=on \
>>   -bios ~/linaro/edk2/QEMU_EFI_DEBUG.fd \
>>   -drive file=/home/petmay01/avocado/data/cache/by_location/0154b7cd3a4f5e135299060c8cabbeec10b70b6d/alpine-standard-3.17.2-aarch64.iso,format=raw
>> \
>>   -device virtio-rng-pci,rng=rng0 \
>>   -object rng-random,id=rng0,filename=/dev/urandom \
>>   -chardev file,id=chr0,path=/tmp/uart0-rev.txt \
>>   -chardev file,id=chr1,path=/tmp/uart1-rev.txt \
>>   -serial chardev:chr0 -serial chardev:chr1
>>
>> (adjust -serial options to taste)
> 
> Is the "uart-edk-investigation" branch still the most recent one for
> this? (I've not checked it yet, but coming close.)

I checked out your branch and built it at commit "virt: Reverse order of UART dtb nodes".

I included my edk2 patches in my edk2 binary.

The QEMU command line is

/opt/qemu-installed-optimized/bin/qemu-system-aarch64 \
  -device virtio-gpu-pci \
  -machine virt,acpi=on,virtualization=on,mte=on,gic-version=max,iommu=smmuv3 \
  -smp 2 \
  -m 1024 \
  -cpu max,pauth-impdef=on \
  -drive if=pflash,file=/home/virt-images/arm/QEMU_EFI.fd.padded,format=raw,readonly=on \
  -drive if=pflash,file=/home/virt-images/arm/QEMU_VARS.fd.padded,format=raw,snapshot=on \
  -drive if=none,file=/home/lacos/tmp/alpine-standard-3.18.4-aarch64.iso,format=raw,readonly=on,id=alpine \
  -device virtio-scsi-pci,id=vscsi \
  -device scsi-cd,drive=alpine,bus=vscsi.0,bootindex=1 \
  -device virtio-rng-pci,rng=rng0 \
  -object rng-random,id=rng0,filename=/dev/urandom \
  -device qemu-xhci,id=xhci \
  -device usb-kbd,bus=xhci.0 \
  -chardev stdio,mux=on,id=monitor-and-console \
  -mon chardev=monitor-and-console,mode=readline \
  -serial chardev:monitor-and-console \
  -chardev file,id=debug-log,path=/home/lacos/tmp/debug.txt \
  -serial chardev:debug-log

With this command line, I have three windows to watch:

(1) a graphical guest display window (virtio-gpu-pci), supposed to serve the following purposes:
(1.1) multiplexed output from the UEFI console,
(1.2) Linux virtual console (in character mode)

(2) the terminal where I start the command from, supposed to serve (mux / C-a) the following purposes:
(2.1) the QEMU monitor,
(2.2) the guest serial port that is *supposed* to serve the following purposes:
(2.2.1) direct SerialPortLib usage from edk2 modules,
(2.2.2) the UEFI console,
(2.2.3) the Linux boot log and a serial login prompt

(3) a separate terminal where I run "tail -F debug.txt", intending to see:
(3.1) the edk2 DEBUG log

Also I intend to provide input via:
(4) USB-3 keyboard when focusing the graphical window (1)
(5) QEMU monitor commands in the terminal where I start QEMU (2)
(6) serial input to the guest in the terminal where I start QEMU (2)

Results:

* edk2 writes the message

  UEFI firmware (version  built at 03:38:24 on Oct  2 2023)

to window (2), with direct SerialPortLib usage.

* the edk2 DEBUG log goes solely to the file "debug.txt" / window (3), not messing up the UEFI console (i.e., windows (1) and (2))

* the edk2 DEBUG log contains the following two lines:

  PlatformPeim: PL011 UART (console) @ 0x9000000
  PlatformPeim: PL011 UART (debug) @ 0x9040000

* the UEFI console output is properly multiplexed to windows (1) and (2), and the UEFI console properly takes input from both (4) and (6). This covers both the edk2 Boot Manager, and Grub from the alpine ISO.

* in Grub, I replace alpine's default kernel option "quiet" with the two options "ignore_loglevel efi=debug". (Because grub uses the UEFI console, it properly takes input from both (4) and (6).)

* I see the guest kernel's EFI Stub's messages on the UEFI console (multiplexed to windows (1) and (2))

* I see the kernel boot log in window (2) -- that is, on the proper serial port. This includes both kernel debug messages, and userspace boot log messages

* Finally I get the following printout:

  Welcome to Alpine Linux 3.18
  Kernel 6.1.55-0-lts on an aarch64 (TTY_LINE)

in *all three* windows. Notably, the TTY_LINE value is the following:

- in window (1) -- virtio-gpu-pci --, it is "/dev/tty1"

- in window (2) (the "console" serial port) it is "/dev/ttyAMA0"

- in window (3) (that is, the "debug.txt" file) it is "/dev/ttyAMA1"

* Linux takes keyboard input in both windows (1) and (2) via (4) and (6) respectively; (3) is not testable (and is not expected to take input) because "-chardev file" is only good for output.

So this is *total success*; the only (positive) surprise is that Linux provides a login prompt on /dev/ttyAMA1 as well.

* Rebuilding QEMU at the *parent* patch -- namely "add to acpi table" --, that is, with patch "virt: Reverse order of UART dtb nodes" *popped*, all of the above results remain identical.

* dropping the "debug-log" chardev, and the serial port that references it, from the QEMU cmdline, there is one difference (and those results generally match the current upstream status): the edk2 DEBUG log is merged into the UEFI console output as the latter is multiplexed to the sole serial port -- making a mess for the reader. And of course Linux cannot offer a login prompt on "/dev/ttyAMA1".

I say we want this!

I have one patch under review ("[edk2-devel] [PATCH 1/1] ArmVirtPkg/FdtPL011SerialPortLib: initialize implicitly"), which is independent, but contextually a pre-requisite for my "armvirt-dual-serial" branch. Once that patch is merged, I'll post this branch. And I suggest going forward with the QEMU-side patches as well.

--*--

... Just for kicks, I also tested the following change to the QEMU command line:

@@ -19,6 +19,7 @@
   -device usb-kbd,bus=xhci.0 \
   -chardev stdio,mux=on,id=monitor-and-console \
   -mon chardev=monitor-and-console,mode=readline \
-  -serial chardev:monitor-and-console \
+  -device virtio-serial-pci,id=vserial \
+  -device virtconsole,bus=vserial.0,chardev=monitor-and-console \
   -chardev file,id=debug-log,path=/home/lacos/tmp/debug.txt \
   -serial chardev:debug-log

The behavior is not immediately intuitive, but it's correct after all. The *sole* PL011 UART -- now in "debug.txt" in window (3) -- unifies several purposes in edk2: UEFI console IO (including edk2 Boot manager and Grub), direct SerialPortLib output, and DEBUG messages. The virtio-gpu-pci window (1) functions undisturbed (UEFI console). The virtio serial console in window (2) properly works as a "head" of the UEFI console.

Even with the virtio serial console, I think that having *two* PL011 serial ports will be beneficial. With virtio-serial plus *one* PL011, we can have undisturbed non-graphical UEFI console, but there still isn't an undisturbed DEBUG log -- the sole PL011 still includes both UEFI console (multiplexed) and DEBUG. If we add in the second PL011, then the DEBUG log is pristine as well, same as with isa-debugcon under x86.

Furthermore, a disadvantage of virtio-serial plus *one* PL011 is that once I boot Linux, the kernel log + userspace log goes to the PL011. That's a bit of a UX disruption, because in this setup, we tend to look at the virtio-serial console, due to the PL011 still being a mixture of UEFI console + DEBUG. So the Linux boot log appears "elsewhere", and even the login prompt does not appear on virtio-serial (probably because in this build of Alpine, no virtio-serial driver is included; I guess anyway).

So either way, to mirror the x86 experience most closely (where we have undisturbed UEFI console on the ISA serial port *and/or* on virtio serial; *plus* undisturbed DEBUG output on isa-debugcon), we need two PL011 UARTs on aarch64. With one PL011 plus virtio-serial, the UEFI console is cleaned up (on virtio-serial), but the sole PL011 still contains a mixure of DEBUG and UEFI console -- and that's the only way we get a (somewhat garbled) DEBUG log in that setup.

Thanks,
Laszlo



-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.
View/Reply Online (#109268): https://edk2.groups.io/g/devel/message/109268
Mute This Topic: https://groups.io/mt/101498371/7686176
Group Owner: devel+owner@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/leave/12367111/7686176/1913456212/xyzzy [rebecca@openfw.io]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [edk2-devel] EDK2 ArmVirtQemu behaviour with multiple UARTs

[Laszlo Ersek]

On 10/3/23 01:05, Laszlo Ersek wrote:
> On 10/2/23 03:51, Laszlo Ersek wrote:
>> On 9/21/23 12:50, peter.maydell at linaro.org (Peter Maydell) wrote:
>>
>>> If you want to play around with this, I have some WIP patches at
>>> https://git.linaro.org/people/pmaydell/qemu-arm.git uart-edk-investigation
>>> (content wise they should be fine, but I haven't cleaned them up into
>>> a coherent set of distinct patches yet, so they're a bit messy.)
>>> A run of QEMU with both UARTs which sends all output to files looks like:
>>>
>>> ./build/arm-clang/qemu-system-aarch64 -display none -vga none \
>>>   -machine virt,acpi=on,virtualization=on,mte=on,gic-version=max,iommu=smmuv3 \
>>>   -smp 2 -m 1024 -cpu max,pauth-impdef=on \
>>>   -bios ~/linaro/edk2/QEMU_EFI_DEBUG.fd \
>>>   -drive file=/home/petmay01/avocado/data/cache/by_location/0154b7cd3a4f5e135299060c8cabbeec10b70b6d/alpine-standard-3.17.2-aarch64.iso,format=raw
>>> \
>>>   -device virtio-rng-pci,rng=rng0 \
>>>   -object rng-random,id=rng0,filename=/dev/urandom \
>>>   -chardev file,id=chr0,path=/tmp/uart0-rev.txt \
>>>   -chardev file,id=chr1,path=/tmp/uart1-rev.txt \
>>>   -serial chardev:chr0 -serial chardev:chr1
>>>
>>> (adjust -serial options to taste)
>>
>> Is the "uart-edk-investigation" branch still the most recent one for
>> this? (I've not checked it yet, but coming close.)
> 
> I checked out your branch and built it at commit "virt: Reverse order of UART dtb nodes".
> 
> I included my edk2 patches in my edk2 binary.
> 
> The QEMU command line is
> 
> /opt/qemu-installed-optimized/bin/qemu-system-aarch64 \
>   -device virtio-gpu-pci \
>   -machine virt,acpi=on,virtualization=on,mte=on,gic-version=max,iommu=smmuv3 \
>   -smp 2 \
>   -m 1024 \
>   -cpu max,pauth-impdef=on \
>   -drive if=pflash,file=/home/virt-images/arm/QEMU_EFI.fd.padded,format=raw,readonly=on \
>   -drive if=pflash,file=/home/virt-images/arm/QEMU_VARS.fd.padded,format=raw,snapshot=on \
>   -drive if=none,file=/home/lacos/tmp/alpine-standard-3.18.4-aarch64.iso,format=raw,readonly=on,id=alpine \
>   -device virtio-scsi-pci,id=vscsi \
>   -device scsi-cd,drive=alpine,bus=vscsi.0,bootindex=1 \
>   -device virtio-rng-pci,rng=rng0 \
>   -object rng-random,id=rng0,filename=/dev/urandom \
>   -device qemu-xhci,id=xhci \
>   -device usb-kbd,bus=xhci.0 \
>   -chardev stdio,mux=on,id=monitor-and-console \
>   -mon chardev=monitor-and-console,mode=readline \
>   -serial chardev:monitor-and-console \
>   -chardev file,id=debug-log,path=/home/lacos/tmp/debug.txt \
>   -serial chardev:debug-log
> 
> With this command line, I have three windows to watch:
> 
> (1) a graphical guest display window (virtio-gpu-pci), supposed to serve the following purposes:
> (1.1) multiplexed output from the UEFI console,
> (1.2) Linux virtual console (in character mode)
> 
> (2) the terminal where I start the command from, supposed to serve (mux / C-a) the following purposes:
> (2.1) the QEMU monitor,
> (2.2) the guest serial port that is *supposed* to serve the following purposes:
> (2.2.1) direct SerialPortLib usage from edk2 modules,
> (2.2.2) the UEFI console,
> (2.2.3) the Linux boot log and a serial login prompt
> 
> (3) a separate terminal where I run "tail -F debug.txt", intending to see:
> (3.1) the edk2 DEBUG log
> 
> Also I intend to provide input via:
> (4) USB-3 keyboard when focusing the graphical window (1)
> (5) QEMU monitor commands in the terminal where I start QEMU (2)
> (6) serial input to the guest in the terminal where I start QEMU (2)
> 
> Results:
> 
> * edk2 writes the message
> 
>   UEFI firmware (version  built at 03:38:24 on Oct  2 2023)
> 
> to window (2), with direct SerialPortLib usage.
> 
> * the edk2 DEBUG log goes solely to the file "debug.txt" / window (3), not messing up the UEFI console (i.e., windows (1) and (2))
> 
> * the edk2 DEBUG log contains the following two lines:
> 
>   PlatformPeim: PL011 UART (console) @ 0x9000000
>   PlatformPeim: PL011 UART (debug) @ 0x9040000
> 
> * the UEFI console output is properly multiplexed to windows (1) and (2), and the UEFI console properly takes input from both (4) and (6). This covers both the edk2 Boot Manager, and Grub from the alpine ISO.
> 
> * in Grub, I replace alpine's default kernel option "quiet" with the two options "ignore_loglevel efi=debug". (Because grub uses the UEFI console, it properly takes input from both (4) and (6).)
> 
> * I see the guest kernel's EFI Stub's messages on the UEFI console (multiplexed to windows (1) and (2))
> 
> * I see the kernel boot log in window (2) -- that is, on the proper serial port. This includes both kernel debug messages, and userspace boot log messages
> 
> * Finally I get the following printout:
> 
>   Welcome to Alpine Linux 3.18
>   Kernel 6.1.55-0-lts on an aarch64 (TTY_LINE)
> 
> in *all three* windows. Notably, the TTY_LINE value is the following:
> 
> - in window (1) -- virtio-gpu-pci --, it is "/dev/tty1"
> 
> - in window (2) (the "console" serial port) it is "/dev/ttyAMA0"
> 
> - in window (3) (that is, the "debug.txt" file) it is "/dev/ttyAMA1"
> 
> * Linux takes keyboard input in both windows (1) and (2) via (4) and (6) respectively; (3) is not testable (and is not expected to take input) because "-chardev file" is only good for output.
> 
> So this is *total success*; the only (positive) surprise is that Linux provides a login prompt on /dev/ttyAMA1 as well.
> 
> * Rebuilding QEMU at the *parent* patch -- namely "add to acpi table" --, that is, with patch "virt: Reverse order of UART dtb nodes" *popped*, all of the above results remain identical.
> 
> * dropping the "debug-log" chardev, and the serial port that references it, from the QEMU cmdline, there is one difference (and those results generally match the current upstream status): the edk2 DEBUG log is merged into the UEFI console output as the latter is multiplexed to the sole serial port -- making a mess for the reader. And of course Linux cannot offer a login prompt on "/dev/ttyAMA1".
> 
> I say we want this!
> 
> I have one patch under review ("[edk2-devel] [PATCH 1/1] ArmVirtPkg/FdtPL011SerialPortLib: initialize implicitly"), which is independent, but contextually a pre-requisite for my "armvirt-dual-serial" branch. Once that patch is merged, I'll post this branch. And I suggest going forward with the QEMU-side patches as well.
> 
> --*--
> 
> ... Just for kicks, I also tested the following change to the QEMU command line:
> 
> @@ -19,6 +19,7 @@
>    -device usb-kbd,bus=xhci.0 \
>    -chardev stdio,mux=on,id=monitor-and-console \
>    -mon chardev=monitor-and-console,mode=readline \
> -  -serial chardev:monitor-and-console \
> +  -device virtio-serial-pci,id=vserial \
> +  -device virtconsole,bus=vserial.0,chardev=monitor-and-console \
>    -chardev file,id=debug-log,path=/home/lacos/tmp/debug.txt \
>    -serial chardev:debug-log
> 
> The behavior is not immediately intuitive, but it's correct after all. The *sole* PL011 UART -- now in "debug.txt" in window (3) -- unifies several purposes in edk2: UEFI console IO (including edk2 Boot manager and Grub), direct SerialPortLib output, and DEBUG messages. The virtio-gpu-pci window (1) functions undisturbed (UEFI console). The virtio serial console in window (2) properly works as a "head" of the UEFI console.
> 
> Even with the virtio serial console, I think that having *two* PL011 serial ports will be beneficial. With virtio-serial plus *one* PL011, we can have undisturbed non-graphical UEFI console, but there still isn't an undisturbed DEBUG log -- the sole PL011 still includes both UEFI console (multiplexed) and DEBUG. If we add in the second PL011, then the DEBUG log is pristine as well, same as with isa-debugcon under x86.
> 
> Furthermore, a disadvantage of virtio-serial plus *one* PL011 is that once I boot Linux, the kernel log + userspace log goes to the PL011. That's a bit of a UX disruption, because in this setup, we tend to look at the virtio-serial console, due to the PL011 still being a mixture of UEFI console + DEBUG. So the Linux boot log appears "elsewhere", and even the login prompt does not appear on virtio-serial (probably because in this build of Alpine, no virtio-serial driver is included; I guess anyway).

Based on Gerd's message <https://edk2.groups.io/g/devel/message/108958>
though, it seems both of these issues can be mitigated: the login prompt
on /dev/hvc0 should appear if the kernel / modules are built properly,
and the kernel messages can be kept on virtio-serial with "console=hvc0".

... I've actually tried testing that now, too, with this build of
Alpine. It seems to work, although the Linux boot log (kernel messages
vs. userspace) doesn't seem to "settle" on virtio-serial versus the one
PL011, for a ways into the Linux boot process. This would require more
analysis (probably Linux kernel analysis), but my laptop battery is at
6%, and either way it's orthogonal to implementing two PL011 UARTs in
qemu and edk2.

Cheers
Laszlo


> 
> So either way, to mirror the x86 experience most closely (where we have undisturbed UEFI console on the ISA serial port *and/or* on virtio serial; *plus* undisturbed DEBUG output on isa-debugcon), we need two PL011 UARTs on aarch64. With one PL011 plus virtio-serial, the UEFI console is cleaned up (on virtio-serial), but the sole PL011 still contains a mixure of DEBUG and UEFI console -- and that's the only way we get a (somewhat garbled) DEBUG log in that setup.
> 
> Thanks,
> Laszlo
> 



-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.
View/Reply Online (#109269): https://edk2.groups.io/g/devel/message/109269
Mute This Topic: https://groups.io/mt/101498371/7686176
Group Owner: devel+owner@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/leave/12367111/7686176/1913456212/xyzzy [rebecca@openfw.io]
-=-=-=-=-=-=-=-=-=-=-=-