From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail02.groups.io (mail02.groups.io [66.175.222.108]) by spool.mail.gandi.net (Postfix) with ESMTPS id 32CDE941E1C for ; Mon, 2 Oct 2023 23:14:35 +0000 (UTC) DKIM-Signature: a=rsa-sha256; bh=3Jk0Wan9VcCXHqJ9QmmLjackDJDzBTM/oiOeqtn0bog=; c=relaxed/simple; d=groups.io; h=Message-ID:Date:MIME-Version:Subject:From:To:Cc:Reply-To:References:In-Reply-To:Precedence:List-Subscribe:List-Help:Sender:List-Id:Mailing-List:Delivered-To:List-Unsubscribe-Post:List-Unsubscribe:Content-Language:Content-Type:Content-Transfer-Encoding; s=20140610; t=1696288473; v=1; b=fAwt2eUmBpy8ZtOE6mTwXF5il/nwmNg4pafD+pEVu/YqNApeFkvULeXhF4H2BJs0xR+9nhv9 JxwuKcHFb57bi+eRU7tPXl4YOzfLlW9PDMl+/2uQwAfFmoGxOqSMYFAdaS0RZvziLy1pv7sP3cs q4HetITRoF/vxzyVSKzKJgt4= X-Received: by 127.0.0.2 with SMTP id vgsBYY7687511xtV74wwdcM8; Mon, 02 Oct 2023 16:14:33 -0700 X-Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by mx.groups.io with SMTP id smtpd.web10.97097.1696288473120452698 for ; Mon, 02 Oct 2023 16:14:33 -0700 X-Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-115-R1hSYggnOpiiWH_eqouKTA-1; Mon, 02 Oct 2023 19:14:30 -0400 X-MC-Unique: R1hSYggnOpiiWH_eqouKTA-1 X-Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 368D18015AB; Mon, 2 Oct 2023 23:14:30 +0000 (UTC) X-Received: from [10.39.192.119] (unknown [10.39.192.119]) by smtp.corp.redhat.com (Postfix) with ESMTPS id ED1332026D4B; Mon, 2 Oct 2023 23:14:28 +0000 (UTC) Message-ID: <975f3bc0-07d7-561d-c6da-04b559969373@redhat.com> Date: Tue, 3 Oct 2023 01:14:27 +0200 MIME-Version: 1.0 Subject: Re: [edk2-devel] EDK2 ArmVirtQemu behaviour with multiple UARTs From: "Laszlo Ersek" To: Peter Maydell Cc: Ard Biesheuvel , Gerd Hoffmann , edk2-devel-groups-io Reply-To: devel@edk2.groups.io,lersek@redhat.com References: <1aee726a-4522-5513-95da-0160d92a1de2@redhat.com> <09b516d7-559d-303e-1532-9b5abe157b5a@redhat.com> In-Reply-To: <09b516d7-559d-303e-1532-9b5abe157b5a@redhat.com> X-Scanned-By: MIMEDefang 3.1 on 10.11.54.4 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Precedence: Bulk List-Subscribe: List-Help: Sender: devel@edk2.groups.io List-Id: Mailing-List: list devel@edk2.groups.io; contact devel+owner@edk2.groups.io List-Unsubscribe-Post: List-Unsubscribe=One-Click List-Unsubscribe: X-Gm-Message-State: yrxnhnDR1UKzR2lX5zjVhkdux7686176AA= Content-Language: en-US Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-GND-Status: LEGIT Authentication-Results: spool.mail.gandi.net; dkim=pass header.d=groups.io header.s=20140610 header.b=fAwt2eUm; dmarc=fail reason="SPF not aligned (relaxed), DKIM not aligned (relaxed)" header.from=redhat.com (policy=none); spf=pass (spool.mail.gandi.net: domain of bounce@groups.io designates 66.175.222.108 as permitted sender) smtp.mailfrom=bounce@groups.io On 10/3/23 01:05, Laszlo Ersek wrote: > On 10/2/23 03:51, Laszlo Ersek wrote: >> On 9/21/23 12:50, peter.maydell at linaro.org (Peter Maydell) wrote: >> >>> If you want to play around with this, I have some WIP patches at >>> https://git.linaro.org/people/pmaydell/qemu-arm.git uart-edk-investigat= ion >>> (content wise they should be fine, but I haven't cleaned them up into >>> a coherent set of distinct patches yet, so they're a bit messy.) >>> A run of QEMU with both UARTs which sends all output to files looks lik= e: >>> >>> ./build/arm-clang/qemu-system-aarch64 -display none -vga none \ >>> -machine virt,acpi=3Don,virtualization=3Don,mte=3Don,gic-version=3Dma= x,iommu=3Dsmmuv3 \ >>> -smp 2 -m 1024 -cpu max,pauth-impdef=3Don \ >>> -bios ~/linaro/edk2/QEMU_EFI_DEBUG.fd \ >>> -drive file=3D/home/petmay01/avocado/data/cache/by_location/0154b7cd3= a4f5e135299060c8cabbeec10b70b6d/alpine-standard-3.17.2-aarch64.iso,format= =3Draw >>> \ >>> -device virtio-rng-pci,rng=3Drng0 \ >>> -object rng-random,id=3Drng0,filename=3D/dev/urandom \ >>> -chardev file,id=3Dchr0,path=3D/tmp/uart0-rev.txt \ >>> -chardev file,id=3Dchr1,path=3D/tmp/uart1-rev.txt \ >>> -serial chardev:chr0 -serial chardev:chr1 >>> >>> (adjust -serial options to taste) >> >> Is the "uart-edk-investigation" branch still the most recent one for >> this? (I've not checked it yet, but coming close.) >=20 > I checked out your branch and built it at commit "virt: Reverse order of = UART dtb nodes". >=20 > I included my edk2 patches in my edk2 binary. >=20 > The QEMU command line is >=20 > /opt/qemu-installed-optimized/bin/qemu-system-aarch64 \ > -device virtio-gpu-pci \ > -machine virt,acpi=3Don,virtualization=3Don,mte=3Don,gic-version=3Dmax,= iommu=3Dsmmuv3 \ > -smp 2 \ > -m 1024 \ > -cpu max,pauth-impdef=3Don \ > -drive if=3Dpflash,file=3D/home/virt-images/arm/QEMU_EFI.fd.padded,form= at=3Draw,readonly=3Don \ > -drive if=3Dpflash,file=3D/home/virt-images/arm/QEMU_VARS.fd.padded,for= mat=3Draw,snapshot=3Don \ > -drive if=3Dnone,file=3D/home/lacos/tmp/alpine-standard-3.18.4-aarch64.= iso,format=3Draw,readonly=3Don,id=3Dalpine \ > -device virtio-scsi-pci,id=3Dvscsi \ > -device scsi-cd,drive=3Dalpine,bus=3Dvscsi.0,bootindex=3D1 \ > -device virtio-rng-pci,rng=3Drng0 \ > -object rng-random,id=3Drng0,filename=3D/dev/urandom \ > -device qemu-xhci,id=3Dxhci \ > -device usb-kbd,bus=3Dxhci.0 \ > -chardev stdio,mux=3Don,id=3Dmonitor-and-console \ > -mon chardev=3Dmonitor-and-console,mode=3Dreadline \ > -serial chardev:monitor-and-console \ > -chardev file,id=3Ddebug-log,path=3D/home/lacos/tmp/debug.txt \ > -serial chardev:debug-log >=20 > With this command line, I have three windows to watch: >=20 > (1) a graphical guest display window (virtio-gpu-pci), supposed to serve = the following purposes: > (1.1) multiplexed output from the UEFI console, > (1.2) Linux virtual console (in character mode) >=20 > (2) the terminal where I start the command from, supposed to serve (mux /= C-a) the following purposes: > (2.1) the QEMU monitor, > (2.2) the guest serial port that is *supposed* to serve the following pur= poses: > (2.2.1) direct SerialPortLib usage from edk2 modules, > (2.2.2) the UEFI console, > (2.2.3) the Linux boot log and a serial login prompt >=20 > (3) a separate terminal where I run "tail -F debug.txt", intending to see= : > (3.1) the edk2 DEBUG log >=20 > Also I intend to provide input via: > (4) USB-3 keyboard when focusing the graphical window (1) > (5) QEMU monitor commands in the terminal where I start QEMU (2) > (6) serial input to the guest in the terminal where I start QEMU (2) >=20 > Results: >=20 > * edk2 writes the message >=20 > UEFI firmware (version built at 03:38:24 on Oct 2 2023) >=20 > to window (2), with direct SerialPortLib usage. >=20 > * the edk2 DEBUG log goes solely to the file "debug.txt" / window (3), no= t messing up the UEFI console (i.e., windows (1) and (2)) >=20 > * the edk2 DEBUG log contains the following two lines: >=20 > PlatformPeim: PL011 UART (console) @ 0x9000000 > PlatformPeim: PL011 UART (debug) @ 0x9040000 >=20 > * the UEFI console output is properly multiplexed to windows (1) and (2),= and the UEFI console properly takes input from both (4) and (6). This cove= rs both the edk2 Boot Manager, and Grub from the alpine ISO. >=20 > * in Grub, I replace alpine's default kernel option "quiet" with the two = options "ignore_loglevel efi=3Ddebug". (Because grub uses the UEFI console,= it properly takes input from both (4) and (6).) >=20 > * I see the guest kernel's EFI Stub's messages on the UEFI console (multi= plexed to windows (1) and (2)) >=20 > * I see the kernel boot log in window (2) -- that is, on the proper seria= l port. This includes both kernel debug messages, and userspace boot log me= ssages >=20 > * Finally I get the following printout: >=20 > Welcome to Alpine Linux 3.18 > Kernel 6.1.55-0-lts on an aarch64 (TTY_LINE) >=20 > in *all three* windows. Notably, the TTY_LINE value is the following: >=20 > - in window (1) -- virtio-gpu-pci --, it is "/dev/tty1" >=20 > - in window (2) (the "console" serial port) it is "/dev/ttyAMA0" >=20 > - in window (3) (that is, the "debug.txt" file) it is "/dev/ttyAMA1" >=20 > * Linux takes keyboard input in both windows (1) and (2) via (4) and (6) = respectively; (3) is not testable (and is not expected to take input) becau= se "-chardev file" is only good for output. >=20 > So this is *total success*; the only (positive) surprise is that Linux pr= ovides a login prompt on /dev/ttyAMA1 as well. >=20 > * Rebuilding QEMU at the *parent* patch -- namely "add to acpi table" --,= that is, with patch "virt: Reverse order of UART dtb nodes" *popped*, all = of the above results remain identical. >=20 > * dropping the "debug-log" chardev, and the serial port that references i= t, from the QEMU cmdline, there is one difference (and those results genera= lly match the current upstream status): the edk2 DEBUG log is merged into t= he UEFI console output as the latter is multiplexed to the sole serial port= -- making a mess for the reader. And of course Linux cannot offer a login = prompt on "/dev/ttyAMA1". >=20 > I say we want this! >=20 > I have one patch under review ("[edk2-devel] [PATCH 1/1] ArmVirtPkg/FdtPL= 011SerialPortLib: initialize implicitly"), which is independent, but contex= tually a pre-requisite for my "armvirt-dual-serial" branch. Once that patch= is merged, I'll post this branch. And I suggest going forward with the QEM= U-side patches as well. >=20 > --*-- >=20 > ... Just for kicks, I also tested the following change to the QEMU comman= d line: >=20 > @@ -19,6 +19,7 @@ > -device usb-kbd,bus=3Dxhci.0 \ > -chardev stdio,mux=3Don,id=3Dmonitor-and-console \ > -mon chardev=3Dmonitor-and-console,mode=3Dreadline \ > - -serial chardev:monitor-and-console \ > + -device virtio-serial-pci,id=3Dvserial \ > + -device virtconsole,bus=3Dvserial.0,chardev=3Dmonitor-and-console \ > -chardev file,id=3Ddebug-log,path=3D/home/lacos/tmp/debug.txt \ > -serial chardev:debug-log >=20 > The behavior is not immediately intuitive, but it's correct after all. Th= e *sole* PL011 UART -- now in "debug.txt" in window (3) -- unifies several = purposes in edk2: UEFI console IO (including edk2 Boot manager and Grub), d= irect SerialPortLib output, and DEBUG messages. The virtio-gpu-pci window (= 1) functions undisturbed (UEFI console). The virtio serial console in windo= w (2) properly works as a "head" of the UEFI console. >=20 > Even with the virtio serial console, I think that having *two* PL011 seri= al ports will be beneficial. With virtio-serial plus *one* PL011, we can ha= ve undisturbed non-graphical UEFI console, but there still isn't an undistu= rbed DEBUG log -- the sole PL011 still includes both UEFI console (multiple= xed) and DEBUG. If we add in the second PL011, then the DEBUG log is pristi= ne as well, same as with isa-debugcon under x86. >=20 > Furthermore, a disadvantage of virtio-serial plus *one* PL011 is that onc= e I boot Linux, the kernel log + userspace log goes to the PL011. That's a = bit of a UX disruption, because in this setup, we tend to look at the virti= o-serial console, due to the PL011 still being a mixture of UEFI console + = DEBUG. So the Linux boot log appears "elsewhere", and even the login prompt= does not appear on virtio-serial (probably because in this build of Alpine= , no virtio-serial driver is included; I guess anyway). Based on Gerd's message though, it seems both of these issues can be mitigated: the login prompt on /dev/hvc0 should appear if the kernel / modules are built properly, and the kernel messages can be kept on virtio-serial with "console=3Dhvc0". ... I've actually tried testing that now, too, with this build of Alpine. It seems to work, although the Linux boot log (kernel messages vs. userspace) doesn't seem to "settle" on virtio-serial versus the one PL011, for a ways into the Linux boot process. This would require more analysis (probably Linux kernel analysis), but my laptop battery is at 6%, and either way it's orthogonal to implementing two PL011 UARTs in qemu and edk2. Cheers Laszlo >=20 > So either way, to mirror the x86 experience most closely (where we have u= ndisturbed UEFI console on the ISA serial port *and/or* on virtio serial; *= plus* undisturbed DEBUG output on isa-debugcon), we need two PL011 UARTs on= aarch64. With one PL011 plus virtio-serial, the UEFI console is cleaned up= (on virtio-serial), but the sole PL011 still contains a mixure of DEBUG an= d UEFI console -- and that's the only way we get a (somewhat garbled) DEBUG= log in that setup. >=20 > Thanks, > Laszlo >=20 -=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D- Groups.io Links: You receive all messages sent to this group. View/Reply Online (#109269): https://edk2.groups.io/g/devel/message/109269 Mute This Topic: https://groups.io/mt/101498371/7686176 Group Owner: devel+owner@edk2.groups.io Unsubscribe: https://edk2.groups.io/g/devel/leave/12367111/7686176/19134562= 12/xyzzy [rebecca@openfw.io] -=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-