From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail02.groups.io (mail02.groups.io [66.175.222.108]) by spool.mail.gandi.net (Postfix) with ESMTPS id 78F3BAC1660 for ; Thu, 21 Sep 2023 17:07:50 +0000 (UTC) DKIM-Signature: a=rsa-sha256; bh=2JP9ZRqOx5hGteJurUODM+Yni/ZqzDkB13H+L8Htjc4=; c=relaxed/simple; d=groups.io; h=MIME-Version:From:Date:Message-ID:Subject:To:Cc:Precedence:List-Subscribe:List-Help:Sender:List-Id:Mailing-List:Delivered-To:Reply-To:List-Unsubscribe-Post:List-Unsubscribe:Content-Type; s=20140610; t=1695316069; v=1; b=klknAPgGZrfR7lUuuKHktzCJ7J5gVF/BI3XxxPccDJJjm8qYrwvE5xZTmINJMs7Gn2wljhhA OBDGufSNxTlx6KgHDhkeL0x07tNoAi1LDGJw8BXcubovHtIFkS/dT+QSI8+CLJgAItVKZDa2XXv NNAFfF4cUSKO6WctGQpdw8sU= X-Received: by 127.0.0.2 with SMTP id YuDEYY7687511xO56xs3NoOe; Thu, 21 Sep 2023 10:07:49 -0700 X-Received: from mail-ed1-f44.google.com (mail-ed1-f44.google.com [209.85.208.44]) by mx.groups.io with SMTP id smtpd.web11.13323.1695293441042005406 for ; Thu, 21 Sep 2023 03:50:41 -0700 X-Received: by mail-ed1-f44.google.com with SMTP id 4fb4d7f45d1cf-5335725cf84so90626a12.2 for ; Thu, 21 Sep 2023 03:50:40 -0700 (PDT) X-Gm-Message-State: BCZA8h3K2C3N50LZfDFcqaE4x7686176AA= X-Google-Smtp-Source: AGHT+IEL5QaN7pY3ypXxZdIA4Tn2VAmkV+jeWUuiGj114EC8xkS6M+CabmBEDnVPVsEvCHcHVa1IU11PmvZWq3MgZvY= X-Received: by 2002:a05:6402:515c:b0:532:bc4d:906f with SMTP id n28-20020a056402515c00b00532bc4d906fmr4455128edd.5.1695293439440; Thu, 21 Sep 2023 03:50:39 -0700 (PDT) MIME-Version: 1.0 From: Peter Maydell Date: Thu, 21 Sep 2023 11:50:20 +0100 Message-ID: Subject: [edk2-devel] EDK2 ArmVirtQemu behaviour with multiple UARTs To: QEMU Developers Cc: devel@edk2.groups.io, Leif Lindholm , Ard Biesheuvel , Sami Mujawar Precedence: Bulk List-Subscribe: List-Help: Sender: devel@edk2.groups.io List-Id: Mailing-List: list devel@edk2.groups.io; contact devel+owner@edk2.groups.io Reply-To: devel@edk2.groups.io,peter.maydell@linaro.org List-Unsubscribe-Post: List-Unsubscribe=One-Click List-Unsubscribe: Content-Type: text/plain; charset="UTF-8" X-GND-Status: LEGIT Authentication-Results: spool.mail.gandi.net; dkim=pass header.d=groups.io header.s=20140610 header.b=klknAPgG; dmarc=fail reason="SPF not aligned (relaxed), DKIM not aligned (relaxed)" header.from=linaro.org (policy=none); spf=pass (spool.mail.gandi.net: domain of bounce@groups.io designates 66.175.222.108 as permitted sender) smtp.mailfrom=bounce@groups.io Hi; I've been looking again at a very long standing missing feature in the QEMU virt board, which is that we only have one UART. One of the things that has stalled this in the past has been the odd behaviour of EDK2 if the DTB that QEMU passes it describes two UARTs. I'm going to describe the behaviour I see in more detail below, but to put the summary up front: * EDK2 puts some debug output on one UART and some on the other (the exact arrangement depends on ordering of the dtb nodes) * EDK2 doesn't look at either stdout-path or the serial* aliases, so its choices about how to use the UARTs differ from those made by the guest kernel it is booting (and it also seems to be iterating through the dtb in the opposite order to the kernel) The current proposal for adding a second UART is that it only happens if you explicitly add one on the command line (with a second "-serial something" option), so whatever we do won't break existing user setups. So we have scope for saying "if you want to use a second UART, you're going to want a newer EDK2 which handles it better". Exactly what "better" means here is up for grabs, but honouring stdout-path and the serial aliases would be the ideal I think. It would also be possible to select a particular ordering for the DTB nodes to produce "least-worst" behaviour from an existing EDK2 binary, but I'm not sure if that's worth doing. What do the EDK2 folks think about what the correct behaviour should be for a 2-UART setup? Anyway, on to the details about the setup and what I see from EDK2: This is all with a debug ArmVirtQemu build, running at EL2 (i.e. entirely non-secure), with some patches I've been working on to add the extra UART to the board and the DTB. The DTB has the two UARTs: pl011@9000000 { clock-names = "uartclk\0apb_pclk"; clocks = <0x8000 0x8000>; interrupts = <0x00 0x01 0x04>; reg = <0x00 0x9000000 0x00 0x1000>; compatible = "arm,pl011\0arm,primecell"; }; pl011@9040000 { clock-names = "uartclk\0apb_pclk"; clocks = <0x8000 0x8000>; interrupts = <0x00 0x08 0x04>; reg = <0x00 0x9040000 0x00 0x1000>; compatible = "arm,pl011\0arm,primecell"; }; and aliases: aliases { serial0 = "/pl011@9000000"; serial1 = "/pl011@9040000"; }; and in the /chosen node: stdout-path = "/pl011@9000000"; The ACPI table fragments generated by QEMU have entries for both UARTs, as COM0 and COM1. Given all this, EDK2 outputs: uart0: * some UEFI output including debug output, starting: UEFI firmware (version built at 15:19:20 on Sep 19 2023) add-symbol-file /home/petmay01/linaro/edk2/Build/ArmVirtQemu-AARCH64/DEBUG_GCC5/AARCH64/ArmPlatformPkg/PrePeiCore/PrePeiCoreUniCore/DEBUG/ArmPlatformPrePeiCore.dll 0x2000 add-symbol-file /home/petmay01/linaro/edk2/Build/ArmVirtQemu-AARCH64/DEBUG_GCC5/AARCH64/MdeModulePkg/Core/Pei/PeiMain/DEBUG/PeiCore.dll 0xD240 Register PPI Notify: DCD0BE23-9586-40F4-B643-06522CED4EDE Install PPI: 8C8CE578-8A3D-4F1C-9935-896185C32DD3 Install PPI: 5473C07A-3DCB-4DCA-BD6F-1E9689E7349A The 0th FV start address is 0x00000001000, size is 0x001FF000, handle is 0x1000 * the guest Linux kernel output (on what Linux says is ttyAMA0) uart1: * a lot of UEFI output including debug output, starting: DxeMain: MemoryBaseAddress=0x48000000 MemoryLength=0x38000000 add-symbol-file /home/petmay01/linaro/edk2/Build/ArmVirtQemu-AARCH64/DEBUG_GCC5/AARCH64/MdeModulePkg/Core/Dxe/DxeMain/DEBUG/DxeCore.dll 0x47860000 HOBLIST address in DXE = 0x7FA35018 Memory Allocation 0x00000004 0x47FFF000 - 0x47FFFFFF Memory Allocation 0x00000004 0x47FFE000 - 0x47FFEFFF * the GNU GRUB OS select screen and other GRUB output The full output dumps can be seen at: https://people.linaro.org/~peter.maydell/uart0.txt https://people.linaro.org/~peter.maydell/uart1.txt With only 1 UART, all the above appears on the single UART: https://people.linaro.org/~peter.maydell/uart-single.txt If I change QEMU to reverse the order of the nodes in the DTB (so the pl011@9040000 nodes is listed first in the dtc output, and pl011@900000 is listed second), then EDK2's output changes: the debug output previously on uart0 is now on uart1, and vice-versa. The GRUB output also switches to uart0. The Linux kernel output remains on uart0 (this makes sense, because Linux is looking at the ACPI tables, which are generated independently from the dtb). Output for this setup is here: https://people.linaro.org/~peter.maydell/uart0-rev.txt https://people.linaro.org/~peter.maydell/uart1-rev.txt A direct boot of Linux doesn't care about the dtb node ordering -- it honours the aliases node and the chosen stdout-path string. (Without the 'aliases' node, only the "pl011@9000000 first" dtb order works, because it assigns ttyAMA0 and ttyAMA1 in the same order as dtc prints them in the dtb disassembly.) I would be happier if I understood why putting the nodes in reverse order works, given that the code in EDK2 seems to be iterating through the dtb forwards. I know there is at least one place in QEMU where the node ordering gets reversed in the process of writing out the dtb, so maybe there are more depending on how exactly the dtb is read. That would I suppose explain why some EDK2 debug output goes to one UART and some to the other, if the dtb read process differs during different phases of EDK2 boot. If you want to play around with this, I have some WIP patches at https://git.linaro.org/people/pmaydell/qemu-arm.git uart-edk-investigation (content wise they should be fine, but I haven't cleaned them up into a coherent set of distinct patches yet, so they're a bit messy.) A run of QEMU with both UARTs which sends all output to files looks like: ./build/arm-clang/qemu-system-aarch64 -display none -vga none \ -machine virt,acpi=on,virtualization=on,mte=on,gic-version=max,iommu=smmuv3 \ -smp 2 -m 1024 -cpu max,pauth-impdef=on \ -bios ~/linaro/edk2/QEMU_EFI_DEBUG.fd \ -drive file=/home/petmay01/avocado/data/cache/by_location/0154b7cd3a4f5e135299060c8cabbeec10b70b6d/alpine-standard-3.17.2-aarch64.iso,format=raw \ -device virtio-rng-pci,rng=rng0 \ -object rng-random,id=rng0,filename=/dev/urandom \ -chardev file,id=chr0,path=/tmp/uart0-rev.txt \ -chardev file,id=chr1,path=/tmp/uart1-rev.txt \ -serial chardev:chr0 -serial chardev:chr1 (adjust -serial options to taste) thanks -- PMM -=-=-=-=-=-=-=-=-=-=-=- Groups.io Links: You receive all messages sent to this group. View/Reply Online (#108959): https://edk2.groups.io/g/devel/message/108959 Mute This Topic: https://groups.io/mt/101498371/7686176 Group Owner: devel+owner@edk2.groups.io Unsubscribe: https://edk2.groups.io/g/devel/unsub [rebecca@openfw.io] -=-=-=-=-=-=-=-=-=-=-=-