Re: [PATCH v4 edk2-platforms 20/23] Platform/Raspberry/Pi3: Add platform readme

[Pete Batard <pete@akeo.ie>]

Hi Leif. Thanks for reviewing this patchset.

On 2019.01.30 21:50, Leif Lindholm wrote:
> Hi Pete,
> 
> I will only have minor comments on this set, but I'll start with this
> documentation.
> 
> On Tue, Jan 29, 2019 at 04:26:52PM +0000, Pete Batard wrote:
>> Contributed-under: TianoCore Contribution Agreement 1.1
>> Signed-off-by: Pete Batard <pete@akeo.ie>
>> ---
>>   Platform/Raspberry/Pi3/Readme.md | 259 ++++++++++++++++++++
>>   Readme.md                        |   3 +
>>   2 files changed, 262 insertions(+)
>>
>> diff --git a/Platform/Raspberry/Pi3/Readme.md b/Platform/Raspberry/Pi3/Readme.md
>> new file mode 100644
>> index 000000000000..7fb59ccdc321
>> --- /dev/null
>> +++ b/Platform/Raspberry/Pi3/Readme.md
>> @@ -0,0 +1,259 @@
>> +Raspberry Pi 3 EDK2 Platform Support
>> +====================================
>> +
>> +# Summary
>> +
>> +This is a port of 64-bit Tiano Core UEFI firmware for the Raspberry Pi 3/3B+ platforms,
>> +based on [Ard Bisheuvel's 64-bit](http://www.workofard.com/2017/02/uefi-on-the-pi/)
>> +and [Microsoft's 32-bit](https://github.com/ms-iot/RPi-UEFI/tree/ms-iot/Pi3BoardPkg)
>> +implementations, as maintained by [Andrei Warkentin](https://github.com/andreiw/RaspberryPiPkg).
>> +
>> +This is meant as a generally useful 64-bit ATF + UEFI implementation for the Raspberry
>> +Pi 3/3B+ which should be good enough for most kind of UEFI development, as well as for
>> +running consummer Operating Systems in such as Linux or Windows.
>> +
>> +Raspberry Pi is a trademark of the [Raspberry Pi Foundation](http://www.raspberrypi.org).
>> +
>> +# Status
>> +
>> +This firmware, that has been validated to compile against the current
>> +[edk2](https://github.com/tianocore/edk2)/[edk2-platforms](https://github.com/tianocore/edk2-platforms),
>> +should be able to boot Linux (SUSE, Ubuntu), NetBSD, FreeBSD as well as Windows 10 ARM64
>> +(full GUI version).
>> +
>> +It also provides support for ATF ([Arm Trusted Platform](https://github.com/ARM-software/arm-trusted-firmware)).
>> +
>> +HDMI and the mini-UART serial port can be used for output devices, with mirrored output.
>> +USB keyboards and the mini-UART serial port can be used as input.
>> +
>> +The boot order is currently hardcoded, first to the USB ports and then to the uSD card.
>> +If there no bootable media media is found, the UEFI Shell is launched.
>> +<kbd>Esc</kbd> enters platform setup. <kbd>F1</kbd> boots the UEFI Shell.
>> +
>> +# Building
>> +
>> +(These instructions were validated against the latest edk2 / edk2-platforms /
>> +edk2-non-osi as of 2019.01.27, on a Debian 9.6 x64 system).
>> +
>> +You may need to install the relevant compilation tools. Especially you should have the
>> +ACPI Source Language (ASL) compiler, `nasm` as well as a native compiler installed.
> 
> nasm? The x86 assembler?

I'll remove that.

>> +On a Debian system, you can get these prerequisites installed with:
>> +```
>> +sudo apt-get install build-essential acpica-tools nasm uuid-dev
>> +```
>> +
>> +**IMPORTANT:** We recommend the use of the Linaro GCC for compilation instead of
>> +your system's native ARM64 GCC cross compiler.
> 
> This sounds like something written in the days of GCC 4.8. I doubt it
> has any relevance today.

It very much had until circa one month ago, as we observed early 
Synchronous Exceptions when trying to use the native Debian ARM64 
compiler, which we did not observe with Linaro's toolchain. We even had 
trouble (similar issue) with recent Linaro toolchains at some stage, 
which is why, until v3, we recommended an older version, but recent 
tests showed that the latest Linaro GCC (2019.02) appeared to be okay, 
which is why I removed the previous requirement to use exclusively 
Linaro's 2017.10 GCC 5.5.

Besides, I think it's preferable when a project highlights precisely how 
they build their own binaries, and with which toolchain, so that, if 
anybody experiences an issue with their own build, they can compare 
their setup with the maintainer's "official" one.

> Also, and this applies both above and below: I am trying very hard to
> get rid of (mostly unnecessary) platform-specific build instructions.
> The top-level Readme.md in this repository contains basic build
> instructions. I would much prefer if you can refer to that instead and
> drop everything after the # Building header above...

Okay. This is a bit at odds with the goal I'm trying to achieve here, 
which is to save time and bewilderment from developers who might be 
trying to build this specific platform and encounter an issue where 
they'll want to eliminate the possibility that their setup/configuration 
is the problem.

I've ran in too many of "works on my machine" not top want to also 
provide "here is exactly how the official developer's machine was set up 
when they ran their built" from the get go, to try to alleviate the 
usual headaches of trying to solve environmental issues... Furthermore, 
given the popularity of the Raspberry Pi platform, my guess is that 
we're going to get quite a few people who aren't that familiar with the 
EDK2, or even building things, in general and who'll want a set of 
"copy/paste exactly this in your shell and you *should* end up with a 
firmware binary" set of instructions, which is why I've attempted to 
provide that, in a single location.

Now, I agree that this may run contrary to what you (and possibly other 
people) want, so I'm not going to push the matter further if you think 
having a set of duplicate "hand holding" set of build instructions in 
this readme is uncalled for. But I'd rather take pre-emptive steps to 
avoid having to deal with a potentially large number of "I tried to 
build and it didn't work" requests that might come from providing 
firmware builds for a very popular platform...

>> +
>> +You can then build the firmware as follows:
>> +
>> +* Standalone instructions
>> +
>> +```
>> +mkdir ~/workspace
>> +cd ~/workspace
>> +git clone https://github.com/tianocore/edk2.git
>> +# The following is only needed once, after you cloned edk2
>> +make -C edk2/BaseTools
>> +git clone https://github.com/tianocore/edk2-platforms.git
>> +git clone https://github.com/tianocore/edk2-non-osi.git
>> +wget https://releases.linaro.org/components/toolchain/binaries/7.4-2019.02/aarch64-linux-gnu/gcc-linaro-7.4.1-2019.02-x86_64_aarch64-linux-gnu.tar.xz
>> +tar -xJvf gcc-linaro-7.4.1-2019.02-x86_64_aarch64-linux-gnu.tar.xz
>> +# If you have multiple AARCH64 toolchains, make sure the above one comes first in your path
>> +export PATH=$PWD/gcc-linaro-7.4.1-2019.02-x86_64_aarch64-linux-gnu/bin:$PATH
>> +export GCC5_AARCH64_PREFIX=aarch64-linux-gnu-
>> +export WORKSPACE=$PWD
>> +export PACKAGES_PATH=$WORKSPACE/edk2:$WORKSPACE/edk2-platforms:$WORKSPACE/edk2-non-osi
>> +. edk2/edksetup.sh
> 
> ...down to here.
> (I would certainly not object if you felt the need to improve on the
> contents of the top-level Readme.md. For example with the explicit
> prerequisite installation steps, rather than the brief list provided
> under the Prerequisites section currently on that page.)
> 
>> +build -a AARCH64 -t GCC5 -p edk2-platforms/Platform/Raspberry/Pi3/RPi3.dsc -DBUILD_EPOCH=`date +%s` -b RELEASE
>> +```
>> +
> 
> The below _is_ platform-specific, so clearly I have no issue with
> it. Not sure that needed poinging out, but...
> And the above line points out the location of the .dsc, so no
> objection there.
> 
>> +# Booting the firmware
>> +
>> +1. Format a uSD card as FAT32
>> +2. Copy the generated `RPI_EFI.fd` firmware onto the partition
>> +3. Download and copy the following files from https://github.com/raspberrypi/firmware/tree/master/boot
>> +  - `bootcode.bin`
>> +  - `fixup.dat`
>> +  - `start.elf`
>> +4. Create a `config.txt` with the following content:
>> +  ```
>> +  arm_control=0x200
>> +  enable_uart=1
>> +  armstub=RPI_EFI.fd
>> +  disable_commandline_tags=1
>> +  ```
>> +5. Insert the uSD card and power up the Pi.
>> +
>> +Note that if you have a model 3+ or a model 3 where you enabled USB boot through OTP
>> +(see [here](https://www.raspberrypi.org/documentation/hardware/raspberrypi/bootmodes/msd.md))
>> +you may also be able to boot from a FAT32 USB driver rather than uSD.
>> +
>> +# Notes
>> +
>> +## ARM Trusted Firmware (ATF)
>> +
>> +The ATF binaries being used were compiled from the ATF mainline.
> 
> With additional modifications?

No. It would have been mentioned otherwise.

>> +
>> +For more details on the ATF compilation, see the [README](./Binary/README.md) in the `Binary/` directory.
> 
> This directory no longer exists.
> You could point at the full URL in the edk2-non-osi repository.

Yes, I will do that, since it contains important data about the 
compilation options that were used for ATF. Ultimately though, unless 
you changed your mind (since I believe this is something you wanted), we 
want to remove the ATF binaries in non-osi and simply point to build 
instructions (as well as links to automated built binaries from AppVeyor 
or something similar, for people who don't want to have to go through 
the extra build step). I am still waiting on a new dot release of ATF to 
work on that.

>> +
>> +## Custom Device Tree
>> +
>> +The default Device Tree included in the firmware is the one for a Raspberry Pi 3 Model B (not B+).
>> +If you want to use a different Device Tree, to boot a Pi 3 Model B+ for instance (for which a
>> +DTB is also provided under `DeviceTree/`), you should copy the relevant `.dtb` into the root of
>> +the SD or USB, and then edit your `config.txt` so that it looks like:
>> +
>> +```
>> +(...)
>> +disable_commandline_tags=2
>> +device_tree_address=0x10000
>> +device_tree_end=0x20000
>> +device_tree=bcm2710-rpi-3-b-plus.dtb
>> +```
>> +
>> +Note: the address range **must** be `[0x10000:0x20000]`.
>> +`dtoverlay` and `dtparam` parameters are also supported **when** providing a Device Tree`.
>> +
>> +## Custom `bootargs`
>> +
>> +This firmware will honor the command line passed by the GPU via `cmdline.txt`.
>> +
>> +Note, that the ultimate contents of `/chosen/bootargs` are a combination of several pieces:
>> +- Original `/chosen/bootargs` if using the internal DTB. Seems to be completely discarded by GPU when booting with a custom device tree.
>> +- GPU-passed hardware configuration. This one is always present.
>> +- Additional boot options passed via `cmdline.txt`.
>> +
>> +# Tested Platforms
> 
> I won't press the issue, but I would prefer for the operating system
> installation instructions not to be included.

Okay. My reasoning here is similar to the build instructions: As opposed 
to what might be the case for other platforms, due to the Raspberry Pi's 
popularity, a lot of people might be coming to this readme for help on 
how to use the firmware to install an OS and they may start to spam the 
list or package maintainers if they don't find the information they 
want. So we want to ensure we provide at least one good walk-through 
example that people can refer to.

> If you do want it in this repo, please break it out into a separate
> Readme.md, which you can link to from here.

I'll do that then, as I believe it's important to have at least getting 
to a fully configured Linux installation documented, especially this 
early in the game. Once relying on this firmware (hopefully) starts to 
become the norm for popular Linux distros, having these instructions 
should be a lot less important.

> Basically, skp straight to "Limitations" from here.
> 
>> +
>> +## Ubuntu
>> +
>> +[Ubuntu 18.04 LTS](http://releases.ubuntu.com/18.04/) has been tested and confirmed to work,
>> +on a Raspberry 3 Model B, including the installation process. Note however that network
>> +installation and networking may not work on the Model B+, due to the `lan78xx` Linux driver
>> +still requiring some support.
>> +
>> +Below are the steps you can follow to install Ubuntu LTS onto SD/USB:
>> +* Download the latest Ubuntu LTS ARM64 [`mini.iso`](http://ports.ubuntu.com/ubuntu-ports/dists/bionic/main/installer-arm64/current/images/netboot/mini.iso).
>> +* Partition the media as MBR and create a ~200 MB FAT32 partition on it with MBR type `0x0c`.
>> +  Note: Do not be tempted to use GPT partition scheme or `0xef` (EFI System Partition) for the
>> +  type, as none of these are supported by the Raspberry Pi's internal boot rom.
>> +* Extract the full content of the ISO onto the partition you created.
>> +* Also extract the GRUB EFI bootloader `bootaa64.efi` from `/boot/grub/efi.img` to `/boot/grub/`.
>> +  Note: Do not be tempted to copy this file to another directory (such as `/efi/boot/`) as GRUB looks for its
>> +  modules and configuration data in the same directory as the EFI loader and also, the installation
>> +  process will create a `bootaa64.efi` into `/efi/boot/`.
>> +* If needed, copy the UEFI firmware files (`RPI_EFI.fd`, `bootcode.bin`, `fixup.dat` and `start.elf`)
>> +  onto the FAT partition.
>> +* Boot the pi and let it go into the UEFI shell.
>> +* Navigate to `fs0:` then `/boot/grub/` and launch the GRUB efi loader.
>> +* Follow the Ubuntu installation process.
>> +
>> +Note: Because Ubuntu operates in quiet mode by default (no boot messages), you may think the system is frozen
>> +on first reboot after installation. However, if you wait long enough you **will** get to a login prompt.
>> +
>> +Once Linux is running, if desired, you can disable quiet boot, as well as force the display
>> +of the GRUB selector, by editing `/etc/default/grub` and changing:
>> +* `GRUB_TIMEOUT_STYLE=hidden` &rarr; `GRUB_TIMEOUT_STYLE=menu`
>> +* `GRUB_CMDLINE_LINUX_DEFAULT="splash quiet"` &rarr; `GRUB_CMDLINE_LINUX_DEFAULT=""`
>> +
>> +Then, to have your changes applied run `update-grub` and reboot.
>> +
>> +## Other Linux distributions
>> +
>> +* Debian ARM64 does not currently work, most likely due to missing required module support
>> +  in its kernel. However its installation process works, so it may be possible to get it
>> +  running with a custom kernel.
>> +
>> +* OpenSUSE Leap 42.3 has been reported to work on Raspberry 3 Model B.
>> +
>> +* Other ARM64 Linux releases, that support UEFI boot and have the required hardware support
>> +  for Pi hardware are expected to run, though their installation process might require some
>> +  cajoling.
>> +
>> +## Windows
>> +
>> +Windows 10 1809 for ARM64 (build 17763) has been tested and confirmed to work (after replacing
>> +`C:\Windows\System32\Drivers\WppRecorder.sys` with an older version, since the one from 1809
>> +appears to be buggy across all archs, and results in a similar BSOD when trying to run Windows
>> +To Go on x64 with native drivers for instance).
>> +
>> +Windows 10 1803 for ARM64 and earlier do not work due to the presence of a hardware ASSERT check
>> +in the Windows kernel, that was removed in later versions.
>> +
>> +You probably want to look at https://www.worproject.ml/ as well as the
>> +[Windows thread in the original RaspberryPiPkg](https://github.com/andreiw/RaspberryPiPkg/issues/12)
>> +for installation details.
>> +
>> +## Other platforms
>> +
>> +Details you may need to run other platforms, including FreeBSD, is provided in the
>> +[Readme from the original RaspberryPiPkg](https://github.com/andreiw/RaspberryPiPkg).
>> +
>> +# Limitations
>> +
>> +## HDMI
>> +
>> +The UEFI HDMI video support relies on the VC (that's the GPU)
>> +firmware to correctly detect and configure the attached screen.
>> +Some screens are slow, and this detection may not occur fast
>> +enough. Finally, you may wish to be able to boot your Pi
>> +headless, yet be able to attach a display to it later for
>> +debugging.
>> +
>> +To accommodate these issues, the following extra lines
>> +are recommended for your `config.txt`:
>> +- `hdmi_force_hotplug=1` to allow plugging in video after system is booted.
>> +- `hdmi_group=1` and `hdmi_mode=4` to force a specific mode, both to accommodate
>> +   late-plugged screens or buggy/slow screens. See [official documentation](https://www.raspberrypi.org/documentation/configuration/config-txt/video.md)
>> +   to make sense of these parameters (example above sets up 720p 60Hz).
>> +
>> +## NVRAM
>> +
>> +The Raspberry Pi has no NVRAM.
>> +
>> +NVRAM is emulated, with the non-volatile store backed by the UEFI image itself. This means
>> +that any changes made in UEFI proper will be persisted, but changes made in HLOS will not.
>> +It would be nice to implement ATF-assisted warm reboot, to allow persisting HLOS
>> +NVRAM changes.
>> +
>> +## RTC
>> +
>> +The Rasberry Pi has no RTC.
>> +
>> +`RtcEpochSeconds` NVRAM variable is used to store the boot time
>> +This should allow you to set whatever date/time you
>> +want using the Shell date and time commands. While in UEFI
>> +or HLOS, the time will tick forward. `RtcEpochSeconds`
>> +is not updated on reboots.
>> +
>> +## uSD
>> +
>> +UEFI supports both the Arasan SDHCI and the Broadcom SDHost controllers to access the uSD slot.
>> +You can use either. The other controller gets routed to the SDIO card. The choice made will
>> +impact ACPI OSes booted (e.g. Windows 10). Arasan, being an SDIO controller, is usually used
>> +with the WiFi adapter where available. SDHost cannot be used with SDIO. In UEFI setup screen:
>> +- go to `Device Manager`
>> +- go to `Raspberry Pi Configuration`
>> +- go to `Chipset`
>> +- configure `Boot uSD Routing`
>> +
>> +Known issues:
>> +- Arasan HS/4bit support is missing.
>> +- No 8 bit mode support for (e)MMC (irrelevant for the Pi 3).
>> +- Hacky (e)MMC support (no HS).
>> +- No card removal/replacement detection, tons of timeouts and slow down during boot without an uSD card present.
>> +
>> +## USB
>> +
>> +- USB1 BBB mass storage devices untested (USB2 and USB3 devices are fine).
>> +- USB1 CBI mass storage devices don't work (e.g. HP FD-05PUB floppy).
>> +
>> +## ACPI
>> +
>> +ACPI should match the MS-IoT one. Both Arasan and SDHost SD controllers are exposed.
> 
> It would be good if you could add a comment here about the limitations
> with regards to proper ACPI description and its unusability for Linux.

Will do.

>> +
>> +## Missing Functionality
>> +
>> +- Network booting via onboard NIC.
>> +- Ability to switch UART use to PL011.
>> diff --git a/Readme.md b/Readme.md
>> index 384b1d3c5e2b..d82b7581ba6d 100644
>> --- a/Readme.md
>> +++ b/Readme.md
>> @@ -217,6 +217,9 @@ they will be documented with the platform.
>>   ## Marvell
>>   * [Armada 70x0](Platform/Marvell/Armada)
>>   
>> +## Raspberry
> 
> Raspberry Pi

Yes, that makes sense with your further note.

> /
>      Leif
> 
>> +* [Pi 3](Platform/Raspberry/Pi3)
>> +
>>   ## Socionext
>>   * [SynQuacer](Platform/Socionext/DeveloperBox)
>>   
>> -- 
>> 2.17.0.windows.1
>>