Re: [edk2-platforms: PATCH v2 00/14] Armada7k8k PCIE support

["Ard Biesheuvel" <ard.biesheuvel@linaro.org>]

On Sat, 25 May 2019 at 00:16, Mark Kettenis <mark.kettenis@xs4all.nl> wrote:
>
> > From: Ard Biesheuvel <ard.biesheuvel@linaro.org>
> > Date: Thu, 23 May 2019 19:13:20 +0100
> > Cc: Leif Lindholm <leif.lindholm@linaro.org>, Marcin Wojtas <mw@semihalf.com>,
> >         edk2-devel-groups-io <devel@edk2.groups.io>,
> >         Jan Dąbroś <jsd@semihalf.com>,
> >         Grzegorz Jaszczyk <jaz@semihalf.com>,
> >         Kostya Porotchkin <kostap@marvell.com>, Jici Gao <Jici.Gao@arm.com>,
> >         Rebecca Cran <rebecca@bluestop.org>, kettenis@openbsd.org
> > Content-Type: text/plain; charset="UTF-8"
> >
> > On Thu, 23 May 2019 at 19:02, Mark Kettenis <kettenis@jive.eu> wrote:
> > >
> > > > Date: Thu, 23 May 2019 15:14:27 +0100
> > > > From: Leif Lindholm <leif.lindholm@linaro.org>
> > > >
> > > > On Thu, May 23, 2019 at 03:27:47PM +0200, Mark Kettenis wrote:
> > > > > > From: Marcin Wojtas <mw@semihalf.com>
> > > > > > Date: Mon, 20 May 2019 17:27:13 +0200
> > > > > >
> > > > > > Hi,
> > > > > >
> > > > > > Thank you for thorough review of v1. I submit second
> > > > > > version of the Armada7k8k PCIE support. I addressed
> > > > > > all comments. There is no functional change to initial
> > > > > > patchset, but mostly clean-up and improvements - please
> > > > > > refer to the changelog below.
> > > > > >
> > > > > > The patches are available in the github:
> > > > > > https://github.com/MarvellEmbeddedProcessors/edk2-open-platform/commits/pcie-upstream-r20190520
> > > > > >
> > > > > > I'm looking forward to your comments or remarks.
> > > > > >
> > > > > > Best regards,
> > > > > > Marcin
> > > > >
> > > > > Tested this on my mcbin running OpenBSD.  It incovers a small issue in
> > > > > our kernel which I'm fixing.  Otherwise this seems to work fine.
> > > > >
> > > > > So tested-by: Mark Kettenis <kettenis@openbsd.org> if that matters.
> > > >
> > > > Always helpful, thanks.
> > > >
> > > > Out of interest, what was the issue?
> > > > Could I still expect the 6.5 installer to run on this
> > > > hardware/firmware combo?
> > >
> > > That should still work fine.
> > >
> > > The issue is with mapping PCI ROMS, which currently fails with the new
> > > firmware due to an oversight in the OpenBSD code.  There are only a
> > > few OpenBSD drivers that attempt to map the PCI ROM.  But one of those
> > > is radeondrm(4) and I stuck an AMD graphics card into the PCIe slot on
> > > my machine.
> > >
> > > There is a bit of a firmware angle to this though.  The issue happens
> > > because the PCI ROM address register has been set to 0xfffe0000.  All
> > > the writable address bits in the register are set to 1.  While it is
> > > possible that the hardware comes up in that state, I suspect this is
> > > done by an attempt by the firmware to determine the size of the ROM
> > > that doesn't properly restore the original contents of the register.
> >
> > UEFI deliberately leaves the ROM BARs unassigned, in order to avoid
> > wasting valuable 32-bit PCI MMIO space. It enables the BAR temporarily
> > to load the ROM, and dispatches it if it can. After that, it changes
> > the BAR back to the old unassigned value.
>
> It seems that that last bit isn't working...
>
> > > It may be related to the following messages that are printed by the
> > > firmware:
> > >
> > > Image type X64 can't be loaded on AARCH64 UEFI system.
> > > Unloading driver at 0x00000000000
> >
> > That address value is quite unexpected.
> >
> > > Connect: PcieRoot(0x0)/Pci(0x0,0x0): Not Found
> > >
> > > Hmm, that's actually interesting.  Maybe I should play with the X86
> > > emulator that Ard added recently to see if that gives me a framebuffer
> > > console.
> > >
> >
> > Yes, if you add edk2-staging to your PACKAGES_PATH env variable, all
> > you need to do is add -D X64EMU_ENABLE to the build command line.
> >
> > If you do end up testing this, could you please report back with the
> > result, i.e., the type of card, PCI PID/VID and whether it all worked
> > as expected? Thanks.
>
> Seems to work fine.  The EFI shell and OpenBSD bootloader prompt show
> up on both the framebuffer and the serial console.

Thanks for the report.

...

> X11 works to some extent, but there are artifacts.  Half the
> characters in my xterms are missing, and there are some random
> characters in places where they shouldn't be.  That may very well be
> an OpenBSD issue though.  Strangely enough 3D acceleration seems to
> work fine.
>

This smells like a DMA coherency issue. I recently disabled an
optimization in the linux version of the radeon/amdgpu drivers that
resulted in similar issues:
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=e02f5c1bb2283cfcee68f2f0feddcc06150f13aa