Re: [edk2-devel] Additional configuration options on Armada/Cn913x

[greg@unrelenting.technology]

June 11, 2020 5:17 PM, "Ard Biesheuvel" <ard.biesheuvel@arm.com> wrote:

> On 6/11/20 4:07 PM, greg@unrelenting.technology wrote:
> 
>> June 11, 2020 4:19 PM, "Ard Biesheuvel" <ard.biesheuvel@arm.com> wrote:
>> On 6/5/20 5:19 PM, Marcin Wojtas via groups.io wrote:
>>> Disabling ECAM shift is just a matter of exposing the iATU controls to the OS, right? Why do you
>>> need to disable it?
>> 
>> I'm not sure what iATU controls are (and we don't want to do anything in the OS),
>> but basically the current address is shifted by 0x8000 to only expose the last device to the OS,
>> to work around the silicon bug (lack of some filtering thing) that causes devices to appear many
>> many times.
>> But actually most modern devices (e.g. AMD RX 480, Mellanox CX2) *do not* get duplicated at all,
>> they show up
>> in the first position, and this shift moves the memory way past that position and the OS sees
>> no PCIe devices at all. The only device that was duplicated into all slots for me was a cheap SATA
>> card.
>> In my experience whether the device is duplicated seems to correlate with the "Legacy" field
>> in the UEFI Shell's pci command. IIRC Marcin has explained the actual technical characteristic
>> of these devices in some mail before. So it might actually be possible to decide whether to do the
>> shift
>> automatically at runtime depending on the inserted device (?)
>> But a setting in the setup menu is easier to do and less magical.
>> I've just been running with the shift reverted:
>> https://github.com/myfreeweb/edk2-platforms/commit/36395be2a8707f6d396e07405eb9fe47b64cf964
>> to make my Radeon GPU work.
> 
> OK, the shift is definitely a hack, and your assertion that 'most modern devices do not get
> duplicated at all' does not match my experience, tbh.
> Are these all devices that support ARI by any chance?

Mellanox (ConnectX-2) and Intel (82599ES and good old 82576) NICs do.

But AMD GPUs actually don't! Still the RX480 (POLARIS10) only works without the shift.

An older GPU (HD7970 I think) I've tried once was duplicated, but only literally duplicated (into two),
not into *all* slots.

> The problem is that the PCIe IP is truly broken, and the lack of a root port means that TLPs get
> emitted that should never reach the device in the first place, and it is not the job of the device
> to filter these TLPs, or reason about their own device # in the first place.

Well, it must be that AMD Radeons do reason about this :)
and that sadly breaks the offset hack.