From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by mx.groups.io with SMTP id smtpd.web11.8371.1591885072293606475 for ; Thu, 11 Jun 2020 07:17:52 -0700 Authentication-Results: mx.groups.io; dkim=missing; spf=pass (domain: arm.com, ip: 217.140.110.172, mailfrom: ard.biesheuvel@arm.com) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 18AF11F1; Thu, 11 Jun 2020 07:17:51 -0700 (PDT) Received: from [192.168.1.69] (unknown [10.37.8.184]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 2A5FC3F6CF; Thu, 11 Jun 2020 07:17:49 -0700 (PDT) Subject: Re: [edk2-devel] Additional configuration options on Armada/Cn913x To: greg@unrelenting.technology, devel@edk2.groups.io, mw@semihalf.com Cc: Leif Lindholm References: <05412f93-d4ca-3ae8-744f-6ba7f1ce7e8e@arm.com> From: "Ard Biesheuvel" Message-ID: <7860963d-c540-1577-600c-106d7236f921@arm.com> Date: Thu, 11 Jun 2020 16:17:47 +0200 User-Agent: Mozilla/5.0 (X11; Linux aarch64; rv:68.0) Gecko/20100101 Thunderbird/68.9.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit On 6/11/20 4:07 PM, greg@unrelenting.technology wrote: > June 11, 2020 4:19 PM, "Ard Biesheuvel" wrote: > >> On 6/5/20 5:19 PM, Marcin Wojtas via groups.io wrote: >> >>> Hi, >>> I'd like to ask for comments before I develop the actual code - > currently we have 2 workarounds >>> done specifically for Linux: >>> a. ECAM shift in PCIE >>> b. SPCR address space definition >> >> What does this mean? > > The SPCR in upstream edk2 is set up to work around some Linux weirdness (?) and I have to do this: > > https://github.com/myfreeweb/edk2-platforms/commit/74ec98a6498e78d2ae6c861db88487bf75f2e1a1 > > to make it work on FreeBSD. > Surely, they can't both be correct. Marcin? >>> Both above are not needed e.g. in FreeBSD and I was requested to add > their optional disabling. >> >> Disabling ECAM shift is just a matter of exposing the iATU controls to the OS, right? Why do you >> need to disable it? > > I'm not sure what iATU controls are (and we don't want to do anything in the OS), > but basically the current address is shifted by 0x8000 to only expose the last device to the OS, > to work around the silicon bug (lack of some filtering thing) that causes devices to appear many many times. > > But actually most modern devices (e.g. AMD RX 480, Mellanox CX2) *do not* get duplicated at all, they show up > in the first position, and this shift moves the memory way past that position and the OS sees > no PCIe devices at all. The only device that was duplicated into all slots for me was a cheap SATA card. > > In my experience whether the device is duplicated seems to correlate with the "Legacy" field > in the UEFI Shell's pci command. IIRC Marcin has explained the actual technical characteristic > of these devices in some mail before. So it might actually be possible to decide whether to do the shift > automatically at runtime depending on the inserted device (?) > But a setting in the setup menu is easier to do and less magical. > > I've just been running with the shift reverted: > > https://github.com/myfreeweb/edk2-platforms/commit/36395be2a8707f6d396e07405eb9fe47b64cf964 > > to make my Radeon GPU work. > OK, the shift is definitely a hack, and your assertion that 'most modern devices do not get duplicated at all' does not match my experience, tbh. Are these all devices that support ARI by any chance? The problem is that the PCIe IP is truly broken, and the lack of a root port means that TLPs get emitted that should never reach the device in the first place, and it is not the job of the device to filter these TLPs, or reason about their own device # in the first place.