From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from out0.migadu.com (out0.migadu.com [94.23.1.103]) by mx.groups.io with SMTP id smtpd.web10.8725.1591886042136844786 for ; Thu, 11 Jun 2020 07:34:02 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=@unrelenting.technology header.s=default header.b=DdmlvzvA; spf=pass (domain: unrelenting.technology, ip: 94.23.1.103, mailfrom: greg@unrelenting.technology) MIME-Version: 1.0 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=unrelenting.technology; s=default; t=1591886039; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=oliOGwUlcwKpfn5Odn6ejy0eRcfEptPOzYgQQruQ3c4=; b=DdmlvzvANc4nGtB+/r7S7EN5shwh6ry3evEb+3JAPViwaz52csPWcp9LyGUHAkCIaJfCp/ VXy4VpdSWRA+E3jbmuHcijcpcpWXmJmSKO7uKNLA3ZzKbWEIo9ndzVbh43J3rePOXDo/Ls C9ygMQreDVmrTn+nnh3WlD1yq2N+3+k= Date: Thu, 11 Jun 2020 14:33:59 +0000 X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. From: greg@unrelenting.technology Message-ID: <38e05018a4b41becb9d6a37f9defef38@unrelenting.technology> Subject: Re: [edk2-devel] Additional configuration options on Armada/Cn913x To: "Ard Biesheuvel" , devel@edk2.groups.io, mw@semihalf.com Cc: "Leif Lindholm" In-Reply-To: <7860963d-c540-1577-600c-106d7236f921@arm.com> References: <7860963d-c540-1577-600c-106d7236f921@arm.com> <05412f93-d4ca-3ae8-744f-6ba7f1ce7e8e@arm.com> X-Spam-Score: -0.10 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable June 11, 2020 5:17 PM, "Ard Biesheuvel" wrote:= =0A=0A> On 6/11/20 4:07 PM, greg@unrelenting.technology wrote:=0A> =0A>> = June 11, 2020 4:19 PM, "Ard Biesheuvel" wrote:= =0A>> On 6/5/20 5:19 PM, Marcin Wojtas via groups.io wrote:=0A>>> Disabli= ng ECAM shift is just a matter of exposing the iATU controls to the OS, r= ight? Why do you=0A>>> need to disable it?=0A>> =0A>> I'm not sure what i= ATU controls are (and we don't want to do anything in the OS),=0A>> but b= asically the current address is shifted by 0x8000 to only expose the last= device to the OS,=0A>> to work around the silicon bug (lack of some filt= ering thing) that causes devices to appear many=0A>> many times.=0A>> But= actually most modern devices (e.g. AMD RX 480, Mellanox CX2) *do not* ge= t duplicated at all,=0A>> they show up=0A>> in the first position, and th= is shift moves the memory way past that position and the OS sees=0A>> no = PCIe devices at all. The only device that was duplicated into all slots f= or me was a cheap SATA=0A>> card.=0A>> In my experience whether the devic= e is duplicated seems to correlate with the "Legacy" field=0A>> in the UE= FI Shell's pci command. IIRC Marcin has explained the actual technical ch= aracteristic=0A>> of these devices in some mail before. So it might actua= lly be possible to decide whether to do the=0A>> shift=0A>> automatically= at runtime depending on the inserted device (?)=0A>> But a setting in th= e setup menu is easier to do and less magical.=0A>> I've just been runnin= g with the shift reverted:=0A>> https://github.com/myfreeweb/edk2-platfor= ms/commit/36395be2a8707f6d396e07405eb9fe47b64cf964=0A>> to make my Radeon= GPU work.=0A> =0A> OK, the shift is definitely a hack, and your assertio= n that 'most modern devices do not get=0A> duplicated at all' does not ma= tch my experience, tbh.=0A> Are these all devices that support ARI by any= chance?=0A=0AMellanox (ConnectX-2) and Intel (82599ES and good old 82576= ) NICs do.=0A=0ABut AMD GPUs actually don't! Still the RX480 (POLARIS10) = only works without the shift.=0A=0AAn older GPU (HD7970 I think) I've tri= ed once was duplicated, but only literally duplicated (into two),=0Anot i= nto *all* slots.=0A=0A> The problem is that the PCIe IP is truly broken, = and the lack of a root port means that TLPs get=0A> emitted that should n= ever reach the device in the first place, and it is not the job of the de= vice=0A> to filter these TLPs, or reason about their own device # in the = first place.=0A=0AWell, it must be that AMD Radeons do reason about this = :)=0Aand that sadly breaks the offset hack.