From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received-SPF: Pass (sender SPF authorized) identity=mailfrom; client-ip=2607:f8b0:4001:c0b::235; helo=mail-it0-x235.google.com; envelope-from=ard.biesheuvel@linaro.org; receiver=edk2-devel@lists.01.org Received: from mail-it0-x235.google.com (mail-it0-x235.google.com [IPv6:2607:f8b0:4001:c0b::235]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ml01.01.org (Postfix) with ESMTPS id 86BEF22198F41 for ; Thu, 21 Dec 2017 01:47:39 -0800 (PST) Received: by mail-it0-x235.google.com with SMTP id u62so10022400ita.2 for ; Thu, 21 Dec 2017 01:52:27 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=cDTzOihUkKCJcCz+yUSgU8JxrYd3Vndk8U7299THqDo=; b=V/DRsip5Vtbc99D7qCHFgl29+iyoJbNAFcf03rsAe4TADNYq8s/eBa4HRptsOjgGoF DU/tpmdMRrIIZJlMSm4nff9oB82RmgC6uokM0icR+fwXI4Bqed7wllGO3OXfj9Buos0s Lbs7CKuf9sRbXo6zcXTRSRYMJ7Q6fYC6dPpgQ= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=cDTzOihUkKCJcCz+yUSgU8JxrYd3Vndk8U7299THqDo=; b=Q/eoAzPQAIGPKJ+zCGcKmwHUn0UXPyQx5NXgymzP8S869BCE9npPOXS58c4l3EJz7B GOmeWcyi/hQCkNRduFV3ebik1dbpJW2JxQ4T8rAQPp8gD407tDmVNeWZbLSJ+RXJtkJl LKdQfpLCoG3Zl/NLlqieRhVPF8cGU9J51iwYngRtdQv+X/pjhnGGFKhnowuF5CHWixkT fdUPPL7YVeQOu8yOsw5s4MeLeqwmqszPwrdX7G/xfZweBP+T/FyfAZAVeaBzLSjUWHVb wrdfMuQNRSxaUeHhT+geBsNbVCzJt9Kp2Yr8TW7BaHkkeBZj5QcynSU0vSDaTGS2PM4y Bh8Q== X-Gm-Message-State: AKGB3mKnW5+l5lkQ8x79ENrEBVhlZoJ0EBK7vl/JawfISoabQWVoJPgY 8/tupJ3gXrrAqXcr7h/nOwmWwarIUvE8eWbWnctzQg== X-Google-Smtp-Source: ACJfBosz1ztjOadPVzqB+qh6pPvGgHGs7RkeC7GTayD++VLTeCxzsYWMUZ8uH6I0QvHmRJQ7HpMZkE4XS25e5j5dtBI= X-Received: by 10.36.55.138 with SMTP id r132mr11995148itr.34.1513849947088; Thu, 21 Dec 2017 01:52:27 -0800 (PST) MIME-Version: 1.0 Received: by 10.107.52.14 with HTTP; Thu, 21 Dec 2017 01:52:26 -0800 (PST) In-Reply-To: <9272c2e4-cd44-b299-480a-bde9946dc3f4@Intel.com> References: <1513758078-99534-1-git-send-email-heyi.guo@linaro.org> <20171220151704.GA2482@iwish> <20171221082726.GA100076@SZX1000114654> <20171221091453.GB100076@SZX1000114654> <9272c2e4-cd44-b299-480a-bde9946dc3f4@Intel.com> From: Ard Biesheuvel Date: Thu, 21 Dec 2017 09:52:26 +0000 Message-ID: To: "Ni, Ruiyu" Cc: Guo Heyi , linaro-uefi , "edk2-devel@lists.01.org" , Star Zeng , Eric Dong , Jason Zhang Subject: Re: [RFC] MdeModulePkg/PciHostBridge: Add address translation support X-BeenThere: edk2-devel@lists.01.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: EDK II Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 21 Dec 2017 09:47:40 -0000 Content-Type: text/plain; charset="UTF-8" On 21 December 2017 at 09:48, Ni, Ruiyu wrote: > On 12/21/2017 5:14 PM, Guo Heyi wrote: >> >> On Thu, Dec 21, 2017 at 08:32:37AM +0000, Ard Biesheuvel wrote: >>> >>> On 21 December 2017 at 08:27, Guo Heyi wrote: >>>> >>>> On Wed, Dec 20, 2017 at 03:26:45PM +0000, Ard Biesheuvel wrote: >>>>> >>>>> On 20 December 2017 at 15:17, gary guo wrote: >>>>>> >>>>>> On Wed, Dec 20, 2017 at 09:13:58AM +0000, Ard Biesheuvel wrote: >>>>>>> >>>>>>> Hi Heyi, >>>>>>> >>>>>>> On 20 December 2017 at 08:21, Heyi Guo wrote: >>>>>>>> >>>>>>>> PCIe on some ARM platforms requires address translation, not only >>>>>>>> for >>>>>>>> legacy IO access, but also for 32bit memory BAR access as well. >>>>>>>> There >>>>>>>> will be "Address Translation Unit" or something similar in PCI host >>>>>>>> bridges to translation CPU address to PCI address and vice versa. So >>>>>>>> we think it may be useful to add address translation support to the >>>>>>>> generic PCI host bridge driver. >>>>>>>> >>>>>>> >>>>>>> I agree. While unusual on a PC, it is quite common on other >>>>>>> architectures to have more complex non 1:1 topologies, which >>>>>>> currently >>>>>>> require a forked PciHostBridgeDxe driver with local changes applied. >>>>>>> >>>>>>>> This RFC only contains one minor change to the definition of >>>>>>>> PciHostBridgeLib, and there certainly will be a lot of other changes >>>>>>>> to make it work, including: >>>>>>>> >>>>>>>> 1. Use CPU address for GCD space add and allocate operations, >>>>>>>> instead >>>>>>>> of PCI address; also IO space will be changed to memory space if >>>>>>>> translation exists. >>>>>>>> >>>>>>> >>>>>>> For I/O space, the translation should simply be applied to the I/O >>>>>>> range. I don't think it makes sense to use memory space here, given >>>>>>> that it is specific to architectures that lack native port I/O. >>>>>>> >>>>>> >>>>>> I made an assumption here that platforms supporting real port IO space >>>>>> do not need address translation, like IA32 and X64, and port IO >>>>>> translation implies the platform does not support real port IO space. >>>>>> >>>>> >>>>> This may be a reasonable assumption. But I still think it is better >>>>> not to encode any assumptions in the first place. >>>>> >>>>>> Indeed the assumption is not so "generic", so I'll agree if you >>>>>> recommend to support IO to IO translation as well. But I still hope to >>>>>> have IO to memory translation support in PCI host bridge driver, >>>>>> rather than in CPU IO protocol, since the faked IO space might only be >>>>>> used for PCI host bridge and we may have overlapping IO ranges for >>>>>> each host bridge, which is compatible with PCIe specification and PCIe >>>>>> ACPI description. >>>>>> >>>>> >>>>> That is fine. Under UEFI, these will translate to non-overlapping I/O >>>>> spaces in the CPU's view. Under the OS, this could be totally >>>>> different. >>>>> >>>>> >>>>> For example, >>>>> >>>>> RC0 IO 0x0000 .. 0xffff -> CPU 0x00000 .. 0x0ffff >>>>> RC1 IO 0x0000 .. 0xffff -> CPU 0x10000 .. 0x1ffff >>>>> >>>>> This is very similar to how MMIO translation works, and makes I/O >>>>> devices behind the host bridges uniquely addressable for drivers. >>>>> >>>>> For our understanding, could you share the host bridge configuration >>>>> that you are targetting? >>>> >>>> >>>> IO translation on one of our platforms is like below: >>>> >>>> PCI IO space CPU memory space >>>> 0x0000 .. 0xffff -> 0xafff0000 .. 0xafffffff >>>> (The sizes are always 0x10000 so I will omit the limit for others) >>>> 0x0000 .. 0xffff -> 0x8abff0000 >>>> 0x0000 .. 0xffff -> 0x8b7ff0000 >>>> ...... >>>> >>>> The translated addresses may be beyond 32bit address, will this >>>> violate IO space limitation? From EDK2 code, I didn't see such >>>> limitation for IO space. >>>> >>> >>> The MMIO address will not be used for I/O port addressing by the CPU. >>> The MMIO to IO translation is an implementation detail of your CpuIo2 >>> protocol implementation. >>> >>> So there will be two stacked translations, one for PCI I/O to CPU I/O, >>> and one for CPU I/O to CPU MMIO. The latter is transparent to the PCI >>> host bridge driver. >> >> >> Yes this should work. >> >> Hi Star, Eric and Ruiyu, >> >> Any comments on this RFC? > > > Let me confirm my understanding: > The PciHostBridge core driver/library interface changes only > take care of the MMIO translation. > > Heyi you will implement a special CpuIo driver in your > platform code to take care of the IO to MMIO translation. > > But let me confirm, will you need to additional translate > the MMIO (translated from IO) to another MMIO using an offset? > If yes, will you handle that translation in your CpuIo driver? > Hi Ray, The issue is that several PCIe root complexes have colliding I/O ranges: >>>> PCI IO space CPU memory space >>>> 0x0000 .. 0xffff -> 0xafff0000 .. 0xafffffff >>>> (The sizes are always 0x10000 so I will omit the limit for others) >>>> 0x0000 .. 0xffff -> 0x8abff0000 >>>> 0x0000 .. 0xffff -> 0x8b7ff0000 So the CPU view is different from the PCI view, and to create a single CPU I/O space where all I/O port ranges behind all host bridges are accessible, we need I/O translation for the CPU. This will result in an intermediate representation >>>> PCI IO space CPU IO space >>>> 0x0000 .. 0xffff -> 0x00000 .. 0x0ffff >>>> 0x0000 .. 0xffff -> 0x10000 .. 0x1ffff >>>> 0x0000 .. 0xffff -> 0x20000 .. 0x2ffff On top of that, given that ARM has no native port I/O instructions, we will need to implement MMIO to IO translation, but this can be implemented in the CpuIo2 protocol.