[PATCH v5 0/6] Add translation support to generic PciHostBridge

[PATCH v5 0/6] Add translation support to generic PciHostBridge

[Heyi Guo]

Patch v5 inherits the code from RFC v4; we don't restart the version number for
RFC to PATCH change.

v5:
- Patch 4/6: Modify the code according to the comments from Ray.
- Patch 1/6 and 2/6 are totally new. They add initialization for all fields of
  PCI_ROOT_BRIDGE_APERTURE temporary variables in PciHostBridgeLib instances, so
  that they will not suffer from extension of PCI_ROOT_BRIDGE_APERTURE
  structure.
- Generate a separate patch (3/6) for PciHostBridgeLib.h change. Though it is a
  prerequisite for patch 4/6, it does not change the code in PciHostBridge
  driver and won't cause any build failure or functional issue.


v4:
- Modify the code according to the comments from Ray, Laszlo and Ard (Please see
  the notes of Patch 1/3)
- Ignore translation of bus in CreateRootBridge.


v3:
- Keep definition of Translation consistent in EDKII code: Translation = device
  address - host address.
- Patch 2/2 is split into 2 patches (2/3 and 3/3).
- Refine comments and commit messages to make the code easier to understand.


v2:
Changs are made according to the discussion on the mailing list, including:

- PciRootBridgeIo->Configuration should return CPU view address, as well as
  PciIo->GetBarAttributes, and Translation Offset should be equal to PCI view
  address - CPU view address.
- Add translation offset to PCI_ROOT_BRIDGE_APERTURE structure definition.
- PciHostBridge driver internally used Base Address is still based on PCI view
  address, and translation offset = CPU view - PCI view, which follows the
  definition in ACPI, and not the same as that in UEFI spec.

Cc: Ruiyu Ni <ruiyu.ni@intel.com>
Cc: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Cc: Star Zeng <star.zeng@intel.com>
Cc: Eric Dong <eric.dong@intel.com>
Cc: Laszlo Ersek <lersek@redhat.com>
Cc: Michael D Kinney <michael.d.kinney@intel.com>
Cc: Maurice Ma <maurice.ma@intel.com>
Cc: Prince Agyeman <prince.agyeman@intel.com>
Cc: Benjamin You <benjamin.you@intel.com>
Cc: Jordan Justen <jordan.l.justen@intel.com>
Cc: Anthony Perard <anthony.perard@citrix.com>
Cc: Julien Grall <julien.grall@linaro.org>

Heyi Guo (6):
  CorebootPayloadPkg/PciHostBridgeLib: Init PCI aperture to 0
  OvmfPkg/PciHostBridgeLib: Init PCI aperture to 0
  MdeModulePkg/PciHostBridgeLib.h: add address Translation
  MdeModulePkg/PciHostBridgeDxe: Add support for address translation
  MdeModulePkg/PciBus: convert host address to device address
  MdeModulePkg/PciBus: return CPU address for GetBarAttributes

 MdeModulePkg/Bus/Pci/PciHostBridgeDxe/PciHostBridge.h              |  21 ++++
 MdeModulePkg/Bus/Pci/PciHostBridgeDxe/PciHostResource.h            |   3 +
 MdeModulePkg/Include/Library/PciHostBridgeLib.h                    |  19 +++
 CorebootPayloadPkg/Library/PciHostBridgeLib/PciHostBridgeSupport.c |   5 +
 MdeModulePkg/Bus/Pci/PciBusDxe/PciIo.c                             |  12 +-
 MdeModulePkg/Bus/Pci/PciHostBridgeDxe/PciHostBridge.c              | 129 +++++++++++++++++---
 MdeModulePkg/Bus/Pci/PciHostBridgeDxe/PciRootBridgeIo.c            | 118 ++++++++++++++++--
 OvmfPkg/Library/PciHostBridgeLib/PciHostBridgeLib.c                |   4 +
 OvmfPkg/Library/PciHostBridgeLib/XenSupport.c                      |   5 +
 9 files changed, 288 insertions(+), 28 deletions(-)

-- 
2.7.4

[PATCH v5 1/6] CorebootPayloadPkg/PciHostBridgeLib: Init PCI aperture to 0

[Heyi Guo]

Use ZeroMem to initialize all fields in temporary
PCI_ROOT_BRIDGE_APERTURE variables to zero. This is not mandatory but
helpful for future extension: when we add new fields to
PCI_ROOT_BRIDGE_APERTURE and the default value of these fields can
safely be zero, this code will not suffer from an additional change.

Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Heyi Guo <heyi.guo@linaro.org>
Cc: Maurice Ma <maurice.ma@intel.com>
Cc: Prince Agyeman <prince.agyeman@intel.com>
Cc: Benjamin You <benjamin.you@intel.com>
Cc: Ruiyu Ni <ruiyu.ni@intel.com>
Cc: Laszlo Ersek <lersek@redhat.com>
Cc: Ard Biesheuvel <ard.biesheuvel@linaro.org>
---
 CorebootPayloadPkg/Library/PciHostBridgeLib/PciHostBridgeSupport.c | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/CorebootPayloadPkg/Library/PciHostBridgeLib/PciHostBridgeSupport.c b/CorebootPayloadPkg/Library/PciHostBridgeLib/PciHostBridgeSupport.c
index 6d94ff72c956..c61609b79cce 100644
--- a/CorebootPayloadPkg/Library/PciHostBridgeLib/PciHostBridgeSupport.c
+++ b/CorebootPayloadPkg/Library/PciHostBridgeLib/PciHostBridgeSupport.c
@@ -319,6 +319,11 @@ ScanForRootBridges (
 
   *NumberOfRootBridges = 0;
   RootBridges = NULL;
+  ZeroMem (&Io, sizeof (Io));
+  ZeroMem (&Mem, sizeof (Mem));
+  ZeroMem (&MemAbove4G, sizeof (MemAbove4G));
+  ZeroMem (&PMem, sizeof (PMem));
+  ZeroMem (&PMemAbove4G, sizeof (PMemAbove4G));
 
   //
   // After scanning all the PCI devices on the PCI root bridge's primary bus,
-- 
2.7.4

Re: [PATCH v5 1/6] CorebootPayloadPkg/PciHostBridgeLib: Init PCI aperture to 0

[Ard Biesheuvel]

On 1 March 2018 at 06:57, Heyi Guo <heyi.guo@linaro.org> wrote:
> Use ZeroMem to initialize all fields in temporary
> PCI_ROOT_BRIDGE_APERTURE variables to zero. This is not mandatory but
> helpful for future extension: when we add new fields to
> PCI_ROOT_BRIDGE_APERTURE and the default value of these fields can
> safely be zero, this code will not suffer from an additional change.
>
> Contributed-under: TianoCore Contribution Agreement 1.1
> Signed-off-by: Heyi Guo <heyi.guo@linaro.org>
> Cc: Maurice Ma <maurice.ma@intel.com>
> Cc: Prince Agyeman <prince.agyeman@intel.com>
> Cc: Benjamin You <benjamin.you@intel.com>
> Cc: Ruiyu Ni <ruiyu.ni@intel.com>
> Cc: Laszlo Ersek <lersek@redhat.com>
> Cc: Ard Biesheuvel <ard.biesheuvel@linaro.org>

Reviewed-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>

Maurice, Prince, Benjamin: any comments?

> ---
>  CorebootPayloadPkg/Library/PciHostBridgeLib/PciHostBridgeSupport.c | 5 +++++
>  1 file changed, 5 insertions(+)
>
> diff --git a/CorebootPayloadPkg/Library/PciHostBridgeLib/PciHostBridgeSupport.c b/CorebootPayloadPkg/Library/PciHostBridgeLib/PciHostBridgeSupport.c
> index 6d94ff72c956..c61609b79cce 100644
> --- a/CorebootPayloadPkg/Library/PciHostBridgeLib/PciHostBridgeSupport.c
> +++ b/CorebootPayloadPkg/Library/PciHostBridgeLib/PciHostBridgeSupport.c
> @@ -319,6 +319,11 @@ ScanForRootBridges (
>
>    *NumberOfRootBridges = 0;
>    RootBridges = NULL;
> +  ZeroMem (&Io, sizeof (Io));
> +  ZeroMem (&Mem, sizeof (Mem));
> +  ZeroMem (&MemAbove4G, sizeof (MemAbove4G));
> +  ZeroMem (&PMem, sizeof (PMem));
> +  ZeroMem (&PMemAbove4G, sizeof (PMemAbove4G));
>
>    //
>    // After scanning all the PCI devices on the PCI root bridge's primary bus,
> --
> 2.7.4
>

[PATCH v5 2/6] OvmfPkg/PciHostBridgeLib: Init PCI aperture to 0

[Heyi Guo]

Use ZeroMem to initialize all fields in temporary
PCI_ROOT_BRIDGE_APERTURE variables to zero. This is not mandatory but
is helpful for future extension: when we add new fields to
PCI_ROOT_BRIDGE_APERTURE and the default value of these fields can
safely be zero, this code will not suffer from an additional
change.

Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Heyi Guo <heyi.guo@linaro.org>

Cc: Jordan Justen <jordan.l.justen@intel.com>
Cc: Anthony Perard <anthony.perard@citrix.com>
Cc: Julien Grall <julien.grall@linaro.org>
Cc: Ruiyu Ni <ruiyu.ni@intel.com>
Cc: Laszlo Ersek <lersek@redhat.com>
Cc: Ard Biesheuvel <ard.biesheuvel@linaro.org>
---
 OvmfPkg/Library/PciHostBridgeLib/PciHostBridgeLib.c | 4 ++++
 OvmfPkg/Library/PciHostBridgeLib/XenSupport.c       | 5 +++++
 2 files changed, 9 insertions(+)

diff --git a/OvmfPkg/Library/PciHostBridgeLib/PciHostBridgeLib.c b/OvmfPkg/Library/PciHostBridgeLib/PciHostBridgeLib.c
index ff837035caff..4a650a4c6df9 100644
--- a/OvmfPkg/Library/PciHostBridgeLib/PciHostBridgeLib.c
+++ b/OvmfPkg/Library/PciHostBridgeLib/PciHostBridgeLib.c
@@ -217,6 +217,10 @@ PciHostBridgeGetRootBridges (
   PCI_ROOT_BRIDGE_APERTURE Mem;
   PCI_ROOT_BRIDGE_APERTURE MemAbove4G;
 
+  ZeroMem (&Io, sizeof (Io));
+  ZeroMem (&Mem, sizeof (Mem));
+  ZeroMem (&MemAbove4G, sizeof (MemAbove4G));
+
   if (PcdGetBool (PcdPciDisableBusEnumeration)) {
     return ScanForRootBridges (Count);
   }
diff --git a/OvmfPkg/Library/PciHostBridgeLib/XenSupport.c b/OvmfPkg/Library/PciHostBridgeLib/XenSupport.c
index 31c63ae19e0a..aaf101dfcb0e 100644
--- a/OvmfPkg/Library/PciHostBridgeLib/XenSupport.c
+++ b/OvmfPkg/Library/PciHostBridgeLib/XenSupport.c
@@ -193,6 +193,11 @@ ScanForRootBridges (
 
   *NumberOfRootBridges = 0;
   RootBridges = NULL;
+  ZeroMem (&Io, sizeof (Io));
+  ZeroMem (&Mem, sizeof (Mem));
+  ZeroMem (&MemAbove4G, sizeof (MemAbove4G));
+  ZeroMem (&PMem, sizeof (PMem));
+  ZeroMem (&PMemAbove4G, sizeof (PMemAbove4G));
 
   //
   // After scanning all the PCI devices on the PCI root bridge's primary bus,
-- 
2.7.4

Re: [PATCH v5 2/6] OvmfPkg/PciHostBridgeLib: Init PCI aperture to 0

[Laszlo Ersek]

Hello Heyi,

On 03/01/18 07:57, Heyi Guo wrote:
> Use ZeroMem to initialize all fields in temporary
> PCI_ROOT_BRIDGE_APERTURE variables to zero. This is not mandatory but
> is helpful for future extension: when we add new fields to
> PCI_ROOT_BRIDGE_APERTURE and the default value of these fields can
> safely be zero, this code will not suffer from an additional
> change.
>
> Contributed-under: TianoCore Contribution Agreement 1.1
> Signed-off-by: Heyi Guo <heyi.guo@linaro.org>
>
> Cc: Jordan Justen <jordan.l.justen@intel.com>
> Cc: Anthony Perard <anthony.perard@citrix.com>
> Cc: Julien Grall <julien.grall@linaro.org>
> Cc: Ruiyu Ni <ruiyu.ni@intel.com>
> Cc: Laszlo Ersek <lersek@redhat.com>
> Cc: Ard Biesheuvel <ard.biesheuvel@linaro.org>
> ---
>  OvmfPkg/Library/PciHostBridgeLib/PciHostBridgeLib.c | 4 ++++
>  OvmfPkg/Library/PciHostBridgeLib/XenSupport.c       | 5 +++++
>  2 files changed, 9 insertions(+)
>
> diff --git a/OvmfPkg/Library/PciHostBridgeLib/PciHostBridgeLib.c b/OvmfPkg/Library/PciHostBridgeLib/PciHostBridgeLib.c
> index ff837035caff..4a650a4c6df9 100644
> --- a/OvmfPkg/Library/PciHostBridgeLib/PciHostBridgeLib.c
> +++ b/OvmfPkg/Library/PciHostBridgeLib/PciHostBridgeLib.c
> @@ -217,6 +217,10 @@ PciHostBridgeGetRootBridges (
>    PCI_ROOT_BRIDGE_APERTURE Mem;
>    PCI_ROOT_BRIDGE_APERTURE MemAbove4G;
>
> +  ZeroMem (&Io, sizeof (Io));
> +  ZeroMem (&Mem, sizeof (Mem));
> +  ZeroMem (&MemAbove4G, sizeof (MemAbove4G));
> +
>    if (PcdGetBool (PcdPciDisableBusEnumeration)) {
>      return ScanForRootBridges (Count);
>    }

This is OK. (Although for a trivial perf improvement, you could move the
ZeroMem() calls after the PcdGetBool() / return. Not necessary, up to
you.)

However:

> diff --git a/OvmfPkg/Library/PciHostBridgeLib/XenSupport.c b/OvmfPkg/Library/PciHostBridgeLib/XenSupport.c
> index 31c63ae19e0a..aaf101dfcb0e 100644
> --- a/OvmfPkg/Library/PciHostBridgeLib/XenSupport.c
> +++ b/OvmfPkg/Library/PciHostBridgeLib/XenSupport.c
> @@ -193,6 +193,11 @@ ScanForRootBridges (
>
>    *NumberOfRootBridges = 0;
>    RootBridges = NULL;
> +  ZeroMem (&Io, sizeof (Io));
> +  ZeroMem (&Mem, sizeof (Mem));
> +  ZeroMem (&MemAbove4G, sizeof (MemAbove4G));
> +  ZeroMem (&PMem, sizeof (PMem));
> +  ZeroMem (&PMemAbove4G, sizeof (PMemAbove4G));
>
>    //
>    // After scanning all the PCI devices on the PCI root bridge's primary bus,
>

these ZeroMem() calls are not in the correct place. Please move them
into the "PrimaryBus" loop just underneath. That loop works like this:

For each primary bus:

  (1) set all of the aperture variables to "nonexistent":

    Io.Base = Mem.Base = MemAbove4G.Base = PMem.Base = PMemAbove4G.Base = MAX_UINT64;
    Io.Limit = Mem.Limit = MemAbove4G.Limit = PMem.Limit = PMemAbove4G.Limit = 0;

  (2) accumulate the BARs of the devices on the bus into the aperture
      variables

  (3) call InitRootBridge() with the aperture variables


That is, the ZeroMem() calls that you are adding have to be part of step
(1). So, please replace the assignments

    Io.Base = Mem.Base = MemAbove4G.Base = PMem.Base = PMemAbove4G.Base = MAX_UINT64;
    Io.Limit = Mem.Limit = MemAbove4G.Limit = PMem.Limit = PMemAbove4G.Limit = 0;

with

    ZeroMem (&Io, sizeof (Io));
    ZeroMem (&Mem, sizeof (Mem));
    ZeroMem (&MemAbove4G, sizeof (MemAbove4G));
    ZeroMem (&PMem, sizeof (PMem));
    ZeroMem (&PMemAbove4G, sizeof (PMemAbove4G));
    Io.Base = Mem.Base = MemAbove4G.Base = PMem.Base = PMemAbove4G.Base = MAX_UINT64;

Thanks!
Laszlo

Re: [PATCH v5 2/6] OvmfPkg/PciHostBridgeLib: Init PCI aperture to 0

[Guo Heyi]

On Thu, Mar 01, 2018 at 11:17:30AM +0100, Laszlo Ersek wrote:
> Hello Heyi,
> 
> On 03/01/18 07:57, Heyi Guo wrote:
> > Use ZeroMem to initialize all fields in temporary
> > PCI_ROOT_BRIDGE_APERTURE variables to zero. This is not mandatory but
> > is helpful for future extension: when we add new fields to
> > PCI_ROOT_BRIDGE_APERTURE and the default value of these fields can
> > safely be zero, this code will not suffer from an additional
> > change.
> >
> > Contributed-under: TianoCore Contribution Agreement 1.1
> > Signed-off-by: Heyi Guo <heyi.guo@linaro.org>
> >
> > Cc: Jordan Justen <jordan.l.justen@intel.com>
> > Cc: Anthony Perard <anthony.perard@citrix.com>
> > Cc: Julien Grall <julien.grall@linaro.org>
> > Cc: Ruiyu Ni <ruiyu.ni@intel.com>
> > Cc: Laszlo Ersek <lersek@redhat.com>
> > Cc: Ard Biesheuvel <ard.biesheuvel@linaro.org>
> > ---
> >  OvmfPkg/Library/PciHostBridgeLib/PciHostBridgeLib.c | 4 ++++
> >  OvmfPkg/Library/PciHostBridgeLib/XenSupport.c       | 5 +++++
> >  2 files changed, 9 insertions(+)
> >
> > diff --git a/OvmfPkg/Library/PciHostBridgeLib/PciHostBridgeLib.c b/OvmfPkg/Library/PciHostBridgeLib/PciHostBridgeLib.c
> > index ff837035caff..4a650a4c6df9 100644
> > --- a/OvmfPkg/Library/PciHostBridgeLib/PciHostBridgeLib.c
> > +++ b/OvmfPkg/Library/PciHostBridgeLib/PciHostBridgeLib.c
> > @@ -217,6 +217,10 @@ PciHostBridgeGetRootBridges (
> >    PCI_ROOT_BRIDGE_APERTURE Mem;
> >    PCI_ROOT_BRIDGE_APERTURE MemAbove4G;
> >
> > +  ZeroMem (&Io, sizeof (Io));
> > +  ZeroMem (&Mem, sizeof (Mem));
> > +  ZeroMem (&MemAbove4G, sizeof (MemAbove4G));
> > +
> >    if (PcdGetBool (PcdPciDisableBusEnumeration)) {
> >      return ScanForRootBridges (Count);
> >    }
> 
> This is OK. (Although for a trivial perf improvement, you could move the
> ZeroMem() calls after the PcdGetBool() / return. Not necessary, up to
> you.)
> 
> However:
> 
> > diff --git a/OvmfPkg/Library/PciHostBridgeLib/XenSupport.c b/OvmfPkg/Library/PciHostBridgeLib/XenSupport.c
> > index 31c63ae19e0a..aaf101dfcb0e 100644
> > --- a/OvmfPkg/Library/PciHostBridgeLib/XenSupport.c
> > +++ b/OvmfPkg/Library/PciHostBridgeLib/XenSupport.c
> > @@ -193,6 +193,11 @@ ScanForRootBridges (
> >
> >    *NumberOfRootBridges = 0;
> >    RootBridges = NULL;
> > +  ZeroMem (&Io, sizeof (Io));
> > +  ZeroMem (&Mem, sizeof (Mem));
> > +  ZeroMem (&MemAbove4G, sizeof (MemAbove4G));
> > +  ZeroMem (&PMem, sizeof (PMem));
> > +  ZeroMem (&PMemAbove4G, sizeof (PMemAbove4G));
> >
> >    //
> >    // After scanning all the PCI devices on the PCI root bridge's primary bus,
> >
> 
> these ZeroMem() calls are not in the correct place. Please move them
> into the "PrimaryBus" loop just underneath. That loop works like this:
> 
> For each primary bus:
> 
>   (1) set all of the aperture variables to "nonexistent":
> 
>     Io.Base = Mem.Base = MemAbove4G.Base = PMem.Base = PMemAbove4G.Base = MAX_UINT64;
>     Io.Limit = Mem.Limit = MemAbove4G.Limit = PMem.Limit = PMemAbove4G.Limit = 0;
> 
>   (2) accumulate the BARs of the devices on the bus into the aperture
>       variables
> 
>   (3) call InitRootBridge() with the aperture variables
> 
> 
> That is, the ZeroMem() calls that you are adding have to be part of step
> (1). So, please replace the assignments
> 
>     Io.Base = Mem.Base = MemAbove4G.Base = PMem.Base = PMemAbove4G.Base = MAX_UINT64;
>     Io.Limit = Mem.Limit = MemAbove4G.Limit = PMem.Limit = PMemAbove4G.Limit = 0;
> 
> with
> 
>     ZeroMem (&Io, sizeof (Io));
>     ZeroMem (&Mem, sizeof (Mem));
>     ZeroMem (&MemAbove4G, sizeof (MemAbove4G));
>     ZeroMem (&PMem, sizeof (PMem));
>     ZeroMem (&PMemAbove4G, sizeof (PMemAbove4G));
>     Io.Base = Mem.Base = MemAbove4G.Base = PMem.Base = PMemAbove4G.Base = MAX_UINT64;

Will it cause functional issue?

My idea of making the change is like this:

1. ZeroMem() is used to initialize all fields of APERTURE to 0; it can make it
   in the current place of the patch;

2. In the loop, some fields may be changed by the end of each iteration, and it
   is the responsibility of the existing code to re-initialize the changed fields
   to expected values explicitly. It seems not necessary to re-initialize the other
   fields which will not be changed.

However, your advice may be better that merges the initialization code together.
I can make the change in the next version of patch.

Thanks,
Heyi

> 
> Thanks!
> Laszlo

Re: [PATCH v5 2/6] OvmfPkg/PciHostBridgeLib: Init PCI aperture to 0

[Laszlo Ersek]

On 03/01/18 11:48, Guo Heyi wrote:
> On Thu, Mar 01, 2018 at 11:17:30AM +0100, Laszlo Ersek wrote:
>> On 03/01/18 07:57, Heyi Guo wrote:

>>> diff --git a/OvmfPkg/Library/PciHostBridgeLib/XenSupport.c b/OvmfPkg/Library/PciHostBridgeLib/XenSupport.c
>>> index 31c63ae19e0a..aaf101dfcb0e 100644
>>> --- a/OvmfPkg/Library/PciHostBridgeLib/XenSupport.c
>>> +++ b/OvmfPkg/Library/PciHostBridgeLib/XenSupport.c
>>> @@ -193,6 +193,11 @@ ScanForRootBridges (
>>>
>>>    *NumberOfRootBridges = 0;
>>>    RootBridges = NULL;
>>> +  ZeroMem (&Io, sizeof (Io));
>>> +  ZeroMem (&Mem, sizeof (Mem));
>>> +  ZeroMem (&MemAbove4G, sizeof (MemAbove4G));
>>> +  ZeroMem (&PMem, sizeof (PMem));
>>> +  ZeroMem (&PMemAbove4G, sizeof (PMemAbove4G));
>>>
>>>    //
>>>    // After scanning all the PCI devices on the PCI root bridge's primary bus,
>>>
>>
>> these ZeroMem() calls are not in the correct place. Please move them
>> into the "PrimaryBus" loop just underneath. That loop works like
>> this:
>>
>> For each primary bus:
>>
>>   (1) set all of the aperture variables to "nonexistent":
>>
>>     Io.Base = Mem.Base = MemAbove4G.Base = PMem.Base = PMemAbove4G.Base = MAX_UINT64;
>>     Io.Limit = Mem.Limit = MemAbove4G.Limit = PMem.Limit = PMemAbove4G.Limit = 0;
>>
>>   (2) accumulate the BARs of the devices on the bus into the aperture
>>       variables
>>
>>   (3) call InitRootBridge() with the aperture variables
>>
>>
>> That is, the ZeroMem() calls that you are adding have to be part of
>> step (1). So, please replace the assignments
>>
>>     Io.Base = Mem.Base = MemAbove4G.Base = PMem.Base = PMemAbove4G.Base = MAX_UINT64;
>>     Io.Limit = Mem.Limit = MemAbove4G.Limit = PMem.Limit = PMemAbove4G.Limit = 0;
>>
>> with
>>
>>     ZeroMem (&Io, sizeof (Io));
>>     ZeroMem (&Mem, sizeof (Mem));
>>     ZeroMem (&MemAbove4G, sizeof (MemAbove4G));
>>     ZeroMem (&PMem, sizeof (PMem));
>>     ZeroMem (&PMemAbove4G, sizeof (PMemAbove4G));
>>     Io.Base = Mem.Base = MemAbove4G.Base = PMem.Base = PMemAbove4G.Base = MAX_UINT64;
>
> Will it cause functional issue?
>
> My idea of making the change is like this:
>
> 1. ZeroMem() is used to initialize all fields of APERTURE to 0; it can
>    make it in the current place of the patch;
>
> 2. In the loop, some fields may be changed by the end of each
>    iteration, and it is the responsibility of the existing code to
>    re-initialize the changed fields to expected values explicitly. It
>    seems not necessary to re-initialize the other fields which will
>    not be changed.
>
> However, your advice may be better that merges the initialization code
> together. I can make the change in the next version of patch.

Yes, if it's not a big problem for you, please implement my request.
Going forward I wouldn't like to depend on such intricate details as
described in your point (2). Namely, in any other C project, I would
suggest that we write:

  for (PrimaryBus = 0; PrimaryBus <= PCI_MAX_BUS; PrimaryBus = SubBus + 1) {
    PCI_ROOT_BRIDGE_APERTURE Io = { .Base = MAX_UINT64 },
      Mem = Io,
      MemAbove4G = Io,
      PMem = Io,
      PMemAbove4G = Io;
    /* ... */
  }

In other words, I would:
- move the definition of the structs into the loop (sort of accepted,
  but not really liked in edk2),
- use real C initialization (forbidden in edk2),
- use designated initializers for the first object, which clears the
  unlisted fields (C99, forbidden in edk2),
- initialize the rest of the structs from the first struct where I used
  the designated initializer explicitly.

Moving the ZeroMem() into the loop is the closest approximation of this,
for edk2.

Thanks!
Laszlo

Re: [PATCH v5 2/6] OvmfPkg/PciHostBridgeLib: Init PCI aperture to 0

[Guo Heyi]

On Fri, Mar 02, 2018 at 11:19:31AM +0100, Laszlo Ersek wrote:
> On 03/01/18 11:48, Guo Heyi wrote:
> > On Thu, Mar 01, 2018 at 11:17:30AM +0100, Laszlo Ersek wrote:
> >> On 03/01/18 07:57, Heyi Guo wrote:
> 
> >>> diff --git a/OvmfPkg/Library/PciHostBridgeLib/XenSupport.c b/OvmfPkg/Library/PciHostBridgeLib/XenSupport.c
> >>> index 31c63ae19e0a..aaf101dfcb0e 100644
> >>> --- a/OvmfPkg/Library/PciHostBridgeLib/XenSupport.c
> >>> +++ b/OvmfPkg/Library/PciHostBridgeLib/XenSupport.c
> >>> @@ -193,6 +193,11 @@ ScanForRootBridges (
> >>>
> >>>    *NumberOfRootBridges = 0;
> >>>    RootBridges = NULL;
> >>> +  ZeroMem (&Io, sizeof (Io));
> >>> +  ZeroMem (&Mem, sizeof (Mem));
> >>> +  ZeroMem (&MemAbove4G, sizeof (MemAbove4G));
> >>> +  ZeroMem (&PMem, sizeof (PMem));
> >>> +  ZeroMem (&PMemAbove4G, sizeof (PMemAbove4G));
> >>>
> >>>    //
> >>>    // After scanning all the PCI devices on the PCI root bridge's primary bus,
> >>>
> >>
> >> these ZeroMem() calls are not in the correct place. Please move them
> >> into the "PrimaryBus" loop just underneath. That loop works like
> >> this:
> >>
> >> For each primary bus:
> >>
> >>   (1) set all of the aperture variables to "nonexistent":
> >>
> >>     Io.Base = Mem.Base = MemAbove4G.Base = PMem.Base = PMemAbove4G.Base = MAX_UINT64;
> >>     Io.Limit = Mem.Limit = MemAbove4G.Limit = PMem.Limit = PMemAbove4G.Limit = 0;
> >>
> >>   (2) accumulate the BARs of the devices on the bus into the aperture
> >>       variables
> >>
> >>   (3) call InitRootBridge() with the aperture variables
> >>
> >>
> >> That is, the ZeroMem() calls that you are adding have to be part of
> >> step (1). So, please replace the assignments
> >>
> >>     Io.Base = Mem.Base = MemAbove4G.Base = PMem.Base = PMemAbove4G.Base = MAX_UINT64;
> >>     Io.Limit = Mem.Limit = MemAbove4G.Limit = PMem.Limit = PMemAbove4G.Limit = 0;
> >>
> >> with
> >>
> >>     ZeroMem (&Io, sizeof (Io));
> >>     ZeroMem (&Mem, sizeof (Mem));
> >>     ZeroMem (&MemAbove4G, sizeof (MemAbove4G));
> >>     ZeroMem (&PMem, sizeof (PMem));
> >>     ZeroMem (&PMemAbove4G, sizeof (PMemAbove4G));
> >>     Io.Base = Mem.Base = MemAbove4G.Base = PMem.Base = PMemAbove4G.Base = MAX_UINT64;
> >
> > Will it cause functional issue?
> >
> > My idea of making the change is like this:
> >
> > 1. ZeroMem() is used to initialize all fields of APERTURE to 0; it can
> >    make it in the current place of the patch;
> >
> > 2. In the loop, some fields may be changed by the end of each
> >    iteration, and it is the responsibility of the existing code to
> >    re-initialize the changed fields to expected values explicitly. It
> >    seems not necessary to re-initialize the other fields which will
> >    not be changed.
> >
> > However, your advice may be better that merges the initialization code
> > together. I can make the change in the next version of patch.
> 
> Yes, if it's not a big problem for you, please implement my request.
> Going forward I wouldn't like to depend on such intricate details as
> described in your point (2). Namely, in any other C project, I would
> suggest that we write:
> 
>   for (PrimaryBus = 0; PrimaryBus <= PCI_MAX_BUS; PrimaryBus = SubBus + 1) {
>     PCI_ROOT_BRIDGE_APERTURE Io = { .Base = MAX_UINT64 },
>       Mem = Io,
>       MemAbove4G = Io,
>       PMem = Io,
>       PMemAbove4G = Io;
>     /* ... */
>   }
> 
> In other words, I would:
> - move the definition of the structs into the loop (sort of accepted,
>   but not really liked in edk2),
> - use real C initialization (forbidden in edk2),
> - use designated initializers for the first object, which clears the
>   unlisted fields (C99, forbidden in edk2),
> - initialize the rest of the structs from the first struct where I used
>   the designated initializer explicitly.
> 
> Moving the ZeroMem() into the loop is the closest approximation of this,
> for edk2.

OK, I can do that in the next version of patch.

Thanks,

Heyi

> 
> Thanks!
> Laszlo

Re: [PATCH v5 2/6] OvmfPkg/PciHostBridgeLib: Init PCI aperture to 0

[Laszlo Ersek]

On 03/01/18 07:57, Heyi Guo wrote:
> Use ZeroMem to initialize all fields in temporary
> PCI_ROOT_BRIDGE_APERTURE variables to zero. This is not mandatory but
> is helpful for future extension: when we add new fields to
> PCI_ROOT_BRIDGE_APERTURE and the default value of these fields can
> safely be zero, this code will not suffer from an additional
> change.
> 
> Contributed-under: TianoCore Contribution Agreement 1.1
> Signed-off-by: Heyi Guo <heyi.guo@linaro.org>
> 
> Cc: Jordan Justen <jordan.l.justen@intel.com>
> Cc: Anthony Perard <anthony.perard@citrix.com>
> Cc: Julien Grall <julien.grall@linaro.org>
> Cc: Ruiyu Ni <ruiyu.ni@intel.com>
> Cc: Laszlo Ersek <lersek@redhat.com>
> Cc: Ard Biesheuvel <ard.biesheuvel@linaro.org>
> ---
>  OvmfPkg/Library/PciHostBridgeLib/PciHostBridgeLib.c | 4 ++++
>  OvmfPkg/Library/PciHostBridgeLib/XenSupport.c       | 5 +++++
>  2 files changed, 9 insertions(+)

I also suggest a different subject line:

OvmfPkg/PciHostBridgeLib: clear PCI_ROOT_BRIDGE_APERTURE vars for (re)init

(74 chars)

Thanks
Laszlo

Re: [PATCH v5 2/6] OvmfPkg/PciHostBridgeLib: Init PCI aperture to 0

[Guo Heyi]

Thanks; I got some trouble in making the subject short and clear :)

Regards,
Heyi

On Thu, Mar 01, 2018 at 11:20:22AM +0100, Laszlo Ersek wrote:
> On 03/01/18 07:57, Heyi Guo wrote:
> > Use ZeroMem to initialize all fields in temporary
> > PCI_ROOT_BRIDGE_APERTURE variables to zero. This is not mandatory but
> > is helpful for future extension: when we add new fields to
> > PCI_ROOT_BRIDGE_APERTURE and the default value of these fields can
> > safely be zero, this code will not suffer from an additional
> > change.
> > 
> > Contributed-under: TianoCore Contribution Agreement 1.1
> > Signed-off-by: Heyi Guo <heyi.guo@linaro.org>
> > 
> > Cc: Jordan Justen <jordan.l.justen@intel.com>
> > Cc: Anthony Perard <anthony.perard@citrix.com>
> > Cc: Julien Grall <julien.grall@linaro.org>
> > Cc: Ruiyu Ni <ruiyu.ni@intel.com>
> > Cc: Laszlo Ersek <lersek@redhat.com>
> > Cc: Ard Biesheuvel <ard.biesheuvel@linaro.org>
> > ---
> >  OvmfPkg/Library/PciHostBridgeLib/PciHostBridgeLib.c | 4 ++++
> >  OvmfPkg/Library/PciHostBridgeLib/XenSupport.c       | 5 +++++
> >  2 files changed, 9 insertions(+)
> 
> I also suggest a different subject line:
> 
> OvmfPkg/PciHostBridgeLib: clear PCI_ROOT_BRIDGE_APERTURE vars for (re)init
> 
> (74 chars)
> 
> Thanks
> Laszlo

Re: [PATCH v5 2/6] OvmfPkg/PciHostBridgeLib: Init PCI aperture to 0

[Ni, Ruiyu]

On 3/1/2018 6:20 PM, Laszlo Ersek wrote:
> On 03/01/18 07:57, Heyi Guo wrote:
>> Use ZeroMem to initialize all fields in temporary
>> PCI_ROOT_BRIDGE_APERTURE variables to zero. This is not mandatory but
>> is helpful for future extension: when we add new fields to
>> PCI_ROOT_BRIDGE_APERTURE and the default value of these fields can
>> safely be zero, this code will not suffer from an additional
>> change.
>>
>> Contributed-under: TianoCore Contribution Agreement 1.1
>> Signed-off-by: Heyi Guo <heyi.guo@linaro.org>
>>
>> Cc: Jordan Justen <jordan.l.justen@intel.com>
>> Cc: Anthony Perard <anthony.perard@citrix.com>
>> Cc: Julien Grall <julien.grall@linaro.org>
>> Cc: Ruiyu Ni <ruiyu.ni@intel.com>
>> Cc: Laszlo Ersek <lersek@redhat.com>
>> Cc: Ard Biesheuvel <ard.biesheuvel@linaro.org>
>> ---
>>   OvmfPkg/Library/PciHostBridgeLib/PciHostBridgeLib.c | 4 ++++
>>   OvmfPkg/Library/PciHostBridgeLib/XenSupport.c       | 5 +++++
>>   2 files changed, 9 insertions(+)
> 
> I also suggest a different subject line:
> 
> OvmfPkg/PciHostBridgeLib: clear PCI_ROOT_BRIDGE_APERTURE vars for (re)init
> 
> (74 chars)

I sometimes tries very hard to make the subject line be <= 70 chars.
74 is acceptable?

> 
> Thanks
> Laszlo
> 


-- 
Thanks,
Ray

Re: [PATCH v5 2/6] OvmfPkg/PciHostBridgeLib: Init PCI aperture to 0

[Laszlo Ersek]

On 03/01/18 13:03, Ni, Ruiyu wrote:
> On 3/1/2018 6:20 PM, Laszlo Ersek wrote:
>> On 03/01/18 07:57, Heyi Guo wrote:
>>> Use ZeroMem to initialize all fields in temporary
>>> PCI_ROOT_BRIDGE_APERTURE variables to zero. This is not mandatory but
>>> is helpful for future extension: when we add new fields to
>>> PCI_ROOT_BRIDGE_APERTURE and the default value of these fields can
>>> safely be zero, this code will not suffer from an additional
>>> change.
>>>
>>> Contributed-under: TianoCore Contribution Agreement 1.1
>>> Signed-off-by: Heyi Guo <heyi.guo@linaro.org>
>>>
>>> Cc: Jordan Justen <jordan.l.justen@intel.com>
>>> Cc: Anthony Perard <anthony.perard@citrix.com>
>>> Cc: Julien Grall <julien.grall@linaro.org>
>>> Cc: Ruiyu Ni <ruiyu.ni@intel.com>
>>> Cc: Laszlo Ersek <lersek@redhat.com>
>>> Cc: Ard Biesheuvel <ard.biesheuvel@linaro.org>
>>> ---
>>>   OvmfPkg/Library/PciHostBridgeLib/PciHostBridgeLib.c | 4 ++++
>>>   OvmfPkg/Library/PciHostBridgeLib/XenSupport.c       | 5 +++++
>>>   2 files changed, 9 insertions(+)
>>
>> I also suggest a different subject line:
>>
>> OvmfPkg/PciHostBridgeLib: clear PCI_ROOT_BRIDGE_APERTURE vars for
>> (re)init
>>
>> (74 chars)
> 
> I sometimes tries very hard to make the subject line be <= 70 chars.
> 74 is acceptable?

To my knowledge, the Linux kernel development guidelines suggest
wrapping the commit message body at 74 characters, and IIRC the same
limit applies to the subject line. I tend to follow these ideas for edk2
development too.

I think anything under 74 chars (for the subject) is an unreasonable
expectation for edk2. First, we start with a prefix of the form

  XxxPkg/Module: ...

Sometimes this prefix is incredibly long alread :/ So what I do (and I
guess most others do as well) is that I first write an "honest" subject
line (which frequently reaches 90-100 chars), and then work on
compressing it down to 74 characters. Sometimes it becomes a real
struggle, with strange abbreviations etc. I might make an exception and
go up to 75-76, and hope that nobody notices :) So, in that range,
limiting ourselves to 70 chars would be catastrophic.

Thanks!
Laszlo

[PATCH v5 3/6] MdeModulePkg/PciHostBridgeLib.h: add address Translation

[Heyi Guo]

Add Translation field to PCI_ROOT_BRIDGE_APERTURE. Translation is used
to represent the difference between device address and host address,
if they are not the same on some platforms.

In UEFI 2.7, "Address Translation Offset" is "Offset to apply to the
Starting address to convert it to a PCI address".  This means:

  Translation = device address - host address

So we also use the above calculation for this Translation field to
keep consistent.

Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Heyi Guo <heyi.guo@linaro.org>
---
 MdeModulePkg/Include/Library/PciHostBridgeLib.h | 19 +++++++++++++++++++
 1 file changed, 19 insertions(+)

diff --git a/MdeModulePkg/Include/Library/PciHostBridgeLib.h b/MdeModulePkg/Include/Library/PciHostBridgeLib.h
index d42e9ecdd763..18963a0d3821 100644
--- a/MdeModulePkg/Include/Library/PciHostBridgeLib.h
+++ b/MdeModulePkg/Include/Library/PciHostBridgeLib.h
@@ -20,8 +20,27 @@
 // (Base > Limit) indicates an aperture is not available.
 //
 typedef struct {
+  //
+  // Base and Limit are the device address instead of host address when
+  // Translation is not zero
+  //
   UINT64 Base;
   UINT64 Limit;
+  //
+  // According to UEFI 2.7, Device Address = Host Address + Translation,
+  // so Translation = Device Address - Host Address.
+  // On platforms where Translation is not zero, the subtraction is probably to
+  // be performed with UINT64 wrap-around semantics, for we may translate an
+  // above-4G host address into a below-4G device address for legacy PCIe device
+  // compatibility.
+  //
+  // NOTE: The alignment of Translation is required to be larger than any BAR
+  // alignment in the same root bridge, so that the same alignment can be
+  // applied to both device address and host address, which simplifies the
+  // situation and makes the current resource allocation code in generic PCI
+  // host bridge driver still work.
+  //
+  UINT64 Translation;
 } PCI_ROOT_BRIDGE_APERTURE;
 
 typedef struct {
-- 
2.7.4

[PATCH v5 4/6] MdeModulePkg/PciHostBridgeDxe: Add support for address translation

[Heyi Guo]

PCI address translation is necessary for some non-x86 platforms. On
such platforms, address value (denoted as "device address" or "address
in PCI view") set to PCI BAR registers in configuration space might be
different from the address which is used by CPU to access the
registers in memory BAR or IO BAR spaces (denoted as "host address" or
"address in CPU view"). The difference between the two addresses is
called "Address Translation Offset" or simply "translation", and can
be represented by "Address Translation Offset" in ACPI QWORD Address
Space Descriptor (Offset 0x1E). However UEFI and ACPI differs on the
definitions of QWORD Address Space Descriptor, and we will follow UEFI
definition on UEFI protocols, such as PCI root bridge IO protocol and
PCI IO protocol. In UEFI 2.7, "Address Translation Offset" is "Offset
to apply to the Starting address to convert it to a PCI address". This
means:

1. Translation = device address - host address.

2. PciRootBridgeIo->Configuration should return CPU view address, as
well as PciIo->GetBarAttributes.

Summary of addresses used in protocol interfaces and internal
implementations:

1. *Only* the following protocol interfaces assume Address is Device
   Address:
(1). PciHostBridgeResourceAllocation.GetProposedResources()
     Otherwise PCI bus driver cannot set correct address into PCI
     BARs.
(2). PciRootBridgeIo.Mem.Read() and PciRootBridgeIo.Mem.Write()
(3). PciRootBridgeIo.CopyMem()
UEFI and PI spec have clear statements for all other protocol
interfaces about the address type.

2. Library interfaces and internal implementation:
(1). Base and Limit in PCI_ROOT_BRIDGE_APERTURE are device address.
     It is easy to check whether the address is below 4G or above 4G.
(2). Addresses in PCI_ROOT_BRIDGE_INSTANCE.ResAllocNode are host
     address, for they are allocated from GCD.
(3). Address passed to PciHostBridgeResourceConflict is host address,
     for it comes from PCI_ROOT_BRIDGE_INSTANCE.ResAllocNode.

RESTRICTION: to simplify the situation, we require the alignment of
Translation must be larger than any BAR alignment in the same root
bridge, so that resource allocation alignment can be applied to both
device address and host address.

Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Heyi Guo <heyi.guo@linaro.org>
Cc: Ruiyu Ni <ruiyu.ni@intel.com>
Cc: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Cc: Star Zeng <star.zeng@intel.com>
Cc: Eric Dong <eric.dong@intel.com>
Cc: Laszlo Ersek <lersek@redhat.com>
Cc: Michael D Kinney <michael.d.kinney@intel.com>
---

Notes:
    v5:
    - Add check for the alignment of Translation against the BAR alignment
      [Ray]
    - Keep coding style of comments consistent with the context [Ray]
    - Comment change for Base in PCI_RES_NODE [Ray]
    - Add macros of TO_HOST_ADDRESS and TO_DEVICE_ADDRESS for address type
      conversion (After that we can also simply the comments about the
      calculation [Ray]
    - Add check for bus translation offset in CreateRootBridge(), making
      sure it is zero, and unify code logic for all types of resource
      after that [Ray]
    - Use GetTranslationByResourceType() to simplify the code in
      RootBridgeIoConfiguration() (also fix a bug in previous patch
      version of missing a break after case TypePMem64) [Ray]
    - Commit message refinement [Ray]
    
    v4:
    - Add ASSERT (FALSE) to default branch in GetTranslationByResourceType
      [Laszlo]
    - Fix bug when passing BaseAddress to gDS->AllocateIoSpace and
      gDS->AllocateMemorySpace [Laszlo]
    - Add comment for applying alignment to both device address and host
      address, and add NOTE for the alignment requirement of Translation,
      as well as in commit message [Laszlo][Ray]
    - Improve indention for the code in CreateRootBridge [Laszlo]
    - Improve comment for Translation in PCI_ROOT_BRIDGE_APERTURE
      definition [Laszlo]
    - Ignore translation of bus in CreateRootBridge
    
    v4:
    - Add ASSERT (FALSE) to default branch in GetTranslationByResourceType
      [Laszlo]
    - Fix bug when passing BaseAddress to gDS->AllocateIoSpace and
      gDS->AllocateMemorySpace [Laszlo]
    - Add comment for applying alignment to both device address and host
      address, and add NOTE for the alignment requirement of Translation,
      as well as in commit message [Laszlo][Ray]
    - Improve indention for the code in CreateRootBridge [Laszlo]
    - Improve comment for Translation in PCI_ROOT_BRIDGE_APERTURE
      definition [Laszlo]
    - Ignore translation of bus in CreateRootBridge

 MdeModulePkg/Bus/Pci/PciHostBridgeDxe/PciHostBridge.h   |  21 ++++
 MdeModulePkg/Bus/Pci/PciHostBridgeDxe/PciHostResource.h |   3 +
 MdeModulePkg/Bus/Pci/PciHostBridgeDxe/PciHostBridge.c   | 129 +++++++++++++++++---
 MdeModulePkg/Bus/Pci/PciHostBridgeDxe/PciRootBridgeIo.c | 118 ++++++++++++++++--
 4 files changed, 245 insertions(+), 26 deletions(-)

diff --git a/MdeModulePkg/Bus/Pci/PciHostBridgeDxe/PciHostBridge.h b/MdeModulePkg/Bus/Pci/PciHostBridgeDxe/PciHostBridge.h
index 9a8ca21f4819..c2791ea5c2a4 100644
--- a/MdeModulePkg/Bus/Pci/PciHostBridgeDxe/PciHostBridge.h
+++ b/MdeModulePkg/Bus/Pci/PciHostBridgeDxe/PciHostBridge.h
@@ -38,6 +38,13 @@ typedef struct {
 #define PCI_HOST_BRIDGE_FROM_THIS(a) CR (a, PCI_HOST_BRIDGE_INSTANCE, ResAlloc, PCI_HOST_BRIDGE_SIGNATURE)
 
 //
+// Macros to translate device address to host address and vice versa. According
+// to UEFI 2.7, device address = host address + translation offset.
+//
+#define TO_HOST_ADDRESS(DeviceAddress,TranslationOffset) ((DeviceAddress) - (TranslationOffset))
+#define TO_DEVICE_ADDRESS(HostAddress,TranslationOffset) ((HostAddress) + (TranslationOffset))
+
+//
 // Driver Entry Point
 //
 /**
@@ -247,6 +254,20 @@ ResourceConflict (
   IN  PCI_HOST_BRIDGE_INSTANCE *HostBridge
   );
 
+/**
+  This routine gets translation offset from a root bridge instance by resource type.
+
+  @param RootBridge The Root Bridge Instance for the resources.
+  @param ResourceType The Resource Type of the translation offset.
+
+  @retval The Translation Offset of the specified resource.
+**/
+UINT64
+GetTranslationByResourceType (
+  IN  PCI_ROOT_BRIDGE_INSTANCE     *RootBridge,
+  IN  PCI_RESOURCE_TYPE            ResourceType
+  );
+
 extern EFI_METRONOME_ARCH_PROTOCOL *mMetronome;
 extern EFI_CPU_IO2_PROTOCOL        *mCpuIo;
 #endif
diff --git a/MdeModulePkg/Bus/Pci/PciHostBridgeDxe/PciHostResource.h b/MdeModulePkg/Bus/Pci/PciHostBridgeDxe/PciHostResource.h
index 8612c0c3251b..a6c3739db368 100644
--- a/MdeModulePkg/Bus/Pci/PciHostBridgeDxe/PciHostResource.h
+++ b/MdeModulePkg/Bus/Pci/PciHostBridgeDxe/PciHostResource.h
@@ -38,6 +38,9 @@ typedef enum {
 
 typedef struct {
   PCI_RESOURCE_TYPE Type;
+  //
+  // Base is a host address
+  //
   UINT64            Base;
   UINT64            Length;
   UINT64            Alignment;
diff --git a/MdeModulePkg/Bus/Pci/PciHostBridgeDxe/PciHostBridge.c b/MdeModulePkg/Bus/Pci/PciHostBridgeDxe/PciHostBridge.c
index 1494848c3e8c..42ded2855c71 100644
--- a/MdeModulePkg/Bus/Pci/PciHostBridgeDxe/PciHostBridge.c
+++ b/MdeModulePkg/Bus/Pci/PciHostBridgeDxe/PciHostBridge.c
@@ -33,6 +33,39 @@ EFI_EVENT                   mIoMmuEvent;
 VOID                        *mIoMmuRegistration;
 
 /**
+  This routine gets translation offset from a root bridge instance by resource type.
+
+  @param RootBridge The Root Bridge Instance for the resources.
+  @param ResourceType The Resource Type of the translation offset.
+
+  @retval The Translation Offset of the specified resource.
+**/
+UINT64
+GetTranslationByResourceType (
+  IN  PCI_ROOT_BRIDGE_INSTANCE     *RootBridge,
+  IN  PCI_RESOURCE_TYPE            ResourceType
+  )
+{
+  switch (ResourceType) {
+    case TypeIo:
+      return RootBridge->Io.Translation;
+    case TypeMem32:
+      return RootBridge->Mem.Translation;
+    case TypePMem32:
+      return RootBridge->PMem.Translation;
+    case TypeMem64:
+      return RootBridge->MemAbove4G.Translation;
+    case TypePMem64:
+      return RootBridge->PMemAbove4G.Translation;
+    case TypeBus:
+      return RootBridge->Bus.Translation;
+    default:
+      ASSERT (FALSE);
+      return 0;
+  }
+}
+
+/**
   Ensure the compatibility of an IO space descriptor with the IO aperture.
 
   The IO space descriptor can come from the GCD IO space map, or it can
@@ -366,6 +399,7 @@ InitializePciHostBridge (
   UINTN                       MemApertureIndex;
   BOOLEAN                     ResourceAssigned;
   LIST_ENTRY                  *Link;
+  UINT64                      HostAddress;
 
   RootBridges = PciHostBridgeGetRootBridges (&RootBridgeCount);
   if ((RootBridges == NULL) || (RootBridgeCount == 0)) {
@@ -411,8 +445,15 @@ InitializePciHostBridge (
     }
 
     if (RootBridges[Index].Io.Base <= RootBridges[Index].Io.Limit) {
+      //
+      // Base and Limit in PCI_ROOT_BRIDGE_APERTURE are device address.
+      // For GCD resource manipulation, we need to use host address.
+      //
+      HostAddress = TO_HOST_ADDRESS (RootBridges[Index].Io.Base,
+        RootBridges[Index].Io.Translation);
+
       Status = AddIoSpace (
-                 RootBridges[Index].Io.Base,
+                 HostAddress,
                  RootBridges[Index].Io.Limit - RootBridges[Index].Io.Base + 1
                  );
       ASSERT_EFI_ERROR (Status);
@@ -422,7 +463,7 @@ InitializePciHostBridge (
                         EfiGcdIoTypeIo,
                         0,
                         RootBridges[Index].Io.Limit - RootBridges[Index].Io.Base + 1,
-                        &RootBridges[Index].Io.Base,
+                        &HostAddress,
                         gImageHandle,
                         NULL
                         );
@@ -443,14 +484,20 @@ InitializePciHostBridge (
 
     for (MemApertureIndex = 0; MemApertureIndex < ARRAY_SIZE (MemApertures); MemApertureIndex++) {
       if (MemApertures[MemApertureIndex]->Base <= MemApertures[MemApertureIndex]->Limit) {
+        //
+        // Base and Limit in PCI_ROOT_BRIDGE_APERTURE are device address.
+        // For GCD resource manipulation, we need to use host address.
+        //
+        HostAddress = TO_HOST_ADDRESS (MemApertures[MemApertureIndex]->Base,
+          MemApertures[MemApertureIndex]->Translation);
         Status = AddMemoryMappedIoSpace (
-                   MemApertures[MemApertureIndex]->Base,
+                   HostAddress,
                    MemApertures[MemApertureIndex]->Limit - MemApertures[MemApertureIndex]->Base + 1,
                    EFI_MEMORY_UC
                    );
         ASSERT_EFI_ERROR (Status);
         Status = gDS->SetMemorySpaceAttributes (
-                        MemApertures[MemApertureIndex]->Base,
+                        HostAddress,
                         MemApertures[MemApertureIndex]->Limit - MemApertures[MemApertureIndex]->Base + 1,
                         EFI_MEMORY_UC
                         );
@@ -463,7 +510,7 @@ InitializePciHostBridge (
                           EfiGcdMemoryTypeMemoryMappedIo,
                           0,
                           MemApertures[MemApertureIndex]->Limit - MemApertures[MemApertureIndex]->Base + 1,
-                          &MemApertures[MemApertureIndex]->Base,
+                          &HostAddress,
                           gImageHandle,
                           NULL
                           );
@@ -654,6 +701,11 @@ AllocateResource (
   if (BaseAddress < Limit) {
     //
     // Have to make sure Aligment is handled since we are doing direct address allocation
+    // Strictly speaking, alignment requirement should be applied to device
+    // address instead of host address which is used in GCD manipulation below,
+    // but as we restrict the alignment of Translation to be larger than any BAR
+    // alignment in the root bridge, we can simplify the situation and consider
+    // the same alignment requirement is also applied to host address.
     //
     BaseAddress = ALIGN_VALUE (BaseAddress, LShiftU64 (1, BitsOfAlignment));
 
@@ -721,6 +773,7 @@ NotifyPhase (
   PCI_RESOURCE_TYPE                     Index2;
   BOOLEAN                               ResNodeHandled[TypeMax];
   UINT64                                MaxAlignment;
+  UINT64                                Translation;
 
   HostBridge = PCI_HOST_BRIDGE_FROM_THIS (This);
 
@@ -822,14 +875,43 @@ NotifyPhase (
           BitsOfAlignment = LowBitSet64 (Alignment + 1);
           BaseAddress = MAX_UINT64;
 
+          //
+          // RESTRICTION: To simplify the situation, we require the alignment of
+          // Translation must be larger than any BAR alignment in the same root
+          // bridge, so that resource allocation alignment can be applied to
+          // both device address and host address.
+          //
+          Translation = GetTranslationByResourceType (RootBridge, Index);
+          if ((Translation & Alignment) != 0) {
+            DEBUG ((DEBUG_ERROR, "[%a:%d] Translation %lx is not aligned to %lx!\n",
+              __FUNCTION__, __LINE__, Translation, Alignment
+              ));
+            ASSERT (FALSE);
+            //
+            // This may be caused by too large alignment or too small
+            // Translation; pick the 1st possibility and return out of resource,
+            // which can also go thru the same process for out of resource
+            // outside the loop.
+            //
+            ReturnStatus = EFI_OUT_OF_RESOURCES;
+            continue;
+          }
+
           switch (Index) {
           case TypeIo:
+            //
+            // Base and Limit in PCI_ROOT_BRIDGE_APERTURE are device address.
+            // For AllocateResource is manipulating GCD resource, we need to use
+            // host address here.
+            //
             BaseAddress = AllocateResource (
                             FALSE,
                             RootBridge->ResAllocNode[Index].Length,
                             MIN (15, BitsOfAlignment),
-                            ALIGN_VALUE (RootBridge->Io.Base, Alignment + 1),
-                            RootBridge->Io.Limit
+                            TO_HOST_ADDRESS (ALIGN_VALUE (RootBridge->Io.Base, Alignment + 1),
+                              RootBridge->Io.Translation),
+                            TO_HOST_ADDRESS (RootBridge->Io.Limit,
+                              RootBridge->Io.Translation)
                             );
             break;
 
@@ -838,8 +920,10 @@ NotifyPhase (
                             TRUE,
                             RootBridge->ResAllocNode[Index].Length,
                             MIN (63, BitsOfAlignment),
-                            ALIGN_VALUE (RootBridge->MemAbove4G.Base, Alignment + 1),
-                            RootBridge->MemAbove4G.Limit
+                            TO_HOST_ADDRESS (ALIGN_VALUE (RootBridge->MemAbove4G.Base, Alignment + 1),
+                              RootBridge->MemAbove4G.Translation),
+                            TO_HOST_ADDRESS (RootBridge->MemAbove4G.Limit,
+                              RootBridge->MemAbove4G.Translation)
                             );
             if (BaseAddress != MAX_UINT64) {
               break;
@@ -853,8 +937,10 @@ NotifyPhase (
                             TRUE,
                             RootBridge->ResAllocNode[Index].Length,
                             MIN (31, BitsOfAlignment),
-                            ALIGN_VALUE (RootBridge->Mem.Base, Alignment + 1),
-                            RootBridge->Mem.Limit
+                            TO_HOST_ADDRESS (ALIGN_VALUE (RootBridge->Mem.Base, Alignment + 1),
+                              RootBridge->Mem.Translation),
+                            TO_HOST_ADDRESS (RootBridge->Mem.Limit,
+                              RootBridge->Mem.Translation)
                             );
             break;
 
@@ -863,8 +949,10 @@ NotifyPhase (
                             TRUE,
                             RootBridge->ResAllocNode[Index].Length,
                             MIN (63, BitsOfAlignment),
-                            ALIGN_VALUE (RootBridge->PMemAbove4G.Base, Alignment + 1),
-                            RootBridge->PMemAbove4G.Limit
+                            TO_HOST_ADDRESS (ALIGN_VALUE (RootBridge->PMemAbove4G.Base, Alignment + 1),
+                              RootBridge->PMemAbove4G.Translation),
+                            TO_HOST_ADDRESS (RootBridge->PMemAbove4G.Limit,
+                              RootBridge->PMemAbove4G.Translation)
                             );
             if (BaseAddress != MAX_UINT64) {
               break;
@@ -877,8 +965,10 @@ NotifyPhase (
                             TRUE,
                             RootBridge->ResAllocNode[Index].Length,
                             MIN (31, BitsOfAlignment),
-                            ALIGN_VALUE (RootBridge->PMem.Base, Alignment + 1),
-                            RootBridge->PMem.Limit
+                            TO_HOST_ADDRESS (ALIGN_VALUE (RootBridge->PMem.Base, Alignment + 1),
+                              RootBridge->PMem.Translation),
+                            TO_HOST_ADDRESS (RootBridge->PMem.Limit,
+                              RootBridge->PMem.Translation)
                             );
             break;
 
@@ -1421,7 +1511,14 @@ GetProposedResources (
           Descriptor->Desc                  = ACPI_ADDRESS_SPACE_DESCRIPTOR;
           Descriptor->Len                   = sizeof (EFI_ACPI_ADDRESS_SPACE_DESCRIPTOR) - 3;;
           Descriptor->GenFlag               = 0;
-          Descriptor->AddrRangeMin          = RootBridge->ResAllocNode[Index].Base;
+          //
+          // AddrRangeMin in Resource Descriptor here should be device address
+          // instead of host address, or else PCI bus driver cannot set correct
+          // address into PCI BAR registers.
+          // Base in ResAllocNode is a host address, so conversion is needed.
+          //
+          Descriptor->AddrRangeMin          = TO_DEVICE_ADDRESS (RootBridge->ResAllocNode[Index].Base,
+            GetTranslationByResourceType (RootBridge, Index));
           Descriptor->AddrRangeMax          = 0;
           Descriptor->AddrTranslationOffset = (ResStatus == ResAllocated) ? EFI_RESOURCE_SATISFIED : PCI_RESOURCE_LESS;
           Descriptor->AddrLen               = RootBridge->ResAllocNode[Index].Length;
diff --git a/MdeModulePkg/Bus/Pci/PciHostBridgeDxe/PciRootBridgeIo.c b/MdeModulePkg/Bus/Pci/PciHostBridgeDxe/PciRootBridgeIo.c
index dc06c16dc038..5dd9d257d46d 100644
--- a/MdeModulePkg/Bus/Pci/PciHostBridgeDxe/PciRootBridgeIo.c
+++ b/MdeModulePkg/Bus/Pci/PciHostBridgeDxe/PciRootBridgeIo.c
@@ -86,12 +86,35 @@ CreateRootBridge (
           (Bridge->AllocationAttributes & EFI_PCI_HOST_BRIDGE_COMBINE_MEM_PMEM) != 0 ? L"CombineMemPMem " : L"",
           (Bridge->AllocationAttributes & EFI_PCI_HOST_BRIDGE_MEM64_DECODE) != 0 ? L"Mem64Decode" : L""
           ));
+  //
+  // Translation for bus is not supported.
+  //
   DEBUG ((EFI_D_INFO, "           Bus: %lx - %lx\n", Bridge->Bus.Base, Bridge->Bus.Limit));
-  DEBUG ((EFI_D_INFO, "            Io: %lx - %lx\n", Bridge->Io.Base, Bridge->Io.Limit));
-  DEBUG ((EFI_D_INFO, "           Mem: %lx - %lx\n", Bridge->Mem.Base, Bridge->Mem.Limit));
-  DEBUG ((EFI_D_INFO, "    MemAbove4G: %lx - %lx\n", Bridge->MemAbove4G.Base, Bridge->MemAbove4G.Limit));
-  DEBUG ((EFI_D_INFO, "          PMem: %lx - %lx\n", Bridge->PMem.Base, Bridge->PMem.Limit));
-  DEBUG ((EFI_D_INFO, "   PMemAbove4G: %lx - %lx\n", Bridge->PMemAbove4G.Base, Bridge->PMemAbove4G.Limit));
+  ASSERT (Bridge->Bus.Translation == 0);
+  if (Bridge->Bus.Translation != 0) {
+    return NULL;
+  }
+
+  DEBUG ((
+    DEBUG_INFO, "            Io: %lx - %lx Translation=%lx\n",
+    Bridge->Io.Base, Bridge->Io.Limit, Bridge->Io.Translation
+    ));
+  DEBUG ((
+    DEBUG_INFO, "           Mem: %lx - %lx Translation=%lx\n",
+    Bridge->Mem.Base, Bridge->Mem.Limit, Bridge->Mem.Translation
+    ));
+  DEBUG ((
+    DEBUG_INFO, "    MemAbove4G: %lx - %lx Translation=%lx\n",
+    Bridge->MemAbove4G.Base, Bridge->MemAbove4G.Limit, Bridge->MemAbove4G.Translation
+    ));
+  DEBUG ((
+    DEBUG_INFO, "          PMem: %lx - %lx Translation=%lx\n",
+    Bridge->PMem.Base, Bridge->PMem.Limit, Bridge->PMem.Translation
+    ));
+  DEBUG ((
+    DEBUG_INFO, "   PMemAbove4G: %lx - %lx Translation=%lx\n",
+    Bridge->PMemAbove4G.Base, Bridge->PMemAbove4G.Limit, Bridge->PMemAbove4G.Translation
+    ));
 
   //
   // Make sure Mem and MemAbove4G apertures are valid
@@ -206,7 +229,12 @@ CreateRootBridge (
     }
     RootBridge->ResAllocNode[Index].Type     = Index;
     if (Bridge->ResourceAssigned && (Aperture->Limit >= Aperture->Base)) {
-      RootBridge->ResAllocNode[Index].Base   = Aperture->Base;
+      //
+      // Base in ResAllocNode is a host address, while Base in Aperture is a
+      // device address.
+      //
+      RootBridge->ResAllocNode[Index].Base   = TO_HOST_ADDRESS (Aperture->Base,
+        Aperture->Translation);
       RootBridge->ResAllocNode[Index].Length = Aperture->Limit - Aperture->Base + 1;
       RootBridge->ResAllocNode[Index].Status = ResAllocated;
     } else {
@@ -403,6 +431,28 @@ RootBridgeIoCheckParameter (
   return EFI_SUCCESS;
 }
 
+EFI_STATUS
+RootBridgeIoGetMemTranslationByAddress (
+  IN PCI_ROOT_BRIDGE_INSTANCE               *RootBridge,
+  IN UINT64                                 Address,
+  IN OUT UINT64                             *Translation
+  )
+{
+  if (Address >= RootBridge->Mem.Base && Address <= RootBridge->Mem.Limit) {
+    *Translation = RootBridge->Mem.Translation;
+  } else if (Address >= RootBridge->PMem.Base && Address <= RootBridge->PMem.Limit) {
+    *Translation = RootBridge->PMem.Translation;
+  } else if (Address >= RootBridge->MemAbove4G.Base && Address <= RootBridge->MemAbove4G.Limit) {
+    *Translation = RootBridge->MemAbove4G.Translation;
+  } else if (Address >= RootBridge->PMemAbove4G.Base && Address <= RootBridge->PMemAbove4G.Limit) {
+    *Translation = RootBridge->PMemAbove4G.Translation;
+  } else {
+    return EFI_INVALID_PARAMETER;
+  }
+
+  return EFI_SUCCESS;
+}
+
 /**
   Polls an address in memory mapped I/O space until an exit condition is met,
   or a timeout occurs.
@@ -658,13 +708,25 @@ RootBridgeIoMemRead (
   )
 {
   EFI_STATUS                             Status;
+  PCI_ROOT_BRIDGE_INSTANCE               *RootBridge;
+  UINT64                                 Translation;
 
   Status = RootBridgeIoCheckParameter (This, MemOperation, Width, Address,
                                        Count, Buffer);
   if (EFI_ERROR (Status)) {
     return Status;
   }
-  return mCpuIo->Mem.Read (mCpuIo, (EFI_CPU_IO_PROTOCOL_WIDTH) Width, Address, Count, Buffer);
+
+  RootBridge = ROOT_BRIDGE_FROM_THIS (This);
+  Status = RootBridgeIoGetMemTranslationByAddress (RootBridge, Address, &Translation);
+  if (EFI_ERROR (Status)) {
+    return Status;
+  }
+
+  // Address passed to CpuIo->Mem.Read needs to be a host address instead of
+  // device address.
+  return mCpuIo->Mem.Read (mCpuIo, (EFI_CPU_IO_PROTOCOL_WIDTH) Width,
+      TO_HOST_ADDRESS (Address, Translation), Count, Buffer);
 }
 
 /**
@@ -705,13 +767,25 @@ RootBridgeIoMemWrite (
   )
 {
   EFI_STATUS                             Status;
+  PCI_ROOT_BRIDGE_INSTANCE               *RootBridge;
+  UINT64                                 Translation;
 
   Status = RootBridgeIoCheckParameter (This, MemOperation, Width, Address,
                                        Count, Buffer);
   if (EFI_ERROR (Status)) {
     return Status;
   }
-  return mCpuIo->Mem.Write (mCpuIo, (EFI_CPU_IO_PROTOCOL_WIDTH) Width, Address, Count, Buffer);
+
+  RootBridge = ROOT_BRIDGE_FROM_THIS (This);
+  Status = RootBridgeIoGetMemTranslationByAddress (RootBridge, Address, &Translation);
+  if (EFI_ERROR (Status)) {
+    return Status;
+  }
+
+  // Address passed to CpuIo->Mem.Write needs to be a host address instead of
+  // device address.
+  return mCpuIo->Mem.Write (mCpuIo, (EFI_CPU_IO_PROTOCOL_WIDTH) Width,
+      TO_HOST_ADDRESS (Address, Translation), Count, Buffer);
 }
 
 /**
@@ -746,6 +820,8 @@ RootBridgeIoIoRead (
   )
 {
   EFI_STATUS                                    Status;
+  PCI_ROOT_BRIDGE_INSTANCE                      *RootBridge;
+
   Status = RootBridgeIoCheckParameter (
              This, IoOperation, Width,
              Address, Count, Buffer
@@ -753,7 +829,13 @@ RootBridgeIoIoRead (
   if (EFI_ERROR (Status)) {
     return Status;
   }
-  return mCpuIo->Io.Read (mCpuIo, (EFI_CPU_IO_PROTOCOL_WIDTH) Width, Address, Count, Buffer);
+
+  RootBridge = ROOT_BRIDGE_FROM_THIS (This);
+
+  // Address passed to CpuIo->Io.Read needs to be a host address instead of
+  // device address.
+  return mCpuIo->Io.Read (mCpuIo, (EFI_CPU_IO_PROTOCOL_WIDTH) Width,
+      TO_HOST_ADDRESS (Address, RootBridge->Io.Translation), Count, Buffer);
 }
 
 /**
@@ -788,6 +870,8 @@ RootBridgeIoIoWrite (
   )
 {
   EFI_STATUS                                    Status;
+  PCI_ROOT_BRIDGE_INSTANCE                      *RootBridge;
+
   Status = RootBridgeIoCheckParameter (
              This, IoOperation, Width,
              Address, Count, Buffer
@@ -795,7 +879,13 @@ RootBridgeIoIoWrite (
   if (EFI_ERROR (Status)) {
     return Status;
   }
-  return mCpuIo->Io.Write (mCpuIo, (EFI_CPU_IO_PROTOCOL_WIDTH) Width, Address, Count, Buffer);
+
+  RootBridge = ROOT_BRIDGE_FROM_THIS (This);
+
+  // Address passed to CpuIo->Io.Write needs to be a host address instead of
+  // device address.
+  return mCpuIo->Io.Write (mCpuIo, (EFI_CPU_IO_PROTOCOL_WIDTH) Width,
+      TO_HOST_ADDRESS (Address, RootBridge->Io.Translation), Count, Buffer);
 }
 
 /**
@@ -1615,9 +1705,17 @@ RootBridgeIoConfiguration (
 
     Descriptor->Desc = ACPI_ADDRESS_SPACE_DESCRIPTOR;
     Descriptor->Len  = sizeof (EFI_ACPI_ADDRESS_SPACE_DESCRIPTOR) - 3;
+    // According to UEFI 2.7, RootBridgeIo->Configuration should return address
+    // range in CPU view (host address), and ResAllocNode->Base is already a CPU
+    // view address (host address).
     Descriptor->AddrRangeMin  = ResAllocNode->Base;
     Descriptor->AddrRangeMax  = ResAllocNode->Base + ResAllocNode->Length - 1;
     Descriptor->AddrLen       = ResAllocNode->Length;
+    Descriptor->AddrTranslationOffset = GetTranslationByResourceType (
+      RootBridge,
+      ResAllocNode->Type
+      );
+
     switch (ResAllocNode->Type) {
 
     case TypeIo:
-- 
2.7.4

Re: [PATCH v5 4/6] MdeModulePkg/PciHostBridgeDxe: Add support for address translation

[Guo Heyi]

Hi Ray,

Any comments for v5?

Regards,

Heyi

On Thu, Mar 01, 2018 at 02:57:22PM +0800, Heyi Guo wrote:
> PCI address translation is necessary for some non-x86 platforms. On
> such platforms, address value (denoted as "device address" or "address
> in PCI view") set to PCI BAR registers in configuration space might be
> different from the address which is used by CPU to access the
> registers in memory BAR or IO BAR spaces (denoted as "host address" or
> "address in CPU view"). The difference between the two addresses is
> called "Address Translation Offset" or simply "translation", and can
> be represented by "Address Translation Offset" in ACPI QWORD Address
> Space Descriptor (Offset 0x1E). However UEFI and ACPI differs on the
> definitions of QWORD Address Space Descriptor, and we will follow UEFI
> definition on UEFI protocols, such as PCI root bridge IO protocol and
> PCI IO protocol. In UEFI 2.7, "Address Translation Offset" is "Offset
> to apply to the Starting address to convert it to a PCI address". This
> means:
> 
> 1. Translation = device address - host address.
> 
> 2. PciRootBridgeIo->Configuration should return CPU view address, as
> well as PciIo->GetBarAttributes.
> 
> Summary of addresses used in protocol interfaces and internal
> implementations:
> 
> 1. *Only* the following protocol interfaces assume Address is Device
>    Address:
> (1). PciHostBridgeResourceAllocation.GetProposedResources()
>      Otherwise PCI bus driver cannot set correct address into PCI
>      BARs.
> (2). PciRootBridgeIo.Mem.Read() and PciRootBridgeIo.Mem.Write()
> (3). PciRootBridgeIo.CopyMem()
> UEFI and PI spec have clear statements for all other protocol
> interfaces about the address type.
> 
> 2. Library interfaces and internal implementation:
> (1). Base and Limit in PCI_ROOT_BRIDGE_APERTURE are device address.
>      It is easy to check whether the address is below 4G or above 4G.
> (2). Addresses in PCI_ROOT_BRIDGE_INSTANCE.ResAllocNode are host
>      address, for they are allocated from GCD.
> (3). Address passed to PciHostBridgeResourceConflict is host address,
>      for it comes from PCI_ROOT_BRIDGE_INSTANCE.ResAllocNode.
> 
> RESTRICTION: to simplify the situation, we require the alignment of
> Translation must be larger than any BAR alignment in the same root
> bridge, so that resource allocation alignment can be applied to both
> device address and host address.
> 
> Contributed-under: TianoCore Contribution Agreement 1.1
> Signed-off-by: Heyi Guo <heyi.guo@linaro.org>
> Cc: Ruiyu Ni <ruiyu.ni@intel.com>
> Cc: Ard Biesheuvel <ard.biesheuvel@linaro.org>
> Cc: Star Zeng <star.zeng@intel.com>
> Cc: Eric Dong <eric.dong@intel.com>
> Cc: Laszlo Ersek <lersek@redhat.com>
> Cc: Michael D Kinney <michael.d.kinney@intel.com>
> ---
> 
> Notes:
>     v5:
>     - Add check for the alignment of Translation against the BAR alignment
>       [Ray]
>     - Keep coding style of comments consistent with the context [Ray]
>     - Comment change for Base in PCI_RES_NODE [Ray]
>     - Add macros of TO_HOST_ADDRESS and TO_DEVICE_ADDRESS for address type
>       conversion (After that we can also simply the comments about the
>       calculation [Ray]
>     - Add check for bus translation offset in CreateRootBridge(), making
>       sure it is zero, and unify code logic for all types of resource
>       after that [Ray]
>     - Use GetTranslationByResourceType() to simplify the code in
>       RootBridgeIoConfiguration() (also fix a bug in previous patch
>       version of missing a break after case TypePMem64) [Ray]
>     - Commit message refinement [Ray]
>     
>     v4:
>     - Add ASSERT (FALSE) to default branch in GetTranslationByResourceType
>       [Laszlo]
>     - Fix bug when passing BaseAddress to gDS->AllocateIoSpace and
>       gDS->AllocateMemorySpace [Laszlo]
>     - Add comment for applying alignment to both device address and host
>       address, and add NOTE for the alignment requirement of Translation,
>       as well as in commit message [Laszlo][Ray]
>     - Improve indention for the code in CreateRootBridge [Laszlo]
>     - Improve comment for Translation in PCI_ROOT_BRIDGE_APERTURE
>       definition [Laszlo]
>     - Ignore translation of bus in CreateRootBridge
>     
>     v4:
>     - Add ASSERT (FALSE) to default branch in GetTranslationByResourceType
>       [Laszlo]
>     - Fix bug when passing BaseAddress to gDS->AllocateIoSpace and
>       gDS->AllocateMemorySpace [Laszlo]
>     - Add comment for applying alignment to both device address and host
>       address, and add NOTE for the alignment requirement of Translation,
>       as well as in commit message [Laszlo][Ray]
>     - Improve indention for the code in CreateRootBridge [Laszlo]
>     - Improve comment for Translation in PCI_ROOT_BRIDGE_APERTURE
>       definition [Laszlo]
>     - Ignore translation of bus in CreateRootBridge
> 
>  MdeModulePkg/Bus/Pci/PciHostBridgeDxe/PciHostBridge.h   |  21 ++++
>  MdeModulePkg/Bus/Pci/PciHostBridgeDxe/PciHostResource.h |   3 +
>  MdeModulePkg/Bus/Pci/PciHostBridgeDxe/PciHostBridge.c   | 129 +++++++++++++++++---
>  MdeModulePkg/Bus/Pci/PciHostBridgeDxe/PciRootBridgeIo.c | 118 ++++++++++++++++--
>  4 files changed, 245 insertions(+), 26 deletions(-)
> 
> diff --git a/MdeModulePkg/Bus/Pci/PciHostBridgeDxe/PciHostBridge.h b/MdeModulePkg/Bus/Pci/PciHostBridgeDxe/PciHostBridge.h
> index 9a8ca21f4819..c2791ea5c2a4 100644
> --- a/MdeModulePkg/Bus/Pci/PciHostBridgeDxe/PciHostBridge.h
> +++ b/MdeModulePkg/Bus/Pci/PciHostBridgeDxe/PciHostBridge.h
> @@ -38,6 +38,13 @@ typedef struct {
>  #define PCI_HOST_BRIDGE_FROM_THIS(a) CR (a, PCI_HOST_BRIDGE_INSTANCE, ResAlloc, PCI_HOST_BRIDGE_SIGNATURE)
>  
>  //
> +// Macros to translate device address to host address and vice versa. According
> +// to UEFI 2.7, device address = host address + translation offset.
> +//
> +#define TO_HOST_ADDRESS(DeviceAddress,TranslationOffset) ((DeviceAddress) - (TranslationOffset))
> +#define TO_DEVICE_ADDRESS(HostAddress,TranslationOffset) ((HostAddress) + (TranslationOffset))
> +
> +//
>  // Driver Entry Point
>  //
>  /**
> @@ -247,6 +254,20 @@ ResourceConflict (
>    IN  PCI_HOST_BRIDGE_INSTANCE *HostBridge
>    );
>  
> +/**
> +  This routine gets translation offset from a root bridge instance by resource type.
> +
> +  @param RootBridge The Root Bridge Instance for the resources.
> +  @param ResourceType The Resource Type of the translation offset.
> +
> +  @retval The Translation Offset of the specified resource.
> +**/
> +UINT64
> +GetTranslationByResourceType (
> +  IN  PCI_ROOT_BRIDGE_INSTANCE     *RootBridge,
> +  IN  PCI_RESOURCE_TYPE            ResourceType
> +  );
> +
>  extern EFI_METRONOME_ARCH_PROTOCOL *mMetronome;
>  extern EFI_CPU_IO2_PROTOCOL        *mCpuIo;
>  #endif
> diff --git a/MdeModulePkg/Bus/Pci/PciHostBridgeDxe/PciHostResource.h b/MdeModulePkg/Bus/Pci/PciHostBridgeDxe/PciHostResource.h
> index 8612c0c3251b..a6c3739db368 100644
> --- a/MdeModulePkg/Bus/Pci/PciHostBridgeDxe/PciHostResource.h
> +++ b/MdeModulePkg/Bus/Pci/PciHostBridgeDxe/PciHostResource.h
> @@ -38,6 +38,9 @@ typedef enum {
>  
>  typedef struct {
>    PCI_RESOURCE_TYPE Type;
> +  //
> +  // Base is a host address
> +  //
>    UINT64            Base;
>    UINT64            Length;
>    UINT64            Alignment;
> diff --git a/MdeModulePkg/Bus/Pci/PciHostBridgeDxe/PciHostBridge.c b/MdeModulePkg/Bus/Pci/PciHostBridgeDxe/PciHostBridge.c
> index 1494848c3e8c..42ded2855c71 100644
> --- a/MdeModulePkg/Bus/Pci/PciHostBridgeDxe/PciHostBridge.c
> +++ b/MdeModulePkg/Bus/Pci/PciHostBridgeDxe/PciHostBridge.c
> @@ -33,6 +33,39 @@ EFI_EVENT                   mIoMmuEvent;
>  VOID                        *mIoMmuRegistration;
>  
>  /**
> +  This routine gets translation offset from a root bridge instance by resource type.
> +
> +  @param RootBridge The Root Bridge Instance for the resources.
> +  @param ResourceType The Resource Type of the translation offset.
> +
> +  @retval The Translation Offset of the specified resource.
> +**/
> +UINT64
> +GetTranslationByResourceType (
> +  IN  PCI_ROOT_BRIDGE_INSTANCE     *RootBridge,
> +  IN  PCI_RESOURCE_TYPE            ResourceType
> +  )
> +{
> +  switch (ResourceType) {
> +    case TypeIo:
> +      return RootBridge->Io.Translation;
> +    case TypeMem32:
> +      return RootBridge->Mem.Translation;
> +    case TypePMem32:
> +      return RootBridge->PMem.Translation;
> +    case TypeMem64:
> +      return RootBridge->MemAbove4G.Translation;
> +    case TypePMem64:
> +      return RootBridge->PMemAbove4G.Translation;
> +    case TypeBus:
> +      return RootBridge->Bus.Translation;
> +    default:
> +      ASSERT (FALSE);
> +      return 0;
> +  }
> +}
> +
> +/**
>    Ensure the compatibility of an IO space descriptor with the IO aperture.
>  
>    The IO space descriptor can come from the GCD IO space map, or it can
> @@ -366,6 +399,7 @@ InitializePciHostBridge (
>    UINTN                       MemApertureIndex;
>    BOOLEAN                     ResourceAssigned;
>    LIST_ENTRY                  *Link;
> +  UINT64                      HostAddress;
>  
>    RootBridges = PciHostBridgeGetRootBridges (&RootBridgeCount);
>    if ((RootBridges == NULL) || (RootBridgeCount == 0)) {
> @@ -411,8 +445,15 @@ InitializePciHostBridge (
>      }
>  
>      if (RootBridges[Index].Io.Base <= RootBridges[Index].Io.Limit) {
> +      //
> +      // Base and Limit in PCI_ROOT_BRIDGE_APERTURE are device address.
> +      // For GCD resource manipulation, we need to use host address.
> +      //
> +      HostAddress = TO_HOST_ADDRESS (RootBridges[Index].Io.Base,
> +        RootBridges[Index].Io.Translation);
> +
>        Status = AddIoSpace (
> -                 RootBridges[Index].Io.Base,
> +                 HostAddress,
>                   RootBridges[Index].Io.Limit - RootBridges[Index].Io.Base + 1
>                   );
>        ASSERT_EFI_ERROR (Status);
> @@ -422,7 +463,7 @@ InitializePciHostBridge (
>                          EfiGcdIoTypeIo,
>                          0,
>                          RootBridges[Index].Io.Limit - RootBridges[Index].Io.Base + 1,
> -                        &RootBridges[Index].Io.Base,
> +                        &HostAddress,
>                          gImageHandle,
>                          NULL
>                          );
> @@ -443,14 +484,20 @@ InitializePciHostBridge (
>  
>      for (MemApertureIndex = 0; MemApertureIndex < ARRAY_SIZE (MemApertures); MemApertureIndex++) {
>        if (MemApertures[MemApertureIndex]->Base <= MemApertures[MemApertureIndex]->Limit) {
> +        //
> +        // Base and Limit in PCI_ROOT_BRIDGE_APERTURE are device address.
> +        // For GCD resource manipulation, we need to use host address.
> +        //
> +        HostAddress = TO_HOST_ADDRESS (MemApertures[MemApertureIndex]->Base,
> +          MemApertures[MemApertureIndex]->Translation);
>          Status = AddMemoryMappedIoSpace (
> -                   MemApertures[MemApertureIndex]->Base,
> +                   HostAddress,
>                     MemApertures[MemApertureIndex]->Limit - MemApertures[MemApertureIndex]->Base + 1,
>                     EFI_MEMORY_UC
>                     );
>          ASSERT_EFI_ERROR (Status);
>          Status = gDS->SetMemorySpaceAttributes (
> -                        MemApertures[MemApertureIndex]->Base,
> +                        HostAddress,
>                          MemApertures[MemApertureIndex]->Limit - MemApertures[MemApertureIndex]->Base + 1,
>                          EFI_MEMORY_UC
>                          );
> @@ -463,7 +510,7 @@ InitializePciHostBridge (
>                            EfiGcdMemoryTypeMemoryMappedIo,
>                            0,
>                            MemApertures[MemApertureIndex]->Limit - MemApertures[MemApertureIndex]->Base + 1,
> -                          &MemApertures[MemApertureIndex]->Base,
> +                          &HostAddress,
>                            gImageHandle,
>                            NULL
>                            );
> @@ -654,6 +701,11 @@ AllocateResource (
>    if (BaseAddress < Limit) {
>      //
>      // Have to make sure Aligment is handled since we are doing direct address allocation
> +    // Strictly speaking, alignment requirement should be applied to device
> +    // address instead of host address which is used in GCD manipulation below,
> +    // but as we restrict the alignment of Translation to be larger than any BAR
> +    // alignment in the root bridge, we can simplify the situation and consider
> +    // the same alignment requirement is also applied to host address.
>      //
>      BaseAddress = ALIGN_VALUE (BaseAddress, LShiftU64 (1, BitsOfAlignment));
>  
> @@ -721,6 +773,7 @@ NotifyPhase (
>    PCI_RESOURCE_TYPE                     Index2;
>    BOOLEAN                               ResNodeHandled[TypeMax];
>    UINT64                                MaxAlignment;
> +  UINT64                                Translation;
>  
>    HostBridge = PCI_HOST_BRIDGE_FROM_THIS (This);
>  
> @@ -822,14 +875,43 @@ NotifyPhase (
>            BitsOfAlignment = LowBitSet64 (Alignment + 1);
>            BaseAddress = MAX_UINT64;
>  
> +          //
> +          // RESTRICTION: To simplify the situation, we require the alignment of
> +          // Translation must be larger than any BAR alignment in the same root
> +          // bridge, so that resource allocation alignment can be applied to
> +          // both device address and host address.
> +          //
> +          Translation = GetTranslationByResourceType (RootBridge, Index);
> +          if ((Translation & Alignment) != 0) {
> +            DEBUG ((DEBUG_ERROR, "[%a:%d] Translation %lx is not aligned to %lx!\n",
> +              __FUNCTION__, __LINE__, Translation, Alignment
> +              ));
> +            ASSERT (FALSE);
> +            //
> +            // This may be caused by too large alignment or too small
> +            // Translation; pick the 1st possibility and return out of resource,
> +            // which can also go thru the same process for out of resource
> +            // outside the loop.
> +            //
> +            ReturnStatus = EFI_OUT_OF_RESOURCES;
> +            continue;
> +          }
> +
>            switch (Index) {
>            case TypeIo:
> +            //
> +            // Base and Limit in PCI_ROOT_BRIDGE_APERTURE are device address.
> +            // For AllocateResource is manipulating GCD resource, we need to use
> +            // host address here.
> +            //
>              BaseAddress = AllocateResource (
>                              FALSE,
>                              RootBridge->ResAllocNode[Index].Length,
>                              MIN (15, BitsOfAlignment),
> -                            ALIGN_VALUE (RootBridge->Io.Base, Alignment + 1),
> -                            RootBridge->Io.Limit
> +                            TO_HOST_ADDRESS (ALIGN_VALUE (RootBridge->Io.Base, Alignment + 1),
> +                              RootBridge->Io.Translation),
> +                            TO_HOST_ADDRESS (RootBridge->Io.Limit,
> +                              RootBridge->Io.Translation)
>                              );
>              break;
>  
> @@ -838,8 +920,10 @@ NotifyPhase (
>                              TRUE,
>                              RootBridge->ResAllocNode[Index].Length,
>                              MIN (63, BitsOfAlignment),
> -                            ALIGN_VALUE (RootBridge->MemAbove4G.Base, Alignment + 1),
> -                            RootBridge->MemAbove4G.Limit
> +                            TO_HOST_ADDRESS (ALIGN_VALUE (RootBridge->MemAbove4G.Base, Alignment + 1),
> +                              RootBridge->MemAbove4G.Translation),
> +                            TO_HOST_ADDRESS (RootBridge->MemAbove4G.Limit,
> +                              RootBridge->MemAbove4G.Translation)
>                              );
>              if (BaseAddress != MAX_UINT64) {
>                break;
> @@ -853,8 +937,10 @@ NotifyPhase (
>                              TRUE,
>                              RootBridge->ResAllocNode[Index].Length,
>                              MIN (31, BitsOfAlignment),
> -                            ALIGN_VALUE (RootBridge->Mem.Base, Alignment + 1),
> -                            RootBridge->Mem.Limit
> +                            TO_HOST_ADDRESS (ALIGN_VALUE (RootBridge->Mem.Base, Alignment + 1),
> +                              RootBridge->Mem.Translation),
> +                            TO_HOST_ADDRESS (RootBridge->Mem.Limit,
> +                              RootBridge->Mem.Translation)
>                              );
>              break;
>  
> @@ -863,8 +949,10 @@ NotifyPhase (
>                              TRUE,
>                              RootBridge->ResAllocNode[Index].Length,
>                              MIN (63, BitsOfAlignment),
> -                            ALIGN_VALUE (RootBridge->PMemAbove4G.Base, Alignment + 1),
> -                            RootBridge->PMemAbove4G.Limit
> +                            TO_HOST_ADDRESS (ALIGN_VALUE (RootBridge->PMemAbove4G.Base, Alignment + 1),
> +                              RootBridge->PMemAbove4G.Translation),
> +                            TO_HOST_ADDRESS (RootBridge->PMemAbove4G.Limit,
> +                              RootBridge->PMemAbove4G.Translation)
>                              );
>              if (BaseAddress != MAX_UINT64) {
>                break;
> @@ -877,8 +965,10 @@ NotifyPhase (
>                              TRUE,
>                              RootBridge->ResAllocNode[Index].Length,
>                              MIN (31, BitsOfAlignment),
> -                            ALIGN_VALUE (RootBridge->PMem.Base, Alignment + 1),
> -                            RootBridge->PMem.Limit
> +                            TO_HOST_ADDRESS (ALIGN_VALUE (RootBridge->PMem.Base, Alignment + 1),
> +                              RootBridge->PMem.Translation),
> +                            TO_HOST_ADDRESS (RootBridge->PMem.Limit,
> +                              RootBridge->PMem.Translation)
>                              );
>              break;
>  
> @@ -1421,7 +1511,14 @@ GetProposedResources (
>            Descriptor->Desc                  = ACPI_ADDRESS_SPACE_DESCRIPTOR;
>            Descriptor->Len                   = sizeof (EFI_ACPI_ADDRESS_SPACE_DESCRIPTOR) - 3;;
>            Descriptor->GenFlag               = 0;
> -          Descriptor->AddrRangeMin          = RootBridge->ResAllocNode[Index].Base;
> +          //
> +          // AddrRangeMin in Resource Descriptor here should be device address
> +          // instead of host address, or else PCI bus driver cannot set correct
> +          // address into PCI BAR registers.
> +          // Base in ResAllocNode is a host address, so conversion is needed.
> +          //
> +          Descriptor->AddrRangeMin          = TO_DEVICE_ADDRESS (RootBridge->ResAllocNode[Index].Base,
> +            GetTranslationByResourceType (RootBridge, Index));
>            Descriptor->AddrRangeMax          = 0;
>            Descriptor->AddrTranslationOffset = (ResStatus == ResAllocated) ? EFI_RESOURCE_SATISFIED : PCI_RESOURCE_LESS;
>            Descriptor->AddrLen               = RootBridge->ResAllocNode[Index].Length;
> diff --git a/MdeModulePkg/Bus/Pci/PciHostBridgeDxe/PciRootBridgeIo.c b/MdeModulePkg/Bus/Pci/PciHostBridgeDxe/PciRootBridgeIo.c
> index dc06c16dc038..5dd9d257d46d 100644
> --- a/MdeModulePkg/Bus/Pci/PciHostBridgeDxe/PciRootBridgeIo.c
> +++ b/MdeModulePkg/Bus/Pci/PciHostBridgeDxe/PciRootBridgeIo.c
> @@ -86,12 +86,35 @@ CreateRootBridge (
>            (Bridge->AllocationAttributes & EFI_PCI_HOST_BRIDGE_COMBINE_MEM_PMEM) != 0 ? L"CombineMemPMem " : L"",
>            (Bridge->AllocationAttributes & EFI_PCI_HOST_BRIDGE_MEM64_DECODE) != 0 ? L"Mem64Decode" : L""
>            ));
> +  //
> +  // Translation for bus is not supported.
> +  //
>    DEBUG ((EFI_D_INFO, "           Bus: %lx - %lx\n", Bridge->Bus.Base, Bridge->Bus.Limit));
> -  DEBUG ((EFI_D_INFO, "            Io: %lx - %lx\n", Bridge->Io.Base, Bridge->Io.Limit));
> -  DEBUG ((EFI_D_INFO, "           Mem: %lx - %lx\n", Bridge->Mem.Base, Bridge->Mem.Limit));
> -  DEBUG ((EFI_D_INFO, "    MemAbove4G: %lx - %lx\n", Bridge->MemAbove4G.Base, Bridge->MemAbove4G.Limit));
> -  DEBUG ((EFI_D_INFO, "          PMem: %lx - %lx\n", Bridge->PMem.Base, Bridge->PMem.Limit));
> -  DEBUG ((EFI_D_INFO, "   PMemAbove4G: %lx - %lx\n", Bridge->PMemAbove4G.Base, Bridge->PMemAbove4G.Limit));
> +  ASSERT (Bridge->Bus.Translation == 0);
> +  if (Bridge->Bus.Translation != 0) {
> +    return NULL;
> +  }
> +
> +  DEBUG ((
> +    DEBUG_INFO, "            Io: %lx - %lx Translation=%lx\n",
> +    Bridge->Io.Base, Bridge->Io.Limit, Bridge->Io.Translation
> +    ));
> +  DEBUG ((
> +    DEBUG_INFO, "           Mem: %lx - %lx Translation=%lx\n",
> +    Bridge->Mem.Base, Bridge->Mem.Limit, Bridge->Mem.Translation
> +    ));
> +  DEBUG ((
> +    DEBUG_INFO, "    MemAbove4G: %lx - %lx Translation=%lx\n",
> +    Bridge->MemAbove4G.Base, Bridge->MemAbove4G.Limit, Bridge->MemAbove4G.Translation
> +    ));
> +  DEBUG ((
> +    DEBUG_INFO, "          PMem: %lx - %lx Translation=%lx\n",
> +    Bridge->PMem.Base, Bridge->PMem.Limit, Bridge->PMem.Translation
> +    ));
> +  DEBUG ((
> +    DEBUG_INFO, "   PMemAbove4G: %lx - %lx Translation=%lx\n",
> +    Bridge->PMemAbove4G.Base, Bridge->PMemAbove4G.Limit, Bridge->PMemAbove4G.Translation
> +    ));
>  
>    //
>    // Make sure Mem and MemAbove4G apertures are valid
> @@ -206,7 +229,12 @@ CreateRootBridge (
>      }
>      RootBridge->ResAllocNode[Index].Type     = Index;
>      if (Bridge->ResourceAssigned && (Aperture->Limit >= Aperture->Base)) {
> -      RootBridge->ResAllocNode[Index].Base   = Aperture->Base;
> +      //
> +      // Base in ResAllocNode is a host address, while Base in Aperture is a
> +      // device address.
> +      //
> +      RootBridge->ResAllocNode[Index].Base   = TO_HOST_ADDRESS (Aperture->Base,
> +        Aperture->Translation);
>        RootBridge->ResAllocNode[Index].Length = Aperture->Limit - Aperture->Base + 1;
>        RootBridge->ResAllocNode[Index].Status = ResAllocated;
>      } else {
> @@ -403,6 +431,28 @@ RootBridgeIoCheckParameter (
>    return EFI_SUCCESS;
>  }
>  
> +EFI_STATUS
> +RootBridgeIoGetMemTranslationByAddress (
> +  IN PCI_ROOT_BRIDGE_INSTANCE               *RootBridge,
> +  IN UINT64                                 Address,
> +  IN OUT UINT64                             *Translation
> +  )
> +{
> +  if (Address >= RootBridge->Mem.Base && Address <= RootBridge->Mem.Limit) {
> +    *Translation = RootBridge->Mem.Translation;
> +  } else if (Address >= RootBridge->PMem.Base && Address <= RootBridge->PMem.Limit) {
> +    *Translation = RootBridge->PMem.Translation;
> +  } else if (Address >= RootBridge->MemAbove4G.Base && Address <= RootBridge->MemAbove4G.Limit) {
> +    *Translation = RootBridge->MemAbove4G.Translation;
> +  } else if (Address >= RootBridge->PMemAbove4G.Base && Address <= RootBridge->PMemAbove4G.Limit) {
> +    *Translation = RootBridge->PMemAbove4G.Translation;
> +  } else {
> +    return EFI_INVALID_PARAMETER;
> +  }
> +
> +  return EFI_SUCCESS;
> +}
> +
>  /**
>    Polls an address in memory mapped I/O space until an exit condition is met,
>    or a timeout occurs.
> @@ -658,13 +708,25 @@ RootBridgeIoMemRead (
>    )
>  {
>    EFI_STATUS                             Status;
> +  PCI_ROOT_BRIDGE_INSTANCE               *RootBridge;
> +  UINT64                                 Translation;
>  
>    Status = RootBridgeIoCheckParameter (This, MemOperation, Width, Address,
>                                         Count, Buffer);
>    if (EFI_ERROR (Status)) {
>      return Status;
>    }
> -  return mCpuIo->Mem.Read (mCpuIo, (EFI_CPU_IO_PROTOCOL_WIDTH) Width, Address, Count, Buffer);
> +
> +  RootBridge = ROOT_BRIDGE_FROM_THIS (This);
> +  Status = RootBridgeIoGetMemTranslationByAddress (RootBridge, Address, &Translation);
> +  if (EFI_ERROR (Status)) {
> +    return Status;
> +  }
> +
> +  // Address passed to CpuIo->Mem.Read needs to be a host address instead of
> +  // device address.
> +  return mCpuIo->Mem.Read (mCpuIo, (EFI_CPU_IO_PROTOCOL_WIDTH) Width,
> +      TO_HOST_ADDRESS (Address, Translation), Count, Buffer);
>  }
>  
>  /**
> @@ -705,13 +767,25 @@ RootBridgeIoMemWrite (
>    )
>  {
>    EFI_STATUS                             Status;
> +  PCI_ROOT_BRIDGE_INSTANCE               *RootBridge;
> +  UINT64                                 Translation;
>  
>    Status = RootBridgeIoCheckParameter (This, MemOperation, Width, Address,
>                                         Count, Buffer);
>    if (EFI_ERROR (Status)) {
>      return Status;
>    }
> -  return mCpuIo->Mem.Write (mCpuIo, (EFI_CPU_IO_PROTOCOL_WIDTH) Width, Address, Count, Buffer);
> +
> +  RootBridge = ROOT_BRIDGE_FROM_THIS (This);
> +  Status = RootBridgeIoGetMemTranslationByAddress (RootBridge, Address, &Translation);
> +  if (EFI_ERROR (Status)) {
> +    return Status;
> +  }
> +
> +  // Address passed to CpuIo->Mem.Write needs to be a host address instead of
> +  // device address.
> +  return mCpuIo->Mem.Write (mCpuIo, (EFI_CPU_IO_PROTOCOL_WIDTH) Width,
> +      TO_HOST_ADDRESS (Address, Translation), Count, Buffer);
>  }
>  
>  /**
> @@ -746,6 +820,8 @@ RootBridgeIoIoRead (
>    )
>  {
>    EFI_STATUS                                    Status;
> +  PCI_ROOT_BRIDGE_INSTANCE                      *RootBridge;
> +
>    Status = RootBridgeIoCheckParameter (
>               This, IoOperation, Width,
>               Address, Count, Buffer
> @@ -753,7 +829,13 @@ RootBridgeIoIoRead (
>    if (EFI_ERROR (Status)) {
>      return Status;
>    }
> -  return mCpuIo->Io.Read (mCpuIo, (EFI_CPU_IO_PROTOCOL_WIDTH) Width, Address, Count, Buffer);
> +
> +  RootBridge = ROOT_BRIDGE_FROM_THIS (This);
> +
> +  // Address passed to CpuIo->Io.Read needs to be a host address instead of
> +  // device address.
> +  return mCpuIo->Io.Read (mCpuIo, (EFI_CPU_IO_PROTOCOL_WIDTH) Width,
> +      TO_HOST_ADDRESS (Address, RootBridge->Io.Translation), Count, Buffer);
>  }
>  
>  /**
> @@ -788,6 +870,8 @@ RootBridgeIoIoWrite (
>    )
>  {
>    EFI_STATUS                                    Status;
> +  PCI_ROOT_BRIDGE_INSTANCE                      *RootBridge;
> +
>    Status = RootBridgeIoCheckParameter (
>               This, IoOperation, Width,
>               Address, Count, Buffer
> @@ -795,7 +879,13 @@ RootBridgeIoIoWrite (
>    if (EFI_ERROR (Status)) {
>      return Status;
>    }
> -  return mCpuIo->Io.Write (mCpuIo, (EFI_CPU_IO_PROTOCOL_WIDTH) Width, Address, Count, Buffer);
> +
> +  RootBridge = ROOT_BRIDGE_FROM_THIS (This);
> +
> +  // Address passed to CpuIo->Io.Write needs to be a host address instead of
> +  // device address.
> +  return mCpuIo->Io.Write (mCpuIo, (EFI_CPU_IO_PROTOCOL_WIDTH) Width,
> +      TO_HOST_ADDRESS (Address, RootBridge->Io.Translation), Count, Buffer);
>  }
>  
>  /**
> @@ -1615,9 +1705,17 @@ RootBridgeIoConfiguration (
>  
>      Descriptor->Desc = ACPI_ADDRESS_SPACE_DESCRIPTOR;
>      Descriptor->Len  = sizeof (EFI_ACPI_ADDRESS_SPACE_DESCRIPTOR) - 3;
> +    // According to UEFI 2.7, RootBridgeIo->Configuration should return address
> +    // range in CPU view (host address), and ResAllocNode->Base is already a CPU
> +    // view address (host address).
>      Descriptor->AddrRangeMin  = ResAllocNode->Base;
>      Descriptor->AddrRangeMax  = ResAllocNode->Base + ResAllocNode->Length - 1;
>      Descriptor->AddrLen       = ResAllocNode->Length;
> +    Descriptor->AddrTranslationOffset = GetTranslationByResourceType (
> +      RootBridge,
> +      ResAllocNode->Type
> +      );
> +
>      switch (ResAllocNode->Type) {
>  
>      case TypeIo:
> -- 
> 2.7.4
> 

Re: [PATCH v5 4/6] MdeModulePkg/PciHostBridgeDxe: Add support for address translation

[Ni, Ruiyu]

On 3/6/2018 10:44 AM, Guo Heyi wrote:
> Hi Ray,
> 
> Any comments for v5?

Heyi,
Some backward compatibility concerns were received from internal 
production teams. Current change will cause silent failure on old 
platforms because TranslationOffset might be random if uninitialized.
I will solve the concern and then send out updates to you, hopefully by 
end of next week.

> 
> Regards,
> 
> Heyi
> 
> On Thu, Mar 01, 2018 at 02:57:22PM +0800, Heyi Guo wrote:
>> PCI address translation is necessary for some non-x86 platforms. On
>> such platforms, address value (denoted as "device address" or "address
>> in PCI view") set to PCI BAR registers in configuration space might be
>> different from the address which is used by CPU to access the
>> registers in memory BAR or IO BAR spaces (denoted as "host address" or
>> "address in CPU view"). The difference between the two addresses is
>> called "Address Translation Offset" or simply "translation", and can
>> be represented by "Address Translation Offset" in ACPI QWORD Address
>> Space Descriptor (Offset 0x1E). However UEFI and ACPI differs on the
>> definitions of QWORD Address Space Descriptor, and we will follow UEFI
>> definition on UEFI protocols, such as PCI root bridge IO protocol and
>> PCI IO protocol. In UEFI 2.7, "Address Translation Offset" is "Offset
>> to apply to the Starting address to convert it to a PCI address". This
>> means:
>>
>> 1. Translation = device address - host address.
>>
>> 2. PciRootBridgeIo->Configuration should return CPU view address, as
>> well as PciIo->GetBarAttributes.
>>
>> Summary of addresses used in protocol interfaces and internal
>> implementations:
>>
>> 1. *Only* the following protocol interfaces assume Address is Device
>>     Address:
>> (1). PciHostBridgeResourceAllocation.GetProposedResources()
>>       Otherwise PCI bus driver cannot set correct address into PCI
>>       BARs.
>> (2). PciRootBridgeIo.Mem.Read() and PciRootBridgeIo.Mem.Write()
>> (3). PciRootBridgeIo.CopyMem()
>> UEFI and PI spec have clear statements for all other protocol
>> interfaces about the address type.
>>
>> 2. Library interfaces and internal implementation:
>> (1). Base and Limit in PCI_ROOT_BRIDGE_APERTURE are device address.
>>       It is easy to check whether the address is below 4G or above 4G.
>> (2). Addresses in PCI_ROOT_BRIDGE_INSTANCE.ResAllocNode are host
>>       address, for they are allocated from GCD.
>> (3). Address passed to PciHostBridgeResourceConflict is host address,
>>       for it comes from PCI_ROOT_BRIDGE_INSTANCE.ResAllocNode.
>>
>> RESTRICTION: to simplify the situation, we require the alignment of
>> Translation must be larger than any BAR alignment in the same root
>> bridge, so that resource allocation alignment can be applied to both
>> device address and host address.
>>
>> Contributed-under: TianoCore Contribution Agreement 1.1
>> Signed-off-by: Heyi Guo <heyi.guo@linaro.org>
>> Cc: Ruiyu Ni <ruiyu.ni@intel.com>
>> Cc: Ard Biesheuvel <ard.biesheuvel@linaro.org>
>> Cc: Star Zeng <star.zeng@intel.com>
>> Cc: Eric Dong <eric.dong@intel.com>
>> Cc: Laszlo Ersek <lersek@redhat.com>
>> Cc: Michael D Kinney <michael.d.kinney@intel.com>
>> ---
>>
>> Notes:
>>      v5:
>>      - Add check for the alignment of Translation against the BAR alignment
>>        [Ray]
>>      - Keep coding style of comments consistent with the context [Ray]
>>      - Comment change for Base in PCI_RES_NODE [Ray]
>>      - Add macros of TO_HOST_ADDRESS and TO_DEVICE_ADDRESS for address type
>>        conversion (After that we can also simply the comments about the
>>        calculation [Ray]
>>      - Add check for bus translation offset in CreateRootBridge(), making
>>        sure it is zero, and unify code logic for all types of resource
>>        after that [Ray]
>>      - Use GetTranslationByResourceType() to simplify the code in
>>        RootBridgeIoConfiguration() (also fix a bug in previous patch
>>        version of missing a break after case TypePMem64) [Ray]
>>      - Commit message refinement [Ray]
>>      
>>      v4:
>>      - Add ASSERT (FALSE) to default branch in GetTranslationByResourceType
>>        [Laszlo]
>>      - Fix bug when passing BaseAddress to gDS->AllocateIoSpace and
>>        gDS->AllocateMemorySpace [Laszlo]
>>      - Add comment for applying alignment to both device address and host
>>        address, and add NOTE for the alignment requirement of Translation,
>>        as well as in commit message [Laszlo][Ray]
>>      - Improve indention for the code in CreateRootBridge [Laszlo]
>>      - Improve comment for Translation in PCI_ROOT_BRIDGE_APERTURE
>>        definition [Laszlo]
>>      - Ignore translation of bus in CreateRootBridge
>>      
>>      v4:
>>      - Add ASSERT (FALSE) to default branch in GetTranslationByResourceType
>>        [Laszlo]
>>      - Fix bug when passing BaseAddress to gDS->AllocateIoSpace and
>>        gDS->AllocateMemorySpace [Laszlo]
>>      - Add comment for applying alignment to both device address and host
>>        address, and add NOTE for the alignment requirement of Translation,
>>        as well as in commit message [Laszlo][Ray]
>>      - Improve indention for the code in CreateRootBridge [Laszlo]
>>      - Improve comment for Translation in PCI_ROOT_BRIDGE_APERTURE
>>        definition [Laszlo]
>>      - Ignore translation of bus in CreateRootBridge
>>
>>   MdeModulePkg/Bus/Pci/PciHostBridgeDxe/PciHostBridge.h   |  21 ++++
>>   MdeModulePkg/Bus/Pci/PciHostBridgeDxe/PciHostResource.h |   3 +
>>   MdeModulePkg/Bus/Pci/PciHostBridgeDxe/PciHostBridge.c   | 129 +++++++++++++++++---
>>   MdeModulePkg/Bus/Pci/PciHostBridgeDxe/PciRootBridgeIo.c | 118 ++++++++++++++++--
>>   4 files changed, 245 insertions(+), 26 deletions(-)
>>
>> diff --git a/MdeModulePkg/Bus/Pci/PciHostBridgeDxe/PciHostBridge.h b/MdeModulePkg/Bus/Pci/PciHostBridgeDxe/PciHostBridge.h
>> index 9a8ca21f4819..c2791ea5c2a4 100644
>> --- a/MdeModulePkg/Bus/Pci/PciHostBridgeDxe/PciHostBridge.h
>> +++ b/MdeModulePkg/Bus/Pci/PciHostBridgeDxe/PciHostBridge.h
>> @@ -38,6 +38,13 @@ typedef struct {
>>   #define PCI_HOST_BRIDGE_FROM_THIS(a) CR (a, PCI_HOST_BRIDGE_INSTANCE, ResAlloc, PCI_HOST_BRIDGE_SIGNATURE)
>>   
>>   //
>> +// Macros to translate device address to host address and vice versa. According
>> +// to UEFI 2.7, device address = host address + translation offset.
>> +//
>> +#define TO_HOST_ADDRESS(DeviceAddress,TranslationOffset) ((DeviceAddress) - (TranslationOffset))
>> +#define TO_DEVICE_ADDRESS(HostAddress,TranslationOffset) ((HostAddress) + (TranslationOffset))
>> +
>> +//
>>   // Driver Entry Point
>>   //
>>   /**
>> @@ -247,6 +254,20 @@ ResourceConflict (
>>     IN  PCI_HOST_BRIDGE_INSTANCE *HostBridge
>>     );
>>   
>> +/**
>> +  This routine gets translation offset from a root bridge instance by resource type.
>> +
>> +  @param RootBridge The Root Bridge Instance for the resources.
>> +  @param ResourceType The Resource Type of the translation offset.
>> +
>> +  @retval The Translation Offset of the specified resource.
>> +**/
>> +UINT64
>> +GetTranslationByResourceType (
>> +  IN  PCI_ROOT_BRIDGE_INSTANCE     *RootBridge,
>> +  IN  PCI_RESOURCE_TYPE            ResourceType
>> +  );
>> +
>>   extern EFI_METRONOME_ARCH_PROTOCOL *mMetronome;
>>   extern EFI_CPU_IO2_PROTOCOL        *mCpuIo;
>>   #endif
>> diff --git a/MdeModulePkg/Bus/Pci/PciHostBridgeDxe/PciHostResource.h b/MdeModulePkg/Bus/Pci/PciHostBridgeDxe/PciHostResource.h
>> index 8612c0c3251b..a6c3739db368 100644
>> --- a/MdeModulePkg/Bus/Pci/PciHostBridgeDxe/PciHostResource.h
>> +++ b/MdeModulePkg/Bus/Pci/PciHostBridgeDxe/PciHostResource.h
>> @@ -38,6 +38,9 @@ typedef enum {
>>   
>>   typedef struct {
>>     PCI_RESOURCE_TYPE Type;
>> +  //
>> +  // Base is a host address
>> +  //
>>     UINT64            Base;
>>     UINT64            Length;
>>     UINT64            Alignment;
>> diff --git a/MdeModulePkg/Bus/Pci/PciHostBridgeDxe/PciHostBridge.c b/MdeModulePkg/Bus/Pci/PciHostBridgeDxe/PciHostBridge.c
>> index 1494848c3e8c..42ded2855c71 100644
>> --- a/MdeModulePkg/Bus/Pci/PciHostBridgeDxe/PciHostBridge.c
>> +++ b/MdeModulePkg/Bus/Pci/PciHostBridgeDxe/PciHostBridge.c
>> @@ -33,6 +33,39 @@ EFI_EVENT                   mIoMmuEvent;
>>   VOID                        *mIoMmuRegistration;
>>   
>>   /**
>> +  This routine gets translation offset from a root bridge instance by resource type.
>> +
>> +  @param RootBridge The Root Bridge Instance for the resources.
>> +  @param ResourceType The Resource Type of the translation offset.
>> +
>> +  @retval The Translation Offset of the specified resource.
>> +**/
>> +UINT64
>> +GetTranslationByResourceType (
>> +  IN  PCI_ROOT_BRIDGE_INSTANCE     *RootBridge,
>> +  IN  PCI_RESOURCE_TYPE            ResourceType
>> +  )
>> +{
>> +  switch (ResourceType) {
>> +    case TypeIo:
>> +      return RootBridge->Io.Translation;
>> +    case TypeMem32:
>> +      return RootBridge->Mem.Translation;
>> +    case TypePMem32:
>> +      return RootBridge->PMem.Translation;
>> +    case TypeMem64:
>> +      return RootBridge->MemAbove4G.Translation;
>> +    case TypePMem64:
>> +      return RootBridge->PMemAbove4G.Translation;
>> +    case TypeBus:
>> +      return RootBridge->Bus.Translation;
>> +    default:
>> +      ASSERT (FALSE);
>> +      return 0;
>> +  }
>> +}
>> +
>> +/**
>>     Ensure the compatibility of an IO space descriptor with the IO aperture.
>>   
>>     The IO space descriptor can come from the GCD IO space map, or it can
>> @@ -366,6 +399,7 @@ InitializePciHostBridge (
>>     UINTN                       MemApertureIndex;
>>     BOOLEAN                     ResourceAssigned;
>>     LIST_ENTRY                  *Link;
>> +  UINT64                      HostAddress;
>>   
>>     RootBridges = PciHostBridgeGetRootBridges (&RootBridgeCount);
>>     if ((RootBridges == NULL) || (RootBridgeCount == 0)) {
>> @@ -411,8 +445,15 @@ InitializePciHostBridge (
>>       }
>>   
>>       if (RootBridges[Index].Io.Base <= RootBridges[Index].Io.Limit) {
>> +      //
>> +      // Base and Limit in PCI_ROOT_BRIDGE_APERTURE are device address.
>> +      // For GCD resource manipulation, we need to use host address.
>> +      //
>> +      HostAddress = TO_HOST_ADDRESS (RootBridges[Index].Io.Base,
>> +        RootBridges[Index].Io.Translation);
>> +
>>         Status = AddIoSpace (
>> -                 RootBridges[Index].Io.Base,
>> +                 HostAddress,
>>                    RootBridges[Index].Io.Limit - RootBridges[Index].Io.Base + 1
>>                    );
>>         ASSERT_EFI_ERROR (Status);
>> @@ -422,7 +463,7 @@ InitializePciHostBridge (
>>                           EfiGcdIoTypeIo,
>>                           0,
>>                           RootBridges[Index].Io.Limit - RootBridges[Index].Io.Base + 1,
>> -                        &RootBridges[Index].Io.Base,
>> +                        &HostAddress,
>>                           gImageHandle,
>>                           NULL
>>                           );
>> @@ -443,14 +484,20 @@ InitializePciHostBridge (
>>   
>>       for (MemApertureIndex = 0; MemApertureIndex < ARRAY_SIZE (MemApertures); MemApertureIndex++) {
>>         if (MemApertures[MemApertureIndex]->Base <= MemApertures[MemApertureIndex]->Limit) {
>> +        //
>> +        // Base and Limit in PCI_ROOT_BRIDGE_APERTURE are device address.
>> +        // For GCD resource manipulation, we need to use host address.
>> +        //
>> +        HostAddress = TO_HOST_ADDRESS (MemApertures[MemApertureIndex]->Base,
>> +          MemApertures[MemApertureIndex]->Translation);
>>           Status = AddMemoryMappedIoSpace (
>> -                   MemApertures[MemApertureIndex]->Base,
>> +                   HostAddress,
>>                      MemApertures[MemApertureIndex]->Limit - MemApertures[MemApertureIndex]->Base + 1,
>>                      EFI_MEMORY_UC
>>                      );
>>           ASSERT_EFI_ERROR (Status);
>>           Status = gDS->SetMemorySpaceAttributes (
>> -                        MemApertures[MemApertureIndex]->Base,
>> +                        HostAddress,
>>                           MemApertures[MemApertureIndex]->Limit - MemApertures[MemApertureIndex]->Base + 1,
>>                           EFI_MEMORY_UC
>>                           );
>> @@ -463,7 +510,7 @@ InitializePciHostBridge (
>>                             EfiGcdMemoryTypeMemoryMappedIo,
>>                             0,
>>                             MemApertures[MemApertureIndex]->Limit - MemApertures[MemApertureIndex]->Base + 1,
>> -                          &MemApertures[MemApertureIndex]->Base,
>> +                          &HostAddress,
>>                             gImageHandle,
>>                             NULL
>>                             );
>> @@ -654,6 +701,11 @@ AllocateResource (
>>     if (BaseAddress < Limit) {
>>       //
>>       // Have to make sure Aligment is handled since we are doing direct address allocation
>> +    // Strictly speaking, alignment requirement should be applied to device
>> +    // address instead of host address which is used in GCD manipulation below,
>> +    // but as we restrict the alignment of Translation to be larger than any BAR
>> +    // alignment in the root bridge, we can simplify the situation and consider
>> +    // the same alignment requirement is also applied to host address.
>>       //
>>       BaseAddress = ALIGN_VALUE (BaseAddress, LShiftU64 (1, BitsOfAlignment));
>>   
>> @@ -721,6 +773,7 @@ NotifyPhase (
>>     PCI_RESOURCE_TYPE                     Index2;
>>     BOOLEAN                               ResNodeHandled[TypeMax];
>>     UINT64                                MaxAlignment;
>> +  UINT64                                Translation;
>>   
>>     HostBridge = PCI_HOST_BRIDGE_FROM_THIS (This);
>>   
>> @@ -822,14 +875,43 @@ NotifyPhase (
>>             BitsOfAlignment = LowBitSet64 (Alignment + 1);
>>             BaseAddress = MAX_UINT64;
>>   
>> +          //
>> +          // RESTRICTION: To simplify the situation, we require the alignment of
>> +          // Translation must be larger than any BAR alignment in the same root
>> +          // bridge, so that resource allocation alignment can be applied to
>> +          // both device address and host address.
>> +          //
>> +          Translation = GetTranslationByResourceType (RootBridge, Index);
>> +          if ((Translation & Alignment) != 0) {
>> +            DEBUG ((DEBUG_ERROR, "[%a:%d] Translation %lx is not aligned to %lx!\n",
>> +              __FUNCTION__, __LINE__, Translation, Alignment
>> +              ));
>> +            ASSERT (FALSE);
>> +            //
>> +            // This may be caused by too large alignment or too small
>> +            // Translation; pick the 1st possibility and return out of resource,
>> +            // which can also go thru the same process for out of resource
>> +            // outside the loop.
>> +            //
>> +            ReturnStatus = EFI_OUT_OF_RESOURCES;
>> +            continue;
>> +          }
>> +
>>             switch (Index) {
>>             case TypeIo:
>> +            //
>> +            // Base and Limit in PCI_ROOT_BRIDGE_APERTURE are device address.
>> +            // For AllocateResource is manipulating GCD resource, we need to use
>> +            // host address here.
>> +            //
>>               BaseAddress = AllocateResource (
>>                               FALSE,
>>                               RootBridge->ResAllocNode[Index].Length,
>>                               MIN (15, BitsOfAlignment),
>> -                            ALIGN_VALUE (RootBridge->Io.Base, Alignment + 1),
>> -                            RootBridge->Io.Limit
>> +                            TO_HOST_ADDRESS (ALIGN_VALUE (RootBridge->Io.Base, Alignment + 1),
>> +                              RootBridge->Io.Translation),
>> +                            TO_HOST_ADDRESS (RootBridge->Io.Limit,
>> +                              RootBridge->Io.Translation)
>>                               );
>>               break;
>>   
>> @@ -838,8 +920,10 @@ NotifyPhase (
>>                               TRUE,
>>                               RootBridge->ResAllocNode[Index].Length,
>>                               MIN (63, BitsOfAlignment),
>> -                            ALIGN_VALUE (RootBridge->MemAbove4G.Base, Alignment + 1),
>> -                            RootBridge->MemAbove4G.Limit
>> +                            TO_HOST_ADDRESS (ALIGN_VALUE (RootBridge->MemAbove4G.Base, Alignment + 1),
>> +                              RootBridge->MemAbove4G.Translation),
>> +                            TO_HOST_ADDRESS (RootBridge->MemAbove4G.Limit,
>> +                              RootBridge->MemAbove4G.Translation)
>>                               );
>>               if (BaseAddress != MAX_UINT64) {
>>                 break;
>> @@ -853,8 +937,10 @@ NotifyPhase (
>>                               TRUE,
>>                               RootBridge->ResAllocNode[Index].Length,
>>                               MIN (31, BitsOfAlignment),
>> -                            ALIGN_VALUE (RootBridge->Mem.Base, Alignment + 1),
>> -                            RootBridge->Mem.Limit
>> +                            TO_HOST_ADDRESS (ALIGN_VALUE (RootBridge->Mem.Base, Alignment + 1),
>> +                              RootBridge->Mem.Translation),
>> +                            TO_HOST_ADDRESS (RootBridge->Mem.Limit,
>> +                              RootBridge->Mem.Translation)
>>                               );
>>               break;
>>   
>> @@ -863,8 +949,10 @@ NotifyPhase (
>>                               TRUE,
>>                               RootBridge->ResAllocNode[Index].Length,
>>                               MIN (63, BitsOfAlignment),
>> -                            ALIGN_VALUE (RootBridge->PMemAbove4G.Base, Alignment + 1),
>> -                            RootBridge->PMemAbove4G.Limit
>> +                            TO_HOST_ADDRESS (ALIGN_VALUE (RootBridge->PMemAbove4G.Base, Alignment + 1),
>> +                              RootBridge->PMemAbove4G.Translation),
>> +                            TO_HOST_ADDRESS (RootBridge->PMemAbove4G.Limit,
>> +                              RootBridge->PMemAbove4G.Translation)
>>                               );
>>               if (BaseAddress != MAX_UINT64) {
>>                 break;
>> @@ -877,8 +965,10 @@ NotifyPhase (
>>                               TRUE,
>>                               RootBridge->ResAllocNode[Index].Length,
>>                               MIN (31, BitsOfAlignment),
>> -                            ALIGN_VALUE (RootBridge->PMem.Base, Alignment + 1),
>> -                            RootBridge->PMem.Limit
>> +                            TO_HOST_ADDRESS (ALIGN_VALUE (RootBridge->PMem.Base, Alignment + 1),
>> +                              RootBridge->PMem.Translation),
>> +                            TO_HOST_ADDRESS (RootBridge->PMem.Limit,
>> +                              RootBridge->PMem.Translation)
>>                               );
>>               break;
>>   
>> @@ -1421,7 +1511,14 @@ GetProposedResources (
>>             Descriptor->Desc                  = ACPI_ADDRESS_SPACE_DESCRIPTOR;
>>             Descriptor->Len                   = sizeof (EFI_ACPI_ADDRESS_SPACE_DESCRIPTOR) - 3;;
>>             Descriptor->GenFlag               = 0;
>> -          Descriptor->AddrRangeMin          = RootBridge->ResAllocNode[Index].Base;
>> +          //
>> +          // AddrRangeMin in Resource Descriptor here should be device address
>> +          // instead of host address, or else PCI bus driver cannot set correct
>> +          // address into PCI BAR registers.
>> +          // Base in ResAllocNode is a host address, so conversion is needed.
>> +          //
>> +          Descriptor->AddrRangeMin          = TO_DEVICE_ADDRESS (RootBridge->ResAllocNode[Index].Base,
>> +            GetTranslationByResourceType (RootBridge, Index));
>>             Descriptor->AddrRangeMax          = 0;
>>             Descriptor->AddrTranslationOffset = (ResStatus == ResAllocated) ? EFI_RESOURCE_SATISFIED : PCI_RESOURCE_LESS;
>>             Descriptor->AddrLen               = RootBridge->ResAllocNode[Index].Length;
>> diff --git a/MdeModulePkg/Bus/Pci/PciHostBridgeDxe/PciRootBridgeIo.c b/MdeModulePkg/Bus/Pci/PciHostBridgeDxe/PciRootBridgeIo.c
>> index dc06c16dc038..5dd9d257d46d 100644
>> --- a/MdeModulePkg/Bus/Pci/PciHostBridgeDxe/PciRootBridgeIo.c
>> +++ b/MdeModulePkg/Bus/Pci/PciHostBridgeDxe/PciRootBridgeIo.c
>> @@ -86,12 +86,35 @@ CreateRootBridge (
>>             (Bridge->AllocationAttributes & EFI_PCI_HOST_BRIDGE_COMBINE_MEM_PMEM) != 0 ? L"CombineMemPMem " : L"",
>>             (Bridge->AllocationAttributes & EFI_PCI_HOST_BRIDGE_MEM64_DECODE) != 0 ? L"Mem64Decode" : L""
>>             ));
>> +  //
>> +  // Translation for bus is not supported.
>> +  //
>>     DEBUG ((EFI_D_INFO, "           Bus: %lx - %lx\n", Bridge->Bus.Base, Bridge->Bus.Limit));
>> -  DEBUG ((EFI_D_INFO, "            Io: %lx - %lx\n", Bridge->Io.Base, Bridge->Io.Limit));
>> -  DEBUG ((EFI_D_INFO, "           Mem: %lx - %lx\n", Bridge->Mem.Base, Bridge->Mem.Limit));
>> -  DEBUG ((EFI_D_INFO, "    MemAbove4G: %lx - %lx\n", Bridge->MemAbove4G.Base, Bridge->MemAbove4G.Limit));
>> -  DEBUG ((EFI_D_INFO, "          PMem: %lx - %lx\n", Bridge->PMem.Base, Bridge->PMem.Limit));
>> -  DEBUG ((EFI_D_INFO, "   PMemAbove4G: %lx - %lx\n", Bridge->PMemAbove4G.Base, Bridge->PMemAbove4G.Limit));
>> +  ASSERT (Bridge->Bus.Translation == 0);
>> +  if (Bridge->Bus.Translation != 0) {
>> +    return NULL;
>> +  }
>> +
>> +  DEBUG ((
>> +    DEBUG_INFO, "            Io: %lx - %lx Translation=%lx\n",
>> +    Bridge->Io.Base, Bridge->Io.Limit, Bridge->Io.Translation
>> +    ));
>> +  DEBUG ((
>> +    DEBUG_INFO, "           Mem: %lx - %lx Translation=%lx\n",
>> +    Bridge->Mem.Base, Bridge->Mem.Limit, Bridge->Mem.Translation
>> +    ));
>> +  DEBUG ((
>> +    DEBUG_INFO, "    MemAbove4G: %lx - %lx Translation=%lx\n",
>> +    Bridge->MemAbove4G.Base, Bridge->MemAbove4G.Limit, Bridge->MemAbove4G.Translation
>> +    ));
>> +  DEBUG ((
>> +    DEBUG_INFO, "          PMem: %lx - %lx Translation=%lx\n",
>> +    Bridge->PMem.Base, Bridge->PMem.Limit, Bridge->PMem.Translation
>> +    ));
>> +  DEBUG ((
>> +    DEBUG_INFO, "   PMemAbove4G: %lx - %lx Translation=%lx\n",
>> +    Bridge->PMemAbove4G.Base, Bridge->PMemAbove4G.Limit, Bridge->PMemAbove4G.Translation
>> +    ));
>>   
>>     //
>>     // Make sure Mem and MemAbove4G apertures are valid
>> @@ -206,7 +229,12 @@ CreateRootBridge (
>>       }
>>       RootBridge->ResAllocNode[Index].Type     = Index;
>>       if (Bridge->ResourceAssigned && (Aperture->Limit >= Aperture->Base)) {
>> -      RootBridge->ResAllocNode[Index].Base   = Aperture->Base;
>> +      //
>> +      // Base in ResAllocNode is a host address, while Base in Aperture is a
>> +      // device address.
>> +      //
>> +      RootBridge->ResAllocNode[Index].Base   = TO_HOST_ADDRESS (Aperture->Base,
>> +        Aperture->Translation);
>>         RootBridge->ResAllocNode[Index].Length = Aperture->Limit - Aperture->Base + 1;
>>         RootBridge->ResAllocNode[Index].Status = ResAllocated;
>>       } else {
>> @@ -403,6 +431,28 @@ RootBridgeIoCheckParameter (
>>     return EFI_SUCCESS;
>>   }
>>   
>> +EFI_STATUS
>> +RootBridgeIoGetMemTranslationByAddress (
>> +  IN PCI_ROOT_BRIDGE_INSTANCE               *RootBridge,
>> +  IN UINT64                                 Address,
>> +  IN OUT UINT64                             *Translation
>> +  )
>> +{
>> +  if (Address >= RootBridge->Mem.Base && Address <= RootBridge->Mem.Limit) {
>> +    *Translation = RootBridge->Mem.Translation;
>> +  } else if (Address >= RootBridge->PMem.Base && Address <= RootBridge->PMem.Limit) {
>> +    *Translation = RootBridge->PMem.Translation;
>> +  } else if (Address >= RootBridge->MemAbove4G.Base && Address <= RootBridge->MemAbove4G.Limit) {
>> +    *Translation = RootBridge->MemAbove4G.Translation;
>> +  } else if (Address >= RootBridge->PMemAbove4G.Base && Address <= RootBridge->PMemAbove4G.Limit) {
>> +    *Translation = RootBridge->PMemAbove4G.Translation;
>> +  } else {
>> +    return EFI_INVALID_PARAMETER;
>> +  }
>> +
>> +  return EFI_SUCCESS;
>> +}
>> +
>>   /**
>>     Polls an address in memory mapped I/O space until an exit condition is met,
>>     or a timeout occurs.
>> @@ -658,13 +708,25 @@ RootBridgeIoMemRead (
>>     )
>>   {
>>     EFI_STATUS                             Status;
>> +  PCI_ROOT_BRIDGE_INSTANCE               *RootBridge;
>> +  UINT64                                 Translation;
>>   
>>     Status = RootBridgeIoCheckParameter (This, MemOperation, Width, Address,
>>                                          Count, Buffer);
>>     if (EFI_ERROR (Status)) {
>>       return Status;
>>     }
>> -  return mCpuIo->Mem.Read (mCpuIo, (EFI_CPU_IO_PROTOCOL_WIDTH) Width, Address, Count, Buffer);
>> +
>> +  RootBridge = ROOT_BRIDGE_FROM_THIS (This);
>> +  Status = RootBridgeIoGetMemTranslationByAddress (RootBridge, Address, &Translation);
>> +  if (EFI_ERROR (Status)) {
>> +    return Status;
>> +  }
>> +
>> +  // Address passed to CpuIo->Mem.Read needs to be a host address instead of
>> +  // device address.
>> +  return mCpuIo->Mem.Read (mCpuIo, (EFI_CPU_IO_PROTOCOL_WIDTH) Width,
>> +      TO_HOST_ADDRESS (Address, Translation), Count, Buffer);
>>   }
>>   
>>   /**
>> @@ -705,13 +767,25 @@ RootBridgeIoMemWrite (
>>     )
>>   {
>>     EFI_STATUS                             Status;
>> +  PCI_ROOT_BRIDGE_INSTANCE               *RootBridge;
>> +  UINT64                                 Translation;
>>   
>>     Status = RootBridgeIoCheckParameter (This, MemOperation, Width, Address,
>>                                          Count, Buffer);
>>     if (EFI_ERROR (Status)) {
>>       return Status;
>>     }
>> -  return mCpuIo->Mem.Write (mCpuIo, (EFI_CPU_IO_PROTOCOL_WIDTH) Width, Address, Count, Buffer);
>> +
>> +  RootBridge = ROOT_BRIDGE_FROM_THIS (This);
>> +  Status = RootBridgeIoGetMemTranslationByAddress (RootBridge, Address, &Translation);
>> +  if (EFI_ERROR (Status)) {
>> +    return Status;
>> +  }
>> +
>> +  // Address passed to CpuIo->Mem.Write needs to be a host address instead of
>> +  // device address.
>> +  return mCpuIo->Mem.Write (mCpuIo, (EFI_CPU_IO_PROTOCOL_WIDTH) Width,
>> +      TO_HOST_ADDRESS (Address, Translation), Count, Buffer);
>>   }
>>   
>>   /**
>> @@ -746,6 +820,8 @@ RootBridgeIoIoRead (
>>     )
>>   {
>>     EFI_STATUS                                    Status;
>> +  PCI_ROOT_BRIDGE_INSTANCE                      *RootBridge;
>> +
>>     Status = RootBridgeIoCheckParameter (
>>                This, IoOperation, Width,
>>                Address, Count, Buffer
>> @@ -753,7 +829,13 @@ RootBridgeIoIoRead (
>>     if (EFI_ERROR (Status)) {
>>       return Status;
>>     }
>> -  return mCpuIo->Io.Read (mCpuIo, (EFI_CPU_IO_PROTOCOL_WIDTH) Width, Address, Count, Buffer);
>> +
>> +  RootBridge = ROOT_BRIDGE_FROM_THIS (This);
>> +
>> +  // Address passed to CpuIo->Io.Read needs to be a host address instead of
>> +  // device address.
>> +  return mCpuIo->Io.Read (mCpuIo, (EFI_CPU_IO_PROTOCOL_WIDTH) Width,
>> +      TO_HOST_ADDRESS (Address, RootBridge->Io.Translation), Count, Buffer);
>>   }
>>   
>>   /**
>> @@ -788,6 +870,8 @@ RootBridgeIoIoWrite (
>>     )
>>   {
>>     EFI_STATUS                                    Status;
>> +  PCI_ROOT_BRIDGE_INSTANCE                      *RootBridge;
>> +
>>     Status = RootBridgeIoCheckParameter (
>>                This, IoOperation, Width,
>>                Address, Count, Buffer
>> @@ -795,7 +879,13 @@ RootBridgeIoIoWrite (
>>     if (EFI_ERROR (Status)) {
>>       return Status;
>>     }
>> -  return mCpuIo->Io.Write (mCpuIo, (EFI_CPU_IO_PROTOCOL_WIDTH) Width, Address, Count, Buffer);
>> +
>> +  RootBridge = ROOT_BRIDGE_FROM_THIS (This);
>> +
>> +  // Address passed to CpuIo->Io.Write needs to be a host address instead of
>> +  // device address.
>> +  return mCpuIo->Io.Write (mCpuIo, (EFI_CPU_IO_PROTOCOL_WIDTH) Width,
>> +      TO_HOST_ADDRESS (Address, RootBridge->Io.Translation), Count, Buffer);
>>   }
>>   
>>   /**
>> @@ -1615,9 +1705,17 @@ RootBridgeIoConfiguration (
>>   
>>       Descriptor->Desc = ACPI_ADDRESS_SPACE_DESCRIPTOR;
>>       Descriptor->Len  = sizeof (EFI_ACPI_ADDRESS_SPACE_DESCRIPTOR) - 3;
>> +    // According to UEFI 2.7, RootBridgeIo->Configuration should return address
>> +    // range in CPU view (host address), and ResAllocNode->Base is already a CPU
>> +    // view address (host address).
>>       Descriptor->AddrRangeMin  = ResAllocNode->Base;
>>       Descriptor->AddrRangeMax  = ResAllocNode->Base + ResAllocNode->Length - 1;
>>       Descriptor->AddrLen       = ResAllocNode->Length;
>> +    Descriptor->AddrTranslationOffset = GetTranslationByResourceType (
>> +      RootBridge,
>> +      ResAllocNode->Type
>> +      );
>> +
>>       switch (ResAllocNode->Type) {
>>   
>>       case TypeIo:
>> -- 
>> 2.7.4
>>


-- 
Thanks,
Ray

Re: [PATCH v5 4/6] MdeModulePkg/PciHostBridgeDxe: Add support for address translation

[Guo Heyi]

Thanks. Please let me know if any further changes are needed.

Regards,

Heyi

On Wed, Mar 07, 2018 at 12:30:59PM +0800, Ni, Ruiyu wrote:
> On 3/6/2018 10:44 AM, Guo Heyi wrote:
> >Hi Ray,
> >
> >Any comments for v5?
> 
> Heyi,
> Some backward compatibility concerns were received from internal production
> teams. Current change will cause silent failure on old platforms because
> TranslationOffset might be random if uninitialized.
> I will solve the concern and then send out updates to you, hopefully by end
> of next week.
> 
> >
> >Regards,
> >
> >Heyi
> >
> >On Thu, Mar 01, 2018 at 02:57:22PM +0800, Heyi Guo wrote:
> >>PCI address translation is necessary for some non-x86 platforms. On
> >>such platforms, address value (denoted as "device address" or "address
> >>in PCI view") set to PCI BAR registers in configuration space might be
> >>different from the address which is used by CPU to access the
> >>registers in memory BAR or IO BAR spaces (denoted as "host address" or
> >>"address in CPU view"). The difference between the two addresses is
> >>called "Address Translation Offset" or simply "translation", and can
> >>be represented by "Address Translation Offset" in ACPI QWORD Address
> >>Space Descriptor (Offset 0x1E). However UEFI and ACPI differs on the
> >>definitions of QWORD Address Space Descriptor, and we will follow UEFI
> >>definition on UEFI protocols, such as PCI root bridge IO protocol and
> >>PCI IO protocol. In UEFI 2.7, "Address Translation Offset" is "Offset
> >>to apply to the Starting address to convert it to a PCI address". This
> >>means:
> >>
> >>1. Translation = device address - host address.
> >>
> >>2. PciRootBridgeIo->Configuration should return CPU view address, as
> >>well as PciIo->GetBarAttributes.
> >>
> >>Summary of addresses used in protocol interfaces and internal
> >>implementations:
> >>
> >>1. *Only* the following protocol interfaces assume Address is Device
> >>    Address:
> >>(1). PciHostBridgeResourceAllocation.GetProposedResources()
> >>      Otherwise PCI bus driver cannot set correct address into PCI
> >>      BARs.
> >>(2). PciRootBridgeIo.Mem.Read() and PciRootBridgeIo.Mem.Write()
> >>(3). PciRootBridgeIo.CopyMem()
> >>UEFI and PI spec have clear statements for all other protocol
> >>interfaces about the address type.
> >>
> >>2. Library interfaces and internal implementation:
> >>(1). Base and Limit in PCI_ROOT_BRIDGE_APERTURE are device address.
> >>      It is easy to check whether the address is below 4G or above 4G.
> >>(2). Addresses in PCI_ROOT_BRIDGE_INSTANCE.ResAllocNode are host
> >>      address, for they are allocated from GCD.
> >>(3). Address passed to PciHostBridgeResourceConflict is host address,
> >>      for it comes from PCI_ROOT_BRIDGE_INSTANCE.ResAllocNode.
> >>
> >>RESTRICTION: to simplify the situation, we require the alignment of
> >>Translation must be larger than any BAR alignment in the same root
> >>bridge, so that resource allocation alignment can be applied to both
> >>device address and host address.
> >>
> >>Contributed-under: TianoCore Contribution Agreement 1.1
> >>Signed-off-by: Heyi Guo <heyi.guo@linaro.org>
> >>Cc: Ruiyu Ni <ruiyu.ni@intel.com>
> >>Cc: Ard Biesheuvel <ard.biesheuvel@linaro.org>
> >>Cc: Star Zeng <star.zeng@intel.com>
> >>Cc: Eric Dong <eric.dong@intel.com>
> >>Cc: Laszlo Ersek <lersek@redhat.com>
> >>Cc: Michael D Kinney <michael.d.kinney@intel.com>
> >>---
> >>
> >>Notes:
> >>     v5:
> >>     - Add check for the alignment of Translation against the BAR alignment
> >>       [Ray]
> >>     - Keep coding style of comments consistent with the context [Ray]
> >>     - Comment change for Base in PCI_RES_NODE [Ray]
> >>     - Add macros of TO_HOST_ADDRESS and TO_DEVICE_ADDRESS for address type
> >>       conversion (After that we can also simply the comments about the
> >>       calculation [Ray]
> >>     - Add check for bus translation offset in CreateRootBridge(), making
> >>       sure it is zero, and unify code logic for all types of resource
> >>       after that [Ray]
> >>     - Use GetTranslationByResourceType() to simplify the code in
> >>       RootBridgeIoConfiguration() (also fix a bug in previous patch
> >>       version of missing a break after case TypePMem64) [Ray]
> >>     - Commit message refinement [Ray]
> >>     v4:
> >>     - Add ASSERT (FALSE) to default branch in GetTranslationByResourceType
> >>       [Laszlo]
> >>     - Fix bug when passing BaseAddress to gDS->AllocateIoSpace and
> >>       gDS->AllocateMemorySpace [Laszlo]
> >>     - Add comment for applying alignment to both device address and host
> >>       address, and add NOTE for the alignment requirement of Translation,
> >>       as well as in commit message [Laszlo][Ray]
> >>     - Improve indention for the code in CreateRootBridge [Laszlo]
> >>     - Improve comment for Translation in PCI_ROOT_BRIDGE_APERTURE
> >>       definition [Laszlo]
> >>     - Ignore translation of bus in CreateRootBridge
> >>     v4:
> >>     - Add ASSERT (FALSE) to default branch in GetTranslationByResourceType
> >>       [Laszlo]
> >>     - Fix bug when passing BaseAddress to gDS->AllocateIoSpace and
> >>       gDS->AllocateMemorySpace [Laszlo]
> >>     - Add comment for applying alignment to both device address and host
> >>       address, and add NOTE for the alignment requirement of Translation,
> >>       as well as in commit message [Laszlo][Ray]
> >>     - Improve indention for the code in CreateRootBridge [Laszlo]
> >>     - Improve comment for Translation in PCI_ROOT_BRIDGE_APERTURE
> >>       definition [Laszlo]
> >>     - Ignore translation of bus in CreateRootBridge
> >>
> >>  MdeModulePkg/Bus/Pci/PciHostBridgeDxe/PciHostBridge.h   |  21 ++++
> >>  MdeModulePkg/Bus/Pci/PciHostBridgeDxe/PciHostResource.h |   3 +
> >>  MdeModulePkg/Bus/Pci/PciHostBridgeDxe/PciHostBridge.c   | 129 +++++++++++++++++---
> >>  MdeModulePkg/Bus/Pci/PciHostBridgeDxe/PciRootBridgeIo.c | 118 ++++++++++++++++--
> >>  4 files changed, 245 insertions(+), 26 deletions(-)
> >>
> >>diff --git a/MdeModulePkg/Bus/Pci/PciHostBridgeDxe/PciHostBridge.h b/MdeModulePkg/Bus/Pci/PciHostBridgeDxe/PciHostBridge.h
> >>index 9a8ca21f4819..c2791ea5c2a4 100644
> >>--- a/MdeModulePkg/Bus/Pci/PciHostBridgeDxe/PciHostBridge.h
> >>+++ b/MdeModulePkg/Bus/Pci/PciHostBridgeDxe/PciHostBridge.h
> >>@@ -38,6 +38,13 @@ typedef struct {
> >>  #define PCI_HOST_BRIDGE_FROM_THIS(a) CR (a, PCI_HOST_BRIDGE_INSTANCE, ResAlloc, PCI_HOST_BRIDGE_SIGNATURE)
> >>  //
> >>+// Macros to translate device address to host address and vice versa. According
> >>+// to UEFI 2.7, device address = host address + translation offset.
> >>+//
> >>+#define TO_HOST_ADDRESS(DeviceAddress,TranslationOffset) ((DeviceAddress) - (TranslationOffset))
> >>+#define TO_DEVICE_ADDRESS(HostAddress,TranslationOffset) ((HostAddress) + (TranslationOffset))
> >>+
> >>+//
> >>  // Driver Entry Point
> >>  //
> >>  /**
> >>@@ -247,6 +254,20 @@ ResourceConflict (
> >>    IN  PCI_HOST_BRIDGE_INSTANCE *HostBridge
> >>    );
> >>+/**
> >>+  This routine gets translation offset from a root bridge instance by resource type.
> >>+
> >>+  @param RootBridge The Root Bridge Instance for the resources.
> >>+  @param ResourceType The Resource Type of the translation offset.
> >>+
> >>+  @retval The Translation Offset of the specified resource.
> >>+**/
> >>+UINT64
> >>+GetTranslationByResourceType (
> >>+  IN  PCI_ROOT_BRIDGE_INSTANCE     *RootBridge,
> >>+  IN  PCI_RESOURCE_TYPE            ResourceType
> >>+  );
> >>+
> >>  extern EFI_METRONOME_ARCH_PROTOCOL *mMetronome;
> >>  extern EFI_CPU_IO2_PROTOCOL        *mCpuIo;
> >>  #endif
> >>diff --git a/MdeModulePkg/Bus/Pci/PciHostBridgeDxe/PciHostResource.h b/MdeModulePkg/Bus/Pci/PciHostBridgeDxe/PciHostResource.h
> >>index 8612c0c3251b..a6c3739db368 100644
> >>--- a/MdeModulePkg/Bus/Pci/PciHostBridgeDxe/PciHostResource.h
> >>+++ b/MdeModulePkg/Bus/Pci/PciHostBridgeDxe/PciHostResource.h
> >>@@ -38,6 +38,9 @@ typedef enum {
> >>  typedef struct {
> >>    PCI_RESOURCE_TYPE Type;
> >>+  //
> >>+  // Base is a host address
> >>+  //
> >>    UINT64            Base;
> >>    UINT64            Length;
> >>    UINT64            Alignment;
> >>diff --git a/MdeModulePkg/Bus/Pci/PciHostBridgeDxe/PciHostBridge.c b/MdeModulePkg/Bus/Pci/PciHostBridgeDxe/PciHostBridge.c
> >>index 1494848c3e8c..42ded2855c71 100644
> >>--- a/MdeModulePkg/Bus/Pci/PciHostBridgeDxe/PciHostBridge.c
> >>+++ b/MdeModulePkg/Bus/Pci/PciHostBridgeDxe/PciHostBridge.c
> >>@@ -33,6 +33,39 @@ EFI_EVENT                   mIoMmuEvent;
> >>  VOID                        *mIoMmuRegistration;
> >>  /**
> >>+  This routine gets translation offset from a root bridge instance by resource type.
> >>+
> >>+  @param RootBridge The Root Bridge Instance for the resources.
> >>+  @param ResourceType The Resource Type of the translation offset.
> >>+
> >>+  @retval The Translation Offset of the specified resource.
> >>+**/
> >>+UINT64
> >>+GetTranslationByResourceType (
> >>+  IN  PCI_ROOT_BRIDGE_INSTANCE     *RootBridge,
> >>+  IN  PCI_RESOURCE_TYPE            ResourceType
> >>+  )
> >>+{
> >>+  switch (ResourceType) {
> >>+    case TypeIo:
> >>+      return RootBridge->Io.Translation;
> >>+    case TypeMem32:
> >>+      return RootBridge->Mem.Translation;
> >>+    case TypePMem32:
> >>+      return RootBridge->PMem.Translation;
> >>+    case TypeMem64:
> >>+      return RootBridge->MemAbove4G.Translation;
> >>+    case TypePMem64:
> >>+      return RootBridge->PMemAbove4G.Translation;
> >>+    case TypeBus:
> >>+      return RootBridge->Bus.Translation;
> >>+    default:
> >>+      ASSERT (FALSE);
> >>+      return 0;
> >>+  }
> >>+}
> >>+
> >>+/**
> >>    Ensure the compatibility of an IO space descriptor with the IO aperture.
> >>    The IO space descriptor can come from the GCD IO space map, or it can
> >>@@ -366,6 +399,7 @@ InitializePciHostBridge (
> >>    UINTN                       MemApertureIndex;
> >>    BOOLEAN                     ResourceAssigned;
> >>    LIST_ENTRY                  *Link;
> >>+  UINT64                      HostAddress;
> >>    RootBridges = PciHostBridgeGetRootBridges (&RootBridgeCount);
> >>    if ((RootBridges == NULL) || (RootBridgeCount == 0)) {
> >>@@ -411,8 +445,15 @@ InitializePciHostBridge (
> >>      }
> >>      if (RootBridges[Index].Io.Base <= RootBridges[Index].Io.Limit) {
> >>+      //
> >>+      // Base and Limit in PCI_ROOT_BRIDGE_APERTURE are device address.
> >>+      // For GCD resource manipulation, we need to use host address.
> >>+      //
> >>+      HostAddress = TO_HOST_ADDRESS (RootBridges[Index].Io.Base,
> >>+        RootBridges[Index].Io.Translation);
> >>+
> >>        Status = AddIoSpace (
> >>-                 RootBridges[Index].Io.Base,
> >>+                 HostAddress,
> >>                   RootBridges[Index].Io.Limit - RootBridges[Index].Io.Base + 1
> >>                   );
> >>        ASSERT_EFI_ERROR (Status);
> >>@@ -422,7 +463,7 @@ InitializePciHostBridge (
> >>                          EfiGcdIoTypeIo,
> >>                          0,
> >>                          RootBridges[Index].Io.Limit - RootBridges[Index].Io.Base + 1,
> >>-                        &RootBridges[Index].Io.Base,
> >>+                        &HostAddress,
> >>                          gImageHandle,
> >>                          NULL
> >>                          );
> >>@@ -443,14 +484,20 @@ InitializePciHostBridge (
> >>      for (MemApertureIndex = 0; MemApertureIndex < ARRAY_SIZE (MemApertures); MemApertureIndex++) {
> >>        if (MemApertures[MemApertureIndex]->Base <= MemApertures[MemApertureIndex]->Limit) {
> >>+        //
> >>+        // Base and Limit in PCI_ROOT_BRIDGE_APERTURE are device address.
> >>+        // For GCD resource manipulation, we need to use host address.
> >>+        //
> >>+        HostAddress = TO_HOST_ADDRESS (MemApertures[MemApertureIndex]->Base,
> >>+          MemApertures[MemApertureIndex]->Translation);
> >>          Status = AddMemoryMappedIoSpace (
> >>-                   MemApertures[MemApertureIndex]->Base,
> >>+                   HostAddress,
> >>                     MemApertures[MemApertureIndex]->Limit - MemApertures[MemApertureIndex]->Base + 1,
> >>                     EFI_MEMORY_UC
> >>                     );
> >>          ASSERT_EFI_ERROR (Status);
> >>          Status = gDS->SetMemorySpaceAttributes (
> >>-                        MemApertures[MemApertureIndex]->Base,
> >>+                        HostAddress,
> >>                          MemApertures[MemApertureIndex]->Limit - MemApertures[MemApertureIndex]->Base + 1,
> >>                          EFI_MEMORY_UC
> >>                          );
> >>@@ -463,7 +510,7 @@ InitializePciHostBridge (
> >>                            EfiGcdMemoryTypeMemoryMappedIo,
> >>                            0,
> >>                            MemApertures[MemApertureIndex]->Limit - MemApertures[MemApertureIndex]->Base + 1,
> >>-                          &MemApertures[MemApertureIndex]->Base,
> >>+                          &HostAddress,
> >>                            gImageHandle,
> >>                            NULL
> >>                            );
> >>@@ -654,6 +701,11 @@ AllocateResource (
> >>    if (BaseAddress < Limit) {
> >>      //
> >>      // Have to make sure Aligment is handled since we are doing direct address allocation
> >>+    // Strictly speaking, alignment requirement should be applied to device
> >>+    // address instead of host address which is used in GCD manipulation below,
> >>+    // but as we restrict the alignment of Translation to be larger than any BAR
> >>+    // alignment in the root bridge, we can simplify the situation and consider
> >>+    // the same alignment requirement is also applied to host address.
> >>      //
> >>      BaseAddress = ALIGN_VALUE (BaseAddress, LShiftU64 (1, BitsOfAlignment));
> >>@@ -721,6 +773,7 @@ NotifyPhase (
> >>    PCI_RESOURCE_TYPE                     Index2;
> >>    BOOLEAN                               ResNodeHandled[TypeMax];
> >>    UINT64                                MaxAlignment;
> >>+  UINT64                                Translation;
> >>    HostBridge = PCI_HOST_BRIDGE_FROM_THIS (This);
> >>@@ -822,14 +875,43 @@ NotifyPhase (
> >>            BitsOfAlignment = LowBitSet64 (Alignment + 1);
> >>            BaseAddress = MAX_UINT64;
> >>+          //
> >>+          // RESTRICTION: To simplify the situation, we require the alignment of
> >>+          // Translation must be larger than any BAR alignment in the same root
> >>+          // bridge, so that resource allocation alignment can be applied to
> >>+          // both device address and host address.
> >>+          //
> >>+          Translation = GetTranslationByResourceType (RootBridge, Index);
> >>+          if ((Translation & Alignment) != 0) {
> >>+            DEBUG ((DEBUG_ERROR, "[%a:%d] Translation %lx is not aligned to %lx!\n",
> >>+              __FUNCTION__, __LINE__, Translation, Alignment
> >>+              ));
> >>+            ASSERT (FALSE);
> >>+            //
> >>+            // This may be caused by too large alignment or too small
> >>+            // Translation; pick the 1st possibility and return out of resource,
> >>+            // which can also go thru the same process for out of resource
> >>+            // outside the loop.
> >>+            //
> >>+            ReturnStatus = EFI_OUT_OF_RESOURCES;
> >>+            continue;
> >>+          }
> >>+
> >>            switch (Index) {
> >>            case TypeIo:
> >>+            //
> >>+            // Base and Limit in PCI_ROOT_BRIDGE_APERTURE are device address.
> >>+            // For AllocateResource is manipulating GCD resource, we need to use
> >>+            // host address here.
> >>+            //
> >>              BaseAddress = AllocateResource (
> >>                              FALSE,
> >>                              RootBridge->ResAllocNode[Index].Length,
> >>                              MIN (15, BitsOfAlignment),
> >>-                            ALIGN_VALUE (RootBridge->Io.Base, Alignment + 1),
> >>-                            RootBridge->Io.Limit
> >>+                            TO_HOST_ADDRESS (ALIGN_VALUE (RootBridge->Io.Base, Alignment + 1),
> >>+                              RootBridge->Io.Translation),
> >>+                            TO_HOST_ADDRESS (RootBridge->Io.Limit,
> >>+                              RootBridge->Io.Translation)
> >>                              );
> >>              break;
> >>@@ -838,8 +920,10 @@ NotifyPhase (
> >>                              TRUE,
> >>                              RootBridge->ResAllocNode[Index].Length,
> >>                              MIN (63, BitsOfAlignment),
> >>-                            ALIGN_VALUE (RootBridge->MemAbove4G.Base, Alignment + 1),
> >>-                            RootBridge->MemAbove4G.Limit
> >>+                            TO_HOST_ADDRESS (ALIGN_VALUE (RootBridge->MemAbove4G.Base, Alignment + 1),
> >>+                              RootBridge->MemAbove4G.Translation),
> >>+                            TO_HOST_ADDRESS (RootBridge->MemAbove4G.Limit,
> >>+                              RootBridge->MemAbove4G.Translation)
> >>                              );
> >>              if (BaseAddress != MAX_UINT64) {
> >>                break;
> >>@@ -853,8 +937,10 @@ NotifyPhase (
> >>                              TRUE,
> >>                              RootBridge->ResAllocNode[Index].Length,
> >>                              MIN (31, BitsOfAlignment),
> >>-                            ALIGN_VALUE (RootBridge->Mem.Base, Alignment + 1),
> >>-                            RootBridge->Mem.Limit
> >>+                            TO_HOST_ADDRESS (ALIGN_VALUE (RootBridge->Mem.Base, Alignment + 1),
> >>+                              RootBridge->Mem.Translation),
> >>+                            TO_HOST_ADDRESS (RootBridge->Mem.Limit,
> >>+                              RootBridge->Mem.Translation)
> >>                              );
> >>              break;
> >>@@ -863,8 +949,10 @@ NotifyPhase (
> >>                              TRUE,
> >>                              RootBridge->ResAllocNode[Index].Length,
> >>                              MIN (63, BitsOfAlignment),
> >>-                            ALIGN_VALUE (RootBridge->PMemAbove4G.Base, Alignment + 1),
> >>-                            RootBridge->PMemAbove4G.Limit
> >>+                            TO_HOST_ADDRESS (ALIGN_VALUE (RootBridge->PMemAbove4G.Base, Alignment + 1),
> >>+                              RootBridge->PMemAbove4G.Translation),
> >>+                            TO_HOST_ADDRESS (RootBridge->PMemAbove4G.Limit,
> >>+                              RootBridge->PMemAbove4G.Translation)
> >>                              );
> >>              if (BaseAddress != MAX_UINT64) {
> >>                break;
> >>@@ -877,8 +965,10 @@ NotifyPhase (
> >>                              TRUE,
> >>                              RootBridge->ResAllocNode[Index].Length,
> >>                              MIN (31, BitsOfAlignment),
> >>-                            ALIGN_VALUE (RootBridge->PMem.Base, Alignment + 1),
> >>-                            RootBridge->PMem.Limit
> >>+                            TO_HOST_ADDRESS (ALIGN_VALUE (RootBridge->PMem.Base, Alignment + 1),
> >>+                              RootBridge->PMem.Translation),
> >>+                            TO_HOST_ADDRESS (RootBridge->PMem.Limit,
> >>+                              RootBridge->PMem.Translation)
> >>                              );
> >>              break;
> >>@@ -1421,7 +1511,14 @@ GetProposedResources (
> >>            Descriptor->Desc                  = ACPI_ADDRESS_SPACE_DESCRIPTOR;
> >>            Descriptor->Len                   = sizeof (EFI_ACPI_ADDRESS_SPACE_DESCRIPTOR) - 3;;
> >>            Descriptor->GenFlag               = 0;
> >>-          Descriptor->AddrRangeMin          = RootBridge->ResAllocNode[Index].Base;
> >>+          //
> >>+          // AddrRangeMin in Resource Descriptor here should be device address
> >>+          // instead of host address, or else PCI bus driver cannot set correct
> >>+          // address into PCI BAR registers.
> >>+          // Base in ResAllocNode is a host address, so conversion is needed.
> >>+          //
> >>+          Descriptor->AddrRangeMin          = TO_DEVICE_ADDRESS (RootBridge->ResAllocNode[Index].Base,
> >>+            GetTranslationByResourceType (RootBridge, Index));
> >>            Descriptor->AddrRangeMax          = 0;
> >>            Descriptor->AddrTranslationOffset = (ResStatus == ResAllocated) ? EFI_RESOURCE_SATISFIED : PCI_RESOURCE_LESS;
> >>            Descriptor->AddrLen               = RootBridge->ResAllocNode[Index].Length;
> >>diff --git a/MdeModulePkg/Bus/Pci/PciHostBridgeDxe/PciRootBridgeIo.c b/MdeModulePkg/Bus/Pci/PciHostBridgeDxe/PciRootBridgeIo.c
> >>index dc06c16dc038..5dd9d257d46d 100644
> >>--- a/MdeModulePkg/Bus/Pci/PciHostBridgeDxe/PciRootBridgeIo.c
> >>+++ b/MdeModulePkg/Bus/Pci/PciHostBridgeDxe/PciRootBridgeIo.c
> >>@@ -86,12 +86,35 @@ CreateRootBridge (
> >>            (Bridge->AllocationAttributes & EFI_PCI_HOST_BRIDGE_COMBINE_MEM_PMEM) != 0 ? L"CombineMemPMem " : L"",
> >>            (Bridge->AllocationAttributes & EFI_PCI_HOST_BRIDGE_MEM64_DECODE) != 0 ? L"Mem64Decode" : L""
> >>            ));
> >>+  //
> >>+  // Translation for bus is not supported.
> >>+  //
> >>    DEBUG ((EFI_D_INFO, "           Bus: %lx - %lx\n", Bridge->Bus.Base, Bridge->Bus.Limit));
> >>-  DEBUG ((EFI_D_INFO, "            Io: %lx - %lx\n", Bridge->Io.Base, Bridge->Io.Limit));
> >>-  DEBUG ((EFI_D_INFO, "           Mem: %lx - %lx\n", Bridge->Mem.Base, Bridge->Mem.Limit));
> >>-  DEBUG ((EFI_D_INFO, "    MemAbove4G: %lx - %lx\n", Bridge->MemAbove4G.Base, Bridge->MemAbove4G.Limit));
> >>-  DEBUG ((EFI_D_INFO, "          PMem: %lx - %lx\n", Bridge->PMem.Base, Bridge->PMem.Limit));
> >>-  DEBUG ((EFI_D_INFO, "   PMemAbove4G: %lx - %lx\n", Bridge->PMemAbove4G.Base, Bridge->PMemAbove4G.Limit));
> >>+  ASSERT (Bridge->Bus.Translation == 0);
> >>+  if (Bridge->Bus.Translation != 0) {
> >>+    return NULL;
> >>+  }
> >>+
> >>+  DEBUG ((
> >>+    DEBUG_INFO, "            Io: %lx - %lx Translation=%lx\n",
> >>+    Bridge->Io.Base, Bridge->Io.Limit, Bridge->Io.Translation
> >>+    ));
> >>+  DEBUG ((
> >>+    DEBUG_INFO, "           Mem: %lx - %lx Translation=%lx\n",
> >>+    Bridge->Mem.Base, Bridge->Mem.Limit, Bridge->Mem.Translation
> >>+    ));
> >>+  DEBUG ((
> >>+    DEBUG_INFO, "    MemAbove4G: %lx - %lx Translation=%lx\n",
> >>+    Bridge->MemAbove4G.Base, Bridge->MemAbove4G.Limit, Bridge->MemAbove4G.Translation
> >>+    ));
> >>+  DEBUG ((
> >>+    DEBUG_INFO, "          PMem: %lx - %lx Translation=%lx\n",
> >>+    Bridge->PMem.Base, Bridge->PMem.Limit, Bridge->PMem.Translation
> >>+    ));
> >>+  DEBUG ((
> >>+    DEBUG_INFO, "   PMemAbove4G: %lx - %lx Translation=%lx\n",
> >>+    Bridge->PMemAbove4G.Base, Bridge->PMemAbove4G.Limit, Bridge->PMemAbove4G.Translation
> >>+    ));
> >>    //
> >>    // Make sure Mem and MemAbove4G apertures are valid
> >>@@ -206,7 +229,12 @@ CreateRootBridge (
> >>      }
> >>      RootBridge->ResAllocNode[Index].Type     = Index;
> >>      if (Bridge->ResourceAssigned && (Aperture->Limit >= Aperture->Base)) {
> >>-      RootBridge->ResAllocNode[Index].Base   = Aperture->Base;
> >>+      //
> >>+      // Base in ResAllocNode is a host address, while Base in Aperture is a
> >>+      // device address.
> >>+      //
> >>+      RootBridge->ResAllocNode[Index].Base   = TO_HOST_ADDRESS (Aperture->Base,
> >>+        Aperture->Translation);
> >>        RootBridge->ResAllocNode[Index].Length = Aperture->Limit - Aperture->Base + 1;
> >>        RootBridge->ResAllocNode[Index].Status = ResAllocated;
> >>      } else {
> >>@@ -403,6 +431,28 @@ RootBridgeIoCheckParameter (
> >>    return EFI_SUCCESS;
> >>  }
> >>+EFI_STATUS
> >>+RootBridgeIoGetMemTranslationByAddress (
> >>+  IN PCI_ROOT_BRIDGE_INSTANCE               *RootBridge,
> >>+  IN UINT64                                 Address,
> >>+  IN OUT UINT64                             *Translation
> >>+  )
> >>+{
> >>+  if (Address >= RootBridge->Mem.Base && Address <= RootBridge->Mem.Limit) {
> >>+    *Translation = RootBridge->Mem.Translation;
> >>+  } else if (Address >= RootBridge->PMem.Base && Address <= RootBridge->PMem.Limit) {
> >>+    *Translation = RootBridge->PMem.Translation;
> >>+  } else if (Address >= RootBridge->MemAbove4G.Base && Address <= RootBridge->MemAbove4G.Limit) {
> >>+    *Translation = RootBridge->MemAbove4G.Translation;
> >>+  } else if (Address >= RootBridge->PMemAbove4G.Base && Address <= RootBridge->PMemAbove4G.Limit) {
> >>+    *Translation = RootBridge->PMemAbove4G.Translation;
> >>+  } else {
> >>+    return EFI_INVALID_PARAMETER;
> >>+  }
> >>+
> >>+  return EFI_SUCCESS;
> >>+}
> >>+
> >>  /**
> >>    Polls an address in memory mapped I/O space until an exit condition is met,
> >>    or a timeout occurs.
> >>@@ -658,13 +708,25 @@ RootBridgeIoMemRead (
> >>    )
> >>  {
> >>    EFI_STATUS                             Status;
> >>+  PCI_ROOT_BRIDGE_INSTANCE               *RootBridge;
> >>+  UINT64                                 Translation;
> >>    Status = RootBridgeIoCheckParameter (This, MemOperation, Width, Address,
> >>                                         Count, Buffer);
> >>    if (EFI_ERROR (Status)) {
> >>      return Status;
> >>    }
> >>-  return mCpuIo->Mem.Read (mCpuIo, (EFI_CPU_IO_PROTOCOL_WIDTH) Width, Address, Count, Buffer);
> >>+
> >>+  RootBridge = ROOT_BRIDGE_FROM_THIS (This);
> >>+  Status = RootBridgeIoGetMemTranslationByAddress (RootBridge, Address, &Translation);
> >>+  if (EFI_ERROR (Status)) {
> >>+    return Status;
> >>+  }
> >>+
> >>+  // Address passed to CpuIo->Mem.Read needs to be a host address instead of
> >>+  // device address.
> >>+  return mCpuIo->Mem.Read (mCpuIo, (EFI_CPU_IO_PROTOCOL_WIDTH) Width,
> >>+      TO_HOST_ADDRESS (Address, Translation), Count, Buffer);
> >>  }
> >>  /**
> >>@@ -705,13 +767,25 @@ RootBridgeIoMemWrite (
> >>    )
> >>  {
> >>    EFI_STATUS                             Status;
> >>+  PCI_ROOT_BRIDGE_INSTANCE               *RootBridge;
> >>+  UINT64                                 Translation;
> >>    Status = RootBridgeIoCheckParameter (This, MemOperation, Width, Address,
> >>                                         Count, Buffer);
> >>    if (EFI_ERROR (Status)) {
> >>      return Status;
> >>    }
> >>-  return mCpuIo->Mem.Write (mCpuIo, (EFI_CPU_IO_PROTOCOL_WIDTH) Width, Address, Count, Buffer);
> >>+
> >>+  RootBridge = ROOT_BRIDGE_FROM_THIS (This);
> >>+  Status = RootBridgeIoGetMemTranslationByAddress (RootBridge, Address, &Translation);
> >>+  if (EFI_ERROR (Status)) {
> >>+    return Status;
> >>+  }
> >>+
> >>+  // Address passed to CpuIo->Mem.Write needs to be a host address instead of
> >>+  // device address.
> >>+  return mCpuIo->Mem.Write (mCpuIo, (EFI_CPU_IO_PROTOCOL_WIDTH) Width,
> >>+      TO_HOST_ADDRESS (Address, Translation), Count, Buffer);
> >>  }
> >>  /**
> >>@@ -746,6 +820,8 @@ RootBridgeIoIoRead (
> >>    )
> >>  {
> >>    EFI_STATUS                                    Status;
> >>+  PCI_ROOT_BRIDGE_INSTANCE                      *RootBridge;
> >>+
> >>    Status = RootBridgeIoCheckParameter (
> >>               This, IoOperation, Width,
> >>               Address, Count, Buffer
> >>@@ -753,7 +829,13 @@ RootBridgeIoIoRead (
> >>    if (EFI_ERROR (Status)) {
> >>      return Status;
> >>    }
> >>-  return mCpuIo->Io.Read (mCpuIo, (EFI_CPU_IO_PROTOCOL_WIDTH) Width, Address, Count, Buffer);
> >>+
> >>+  RootBridge = ROOT_BRIDGE_FROM_THIS (This);
> >>+
> >>+  // Address passed to CpuIo->Io.Read needs to be a host address instead of
> >>+  // device address.
> >>+  return mCpuIo->Io.Read (mCpuIo, (EFI_CPU_IO_PROTOCOL_WIDTH) Width,
> >>+      TO_HOST_ADDRESS (Address, RootBridge->Io.Translation), Count, Buffer);
> >>  }
> >>  /**
> >>@@ -788,6 +870,8 @@ RootBridgeIoIoWrite (
> >>    )
> >>  {
> >>    EFI_STATUS                                    Status;
> >>+  PCI_ROOT_BRIDGE_INSTANCE                      *RootBridge;
> >>+
> >>    Status = RootBridgeIoCheckParameter (
> >>               This, IoOperation, Width,
> >>               Address, Count, Buffer
> >>@@ -795,7 +879,13 @@ RootBridgeIoIoWrite (
> >>    if (EFI_ERROR (Status)) {
> >>      return Status;
> >>    }
> >>-  return mCpuIo->Io.Write (mCpuIo, (EFI_CPU_IO_PROTOCOL_WIDTH) Width, Address, Count, Buffer);
> >>+
> >>+  RootBridge = ROOT_BRIDGE_FROM_THIS (This);
> >>+
> >>+  // Address passed to CpuIo->Io.Write needs to be a host address instead of
> >>+  // device address.
> >>+  return mCpuIo->Io.Write (mCpuIo, (EFI_CPU_IO_PROTOCOL_WIDTH) Width,
> >>+      TO_HOST_ADDRESS (Address, RootBridge->Io.Translation), Count, Buffer);
> >>  }
> >>  /**
> >>@@ -1615,9 +1705,17 @@ RootBridgeIoConfiguration (
> >>      Descriptor->Desc = ACPI_ADDRESS_SPACE_DESCRIPTOR;
> >>      Descriptor->Len  = sizeof (EFI_ACPI_ADDRESS_SPACE_DESCRIPTOR) - 3;
> >>+    // According to UEFI 2.7, RootBridgeIo->Configuration should return address
> >>+    // range in CPU view (host address), and ResAllocNode->Base is already a CPU
> >>+    // view address (host address).
> >>      Descriptor->AddrRangeMin  = ResAllocNode->Base;
> >>      Descriptor->AddrRangeMax  = ResAllocNode->Base + ResAllocNode->Length - 1;
> >>      Descriptor->AddrLen       = ResAllocNode->Length;
> >>+    Descriptor->AddrTranslationOffset = GetTranslationByResourceType (
> >>+      RootBridge,
> >>+      ResAllocNode->Type
> >>+      );
> >>+
> >>      switch (ResAllocNode->Type) {
> >>      case TypeIo:
> >>-- 
> >>2.7.4
> >>
> 
> 
> -- 
> Thanks,
> Ray

Re: [PATCH v5 4/6] MdeModulePkg/PciHostBridgeDxe: Add support for address translation

[Ni, Ruiyu]

On 3/7/2018 2:01 PM, Guo Heyi wrote:
> Thanks. Please let me know if any further changes are needed.
> 
> Regards,
> 
> Heyi
> 
> On Wed, Mar 07, 2018 at 12:30:59PM +0800, Ni, Ruiyu wrote:
>> On 3/6/2018 10:44 AM, Guo Heyi wrote:
>>> Hi Ray,
>>>
>>> Any comments for v5?
>>
>> Heyi,
>> Some backward compatibility concerns were received from internal production
>> teams. Current change will cause silent failure on old platforms because
>> TranslationOffset might be random if uninitialized.
>> I will solve the concern and then send out updates to you, hopefully by end
>> of next week.
>>
>>>
>>> Regards,
>>>
>>> Heyi
>>>
>>> On Thu, Mar 01, 2018 at 02:57:22PM +0800, Heyi Guo wrote:
>>>> PCI address translation is necessary for some non-x86 platforms. On
>>>> such platforms, address value (denoted as "device address" or "address
>>>> in PCI view") set to PCI BAR registers in configuration space might be
>>>> different from the address which is used by CPU to access the
>>>> registers in memory BAR or IO BAR spaces (denoted as "host address" or
>>>> "address in CPU view"). The difference between the two addresses is
>>>> called "Address Translation Offset" or simply "translation", and can
>>>> be represented by "Address Translation Offset" in ACPI QWORD Address
>>>> Space Descriptor (Offset 0x1E). However UEFI and ACPI differs on the
>>>> definitions of QWORD Address Space Descriptor, and we will follow UEFI
>>>> definition on UEFI protocols, such as PCI root bridge IO protocol and
>>>> PCI IO protocol. In UEFI 2.7, "Address Translation Offset" is "Offset
>>>> to apply to the Starting address to convert it to a PCI address". This
>>>> means:
>>>>
>>>> 1. Translation = device address - host address.
>>>>
>>>> 2. PciRootBridgeIo->Configuration should return CPU view address, as
>>>> well as PciIo->GetBarAttributes.
>>>>
>>>> Summary of addresses used in protocol interfaces and internal
>>>> implementations:
>>>>
>>>> 1. *Only* the following protocol interfaces assume Address is Device
>>>>     Address:
>>>> (1). PciHostBridgeResourceAllocation.GetProposedResources()
>>>>       Otherwise PCI bus driver cannot set correct address into PCI
>>>>       BARs.
>>>> (2). PciRootBridgeIo.Mem.Read() and PciRootBridgeIo.Mem.Write()
>>>> (3). PciRootBridgeIo.CopyMem()
>>>> UEFI and PI spec have clear statements for all other protocol
>>>> interfaces about the address type.
>>>>
>>>> 2. Library interfaces and internal implementation:
>>>> (1). Base and Limit in PCI_ROOT_BRIDGE_APERTURE are device address.
>>>>       It is easy to check whether the address is below 4G or above 4G.
>>>> (2). Addresses in PCI_ROOT_BRIDGE_INSTANCE.ResAllocNode are host
>>>>       address, for they are allocated from GCD.
>>>> (3). Address passed to PciHostBridgeResourceConflict is host address,
>>>>       for it comes from PCI_ROOT_BRIDGE_INSTANCE.ResAllocNode.
>>>>
>>>> RESTRICTION: to simplify the situation, we require the alignment of
>>>> Translation must be larger than any BAR alignment in the same root
>>>> bridge, so that resource allocation alignment can be applied to both
>>>> device address and host address.
>>>>
>>>> Contributed-under: TianoCore Contribution Agreement 1.1
>>>> Signed-off-by: Heyi Guo <heyi.guo@linaro.org>
>>>> Cc: Ruiyu Ni <ruiyu.ni@intel.com>
>>>> Cc: Ard Biesheuvel <ard.biesheuvel@linaro.org>
>>>> Cc: Star Zeng <star.zeng@intel.com>
>>>> Cc: Eric Dong <eric.dong@intel.com>
>>>> Cc: Laszlo Ersek <lersek@redhat.com>
>>>> Cc: Michael D Kinney <michael.d.kinney@intel.com>
>>>> ---
>>>>
>>>> Notes:
>>>>      v5:
>>>>      - Add check for the alignment of Translation against the BAR alignment
>>>>        [Ray]
>>>>      - Keep coding style of comments consistent with the context [Ray]
>>>>      - Comment change for Base in PCI_RES_NODE [Ray]
>>>>      - Add macros of TO_HOST_ADDRESS and TO_DEVICE_ADDRESS for address type
>>>>        conversion (After that we can also simply the comments about the
>>>>        calculation [Ray]
>>>>      - Add check for bus translation offset in CreateRootBridge(), making
>>>>        sure it is zero, and unify code logic for all types of resource
>>>>        after that [Ray]
>>>>      - Use GetTranslationByResourceType() to simplify the code in
>>>>        RootBridgeIoConfiguration() (also fix a bug in previous patch
>>>>        version of missing a break after case TypePMem64) [Ray]
>>>>      - Commit message refinement [Ray]
>>>>      v4:
>>>>      - Add ASSERT (FALSE) to default branch in GetTranslationByResourceType
>>>>        [Laszlo]
>>>>      - Fix bug when passing BaseAddress to gDS->AllocateIoSpace and
>>>>        gDS->AllocateMemorySpace [Laszlo]
>>>>      - Add comment for applying alignment to both device address and host
>>>>        address, and add NOTE for the alignment requirement of Translation,
>>>>        as well as in commit message [Laszlo][Ray]
>>>>      - Improve indention for the code in CreateRootBridge [Laszlo]
>>>>      - Improve comment for Translation in PCI_ROOT_BRIDGE_APERTURE
>>>>        definition [Laszlo]
>>>>      - Ignore translation of bus in CreateRootBridge
>>>>      v4:
>>>>      - Add ASSERT (FALSE) to default branch in GetTranslationByResourceType
>>>>        [Laszlo]
>>>>      - Fix bug when passing BaseAddress to gDS->AllocateIoSpace and
>>>>        gDS->AllocateMemorySpace [Laszlo]
>>>>      - Add comment for applying alignment to both device address and host
>>>>        address, and add NOTE for the alignment requirement of Translation,
>>>>        as well as in commit message [Laszlo][Ray]
>>>>      - Improve indention for the code in CreateRootBridge [Laszlo]
>>>>      - Improve comment for Translation in PCI_ROOT_BRIDGE_APERTURE
>>>>        definition [Laszlo]
>>>>      - Ignore translation of bus in CreateRootBridge
>>>>
>>>>   MdeModulePkg/Bus/Pci/PciHostBridgeDxe/PciHostBridge.h   |  21 ++++
>>>>   MdeModulePkg/Bus/Pci/PciHostBridgeDxe/PciHostResource.h |   3 +
>>>>   MdeModulePkg/Bus/Pci/PciHostBridgeDxe/PciHostBridge.c   | 129 +++++++++++++++++---
>>>>   MdeModulePkg/Bus/Pci/PciHostBridgeDxe/PciRootBridgeIo.c | 118 ++++++++++++++++--
>>>>   4 files changed, 245 insertions(+), 26 deletions(-)
>>>>
>>>> diff --git a/MdeModulePkg/Bus/Pci/PciHostBridgeDxe/PciHostBridge.h b/MdeModulePkg/Bus/Pci/PciHostBridgeDxe/PciHostBridge.h
>>>> index 9a8ca21f4819..c2791ea5c2a4 100644
>>>> --- a/MdeModulePkg/Bus/Pci/PciHostBridgeDxe/PciHostBridge.h
>>>> +++ b/MdeModulePkg/Bus/Pci/PciHostBridgeDxe/PciHostBridge.h
>>>> @@ -38,6 +38,13 @@ typedef struct {
>>>>   #define PCI_HOST_BRIDGE_FROM_THIS(a) CR (a, PCI_HOST_BRIDGE_INSTANCE, ResAlloc, PCI_HOST_BRIDGE_SIGNATURE)
>>>>   //
>>>> +// Macros to translate device address to host address and vice versa. According
>>>> +// to UEFI 2.7, device address = host address + translation offset.
>>>> +//
>>>> +#define TO_HOST_ADDRESS(DeviceAddress,TranslationOffset) ((DeviceAddress) - (TranslationOffset))
>>>> +#define TO_DEVICE_ADDRESS(HostAddress,TranslationOffset) ((HostAddress) + (TranslationOffset))
>>>> +
>>>> +//
>>>>   // Driver Entry Point
>>>>   //
>>>>   /**
>>>> @@ -247,6 +254,20 @@ ResourceConflict (
>>>>     IN  PCI_HOST_BRIDGE_INSTANCE *HostBridge
>>>>     );
>>>> +/**
>>>> +  This routine gets translation offset from a root bridge instance by resource type.
>>>> +
>>>> +  @param RootBridge The Root Bridge Instance for the resources.
>>>> +  @param ResourceType The Resource Type of the translation offset.
>>>> +
>>>> +  @retval The Translation Offset of the specified resource.
>>>> +**/
>>>> +UINT64
>>>> +GetTranslationByResourceType (
>>>> +  IN  PCI_ROOT_BRIDGE_INSTANCE     *RootBridge,
>>>> +  IN  PCI_RESOURCE_TYPE            ResourceType
>>>> +  );
>>>> +
>>>>   extern EFI_METRONOME_ARCH_PROTOCOL *mMetronome;
>>>>   extern EFI_CPU_IO2_PROTOCOL        *mCpuIo;
>>>>   #endif
>>>> diff --git a/MdeModulePkg/Bus/Pci/PciHostBridgeDxe/PciHostResource.h b/MdeModulePkg/Bus/Pci/PciHostBridgeDxe/PciHostResource.h
>>>> index 8612c0c3251b..a6c3739db368 100644
>>>> --- a/MdeModulePkg/Bus/Pci/PciHostBridgeDxe/PciHostResource.h
>>>> +++ b/MdeModulePkg/Bus/Pci/PciHostBridgeDxe/PciHostResource.h
>>>> @@ -38,6 +38,9 @@ typedef enum {
>>>>   typedef struct {
>>>>     PCI_RESOURCE_TYPE Type;
>>>> +  //
>>>> +  // Base is a host address
>>>> +  //
>>>>     UINT64            Base;
>>>>     UINT64            Length;
>>>>     UINT64            Alignment;
>>>> diff --git a/MdeModulePkg/Bus/Pci/PciHostBridgeDxe/PciHostBridge.c b/MdeModulePkg/Bus/Pci/PciHostBridgeDxe/PciHostBridge.c
>>>> index 1494848c3e8c..42ded2855c71 100644
>>>> --- a/MdeModulePkg/Bus/Pci/PciHostBridgeDxe/PciHostBridge.c
>>>> +++ b/MdeModulePkg/Bus/Pci/PciHostBridgeDxe/PciHostBridge.c
>>>> @@ -33,6 +33,39 @@ EFI_EVENT                   mIoMmuEvent;
>>>>   VOID                        *mIoMmuRegistration;
>>>>   /**
>>>> +  This routine gets translation offset from a root bridge instance by resource type.
>>>> +
>>>> +  @param RootBridge The Root Bridge Instance for the resources.
>>>> +  @param ResourceType The Resource Type of the translation offset.
>>>> +
>>>> +  @retval The Translation Offset of the specified resource.
>>>> +**/
>>>> +UINT64
>>>> +GetTranslationByResourceType (
>>>> +  IN  PCI_ROOT_BRIDGE_INSTANCE     *RootBridge,
>>>> +  IN  PCI_RESOURCE_TYPE            ResourceType
>>>> +  )
>>>> +{
>>>> +  switch (ResourceType) {
>>>> +    case TypeIo:
>>>> +      return RootBridge->Io.Translation;
>>>> +    case TypeMem32:
>>>> +      return RootBridge->Mem.Translation;
>>>> +    case TypePMem32:
>>>> +      return RootBridge->PMem.Translation;
>>>> +    case TypeMem64:
>>>> +      return RootBridge->MemAbove4G.Translation;
>>>> +    case TypePMem64:
>>>> +      return RootBridge->PMemAbove4G.Translation;
>>>> +    case TypeBus:
>>>> +      return RootBridge->Bus.Translation;
>>>> +    default:
>>>> +      ASSERT (FALSE);
>>>> +      return 0;
>>>> +  }
>>>> +}
>>>> +
>>>> +/**
>>>>     Ensure the compatibility of an IO space descriptor with the IO aperture.
>>>>     The IO space descriptor can come from the GCD IO space map, or it can
>>>> @@ -366,6 +399,7 @@ InitializePciHostBridge (
>>>>     UINTN                       MemApertureIndex;
>>>>     BOOLEAN                     ResourceAssigned;
>>>>     LIST_ENTRY                  *Link;
>>>> +  UINT64                      HostAddress;
>>>>     RootBridges = PciHostBridgeGetRootBridges (&RootBridgeCount);
>>>>     if ((RootBridges == NULL) || (RootBridgeCount == 0)) {
>>>> @@ -411,8 +445,15 @@ InitializePciHostBridge (
>>>>       }
>>>>       if (RootBridges[Index].Io.Base <= RootBridges[Index].Io.Limit) {
>>>> +      //
>>>> +      // Base and Limit in PCI_ROOT_BRIDGE_APERTURE are device address.
>>>> +      // For GCD resource manipulation, we need to use host address.
>>>> +      //
>>>> +      HostAddress = TO_HOST_ADDRESS (RootBridges[Index].Io.Base,
>>>> +        RootBridges[Index].Io.Translation);
>>>> +
>>>>         Status = AddIoSpace (
>>>> -                 RootBridges[Index].Io.Base,
>>>> +                 HostAddress,
>>>>                    RootBridges[Index].Io.Limit - RootBridges[Index].Io.Base + 1
>>>>                    );
>>>>         ASSERT_EFI_ERROR (Status);
>>>> @@ -422,7 +463,7 @@ InitializePciHostBridge (
>>>>                           EfiGcdIoTypeIo,
>>>>                           0,
>>>>                           RootBridges[Index].Io.Limit - RootBridges[Index].Io.Base + 1,
>>>> -                        &RootBridges[Index].Io.Base,
>>>> +                        &HostAddress,
>>>>                           gImageHandle,
>>>>                           NULL
>>>>                           );
>>>> @@ -443,14 +484,20 @@ InitializePciHostBridge (
>>>>       for (MemApertureIndex = 0; MemApertureIndex < ARRAY_SIZE (MemApertures); MemApertureIndex++) {
>>>>         if (MemApertures[MemApertureIndex]->Base <= MemApertures[MemApertureIndex]->Limit) {
>>>> +        //
>>>> +        // Base and Limit in PCI_ROOT_BRIDGE_APERTURE are device address.
>>>> +        // For GCD resource manipulation, we need to use host address.
>>>> +        //
>>>> +        HostAddress = TO_HOST_ADDRESS (MemApertures[MemApertureIndex]->Base,
>>>> +          MemApertures[MemApertureIndex]->Translation);
>>>>           Status = AddMemoryMappedIoSpace (
>>>> -                   MemApertures[MemApertureIndex]->Base,
>>>> +                   HostAddress,
>>>>                      MemApertures[MemApertureIndex]->Limit - MemApertures[MemApertureIndex]->Base + 1,
>>>>                      EFI_MEMORY_UC
>>>>                      );
>>>>           ASSERT_EFI_ERROR (Status);
>>>>           Status = gDS->SetMemorySpaceAttributes (
>>>> -                        MemApertures[MemApertureIndex]->Base,
>>>> +                        HostAddress,
>>>>                           MemApertures[MemApertureIndex]->Limit - MemApertures[MemApertureIndex]->Base + 1,
>>>>                           EFI_MEMORY_UC
>>>>                           );
>>>> @@ -463,7 +510,7 @@ InitializePciHostBridge (
>>>>                             EfiGcdMemoryTypeMemoryMappedIo,
>>>>                             0,
>>>>                             MemApertures[MemApertureIndex]->Limit - MemApertures[MemApertureIndex]->Base + 1,
>>>> -                          &MemApertures[MemApertureIndex]->Base,
>>>> +                          &HostAddress,
>>>>                             gImageHandle,
>>>>                             NULL
>>>>                             );
>>>> @@ -654,6 +701,11 @@ AllocateResource (
>>>>     if (BaseAddress < Limit) {
>>>>       //
>>>>       // Have to make sure Aligment is handled since we are doing direct address allocation
>>>> +    // Strictly speaking, alignment requirement should be applied to device
>>>> +    // address instead of host address which is used in GCD manipulation below,
>>>> +    // but as we restrict the alignment of Translation to be larger than any BAR
>>>> +    // alignment in the root bridge, we can simplify the situation and consider
>>>> +    // the same alignment requirement is also applied to host address.
>>>>       //
>>>>       BaseAddress = ALIGN_VALUE (BaseAddress, LShiftU64 (1, BitsOfAlignment));
>>>> @@ -721,6 +773,7 @@ NotifyPhase (
>>>>     PCI_RESOURCE_TYPE                     Index2;
>>>>     BOOLEAN                               ResNodeHandled[TypeMax];
>>>>     UINT64                                MaxAlignment;
>>>> +  UINT64                                Translation;
>>>>     HostBridge = PCI_HOST_BRIDGE_FROM_THIS (This);
>>>> @@ -822,14 +875,43 @@ NotifyPhase (
>>>>             BitsOfAlignment = LowBitSet64 (Alignment + 1);
>>>>             BaseAddress = MAX_UINT64;
>>>> +          //
>>>> +          // RESTRICTION: To simplify the situation, we require the alignment of
>>>> +          // Translation must be larger than any BAR alignment in the same root
>>>> +          // bridge, so that resource allocation alignment can be applied to
>>>> +          // both device address and host address.
>>>> +          //
>>>> +          Translation = GetTranslationByResourceType (RootBridge, Index);
>>>> +          if ((Translation & Alignment) != 0) {
>>>> +            DEBUG ((DEBUG_ERROR, "[%a:%d] Translation %lx is not aligned to %lx!\n",
>>>> +              __FUNCTION__, __LINE__, Translation, Alignment
>>>> +              ));
>>>> +            ASSERT (FALSE);
1. Cool! But can you replace ASSERT(FALSE) with
    ASSERT ((Translation & Alignment) == 0)?
>>>> +            //
>>>> +            // This may be caused by too large alignment or too small
>>>> +            // Translation; pick the 1st possibility and return out of resource,
>>>> +            // which can also go thru the same process for out of resource
>>>> +            // outside the loop.
>>>> +            //
>>>> +            ReturnStatus = EFI_OUT_OF_RESOURCES;
>>>> +            continue;
>>>> +          }
>>>> +
>>>>             switch (Index) {
>>>>             case TypeIo:
>>>> +            //
>>>> +            // Base and Limit in PCI_ROOT_BRIDGE_APERTURE are device address.
>>>> +            // For AllocateResource is manipulating GCD resource, we need to use
>>>> +            // host address here.
>>>> +            //
>>>>               BaseAddress = AllocateResource (
>>>>                               FALSE,
>>>>                               RootBridge->ResAllocNode[Index].Length,
>>>>                               MIN (15, BitsOfAlignment),
>>>> -                            ALIGN_VALUE (RootBridge->Io.Base, Alignment + 1),
>>>> -                            RootBridge->Io.Limit
>>>> +                            TO_HOST_ADDRESS (ALIGN_VALUE (RootBridge->Io.Base, Alignment + 1),
>>>> +                              RootBridge->Io.Translation),
>>>> +                            TO_HOST_ADDRESS (RootBridge->Io.Limit,
>>>> +                              RootBridge->Io.Translation)
>>>>                               );
>>>>               break;
>>>> @@ -838,8 +920,10 @@ NotifyPhase (
>>>>                               TRUE,
>>>>                               RootBridge->ResAllocNode[Index].Length,
>>>>                               MIN (63, BitsOfAlignment),
>>>> -                            ALIGN_VALUE (RootBridge->MemAbove4G.Base, Alignment + 1),
>>>> -                            RootBridge->MemAbove4G.Limit
>>>> +                            TO_HOST_ADDRESS (ALIGN_VALUE (RootBridge->MemAbove4G.Base, Alignment + 1),
>>>> +                              RootBridge->MemAbove4G.Translation),
>>>> +                            TO_HOST_ADDRESS (RootBridge->MemAbove4G.Limit,
>>>> +                              RootBridge->MemAbove4G.Translation)
>>>>                               );
>>>>               if (BaseAddress != MAX_UINT64) {
>>>>                 break;
>>>> @@ -853,8 +937,10 @@ NotifyPhase (
>>>>                               TRUE,
>>>>                               RootBridge->ResAllocNode[Index].Length,
>>>>                               MIN (31, BitsOfAlignment),
>>>> -                            ALIGN_VALUE (RootBridge->Mem.Base, Alignment + 1),
>>>> -                            RootBridge->Mem.Limit
>>>> +                            TO_HOST_ADDRESS (ALIGN_VALUE (RootBridge->Mem.Base, Alignment + 1),
>>>> +                              RootBridge->Mem.Translation),
>>>> +                            TO_HOST_ADDRESS (RootBridge->Mem.Limit,
>>>> +                              RootBridge->Mem.Translation)
>>>>                               );
>>>>               break;
>>>> @@ -863,8 +949,10 @@ NotifyPhase (
>>>>                               TRUE,
>>>>                               RootBridge->ResAllocNode[Index].Length,
>>>>                               MIN (63, BitsOfAlignment),
>>>> -                            ALIGN_VALUE (RootBridge->PMemAbove4G.Base, Alignment + 1),
>>>> -                            RootBridge->PMemAbove4G.Limit
>>>> +                            TO_HOST_ADDRESS (ALIGN_VALUE (RootBridge->PMemAbove4G.Base, Alignment + 1),
>>>> +                              RootBridge->PMemAbove4G.Translation),
>>>> +                            TO_HOST_ADDRESS (RootBridge->PMemAbove4G.Limit,
>>>> +                              RootBridge->PMemAbove4G.Translation)
>>>>                               );
>>>>               if (BaseAddress != MAX_UINT64) {
>>>>                 break;
>>>> @@ -877,8 +965,10 @@ NotifyPhase (
>>>>                               TRUE,
>>>>                               RootBridge->ResAllocNode[Index].Length,
>>>>                               MIN (31, BitsOfAlignment),
>>>> -                            ALIGN_VALUE (RootBridge->PMem.Base, Alignment + 1),
>>>> -                            RootBridge->PMem.Limit
>>>> +                            TO_HOST_ADDRESS (ALIGN_VALUE (RootBridge->PMem.Base, Alignment + 1),
>>>> +                              RootBridge->PMem.Translation),
>>>> +                            TO_HOST_ADDRESS (RootBridge->PMem.Limit,
>>>> +                              RootBridge->PMem.Translation)
>>>>                               );
>>>>               break;
>>>> @@ -1421,7 +1511,14 @@ GetProposedResources (
>>>>             Descriptor->Desc                  = ACPI_ADDRESS_SPACE_DESCRIPTOR;
>>>>             Descriptor->Len                   = sizeof (EFI_ACPI_ADDRESS_SPACE_DESCRIPTOR) - 3;;
>>>>             Descriptor->GenFlag               = 0;
>>>> -          Descriptor->AddrRangeMin          = RootBridge->ResAllocNode[Index].Base;
>>>> +          //
>>>> +          // AddrRangeMin in Resource Descriptor here should be device address
>>>> +          // instead of host address, or else PCI bus driver cannot set correct
>>>> +          // address into PCI BAR registers.
>>>> +          // Base in ResAllocNode is a host address, so conversion is needed.
>>>> +          //
>>>> +          Descriptor->AddrRangeMin          = TO_DEVICE_ADDRESS (RootBridge->ResAllocNode[Index].Base,
>>>> +            GetTranslationByResourceType (RootBridge, Index));
>>>>             Descriptor->AddrRangeMax          = 0;
>>>>             Descriptor->AddrTranslationOffset = (ResStatus == ResAllocated) ? EFI_RESOURCE_SATISFIED : PCI_RESOURCE_LESS;
>>>>             Descriptor->AddrLen               = RootBridge->ResAllocNode[Index].Length;
>>>> diff --git a/MdeModulePkg/Bus/Pci/PciHostBridgeDxe/PciRootBridgeIo.c b/MdeModulePkg/Bus/Pci/PciHostBridgeDxe/PciRootBridgeIo.c
>>>> index dc06c16dc038..5dd9d257d46d 100644
>>>> --- a/MdeModulePkg/Bus/Pci/PciHostBridgeDxe/PciRootBridgeIo.c
>>>> +++ b/MdeModulePkg/Bus/Pci/PciHostBridgeDxe/PciRootBridgeIo.c
>>>> @@ -86,12 +86,35 @@ CreateRootBridge (
>>>>             (Bridge->AllocationAttributes & EFI_PCI_HOST_BRIDGE_COMBINE_MEM_PMEM) != 0 ? L"CombineMemPMem " : L"",
>>>>             (Bridge->AllocationAttributes & EFI_PCI_HOST_BRIDGE_MEM64_DECODE) != 0 ? L"Mem64Decode" : L""
>>>>             ));
>>>> +  //
>>>> +  // Translation for bus is not supported.
>>>> +  //
>>>>     DEBUG ((EFI_D_INFO, "           Bus: %lx - %lx\n", Bridge->Bus.Base, Bridge->Bus.Limit));
>>>> -  DEBUG ((EFI_D_INFO, "            Io: %lx - %lx\n", Bridge->Io.Base, Bridge->Io.Limit));
>>>> -  DEBUG ((EFI_D_INFO, "           Mem: %lx - %lx\n", Bridge->Mem.Base, Bridge->Mem.Limit));
>>>> -  DEBUG ((EFI_D_INFO, "    MemAbove4G: %lx - %lx\n", Bridge->MemAbove4G.Base, Bridge->MemAbove4G.Limit));
>>>> -  DEBUG ((EFI_D_INFO, "          PMem: %lx - %lx\n", Bridge->PMem.Base, Bridge->PMem.Limit));
>>>> -  DEBUG ((EFI_D_INFO, "   PMemAbove4G: %lx - %lx\n", Bridge->PMemAbove4G.Base, Bridge->PMemAbove4G.Limit));
>>>> +  ASSERT (Bridge->Bus.Translation == 0);
>>>> +  if (Bridge->Bus.Translation != 0) {
>>>> +    return NULL;
>>>> +  }

2. Can you please use the same debug message format for Bus range? I 
mean to print the Translation for Bus as well. Then in the end of debug 
message, use assertion and if-check. This way developer can know what 
the Bus.Translation is from the debug log.

>>>> +
>>>> +  DEBUG ((
>>>> +    DEBUG_INFO, "            Io: %lx - %lx Translation=%lx\n",
>>>> +    Bridge->Io.Base, Bridge->Io.Limit, Bridge->Io.Translation
>>>> +    ));
>>>> +  DEBUG ((
>>>> +    DEBUG_INFO, "           Mem: %lx - %lx Translation=%lx\n",
>>>> +    Bridge->Mem.Base, Bridge->Mem.Limit, Bridge->Mem.Translation
>>>> +    ));
>>>> +  DEBUG ((
>>>> +    DEBUG_INFO, "    MemAbove4G: %lx - %lx Translation=%lx\n",
>>>> +    Bridge->MemAbove4G.Base, Bridge->MemAbove4G.Limit, Bridge->MemAbove4G.Translation
>>>> +    ));
>>>> +  DEBUG ((
>>>> +    DEBUG_INFO, "          PMem: %lx - %lx Translation=%lx\n",
>>>> +    Bridge->PMem.Base, Bridge->PMem.Limit, Bridge->PMem.Translation
>>>> +    ));
>>>> +  DEBUG ((
>>>> +    DEBUG_INFO, "   PMemAbove4G: %lx - %lx Translation=%lx\n",
>>>> +    Bridge->PMemAbove4G.Base, Bridge->PMemAbove4G.Limit, Bridge->PMemAbove4G.Translation
>>>> +    ));
>>>>     //
>>>>     // Make sure Mem and MemAbove4G apertures are valid
>>>> @@ -206,7 +229,12 @@ CreateRootBridge (
>>>>       }
>>>>       RootBridge->ResAllocNode[Index].Type     = Index;
>>>>       if (Bridge->ResourceAssigned && (Aperture->Limit >= Aperture->Base)) {
>>>> -      RootBridge->ResAllocNode[Index].Base   = Aperture->Base;
>>>> +      //
>>>> +      // Base in ResAllocNode is a host address, while Base in Aperture is a
>>>> +      // device address.
>>>> +      //
>>>> +      RootBridge->ResAllocNode[Index].Base   = TO_HOST_ADDRESS (Aperture->Base,
>>>> +        Aperture->Translation);
>>>>         RootBridge->ResAllocNode[Index].Length = Aperture->Limit - Aperture->Base + 1;
>>>>         RootBridge->ResAllocNode[Index].Status = ResAllocated;
>>>>       } else {
>>>> @@ -403,6 +431,28 @@ RootBridgeIoCheckParameter (
>>>>     return EFI_SUCCESS;
>>>>   }

3. Please add the missing function header comments.

>>>> +EFI_STATUS
>>>> +RootBridgeIoGetMemTranslationByAddress (
>>>> +  IN PCI_ROOT_BRIDGE_INSTANCE               *RootBridge,
>>>> +  IN UINT64                                 Address,
>>>> +  IN OUT UINT64                             *Translation
>>>> +  )
>>>> +{
>>>> +  if (Address >= RootBridge->Mem.Base && Address <= RootBridge->Mem.Limit) {
>>>> +    *Translation = RootBridge->Mem.Translation;
>>>> +  } else if (Address >= RootBridge->PMem.Base && Address <= RootBridge->PMem.Limit) {
>>>> +    *Translation = RootBridge->PMem.Translation;
>>>> +  } else if (Address >= RootBridge->MemAbove4G.Base && Address <= RootBridge->MemAbove4G.Limit) {
>>>> +    *Translation = RootBridge->MemAbove4G.Translation;
>>>> +  } else if (Address >= RootBridge->PMemAbove4G.Base && Address <= RootBridge->PMemAbove4G.Limit) {
>>>> +    *Translation = RootBridge->PMemAbove4G.Translation;
>>>> +  } else {
>>>> +    return EFI_INVALID_PARAMETER;
>>>> +  }
>>>> +
>>>> +  return EFI_SUCCESS;
>>>> +}
>>>> +
>>>>   /**
>>>>     Polls an address in memory mapped I/O space until an exit condition is met,
>>>>     or a timeout occurs.
>>>> @@ -658,13 +708,25 @@ RootBridgeIoMemRead (
>>>>     )
>>>>   {
>>>>     EFI_STATUS                             Status;
>>>> +  PCI_ROOT_BRIDGE_INSTANCE               *RootBridge;
>>>> +  UINT64                                 Translation;
>>>>     Status = RootBridgeIoCheckParameter (This, MemOperation, Width, Address,
>>>>                                          Count, Buffer);
>>>>     if (EFI_ERROR (Status)) {
>>>>       return Status;
>>>>     }
>>>> -  return mCpuIo->Mem.Read (mCpuIo, (EFI_CPU_IO_PROTOCOL_WIDTH) Width, Address, Count, Buffer);
>>>> +
>>>> +  RootBridge = ROOT_BRIDGE_FROM_THIS (This);
>>>> +  Status = RootBridgeIoGetMemTranslationByAddress (RootBridge, Address, &Translation);
>>>> +  if (EFI_ERROR (Status)) {
>>>> +    return Status;
>>>> +  }
>>>> +
>>>> +  // Address passed to CpuIo->Mem.Read needs to be a host address instead of
>>>> +  // device address.
>>>> +  return mCpuIo->Mem.Read (mCpuIo, (EFI_CPU_IO_PROTOCOL_WIDTH) Width,
>>>> +      TO_HOST_ADDRESS (Address, Translation), Count, Buffer);
>>>>   }
>>>>   /**
>>>> @@ -705,13 +767,25 @@ RootBridgeIoMemWrite (
>>>>     )
>>>>   {
>>>>     EFI_STATUS                             Status;
>>>> +  PCI_ROOT_BRIDGE_INSTANCE               *RootBridge;
>>>> +  UINT64                                 Translation;
>>>>     Status = RootBridgeIoCheckParameter (This, MemOperation, Width, Address,
>>>>                                          Count, Buffer);
>>>>     if (EFI_ERROR (Status)) {
>>>>       return Status;
>>>>     }
>>>> -  return mCpuIo->Mem.Write (mCpuIo, (EFI_CPU_IO_PROTOCOL_WIDTH) Width, Address, Count, Buffer);
>>>> +
>>>> +  RootBridge = ROOT_BRIDGE_FROM_THIS (This);
>>>> +  Status = RootBridgeIoGetMemTranslationByAddress (RootBridge, Address, &Translation);
>>>> +  if (EFI_ERROR (Status)) {
>>>> +    return Status;
>>>> +  }
>>>> +
>>>> +  // Address passed to CpuIo->Mem.Write needs to be a host address instead of
>>>> +  // device address.
>>>> +  return mCpuIo->Mem.Write (mCpuIo, (EFI_CPU_IO_PROTOCOL_WIDTH) Width,
>>>> +      TO_HOST_ADDRESS (Address, Translation), Count, Buffer);
>>>>   }
>>>>   /**
>>>> @@ -746,6 +820,8 @@ RootBridgeIoIoRead (
>>>>     )
>>>>   {
>>>>     EFI_STATUS                                    Status;
>>>> +  PCI_ROOT_BRIDGE_INSTANCE                      *RootBridge;
>>>> +
>>>>     Status = RootBridgeIoCheckParameter (
>>>>                This, IoOperation, Width,
>>>>                Address, Count, Buffer
>>>> @@ -753,7 +829,13 @@ RootBridgeIoIoRead (
>>>>     if (EFI_ERROR (Status)) {
>>>>       return Status;
>>>>     }
>>>> -  return mCpuIo->Io.Read (mCpuIo, (EFI_CPU_IO_PROTOCOL_WIDTH) Width, Address, Count, Buffer);
>>>> +
>>>> +  RootBridge = ROOT_BRIDGE_FROM_THIS (This);
>>>> +
>>>> +  // Address passed to CpuIo->Io.Read needs to be a host address instead of
>>>> +  // device address.
>>>> +  return mCpuIo->Io.Read (mCpuIo, (EFI_CPU_IO_PROTOCOL_WIDTH) Width,
>>>> +      TO_HOST_ADDRESS (Address, RootBridge->Io.Translation), Count, Buffer);
>>>>   }
>>>>   /**
>>>> @@ -788,6 +870,8 @@ RootBridgeIoIoWrite (
>>>>     )
>>>>   {
>>>>     EFI_STATUS                                    Status;
>>>> +  PCI_ROOT_BRIDGE_INSTANCE                      *RootBridge;
>>>> +
>>>>     Status = RootBridgeIoCheckParameter (
>>>>                This, IoOperation, Width,
>>>>                Address, Count, Buffer
>>>> @@ -795,7 +879,13 @@ RootBridgeIoIoWrite (
>>>>     if (EFI_ERROR (Status)) {
>>>>       return Status;
>>>>     }
>>>> -  return mCpuIo->Io.Write (mCpuIo, (EFI_CPU_IO_PROTOCOL_WIDTH) Width, Address, Count, Buffer);
>>>> +
>>>> +  RootBridge = ROOT_BRIDGE_FROM_THIS (This);
>>>> +
>>>> +  // Address passed to CpuIo->Io.Write needs to be a host address instead of
>>>> +  // device address.
>>>> +  return mCpuIo->Io.Write (mCpuIo, (EFI_CPU_IO_PROTOCOL_WIDTH) Width,
>>>> +      TO_HOST_ADDRESS (Address, RootBridge->Io.Translation), Count, Buffer);
>>>>   }
>>>>   /**
>>>> @@ -1615,9 +1705,17 @@ RootBridgeIoConfiguration (
>>>>       Descriptor->Desc = ACPI_ADDRESS_SPACE_DESCRIPTOR;
>>>>       Descriptor->Len  = sizeof (EFI_ACPI_ADDRESS_SPACE_DESCRIPTOR) - 3;
>>>> +    // According to UEFI 2.7, RootBridgeIo->Configuration should return address
>>>> +    // range in CPU view (host address), and ResAllocNode->Base is already a CPU
>>>> +    // view address (host address).
>>>>       Descriptor->AddrRangeMin  = ResAllocNode->Base;
>>>>       Descriptor->AddrRangeMax  = ResAllocNode->Base + ResAllocNode->Length - 1;
>>>>       Descriptor->AddrLen       = ResAllocNode->Length;
>>>> +    Descriptor->AddrTranslationOffset = GetTranslationByResourceType (
>>>> +      RootBridge,
>>>> +      ResAllocNode->Type
>>>> +      );
>>>> +
>>>>       switch (ResAllocNode->Type) {
>>>>       case TypeIo:
>>>> -- 
>>>> 2.7.4
>>>>
>>
>>
>> -- 
>> Thanks,
>> Ray


Reviewed-by: Ruiyu Ni <ruiyu.ni@intel.com>
Please modify code according to the above 3 comments and make sure
the changes pass ECC check:
BaseTools/Source/Python/Ecc/Ecc.py -t -s \
  MdeModulePkg/Pci/PciHostBridgeDxe

> _______________________________________________
> edk2-devel mailing list
> edk2-devel@lists.01.org
> https://lists.01.org/mailman/listinfo/edk2-devel
> 


-- 
Thanks,
Ray

Re: [PATCH v5 4/6] MdeModulePkg/PciHostBridgeDxe: Add support for address translation

[Ard Biesheuvel]

On 14 March 2018 at 07:57, Ni, Ruiyu <ruiyu.ni@intel.com> wrote:
> On 3/7/2018 2:01 PM, Guo Heyi wrote:
>>
>> Thanks. Please let me know if any further changes are needed.
>>
>> Regards,
>>
>> Heyi
>>
>> On Wed, Mar 07, 2018 at 12:30:59PM +0800, Ni, Ruiyu wrote:
>>>
>>> On 3/6/2018 10:44 AM, Guo Heyi wrote:
>>>>
>>>> Hi Ray,
>>>>
>>>> Any comments for v5?
>>>
>>>
>>> Heyi,
>>> Some backward compatibility concerns were received from internal
>>> production
>>> teams. Current change will cause silent failure on old platforms because
>>> TranslationOffset might be random if uninitialized.
>>> I will solve the concern and then send out updates to you, hopefully by
>>> end
>>> of next week.
>>>
>>>>
>>>> Regards,
>>>>
>>>> Heyi
>>>>
>>>> On Thu, Mar 01, 2018 at 02:57:22PM +0800, Heyi Guo wrote:
>>>>>
>>>>> PCI address translation is necessary for some non-x86 platforms. On
>>>>> such platforms, address value (denoted as "device address" or "address
>>>>> in PCI view") set to PCI BAR registers in configuration space might be
>>>>> different from the address which is used by CPU to access the
>>>>> registers in memory BAR or IO BAR spaces (denoted as "host address" or
>>>>> "address in CPU view"). The difference between the two addresses is
>>>>> called "Address Translation Offset" or simply "translation", and can
>>>>> be represented by "Address Translation Offset" in ACPI QWORD Address
>>>>> Space Descriptor (Offset 0x1E). However UEFI and ACPI differs on the
>>>>> definitions of QWORD Address Space Descriptor, and we will follow UEFI
>>>>> definition on UEFI protocols, such as PCI root bridge IO protocol and
>>>>> PCI IO protocol. In UEFI 2.7, "Address Translation Offset" is "Offset
>>>>> to apply to the Starting address to convert it to a PCI address". This
>>>>> means:
>>>>>
>>>>> 1. Translation = device address - host address.
>>>>>
>>>>> 2. PciRootBridgeIo->Configuration should return CPU view address, as
>>>>> well as PciIo->GetBarAttributes.
>>>>>
>>>>> Summary of addresses used in protocol interfaces and internal
>>>>> implementations:
>>>>>
>>>>> 1. *Only* the following protocol interfaces assume Address is Device
>>>>>     Address:
>>>>> (1). PciHostBridgeResourceAllocation.GetProposedResources()
>>>>>       Otherwise PCI bus driver cannot set correct address into PCI
>>>>>       BARs.
>>>>> (2). PciRootBridgeIo.Mem.Read() and PciRootBridgeIo.Mem.Write()
>>>>> (3). PciRootBridgeIo.CopyMem()
>>>>> UEFI and PI spec have clear statements for all other protocol
>>>>> interfaces about the address type.
>>>>>
>>>>> 2. Library interfaces and internal implementation:
>>>>> (1). Base and Limit in PCI_ROOT_BRIDGE_APERTURE are device address.
>>>>>       It is easy to check whether the address is below 4G or above 4G.
>>>>> (2). Addresses in PCI_ROOT_BRIDGE_INSTANCE.ResAllocNode are host
>>>>>       address, for they are allocated from GCD.
>>>>> (3). Address passed to PciHostBridgeResourceConflict is host address,
>>>>>       for it comes from PCI_ROOT_BRIDGE_INSTANCE.ResAllocNode.
>>>>>
>>>>> RESTRICTION: to simplify the situation, we require the alignment of
>>>>> Translation must be larger than any BAR alignment in the same root
>>>>> bridge, so that resource allocation alignment can be applied to both
>>>>> device address and host address.
>>>>>
>>>>> Contributed-under: TianoCore Contribution Agreement 1.1
>>>>> Signed-off-by: Heyi Guo <heyi.guo@linaro.org>
>>>>> Cc: Ruiyu Ni <ruiyu.ni@intel.com>
>>>>> Cc: Ard Biesheuvel <ard.biesheuvel@linaro.org>
>>>>> Cc: Star Zeng <star.zeng@intel.com>
>>>>> Cc: Eric Dong <eric.dong@intel.com>
>>>>> Cc: Laszlo Ersek <lersek@redhat.com>
>>>>> Cc: Michael D Kinney <michael.d.kinney@intel.com>
>>>>> ---
>>>>>
>>>>> Notes:
>>>>>      v5:
>>>>>      - Add check for the alignment of Translation against the BAR
>>>>> alignment
>>>>>        [Ray]
>>>>>      - Keep coding style of comments consistent with the context [Ray]
>>>>>      - Comment change for Base in PCI_RES_NODE [Ray]
>>>>>      - Add macros of TO_HOST_ADDRESS and TO_DEVICE_ADDRESS for address
>>>>> type
>>>>>        conversion (After that we can also simply the comments about the
>>>>>        calculation [Ray]
>>>>>      - Add check for bus translation offset in CreateRootBridge(),
>>>>> making
>>>>>        sure it is zero, and unify code logic for all types of resource
>>>>>        after that [Ray]
>>>>>      - Use GetTranslationByResourceType() to simplify the code in
>>>>>        RootBridgeIoConfiguration() (also fix a bug in previous patch
>>>>>        version of missing a break after case TypePMem64) [Ray]
>>>>>      - Commit message refinement [Ray]
>>>>>      v4:
>>>>>      - Add ASSERT (FALSE) to default branch in
>>>>> GetTranslationByResourceType
>>>>>        [Laszlo]
>>>>>      - Fix bug when passing BaseAddress to gDS->AllocateIoSpace and
>>>>>        gDS->AllocateMemorySpace [Laszlo]
>>>>>      - Add comment for applying alignment to both device address and
>>>>> host
>>>>>        address, and add NOTE for the alignment requirement of
>>>>> Translation,
>>>>>        as well as in commit message [Laszlo][Ray]
>>>>>      - Improve indention for the code in CreateRootBridge [Laszlo]
>>>>>      - Improve comment for Translation in PCI_ROOT_BRIDGE_APERTURE
>>>>>        definition [Laszlo]
>>>>>      - Ignore translation of bus in CreateRootBridge
>>>>>      v4:
>>>>>      - Add ASSERT (FALSE) to default branch in
>>>>> GetTranslationByResourceType
>>>>>        [Laszlo]
>>>>>      - Fix bug when passing BaseAddress to gDS->AllocateIoSpace and
>>>>>        gDS->AllocateMemorySpace [Laszlo]
>>>>>      - Add comment for applying alignment to both device address and
>>>>> host
>>>>>        address, and add NOTE for the alignment requirement of
>>>>> Translation,
>>>>>        as well as in commit message [Laszlo][Ray]
>>>>>      - Improve indention for the code in CreateRootBridge [Laszlo]
>>>>>      - Improve comment for Translation in PCI_ROOT_BRIDGE_APERTURE
>>>>>        definition [Laszlo]
>>>>>      - Ignore translation of bus in CreateRootBridge
>>>>>
>>>>>   MdeModulePkg/Bus/Pci/PciHostBridgeDxe/PciHostBridge.h   |  21 ++++
>>>>>   MdeModulePkg/Bus/Pci/PciHostBridgeDxe/PciHostResource.h |   3 +
>>>>>   MdeModulePkg/Bus/Pci/PciHostBridgeDxe/PciHostBridge.c   | 129
>>>>> +++++++++++++++++---
>>>>>   MdeModulePkg/Bus/Pci/PciHostBridgeDxe/PciRootBridgeIo.c | 118
>>>>> ++++++++++++++++--
>>>>>   4 files changed, 245 insertions(+), 26 deletions(-)
>>>>>
>>>>> diff --git a/MdeModulePkg/Bus/Pci/PciHostBridgeDxe/PciHostBridge.h
>>>>> b/MdeModulePkg/Bus/Pci/PciHostBridgeDxe/PciHostBridge.h
>>>>> index 9a8ca21f4819..c2791ea5c2a4 100644
>>>>> --- a/MdeModulePkg/Bus/Pci/PciHostBridgeDxe/PciHostBridge.h
>>>>> +++ b/MdeModulePkg/Bus/Pci/PciHostBridgeDxe/PciHostBridge.h
>>>>> @@ -38,6 +38,13 @@ typedef struct {
>>>>>   #define PCI_HOST_BRIDGE_FROM_THIS(a) CR (a, PCI_HOST_BRIDGE_INSTANCE,
>>>>> ResAlloc, PCI_HOST_BRIDGE_SIGNATURE)
>>>>>   //
>>>>> +// Macros to translate device address to host address and vice versa.
>>>>> According
>>>>> +// to UEFI 2.7, device address = host address + translation offset.
>>>>> +//
>>>>> +#define TO_HOST_ADDRESS(DeviceAddress,TranslationOffset)
>>>>> ((DeviceAddress) - (TranslationOffset))
>>>>> +#define TO_DEVICE_ADDRESS(HostAddress,TranslationOffset)
>>>>> ((HostAddress) + (TranslationOffset))
>>>>> +
>>>>> +//
>>>>>   // Driver Entry Point
>>>>>   //
>>>>>   /**
>>>>> @@ -247,6 +254,20 @@ ResourceConflict (
>>>>>     IN  PCI_HOST_BRIDGE_INSTANCE *HostBridge
>>>>>     );
>>>>> +/**
>>>>> +  This routine gets translation offset from a root bridge instance by
>>>>> resource type.
>>>>> +
>>>>> +  @param RootBridge The Root Bridge Instance for the resources.
>>>>> +  @param ResourceType The Resource Type of the translation offset.
>>>>> +
>>>>> +  @retval The Translation Offset of the specified resource.
>>>>> +**/
>>>>> +UINT64
>>>>> +GetTranslationByResourceType (
>>>>> +  IN  PCI_ROOT_BRIDGE_INSTANCE     *RootBridge,
>>>>> +  IN  PCI_RESOURCE_TYPE            ResourceType
>>>>> +  );
>>>>> +
>>>>>   extern EFI_METRONOME_ARCH_PROTOCOL *mMetronome;
>>>>>   extern EFI_CPU_IO2_PROTOCOL        *mCpuIo;
>>>>>   #endif
>>>>> diff --git a/MdeModulePkg/Bus/Pci/PciHostBridgeDxe/PciHostResource.h
>>>>> b/MdeModulePkg/Bus/Pci/PciHostBridgeDxe/PciHostResource.h
>>>>> index 8612c0c3251b..a6c3739db368 100644
>>>>> --- a/MdeModulePkg/Bus/Pci/PciHostBridgeDxe/PciHostResource.h
>>>>> +++ b/MdeModulePkg/Bus/Pci/PciHostBridgeDxe/PciHostResource.h
>>>>> @@ -38,6 +38,9 @@ typedef enum {
>>>>>   typedef struct {
>>>>>     PCI_RESOURCE_TYPE Type;
>>>>> +  //
>>>>> +  // Base is a host address
>>>>> +  //
>>>>>     UINT64            Base;
>>>>>     UINT64            Length;
>>>>>     UINT64            Alignment;
>>>>> diff --git a/MdeModulePkg/Bus/Pci/PciHostBridgeDxe/PciHostBridge.c
>>>>> b/MdeModulePkg/Bus/Pci/PciHostBridgeDxe/PciHostBridge.c
>>>>> index 1494848c3e8c..42ded2855c71 100644
>>>>> --- a/MdeModulePkg/Bus/Pci/PciHostBridgeDxe/PciHostBridge.c
>>>>> +++ b/MdeModulePkg/Bus/Pci/PciHostBridgeDxe/PciHostBridge.c
>>>>> @@ -33,6 +33,39 @@ EFI_EVENT                   mIoMmuEvent;
>>>>>   VOID                        *mIoMmuRegistration;
>>>>>   /**
>>>>> +  This routine gets translation offset from a root bridge instance by
>>>>> resource type.
>>>>> +
>>>>> +  @param RootBridge The Root Bridge Instance for the resources.
>>>>> +  @param ResourceType The Resource Type of the translation offset.
>>>>> +
>>>>> +  @retval The Translation Offset of the specified resource.
>>>>> +**/
>>>>> +UINT64
>>>>> +GetTranslationByResourceType (
>>>>> +  IN  PCI_ROOT_BRIDGE_INSTANCE     *RootBridge,
>>>>> +  IN  PCI_RESOURCE_TYPE            ResourceType
>>>>> +  )
>>>>> +{
>>>>> +  switch (ResourceType) {
>>>>> +    case TypeIo:
>>>>> +      return RootBridge->Io.Translation;
>>>>> +    case TypeMem32:
>>>>> +      return RootBridge->Mem.Translation;
>>>>> +    case TypePMem32:
>>>>> +      return RootBridge->PMem.Translation;
>>>>> +    case TypeMem64:
>>>>> +      return RootBridge->MemAbove4G.Translation;
>>>>> +    case TypePMem64:
>>>>> +      return RootBridge->PMemAbove4G.Translation;
>>>>> +    case TypeBus:
>>>>> +      return RootBridge->Bus.Translation;
>>>>> +    default:
>>>>> +      ASSERT (FALSE);
>>>>> +      return 0;
>>>>> +  }
>>>>> +}
>>>>> +
>>>>> +/**
>>>>>     Ensure the compatibility of an IO space descriptor with the IO
>>>>> aperture.
>>>>>     The IO space descriptor can come from the GCD IO space map, or it
>>>>> can
>>>>> @@ -366,6 +399,7 @@ InitializePciHostBridge (
>>>>>     UINTN                       MemApertureIndex;
>>>>>     BOOLEAN                     ResourceAssigned;
>>>>>     LIST_ENTRY                  *Link;
>>>>> +  UINT64                      HostAddress;
>>>>>     RootBridges = PciHostBridgeGetRootBridges (&RootBridgeCount);
>>>>>     if ((RootBridges == NULL) || (RootBridgeCount == 0)) {
>>>>> @@ -411,8 +445,15 @@ InitializePciHostBridge (
>>>>>       }
>>>>>       if (RootBridges[Index].Io.Base <= RootBridges[Index].Io.Limit) {
>>>>> +      //
>>>>> +      // Base and Limit in PCI_ROOT_BRIDGE_APERTURE are device
>>>>> address.
>>>>> +      // For GCD resource manipulation, we need to use host address.
>>>>> +      //
>>>>> +      HostAddress = TO_HOST_ADDRESS (RootBridges[Index].Io.Base,
>>>>> +        RootBridges[Index].Io.Translation);
>>>>> +
>>>>>         Status = AddIoSpace (
>>>>> -                 RootBridges[Index].Io.Base,
>>>>> +                 HostAddress,
>>>>>                    RootBridges[Index].Io.Limit -
>>>>> RootBridges[Index].Io.Base + 1
>>>>>                    );
>>>>>         ASSERT_EFI_ERROR (Status);
>>>>> @@ -422,7 +463,7 @@ InitializePciHostBridge (
>>>>>                           EfiGcdIoTypeIo,
>>>>>                           0,
>>>>>                           RootBridges[Index].Io.Limit -
>>>>> RootBridges[Index].Io.Base + 1,
>>>>> -                        &RootBridges[Index].Io.Base,
>>>>> +                        &HostAddress,
>>>>>                           gImageHandle,
>>>>>                           NULL
>>>>>                           );
>>>>> @@ -443,14 +484,20 @@ InitializePciHostBridge (
>>>>>       for (MemApertureIndex = 0; MemApertureIndex < ARRAY_SIZE
>>>>> (MemApertures); MemApertureIndex++) {
>>>>>         if (MemApertures[MemApertureIndex]->Base <=
>>>>> MemApertures[MemApertureIndex]->Limit) {
>>>>> +        //
>>>>> +        // Base and Limit in PCI_ROOT_BRIDGE_APERTURE are device
>>>>> address.
>>>>> +        // For GCD resource manipulation, we need to use host address.
>>>>> +        //
>>>>> +        HostAddress = TO_HOST_ADDRESS
>>>>> (MemApertures[MemApertureIndex]->Base,
>>>>> +          MemApertures[MemApertureIndex]->Translation);
>>>>>           Status = AddMemoryMappedIoSpace (
>>>>> -                   MemApertures[MemApertureIndex]->Base,
>>>>> +                   HostAddress,
>>>>>                      MemApertures[MemApertureIndex]->Limit -
>>>>> MemApertures[MemApertureIndex]->Base + 1,
>>>>>                      EFI_MEMORY_UC
>>>>>                      );
>>>>>           ASSERT_EFI_ERROR (Status);
>>>>>           Status = gDS->SetMemorySpaceAttributes (
>>>>> -                        MemApertures[MemApertureIndex]->Base,
>>>>> +                        HostAddress,
>>>>>                           MemApertures[MemApertureIndex]->Limit -
>>>>> MemApertures[MemApertureIndex]->Base + 1,
>>>>>                           EFI_MEMORY_UC
>>>>>                           );
>>>>> @@ -463,7 +510,7 @@ InitializePciHostBridge (
>>>>>                             EfiGcdMemoryTypeMemoryMappedIo,
>>>>>                             0,
>>>>>                             MemApertures[MemApertureIndex]->Limit -
>>>>> MemApertures[MemApertureIndex]->Base + 1,
>>>>> -                          &MemApertures[MemApertureIndex]->Base,
>>>>> +                          &HostAddress,
>>>>>                             gImageHandle,
>>>>>                             NULL
>>>>>                             );
>>>>> @@ -654,6 +701,11 @@ AllocateResource (
>>>>>     if (BaseAddress < Limit) {
>>>>>       //
>>>>>       // Have to make sure Aligment is handled since we are doing
>>>>> direct address allocation
>>>>> +    // Strictly speaking, alignment requirement should be applied to
>>>>> device
>>>>> +    // address instead of host address which is used in GCD
>>>>> manipulation below,
>>>>> +    // but as we restrict the alignment of Translation to be larger
>>>>> than any BAR
>>>>> +    // alignment in the root bridge, we can simplify the situation and
>>>>> consider
>>>>> +    // the same alignment requirement is also applied to host address.
>>>>>       //
>>>>>       BaseAddress = ALIGN_VALUE (BaseAddress, LShiftU64 (1,
>>>>> BitsOfAlignment));
>>>>> @@ -721,6 +773,7 @@ NotifyPhase (
>>>>>     PCI_RESOURCE_TYPE                     Index2;
>>>>>     BOOLEAN                               ResNodeHandled[TypeMax];
>>>>>     UINT64                                MaxAlignment;
>>>>> +  UINT64                                Translation;
>>>>>     HostBridge = PCI_HOST_BRIDGE_FROM_THIS (This);
>>>>> @@ -822,14 +875,43 @@ NotifyPhase (
>>>>>             BitsOfAlignment = LowBitSet64 (Alignment + 1);
>>>>>             BaseAddress = MAX_UINT64;
>>>>> +          //
>>>>> +          // RESTRICTION: To simplify the situation, we require the
>>>>> alignment of
>>>>> +          // Translation must be larger than any BAR alignment in the
>>>>> same root
>>>>> +          // bridge, so that resource allocation alignment can be
>>>>> applied to
>>>>> +          // both device address and host address.
>>>>> +          //
>>>>> +          Translation = GetTranslationByResourceType (RootBridge,
>>>>> Index);
>>>>> +          if ((Translation & Alignment) != 0) {
>>>>> +            DEBUG ((DEBUG_ERROR, "[%a:%d] Translation %lx is not
>>>>> aligned to %lx!\n",
>>>>> +              __FUNCTION__, __LINE__, Translation, Alignment
>>>>> +              ));
>>>>> +            ASSERT (FALSE);
>
> 1. Cool! But can you replace ASSERT(FALSE) with
>    ASSERT ((Translation & Alignment) == 0)?
>
>>>>> +            //
>>>>> +            // This may be caused by too large alignment or too small
>>>>> +            // Translation; pick the 1st possibility and return out of
>>>>> resource,
>>>>> +            // which can also go thru the same process for out of
>>>>> resource
>>>>> +            // outside the loop.
>>>>> +            //
>>>>> +            ReturnStatus = EFI_OUT_OF_RESOURCES;
>>>>> +            continue;
>>>>> +          }
>>>>> +
>>>>>             switch (Index) {
>>>>>             case TypeIo:
>>>>> +            //
>>>>> +            // Base and Limit in PCI_ROOT_BRIDGE_APERTURE are device
>>>>> address.
>>>>> +            // For AllocateResource is manipulating GCD resource, we
>>>>> need to use
>>>>> +            // host address here.
>>>>> +            //
>>>>>               BaseAddress = AllocateResource (
>>>>>                               FALSE,
>>>>>                               RootBridge->ResAllocNode[Index].Length,
>>>>>                               MIN (15, BitsOfAlignment),
>>>>> -                            ALIGN_VALUE (RootBridge->Io.Base,
>>>>> Alignment + 1),
>>>>> -                            RootBridge->Io.Limit
>>>>> +                            TO_HOST_ADDRESS (ALIGN_VALUE
>>>>> (RootBridge->Io.Base, Alignment + 1),
>>>>> +                              RootBridge->Io.Translation),
>>>>> +                            TO_HOST_ADDRESS (RootBridge->Io.Limit,
>>>>> +                              RootBridge->Io.Translation)
>>>>>                               );
>>>>>               break;
>>>>> @@ -838,8 +920,10 @@ NotifyPhase (
>>>>>                               TRUE,
>>>>>                               RootBridge->ResAllocNode[Index].Length,
>>>>>                               MIN (63, BitsOfAlignment),
>>>>> -                            ALIGN_VALUE (RootBridge->MemAbove4G.Base,
>>>>> Alignment + 1),
>>>>> -                            RootBridge->MemAbove4G.Limit
>>>>> +                            TO_HOST_ADDRESS (ALIGN_VALUE
>>>>> (RootBridge->MemAbove4G.Base, Alignment + 1),
>>>>> +                              RootBridge->MemAbove4G.Translation),
>>>>> +                            TO_HOST_ADDRESS
>>>>> (RootBridge->MemAbove4G.Limit,
>>>>> +                              RootBridge->MemAbove4G.Translation)
>>>>>                               );
>>>>>               if (BaseAddress != MAX_UINT64) {
>>>>>                 break;
>>>>> @@ -853,8 +937,10 @@ NotifyPhase (
>>>>>                               TRUE,
>>>>>                               RootBridge->ResAllocNode[Index].Length,
>>>>>                               MIN (31, BitsOfAlignment),
>>>>> -                            ALIGN_VALUE (RootBridge->Mem.Base,
>>>>> Alignment + 1),
>>>>> -                            RootBridge->Mem.Limit
>>>>> +                            TO_HOST_ADDRESS (ALIGN_VALUE
>>>>> (RootBridge->Mem.Base, Alignment + 1),
>>>>> +                              RootBridge->Mem.Translation),
>>>>> +                            TO_HOST_ADDRESS (RootBridge->Mem.Limit,
>>>>> +                              RootBridge->Mem.Translation)
>>>>>                               );
>>>>>               break;
>>>>> @@ -863,8 +949,10 @@ NotifyPhase (
>>>>>                               TRUE,
>>>>>                               RootBridge->ResAllocNode[Index].Length,
>>>>>                               MIN (63, BitsOfAlignment),
>>>>> -                            ALIGN_VALUE (RootBridge->PMemAbove4G.Base,
>>>>> Alignment + 1),
>>>>> -                            RootBridge->PMemAbove4G.Limit
>>>>> +                            TO_HOST_ADDRESS (ALIGN_VALUE
>>>>> (RootBridge->PMemAbove4G.Base, Alignment + 1),
>>>>> +                              RootBridge->PMemAbove4G.Translation),
>>>>> +                            TO_HOST_ADDRESS
>>>>> (RootBridge->PMemAbove4G.Limit,
>>>>> +                              RootBridge->PMemAbove4G.Translation)
>>>>>                               );
>>>>>               if (BaseAddress != MAX_UINT64) {
>>>>>                 break;
>>>>> @@ -877,8 +965,10 @@ NotifyPhase (
>>>>>                               TRUE,
>>>>>                               RootBridge->ResAllocNode[Index].Length,
>>>>>                               MIN (31, BitsOfAlignment),
>>>>> -                            ALIGN_VALUE (RootBridge->PMem.Base,
>>>>> Alignment + 1),
>>>>> -                            RootBridge->PMem.Limit
>>>>> +                            TO_HOST_ADDRESS (ALIGN_VALUE
>>>>> (RootBridge->PMem.Base, Alignment + 1),
>>>>> +                              RootBridge->PMem.Translation),
>>>>> +                            TO_HOST_ADDRESS (RootBridge->PMem.Limit,
>>>>> +                              RootBridge->PMem.Translation)
>>>>>                               );
>>>>>               break;
>>>>> @@ -1421,7 +1511,14 @@ GetProposedResources (
>>>>>             Descriptor->Desc                  =
>>>>> ACPI_ADDRESS_SPACE_DESCRIPTOR;
>>>>>             Descriptor->Len                   = sizeof
>>>>> (EFI_ACPI_ADDRESS_SPACE_DESCRIPTOR) - 3;;
>>>>>             Descriptor->GenFlag               = 0;
>>>>> -          Descriptor->AddrRangeMin          =
>>>>> RootBridge->ResAllocNode[Index].Base;
>>>>> +          //
>>>>> +          // AddrRangeMin in Resource Descriptor here should be device
>>>>> address
>>>>> +          // instead of host address, or else PCI bus driver cannot
>>>>> set correct
>>>>> +          // address into PCI BAR registers.
>>>>> +          // Base in ResAllocNode is a host address, so conversion is
>>>>> needed.
>>>>> +          //
>>>>> +          Descriptor->AddrRangeMin          = TO_DEVICE_ADDRESS
>>>>> (RootBridge->ResAllocNode[Index].Base,
>>>>> +            GetTranslationByResourceType (RootBridge, Index));
>>>>>             Descriptor->AddrRangeMax          = 0;
>>>>>             Descriptor->AddrTranslationOffset = (ResStatus ==
>>>>> ResAllocated) ? EFI_RESOURCE_SATISFIED : PCI_RESOURCE_LESS;
>>>>>             Descriptor->AddrLen               =
>>>>> RootBridge->ResAllocNode[Index].Length;
>>>>> diff --git a/MdeModulePkg/Bus/Pci/PciHostBridgeDxe/PciRootBridgeIo.c
>>>>> b/MdeModulePkg/Bus/Pci/PciHostBridgeDxe/PciRootBridgeIo.c
>>>>> index dc06c16dc038..5dd9d257d46d 100644
>>>>> --- a/MdeModulePkg/Bus/Pci/PciHostBridgeDxe/PciRootBridgeIo.c
>>>>> +++ b/MdeModulePkg/Bus/Pci/PciHostBridgeDxe/PciRootBridgeIo.c
>>>>> @@ -86,12 +86,35 @@ CreateRootBridge (
>>>>>             (Bridge->AllocationAttributes &
>>>>> EFI_PCI_HOST_BRIDGE_COMBINE_MEM_PMEM) != 0 ? L"CombineMemPMem " : L"",
>>>>>             (Bridge->AllocationAttributes &
>>>>> EFI_PCI_HOST_BRIDGE_MEM64_DECODE) != 0 ? L"Mem64Decode" : L""
>>>>>             ));
>>>>> +  //
>>>>> +  // Translation for bus is not supported.
>>>>> +  //
>>>>>     DEBUG ((EFI_D_INFO, "           Bus: %lx - %lx\n",
>>>>> Bridge->Bus.Base, Bridge->Bus.Limit));
>>>>> -  DEBUG ((EFI_D_INFO, "            Io: %lx - %lx\n", Bridge->Io.Base,
>>>>> Bridge->Io.Limit));
>>>>> -  DEBUG ((EFI_D_INFO, "           Mem: %lx - %lx\n", Bridge->Mem.Base,
>>>>> Bridge->Mem.Limit));
>>>>> -  DEBUG ((EFI_D_INFO, "    MemAbove4G: %lx - %lx\n",
>>>>> Bridge->MemAbove4G.Base, Bridge->MemAbove4G.Limit));
>>>>> -  DEBUG ((EFI_D_INFO, "          PMem: %lx - %lx\n",
>>>>> Bridge->PMem.Base, Bridge->PMem.Limit));
>>>>> -  DEBUG ((EFI_D_INFO, "   PMemAbove4G: %lx - %lx\n",
>>>>> Bridge->PMemAbove4G.Base, Bridge->PMemAbove4G.Limit));
>>>>> +  ASSERT (Bridge->Bus.Translation == 0);
>>>>> +  if (Bridge->Bus.Translation != 0) {
>>>>> +    return NULL;
>>>>> +  }
>
>
> 2. Can you please use the same debug message format for Bus range? I mean to
> print the Translation for Bus as well. Then in the end of debug message, use
> assertion and if-check. This way developer can know what the Bus.Translation
> is from the debug log.
>
>
>>>>> +
>>>>> +  DEBUG ((
>>>>> +    DEBUG_INFO, "            Io: %lx - %lx Translation=%lx\n",
>>>>> +    Bridge->Io.Base, Bridge->Io.Limit, Bridge->Io.Translation
>>>>> +    ));
>>>>> +  DEBUG ((
>>>>> +    DEBUG_INFO, "           Mem: %lx - %lx Translation=%lx\n",
>>>>> +    Bridge->Mem.Base, Bridge->Mem.Limit, Bridge->Mem.Translation
>>>>> +    ));
>>>>> +  DEBUG ((
>>>>> +    DEBUG_INFO, "    MemAbove4G: %lx - %lx Translation=%lx\n",
>>>>> +    Bridge->MemAbove4G.Base, Bridge->MemAbove4G.Limit,
>>>>> Bridge->MemAbove4G.Translation
>>>>> +    ));
>>>>> +  DEBUG ((
>>>>> +    DEBUG_INFO, "          PMem: %lx - %lx Translation=%lx\n",
>>>>> +    Bridge->PMem.Base, Bridge->PMem.Limit, Bridge->PMem.Translation
>>>>> +    ));
>>>>> +  DEBUG ((
>>>>> +    DEBUG_INFO, "   PMemAbove4G: %lx - %lx Translation=%lx\n",
>>>>> +    Bridge->PMemAbove4G.Base, Bridge->PMemAbove4G.Limit,
>>>>> Bridge->PMemAbove4G.Translation
>>>>> +    ));
>>>>>     //
>>>>>     // Make sure Mem and MemAbove4G apertures are valid
>>>>> @@ -206,7 +229,12 @@ CreateRootBridge (
>>>>>       }
>>>>>       RootBridge->ResAllocNode[Index].Type     = Index;
>>>>>       if (Bridge->ResourceAssigned && (Aperture->Limit >=
>>>>> Aperture->Base)) {
>>>>> -      RootBridge->ResAllocNode[Index].Base   = Aperture->Base;
>>>>> +      //
>>>>> +      // Base in ResAllocNode is a host address, while Base in
>>>>> Aperture is a
>>>>> +      // device address.
>>>>> +      //
>>>>> +      RootBridge->ResAllocNode[Index].Base   = TO_HOST_ADDRESS
>>>>> (Aperture->Base,
>>>>> +        Aperture->Translation);
>>>>>         RootBridge->ResAllocNode[Index].Length = Aperture->Limit -
>>>>> Aperture->Base + 1;
>>>>>         RootBridge->ResAllocNode[Index].Status = ResAllocated;
>>>>>       } else {
>>>>> @@ -403,6 +431,28 @@ RootBridgeIoCheckParameter (
>>>>>     return EFI_SUCCESS;
>>>>>   }
>
>
> 3. Please add the missing function header comments.
>
>
>>>>> +EFI_STATUS
>>>>> +RootBridgeIoGetMemTranslationByAddress (
>>>>> +  IN PCI_ROOT_BRIDGE_INSTANCE               *RootBridge,
>>>>> +  IN UINT64                                 Address,
>>>>> +  IN OUT UINT64                             *Translation
>>>>> +  )
>>>>> +{
>>>>> +  if (Address >= RootBridge->Mem.Base && Address <=
>>>>> RootBridge->Mem.Limit) {
>>>>> +    *Translation = RootBridge->Mem.Translation;
>>>>> +  } else if (Address >= RootBridge->PMem.Base && Address <=
>>>>> RootBridge->PMem.Limit) {
>>>>> +    *Translation = RootBridge->PMem.Translation;
>>>>> +  } else if (Address >= RootBridge->MemAbove4G.Base && Address <=
>>>>> RootBridge->MemAbove4G.Limit) {
>>>>> +    *Translation = RootBridge->MemAbove4G.Translation;
>>>>> +  } else if (Address >= RootBridge->PMemAbove4G.Base && Address <=
>>>>> RootBridge->PMemAbove4G.Limit) {
>>>>> +    *Translation = RootBridge->PMemAbove4G.Translation;
>>>>> +  } else {
>>>>> +    return EFI_INVALID_PARAMETER;
>>>>> +  }
>>>>> +
>>>>> +  return EFI_SUCCESS;
>>>>> +}
>>>>> +
>>>>>   /**
>>>>>     Polls an address in memory mapped I/O space until an exit condition
>>>>> is met,
>>>>>     or a timeout occurs.
>>>>> @@ -658,13 +708,25 @@ RootBridgeIoMemRead (
>>>>>     )
>>>>>   {
>>>>>     EFI_STATUS                             Status;
>>>>> +  PCI_ROOT_BRIDGE_INSTANCE               *RootBridge;
>>>>> +  UINT64                                 Translation;
>>>>>     Status = RootBridgeIoCheckParameter (This, MemOperation, Width,
>>>>> Address,
>>>>>                                          Count, Buffer);
>>>>>     if (EFI_ERROR (Status)) {
>>>>>       return Status;
>>>>>     }
>>>>> -  return mCpuIo->Mem.Read (mCpuIo, (EFI_CPU_IO_PROTOCOL_WIDTH) Width,
>>>>> Address, Count, Buffer);
>>>>> +
>>>>> +  RootBridge = ROOT_BRIDGE_FROM_THIS (This);
>>>>> +  Status = RootBridgeIoGetMemTranslationByAddress (RootBridge,
>>>>> Address, &Translation);
>>>>> +  if (EFI_ERROR (Status)) {
>>>>> +    return Status;
>>>>> +  }
>>>>> +
>>>>> +  // Address passed to CpuIo->Mem.Read needs to be a host address
>>>>> instead of
>>>>> +  // device address.
>>>>> +  return mCpuIo->Mem.Read (mCpuIo, (EFI_CPU_IO_PROTOCOL_WIDTH) Width,
>>>>> +      TO_HOST_ADDRESS (Address, Translation), Count, Buffer);
>>>>>   }
>>>>>   /**
>>>>> @@ -705,13 +767,25 @@ RootBridgeIoMemWrite (
>>>>>     )
>>>>>   {
>>>>>     EFI_STATUS                             Status;
>>>>> +  PCI_ROOT_BRIDGE_INSTANCE               *RootBridge;
>>>>> +  UINT64                                 Translation;
>>>>>     Status = RootBridgeIoCheckParameter (This, MemOperation, Width,
>>>>> Address,
>>>>>                                          Count, Buffer);
>>>>>     if (EFI_ERROR (Status)) {
>>>>>       return Status;
>>>>>     }
>>>>> -  return mCpuIo->Mem.Write (mCpuIo, (EFI_CPU_IO_PROTOCOL_WIDTH) Width,
>>>>> Address, Count, Buffer);
>>>>> +
>>>>> +  RootBridge = ROOT_BRIDGE_FROM_THIS (This);
>>>>> +  Status = RootBridgeIoGetMemTranslationByAddress (RootBridge,
>>>>> Address, &Translation);
>>>>> +  if (EFI_ERROR (Status)) {
>>>>> +    return Status;
>>>>> +  }
>>>>> +
>>>>> +  // Address passed to CpuIo->Mem.Write needs to be a host address
>>>>> instead of
>>>>> +  // device address.
>>>>> +  return mCpuIo->Mem.Write (mCpuIo, (EFI_CPU_IO_PROTOCOL_WIDTH) Width,
>>>>> +      TO_HOST_ADDRESS (Address, Translation), Count, Buffer);
>>>>>   }
>>>>>   /**
>>>>> @@ -746,6 +820,8 @@ RootBridgeIoIoRead (
>>>>>     )
>>>>>   {
>>>>>     EFI_STATUS                                    Status;
>>>>> +  PCI_ROOT_BRIDGE_INSTANCE                      *RootBridge;
>>>>> +
>>>>>     Status = RootBridgeIoCheckParameter (
>>>>>                This, IoOperation, Width,
>>>>>                Address, Count, Buffer
>>>>> @@ -753,7 +829,13 @@ RootBridgeIoIoRead (
>>>>>     if (EFI_ERROR (Status)) {
>>>>>       return Status;
>>>>>     }
>>>>> -  return mCpuIo->Io.Read (mCpuIo, (EFI_CPU_IO_PROTOCOL_WIDTH) Width,
>>>>> Address, Count, Buffer);
>>>>> +
>>>>> +  RootBridge = ROOT_BRIDGE_FROM_THIS (This);
>>>>> +
>>>>> +  // Address passed to CpuIo->Io.Read needs to be a host address
>>>>> instead of
>>>>> +  // device address.
>>>>> +  return mCpuIo->Io.Read (mCpuIo, (EFI_CPU_IO_PROTOCOL_WIDTH) Width,
>>>>> +      TO_HOST_ADDRESS (Address, RootBridge->Io.Translation), Count,
>>>>> Buffer);
>>>>>   }
>>>>>   /**
>>>>> @@ -788,6 +870,8 @@ RootBridgeIoIoWrite (
>>>>>     )
>>>>>   {
>>>>>     EFI_STATUS                                    Status;
>>>>> +  PCI_ROOT_BRIDGE_INSTANCE                      *RootBridge;
>>>>> +
>>>>>     Status = RootBridgeIoCheckParameter (
>>>>>                This, IoOperation, Width,
>>>>>                Address, Count, Buffer
>>>>> @@ -795,7 +879,13 @@ RootBridgeIoIoWrite (
>>>>>     if (EFI_ERROR (Status)) {
>>>>>       return Status;
>>>>>     }
>>>>> -  return mCpuIo->Io.Write (mCpuIo, (EFI_CPU_IO_PROTOCOL_WIDTH) Width,
>>>>> Address, Count, Buffer);
>>>>> +
>>>>> +  RootBridge = ROOT_BRIDGE_FROM_THIS (This);
>>>>> +
>>>>> +  // Address passed to CpuIo->Io.Write needs to be a host address
>>>>> instead of
>>>>> +  // device address.
>>>>> +  return mCpuIo->Io.Write (mCpuIo, (EFI_CPU_IO_PROTOCOL_WIDTH) Width,
>>>>> +      TO_HOST_ADDRESS (Address, RootBridge->Io.Translation), Count,
>>>>> Buffer);
>>>>>   }
>>>>>   /**
>>>>> @@ -1615,9 +1705,17 @@ RootBridgeIoConfiguration (
>>>>>       Descriptor->Desc = ACPI_ADDRESS_SPACE_DESCRIPTOR;
>>>>>       Descriptor->Len  = sizeof (EFI_ACPI_ADDRESS_SPACE_DESCRIPTOR) -
>>>>> 3;
>>>>> +    // According to UEFI 2.7, RootBridgeIo->Configuration should
>>>>> return address
>>>>> +    // range in CPU view (host address), and ResAllocNode->Base is
>>>>> already a CPU
>>>>> +    // view address (host address).
>>>>>       Descriptor->AddrRangeMin  = ResAllocNode->Base;
>>>>>       Descriptor->AddrRangeMax  = ResAllocNode->Base +
>>>>> ResAllocNode->Length - 1;
>>>>>       Descriptor->AddrLen       = ResAllocNode->Length;
>>>>> +    Descriptor->AddrTranslationOffset = GetTranslationByResourceType (
>>>>> +      RootBridge,
>>>>> +      ResAllocNode->Type
>>>>> +      );
>>>>> +
>>>>>       switch (ResAllocNode->Type) {
>>>>>       case TypeIo:
>>>>> --
>>>>> 2.7.4
>>>>>
>>>
>>>
>>> --
>>> Thanks,
>>> Ray
>
>
>
> Reviewed-by: Ruiyu Ni <ruiyu.ni@intel.com>
> Please modify code according to the above 3 comments and make sure
> the changes pass ECC check:
> BaseTools/Source/Python/Ecc/Ecc.py -t -s \
>  MdeModulePkg/Pci/PciHostBridgeDxe
>

Thanks Ray

Does that apply to the whole series?

Re: [PATCH v5 4/6] MdeModulePkg/PciHostBridgeDxe: Add support for address translation

[Guo Heyi]

Hi Ray,

Sorry I never tried Ecc tool before, for there is few documents about it. When I
tried running python BaseTools/Source/Python/Ecc/Ecc.py -t -s xxxx,
it showed me with error message:

ImportError: No module named Common.LongFilePathOs

Then I found there was an executable named "Ecc" in
BaseTools/BinWrappers/PosixLike/, and change the place of "-t" and "-s" after it
prompted:

 : error 1003: Invalid value of option
        Target [-s] does NOT exist

However, it still failed with error:

RuntimeError: ANTLR version mismatch: The recognizer has been generated with API
V0, but this runtime does not support this.

I could not find the reason for this error, maybe it was caused by incompatible
version of python and antlr3 (mine is Ubuntu 16.04.3 LTS, python 2.7.12,
python-antlr3 3.5.2-1).

Finally I downloaded prebuilt Win32 BaseTools and ran Ecc.exe on Windows. It
success and produced a "Report.csv" with only a titel line. I guess this means
there is no error in the coding style, isn't it?

Please let me know if there is anything wrong with what I did.

Thanks and regards,

Heyi

On Wed, Mar 14, 2018 at 03:57:14PM +0800, Ni, Ruiyu wrote:
> On 3/7/2018 2:01 PM, Guo Heyi wrote:
> >Thanks. Please let me know if any further changes are needed.
> >
> >Regards,
> >
> >Heyi
> >
> >On Wed, Mar 07, 2018 at 12:30:59PM +0800, Ni, Ruiyu wrote:
> >>On 3/6/2018 10:44 AM, Guo Heyi wrote:
> >>>Hi Ray,
> >>>
> >>>Any comments for v5?
> >>
> >>Heyi,
> >>Some backward compatibility concerns were received from internal production
> >>teams. Current change will cause silent failure on old platforms because
> >>TranslationOffset might be random if uninitialized.
> >>I will solve the concern and then send out updates to you, hopefully by end
> >>of next week.
> >>
> >>>
> >>>Regards,
> >>>
> >>>Heyi
> >>>
> >>>On Thu, Mar 01, 2018 at 02:57:22PM +0800, Heyi Guo wrote:
> >>>>PCI address translation is necessary for some non-x86 platforms. On
> >>>>such platforms, address value (denoted as "device address" or "address
> >>>>in PCI view") set to PCI BAR registers in configuration space might be
> >>>>different from the address which is used by CPU to access the
> >>>>registers in memory BAR or IO BAR spaces (denoted as "host address" or
> >>>>"address in CPU view"). The difference between the two addresses is
> >>>>called "Address Translation Offset" or simply "translation", and can
> >>>>be represented by "Address Translation Offset" in ACPI QWORD Address
> >>>>Space Descriptor (Offset 0x1E). However UEFI and ACPI differs on the
> >>>>definitions of QWORD Address Space Descriptor, and we will follow UEFI
> >>>>definition on UEFI protocols, such as PCI root bridge IO protocol and
> >>>>PCI IO protocol. In UEFI 2.7, "Address Translation Offset" is "Offset
> >>>>to apply to the Starting address to convert it to a PCI address". This
> >>>>means:
> >>>>
> >>>>1. Translation = device address - host address.
> >>>>
> >>>>2. PciRootBridgeIo->Configuration should return CPU view address, as
> >>>>well as PciIo->GetBarAttributes.
> >>>>
> >>>>Summary of addresses used in protocol interfaces and internal
> >>>>implementations:
> >>>>
> >>>>1. *Only* the following protocol interfaces assume Address is Device
> >>>>    Address:
> >>>>(1). PciHostBridgeResourceAllocation.GetProposedResources()
> >>>>      Otherwise PCI bus driver cannot set correct address into PCI
> >>>>      BARs.
> >>>>(2). PciRootBridgeIo.Mem.Read() and PciRootBridgeIo.Mem.Write()
> >>>>(3). PciRootBridgeIo.CopyMem()
> >>>>UEFI and PI spec have clear statements for all other protocol
> >>>>interfaces about the address type.
> >>>>
> >>>>2. Library interfaces and internal implementation:
> >>>>(1). Base and Limit in PCI_ROOT_BRIDGE_APERTURE are device address.
> >>>>      It is easy to check whether the address is below 4G or above 4G.
> >>>>(2). Addresses in PCI_ROOT_BRIDGE_INSTANCE.ResAllocNode are host
> >>>>      address, for they are allocated from GCD.
> >>>>(3). Address passed to PciHostBridgeResourceConflict is host address,
> >>>>      for it comes from PCI_ROOT_BRIDGE_INSTANCE.ResAllocNode.
> >>>>
> >>>>RESTRICTION: to simplify the situation, we require the alignment of
> >>>>Translation must be larger than any BAR alignment in the same root
> >>>>bridge, so that resource allocation alignment can be applied to both
> >>>>device address and host address.
> >>>>
> >>>>Contributed-under: TianoCore Contribution Agreement 1.1
> >>>>Signed-off-by: Heyi Guo <heyi.guo@linaro.org>
> >>>>Cc: Ruiyu Ni <ruiyu.ni@intel.com>
> >>>>Cc: Ard Biesheuvel <ard.biesheuvel@linaro.org>
> >>>>Cc: Star Zeng <star.zeng@intel.com>
> >>>>Cc: Eric Dong <eric.dong@intel.com>
> >>>>Cc: Laszlo Ersek <lersek@redhat.com>
> >>>>Cc: Michael D Kinney <michael.d.kinney@intel.com>
> >>>>---
> >>>>
> >>>>Notes:
> >>>>     v5:
> >>>>     - Add check for the alignment of Translation against the BAR alignment
> >>>>       [Ray]
> >>>>     - Keep coding style of comments consistent with the context [Ray]
> >>>>     - Comment change for Base in PCI_RES_NODE [Ray]
> >>>>     - Add macros of TO_HOST_ADDRESS and TO_DEVICE_ADDRESS for address type
> >>>>       conversion (After that we can also simply the comments about the
> >>>>       calculation [Ray]
> >>>>     - Add check for bus translation offset in CreateRootBridge(), making
> >>>>       sure it is zero, and unify code logic for all types of resource
> >>>>       after that [Ray]
> >>>>     - Use GetTranslationByResourceType() to simplify the code in
> >>>>       RootBridgeIoConfiguration() (also fix a bug in previous patch
> >>>>       version of missing a break after case TypePMem64) [Ray]
> >>>>     - Commit message refinement [Ray]
> >>>>     v4:
> >>>>     - Add ASSERT (FALSE) to default branch in GetTranslationByResourceType
> >>>>       [Laszlo]
> >>>>     - Fix bug when passing BaseAddress to gDS->AllocateIoSpace and
> >>>>       gDS->AllocateMemorySpace [Laszlo]
> >>>>     - Add comment for applying alignment to both device address and host
> >>>>       address, and add NOTE for the alignment requirement of Translation,
> >>>>       as well as in commit message [Laszlo][Ray]
> >>>>     - Improve indention for the code in CreateRootBridge [Laszlo]
> >>>>     - Improve comment for Translation in PCI_ROOT_BRIDGE_APERTURE
> >>>>       definition [Laszlo]
> >>>>     - Ignore translation of bus in CreateRootBridge
> >>>>     v4:
> >>>>     - Add ASSERT (FALSE) to default branch in GetTranslationByResourceType
> >>>>       [Laszlo]
> >>>>     - Fix bug when passing BaseAddress to gDS->AllocateIoSpace and
> >>>>       gDS->AllocateMemorySpace [Laszlo]
> >>>>     - Add comment for applying alignment to both device address and host
> >>>>       address, and add NOTE for the alignment requirement of Translation,
> >>>>       as well as in commit message [Laszlo][Ray]
> >>>>     - Improve indention for the code in CreateRootBridge [Laszlo]
> >>>>     - Improve comment for Translation in PCI_ROOT_BRIDGE_APERTURE
> >>>>       definition [Laszlo]
> >>>>     - Ignore translation of bus in CreateRootBridge
> >>>>
> >>>>  MdeModulePkg/Bus/Pci/PciHostBridgeDxe/PciHostBridge.h   |  21 ++++
> >>>>  MdeModulePkg/Bus/Pci/PciHostBridgeDxe/PciHostResource.h |   3 +
> >>>>  MdeModulePkg/Bus/Pci/PciHostBridgeDxe/PciHostBridge.c   | 129 +++++++++++++++++---
> >>>>  MdeModulePkg/Bus/Pci/PciHostBridgeDxe/PciRootBridgeIo.c | 118 ++++++++++++++++--
> >>>>  4 files changed, 245 insertions(+), 26 deletions(-)
> >>>>
> >>>>diff --git a/MdeModulePkg/Bus/Pci/PciHostBridgeDxe/PciHostBridge.h b/MdeModulePkg/Bus/Pci/PciHostBridgeDxe/PciHostBridge.h
> >>>>index 9a8ca21f4819..c2791ea5c2a4 100644
> >>>>--- a/MdeModulePkg/Bus/Pci/PciHostBridgeDxe/PciHostBridge.h
> >>>>+++ b/MdeModulePkg/Bus/Pci/PciHostBridgeDxe/PciHostBridge.h
> >>>>@@ -38,6 +38,13 @@ typedef struct {
> >>>>  #define PCI_HOST_BRIDGE_FROM_THIS(a) CR (a, PCI_HOST_BRIDGE_INSTANCE, ResAlloc, PCI_HOST_BRIDGE_SIGNATURE)
> >>>>  //
> >>>>+// Macros to translate device address to host address and vice versa. According
> >>>>+// to UEFI 2.7, device address = host address + translation offset.
> >>>>+//
> >>>>+#define TO_HOST_ADDRESS(DeviceAddress,TranslationOffset) ((DeviceAddress) - (TranslationOffset))
> >>>>+#define TO_DEVICE_ADDRESS(HostAddress,TranslationOffset) ((HostAddress) + (TranslationOffset))
> >>>>+
> >>>>+//
> >>>>  // Driver Entry Point
> >>>>  //
> >>>>  /**
> >>>>@@ -247,6 +254,20 @@ ResourceConflict (
> >>>>    IN  PCI_HOST_BRIDGE_INSTANCE *HostBridge
> >>>>    );
> >>>>+/**
> >>>>+  This routine gets translation offset from a root bridge instance by resource type.
> >>>>+
> >>>>+  @param RootBridge The Root Bridge Instance for the resources.
> >>>>+  @param ResourceType The Resource Type of the translation offset.
> >>>>+
> >>>>+  @retval The Translation Offset of the specified resource.
> >>>>+**/
> >>>>+UINT64
> >>>>+GetTranslationByResourceType (
> >>>>+  IN  PCI_ROOT_BRIDGE_INSTANCE     *RootBridge,
> >>>>+  IN  PCI_RESOURCE_TYPE            ResourceType
> >>>>+  );
> >>>>+
> >>>>  extern EFI_METRONOME_ARCH_PROTOCOL *mMetronome;
> >>>>  extern EFI_CPU_IO2_PROTOCOL        *mCpuIo;
> >>>>  #endif
> >>>>diff --git a/MdeModulePkg/Bus/Pci/PciHostBridgeDxe/PciHostResource.h b/MdeModulePkg/Bus/Pci/PciHostBridgeDxe/PciHostResource.h
> >>>>index 8612c0c3251b..a6c3739db368 100644
> >>>>--- a/MdeModulePkg/Bus/Pci/PciHostBridgeDxe/PciHostResource.h
> >>>>+++ b/MdeModulePkg/Bus/Pci/PciHostBridgeDxe/PciHostResource.h
> >>>>@@ -38,6 +38,9 @@ typedef enum {
> >>>>  typedef struct {
> >>>>    PCI_RESOURCE_TYPE Type;
> >>>>+  //
> >>>>+  // Base is a host address
> >>>>+  //
> >>>>    UINT64            Base;
> >>>>    UINT64            Length;
> >>>>    UINT64            Alignment;
> >>>>diff --git a/MdeModulePkg/Bus/Pci/PciHostBridgeDxe/PciHostBridge.c b/MdeModulePkg/Bus/Pci/PciHostBridgeDxe/PciHostBridge.c
> >>>>index 1494848c3e8c..42ded2855c71 100644
> >>>>--- a/MdeModulePkg/Bus/Pci/PciHostBridgeDxe/PciHostBridge.c
> >>>>+++ b/MdeModulePkg/Bus/Pci/PciHostBridgeDxe/PciHostBridge.c
> >>>>@@ -33,6 +33,39 @@ EFI_EVENT                   mIoMmuEvent;
> >>>>  VOID                        *mIoMmuRegistration;
> >>>>  /**
> >>>>+  This routine gets translation offset from a root bridge instance by resource type.
> >>>>+
> >>>>+  @param RootBridge The Root Bridge Instance for the resources.
> >>>>+  @param ResourceType The Resource Type of the translation offset.
> >>>>+
> >>>>+  @retval The Translation Offset of the specified resource.
> >>>>+**/
> >>>>+UINT64
> >>>>+GetTranslationByResourceType (
> >>>>+  IN  PCI_ROOT_BRIDGE_INSTANCE     *RootBridge,
> >>>>+  IN  PCI_RESOURCE_TYPE            ResourceType
> >>>>+  )
> >>>>+{
> >>>>+  switch (ResourceType) {
> >>>>+    case TypeIo:
> >>>>+      return RootBridge->Io.Translation;
> >>>>+    case TypeMem32:
> >>>>+      return RootBridge->Mem.Translation;
> >>>>+    case TypePMem32:
> >>>>+      return RootBridge->PMem.Translation;
> >>>>+    case TypeMem64:
> >>>>+      return RootBridge->MemAbove4G.Translation;
> >>>>+    case TypePMem64:
> >>>>+      return RootBridge->PMemAbove4G.Translation;
> >>>>+    case TypeBus:
> >>>>+      return RootBridge->Bus.Translation;
> >>>>+    default:
> >>>>+      ASSERT (FALSE);
> >>>>+      return 0;
> >>>>+  }
> >>>>+}
> >>>>+
> >>>>+/**
> >>>>    Ensure the compatibility of an IO space descriptor with the IO aperture.
> >>>>    The IO space descriptor can come from the GCD IO space map, or it can
> >>>>@@ -366,6 +399,7 @@ InitializePciHostBridge (
> >>>>    UINTN                       MemApertureIndex;
> >>>>    BOOLEAN                     ResourceAssigned;
> >>>>    LIST_ENTRY                  *Link;
> >>>>+  UINT64                      HostAddress;
> >>>>    RootBridges = PciHostBridgeGetRootBridges (&RootBridgeCount);
> >>>>    if ((RootBridges == NULL) || (RootBridgeCount == 0)) {
> >>>>@@ -411,8 +445,15 @@ InitializePciHostBridge (
> >>>>      }
> >>>>      if (RootBridges[Index].Io.Base <= RootBridges[Index].Io.Limit) {
> >>>>+      //
> >>>>+      // Base and Limit in PCI_ROOT_BRIDGE_APERTURE are device address.
> >>>>+      // For GCD resource manipulation, we need to use host address.
> >>>>+      //
> >>>>+      HostAddress = TO_HOST_ADDRESS (RootBridges[Index].Io.Base,
> >>>>+        RootBridges[Index].Io.Translation);
> >>>>+
> >>>>        Status = AddIoSpace (
> >>>>-                 RootBridges[Index].Io.Base,
> >>>>+                 HostAddress,
> >>>>                   RootBridges[Index].Io.Limit - RootBridges[Index].Io.Base + 1
> >>>>                   );
> >>>>        ASSERT_EFI_ERROR (Status);
> >>>>@@ -422,7 +463,7 @@ InitializePciHostBridge (
> >>>>                          EfiGcdIoTypeIo,
> >>>>                          0,
> >>>>                          RootBridges[Index].Io.Limit - RootBridges[Index].Io.Base + 1,
> >>>>-                        &RootBridges[Index].Io.Base,
> >>>>+                        &HostAddress,
> >>>>                          gImageHandle,
> >>>>                          NULL
> >>>>                          );
> >>>>@@ -443,14 +484,20 @@ InitializePciHostBridge (
> >>>>      for (MemApertureIndex = 0; MemApertureIndex < ARRAY_SIZE (MemApertures); MemApertureIndex++) {
> >>>>        if (MemApertures[MemApertureIndex]->Base <= MemApertures[MemApertureIndex]->Limit) {
> >>>>+        //
> >>>>+        // Base and Limit in PCI_ROOT_BRIDGE_APERTURE are device address.
> >>>>+        // For GCD resource manipulation, we need to use host address.
> >>>>+        //
> >>>>+        HostAddress = TO_HOST_ADDRESS (MemApertures[MemApertureIndex]->Base,
> >>>>+          MemApertures[MemApertureIndex]->Translation);
> >>>>          Status = AddMemoryMappedIoSpace (
> >>>>-                   MemApertures[MemApertureIndex]->Base,
> >>>>+                   HostAddress,
> >>>>                     MemApertures[MemApertureIndex]->Limit - MemApertures[MemApertureIndex]->Base + 1,
> >>>>                     EFI_MEMORY_UC
> >>>>                     );
> >>>>          ASSERT_EFI_ERROR (Status);
> >>>>          Status = gDS->SetMemorySpaceAttributes (
> >>>>-                        MemApertures[MemApertureIndex]->Base,
> >>>>+                        HostAddress,
> >>>>                          MemApertures[MemApertureIndex]->Limit - MemApertures[MemApertureIndex]->Base + 1,
> >>>>                          EFI_MEMORY_UC
> >>>>                          );
> >>>>@@ -463,7 +510,7 @@ InitializePciHostBridge (
> >>>>                            EfiGcdMemoryTypeMemoryMappedIo,
> >>>>                            0,
> >>>>                            MemApertures[MemApertureIndex]->Limit - MemApertures[MemApertureIndex]->Base + 1,
> >>>>-                          &MemApertures[MemApertureIndex]->Base,
> >>>>+                          &HostAddress,
> >>>>                            gImageHandle,
> >>>>                            NULL
> >>>>                            );
> >>>>@@ -654,6 +701,11 @@ AllocateResource (
> >>>>    if (BaseAddress < Limit) {
> >>>>      //
> >>>>      // Have to make sure Aligment is handled since we are doing direct address allocation
> >>>>+    // Strictly speaking, alignment requirement should be applied to device
> >>>>+    // address instead of host address which is used in GCD manipulation below,
> >>>>+    // but as we restrict the alignment of Translation to be larger than any BAR
> >>>>+    // alignment in the root bridge, we can simplify the situation and consider
> >>>>+    // the same alignment requirement is also applied to host address.
> >>>>      //
> >>>>      BaseAddress = ALIGN_VALUE (BaseAddress, LShiftU64 (1, BitsOfAlignment));
> >>>>@@ -721,6 +773,7 @@ NotifyPhase (
> >>>>    PCI_RESOURCE_TYPE                     Index2;
> >>>>    BOOLEAN                               ResNodeHandled[TypeMax];
> >>>>    UINT64                                MaxAlignment;
> >>>>+  UINT64                                Translation;
> >>>>    HostBridge = PCI_HOST_BRIDGE_FROM_THIS (This);
> >>>>@@ -822,14 +875,43 @@ NotifyPhase (
> >>>>            BitsOfAlignment = LowBitSet64 (Alignment + 1);
> >>>>            BaseAddress = MAX_UINT64;
> >>>>+          //
> >>>>+          // RESTRICTION: To simplify the situation, we require the alignment of
> >>>>+          // Translation must be larger than any BAR alignment in the same root
> >>>>+          // bridge, so that resource allocation alignment can be applied to
> >>>>+          // both device address and host address.
> >>>>+          //
> >>>>+          Translation = GetTranslationByResourceType (RootBridge, Index);
> >>>>+          if ((Translation & Alignment) != 0) {
> >>>>+            DEBUG ((DEBUG_ERROR, "[%a:%d] Translation %lx is not aligned to %lx!\n",
> >>>>+              __FUNCTION__, __LINE__, Translation, Alignment
> >>>>+              ));
> >>>>+            ASSERT (FALSE);
> 1. Cool! But can you replace ASSERT(FALSE) with
>    ASSERT ((Translation & Alignment) == 0)?
> >>>>+            //
> >>>>+            // This may be caused by too large alignment or too small
> >>>>+            // Translation; pick the 1st possibility and return out of resource,
> >>>>+            // which can also go thru the same process for out of resource
> >>>>+            // outside the loop.
> >>>>+            //
> >>>>+            ReturnStatus = EFI_OUT_OF_RESOURCES;
> >>>>+            continue;
> >>>>+          }
> >>>>+
> >>>>            switch (Index) {
> >>>>            case TypeIo:
> >>>>+            //
> >>>>+            // Base and Limit in PCI_ROOT_BRIDGE_APERTURE are device address.
> >>>>+            // For AllocateResource is manipulating GCD resource, we need to use
> >>>>+            // host address here.
> >>>>+            //
> >>>>              BaseAddress = AllocateResource (
> >>>>                              FALSE,
> >>>>                              RootBridge->ResAllocNode[Index].Length,
> >>>>                              MIN (15, BitsOfAlignment),
> >>>>-                            ALIGN_VALUE (RootBridge->Io.Base, Alignment + 1),
> >>>>-                            RootBridge->Io.Limit
> >>>>+                            TO_HOST_ADDRESS (ALIGN_VALUE (RootBridge->Io.Base, Alignment + 1),
> >>>>+                              RootBridge->Io.Translation),
> >>>>+                            TO_HOST_ADDRESS (RootBridge->Io.Limit,
> >>>>+                              RootBridge->Io.Translation)
> >>>>                              );
> >>>>              break;
> >>>>@@ -838,8 +920,10 @@ NotifyPhase (
> >>>>                              TRUE,
> >>>>                              RootBridge->ResAllocNode[Index].Length,
> >>>>                              MIN (63, BitsOfAlignment),
> >>>>-                            ALIGN_VALUE (RootBridge->MemAbove4G.Base, Alignment + 1),
> >>>>-                            RootBridge->MemAbove4G.Limit
> >>>>+                            TO_HOST_ADDRESS (ALIGN_VALUE (RootBridge->MemAbove4G.Base, Alignment + 1),
> >>>>+                              RootBridge->MemAbove4G.Translation),
> >>>>+                            TO_HOST_ADDRESS (RootBridge->MemAbove4G.Limit,
> >>>>+                              RootBridge->MemAbove4G.Translation)
> >>>>                              );
> >>>>              if (BaseAddress != MAX_UINT64) {
> >>>>                break;
> >>>>@@ -853,8 +937,10 @@ NotifyPhase (
> >>>>                              TRUE,
> >>>>                              RootBridge->ResAllocNode[Index].Length,
> >>>>                              MIN (31, BitsOfAlignment),
> >>>>-                            ALIGN_VALUE (RootBridge->Mem.Base, Alignment + 1),
> >>>>-                            RootBridge->Mem.Limit
> >>>>+                            TO_HOST_ADDRESS (ALIGN_VALUE (RootBridge->Mem.Base, Alignment + 1),
> >>>>+                              RootBridge->Mem.Translation),
> >>>>+                            TO_HOST_ADDRESS (RootBridge->Mem.Limit,
> >>>>+                              RootBridge->Mem.Translation)
> >>>>                              );
> >>>>              break;
> >>>>@@ -863,8 +949,10 @@ NotifyPhase (
> >>>>                              TRUE,
> >>>>                              RootBridge->ResAllocNode[Index].Length,
> >>>>                              MIN (63, BitsOfAlignment),
> >>>>-                            ALIGN_VALUE (RootBridge->PMemAbove4G.Base, Alignment + 1),
> >>>>-                            RootBridge->PMemAbove4G.Limit
> >>>>+                            TO_HOST_ADDRESS (ALIGN_VALUE (RootBridge->PMemAbove4G.Base, Alignment + 1),
> >>>>+                              RootBridge->PMemAbove4G.Translation),
> >>>>+                            TO_HOST_ADDRESS (RootBridge->PMemAbove4G.Limit,
> >>>>+                              RootBridge->PMemAbove4G.Translation)
> >>>>                              );
> >>>>              if (BaseAddress != MAX_UINT64) {
> >>>>                break;
> >>>>@@ -877,8 +965,10 @@ NotifyPhase (
> >>>>                              TRUE,
> >>>>                              RootBridge->ResAllocNode[Index].Length,
> >>>>                              MIN (31, BitsOfAlignment),
> >>>>-                            ALIGN_VALUE (RootBridge->PMem.Base, Alignment + 1),
> >>>>-                            RootBridge->PMem.Limit
> >>>>+                            TO_HOST_ADDRESS (ALIGN_VALUE (RootBridge->PMem.Base, Alignment + 1),
> >>>>+                              RootBridge->PMem.Translation),
> >>>>+                            TO_HOST_ADDRESS (RootBridge->PMem.Limit,
> >>>>+                              RootBridge->PMem.Translation)
> >>>>                              );
> >>>>              break;
> >>>>@@ -1421,7 +1511,14 @@ GetProposedResources (
> >>>>            Descriptor->Desc                  = ACPI_ADDRESS_SPACE_DESCRIPTOR;
> >>>>            Descriptor->Len                   = sizeof (EFI_ACPI_ADDRESS_SPACE_DESCRIPTOR) - 3;;
> >>>>            Descriptor->GenFlag               = 0;
> >>>>-          Descriptor->AddrRangeMin          = RootBridge->ResAllocNode[Index].Base;
> >>>>+          //
> >>>>+          // AddrRangeMin in Resource Descriptor here should be device address
> >>>>+          // instead of host address, or else PCI bus driver cannot set correct
> >>>>+          // address into PCI BAR registers.
> >>>>+          // Base in ResAllocNode is a host address, so conversion is needed.
> >>>>+          //
> >>>>+          Descriptor->AddrRangeMin          = TO_DEVICE_ADDRESS (RootBridge->ResAllocNode[Index].Base,
> >>>>+            GetTranslationByResourceType (RootBridge, Index));
> >>>>            Descriptor->AddrRangeMax          = 0;
> >>>>            Descriptor->AddrTranslationOffset = (ResStatus == ResAllocated) ? EFI_RESOURCE_SATISFIED : PCI_RESOURCE_LESS;
> >>>>            Descriptor->AddrLen               = RootBridge->ResAllocNode[Index].Length;
> >>>>diff --git a/MdeModulePkg/Bus/Pci/PciHostBridgeDxe/PciRootBridgeIo.c b/MdeModulePkg/Bus/Pci/PciHostBridgeDxe/PciRootBridgeIo.c
> >>>>index dc06c16dc038..5dd9d257d46d 100644
> >>>>--- a/MdeModulePkg/Bus/Pci/PciHostBridgeDxe/PciRootBridgeIo.c
> >>>>+++ b/MdeModulePkg/Bus/Pci/PciHostBridgeDxe/PciRootBridgeIo.c
> >>>>@@ -86,12 +86,35 @@ CreateRootBridge (
> >>>>            (Bridge->AllocationAttributes & EFI_PCI_HOST_BRIDGE_COMBINE_MEM_PMEM) != 0 ? L"CombineMemPMem " : L"",
> >>>>            (Bridge->AllocationAttributes & EFI_PCI_HOST_BRIDGE_MEM64_DECODE) != 0 ? L"Mem64Decode" : L""
> >>>>            ));
> >>>>+  //
> >>>>+  // Translation for bus is not supported.
> >>>>+  //
> >>>>    DEBUG ((EFI_D_INFO, "           Bus: %lx - %lx\n", Bridge->Bus.Base, Bridge->Bus.Limit));
> >>>>-  DEBUG ((EFI_D_INFO, "            Io: %lx - %lx\n", Bridge->Io.Base, Bridge->Io.Limit));
> >>>>-  DEBUG ((EFI_D_INFO, "           Mem: %lx - %lx\n", Bridge->Mem.Base, Bridge->Mem.Limit));
> >>>>-  DEBUG ((EFI_D_INFO, "    MemAbove4G: %lx - %lx\n", Bridge->MemAbove4G.Base, Bridge->MemAbove4G.Limit));
> >>>>-  DEBUG ((EFI_D_INFO, "          PMem: %lx - %lx\n", Bridge->PMem.Base, Bridge->PMem.Limit));
> >>>>-  DEBUG ((EFI_D_INFO, "   PMemAbove4G: %lx - %lx\n", Bridge->PMemAbove4G.Base, Bridge->PMemAbove4G.Limit));
> >>>>+  ASSERT (Bridge->Bus.Translation == 0);
> >>>>+  if (Bridge->Bus.Translation != 0) {
> >>>>+    return NULL;
> >>>>+  }
> 
> 2. Can you please use the same debug message format for Bus range? I mean to
> print the Translation for Bus as well. Then in the end of debug message, use
> assertion and if-check. This way developer can know what the Bus.Translation
> is from the debug log.
> 
> >>>>+
> >>>>+  DEBUG ((
> >>>>+    DEBUG_INFO, "            Io: %lx - %lx Translation=%lx\n",
> >>>>+    Bridge->Io.Base, Bridge->Io.Limit, Bridge->Io.Translation
> >>>>+    ));
> >>>>+  DEBUG ((
> >>>>+    DEBUG_INFO, "           Mem: %lx - %lx Translation=%lx\n",
> >>>>+    Bridge->Mem.Base, Bridge->Mem.Limit, Bridge->Mem.Translation
> >>>>+    ));
> >>>>+  DEBUG ((
> >>>>+    DEBUG_INFO, "    MemAbove4G: %lx - %lx Translation=%lx\n",
> >>>>+    Bridge->MemAbove4G.Base, Bridge->MemAbove4G.Limit, Bridge->MemAbove4G.Translation
> >>>>+    ));
> >>>>+  DEBUG ((
> >>>>+    DEBUG_INFO, "          PMem: %lx - %lx Translation=%lx\n",
> >>>>+    Bridge->PMem.Base, Bridge->PMem.Limit, Bridge->PMem.Translation
> >>>>+    ));
> >>>>+  DEBUG ((
> >>>>+    DEBUG_INFO, "   PMemAbove4G: %lx - %lx Translation=%lx\n",
> >>>>+    Bridge->PMemAbove4G.Base, Bridge->PMemAbove4G.Limit, Bridge->PMemAbove4G.Translation
> >>>>+    ));
> >>>>    //
> >>>>    // Make sure Mem and MemAbove4G apertures are valid
> >>>>@@ -206,7 +229,12 @@ CreateRootBridge (
> >>>>      }
> >>>>      RootBridge->ResAllocNode[Index].Type     = Index;
> >>>>      if (Bridge->ResourceAssigned && (Aperture->Limit >= Aperture->Base)) {
> >>>>-      RootBridge->ResAllocNode[Index].Base   = Aperture->Base;
> >>>>+      //
> >>>>+      // Base in ResAllocNode is a host address, while Base in Aperture is a
> >>>>+      // device address.
> >>>>+      //
> >>>>+      RootBridge->ResAllocNode[Index].Base   = TO_HOST_ADDRESS (Aperture->Base,
> >>>>+        Aperture->Translation);
> >>>>        RootBridge->ResAllocNode[Index].Length = Aperture->Limit - Aperture->Base + 1;
> >>>>        RootBridge->ResAllocNode[Index].Status = ResAllocated;
> >>>>      } else {
> >>>>@@ -403,6 +431,28 @@ RootBridgeIoCheckParameter (
> >>>>    return EFI_SUCCESS;
> >>>>  }
> 
> 3. Please add the missing function header comments.
> 
> >>>>+EFI_STATUS
> >>>>+RootBridgeIoGetMemTranslationByAddress (
> >>>>+  IN PCI_ROOT_BRIDGE_INSTANCE               *RootBridge,
> >>>>+  IN UINT64                                 Address,
> >>>>+  IN OUT UINT64                             *Translation
> >>>>+  )
> >>>>+{
> >>>>+  if (Address >= RootBridge->Mem.Base && Address <= RootBridge->Mem.Limit) {
> >>>>+    *Translation = RootBridge->Mem.Translation;
> >>>>+  } else if (Address >= RootBridge->PMem.Base && Address <= RootBridge->PMem.Limit) {
> >>>>+    *Translation = RootBridge->PMem.Translation;
> >>>>+  } else if (Address >= RootBridge->MemAbove4G.Base && Address <= RootBridge->MemAbove4G.Limit) {
> >>>>+    *Translation = RootBridge->MemAbove4G.Translation;
> >>>>+  } else if (Address >= RootBridge->PMemAbove4G.Base && Address <= RootBridge->PMemAbove4G.Limit) {
> >>>>+    *Translation = RootBridge->PMemAbove4G.Translation;
> >>>>+  } else {
> >>>>+    return EFI_INVALID_PARAMETER;
> >>>>+  }
> >>>>+
> >>>>+  return EFI_SUCCESS;
> >>>>+}
> >>>>+
> >>>>  /**
> >>>>    Polls an address in memory mapped I/O space until an exit condition is met,
> >>>>    or a timeout occurs.
> >>>>@@ -658,13 +708,25 @@ RootBridgeIoMemRead (
> >>>>    )
> >>>>  {
> >>>>    EFI_STATUS                             Status;
> >>>>+  PCI_ROOT_BRIDGE_INSTANCE               *RootBridge;
> >>>>+  UINT64                                 Translation;
> >>>>    Status = RootBridgeIoCheckParameter (This, MemOperation, Width, Address,
> >>>>                                         Count, Buffer);
> >>>>    if (EFI_ERROR (Status)) {
> >>>>      return Status;
> >>>>    }
> >>>>-  return mCpuIo->Mem.Read (mCpuIo, (EFI_CPU_IO_PROTOCOL_WIDTH) Width, Address, Count, Buffer);
> >>>>+
> >>>>+  RootBridge = ROOT_BRIDGE_FROM_THIS (This);
> >>>>+  Status = RootBridgeIoGetMemTranslationByAddress (RootBridge, Address, &Translation);
> >>>>+  if (EFI_ERROR (Status)) {
> >>>>+    return Status;
> >>>>+  }
> >>>>+
> >>>>+  // Address passed to CpuIo->Mem.Read needs to be a host address instead of
> >>>>+  // device address.
> >>>>+  return mCpuIo->Mem.Read (mCpuIo, (EFI_CPU_IO_PROTOCOL_WIDTH) Width,
> >>>>+      TO_HOST_ADDRESS (Address, Translation), Count, Buffer);
> >>>>  }
> >>>>  /**
> >>>>@@ -705,13 +767,25 @@ RootBridgeIoMemWrite (
> >>>>    )
> >>>>  {
> >>>>    EFI_STATUS                             Status;
> >>>>+  PCI_ROOT_BRIDGE_INSTANCE               *RootBridge;
> >>>>+  UINT64                                 Translation;
> >>>>    Status = RootBridgeIoCheckParameter (This, MemOperation, Width, Address,
> >>>>                                         Count, Buffer);
> >>>>    if (EFI_ERROR (Status)) {
> >>>>      return Status;
> >>>>    }
> >>>>-  return mCpuIo->Mem.Write (mCpuIo, (EFI_CPU_IO_PROTOCOL_WIDTH) Width, Address, Count, Buffer);
> >>>>+
> >>>>+  RootBridge = ROOT_BRIDGE_FROM_THIS (This);
> >>>>+  Status = RootBridgeIoGetMemTranslationByAddress (RootBridge, Address, &Translation);
> >>>>+  if (EFI_ERROR (Status)) {
> >>>>+    return Status;
> >>>>+  }
> >>>>+
> >>>>+  // Address passed to CpuIo->Mem.Write needs to be a host address instead of
> >>>>+  // device address.
> >>>>+  return mCpuIo->Mem.Write (mCpuIo, (EFI_CPU_IO_PROTOCOL_WIDTH) Width,
> >>>>+      TO_HOST_ADDRESS (Address, Translation), Count, Buffer);
> >>>>  }
> >>>>  /**
> >>>>@@ -746,6 +820,8 @@ RootBridgeIoIoRead (
> >>>>    )
> >>>>  {
> >>>>    EFI_STATUS                                    Status;
> >>>>+  PCI_ROOT_BRIDGE_INSTANCE                      *RootBridge;
> >>>>+
> >>>>    Status = RootBridgeIoCheckParameter (
> >>>>               This, IoOperation, Width,
> >>>>               Address, Count, Buffer
> >>>>@@ -753,7 +829,13 @@ RootBridgeIoIoRead (
> >>>>    if (EFI_ERROR (Status)) {
> >>>>      return Status;
> >>>>    }
> >>>>-  return mCpuIo->Io.Read (mCpuIo, (EFI_CPU_IO_PROTOCOL_WIDTH) Width, Address, Count, Buffer);
> >>>>+
> >>>>+  RootBridge = ROOT_BRIDGE_FROM_THIS (This);
> >>>>+
> >>>>+  // Address passed to CpuIo->Io.Read needs to be a host address instead of
> >>>>+  // device address.
> >>>>+  return mCpuIo->Io.Read (mCpuIo, (EFI_CPU_IO_PROTOCOL_WIDTH) Width,
> >>>>+      TO_HOST_ADDRESS (Address, RootBridge->Io.Translation), Count, Buffer);
> >>>>  }
> >>>>  /**
> >>>>@@ -788,6 +870,8 @@ RootBridgeIoIoWrite (
> >>>>    )
> >>>>  {
> >>>>    EFI_STATUS                                    Status;
> >>>>+  PCI_ROOT_BRIDGE_INSTANCE                      *RootBridge;
> >>>>+
> >>>>    Status = RootBridgeIoCheckParameter (
> >>>>               This, IoOperation, Width,
> >>>>               Address, Count, Buffer
> >>>>@@ -795,7 +879,13 @@ RootBridgeIoIoWrite (
> >>>>    if (EFI_ERROR (Status)) {
> >>>>      return Status;
> >>>>    }
> >>>>-  return mCpuIo->Io.Write (mCpuIo, (EFI_CPU_IO_PROTOCOL_WIDTH) Width, Address, Count, Buffer);
> >>>>+
> >>>>+  RootBridge = ROOT_BRIDGE_FROM_THIS (This);
> >>>>+
> >>>>+  // Address passed to CpuIo->Io.Write needs to be a host address instead of
> >>>>+  // device address.
> >>>>+  return mCpuIo->Io.Write (mCpuIo, (EFI_CPU_IO_PROTOCOL_WIDTH) Width,
> >>>>+      TO_HOST_ADDRESS (Address, RootBridge->Io.Translation), Count, Buffer);
> >>>>  }
> >>>>  /**
> >>>>@@ -1615,9 +1705,17 @@ RootBridgeIoConfiguration (
> >>>>      Descriptor->Desc = ACPI_ADDRESS_SPACE_DESCRIPTOR;
> >>>>      Descriptor->Len  = sizeof (EFI_ACPI_ADDRESS_SPACE_DESCRIPTOR) - 3;
> >>>>+    // According to UEFI 2.7, RootBridgeIo->Configuration should return address
> >>>>+    // range in CPU view (host address), and ResAllocNode->Base is already a CPU
> >>>>+    // view address (host address).
> >>>>      Descriptor->AddrRangeMin  = ResAllocNode->Base;
> >>>>      Descriptor->AddrRangeMax  = ResAllocNode->Base + ResAllocNode->Length - 1;
> >>>>      Descriptor->AddrLen       = ResAllocNode->Length;
> >>>>+    Descriptor->AddrTranslationOffset = GetTranslationByResourceType (
> >>>>+      RootBridge,
> >>>>+      ResAllocNode->Type
> >>>>+      );
> >>>>+
> >>>>      switch (ResAllocNode->Type) {
> >>>>      case TypeIo:
> >>>>-- 
> >>>>2.7.4
> >>>>
> >>
> >>
> >>-- 
> >>Thanks,
> >>Ray
> 
> 
> Reviewed-by: Ruiyu Ni <ruiyu.ni@intel.com>
> Please modify code according to the above 3 comments and make sure
> the changes pass ECC check:
> BaseTools/Source/Python/Ecc/Ecc.py -t -s \
>  MdeModulePkg/Pci/PciHostBridgeDxe
> 
> >_______________________________________________
> >edk2-devel mailing list
> >edk2-devel@lists.01.org
> >https://lists.01.org/mailman/listinfo/edk2-devel
> >
> 
> 
> -- 
> Thanks,
> Ray

Re: [PATCH v5 4/6] MdeModulePkg/PciHostBridgeDxe: Add support for address translation

[Ni, Ruiyu]

On 3/15/2018 10:57 AM, Guo Heyi wrote:
> Hi Ray,
> 
> Sorry I never tried Ecc tool before, for there is few documents about it. When I
> tried running python BaseTools/Source/Python/Ecc/Ecc.py -t -s xxxx,
> it showed me with error message:
> 
> ImportError: No module named Common.LongFilePathOs
> 
> Then I found there was an executable named "Ecc" in
> BaseTools/BinWrappers/PosixLike/, and change the place of "-t" and "-s" after it
> prompted:
> 
>   : error 1003: Invalid value of option
>          Target [-s] does NOT exist
> 
> However, it still failed with error:
> 
> RuntimeError: ANTLR version mismatch: The recognizer has been generated with API
> V0, but this runtime does not support this.
> 
> I could not find the reason for this error, maybe it was caused by incompatible
> version of python and antlr3 (mine is Ubuntu 16.04.3 LTS, python 2.7.12,
> python-antlr3 3.5.2-1).
> 
> Finally I downloaded prebuilt Win32 BaseTools and ran Ecc.exe on Windows. It
> success and produced a "Report.csv" with only a titel line. I guess this means
> there is no error in the coding style, isn't it?
> 
> Please let me know if there is anything wrong with what I did.
Sorry I provided the wrong command line. It should be
"...Ecc.py -t <path> -s".

If you haven't add the function header comments as I pointed out.
You should get at least one ECC error.
Otherwise, the code change should be fine.

Thanks for the try.

> 
> Thanks and regards,
> 
> Heyi
> 

Re: [PATCH v5 4/6] MdeModulePkg/PciHostBridgeDxe: Add support for address translation

[Ard Biesheuvel]

On 7 March 2018 at 04:30, Ni, Ruiyu <ruiyu.ni@intel.com> wrote:
> On 3/6/2018 10:44 AM, Guo Heyi wrote:
>>
>> Hi Ray,
>>
>> Any comments for v5?
>
>
> Heyi,
> Some backward compatibility concerns were received from internal production
> teams. Current change will cause silent failure on old platforms because
> TranslationOffset might be random if uninitialized.
> I will solve the concern and then send out updates to you, hopefully by end
> of next week.
>

Hello Ray,

Do you have any updates on these issues? I'd like to proceed with
getting this code merged if we can

Thanks,
Ard.

Re: [PATCH v5 4/6] MdeModulePkg/PciHostBridgeDxe: Add support for address translation

[Ni, Ruiyu]

On 3/13/2018 1:18 AM, Ard Biesheuvel wrote:
> On 7 March 2018 at 04:30, Ni, Ruiyu <ruiyu.ni@intel.com> wrote:
>> On 3/6/2018 10:44 AM, Guo Heyi wrote:
>>>
>>> Hi Ray,
>>>
>>> Any comments for v5?
>>
>>
>> Heyi,
>> Some backward compatibility concerns were received from internal production
>> teams. Current change will cause silent failure on old platforms because
>> TranslationOffset might be random if uninitialized.
>> I will solve the concern and then send out updates to you, hopefully by end
>> of next week.
>>
> 
> Hello Ray,
> 
> Do you have any updates on these issues? I'd like to proceed with
> getting this code merged if we can
> 
> Thanks,
> Ard.
> 
Ard，
I will try to give updates tomorrow.
If I cannot, I will let you know.
Then I will give you updates by the end of this week.

-- 
Thanks,
Ray

Re: [PATCH v5 4/6] MdeModulePkg/PciHostBridgeDxe: Add support for address translation

[Guo Heyi]

Hi Ray,

If the major concern is about backward compatibility, we can think about ways to
get around of this, e.g. add a feature PCD to turn on translation and set the
default value to false.

Thanks,
Heyi

On Tue, Mar 13, 2018 at 11:00:12AM +0800, Ni, Ruiyu wrote:
> On 3/13/2018 1:18 AM, Ard Biesheuvel wrote:
> >On 7 March 2018 at 04:30, Ni, Ruiyu <ruiyu.ni@intel.com> wrote:
> >>On 3/6/2018 10:44 AM, Guo Heyi wrote:
> >>>
> >>>Hi Ray,
> >>>
> >>>Any comments for v5?
> >>
> >>
> >>Heyi,
> >>Some backward compatibility concerns were received from internal production
> >>teams. Current change will cause silent failure on old platforms because
> >>TranslationOffset might be random if uninitialized.
> >>I will solve the concern and then send out updates to you, hopefully by end
> >>of next week.
> >>
> >
> >Hello Ray,
> >
> >Do you have any updates on these issues? I'd like to proceed with
> >getting this code merged if we can
> >
> >Thanks,
> >Ard.
> >
> Ard，
> I will try to give updates tomorrow.
> If I cannot, I will let you know.
> Then I will give you updates by the end of this week.
> 
> -- 
> Thanks,
> Ray

Re: [PATCH v5 4/6] MdeModulePkg/PciHostBridgeDxe: Add support for address translation

[Ard Biesheuvel]

On 13 March 2018 at 07:31, Guo Heyi <heyi.guo@linaro.org> wrote:
> Hi Ray,
>
> If the major concern is about backward compatibility, we can think about ways to
> get around of this, e.g. add a feature PCD to turn on translation and set the
> default value to false.
>

I don't think that is necessary. Like any other project, internal
Intel projects should occasionally expect to have to adapt their code
to accommodate upstream changes. If that is an issue, they should use
a UDK branch instead.

[PATCH v5 5/6] MdeModulePkg/PciBus: convert host address to device address

[Heyi Guo]

According to UEFI spec 2.7, PciRootBridgeIo->Configuration() should
return host address (CPU view ddress) rather than device address
(PCI view address), so in function GetMmioAddressTranslationOffset we
need to convert the range to device address before comparing.

And device address = host address + translation offset.

Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Heyi Guo <heyi.guo@linaro.org>
Cc: Ruiyu Ni <ruiyu.ni@intel.com>
Cc: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Cc: Star Zeng <star.zeng@intel.com>
Cc: Eric Dong <eric.dong@intel.com>
Cc: Laszlo Ersek <lersek@redhat.com>
Cc: Michael D Kinney <michael.d.kinney@intel.com>
---
 MdeModulePkg/Bus/Pci/PciBusDxe/PciIo.c | 8 ++++++--
 1 file changed, 6 insertions(+), 2 deletions(-)

diff --git a/MdeModulePkg/Bus/Pci/PciBusDxe/PciIo.c b/MdeModulePkg/Bus/Pci/PciBusDxe/PciIo.c
index 190f4b0dc7ed..fef3eceb7f62 100644
--- a/MdeModulePkg/Bus/Pci/PciBusDxe/PciIo.c
+++ b/MdeModulePkg/Bus/Pci/PciBusDxe/PciIo.c
@@ -1812,10 +1812,14 @@ GetMmioAddressTranslationOffset (
     return (UINT64) -1;
   }
 
+  // According to UEFI 2.7, EFI_PCI_ROOT_BRIDGE_IO_PROTOCOL::Configuration()
+  // returns host address instead of device address, while AddrTranslationOffset
+  // is not zero, and device address = host address + AddrTranslationOffset, so
+  // we convert host address to device address for range compare.
   while (Configuration->Desc == ACPI_ADDRESS_SPACE_DESCRIPTOR) {
     if ((Configuration->ResType == ACPI_ADDRESS_SPACE_TYPE_MEM) &&
-        (Configuration->AddrRangeMin <= AddrRangeMin) &&
-        (Configuration->AddrRangeMin + Configuration->AddrLen >= AddrRangeMin + AddrLen)
+        (Configuration->AddrRangeMin + Configuration->AddrTranslationOffset <= AddrRangeMin) &&
+        (Configuration->AddrRangeMin + Configuration->AddrLen + Configuration->AddrTranslationOffset >= AddrRangeMin + AddrLen)
         ) {
       return Configuration->AddrTranslationOffset;
     }
-- 
2.7.4

[PATCH v5 6/6] MdeModulePkg/PciBus: return CPU address for GetBarAttributes

[Heyi Guo]

According to UEFI spec 2.7, PciIo->GetBarAttributes should return host
address (CPU view ddress) rather than device address (PCI view
address), and
device address = host address + address translation offset,
so we subtract translation from device address before returning.

Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Heyi Guo <heyi.guo@linaro.org>
Cc: Ruiyu Ni <ruiyu.ni@intel.com>
Cc: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Cc: Star Zeng <star.zeng@intel.com>
Cc: Eric Dong <eric.dong@intel.com>
Cc: Laszlo Ersek <lersek@redhat.com>
Cc: Michael D Kinney <michael.d.kinney@intel.com>
---
 MdeModulePkg/Bus/Pci/PciBusDxe/PciIo.c | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/MdeModulePkg/Bus/Pci/PciBusDxe/PciIo.c b/MdeModulePkg/Bus/Pci/PciBusDxe/PciIo.c
index fef3eceb7f62..62179eb44bbd 100644
--- a/MdeModulePkg/Bus/Pci/PciBusDxe/PciIo.c
+++ b/MdeModulePkg/Bus/Pci/PciBusDxe/PciIo.c
@@ -1972,6 +1972,10 @@ PciIoGetBarAttributes (
         return EFI_UNSUPPORTED;
       }
     }
+
+    // According to UEFI spec 2.7, we need return host address for
+    // PciIo->GetBarAttributes, and host address = device address - translation.
+    Descriptor->AddrRangeMin -= Descriptor->AddrTranslationOffset;
   }
 
   return EFI_SUCCESS;
-- 
2.7.4

Re: [PATCH v5 0/6] Add translation support to generic PciHostBridge

[Ard Biesheuvel]

On 1 March 2018 at 06:57, Heyi Guo <heyi.guo@linaro.org> wrote:
> Patch v5 inherits the code from RFC v4; we don't restart the version number for
> RFC to PATCH change.
>
> v5:
> - Patch 4/6: Modify the code according to the comments from Ray.
> - Patch 1/6 and 2/6 are totally new. They add initialization for all fields of
>   PCI_ROOT_BRIDGE_APERTURE temporary variables in PciHostBridgeLib instances, so
>   that they will not suffer from extension of PCI_ROOT_BRIDGE_APERTURE
>   structure.
> - Generate a separate patch (3/6) for PciHostBridgeLib.h change. Though it is a
>   prerequisite for patch 4/6, it does not change the code in PciHostBridge
>   driver and won't cause any build failure or functional issue.
>
>
> v4:
> - Modify the code according to the comments from Ray, Laszlo and Ard (Please see
>   the notes of Patch 1/3)
> - Ignore translation of bus in CreateRootBridge.
>
>
> v3:
> - Keep definition of Translation consistent in EDKII code: Translation = device
>   address - host address.
> - Patch 2/2 is split into 2 patches (2/3 and 3/3).
> - Refine comments and commit messages to make the code easier to understand.
>
>
> v2:
> Changs are made according to the discussion on the mailing list, including:
>
> - PciRootBridgeIo->Configuration should return CPU view address, as well as
>   PciIo->GetBarAttributes, and Translation Offset should be equal to PCI view
>   address - CPU view address.
> - Add translation offset to PCI_ROOT_BRIDGE_APERTURE structure definition.
> - PciHostBridge driver internally used Base Address is still based on PCI view
>   address, and translation offset = CPU view - PCI view, which follows the
>   definition in ACPI, and not the same as that in UEFI spec.
>

Heyi,

Thanks again for taking the time to implement this.

I have applied the patches and they appear to work on my SynQuacer
system, which has two PCIe RCs of which one uses translation for the
I/O space

I.e.,

PCI0 has 0x0..0xffff mapped to MMIO offset 0x67f00000
PCI1 has 0x0..0xffff mapped to MMIO offset 0x77f00000

The only problem I am hitting now is that the 'mm' shell command has a
hard coded 16-bit limit for IO space, which we should probably fix as
well.

If I remove this limit, I can correctly access the I/O space of PCI1 like this:

Shell> mm 77f00010
MEM  0x0000000077F00010 : 0x00 >
MEM  0x0000000077F00011 : 0x03 >
MEM  0x0000000077F00012 : 0x00 >
MEM  0x0000000077F00013 : 0x00 >
MEM  0x0000000077F00014 : 0x00 >
MEM  0x0000000077F00015 : 0xF0 >
MEM  0x0000000077F00016 : 0x35 >
MEM  0x0000000077F00017 : 0xFD > q

Shell> mm 10010 -io
IO  0x0000000000010010 : 0x00 >
IO  0x0000000000010011 : 0x03 >
IO  0x0000000000010012 : 0x00 >
IO  0x0000000000010013 : 0x00 >
IO  0x0000000000010014 : 0x00 >
IO  0x0000000000010015 : 0xF0 >
IO  0x0000000000010016 : 0x35 >
IO  0x0000000000010017 : 0xFD > q

Tested-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>


> Cc: Ruiyu Ni <ruiyu.ni@intel.com>
> Cc: Ard Biesheuvel <ard.biesheuvel@linaro.org>
> Cc: Star Zeng <star.zeng@intel.com>
> Cc: Eric Dong <eric.dong@intel.com>
> Cc: Laszlo Ersek <lersek@redhat.com>
> Cc: Michael D Kinney <michael.d.kinney@intel.com>
> Cc: Maurice Ma <maurice.ma@intel.com>
> Cc: Prince Agyeman <prince.agyeman@intel.com>
> Cc: Benjamin You <benjamin.you@intel.com>
> Cc: Jordan Justen <jordan.l.justen@intel.com>
> Cc: Anthony Perard <anthony.perard@citrix.com>
> Cc: Julien Grall <julien.grall@linaro.org>
>
> Heyi Guo (6):
>   CorebootPayloadPkg/PciHostBridgeLib: Init PCI aperture to 0
>   OvmfPkg/PciHostBridgeLib: Init PCI aperture to 0
>   MdeModulePkg/PciHostBridgeLib.h: add address Translation
>   MdeModulePkg/PciHostBridgeDxe: Add support for address translation
>   MdeModulePkg/PciBus: convert host address to device address
>   MdeModulePkg/PciBus: return CPU address for GetBarAttributes
>
>  MdeModulePkg/Bus/Pci/PciHostBridgeDxe/PciHostBridge.h              |  21 ++++
>  MdeModulePkg/Bus/Pci/PciHostBridgeDxe/PciHostResource.h            |   3 +
>  MdeModulePkg/Include/Library/PciHostBridgeLib.h                    |  19 +++
>  CorebootPayloadPkg/Library/PciHostBridgeLib/PciHostBridgeSupport.c |   5 +
>  MdeModulePkg/Bus/Pci/PciBusDxe/PciIo.c                             |  12 +-
>  MdeModulePkg/Bus/Pci/PciHostBridgeDxe/PciHostBridge.c              | 129 +++++++++++++++++---
>  MdeModulePkg/Bus/Pci/PciHostBridgeDxe/PciRootBridgeIo.c            | 118 ++++++++++++++++--
>  OvmfPkg/Library/PciHostBridgeLib/PciHostBridgeLib.c                |   4 +
>  OvmfPkg/Library/PciHostBridgeLib/XenSupport.c                      |   5 +
>  9 files changed, 288 insertions(+), 28 deletions(-)
>
> --
> 2.7.4
>

Re: [PATCH v5 0/6] Add translation support to generic PciHostBridge

[Ni, Ruiyu]

On 3/1/2018 2:57 PM, Heyi Guo wrote:
> Patch v5 inherits the code from RFC v4; we don't restart the version number for
> RFC to PATCH change.
> 
> v5:
> - Patch 4/6: Modify the code according to the comments from Ray.
> - Patch 1/6 and 2/6 are totally new. They add initialization for all fields of
>    PCI_ROOT_BRIDGE_APERTURE temporary variables in PciHostBridgeLib instances, so
>    that they will not suffer from extension of PCI_ROOT_BRIDGE_APERTURE
>    structure.
> - Generate a separate patch (3/6) for PciHostBridgeLib.h change. Though it is a
>    prerequisite for patch 4/6, it does not change the code in PciHostBridge
>    driver and won't cause any build failure or functional issue.
> 
> 
> v4:
> - Modify the code according to the comments from Ray, Laszlo and Ard (Please see
>    the notes of Patch 1/3)
> - Ignore translation of bus in CreateRootBridge.
> 
> 
> v3:
> - Keep definition of Translation consistent in EDKII code: Translation = device
>    address - host address.
> - Patch 2/2 is split into 2 patches (2/3 and 3/3).
> - Refine comments and commit messages to make the code easier to understand.
> 
> 
> v2:
> Changs are made according to the discussion on the mailing list, including:
> 
> - PciRootBridgeIo->Configuration should return CPU view address, as well as
>    PciIo->GetBarAttributes, and Translation Offset should be equal to PCI view
>    address - CPU view address.
> - Add translation offset to PCI_ROOT_BRIDGE_APERTURE structure definition.
> - PciHostBridge driver internally used Base Address is still based on PCI view
>    address, and translation offset = CPU view - PCI view, which follows the
>    definition in ACPI, and not the same as that in UEFI spec.
> 
> Cc: Ruiyu Ni <ruiyu.ni@intel.com>
> Cc: Ard Biesheuvel <ard.biesheuvel@linaro.org>
> Cc: Star Zeng <star.zeng@intel.com>
> Cc: Eric Dong <eric.dong@intel.com>
> Cc: Laszlo Ersek <lersek@redhat.com>
> Cc: Michael D Kinney <michael.d.kinney@intel.com>
> Cc: Maurice Ma <maurice.ma@intel.com>
> Cc: Prince Agyeman <prince.agyeman@intel.com>
> Cc: Benjamin You <benjamin.you@intel.com>
> Cc: Jordan Justen <jordan.l.justen@intel.com>
> Cc: Anthony Perard <anthony.perard@citrix.com>
> Cc: Julien Grall <julien.grall@linaro.org>
> 
> Heyi Guo (6):
>    CorebootPayloadPkg/PciHostBridgeLib: Init PCI aperture to 0
>    OvmfPkg/PciHostBridgeLib: Init PCI aperture to 0
>    MdeModulePkg/PciHostBridgeLib.h: add address Translation
>    MdeModulePkg/PciHostBridgeDxe: Add support for address translation
>    MdeModulePkg/PciBus: convert host address to device address
>    MdeModulePkg/PciBus: return CPU address for GetBarAttributes
> 
>   MdeModulePkg/Bus/Pci/PciHostBridgeDxe/PciHostBridge.h              |  21 ++++
>   MdeModulePkg/Bus/Pci/PciHostBridgeDxe/PciHostResource.h            |   3 +
>   MdeModulePkg/Include/Library/PciHostBridgeLib.h                    |  19 +++
>   CorebootPayloadPkg/Library/PciHostBridgeLib/PciHostBridgeSupport.c |   5 +
>   MdeModulePkg/Bus/Pci/PciBusDxe/PciIo.c                             |  12 +-
>   MdeModulePkg/Bus/Pci/PciHostBridgeDxe/PciHostBridge.c              | 129 +++++++++++++++++---
>   MdeModulePkg/Bus/Pci/PciHostBridgeDxe/PciRootBridgeIo.c            | 118 ++++++++++++++++--
>   OvmfPkg/Library/PciHostBridgeLib/PciHostBridgeLib.c                |   4 +
>   OvmfPkg/Library/PciHostBridgeLib/XenSupport.c                      |   5 +
>   9 files changed, 288 insertions(+), 28 deletions(-)
> 
Reviewed-by: Ruiyu Ni <ruiyu.ni@intel.com>

-- 
Thanks,
Ray