Re: [RFC v2 1/2] MdeModulePkg/PciHostBridgeDxe: Add support for address translation

[Laszlo Ersek <lersek@redhat.com>]

Hello Heyi,

On 02/22/18 07:54, Heyi Guo wrote:
> This is the draft patch for the discussion posted in edk2-devel
> mailing list:
> https://lists.01.org/pipermail/edk2-devel/2017-December/019289.html
> 
> As discussed in the mailing list, we'd like to add support for PCI
> address translation which is necessary for some non-x86 platforms. I
> also want to minimize the changes to the generic host bridge driver
> and platform PciHostBridgeLib implemetations, so additional two
> interfaces are added to expose translation information of the
> platform. To be generic, I add translation for each type of IO or
> memory resources.
> 
> The patch is still a RFC, so I only passed the build for qemu64 and
> the function has not been tested yet.
> 
> Please let me know your comments about it.
> 
> Thanks.

I have two requests here:

(1) In the commit message -- and, as requested by Ray, near the
PCI_ROOT_BRIDGE_APERTURE typedef too --, please document the *exact*
mathematical formula that defines how the translation is supposed to work:

  DeviceAddress = HostAddress + AddressTranslationOffset

versus

  HostAddress = DeviceAddress + AddressTranslationOffset

We've have a whole lot of discussion around that, both on this list and
on the PIWG and/or USWG lists, and it's still unclear to me.

The last discussion I recall (that I was involved in) starts here:

  [edk2] [PATCH] MdeModulePkg/PciBusDxe: GetBarAttributes() returns Host
                 address
  https://lists.01.org/pipermail/edk2-devel/2017-September/014425.html

Due to our (apparently!) collective confusion in that thread, Ray
decided back then not to change the code at all.

So, have we cleared up the confusion? Do we have an agreement on what
formula to use? Is the formula consistent with the UEFI spec?

Reviewing the (more recent, December 2017) thread that you link in the
commit message (which I apparently didn't participate in, apologies), I
find the following message from Ray:

https://lists.01.org/pipermail/edk2-devel/2017-December/019335.html

> And further more, there was a discussion in the mailing list
> regarding how to utilize the Offset. There was 2 points
> about the meaning of Offset and we cannot agree with each
> other. Spec is not quite clear.
> 1. Offset = Host - Device
> 2. Offset = Device - Host

Based on your followup to that,
<https://lists.01.org/pipermail/edk2-devel/2017-December/019341.html>,
it looks like you picked #1, or in other words,

  HostAddress = DeviceAddress + AddressTranslationOffset

To quote Ray again from the September 2017 thread, 'Your understanding
to "apply to" is "add", my understanding is "minus"'.

So, if we go with this approach, please quote the
EFI_PCI_IO_PROTOCOL.GetBarAttributes() language from the UEFI 2.7 spec
in the code, and explicitly define "apply" as *subtract*:

    "Translation Offset. Offset to apply to the Starting
    address of a BAR to convert it to a PCI address"

    with "apply" defined as "subtract"; in other words

    DeviceAddress = HostAddress - AddressTranslationOffset

> diff --git a/MdeModulePkg/Include/Library/PciHostBridgeLib.h b/MdeModulePkg/Include/Library/PciHostBridgeLib.h
> index d42e9ec..b9e8c0f 100644
> --- a/MdeModulePkg/Include/Library/PciHostBridgeLib.h
> +++ b/MdeModulePkg/Include/Library/PciHostBridgeLib.h
> @@ -22,6 +22,7 @@
>  typedef struct {
>    UINT64 Base;
>    UINT64 Limit;
> +  UINT64 Translation;
>  } PCI_ROOT_BRIDGE_APERTURE;
>  
>  typedef struct {
> 

(2) My second request is that you please audit all uses of
PCI_ROOT_BRIDGE_APERTURE and PCI_ROOT_BRIDGE in the edk2 tree. For example:

(2a) "ArmVirtPkg/Library/FdtPciHostBridgeLib/FdtPciHostBridgeLib.c" is
fine, it needs no updates. It uses

STATIC PCI_ROOT_BRIDGE mRootBridge;

so the new Translation field will be zero, in all the aperture fields. OK.

(2b) However, under "OvmfPkg/Library/PciHostBridgeLib", several updates
are needed. This library instance behaves differently on Xen vs. QEMU,
and both branches need fixes.

The InitRootBridge() function needs no change, because it calls
ZeroMem(), in order to "Be safe if other fields are added to
PCI_ROOT_BRIDGE later". OK.

There's also a global variable called "mNonExistAperture" where the
Translation field will be initialized to zero, implicitly. So that's OK too.

However, both the QEMU and the Xen code use *local variables* of type
PCI_ROOT_BRIDGE_APERTURE:

- in PciHostBridgeGetRootBridges(): Io, Mem, MemAbove4G
- in ScanForRootBridges(): Io, Mem, MemAbove4G, PMem, PMemAbove4G

The Translation field in all of these local structs would be
indeterminate, and then that would be copied into the PCI_ROOT_BRIDGE
object(s) exposed by PciHostBridgeGetRootBridges().

So, please zero out all of these local structs (with ZeroMem()) in their
respective functions, before they are populated member-wise. (Note that
in ScanForRootBridges(), the zeroing should occur at the top of the loop
body, replacing the manual zeroing of the .Limit fields.)

Thanks
Laszlo