Re: [edk2-devel] [PATCH] MdeModulePkg/PciHostBridge: Update Mem and PMem Limit Checks

["Wasim Khan" <wasim.khan@nxp.com>]

> -----Original Message-----
> From: Ni, Ray <ray.ni@intel.com>
> Sent: Thursday, April 23, 2020 8:47 PM
> To: Wasim Khan <wasim.khan@nxp.com>; devel@edk2.groups.io
> Cc: ard.biesheuvel@linaro.org; Varun Sethi <V.Sethi@nxp.com>; Wu, Hao A
> <hao.a.wu@intel.com>
> Subject: RE: [edk2-devel] [PATCH] MdeModulePkg/PciHostBridge: Update Mem
> and PMem Limit Checks
> 
> 
> 
> > -----Original Message-----
> > From: Wasim Khan <wasim.khan@nxp.com>
> > Sent: Thursday, April 23, 2020 10:54 PM
> > To: devel@edk2.groups.io; Ni, Ray <ray.ni@intel.com>
> > Cc: ard.biesheuvel@linaro.org; Varun Sethi <V.Sethi@nxp.com>; Wu, Hao
> > A <hao.a.wu@intel.com>
> > Subject: RE: [edk2-devel] [PATCH] MdeModulePkg/PciHostBridge: Update
> > Mem and PMem Limit Checks
> > > > >
> > > > > Thanks for fixing the check.
> > > > >
> > > > > PCI_ROOT_BRIDGE_APERTURE.Base/Limit are device address while the
> > > > > memory space in GCD belongs to host domain.
> > > > > So, host address for Mem/Pmem should be below 4GB while device
> > > > > address can across 4GB.
> > > > >
> > > >
> > > >
> > > > Hi Ray,
> > > > Thank you for the review.
> > > > There are cases when we don't have PCIe host address below 4GB,
> > > > and the PCIe HOST Address space is only available above 4GB.
> > > > For Example, Mem: 40000000 - FFFFFFFF Translation=FFFFFF6000000000
> > > > will result in HOST Address = 0xA0FFFFFFFF . This is a valid use
> > > > case, but below
> > > check will report ASSERT for this HOST ADDRESS.
> > >
> > > OK. Now I remember that "Mem" reports the 32bit memory space (device
> > > address) and "MemAbove4GB" reports the 64bit memory space (device
> address).
> > >
> > > Then if "Mem" reports memory range that across 4GB, it means the
> > > range above 4GB should be reported through "MemAbove4GB".
> > >
> > Yes this is true, but some devices needs MMIO 32bit space only as per their
> BAR property, including E1000 EP.
> 
> I understand some devices contain only 32bit MMIO BAR so only 32bit memory
> space (device address) can be assigned to them.
> Can you tell me the value of Mem/MemAbove4GB/Pmem/PmemAbove4GB in
> your real case?
> Can you also tell me the PCI(e) device BAR information you want to initialize
> through the EDKII PCI stack?

So as mentioned that in our case, we do not have Host region below 4GB. So we use MMIO + Translation.
For a PCIe device which needs 32 Bit, Non-prefetchable memory , below are the two experiments I could run.

With MMIO + translation , and a MMIO64 , for a device : (Works fine)
 
Mem: 40000000 - FFFFFFFF Translation=FFFFFF7000000000
 MemAbove4G: 9100000000 - 91FFFFFFFF Translation=0
 PMem: FFFFFFFFFFFFFFFF - 0 Translation=0
 PMemAbove4G: FFFFFFFFFFFFFFFF - 0 Translation=0

PciHostBridge: SubmitResources for PcieRoot(0x2)
Mem: Granularity/SpecificFlag = 32 / 00
      Length/Alignment = 0x6100000 / 0x3FFFFFF

PciHostBridge: NotifyPhase (AllocateResources)
 RootBridge: PcieRoot(0x2)
  Mem: Base/Length/Alignment = 9040000000/6100000/3FFFFFF - Success

EP's BAR0 : 0x46080000


With Only MMIO64: (Does not work)
 Mem: FFFFFFFFFFFFFFFF - 0 Translation=0
 MemAbove4G: 9100000000 - 91FFFFFFFF Translation=0
 PMem: FFFFFFFFFFFFFFFF - 0 Translation=0
 PMemAbove4G: FFFFFFFFFFFFFFFF - 0 Translation=0

PciHostBridge: SubmitResources for PcieRoot(0x2)
Mem: Granularity/SpecificFlag = 32 / 00
      Length/Alignment = 0x6100000 / 0x3FFFFFF

PciHostBridge: NotifyPhase (AllocateResources)
 RootBridge: PcieRoot(0x2)
  Mem: Base/Length/Alignment = FFFFFFFFFFFFFFFF/6100000/3FFFFFF - Out Of Resource!

EP's BAR0: 0xFFFE0000