Re: How do I access Prefetchable Memory region through EFI_PCI_IO_PROTOCOL

["Kirkendall, Garrett" <Garrett.Kirkendall@amd.com>]

Ray, did you get a chance to look at the code and see how Prefetchable Memory regions should be handled?

GARRETT KIRKENDALL
SMTS Firmware Engineer | CTE
7171 Southwest Parkway, Austin, TX 78735 USA 
AMD   facebook  |  amd.com

-----Original Message-----
From: Laszlo Ersek <lersek@redhat.com> 
Sent: Sunday, September 16, 2018 5:20 AM
To: Kirkendall, Garrett <Garrett.Kirkendall@amd.com>
Cc: edk2-devel@lists.01.org; Ruiyu Ni <ruiyu.ni@intel.com>
Subject: Re: [edk2] How do I access Prefetchable Memory region through EFI_PCI_IO_PROTOCOL

Adding Ray, and a comment at the bottom:

On 09/14/18 20:24, Kirkendall, Garrett wrote:
> Hopefully someone can help me understand.
> 
> I assume I should be able to access MMIO pointed to by a PCI devices Prefetchable Memory BAR.  I would think I would use EFI_PCI_IO_PROTOCOL.Mem.Read or Write.  This in turn will send the request up to EFI_PCI_ROOT_BRIDGE_IO_PROTOCOL.Mem.Read or Write.  
> 
> MdeModulePkg/Bus/Pci/PciHostBridgeDxe/PciRootBridgeIo.c: 
> RootBridgeIoMemRead (Write) calls RootBridgeIoCheckParameter() to 
> verify that the request falls within the confines of the PCI root 
> bridge.  The below code verifies the address against non-prefetchable 
> MMIO regions and skips the prefetchable memory regions.  Is this 
> correct, and if so what is the method to read/write prefetchable 
> memory regions?  If I force EFI_PCI_HOST_BRIDGE_COMBINE_MEM_PMEM, then 
> I can access prefetchable memory regions because they are forced 
> within Mem.* and MemAbove4G.*
> 
> 
> 
> EFI_STATUS
> RootBridgeIoCheckParameter (
>   IN EFI_PCI_ROOT_BRIDGE_IO_PROTOCOL        *This,
>   IN OPERATION_TYPE                         OperationType,
>   IN EFI_PCI_ROOT_BRIDGE_IO_PROTOCOL_WIDTH  Width,
>   IN UINT64                                 Address,
>   IN UINTN                                  Count,
>   IN VOID                                   *Buffer
>   )
> {
> ...
>   } else if (OperationType == MemOperation) {
>     //
>     // Allow Legacy MMIO access
>     //
>     if ((Address >= 0xA0000) && (Address + MultU64x32 (Count, Size)) <= 0xC0000) {
>       if ((RootBridge->Attributes & EFI_PCI_ATTRIBUTE_VGA_MEMORY) != 0) {
>         return EFI_SUCCESS;
>       }
>     }
>     //
>     // By comparing the Address against Limit we know which range to be used
>     // for checking
>     //
>     if (Address + MultU64x32 (Count, Size) <= RootBridge->Mem.Limit + 1) {
>       Base = RootBridge->Mem.Base;
>       Limit = RootBridge->Mem.Limit;
>     } else {
>       Base = RootBridge->MemAbove4G.Base;
>       Limit = RootBridge->MemAbove4G.Limit;
>     }
>   } else {
> ...
> 

I'm not a PCI expert, but I think you are right; the code should permit access to the PMem / PMemAbove4G apertures as well. According to the PCI spec (--> fair use citation below),

"A PCI Express Function requesting memory resources through a BAR must set the BAR's Prefetchable bit unless the range contains locations with read side effects or locations in which the Function does not tolerate write merging."

IOW "prefetchable" means "there are no side effects to reading, and the Function tolerates write merging".

I don't see why that's reason for RootBridgeIoCheckParameter() to reject the access.

Ray, what's your take?

Thanks
Laszlo