Re: Mapping of PrpList in NvmExpressDxe

[Andrew Fish <afish@apple.com>]

> On Feb 17, 2017, at 5:54 AM, Arka Sharma <arka.sw1988@gmail.com> wrote:
> 
> I am wondering what is the reason for mapping the PrpList buffer with
> EfiPciIoOperationBusMasterCommonBuffer, as host will fill the Prp
> entries and after updating the submission queue doorbell device will
> start processing the command and fetch the Prp entries. So I am
> thinking the Prplist buffer could have been mapped as
> EfiPciIoOperationBusMasterRead. Is there any reason for mapping it as
> CommonBuffer that I am not able to figure out ?
> 

Arka,

Good question. Historically there have been a lot of bugs in DMA code in EFI. The reason being if you don't follow the rules the code still works since DMA is so coherent in hardware on an x86 PC. This is not the case for a lot of ARM platforms. 

There is a good overview of UEFI DMA operations in the UEFI Spec PCI Bus Support Chapter. 

DMA Bus Master Read Operation
• Call Map() for EfiPciOperationBusMasterRead or EfiPciOperationBusMasterRead64.
• Program the DMA Bus Master with the DeviceAddress returned by Map().
• Start the DMA Bus Master.
• Wait for DMA Bus Master to complete the read operation.
• Call Unmap().

DMA Bus Master Write Operation
• Call Map() for EfiPciOperationBusMasterWrite or EfiPciOperationBusMasterRead64.
• Program the DMA Bus Master with the DeviceAddress returned by Map().
• Start the DMA Bus Master.
• Wait for DMA Bus Master to complete the write operation.
• Perform a PCI controller specific read transaction to flush all PCI write buffers (See PCI Specification Section 3.2.5.2) .
• Call Flush().
• Call Unmap().

DMA Bus Master Common Buffer Operation
• Call AllocateBuffer() to allocate a common buffer.
• Call Map() for EfiPciOperationBusMasterCommonBuffer or EfiPciOperationBusMasterCommonBuffer64.
• Program the DMA Bus Master with the DeviceAddress returned by Map().
• The common buffer can now be accessed equally by the processor and the DMA bus master.
• Call Unmap().
• Call FreeBuffer().

So to answer your question. The Read and Write operations are one shot on the buffer, while Common Buffer is a buffer that is DMA coherent and can be reused. Thats probably why Common Buffer is used by the NVMe driver. 

Basically the common usage for the Read and Write mappings are the caller passing a buffer (like a block on the disk). The Queues that run DMA are common buffer as the PCI hardware and the CPU both need to access them intermittently. To be clear for the read and write case the CPU only has a coherent view of the buffer after the Unmap() is called. 

I think the only way you can enforce most of the UEFI driver DMA rules on x86 is to turn on an IOMMU that would cause faults if you don't follow the rules. Basically you have the IOMMU fault on DMA transactions to a buffer that is not following the rules above. 

Thanks,

Andrew Fish

> Regards,
> Arka
> _______________________________________________
> edk2-devel mailing list
> edk2-devel@lists.01.org
> https://lists.01.org/mailman/listinfo/edk2-devel