Re: Mapping of PrpList in NvmExpressDxe

[Arka Sharma <arka.sw1988@gmail.com>]

Thanks Andrew for your response. Actually my question related to
EfiPciOperationBusMasterCommonBuffer was only in case of PrpList
buffer which is allocated and mapped in NvmeCreatePrpList function. In
case of queues the mapping with EfiPciOperationBusMasterCommonBuffer
is understood as in case of Completion queue the device will post the
response of a command and driver will also poll the phase tag to
detect a new completion entry so coherent buffer is needed. But in
case of PrpList buffer such simultaneous access is not done. The host
will update the prp entries and device will pick them up only after
host update submission queue doorbell. So my question is in case of
particularly PrpList buffer why it is mapped as
EfiPciOperationBusMasterCommonBuffer, where as per my understanding
EfiPciOperationBusMasterRead would have sufficed.

Regards,
Arka

On Fri, Feb 17, 2017 at 10:13 PM, Andrew Fish <afish@apple.com> wrote:
>
>> On Feb 17, 2017, at 5:54 AM, Arka Sharma <arka.sw1988@gmail.com> wrote:
>>
>> I am wondering what is the reason for mapping the PrpList buffer with
>> EfiPciIoOperationBusMasterCommonBuffer, as host will fill the Prp
>> entries and after updating the submission queue doorbell device will
>> start processing the command and fetch the Prp entries. So I am
>> thinking the Prplist buffer could have been mapped as
>> EfiPciIoOperationBusMasterRead. Is there any reason for mapping it as
>> CommonBuffer that I am not able to figure out ?
>>
>
> Arka,
>
> Good question. Historically there have been a lot of bugs in DMA code in EFI. The reason being if you don't follow the rules the code still works since DMA is so coherent in hardware on an x86 PC. This is not the case for a lot of ARM platforms.
>
> There is a good overview of UEFI DMA operations in the UEFI Spec PCI Bus Support Chapter.
>
> DMA Bus Master Read Operation
> • Call Map() for EfiPciOperationBusMasterRead or EfiPciOperationBusMasterRead64.
> • Program the DMA Bus Master with the DeviceAddress returned by Map().
> • Start the DMA Bus Master.
> • Wait for DMA Bus Master to complete the read operation.
> • Call Unmap().
>
> DMA Bus Master Write Operation
> • Call Map() for EfiPciOperationBusMasterWrite or EfiPciOperationBusMasterRead64.
> • Program the DMA Bus Master with the DeviceAddress returned by Map().
> • Start the DMA Bus Master.
> • Wait for DMA Bus Master to complete the write operation.
> • Perform a PCI controller specific read transaction to flush all PCI write buffers (See PCI Specification Section 3.2.5.2) .
> • Call Flush().
> • Call Unmap().
>
> DMA Bus Master Common Buffer Operation
> • Call AllocateBuffer() to allocate a common buffer.
> • Call Map() for EfiPciOperationBusMasterCommonBuffer or EfiPciOperationBusMasterCommonBuffer64.
> • Program the DMA Bus Master with the DeviceAddress returned by Map().
> • The common buffer can now be accessed equally by the processor and the DMA bus master.
> • Call Unmap().
> • Call FreeBuffer().
>
> So to answer your question. The Read and Write operations are one shot on the buffer, while Common Buffer is a buffer that is DMA coherent and can be reused. Thats probably why Common Buffer is used by the NVMe driver.
>
> Basically the common usage for the Read and Write mappings are the caller passing a buffer (like a block on the disk). The Queues that run DMA are common buffer as the PCI hardware and the CPU both need to access them intermittently. To be clear for the read and write case the CPU only has a coherent view of the buffer after the Unmap() is called.
>
> I think the only way you can enforce most of the UEFI driver DMA rules on x86 is to turn on an IOMMU that would cause faults if you don't follow the rules. Basically you have the IOMMU fault on DMA transactions to a buffer that is not following the rules above.
>
> Thanks,
>
> Andrew Fish
>
>> Regards,
>> Arka
>> _______________________________________________
>> edk2-devel mailing list
>> edk2-devel@lists.01.org
>> https://lists.01.org/mailman/listinfo/edk2-devel
>