From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-in21.apple.com (mail-out21.apple.com [17.171.2.31]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ml01.01.org (Postfix) with ESMTPS id 3E9A48211E for ; Fri, 17 Feb 2017 09:45:44 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; d=apple.com; s=mailout2048s; c=relaxed/simple; q=dns/txt; i=@apple.com; t=1487353543; h=From:Sender:Reply-To:Subject:Date:Message-id:To:Cc:MIME-version:Content-type: Content-transfer-encoding:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:In-reply-to:References:List-Id: List-Help:List-Unsubscribe:List-Subscribe:List-Post:List-Owner:List-Archive; bh=dR7afu0Qz0LYoqI7Do00uwqEAb8aDSTKygbeOYH0HSc=; b=lPrXUo18rCDXUKPwvHBVimt5JDyffPagUxClzvMwSQdXkn6oqaSkyyruRudWLtiX YSFQ5PKPvYwYfkzGKuOEsUS9KjxswEe7CUely4typD5N8hOSHAb+GP8hiWQo4CWh 2c1OtgSmY75N4DyNVGp06j+bpbejh1o85yI/SaJ7WUT18oesK8jUdSZkSPNO9Ur4 L8EyP0aklvJSq1hN2//HYzJl/5MzuCqphfmdk6Zfbdd0XEj0AZ7v9EPfVfpBjkpT 1n0tzzZjwlKnX2rWa6QHrlSAzSAXyyj99IChyA4xYPQFjX5h0dN4wMD3gO+K6qTh ikzTc76jp51h6EV49o04KQ==; Received: from relay5.apple.com (relay5.apple.com [17.128.113.88]) by mail-in21.apple.com (Apple Secure Mail Relay) with SMTP id 4B.7A.09934.6C637A85; Fri, 17 Feb 2017 09:45:43 -0800 (PST) X-AuditID: 11ab0215-3c5ff700000026ce-e3-58a736c63578 Received: from nwk-mmpp-sz09.apple.com (nwk-mmpp-sz09.apple.com [17.128.115.80]) by relay5.apple.com (Apple SCV relay) with SMTP id C9.87.05881.6C637A85; Fri, 17 Feb 2017 09:45:42 -0800 (PST) MIME-version: 1.0 Received: from [17.153.70.48] (unknown [17.153.70.48]) by nwk-mmpp-sz09.apple.com (Oracle Communications Messaging Server 8.0.1.2.20170210 64bit (built Feb 10 2017)) with ESMTPSA id <0OLJ00BM45C4ID00@nwk-mmpp-sz09.apple.com>; Fri, 17 Feb 2017 09:45:41 -0800 (PST) Sender: afish@apple.com From: Andrew Fish In-reply-to: Date: Fri, 17 Feb 2017 09:45:40 -0800 Cc: edk2-devel@lists.01.org Message-id: References: To: Arka Sharma X-Mailer: Apple Mail (2.3226) X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFtrNLMWRmVeSWpSXmKPExsUi2FAYoXvcbHmEwbPzMhabe3rYLfYcOsrs wOSxc9Zddo/u2f9YApiiuGxSUnMyy1KL9O0SuDLuX9vNUrBbs2LT04PMDYwXFbsYOTkkBEwk JvS/YO9i5OIQEtjHKLF4+zbWLkYOsMTvh0EQ8YOMEjvO3GEBaeAVEJT4MfkeC0gNs4C6xJQp uRA1/UwSSw40M4LUCAuIS7w7s4kZwjaXOPToLJjNJqAssWL+B3YQm1MgWOJjxxGwmSwCqhKX vuxgBbGZBaQlHk48DmVrSzx5d4EVYq+NxM5DKxkhlp1hlDg87SHYIBGgI/rvzmKEOFpWYvYv L5AaCYENbBINOzuYJjAKz0Jy9yyEu2chWbGAkXkVo3BuYmaObmaekaFeYkFBTqpecn7uJkZQ aK9mEt3BOP+V4SFGAQ5GJR7eDL3lEUKsiWXFlbmHGKU5WJTEee3tlkUICaQnlqRmp6YWpBbF F5XmpBYfYmTi4JRqYEwpeDuX49NVW6eEA+vO/9/78N3s+/lTZ3bfXX89I6fd2b9Kq+Nthz4L W5PA58NSr0Tyc6/q6y+8I31Q7lJfWfcl7vjJ0f89Xpd8667d23h8ZrL9u9hXm0x+id/4zJkz L8XqD4va3ekaTL8/rNqW1bUw5/mTvMnz6/eof2Xq4DudcHbN3EfMK74qsRRnJBpqMRcVJwIA jDEgPE4CAAA= X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFmpkkeLIzCtJLcpLzFFi42IRbCgO0D1mtjzCoGsqr8Xmnh52iz2HjjI7 MHnsnHWX3aN79j+WAKYoLpuU1JzMstQifbsEroz713azFOzWrNj09CBzA+NFxS5GDg4JAROJ 3w+Duhg5gUwxiQv31rN1MXJxCAkcZJTYceYOC0iCV0BQ4sfkeywg9cwC6hJTpuRC1PQzSSw5 0MwIUiMsIC7x7swmZgjbXOLQo7NgNpuAssSK+R/YQWxOgWCJjx1HwGayCKhKXPqygxXEZhaQ lng48TiUrS3x5N0FVoi9NhI7D61khFh2hlHi8LSHYINEgI7ovzuLEeIBWYnZv7wmMArOQnLq LIRTZyGZuoCReRWjQFFqTmKlqV5iQUFOql5yfu4mRnCAFkbsYPy/zOoQowAHoxIPrwP38ggh 1sSy4srcQ4wSHMxKIrwr9YBCvCmJlVWpRfnxRaU5qcWHGJOBfpnILCWanA+MnrySeEMTEwMT Y2MzY2NzE3PShJXEeROklkUICaQnlqRmp6YWpBbBbGHi4JRqYJyyQebf3WUvfr65M+WE4IO9 +87FmDecfXs8yCZgarrRjunae98I6turpO2bfcem8fWxy+eLM/wXfSqsi/rw5yfLJ4nA6ZcU I5r64u7ebbhxaJFokH/r0p2GOkztzku/3ipl/NCky6a9dtmuyW4rpGfW87rwf1jNwce+r2TH pcXZuz9x+m1TXOmnxFKckWioxVxUnAgAc74GzZQCAAA= Subject: Re: Mapping of PrpList in NvmExpressDxe X-BeenThere: edk2-devel@lists.01.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: EDK II Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 17 Feb 2017 17:45:44 -0000 Content-type: text/plain; charset=utf-8 Content-transfer-encoding: quoted-printable > On Feb 17, 2017, at 9:30 AM, Arka Sharma = wrote: >=20 > Thanks Andrew for your response. Actually my question related to > EfiPciOperationBusMasterCommonBuffer was only in case of PrpList > buffer which is allocated and mapped in NvmeCreatePrpList function. In > case of queues the mapping with EfiPciOperationBusMasterCommonBuffer > is understood as in case of Completion queue the device will post the > response of a command and driver will also poll the phase tag to > detect a new completion entry so coherent buffer is needed. But in > case of PrpList buffer such simultaneous access is not done. The host > will update the prp entries and device will pick them up only after > host update submission queue doorbell. So my question is in case of > particularly PrpList buffer why it is mapped as > EfiPciOperationBusMasterCommonBuffer, where as per my understanding > EfiPciOperationBusMasterRead would have sufficed. >=20 Arka, The EfiPciOperationBusMasterRead implies the driver would have to call = Map() prior to every DMA, and Unmap() after every DMA before the CPU = looks at the data. For EfiPciOperationBusMasterCommonBuffer the Map() is = part of the malloc, and the unmap is part of the free of the buffer. So = in general EfiPciOperationBusMasterCommonBuffer is more efficient and = there are less coherency race conditions, so it is easier to write the = code.=20 On systems where DMA is not coherent a Map()/Unmap() call may have to do = caches flushes and other operations on every call (if you miss a call = you can get data corruption in your buffer), while a Common Buffer might = just be uncached memory.=20 Thanks, Andrew Fish > Regards, > Arka >=20 > On Fri, Feb 17, 2017 at 10:13 PM, Andrew Fish wrote: >>=20 >>> On Feb 17, 2017, at 5:54 AM, Arka Sharma = wrote: >>>=20 >>> I am wondering what is the reason for mapping the PrpList buffer = with >>> EfiPciIoOperationBusMasterCommonBuffer, as host will fill the Prp >>> entries and after updating the submission queue doorbell device will >>> start processing the command and fetch the Prp entries. So I am >>> thinking the Prplist buffer could have been mapped as >>> EfiPciIoOperationBusMasterRead. Is there any reason for mapping it = as >>> CommonBuffer that I am not able to figure out ? >>>=20 >>=20 >> Arka, >>=20 >> Good question. Historically there have been a lot of bugs in DMA code = in EFI. The reason being if you don't follow the rules the code still = works since DMA is so coherent in hardware on an x86 PC. This is not the = case for a lot of ARM platforms. >>=20 >> There is a good overview of UEFI DMA operations in the UEFI Spec PCI = Bus Support Chapter. >>=20 >> DMA Bus Master Read Operation >> =E2=80=A2 Call Map() for EfiPciOperationBusMasterRead or = EfiPciOperationBusMasterRead64. >> =E2=80=A2 Program the DMA Bus Master with the DeviceAddress returned = by Map(). >> =E2=80=A2 Start the DMA Bus Master. >> =E2=80=A2 Wait for DMA Bus Master to complete the read operation. >> =E2=80=A2 Call Unmap(). >>=20 >> DMA Bus Master Write Operation >> =E2=80=A2 Call Map() for EfiPciOperationBusMasterWrite or = EfiPciOperationBusMasterRead64. >> =E2=80=A2 Program the DMA Bus Master with the DeviceAddress returned = by Map(). >> =E2=80=A2 Start the DMA Bus Master. >> =E2=80=A2 Wait for DMA Bus Master to complete the write operation. >> =E2=80=A2 Perform a PCI controller specific read transaction to flush = all PCI write buffers (See PCI Specification Section 3.2.5.2) . >> =E2=80=A2 Call Flush(). >> =E2=80=A2 Call Unmap(). >>=20 >> DMA Bus Master Common Buffer Operation >> =E2=80=A2 Call AllocateBuffer() to allocate a common buffer. >> =E2=80=A2 Call Map() for EfiPciOperationBusMasterCommonBuffer or = EfiPciOperationBusMasterCommonBuffer64. >> =E2=80=A2 Program the DMA Bus Master with the DeviceAddress returned = by Map(). >> =E2=80=A2 The common buffer can now be accessed equally by the = processor and the DMA bus master. >> =E2=80=A2 Call Unmap(). >> =E2=80=A2 Call FreeBuffer(). >>=20 >> So to answer your question. The Read and Write operations are one = shot on the buffer, while Common Buffer is a buffer that is DMA coherent = and can be reused. Thats probably why Common Buffer is used by the NVMe = driver. >>=20 >> Basically the common usage for the Read and Write mappings are the = caller passing a buffer (like a block on the disk). The Queues that run = DMA are common buffer as the PCI hardware and the CPU both need to = access them intermittently. To be clear for the read and write case the = CPU only has a coherent view of the buffer after the Unmap() is called. >>=20 >> I think the only way you can enforce most of the UEFI driver DMA = rules on x86 is to turn on an IOMMU that would cause faults if you don't = follow the rules. Basically you have the IOMMU fault on DMA transactions = to a buffer that is not following the rules above. >>=20 >> Thanks, >>=20 >> Andrew Fish >>=20 >>> Regards, >>> Arka >>> _______________________________________________ >>> edk2-devel mailing list >>> edk2-devel@lists.01.org >>> https://lists.01.org/mailman/listinfo/edk2-devel >>=20