From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-in25.apple.com (mail-out25.apple.com [17.171.2.35]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ml01.01.org (Postfix) with ESMTPS id 915F021D491B7 for ; Thu, 27 Jul 2017 14:36:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; d=apple.com; s=mailout2048s; c=relaxed/simple; q=dns/txt; i=@apple.com; t=1501191503; h=From:Sender:Reply-To:Subject:Date:Message-id:To:Cc:MIME-version:Content-type: Content-Transfer-Encoding:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:In-reply-to:References:List-Id: List-Help:List-Unsubscribe:List-Subscribe:List-Post:List-Owner:List-Archive; bh=5plB/xYV7umBu0QUdg2HHd8EfIvWAe3aZge+/SWWER0=; b=H8YrGGukMAGyUV6hKYd/gVq+ZCS+uZOfjj6ddTz22bgxU8vF/vV8XdH8OCcrrHJO BbxdMapaa8WNFHykm0IT6CsMeyya+CqqwRLTp3pQ6wk1hHDLNr09g6L7yPtN6KLE FiIaFomr7qAZN0bdFo4NHgB5GaogSbk0SafJujkbmq7HftkcisjC5LjhQd+YAWD9 XS3OEI0/BcjH8G67a3Qg/B2VGGgpvozdYXFcmsAV3lx6rQcfEyMnXoEpxe9sJGAG MXGSd0wyU7FP36d8H5l9YZe8rDNlsSNjHeQprVGTdiOisw7dqw0AWQPsgAJxiP1/ rMsEanabpteUsll6Y6Qv+g==; Received: from relay7.apple.com (relay7.apple.com [17.128.113.101]) (using TLS with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mail-in25.apple.com (Apple Secure Mail Relay) with SMTP id 4B.9C.05744.F4D5A795; Thu, 27 Jul 2017 14:38:23 -0700 (PDT) X-AuditID: 11ab0219-dacb19c000001670-af-597a5d4f64cf Received: from nwk-mmpp-sz12.apple.com (nwk-mmpp-sz12.apple.com [17.128.115.204]) by relay7.apple.com (Apple SCV relay) with SMTP id 06.86.07283.E4D5A795; Thu, 27 Jul 2017 14:38:23 -0700 (PDT) MIME-version: 1.0 Received: from [17.235.18.114] by nwk-mmpp-sz12.apple.com (Oracle Communications Messaging Server 8.0.1.2.20170621 64bit (built Jun 21 2017)) with ESMTPSA id <0OTR00E1KQRXVO80@nwk-mmpp-sz12.apple.com>; Thu, 27 Jul 2017 14:38:22 -0700 (PDT) Sender: afish@apple.com From: Andrew Fish Message-id: Date: Thu, 27 Jul 2017 14:38:20 -0700 In-reply-to: <9C4ABF62-4018-4014-A3C4-0A8B3B3CE1C2@linaro.org> Cc: Tom Lendacky , "Michael S . Tsirkin" , Jordan Justen , "edk2-devel@lists.01.org" , Laszlo Ersek , Jason Wang , Ard Biesheuvel To: Brijesh Singh References: <1500502151-13508-1-git-send-email-brijesh.singh@amd.com> <841bec5f-6f6e-8b1f-25ba-0fd37a915b72@redhat.com> <4e2fc623-3656-eea7-09a8-b5c6d2f694e1@amd.com> <4071596d-32c9-e6d9-8c93-0d43d28e9b5a@redhat.com> <6517a7f8-5564-35e1-dc27-1b85a23c815e@amd.com> <9C4ABF62-4018-4014-A3C4-0A8B3B3CE1C2@linaro.org> X-Mailer: Apple Mail (2.3273) X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFmphkeLIzCtJLcpLzFFi42IRbChM1fWPrYo0mHTMwuL/h92MFjM39TFa 7Dl0lNli2aXPTBY7rvWzWCw7toPF4v+vV6wWR6bsY3Xg8Gi99JfNY/Gel0wed67tYfPonv2P xeP9vqtsAaxRXDYpqTmZZalF+nYJXBm39rYwFfyazVjRfeMZUwPjrnbGLkZODgkBE4mFc98w dTFycQgJrGWSaL/6mQ0msefjMUaIxCFGia1vutlBErwCghI/Jt9jAbGZBcIkLlzuZ4Yo+soo sfTaO7CxwgLiEu/ObGIGsdkElCVWzP8A1WwjMWd5G9A6DqAaF4mdzfEgYRYBVYmfE/uZQGxO ATuJU4cfsIPMZBZYwyRxtfMWWK+IgKbEw3M/WEFsIYHdzBI9dzwhLpWVuDX7EtgREgLd7BL7 p95lmsAoNAvJsbOQHAtha0l8f9QKFOcAsuUlDp6XhQhrSjy794kdwtaWePLuAusCRrZVjMK5 iZk5upl5RqZ6iQUFOal6yfm5mxhB0baaSXIH49fXhocYBTgYlXh4HdyqIoVYE8uKK3MPMUpz sCiJ864xAgoJpCeWpGanphakFsUXleakFh9iZOLglGpgTKjZuWXX4oVT5pxKVFNg4/FUUPgx R14upzzg18qnbpb+KkU/vecoVxyK9OTXeC+ScnZjxZ5pbgFF8Vv2vb2y6I3UpkkfHa41dd+2 nprYP1eh8P4dq6Pb1/t57eRi/Xp68vv7TVZ7o25Ib5tXLL/8Vxmn9/pQzpi0y/8iss62vZ72 7+qVroL7yUosxRmJhlrMRcWJAMg2xhaXAgAA X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFnrLIsWRmVeSWpSXmKPExsUi2FB8Rtc/tirSoKOJ0eL/h92MFjM39TFa 7Dl0lNli2aXPTBY7rvWzWCw7toPF4v+vV6wWR6bsY3Xg8Gi99JfNY/Gel0wed67tYfPonv2P xeP9vqtsAaxRXDYpqTmZZalF+nYJXBm39rYwFfyazVjRfeMZUwPjrnbGLkZODgkBE4k9H48B 2VwcQgKHGCW2vulmB0nwCghK/Jh8jwXEZhYIk7hwuZ8Zougro8TSa+/AuoUFxCXendnEDGKz CShLrJj/AarZRmLO8jamLkYOoBoXiZ3N8SBhFgFViZ8T+5lAbE4BO4lThx+wg8xkFljDJHG1 8xZYr4iApsTDcz9YQWwhgd3MEj13PCEulZW4NfsS8wRG/llI7puF5D4IW0vi+6NWoDgHkC0v cfC8LERYU+LZvU/sELa2xJN3F1gXMLKtYhQoSs1JrDTXSywoyEnVS87P3cQIjo3C1B2Mjcut DjEKcDAq8fA++FgZKcSaWFZcmQsMJA5mJRHexuiqSCHelMTKqtSi/Pii0pzU4kOMExmBvpzI LCWanA+M3LySeEMTEwMTY2MzY2NzE3NaCiuJ887XL4wUEkhPLEnNTk0tSC2COYqJg1OqgbGt sPF71rx9G+5FP9o3wblVYxPDy3cL9r77tjHjfeFe0W3lXBv2m3nOi2ZPEtw2L0zrgtCjSMb3 38XP9foI7X5l/K+9t5/hnWmBk1tim3ay0QKO4m653DQbnl2BdWm8VUWPn12PupS5vmX2mpCi GMPXEe8X5Bd/v/lYh2H1yb2G0pz6sUIbQpVYijMSDbWYi4oTAdsqF2kAAwAA X-Content-Filtered-By: Mailman/MimeDel 2.1.22 Subject: Re: [RFC v1 0/3] Add VIRTIO_F_IOMMU_PLATFORM support X-BeenThere: edk2-devel@lists.01.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: EDK II Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 27 Jul 2017 21:36:20 -0000 Content-Type: text/plain; CHARSET=US-ASCII Content-Transfer-Encoding: 7BIT > On Jul 27, 2017, at 2:31 PM, Ard Biesheuvel wrote: > >> >> On 27 Jul 2017, at 21:55, Brijesh Singh wrote: >> >> >> >> On 07/27/2017 02:00 PM, Brijesh Singh wrote: >> >>>>> This distribution of operations seems wrong. The key point is that >>>>> AllocateBuffer() *need not* result in a buffer that is immediately >>>>> usable, and that client code is required to call Map() >>>>> *unconditionally*, even if BusMasterCommonBuffer is the desired >>>>> operation. Therefore, the right distribution of operations is: >>>>> >>>>> - IoMmuAllocateBuffer() allocates pages and does not touch the >>>>> encryption mask.. >>>>> >>>>> - IoMmuFreeBuffer() deallocates pages and does not touch the encryption >>>>> mask. >>>>> >>> Actually one of main reason why we cleared and restored the memory encryption mask >>> during allocate/free is because we also consume the IOMMU protocol in QemuFwCfgLib >>> as a method to allocate and free a DMA buffer. I am certainly open to suggestions. >>> [1] https://github.com/tianocore/edk2/blob/master/OvmfPkg/Library/QemuFwCfgLib/QemuFwCfgDxe.c#L159 >>> [2] https://github.com/tianocore/edk2/blob/master/OvmfPkg/Library/QemuFwCfgLib/QemuFwCfgDxe.c#L197 >>>>> - IoMmuMap() does not allocate pages when BusMasterCommonBuffer is >>>>> requested, and it allocates pages (bounce buffer) otherwise. >>>>> >>> I am trying to wrap my head around how we can support BusMasterCommonBuffer >>> when buffer was not allocated by us. Changing the memory encryption mask in >>> a page table will not update the contents. Also since the memory encryption >>> mask works on PAGE_SIZE hence changing the encryption mask on not our allocated >>> buffer could mess things up (e.g if NumberOfBytes is not PAGE_SIZE aligned). >> >> I may be missing something in my understanding. Here is a flow I have in my >> mind, please correct me. >> >> OvmfPkg/VirtIoBlk.c: >> >> VirtioBlkInit() >> .... >> .... >> VirtioRingInit >> Virtio->AllocateSharedPages(RingSize, &Ring->Base) >> PciIo->AllocatePages(RingSize, &RingAddress) >> Virtio->MapSharedPages(...,BusMasterCommonBuffer, Ring->Base, RingSize, &RingDeviceAddress) >> ..... >> ..... >> >> This case is straight forward and we can easily maps. No need for bounce buffering. >> >> VirtioBlkReadBlocks(..., BufferSize, Buffer,) >> ...... >> ...... >> SynchronousRequest(..., BufferSize, Buffer) >> .... >> Virtio->MapSharedPages(..., BusMasterCommonBuffer, Buffer, BufferSize, &DeviceAddress) >> VirtioAppendDesc(DeviceAddress, BufferSize, ...) >> VirtioFlush (...) >> >> In the above case, "Buffer" was not allocated by us hence we will not able to change the >> memory encryption attributes. Am I missing something in the flow ? >> > > > Common buffer mappings may only be created from buffers that were allocated by AllocateBuffer(). In fact, that is its main purpose Brijesh, If you look in the UEFI Spec 13.4 EFI PCI I/O Protocol there is a good write on DMA. DMA Bus Master Read Operation ========================= Call Map() for EfiPciIoOperationBusMasterRead. Program the DMA Bus Master with the DeviceAddress returned by Map(). Start the DMA Bus Master. Wait for DMA Bus Master to complete the read operation. Call Unmap(). DMA Bus Master Write Operation ========================== Call Map() for EfiPciOperationBusMasterWrite. Program the DMA Bus Master with the DeviceAddress returned by Map(). Start the DMA Bus Master. Wait for DMA Bus Master to complete the write operation. Perform a PCI controller specific read transaction to flush all PCI write buffers (See PCI Specification Section 3.2.5.2) . Call Flush(). Call Unmap(). DMA Bus Master Common Buffer Operation ================================== Call AllocateBuffer() to allocate a common buffer. Call Map() for EfiPciIoOperationBusMasterCommonBuffer. Program the DMA Bus Master with the DeviceAddress returned by Map(). The common buffer can now be accessed equally by the processor and the DMA bus master. Call Unmap(). Call FreeBuffer(). Thanks, Andrew Fish >> >>>>> *Regardless* of BusMaster operation, the following actions are carried >>>>> out unconditionally: >>>>> >>>>> . the memory encryption mask is cleared in this function (and in this >>>>> function only), >>>>> >>>>> . An attempt is made to grab a MAP_INFO structure from an internal >>>>> free list (to be introduced!). The head of the list is a new static >>>>> variable. If the free list is empty, then a MAP_INFO structure is >>>>> allocated with AllocatePool(). The NO_MAPPING macro becomes unused >>>>> and can be deleted from the source code. >>>>> >>>>> - IoMmuUnmap() clears the encryption mask unconditionally. (For this, it >>>>> has to consult the MAP_INFO structure that is being passed in from the >>>>> caller.) In addition: >>>>> >>>>> . If MapInfo->Operation is BusMasterCommonBuffer, then we know the >>>>> allocation was done separately in AllocateBuffer, so we do not >>>>> release the pages. Otherwise, we do release the pages. >>>>> >>>>> . MapInfo is linked back on the internal free list (see above). It is >>>>> *never* released with FreePool(). >>>>> >>>>> This approach guarantees that IoMmuUnmap() can de-program the IOMMU (= >>>>> re-set the memory encryption mask) without changing the UEFI memory >>>>> map. (I trust that MemEncryptSevSetPageEncMask() will not split page >>>>> tables internally when it *re*sets the encryption mask -- is that >>>>> correct?) >> >> >> > _______________________________________________ > edk2-devel mailing list > edk2-devel@lists.01.org > https://lists.01.org/mailman/listinfo/edk2-devel