From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-it0-x229.google.com (mail-it0-x229.google.com [IPv6:2607:f8b0:4001:c0b::229]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ml01.01.org (Postfix) with ESMTPS id 20B1421D19954 for ; Mon, 31 Jul 2017 12:46:59 -0700 (PDT) Received: by mail-it0-x229.google.com with SMTP id h199so239434ith.0 for ; Mon, 31 Jul 2017 12:49:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=g889g7s3r6eU81yeNqvrr2o1w3PD2HSbtp3oRrB3uRU=; b=B7Oaf7IdTKWzLRb+pT3eMa9iVLXfZ5G0IJ9V9eZHn6s/SL8ZVldLgFLbYeHjN2egfG eaGSCRTihuYVQc4nqeLRH6Y5p95MciqzxVvUCy7HpQpJxp/SPAdD/Yks05kUae+vpcIo ykj+Tw3M6LpUAjx+xVsfPMqUydsXrfixlM0Xc= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=g889g7s3r6eU81yeNqvrr2o1w3PD2HSbtp3oRrB3uRU=; b=Jg5BizXRvLqCPl+EB3RkM2bZ2xCdFAel6YH13JGwQWWjkVDpQo1FI6lXGFZdbnff7o dxo9heDyu7I/oQHOa6RFRVwc4Bg2Z/Wb+OKbLQuPAufaRMy38j00hqES3tfipeS+MN97 74FlElW3YIKmDVGHfBuxJRLl5okmOWaB8mQOKp/ILCg6GZ4OEzgHaSicIhRryNXPNR/J 5I8aTxUJ+dqvaTGK/SfHi9I5uI7GBoRLqmFbCq+2/O30xCiKVSkfZK2DhNu8asbIkTRu YBXl7oC5/m+++Tqb/hxhtri/osQuj+0l5F8UbhDNCHTyKdxZ2h7JI9731JVXIwpGldYL gEWQ== X-Gm-Message-State: AIVw112UCnLwmMls3TEgG6/6iPiSd9O0lEexByAlYKAg2KUhNVT0098O AbZNfHATFectD3Rxs/5sLJnkY0c7apaOhpg= X-Received: by 10.36.170.7 with SMTP id b7mr17503383itf.10.1501530546929; Mon, 31 Jul 2017 12:49:06 -0700 (PDT) MIME-Version: 1.0 Received: by 10.107.162.1 with HTTP; Mon, 31 Jul 2017 12:49:06 -0700 (PDT) In-Reply-To: <1501529474-20550-3-git-send-email-brijesh.singh@amd.com> References: <1501529474-20550-1-git-send-email-brijesh.singh@amd.com> <1501529474-20550-3-git-send-email-brijesh.singh@amd.com> From: Ard Biesheuvel Date: Mon, 31 Jul 2017 20:49:06 +0100 Message-ID: To: Brijesh Singh Cc: "edk2-devel@lists.01.org" , Tom Lendacky , Laszlo Ersek , Jordan Justen Subject: Re: [PATCH v1 2/4] OvmfPkg: IommuDxe: Provide support for mapping BusMasterCommonBuffer operation X-BeenThere: edk2-devel@lists.01.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: EDK II Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 31 Jul 2017 19:46:59 -0000 Content-Type: text/plain; charset="UTF-8" On 31 July 2017 at 20:31, Brijesh Singh wrote: > The current implementation was making assumption that AllocateBuffer() > returns a buffer with C-bit cleared. Hence when we were asked to > Map() with BusMasterCommonBuffer, we do not change the C-bit on > host buffer. > > In previous patch, we changed the AllocateBuffer() to not clear > C-bit during allocation. The patch adds support for handling the > BusMasterCommonBuffer operations when SEV is active. > > A typical DMA Bus master Common Operation follows the below step: > > 1. Client calls AllocateBuffer() to allocate a common buffer > 2. Client fill some data in common buffer (optional) > 3. Client calls Map() with BusMasterCommonBuffer > 4. Programs the DMA bus master with the device address returned by Map() > 5. The common buffer can now be accessed equally by the processor and > the DMA bus master. > 6. Client calls Unmap() > 7. Client calls FreeBuffer() > > In order to handle steps #2 (in which common buffer may contain > data), we perform in-place encryption to ensure that device > address returned by the Map() contains the correct data after > we clear the C-bit during Map(). > > In my measurement I do not see any noticable perform degradation when > performing in-place encryption/decryption on common buffer. > > Suggested-by: Laszlo Ersek > Contributed-under: TianoCore Contribution Agreement 1.0 > Cc: Laszlo Ersek > Cc: Jordan Justen > Signed-off-by: Brijesh Singh > --- > OvmfPkg/IoMmuDxe/AmdSevIoMmu.c | 190 +++++++++++++++++--- > 1 file changed, 164 insertions(+), 26 deletions(-) > Hello Brijesh, I haven't looked in detail at the existing code, but please don't conflate the device address with the address of a bounce buffer. These are very different things, although the confusion is understandable (and precedented) when not used to dealing with non-1:1 DMA. The device address is what gets programmed into the device's DMA registers. If there is a fixed [non-zero] offset between the device's view of memory and the host's (as may be the case with PCI, or generally when using an IOMMU), then the device is the only one who should attempt to perform memory accesses using this address. So please void SetMem() or other CPU dereferences involving the device address, and treat it as an opaque handle instead. In your case, you are dealing with a bounce buffer. So call it bounce buffer in the MapInfo struct. Imagine when dealing with a non-linear host to PCI mapping, you will still need to perform an additional translation to derive the device address from the bounce buffer address. > diff --git a/OvmfPkg/IoMmuDxe/AmdSevIoMmu.c b/OvmfPkg/IoMmuDxe/AmdSevIoMmu.c > index cc3c979d4484..5ae54482fffe 100644 > --- a/OvmfPkg/IoMmuDxe/AmdSevIoMmu.c > +++ b/OvmfPkg/IoMmuDxe/AmdSevIoMmu.c > @@ -28,7 +28,127 @@ typedef struct { > EFI_PHYSICAL_ADDRESS DeviceAddress; > } MAP_INFO; > > -#define NO_MAPPING (VOID *) (UINTN) -1 > +/** > + > + The function is used for mapping and unmapping the Host buffer with > + BusMasterCommonBuffer. Since the buffer can be accessed equally by the > + processor and the DMA bus master hence we can not use the bounce buffer. > + > + The function changes the underlying encryption mask of the pages that maps the > + host buffer. It also ensures that buffer contents are updated with the desired > + state. > + > +**/ > +STATIC > +EFI_STATUS > +SetBufferAsEncDec ( > + IN MAP_INFO *MapInfo, > + IN BOOLEAN Enc > + ) > +{ > + EFI_STATUS Status; > + EFI_PHYSICAL_ADDRESS TempBuffer; > + > + // > + // Allocate an intermediate buffer to hold the host buffer contents > + // > + Status = gBS->AllocatePages ( > + AllocateAnyPages, > + EfiBootServicesData, > + MapInfo->NumberOfPages, > + &TempBuffer > + ); > + if (EFI_ERROR (Status)) { > + return Status; > + } > + > + // > + // If the host buffer has C-bit cleared, then make sure the intermediate > + // buffer matches with same encryption mask. > + // > + if (!Enc) { > + Status = MemEncryptSevClearPageEncMask (0, MapInfo->DeviceAddress, > + MapInfo->NumberOfPages, TRUE); > + ASSERT_EFI_ERROR (Status); > + } > + > + // > + // Copy the data from host buffer into a temporary buffer. At this > + // time both host and intermediate buffer will have same encryption > + // mask. > + // > + CopyMem ( > + (VOID *) (UINTN) TempBuffer, > + (VOID *) (UINTN)MapInfo->HostAddress, > + MapInfo->NumberOfBytes); > + > + // > + // Now change the encryption mask of the host buffer > + // > + if (Enc) { > + Status = MemEncryptSevSetPageEncMask (0, MapInfo->HostAddress, > + MapInfo->NumberOfPages, TRUE); > + ASSERT_EFI_ERROR (Status); > + } else { > + Status = MemEncryptSevClearPageEncMask (0, MapInfo->HostAddress, > + MapInfo->NumberOfPages, TRUE); > + ASSERT_EFI_ERROR (Status); > + } > + > + // > + // Copy the data from intermediate buffer into host buffer. At this > + // time encryption masks will be different on host and intermediate > + // buffer and the hardware will perform encryption/decryption on > + // accesses. > + // > + CopyMem ( > + (VOID *) (UINTN)MapInfo->HostAddress, > + (VOID *) (UINTN)TempBuffer, > + MapInfo->NumberOfBytes); > + > + // > + // Restore the encryption mask of the intermediate buffer > + // > + if (!Enc) { > + Status = MemEncryptSevSetPageEncMask (0, MapInfo->DeviceAddress, > + MapInfo->NumberOfPages, TRUE); > + ASSERT_EFI_ERROR (Status); > + } > + > + // > + // Free the intermediate buffer > + // > + gBS->FreePages (TempBuffer, MapInfo->NumberOfPages); > + return EFI_SUCCESS; > +} > + > +/** > + This function will be called by Map() when mapping the buffer buffer to > + BusMasterCommonBuffer type. > + > +**/ > +STATIC > +EFI_STATUS > +SetHostBufferAsEncrypted ( > + IN MAP_INFO *MapInfo > + ) > +{ > + return SetBufferAsEncDec (MapInfo, TRUE); > +} > + > +/** > + This function will be called by Unmap() when unmapping host buffer > + from the BusMasterCommonBuffer type. > + > +**/ > +STATIC > +EFI_STATUS > +SetHostBufferAsDecrypted ( > + IN MAP_INFO *MapInfo > + ) > +{ > + return SetBufferAsEncDec (MapInfo, FALSE); > +} > > /** > Provides the controller-specific addresses required to access system memory from a > @@ -113,18 +233,6 @@ IoMmuMap ( > } > > // > - // CommandBuffer was allocated by us (AllocateBuffer) and is already in > - // unencryted buffer so no need to create bounce buffer > - // > - if (Operation == EdkiiIoMmuOperationBusMasterCommonBuffer || > - Operation == EdkiiIoMmuOperationBusMasterCommonBuffer64) { > - *Mapping = NO_MAPPING; > - *DeviceAddress = PhysicalAddress; > - > - return EFI_SUCCESS; > - } > - > - // > // Allocate a MAP_INFO structure to remember the mapping when Unmap() is > // called later. > // > @@ -144,6 +252,25 @@ IoMmuMap ( > MapInfo->DeviceAddress = DmaMemoryTop; > > // > + // If the requested Map() operation is BusMasterCommandBuffer then map > + // using internal function otherwise allocate a bounce buffer to map > + // the host buffer to device buffer > + // > + if (Operation == EdkiiIoMmuOperationBusMasterCommonBuffer || > + Operation == EdkiiIoMmuOperationBusMasterCommonBuffer64) { > + > + Status = SetHostBufferAsDecrypted (MapInfo); > + if (EFI_ERROR (Status)) { > + FreePool (MapInfo); > + *NumberOfBytes = 0; > + return Status; > + } > + > + MapInfo->DeviceAddress = MapInfo->HostAddress; > + goto Done; > + } > + > + // > // Allocate a buffer to map the transfer to. > // > Status = gBS->AllocatePages ( > @@ -178,6 +305,7 @@ IoMmuMap ( > ); > } > > +Done: > // > // The DeviceAddress is the address of the maped buffer below 4GB > // > @@ -219,18 +347,25 @@ IoMmuUnmap ( > return EFI_INVALID_PARAMETER; > } > > - // > - // See if the Map() operation associated with this Unmap() required a mapping > - // buffer. If a mapping buffer was not required, then this function simply > - // buffer. If a mapping buffer was not required, then this function simply > - // > - if (Mapping == NO_MAPPING) { > - return EFI_SUCCESS; > - } > - > MapInfo = (MAP_INFO *)Mapping; > > // > + // If this is a CommonBuffer operation from the Bus Master's point of > + // view then Map() have cleared the memory encryption mask from Host > + // buffer. Lets restore the memory encryption mask before returning > + // > + if (MapInfo->Operation == EdkiiIoMmuOperationBusMasterCommonBuffer || > + MapInfo->Operation == EdkiiIoMmuOperationBusMasterCommonBuffer64) { > + > + Status = SetHostBufferAsEncrypted (MapInfo); > + if (EFI_ERROR (Status)) { > + return Status; > + } > + > + goto Done; > + } > + > + // > // If this is a write operation from the Bus Master's point of view, > // then copy the contents of the mapped buffer into the real buffer > // so the processor can read the contents of the real buffer. > @@ -244,9 +379,6 @@ IoMmuUnmap ( > ); > } > > - DEBUG ((DEBUG_VERBOSE, "%a Device 0x%Lx Host 0x%Lx Pages 0x%Lx Bytes 0x%Lx\n", > - __FUNCTION__, MapInfo->DeviceAddress, MapInfo->HostAddress, > - MapInfo->NumberOfPages, MapInfo->NumberOfBytes)); > // > // Restore the memory encryption mask > // > @@ -254,9 +386,15 @@ IoMmuUnmap ( > ASSERT_EFI_ERROR(Status); > > // > - // Free the mapped buffer and the MAP_INFO structure. > + // Free the bounce buffer > // > gBS->FreePages (MapInfo->DeviceAddress, MapInfo->NumberOfPages); > + > +Done: > + DEBUG ((DEBUG_VERBOSE, "%a Device 0x%Lx Host 0x%Lx Pages 0x%Lx Bytes 0x%Lx\n", > + __FUNCTION__, MapInfo->DeviceAddress, MapInfo->HostAddress, > + MapInfo->NumberOfPages, MapInfo->NumberOfBytes)); > + > FreePool (Mapping); > return EFI_SUCCESS; > } > -- > 2.7.4 >