Re: [PATCH 2/2] EmbeddedPkg: implement EDK2 IoMmu protocol wrapping DmaLib

["Ard Biesheuvel" <ard.biesheuvel@linaro.org>]

On Mon, 25 Nov 2019 at 13:49, Leif Lindholm <leif.lindholm@linaro.org> wrote:
>
> On Thu, Nov 21, 2019 at 09:32:27 +0100, Ard Biesheuvel wrote:
> > Implement a version of the EDK2 IoMmu protocol that is a simple wrapper
> > around DmaLib. This is intended to be used to wrap NonCoherentDmaLib so
> > that the generic PCI infrastructure can be used to implement support for
> > non cache-coherent DMA.
> >
> > Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
>
> Reviewed-by: Leif Lindholm <leif.lindholm@linaro.org>
>

Cheers.

Note that 1/2 of this set actually has problems, so I'll resend shortly.

> > ---
> >  EmbeddedPkg/Drivers/NonCoherentIoMmuDxe/NonCoherentIoMmuDxe.c   | 257 ++++++++++++++++++++
> >  EmbeddedPkg/Drivers/NonCoherentIoMmuDxe/NonCoherentIoMmuDxe.inf |  43 ++++
> >  EmbeddedPkg/EmbeddedPkg.dsc                                     |   5 +
> >  3 files changed, 305 insertions(+)
> >
> > diff --git a/EmbeddedPkg/Drivers/NonCoherentIoMmuDxe/NonCoherentIoMmuDxe.c b/EmbeddedPkg/Drivers/NonCoherentIoMmuDxe/NonCoherentIoMmuDxe.c
> > new file mode 100644
> > index 000000000000..4b0afe47de4c
> > --- /dev/null
> > +++ b/EmbeddedPkg/Drivers/NonCoherentIoMmuDxe/NonCoherentIoMmuDxe.c
> > @@ -0,0 +1,257 @@
> > +/** @file
> > +
> > +  Copyright (c) 2019, Linaro, Ltd. All rights reserved.<BR>
> > +
> > +  This program and the accompanying materials are licensed and made available
> > +  under the terms and conditions of the BSD License which accompanies this
> > +  distribution.  The full text of the license may be found at
> > +  http://opensource.org/licenses/bsd-license.php
> > +
> > +  THE PROGRAM IS DISTRIBUTED UNDER THE BSD LICENSE ON AN "AS IS" BASIS,
> > +  WITHOUT WARRANTIES OR REPRESENTATIONS OF ANY KIND, EITHER EXPRESS OR IMPLIED.
> > +
> > +**/
> > +
> > +#include <PiDxe.h>
> > +#include <Library/BaseLib.h>
> > +#include <Library/DebugLib.h>
> > +#include <Library/DmaLib.h>
> > +#include <Library/UefiBootServicesTableLib.h>
> > +#include <Protocol/IoMmu.h>
> > +
> > +/**
> > +  Set IOMMU attribute for a system memory.
> > +
> > +  If the IOMMU protocol exists, the system memory cannot be used
> > +  for DMA by default.
> > +
> > +  When a device requests a DMA access for a system memory,
> > +  the device driver need use SetAttribute() to update the IOMMU
> > +  attribute to request DMA access (read and/or write).
> > +
> > +  The DeviceHandle is used to identify which device submits the request.
> > +  The IOMMU implementation need translate the device path to an IOMMU device
> > +  ID, and set IOMMU hardware register accordingly.
> > +  1) DeviceHandle can be a standard PCI device.
> > +     The memory for BusMasterRead need set EDKII_IOMMU_ACCESS_READ.
> > +     The memory for BusMasterWrite need set EDKII_IOMMU_ACCESS_WRITE.
> > +     The memory for BusMasterCommonBuffer need set
> > +     EDKII_IOMMU_ACCESS_READ|EDKII_IOMMU_ACCESS_WRITE.
> > +     After the memory is used, the memory need set 0 to keep it being
> > +     protected.
> > +  2) DeviceHandle can be an ACPI device (ISA, I2C, SPI, etc).
> > +     The memory for DMA access need set EDKII_IOMMU_ACCESS_READ and/or
> > +     EDKII_IOMMU_ACCESS_WRITE.
> > +
> > +  @param[in]  This              The protocol instance pointer.
> > +  @param[in]  DeviceHandle      The device who initiates the DMA access
> > +                                request.
> > +  @param[in]  Mapping           The mapping value returned from Map().
> > +  @param[in]  IoMmuAccess       The IOMMU access.
> > +
> > +  @retval EFI_SUCCESS            The IoMmuAccess is set for the memory range
> > +                                 specified by DeviceAddress and Length.
> > +  @retval EFI_INVALID_PARAMETER  DeviceHandle is an invalid handle.
> > +  @retval EFI_INVALID_PARAMETER  Mapping is not a value that was returned by
> > +                                 Map().
> > +  @retval EFI_INVALID_PARAMETER  IoMmuAccess specified an illegal combination
> > +                                 of access.
> > +  @retval EFI_UNSUPPORTED        DeviceHandle is unknown by the IOMMU.
> > +  @retval EFI_UNSUPPORTED        The bit mask of IoMmuAccess is not supported
> > +                                 by the IOMMU.
> > +  @retval EFI_UNSUPPORTED        The IOMMU does not support the memory range
> > +                                 specified by Mapping.
> > +  @retval EFI_OUT_OF_RESOURCES   There are not enough resources available to
> > +                                 modify the IOMMU access.
> > +  @retval EFI_DEVICE_ERROR       The IOMMU device reported an error while
> > +                                 attempting the operation.
> > +
> > +**/
> > +STATIC
> > +EFI_STATUS
> > +EFIAPI
> > +NonCoherentIoMmuSetAttribute (
> > +  IN EDKII_IOMMU_PROTOCOL  *This,
> > +  IN EFI_HANDLE            DeviceHandle,
> > +  IN VOID                  *Mapping,
> > +  IN UINT64                IoMmuAccess
> > +  )
> > +{
> > +  return EFI_UNSUPPORTED;
> > +}
> > +
> > +/**
> > +  Provides the controller-specific addresses required to access system memory
> > +  from a DMA bus master. On SEV guest, the DMA operations must be performed on
> > +  shared buffer hence we allocate a bounce buffer to map the HostAddress to a
> > +  DeviceAddress. The Encryption attribute is removed from the DeviceAddress
> > +  buffer.
> > +
> > +  @param  This                  The protocol instance pointer.
> > +  @param  Operation             Indicates if the bus master is going to read or
> > +                                write to system memory.
> > +  @param  HostAddress           The system memory address to map to the PCI
> > +                                controller.
> > +  @param  NumberOfBytes         On input the number of bytes to map. On output
> > +                                the number of bytes that were mapped.
> > +  @param  DeviceAddress         The resulting map address for the bus master
> > +                                PCI controller to use to access the hosts
> > +                                HostAddress.
> > +  @param  Mapping               A resulting value to pass to Unmap().
> > +
> > +  @retval EFI_SUCCESS           The range was mapped for the returned
> > +                                NumberOfBytes.
> > +  @retval EFI_UNSUPPORTED       The HostAddress cannot be mapped as a common
> > +                                buffer.
> > +  @retval EFI_INVALID_PARAMETER One or more parameters are invalid.
> > +  @retval EFI_OUT_OF_RESOURCES  The request could not be completed due to a
> > +                                lack of resources.
> > +  @retval EFI_DEVICE_ERROR      The system hardware could not map the requested
> > +                                address.
> > +
> > +**/
> > +STATIC
> > +EFI_STATUS
> > +EFIAPI
> > +NonCoherentIoMmuMap (
> > +  IN     EDKII_IOMMU_PROTOCOL                       *This,
> > +  IN     EDKII_IOMMU_OPERATION                      Operation,
> > +  IN     VOID                                       *HostAddress,
> > +  IN OUT UINTN                                      *NumberOfBytes,
> > +  OUT    EFI_PHYSICAL_ADDRESS                       *DeviceAddress,
> > +  OUT    VOID                                       **Mapping
> > +  )
> > +{
> > +  DMA_MAP_OPERATION     DmaOperation;
> > +
> > +  switch (Operation) {
> > +    case EdkiiIoMmuOperationBusMasterRead:
> > +    case EdkiiIoMmuOperationBusMasterRead64:
> > +      DmaOperation = MapOperationBusMasterRead;
> > +      break;
> > +
> > +    case EdkiiIoMmuOperationBusMasterWrite:
> > +    case EdkiiIoMmuOperationBusMasterWrite64:
> > +      DmaOperation = MapOperationBusMasterWrite;
> > +      break;
> > +
> > +    case EdkiiIoMmuOperationBusMasterCommonBuffer:
> > +    case EdkiiIoMmuOperationBusMasterCommonBuffer64:
> > +      DmaOperation = MapOperationBusMasterCommonBuffer;
> > +      break;
> > +
> > +    default:
> > +      ASSERT (FALSE);
> > +      return EFI_INVALID_PARAMETER;
> > +  }
> > +
> > +  return DmaMap (DmaOperation, HostAddress, NumberOfBytes,
> > +           DeviceAddress, Mapping);
> > +}
> > +
> > +/**
> > +  Completes the Map() operation and releases any corresponding resources.
> > +
> > +  @param  This                  The protocol instance pointer.
> > +  @param  Mapping               The mapping value returned from Map().
> > +
> > +  @retval EFI_SUCCESS           The range was unmapped.
> > +  @retval EFI_INVALID_PARAMETER Mapping is not a value that was returned by
> > +                                Map().
> > +  @retval EFI_DEVICE_ERROR      The data was not committed to the target system
> > +                                memory.
> > +**/
> > +STATIC
> > +EFI_STATUS
> > +EFIAPI
> > +NonCoherentIoMmuUnmap (
> > +  IN  EDKII_IOMMU_PROTOCOL                     *This,
> > +  IN  VOID                                     *Mapping
> > +  )
> > +{
> > +  return DmaUnmap (Mapping);
> > +}
> > +
> > +/**
> > +  Allocates pages that are suitable for an OperationBusMasterCommonBuffer or
> > +  OperationBusMasterCommonBuffer64 mapping.
> > +
> > +  @param  This                  The protocol instance pointer.
> > +  @param  Type                  This parameter is not used and must be ignored.
> > +  @param  MemoryType            The type of memory to allocate,
> > +                                EfiBootServicesData or EfiRuntimeServicesData.
> > +  @param  Pages                 The number of pages to allocate.
> > +  @param  HostAddress           A pointer to store the base system memory
> > +                                address of the allocated range.
> > +  @param  Attributes            The requested bit mask of attributes for the
> > +                                allocated range.
> > +
> > +  @retval EFI_SUCCESS           The requested memory pages were allocated.
> > +  @retval EFI_UNSUPPORTED       Attributes is unsupported. The only legal
> > +                                attribute bits are MEMORY_WRITE_COMBINE and
> > +                                MEMORY_CACHED.
> > +  @retval EFI_INVALID_PARAMETER One or more parameters are invalid.
> > +  @retval EFI_OUT_OF_RESOURCES  The memory pages could not be allocated.
> > +
> > +**/
> > +STATIC
> > +EFI_STATUS
> > +EFIAPI
> > +NonCoherentIoMmuAllocateBuffer (
> > +  IN     EDKII_IOMMU_PROTOCOL                     *This,
> > +  IN     EFI_ALLOCATE_TYPE                        Type,
> > +  IN     EFI_MEMORY_TYPE                          MemoryType,
> > +  IN     UINTN                                    Pages,
> > +  IN OUT VOID                                     **HostAddress,
> > +  IN     UINT64                                   Attributes
> > +  )
> > +{
> > +  return DmaAllocateBuffer (MemoryType, Pages, HostAddress);
> > +}
> > +
> > +/**
> > +  Frees memory that was allocated with AllocateBuffer().
> > +
> > +  @param  This                  The protocol instance pointer.
> > +  @param  Pages                 The number of pages to free.
> > +  @param  HostAddress           The base system memory address of the allocated
> > +                                range.
> > +
> > +  @retval EFI_SUCCESS           The requested memory pages were freed.
> > +  @retval EFI_INVALID_PARAMETER The memory range specified by HostAddress and
> > +                                Pages was not allocated with AllocateBuffer().
> > +
> > +**/
> > +STATIC
> > +EFI_STATUS
> > +EFIAPI
> > +NonCoherentIoMmuFreeBuffer (
> > +  IN  EDKII_IOMMU_PROTOCOL                     *This,
> > +  IN  UINTN                                    Pages,
> > +  IN  VOID                                     *HostAddress
> > +  )
> > +{
> > +  return DmaFreeBuffer (Pages, HostAddress);
> > +}
> > +
> > +STATIC EDKII_IOMMU_PROTOCOL  mNonCoherentIoMmuOps = {
> > +  EDKII_IOMMU_PROTOCOL_REVISION,
> > +  NonCoherentIoMmuSetAttribute,
> > +  NonCoherentIoMmuMap,
> > +  NonCoherentIoMmuUnmap,
> > +  NonCoherentIoMmuAllocateBuffer,
> > +  NonCoherentIoMmuFreeBuffer,
> > +};
> > +
> > +
> > +EFI_STATUS
> > +EFIAPI
> > +NonCoherentIoMmuDxeEntryPoint (
> > +  IN EFI_HANDLE         ImageHandle,
> > +  IN EFI_SYSTEM_TABLE   *SystemTable
> > +  )
> > +{
> > +  return gBS->InstallMultipleProtocolInterfaces (&ImageHandle,
> > +                &gEdkiiIoMmuProtocolGuid, &mNonCoherentIoMmuOps,
> > +                NULL);
> > +}
> > diff --git a/EmbeddedPkg/Drivers/NonCoherentIoMmuDxe/NonCoherentIoMmuDxe.inf b/EmbeddedPkg/Drivers/NonCoherentIoMmuDxe/NonCoherentIoMmuDxe.inf
> > new file mode 100644
> > index 000000000000..de70cfb4cad7
> > --- /dev/null
> > +++ b/EmbeddedPkg/Drivers/NonCoherentIoMmuDxe/NonCoherentIoMmuDxe.inf
> > @@ -0,0 +1,43 @@
> > +#/** @file
> > +#
> > +#  Copyright (c) 2019, Linaro, Ltd. All rights reserved.<BR>
> > +#
> > +#  This program and the accompanying materials are licensed and made available
> > +#  under the terms and conditions of the BSD License which accompanies this
> > +#  distribution.  The full text of the license may be found at
> > +#  http://opensource.org/licenses/bsd-license.php
> > +#
> > +#  THE PROGRAM IS DISTRIBUTED UNDER THE BSD LICENSE ON AN "AS IS" BASIS,
> > +#  WITHOUT WARRANTIES OR REPRESENTATIONS OF ANY KIND, EITHER EXPRESS OR
> > +#  IMPLIED.
> > +#
> > +#**/
> > +
> > +[Defines]
> > +  INF_VERSION                    = 1.27
> > +  BASE_NAME                      = NonCoherentIoMmuDxe
> > +  FILE_GUID                      = 7ed510aa-9cdc-49d2-a306-6e11e359f9b3
> > +  MODULE_TYPE                    = DXE_DRIVER
> > +  VERSION_STRING                 = 1.0
> > +  ENTRY_POINT                    = NonCoherentIoMmuDxeEntryPoint
> > +
> > +[Sources]
> > +  NonCoherentIoMmuDxe.c
> > +
> > +[Packages]
> > +  EmbeddedPkg/EmbeddedPkg.dec
> > +  MdeModulePkg/MdeModulePkg.dec
> > +  MdePkg/MdePkg.dec
> > +
> > +[LibraryClasses]
> > +  BaseLib
> > +  DebugLib
> > +  DmaLib
> > +  UefiBootServicesTableLib
> > +  UefiDriverEntryPoint
> > +
> > +[Protocols]
> > +  gEdkiiIoMmuProtocolGuid                     ## PRODUCES
> > +
> > +[Depex]
> > +  TRUE
> > diff --git a/EmbeddedPkg/EmbeddedPkg.dsc b/EmbeddedPkg/EmbeddedPkg.dsc
> > index a8a151eb40cb..8842acc4cbf4 100644
> > --- a/EmbeddedPkg/EmbeddedPkg.dsc
> > +++ b/EmbeddedPkg/EmbeddedPkg.dsc
> > @@ -238,6 +238,11 @@ [Components.common]
> >    EmbeddedPkg/Drivers/ConsolePrefDxe/ConsolePrefDxe.inf
> >    EmbeddedPkg/Drivers/DtPlatformDxe/DtPlatformDxe.inf
> >
> > +  EmbeddedPkg/Drivers/NonCoherentIoMmuDxe/NonCoherentIoMmuDxe.inf {
> > +    <LibraryClasses>
> > +      DmaLib|EmbeddedPkg/Library/NonCoherentDmaLib/NonCoherentDmaLib.inf
> > +  }
> > +
> >  [Components.ARM]
> >    EmbeddedPkg/Drivers/Isp1761UsbDxe/Isp1761UsbDxe.inf
> >
> > --
> > 2.17.1
> >