From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-wm0-x22c.google.com (mail-wm0-x22c.google.com [IPv6:2a00:1450:400c:c09::22c]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ml01.01.org (Postfix) with ESMTPS id 1669981D5B for ; Thu, 3 Nov 2016 04:41:56 -0700 (PDT) Received: by mail-wm0-x22c.google.com with SMTP id n67so95148764wme.1 for ; Thu, 03 Nov 2016 04:41:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=IL6+nAZdU7YyWPHBf+wRq6Cwj9foKorENWvJs3qcgns=; b=DyLzGzQrtJDpbi6gLxZVLXvi2CCBAjSOni9JohwIRaEQs30sGpd2wMH2O19xd/3hpK GpTAlFbjH582yeMjzwc/etS+bAJnQxYFZoL7IHO7QNVO7lLOAVMeuxdknX395gWngGKc ggkIy8yOAZFMW/1rVwy3Rv+ZtjkBhuLCHAqTo= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=IL6+nAZdU7YyWPHBf+wRq6Cwj9foKorENWvJs3qcgns=; b=AIH80sB0aZxnEdfg/hGam/3HrP2jjkvJynMc1JLOB8Wjg5DTbOIV0poH7LKCx6C8zt 5uo0qUwAh4Z2xSpvFIepBSkOJtlqD5+AJ9S7ARo7Gi6UQc3ClXSV4+9G7ubGyWrbZBlL vGwaWfj2lVqvUck3AMoPvty1Li1XIghUux9oURJ3kUrEhlb19NF2EOj7qXi/IEjM3mvI 4w4Go4XBbv/V4Gwn8tQqxjVHn6xs2s3OgbTQJc0PtzIFufdFswqMvU4nViLkml3UtVuV i9dqegVrtNyos02sAZyLR5HD34/0q2ZZTNEPLVnZFvq6cY3iMY0/GPJU2xY/XW0cW1L7 pWjA== X-Gm-Message-State: ABUngvcBrGIoIglA9kYsBqu+sQ5GG6o4zcHfrKrFWC3L6MsrQEnKg11UdxJb2EUBzmL3P3Sf X-Received: by 10.28.19.67 with SMTP id 64mr1498202wmt.111.1478173316468; Thu, 03 Nov 2016 04:41:56 -0700 (PDT) Received: from localhost.localdomain ([105.151.153.127]) by smtp.gmail.com with ESMTPSA id za1sm8108371wjb.8.2016.11.03.04.41.53 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Thu, 03 Nov 2016 04:41:55 -0700 (PDT) From: Ard Biesheuvel To: edk2-devel@lists.01.org, leif.lindholm@linaro.org, michael.d.kinney@intel.com, afish@apple.com Cc: mw@semihalf.com, feng.tian@intel.com, star.zeng@intel.com, Ard Biesheuvel Date: Thu, 3 Nov 2016 11:41:41 +0000 Message-Id: <1478173302-22349-4-git-send-email-ard.biesheuvel@linaro.org> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1478173302-22349-1-git-send-email-ard.biesheuvel@linaro.org> References: <1478173302-22349-1-git-send-email-ard.biesheuvel@linaro.org> Subject: [PATCH v2 4/5] MdeModulePkg/NonDiscoverablePciDeviceDxe: add support for non-coherent DMA X-BeenThere: edk2-devel@lists.01.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: EDK II Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 03 Nov 2016 11:41:56 -0000 Add support for non-coherent DMA, either by performing explicit cache maintenance when DMA mappings are aligned to the CPU's DMA buffer alignment, or by bounce buffering via uncached mappings otherwise. Contributed-under: TianoCore Contribution Agreement 1.0 Signed-off-by: Ard Biesheuvel --- MdeModulePkg/Bus/Pci/NonDiscoverablePciDeviceDxe/NonDiscoverablePciDeviceDxe.c | 17 +- MdeModulePkg/Bus/Pci/NonDiscoverablePciDeviceDxe/NonDiscoverablePciDeviceDxe.inf | 2 + MdeModulePkg/Bus/Pci/NonDiscoverablePciDeviceDxe/NonDiscoverablePciDeviceIo.c | 254 ++++++++++++++++++++ MdeModulePkg/Bus/Pci/NonDiscoverablePciDeviceDxe/NonDiscoverablePciDeviceIo.h | 5 + 4 files changed, 271 insertions(+), 7 deletions(-) diff --git a/MdeModulePkg/Bus/Pci/NonDiscoverablePciDeviceDxe/NonDiscoverablePciDeviceDxe.c b/MdeModulePkg/Bus/Pci/NonDiscoverablePciDeviceDxe/NonDiscoverablePciDeviceDxe.c index c7f1c42208d7..16cc3fb7e242 100644 --- a/MdeModulePkg/Bus/Pci/NonDiscoverablePciDeviceDxe/NonDiscoverablePciDeviceDxe.c +++ b/MdeModulePkg/Bus/Pci/NonDiscoverablePciDeviceDxe/NonDiscoverablePciDeviceDxe.c @@ -16,6 +16,8 @@ #include +EFI_CPU_ARCH_PROTOCOL *mCpu; + // // Probe, start and stop functions of this driver, called by the DXE core for // specific devices. @@ -62,13 +64,9 @@ NonDiscoverablePciDeviceSupported ( case NonDiscoverableDeviceTypeSdhci: case NonDiscoverableDeviceTypeUfs: case NonDiscoverableDeviceTypeNvme: - // - // Restricted to DMA coherent for now - // - if (Device->DmaType == NonDiscoverableDeviceDmaTypeCoherent) { - Status = EFI_SUCCESS; - break; - } + Status = EFI_SUCCESS; + break; + default: Status = EFI_UNSUPPORTED; } @@ -194,6 +192,11 @@ NonDiscoverablePciDeviceDxeEntryPoint ( IN EFI_SYSTEM_TABLE *SystemTable ) { + EFI_STATUS Status; + + Status = gBS->LocateProtocol (&gEfiCpuArchProtocolGuid, NULL, (VOID **)&mCpu); + ASSERT_EFI_ERROR(Status); + return EfiLibInstallDriverBindingComponentName2 ( ImageHandle, SystemTable, diff --git a/MdeModulePkg/Bus/Pci/NonDiscoverablePciDeviceDxe/NonDiscoverablePciDeviceDxe.inf b/MdeModulePkg/Bus/Pci/NonDiscoverablePciDeviceDxe/NonDiscoverablePciDeviceDxe.inf index da1e986b6e9e..e28562533ae8 100644 --- a/MdeModulePkg/Bus/Pci/NonDiscoverablePciDeviceDxe/NonDiscoverablePciDeviceDxe.inf +++ b/MdeModulePkg/Bus/Pci/NonDiscoverablePciDeviceDxe/NonDiscoverablePciDeviceDxe.inf @@ -32,11 +32,13 @@ [Packages] [LibraryClasses] BaseMemoryLib DebugLib + DxeServicesTableLib MemoryAllocationLib UefiBootServicesTableLib UefiDriverEntryPoint UefiLib [Protocols] + gEfiCpuArchProtocolGuid ## CONSUMES gEfiPciIoProtocolGuid ## BY_START gNonDiscoverableDeviceProtocolGuid ## TO_START diff --git a/MdeModulePkg/Bus/Pci/NonDiscoverablePciDeviceDxe/NonDiscoverablePciDeviceIo.c b/MdeModulePkg/Bus/Pci/NonDiscoverablePciDeviceDxe/NonDiscoverablePciDeviceIo.c index 1269194a9eca..d7d687a853bb 100644 --- a/MdeModulePkg/Bus/Pci/NonDiscoverablePciDeviceDxe/NonDiscoverablePciDeviceIo.c +++ b/MdeModulePkg/Bus/Pci/NonDiscoverablePciDeviceDxe/NonDiscoverablePciDeviceIo.c @@ -15,6 +15,8 @@ #include "NonDiscoverablePciDeviceIo.h" +#include + #include #include @@ -475,6 +477,251 @@ CoherentPciIoFreeBuffer ( return EFI_SUCCESS; } +STATIC +EFI_STATUS +NonCoherentPciIoFreeBuffer ( + IN EFI_PCI_IO_PROTOCOL *This, + IN UINTN Pages, + IN VOID *HostAddress + ) +{ + EFI_STATUS Status; + + Status = gDS->SetMemorySpaceAttributes ( + (EFI_PHYSICAL_ADDRESS)(UINTN)HostAddress, + EFI_PAGES_TO_SIZE (Pages), + EFI_MEMORY_WB); + if (EFI_ERROR (Status)) { + return Status; + } + + FreePages (HostAddress, Pages); + return EFI_SUCCESS; +} + +STATIC +EFI_STATUS +NonCoherentPciIoAllocateBuffer ( + IN EFI_PCI_IO_PROTOCOL *This, + IN EFI_ALLOCATE_TYPE Type, + IN EFI_MEMORY_TYPE MemoryType, + IN UINTN Pages, + OUT VOID **HostAddress, + IN UINT64 Attributes + ) +{ + EFI_GCD_MEMORY_SPACE_DESCRIPTOR GcdDescriptor; + EFI_STATUS Status; + UINT64 MemType; + + Status = CoherentPciIoAllocateBuffer (This, Type, MemoryType, Pages, + HostAddress, Attributes); + if (EFI_ERROR (Status)) { + return Status; + } + + Status = gDS->GetMemorySpaceDescriptor ( + (EFI_PHYSICAL_ADDRESS)(UINTN)HostAddress, + &GcdDescriptor); + if (EFI_ERROR (Status)) { + goto FreeBuffer; + } + + if ((GcdDescriptor.Capabilities & (EFI_MEMORY_WC | EFI_MEMORY_UC)) == 0) { + return EFI_UNSUPPORTED; + } + + // + // Set the preferred memory attributes + // + if ((Attributes & EFI_PCI_ATTRIBUTE_MEMORY_WRITE_COMBINE) != 0 || + (GcdDescriptor.Capabilities & EFI_MEMORY_UC) == 0) { + // + // Use write combining if it was requested, or if it is the only + // type supported by the region. + // + MemType = EFI_MEMORY_WC; + } else { + MemType = EFI_MEMORY_UC; + } + + Status = gDS->SetMemorySpaceAttributes ( + (EFI_PHYSICAL_ADDRESS)(UINTN)*HostAddress, + EFI_PAGES_TO_SIZE (Pages), + MemType); + if (EFI_ERROR (Status)) { + goto FreeBuffer; + } + + Status = mCpu->FlushDataCache ( + mCpu, + (EFI_PHYSICAL_ADDRESS)(UINTN)*HostAddress, + EFI_PAGES_TO_SIZE (Pages), + EfiCpuFlushTypeInvalidate); + if (EFI_ERROR (Status)) { + goto FreeBuffer; + } + return EFI_SUCCESS; + +FreeBuffer: + NonCoherentPciIoFreeBuffer (This, Pages, *HostAddress); + return Status; +} + +STATIC +EFI_STATUS +NonCoherentPciIoMap ( + IN EFI_PCI_IO_PROTOCOL *This, + IN EFI_PCI_IO_PROTOCOL_OPERATION Operation, + IN VOID *HostAddress, + IN OUT UINTN *NumberOfBytes, + OUT EFI_PHYSICAL_ADDRESS *DeviceAddress, + OUT VOID **Mapping + ) +{ + NON_DISCOVERABLE_PCI_DEVICE *Dev; + EFI_STATUS Status; + NON_DISCOVERABLE_PCI_DEVICE_MAP_INFO *MapInfo; + UINTN AlignMask; + VOID *AllocAddress; + EFI_GCD_MEMORY_SPACE_DESCRIPTOR GcdDescriptor; + BOOLEAN Bounce; + + MapInfo = AllocatePool (sizeof *MapInfo); + if (MapInfo == NULL) { + return EFI_OUT_OF_RESOURCES; + } + + MapInfo->HostAddress = HostAddress; + MapInfo->Operation = Operation; + MapInfo->NumberOfBytes = *NumberOfBytes; + + Dev = NON_DISCOVERABLE_PCI_DEVICE_FROM_PCI_IO(This); + + // + // If this device does not support 64-bit DMA addressing, we need to allocate + // a bounce buffer and copy over the data in case HostAddress >= 4 GB. + // + Bounce = ((Dev->Attributes & EFI_PCI_IO_ATTRIBUTE_DUAL_ADDRESS_CYCLE) == 0 && + (UINTN) HostAddress >= SIZE_4GB); + + if (!Bounce) { + switch (Operation) { + case EfiPciIoOperationBusMasterRead: + case EfiPciIoOperationBusMasterWrite: + // + // For streaming DMA, it is sufficient if the buffer is aligned to + // the CPUs DMA buffer alignment. + // + AlignMask = mCpu->DmaBufferAlignment - 1; + if ((((UINTN) HostAddress | *NumberOfBytes) & AlignMask) == 0) { + break; + } + // fall through + + case EfiPciIoOperationBusMasterCommonBuffer: + // + // Check whether the host address refers to an uncached mapping. + // + Status = gDS->GetMemorySpaceDescriptor ( + (EFI_PHYSICAL_ADDRESS)(UINTN)HostAddress, + &GcdDescriptor); + if (EFI_ERROR (Status) || + (GcdDescriptor.Attributes & (EFI_MEMORY_WB|EFI_MEMORY_WT)) != 0) { + Bounce = TRUE; + } + break; + + default: + ASSERT (FALSE); + } + } + + if (Bounce) { + if (Operation == EfiPciIoOperationBusMasterCommonBuffer) { + Status = EFI_DEVICE_ERROR; + goto FreeMapInfo; + } + + Status = NonCoherentPciIoAllocateBuffer (This, AllocateAnyPages, + EfiBootServicesData, EFI_SIZE_TO_PAGES (MapInfo->NumberOfBytes), + &AllocAddress, EFI_PCI_ATTRIBUTE_MEMORY_WRITE_COMBINE); + if (EFI_ERROR (Status)) { + goto FreeMapInfo; + } + MapInfo->AllocAddress = (EFI_PHYSICAL_ADDRESS)(UINTN)AllocAddress; + if (Operation == EfiPciIoOperationBusMasterRead) { + gBS->CopyMem (AllocAddress, HostAddress, *NumberOfBytes); + } + *DeviceAddress = MapInfo->AllocAddress; + } else { + MapInfo->AllocAddress = 0; + *DeviceAddress = (EFI_PHYSICAL_ADDRESS)(UINTN)HostAddress; + + // + // We are not using a bounce buffer: the mapping is sufficiently + // aligned to allow us to simply flush the caches. Note that cleaning + // the caches is necessary for both data directions: + // - for bus master read, we want the latest data to be present + // in main memory + // - for bus master write, we don't want any stale dirty cachelines that + // may be written back unexpectedly, and clobber the data written to + // main memory by the device. + // + mCpu->FlushDataCache (mCpu, (EFI_PHYSICAL_ADDRESS)(UINTN)HostAddress, + *NumberOfBytes, EfiCpuFlushTypeWriteBack); + } + + *Mapping = MapInfo; + return EFI_SUCCESS; + +FreeMapInfo: + FreePool (MapInfo); + + return Status; +} + +STATIC +EFI_STATUS +NonCoherentPciIoUnmap ( + IN EFI_PCI_IO_PROTOCOL *This, + IN VOID *Mapping + ) +{ + NON_DISCOVERABLE_PCI_DEVICE_MAP_INFO *MapInfo; + + if (Mapping == NULL) { + return EFI_DEVICE_ERROR; + } + + MapInfo = Mapping; + if (MapInfo->AllocAddress != 0) { + // + // We are using a bounce buffer: copy back the data if necessary, + // and free the buffer. + // + if (MapInfo->Operation == EfiPciIoOperationBusMasterWrite) { + gBS->CopyMem (MapInfo->HostAddress, (VOID *)(UINTN)MapInfo->AllocAddress, + MapInfo->NumberOfBytes); + } + NonCoherentPciIoFreeBuffer (This, + EFI_SIZE_TO_PAGES (MapInfo->NumberOfBytes), + (VOID *)(UINTN)MapInfo->AllocAddress); + } else { + // + // We are *not* using a bounce buffer: if this is a bus master write, + // we have to invalidate the caches so the CPU will see the uncached + // data written by the device. + // + if (MapInfo->Operation == EfiPciIoOperationBusMasterWrite) { + mCpu->FlushDataCache (mCpu, + (EFI_PHYSICAL_ADDRESS)(UINTN)MapInfo->HostAddress, + MapInfo->NumberOfBytes, EfiCpuFlushTypeInvalidate); + } + } + FreePool (MapInfo); + return EFI_SUCCESS; +} STATIC EFI_STATUS @@ -737,4 +984,11 @@ InitializePciIoProtocol ( // Copy protocol structure CopyMem(&Dev->PciIo, &PciIoTemplate, sizeof PciIoTemplate); + + if (Dev->Device->DmaType == NonDiscoverableDeviceDmaTypeNonCoherent) { + Dev->PciIo.AllocateBuffer = NonCoherentPciIoAllocateBuffer; + Dev->PciIo.FreeBuffer = NonCoherentPciIoFreeBuffer; + Dev->PciIo.Map = NonCoherentPciIoMap; + Dev->PciIo.Unmap = NonCoherentPciIoUnmap; + } } diff --git a/MdeModulePkg/Bus/Pci/NonDiscoverablePciDeviceDxe/NonDiscoverablePciDeviceIo.h b/MdeModulePkg/Bus/Pci/NonDiscoverablePciDeviceDxe/NonDiscoverablePciDeviceIo.h index 5c6086fe6c6b..e7ab60338141 100644 --- a/MdeModulePkg/Bus/Pci/NonDiscoverablePciDeviceDxe/NonDiscoverablePciDeviceIo.h +++ b/MdeModulePkg/Bus/Pci/NonDiscoverablePciDeviceDxe/NonDiscoverablePciDeviceIo.h @@ -15,6 +15,8 @@ #ifndef __NON_DISCOVERABLE_PCI_DEVICE_IO_H__ #define __NON_DISCOVERABLE_PCI_DEVICE_IO_H__ +#include + #include #include #include @@ -25,6 +27,7 @@ #include #include +#include #include #define NON_DISCOVERABLE_PCI_DEVICE_SIG SIGNATURE_32 ('P', 'P', 'I', 'D') @@ -33,6 +36,8 @@ CR (PciIoPointer, NON_DISCOVERABLE_PCI_DEVICE, PciIo, \ NON_DISCOVERABLE_PCI_DEVICE_SIG) +extern EFI_CPU_ARCH_PROTOCOL *mCpu; + typedef struct { UINT32 Signature; // -- 2.7.4