From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mga02.intel.com (mga02.intel.com [134.134.136.20]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ml01.01.org (Postfix) with ESMTPS id 7F84E203BEBE8 for ; Sun, 26 Mar 2017 23:15:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=intel.com; i=@intel.com; q=dns/txt; s=intel; t=1490595338; x=1522131338; h=from:to:cc:subject:date:message-id:references: in-reply-to:mime-version; bh=iI4eQObggHWvAwLFV/b+g/pans7GCxgYauWNzGVD37M=; b=as4I+l+rXAovZ1fXNEFEMucoRpXRtsMH74F944n7tlTMJeDfGYPemPN/ X6aJ5a/W3B/dk7Wv52NjMWw58ieMuw==; Received: from orsmga004.jf.intel.com ([10.7.209.38]) by orsmga101.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 26 Mar 2017 23:15:38 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.36,229,1486454400"; d="scan'208,217";a="70850254" Received: from fmsmsx103.amr.corp.intel.com ([10.18.124.201]) by orsmga004.jf.intel.com with ESMTP; 26 Mar 2017 23:15:37 -0700 Received: from fmsmsx113.amr.corp.intel.com (10.18.116.7) by FMSMSX103.amr.corp.intel.com (10.18.124.201) with Microsoft SMTP Server (TLS) id 14.3.319.2; Sun, 26 Mar 2017 23:15:37 -0700 Received: from shsmsx103.ccr.corp.intel.com (10.239.4.69) by FMSMSX113.amr.corp.intel.com (10.18.116.7) with Microsoft SMTP Server (TLS) id 14.3.319.2; Sun, 26 Mar 2017 23:15:36 -0700 Received: from shsmsx102.ccr.corp.intel.com ([169.254.2.212]) by SHSMSX103.ccr.corp.intel.com ([169.254.4.253]) with mapi id 14.03.0248.002; Mon, 27 Mar 2017 14:15:35 +0800 From: "Yao, Jiewen" To: "Ni, Ruiyu" , "edk2-devel@lists.01.org" CC: Leo Duran , Brijesh Singh Thread-Topic: [RFC] [PATCH 3/3] MdeModulePkg/PciBus: Add IOMMU support. Thread-Index: AQHSpryRB9JMlEdxQ0OPJ4QOdPQKRKGoMtzQ Date: Mon, 27 Mar 2017 06:15:34 +0000 Message-ID: <74D8A39837DF1E4DA445A8C0B3885C503A9143FB@shsmsx102.ccr.corp.intel.com> References: <1490434122-16200-1-git-send-email-jiewen.yao@intel.com> <1490434122-16200-4-git-send-email-jiewen.yao@intel.com> <734D49CCEBEEF84792F5B80ED585239D5B8EC287@SHSMSX104.ccr.corp.intel.com> In-Reply-To: <734D49CCEBEEF84792F5B80ED585239D5B8EC287@SHSMSX104.ccr.corp.intel.com> Accept-Language: zh-CN, en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [10.239.127.40] MIME-Version: 1.0 X-Content-Filtered-By: Mailman/MimeDel 2.1.22 Subject: Re: [RFC] [PATCH 3/3] MdeModulePkg/PciBus: Add IOMMU support. X-BeenThere: edk2-devel@lists.01.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: EDK II Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 27 Mar 2017 06:15:38 -0000 Content-Language: en-US Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Answer below: From: Ni, Ruiyu Sent: Monday, March 27, 2017 1:40 PM To: Yao, Jiewen ; edk2-devel@lists.01.org Cc: Leo Duran ; Brijesh Singh Subject: RE: [RFC] [PATCH 3/3] MdeModulePkg/PciBus: Add IOMMU support. #1. If PciHostBridge driver doesn't consume IOMMU protocol, will the change= in PciBus driver cause any negative impact? For example, causing system hang? Do we need to int= roduce a bit to let PciHostBridge report the capability bit about IOMMU through PciHostBridge.GetAlloc= ationAttribute()? [Jiewen] That is a very good question. I thought it before, but I finally give up the idea to define capability bi= t, because I want to make things simple. I assume the system integration engineer should carefully pick up all drive= rs and validate the combination. If the system integration engineer choose to integrate IOMMU driver, he/she= should also choose a PciHostBridge which IOMMU capability. I do not deny the goodness of such capability. But I want to add them later= until we see the real usage. #2, Can you add a Maps field into the PciIoDevice structure? So that the Pc= iIoMapInfo can be inserted to the list and we can check whether the Mapping is invalid by checking= the existence in the list. This is better than checking the memory signature (PciIoMapInfo->Sig= nature). [Jiewen] I am not clear about this. Can you provide some sample code to show your idea? Maybe just submit a patch based upon current code. If it is good, I can int= egrate the patch directly. #3. I saw PciIo.Map/Allocate sets the IOMMU attribute and Unmap/Free clear= s the IOMMU attribute. Do we have the case that the original IOMMU attribute is not 0? If = that may happen, we may need to have an additional interface in IOMMU protocol to get the original = attribute. [Jiewen] Good question again. I added below word in IOMMU protocol header file. Allocation success mean no one use this memory before, so that it should be= DMA blocked. +/** + Set IOMMU attribute for a system memory. + + If the IOMMU protocol exists, the system memory cannot be used + for DMA by default. + + When a device requests a DMA access for a system memory, + the device driver need use SetAttribute() to update the IOMMU + attribute to request DMA access (read and/or write). Thanks/Ray > -----Original Message----- > From: Yao, Jiewen > Sent: Saturday, March 25, 2017 5:29 PM > To: edk2-devel@lists.01.org > Cc: Ni, Ruiyu >; Leo Duran = >; > Brijesh Singh > > Subject: [RFC] [PATCH 3/3] MdeModulePkg/PciBus: Add IOMMU support. > > The responsibility of PciBus driver is to set IOMMU attribute, because on= ly > PciBus knows which device submits DMA access request. > > PciBus driver assumes that PciHostBridge driver can allocate IOMMU page > aligned memory, if IOMMU protocol exists. > > Cc: Ruiyu Ni > > Cc: Leo Duran > > Cc: Brijesh Singh > > Contributed-under: TianoCore Contribution Agreement 1.0 > Signed-off-by: Jiewen Yao > > --- > MdeModulePkg/Bus/Pci/PciBusDxe/PciBus.c | 12 +++ > MdeModulePkg/Bus/Pci/PciBusDxe/PciBus.h | 10 ++ > MdeModulePkg/Bus/Pci/PciBusDxe/PciBusDxe.inf | 1 + > MdeModulePkg/Bus/Pci/PciBusDxe/PciIo.c | 100 > ++++++++++++++++++++ > 4 files changed, 123 insertions(+) > > diff --git a/MdeModulePkg/Bus/Pci/PciBusDxe/PciBus.c > b/MdeModulePkg/Bus/Pci/PciBusDxe/PciBus.c > index f3be47a..c9ee4de 100644 > --- a/MdeModulePkg/Bus/Pci/PciBusDxe/PciBus.c > +++ b/MdeModulePkg/Bus/Pci/PciBusDxe/PciBus.c > @@ -42,6 +42,8 @@ UINT64 gAllZero = =3D 0; > > EFI_PCI_PLATFORM_PROTOCOL *gPciPlatformProtocol; > EFI_PCI_OVERRIDE_PROTOCOL *gPciOverrideProtocol; > +EDKII_IOMMU_PROTOCOL *gIoMmuProtocol; > +UINTN mIoMmuPageSize =3D 1; > > > GLOBAL_REMOVE_IF_UNREFERENCED > EFI_PCI_HOTPLUG_REQUEST_PROTOCOL mPciHotPlugRequest =3D { @@ - > 256,6 +258,16 @@ PciBusDriverBindingStart ( > } > > gBS->LocateProtocol ( > + &gEdkiiIoMmuProtocolGuid, > + NULL, > + (VOID **) &gIoMmuProtocol > + ); > + if (gIoMmuProtocol !=3D NULL) { > + gIoMmuProtocol->GetPageSize (gIoMmuProtocol, &mIoMmuPageSize); > + ASSERT ((mIoMmuPageSize & (mIoMmuPageSize - 1)) =3D=3D 0); } > + > + gBS->LocateProtocol ( > &gEfiIncompatiblePciDeviceSupportProtocolGuid, > NULL, > (VOID **) &gIncompatiblePciDeviceSupport diff --git > a/MdeModulePkg/Bus/Pci/PciBusDxe/PciBus.h > b/MdeModulePkg/Bus/Pci/PciBusDxe/PciBus.h > index 39ba8b9..6f96696 100644 > --- a/MdeModulePkg/Bus/Pci/PciBusDxe/PciBus.h > +++ b/MdeModulePkg/Bus/Pci/PciBusDxe/PciBus.h > @@ -32,6 +32,7 @@ WITHOUT WARRANTIES OR REPRESENTATIONS OF ANY > KIND, EITHER EXPRESS OR IMPLIED. > #include > #include > #include > +#include > > #include > #include @@ -304,6 +305,13 @@ struct > _PCI_IO_DEVICE { > CR (a, PCI_IO_DEVICE, LoadFile2, PCI_IO_DEVICE_SIGNATURE) > > > +#define PCI_IO_MAP_INFO_SIGNATURE SIGNATURE_32 ('p', 'm', 'a', 'p') > +typedef struct { > + UINT32 Signature; > + UINTN NumberOfBytes; > + EFI_PHYSICAL_ADDRESS DeviceAddress; > + VOID *PciRootBridgeIoMapping; > +} PCI_IO_MAP_INFO; > > // > // Global Variables > @@ -319,6 +327,8 @@ extern UINT64 g= AllOne; > extern UINT64 gAllZero; > extern EFI_PCI_PLATFORM_PROTOCOL *gPciPlatformProtoco= l; > extern EFI_PCI_OVERRIDE_PROTOCOL *gPciOverrideProtoco= l; > +extern EDKII_IOMMU_PROTOCOL *gIoMmuProtocol; > +extern UINTN mIoMmuPageSize; > extern BOOLEAN mReserveIsaAliases; > extern BOOLEAN mReserveVgaAliases; > > diff --git a/MdeModulePkg/Bus/Pci/PciBusDxe/PciBusDxe.inf > b/MdeModulePkg/Bus/Pci/PciBusDxe/PciBusDxe.inf > index a3ab11f..5da094f 100644 > --- a/MdeModulePkg/Bus/Pci/PciBusDxe/PciBusDxe.inf > +++ b/MdeModulePkg/Bus/Pci/PciBusDxe/PciBusDxe.inf > @@ -95,6 +95,7 @@ > gEfiPciRootBridgeIoProtocolGuid ## TO_START > gEfiIncompatiblePciDeviceSupportProtocolGuid ## > SOMETIMES_CONSUMES > gEfiLoadFile2ProtocolGuid ## SOMETIMES_PRODUCES > + gEdkiiIoMmuProtocolGuid ## SOMETIMES_CONSUMES > > [FeaturePcd] > gEfiMdeModulePkgTokenSpaceGuid.PcdPciBusHotplugDeviceSupport ## > CONSUMES > diff --git a/MdeModulePkg/Bus/Pci/PciBusDxe/PciIo.c > b/MdeModulePkg/Bus/Pci/PciBusDxe/PciIo.c > index f72598d..01786c1 100644 > --- a/MdeModulePkg/Bus/Pci/PciBusDxe/PciIo.c > +++ b/MdeModulePkg/Bus/Pci/PciBusDxe/PciIo.c > @@ -967,6 +967,8 @@ PciIoMap ( > { > EFI_STATUS Status; > PCI_IO_DEVICE *PciIoDevice; > + PCI_IO_MAP_INFO *PciIoMapInfo; > + UINT64 IoMmuAttribute; > > PciIoDevice =3D PCI_IO_DEVICE_FROM_PCI_IO_THIS (This); > > @@ -999,6 +1001,46 @@ PciIoMap ( > ); > } > > + if (gIoMmuProtocol !=3D NULL) { > + if (!EFI_ERROR(Status)) { > + PciIoMapInfo =3D AllocatePool (sizeof(*PciIoMapInfo)); > + if (PciIoMapInfo =3D=3D NULL) { > + PciIoDevice->PciRootBridgeIo->Unmap (PciIoDevice->PciRootBridgeI= o, > *Mapping); > + return EFI_OUT_OF_RESOURCES; > + } > + > + PciIoMapInfo->Signature =3D PCI_IO_MAP_INFO_SIGNATURE= ; > + PciIoMapInfo->NumberOfBytes =3D *NumberOfBytes; > + PciIoMapInfo->DeviceAddress =3D *DeviceAddress; > + PciIoMapInfo->PciRootBridgeIoMapping =3D *Mapping; > + *Mapping =3D PciIoMapInfo; > + > + switch (Operation) { > + case EfiPciIoOperationBusMasterRead: > + IoMmuAttribute =3D EDKII_IOMMU_ATTRIBUTE_READ; > + break; > + case EfiPciIoOperationBusMasterWrite: > + IoMmuAttribute =3D EDKII_IOMMU_ATTRIBUTE_WRITE; > + break; > + case EfiPciIoOperationBusMasterCommonBuffer: > + IoMmuAttribute =3D EDKII_IOMMU_ATTRIBUTE_READ | > EDKII_IOMMU_ATTRIBUTE_WRITE; > + break; > + default: > + ASSERT(FALSE); > + return Status; > + } > + // > + // PciHostBridge should map IOMMU page aligned HostAddress. > + // > + gIoMmuProtocol->SetAttribute ( > + gIoMmuProtocol, > + PciIoDevice->Handle, > + PciIoMapInfo->DeviceAddress, > + ALIGN_VALUE(PciIoMapInfo->NumberOfBytes, > mIoMmuPageSize), > + IoMmuAttribute > + ); > + } > + } > return Status; > } > > @@ -1021,9 +1063,19 @@ PciIoUnmap ( > { > EFI_STATUS Status; > PCI_IO_DEVICE *PciIoDevice; > + PCI_IO_MAP_INFO *PciIoMapInfo; > > PciIoDevice =3D PCI_IO_DEVICE_FROM_PCI_IO_THIS (This); > > + PciIoMapInfo =3D NULL; > + if (gIoMmuProtocol !=3D NULL) { > + PciIoMapInfo =3D Mapping; > + if (PciIoMapInfo->Signature !=3D PCI_IO_MAP_INFO_SIGNATURE) { > + return EFI_INVALID_PARAMETER; > + } > + Mapping =3D PciIoMapInfo->PciRootBridgeIoMapping; > + } > + > Status =3D PciIoDevice->PciRootBridgeIo->Unmap ( > PciIoDevice->PciRootBridgeIo, > Mapping @@ -1037,6 +1089,22 @@= PciIoUnmap ( > ); > } > > + if (gIoMmuProtocol !=3D NULL) { > + if (!EFI_ERROR(Status)) { > + // > + // PciHostBridge should map IOMMU page aligned HostAddress. > + // > + gIoMmuProtocol->SetAttribute ( > + gIoMmuProtocol, > + PciIoDevice->Handle, > + PciIoMapInfo->DeviceAddress, > + ALIGN_VALUE(PciIoMapInfo->NumberOfBytes, > mIoMmuPageSize), > + 0 > + ); > + FreePool (PciIoMapInfo); > + } > + } > + > return Status; > } > > @@ -1073,6 +1141,7 @@ PciIoAllocateBuffer ( { > EFI_STATUS Status; > PCI_IO_DEVICE *PciIoDevice; > + UINTN Size; > > if ((Attributes & > (~(EFI_PCI_ATTRIBUTE_MEMORY_WRITE_COMBINE | > EFI_PCI_ATTRIBUTE_MEMORY_CACHED))) !=3D 0){ @@ -1102,6 +1171,21 @@ > PciIoAllocateBuffer ( > ); > } > > + if (gIoMmuProtocol !=3D NULL) { > + if (!EFI_ERROR(Status)) { > + // > + // PciHostBridge should allocate IOMMU page aligned HostAddress. > + // > + Size =3D EFI_PAGES_TO_SIZE(Pages); > + gIoMmuProtocol->SetAttribute ( > + gIoMmuProtocol, > + PciIoDevice->Handle, > + (UINTN)*HostAddress, > + ALIGN_VALUE(Size, mIoMmuPageSize), > + EDKII_IOMMU_ATTRIBUTE_READ | > EDKII_IOMMU_ATTRIBUTE_WRITE > + ); > + } > + } > return Status; > } > > @@ -1127,6 +1211,7 @@ PciIoFreeBuffer ( > { > EFI_STATUS Status; > PCI_IO_DEVICE *PciIoDevice; > + UINTN Size; > > PciIoDevice =3D PCI_IO_DEVICE_FROM_PCI_IO_THIS (This); > > @@ -1144,6 +1229,21 @@ PciIoFreeBuffer ( > ); > } > > + if (gIoMmuProtocol !=3D NULL) { > + if (!EFI_ERROR(Status)) { > + // > + // PciHostBridge should allocate IOMMU page aligned HostAddress. > + // > + Size =3D EFI_PAGES_TO_SIZE(Pages); > + gIoMmuProtocol->SetAttribute ( > + gIoMmuProtocol, > + PciIoDevice->Handle, > + (UINTN)HostAddress, > + ALIGN_VALUE(Size, mIoMmuPageSize), > + 0 > + ); > + } > + } > return Status; > } > > -- > 2.7.4.windows.1