* [PATCH V2 0/2] Improve IntelVTd performance.
@ 2017-08-11 14:54 Jiewen Yao
2017-08-11 14:54 ` [PATCH V2 1/2] IntelSiliconPkg/dsc: Add CacheMaintenanceLib Jiewen Yao
2017-08-11 14:54 ` [PATCH V2 2/2] IntelSiliconPkg/IntelVTdDxe: Improve performance Jiewen Yao
0 siblings, 2 replies; 5+ messages in thread
From: Jiewen Yao @ 2017-08-11 14:54 UTC (permalink / raw)
To: edk2-devel
=========== V2 =============
I found CacheLineFlush() is already defined in CacheMaintenanceLib.
Drop the CpuLib update, but use in CacheMaintenanceLib directly.
=========== V1 =============
This series patch replace WBINVD with CLFlush
in IntelVTdDxe driver, which improves boot performance.
This patch is validated on Intel Kabylake generation.
Jiewen Yao (2):
IntelSiliconPkg/dsc: Add CacheMaintenanceLib.
IntelSiliconPkg/IntelVTdDxe: Improve performance.
IntelSiliconPkg/IntelSiliconPkg.dsc | 1 +
IntelSiliconPkg/IntelVTdDxe/DmaProtection.h | 57 ++---
IntelSiliconPkg/IntelVTdDxe/IntelVTdDxe.c | 10 +
IntelSiliconPkg/IntelVTdDxe/IntelVTdDxe.inf | 3 +
IntelSiliconPkg/IntelVTdDxe/PciInfo.c | 12 +
IntelSiliconPkg/IntelVTdDxe/TranslationTable.c | 58 ++++-
IntelSiliconPkg/IntelVTdDxe/TranslationTableEx.c | 2 +
IntelSiliconPkg/IntelVTdDxe/VtdReg.c | 260 +++++++-------------
8 files changed, 187 insertions(+), 216 deletions(-)
--
2.7.4.windows.1
^ permalink raw reply [flat|nested] 5+ messages in thread
* [PATCH V2 1/2] IntelSiliconPkg/dsc: Add CacheMaintenanceLib.
2017-08-11 14:54 [PATCH V2 0/2] Improve IntelVTd performance Jiewen Yao
@ 2017-08-11 14:54 ` Jiewen Yao
2017-08-11 14:54 ` [PATCH V2 2/2] IntelSiliconPkg/IntelVTdDxe: Improve performance Jiewen Yao
1 sibling, 0 replies; 5+ messages in thread
From: Jiewen Yao @ 2017-08-11 14:54 UTC (permalink / raw)
To: edk2-devel; +Cc: Star Zeng
It will be used by IntelVTdDxe.
Cc: Star Zeng <star.zeng@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.0
Signed-off-by: Jiewen Yao <jiewen.yao@intel.com>
---
IntelSiliconPkg/IntelSiliconPkg.dsc | 1 +
1 file changed, 1 insertion(+)
diff --git a/IntelSiliconPkg/IntelSiliconPkg.dsc b/IntelSiliconPkg/IntelSiliconPkg.dsc
index d837d84..9a9920f 100644
--- a/IntelSiliconPkg/IntelSiliconPkg.dsc
+++ b/IntelSiliconPkg/IntelSiliconPkg.dsc
@@ -38,6 +38,7 @@
UefiDecompressLib|MdePkg/Library/BaseUefiDecompressLib/BaseUefiDecompressLib.inf
PerformanceLib|MdePkg/Library/BasePerformanceLibNull/BasePerformanceLibNull.inf
SerialPortLib|MdePkg/Library/BaseSerialPortLibNull/BaseSerialPortLibNull.inf
+ CacheMaintenanceLib|MdePkg/Library/BaseCacheMaintenanceLib/BaseCacheMaintenanceLib.inf
[LibraryClasses.common.DXE_DRIVER]
UefiDriverEntryPoint|MdePkg/Library/UefiDriverEntryPoint/UefiDriverEntryPoint.inf
--
2.7.4.windows.1
^ permalink raw reply related [flat|nested] 5+ messages in thread
* [PATCH V2 2/2] IntelSiliconPkg/IntelVTdDxe: Improve performance.
2017-08-11 14:54 [PATCH V2 0/2] Improve IntelVTd performance Jiewen Yao
2017-08-11 14:54 ` [PATCH V2 1/2] IntelSiliconPkg/dsc: Add CacheMaintenanceLib Jiewen Yao
@ 2017-08-11 14:54 ` Jiewen Yao
2017-08-14 3:53 ` Zeng, Star
1 sibling, 1 reply; 5+ messages in thread
From: Jiewen Yao @ 2017-08-11 14:54 UTC (permalink / raw)
To: edk2-devel; +Cc: Star Zeng
This patch is to improve IOMMU performance.
All WBINVD is removed due to performance issue.
CLFLUSH is used to to only flush the context table or
second level page table if they are changed.
This patch also removed some unused functions.
Cc: Star Zeng <star.zeng@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.0
Signed-off-by: Jiewen Yao <jiewen.yao@intel.com>
---
IntelSiliconPkg/IntelVTdDxe/DmaProtection.h | 57 ++---
IntelSiliconPkg/IntelVTdDxe/IntelVTdDxe.c | 10 +
IntelSiliconPkg/IntelVTdDxe/IntelVTdDxe.inf | 3 +
IntelSiliconPkg/IntelVTdDxe/PciInfo.c | 12 +
IntelSiliconPkg/IntelVTdDxe/TranslationTable.c | 58 ++++-
IntelSiliconPkg/IntelVTdDxe/TranslationTableEx.c | 2 +
IntelSiliconPkg/IntelVTdDxe/VtdReg.c | 260 +++++++-------------
7 files changed, 186 insertions(+), 216 deletions(-)
diff --git a/IntelSiliconPkg/IntelVTdDxe/DmaProtection.h b/IntelSiliconPkg/IntelVTdDxe/DmaProtection.h
index 8cfa69c..c3b57a0 100644
--- a/IntelSiliconPkg/IntelVTdDxe/DmaProtection.h
+++ b/IntelSiliconPkg/IntelVTdDxe/DmaProtection.h
@@ -25,6 +25,9 @@
#include <Library/PciSegmentLib.h>
#include <Library/DebugLib.h>
#include <Library/UefiLib.h>
+#include <Library/CacheMaintenanceLib.h>
+#include <Library/PerformanceLib.h>
+#include <Library/PrintLib.h>
#include <Guid/EventGroup.h>
#include <Guid/Acpi.h>
@@ -58,6 +61,8 @@ typedef struct {
UINTN PciDescriptorMaxNumber;
BOOLEAN *IsRealPciDevice;
VTD_SOURCE_ID *PciDescriptors;
+ // for statistic analysis
+ UINTN *AccessCount;
} PCI_DEVICE_INFORMATION;
typedef struct {
@@ -68,6 +73,7 @@ typedef struct {
VTD_ROOT_ENTRY *RootEntryTable;
VTD_EXT_ROOT_ENTRY *ExtRootEntryTable;
VTD_SECOND_LEVEL_PAGING_ENTRY *FixedSecondLevelPagingEntry;
+ BOOLEAN HasDirtyContext;
BOOLEAN HasDirtyPages;
PCI_DEVICE_INFORMATION PciDeviceInfo;
} VTD_UNIT_INFORMATION;
@@ -125,40 +131,6 @@ DisableDmar (
);
/**
- Invalid VTd IOTLB page.
-
- @param[in] VtdIndex The index of VTd engine.
- @param[in] Address The address of IOTLB page.
- @param[in] AddressMode The address mode of IOTLB page.
- @param[in] DomainIdentifier The domain ID of the source.
-
- @retval EFI_SUCCESS VTd IOTLB page is invalidated.
- @retval EFI_DEVICE_ERROR VTd IOTLB page is not invalidated.
-**/
-EFI_STATUS
-InvalidateVtdIOTLBPage (
- IN UINTN VtdIndex,
- IN UINT64 Address,
- IN UINT8 AddressMode,
- IN UINT16 DomainIdentifier
- );
-
-/**
- Invalid VTd IOTLB domain.
-
- @param[in] VtdIndex The index of VTd engine.
- @param[in] DomainIdentifier The domain ID of the source.
-
- @retval EFI_SUCCESS VTd IOTLB domain is invalidated.
- @retval EFI_DEVICE_ERROR VTd IOTLB domain is not invalidated.
-**/
-EFI_STATUS
-InvalidateVtdIOTLBDomain (
- IN UINTN VtdIndex,
- IN UINT16 DomainIdentifier
- );
-
-/**
Invalid VTd global IOTLB.
@param[in] VtdIndex The index of VTd engine.
@@ -362,6 +334,7 @@ DumpSecondLevelPagingEntry (
EFI_STATUS
SetPageAttribute (
IN UINTN VtdIndex,
+ IN UINT16 DomainIdentifier,
IN VTD_SECOND_LEVEL_PAGING_ENTRY *SecondLevelPagingEntry,
IN UINT64 BaseAddress,
IN UINT64 Length,
@@ -500,4 +473,20 @@ AllocateZeroPages (
IN UINTN Pages
);
+/**
+ Flush VTD page table and context table memory.
+
+ This action is to make sure the IOMMU engine can get final data in memory.
+
+ @param[in] VtdIndex The index used to identify a VTd engine.
+ @param[in] Base The base address of memory to be flushed.
+ @param[in] Size The size of memory in bytes to be flushed.
+**/
+VOID
+FlushPageTableMemory (
+ IN UINTN VtdIndex,
+ IN UINTN Base,
+ IN UINTN Size
+ );
+
#endif
diff --git a/IntelSiliconPkg/IntelVTdDxe/IntelVTdDxe.c b/IntelSiliconPkg/IntelVTdDxe/IntelVTdDxe.c
index d22222d..7feaaf5 100644
--- a/IntelSiliconPkg/IntelVTdDxe/IntelVTdDxe.c
+++ b/IntelSiliconPkg/IntelVTdDxe/IntelVTdDxe.c
@@ -227,6 +227,7 @@ VTdSetAttribute (
EFI_STATUS Status;
UINT16 Segment;
VTD_SOURCE_ID SourceId;
+ CHAR8 PerfToken[sizeof("VTD(S0000.B00.D00.F00)")];
DumpVtdIfError ();
@@ -239,8 +240,17 @@ VTdSetAttribute (
DEBUG ((DEBUG_VERBOSE, "PCI(S%x.B%x.D%x.F%x) ", Segment, SourceId.Bits.Bus, SourceId.Bits.Device, SourceId.Bits.Function));
DEBUG ((DEBUG_VERBOSE, "(0x%lx~0x%lx) - %lx\n", DeviceAddress, Length, IoMmuAccess));
+ PERF_CODE_BEGIN();
+ AsciiSPrint (PerfToken, sizeof(PerfToken), "VTD(S%04x.B%02x.D%02x.F%02x)", Segment, SourceId.Bits.Bus, SourceId.Bits.Device, SourceId.Bits.Function);
+ PERF_START (gImageHandle, PerfToken, "IntelVTD", 0);
+ PERF_CODE_END();
+
Status = SetAccessAttribute (Segment, SourceId, DeviceAddress, Length, IoMmuAccess);
+ PERF_CODE_BEGIN();
+ PERF_END (gImageHandle, PerfToken, "IntelVTD", 0);
+ PERF_CODE_END();
+
return Status;
}
diff --git a/IntelSiliconPkg/IntelVTdDxe/IntelVTdDxe.inf b/IntelSiliconPkg/IntelVTdDxe/IntelVTdDxe.inf
index 6a61c13..697932e 100644
--- a/IntelSiliconPkg/IntelVTdDxe/IntelVTdDxe.inf
+++ b/IntelSiliconPkg/IntelVTdDxe/IntelVTdDxe.inf
@@ -57,6 +57,9 @@
BaseMemoryLib
MemoryAllocationLib
UefiLib
+ CacheMaintenanceLib
+ PerformanceLib
+ PrintLib
[Guids]
gEfiEventExitBootServicesGuid ## CONSUMES ## Event
diff --git a/IntelSiliconPkg/IntelVTdDxe/PciInfo.c b/IntelSiliconPkg/IntelVTdDxe/PciInfo.c
index d5f096f..27e253d 100644
--- a/IntelSiliconPkg/IntelVTdDxe/PciInfo.c
+++ b/IntelSiliconPkg/IntelVTdDxe/PciInfo.c
@@ -77,6 +77,7 @@ RegisterPciDevice (
UINTN Index;
BOOLEAN *NewIsRealPciDevice;
VTD_SOURCE_ID *NewPciDescriptors;
+ UINTN *NewAccessCount;
PciDeviceInfo = &mVtdUnitInformation[VtdIndex].PciDeviceInfo;
@@ -112,6 +113,12 @@ RegisterPciDevice (
FreePool (NewIsRealPciDevice);
return EFI_OUT_OF_RESOURCES;
}
+ NewAccessCount = AllocateZeroPool (sizeof(*NewAccessCount) * (PciDeviceInfo->PciDescriptorMaxNumber + MAX_PCI_DESCRIPTORS));
+ if (NewAccessCount == NULL) {
+ FreePool (NewIsRealPciDevice);
+ FreePool (NewPciDescriptors);
+ return EFI_OUT_OF_RESOURCES;
+ }
PciDeviceInfo->PciDescriptorMaxNumber += MAX_PCI_DESCRIPTORS;
if (PciDeviceInfo->IsRealPciDevice != NULL) {
CopyMem (NewIsRealPciDevice, PciDeviceInfo->IsRealPciDevice, sizeof(*NewIsRealPciDevice) * PciDeviceInfo->PciDescriptorNumber);
@@ -123,6 +130,11 @@ RegisterPciDevice (
FreePool (PciDeviceInfo->PciDescriptors);
}
PciDeviceInfo->PciDescriptors = NewPciDescriptors;
+ if (PciDeviceInfo->AccessCount != NULL) {
+ CopyMem (NewAccessCount, PciDeviceInfo->AccessCount, sizeof(*NewAccessCount) * PciDeviceInfo->PciDescriptorNumber);
+ FreePool (PciDeviceInfo->AccessCount);
+ }
+ PciDeviceInfo->AccessCount = NewAccessCount;
}
ASSERT (PciDeviceInfo->PciDescriptorNumber < PciDeviceInfo->PciDescriptorMaxNumber);
diff --git a/IntelSiliconPkg/IntelVTdDxe/TranslationTable.c b/IntelSiliconPkg/IntelVTdDxe/TranslationTable.c
index 961d7ca..80fc823 100644
--- a/IntelSiliconPkg/IntelVTdDxe/TranslationTable.c
+++ b/IntelSiliconPkg/IntelVTdDxe/TranslationTable.c
@@ -124,6 +124,7 @@ CreateContextEntry (
RootEntry->Bits.ContextTablePointerHi = (UINT32) RShiftU64 ((UINT64)(UINTN)Buffer, 32);
RootEntry->Bits.Present = 1;
Buffer = (UINT8 *)Buffer + EFI_PAGES_TO_SIZE (ContextPages);
+ FlushPageTableMemory (VtdIndex, (UINTN)RootEntry, sizeof(*RootEntry));
}
ContextEntryTable = (VTD_CONTEXT_ENTRY *)(UINTN)VTD_64BITS_ADDRESS(RootEntry->Bits.ContextTablePointerLo, RootEntry->Bits.ContextTablePointerHi) ;
@@ -142,6 +143,7 @@ CreateContextEntry (
ContextEntry->Bits.AddressWidth = 0x2;
break;
}
+ FlushPageTableMemory (VtdIndex, (UINTN)ContextEntry, sizeof(*ContextEntry));
}
return EFI_SUCCESS;
@@ -250,8 +252,11 @@ CreateSecondLevelPagingEntryTable (
goto Done;
}
}
+ FlushPageTableMemory (VtdIndex, (UINTN)Lvl2PtEntry, SIZE_4KB);
}
+ FlushPageTableMemory (VtdIndex, (UINTN)&Lvl3PtEntry[Lvl3Start], (UINTN)&Lvl3PtEntry[Lvl3End + 1] - (UINTN)&Lvl3PtEntry[Lvl3Start]);
}
+ FlushPageTableMemory (VtdIndex, (UINTN)&Lvl4PtEntry[Lvl4Start], (UINTN)&Lvl4PtEntry[Lvl4End + 1] - (UINTN)&Lvl4PtEntry[Lvl4Start]);
Done:
return SecondLevelPagingEntry;
@@ -429,9 +434,10 @@ InvalidatePageEntry (
IN UINTN VtdIndex
)
{
- if (mVtdUnitInformation[VtdIndex].HasDirtyPages) {
+ if (mVtdUnitInformation[VtdIndex].HasDirtyContext || mVtdUnitInformation[VtdIndex].HasDirtyPages) {
InvalidateVtdIOTLBGlobal (VtdIndex);
}
+ mVtdUnitInformation[VtdIndex].HasDirtyContext = FALSE;
mVtdUnitInformation[VtdIndex].HasDirtyPages = FALSE;
}
@@ -498,6 +504,7 @@ PageAttributeToLength (
/**
Return page table entry to match the address.
+ @param[in] VtdIndex The index used to identify a VTd engine.
@param[in] SecondLevelPagingEntry The second level paging entry in VTd table for the device.
@param[in] Address The address to be checked.
@param[out] PageAttributes The page attribute of the page entry.
@@ -506,6 +513,7 @@ PageAttributeToLength (
**/
VOID *
GetSecondLevelPageTableEntry (
+ IN UINTN VtdIndex,
IN VTD_SECOND_LEVEL_PAGING_ENTRY *SecondLevelPagingEntry,
IN PHYSICAL_ADDRESS Address,
OUT PAGE_ATTRIBUTE *PageAttribute
@@ -535,6 +543,7 @@ GetSecondLevelPageTableEntry (
return NULL;
}
SetSecondLevelPagingEntryAttribute ((VTD_SECOND_LEVEL_PAGING_ENTRY *)&L4PageTable[Index4], EDKII_IOMMU_ACCESS_READ | EDKII_IOMMU_ACCESS_WRITE);
+ FlushPageTableMemory (VtdIndex, (UINTN)&L4PageTable[Index4], sizeof(L4PageTable[Index4]));
}
L3PageTable = (UINT64 *)(UINTN)(L4PageTable[Index4] & PAGING_4K_ADDRESS_MASK_64);
@@ -547,6 +556,7 @@ GetSecondLevelPageTableEntry (
return NULL;
}
SetSecondLevelPagingEntryAttribute ((VTD_SECOND_LEVEL_PAGING_ENTRY *)&L3PageTable[Index3], EDKII_IOMMU_ACCESS_READ | EDKII_IOMMU_ACCESS_WRITE);
+ FlushPageTableMemory (VtdIndex, (UINTN)&L3PageTable[Index3], sizeof(L3PageTable[Index3]));
}
if ((L3PageTable[Index3] & VTD_PG_PS) != 0) {
// 1G
@@ -559,6 +569,7 @@ GetSecondLevelPageTableEntry (
L2PageTable[Index2] = Address & PAGING_2M_ADDRESS_MASK_64;
SetSecondLevelPagingEntryAttribute ((VTD_SECOND_LEVEL_PAGING_ENTRY *)&L2PageTable[Index2], 0);
L2PageTable[Index2] |= VTD_PG_PS;
+ FlushPageTableMemory (VtdIndex, (UINTN)&L2PageTable[Index2], sizeof(L2PageTable[Index2]));
}
if ((L2PageTable[Index2] & VTD_PG_PS) != 0) {
// 2M
@@ -579,12 +590,14 @@ GetSecondLevelPageTableEntry (
/**
Modify memory attributes of page entry.
+ @param[in] VtdIndex The index used to identify a VTd engine.
@param[in] PageEntry The page entry.
@param[in] IoMmuAccess The IOMMU access.
@param[out] IsModified TRUE means page table modified. FALSE means page table not modified.
**/
VOID
ConvertSecondLevelPageEntryAttribute (
+ IN UINTN VtdIndex,
IN VTD_SECOND_LEVEL_PAGING_ENTRY *PageEntry,
IN UINT64 IoMmuAccess,
OUT BOOLEAN *IsModified
@@ -595,6 +608,7 @@ ConvertSecondLevelPageEntryAttribute (
CurrentPageEntry = PageEntry->Uint64;
SetSecondLevelPagingEntryAttribute (PageEntry, IoMmuAccess);
+ FlushPageTableMemory (VtdIndex, (UINTN)PageEntry, sizeof(*PageEntry));
NewPageEntry = PageEntry->Uint64;
if (CurrentPageEntry != NewPageEntry) {
*IsModified = TRUE;
@@ -639,6 +653,7 @@ NeedSplitPage (
/**
This function splits one page entry to small page entries.
+ @param[in] VtdIndex The index used to identify a VTd engine.
@param[in] PageEntry The page entry to be splitted.
@param[in] PageAttribute The page attribute of the page entry.
@param[in] SplitAttribute How to split the page entry.
@@ -649,6 +664,7 @@ NeedSplitPage (
**/
RETURN_STATUS
SplitSecondLevelPage (
+ IN UINTN VtdIndex,
IN VTD_SECOND_LEVEL_PAGING_ENTRY *PageEntry,
IN PAGE_ATTRIBUTE PageAttribute,
IN PAGE_ATTRIBUTE SplitAttribute
@@ -675,8 +691,11 @@ SplitSecondLevelPage (
for (Index = 0; Index < SIZE_4KB / sizeof(UINT64); Index++) {
NewPageEntry[Index] = (BaseAddress + SIZE_4KB * Index) | (PageEntry->Uint64 & PAGE_PROGATE_BITS);
}
+ FlushPageTableMemory (VtdIndex, (UINTN)NewPageEntry, SIZE_4KB);
+
PageEntry->Uint64 = (UINT64)(UINTN)NewPageEntry;
SetSecondLevelPagingEntryAttribute (PageEntry, EDKII_IOMMU_ACCESS_READ | EDKII_IOMMU_ACCESS_WRITE);
+ FlushPageTableMemory (VtdIndex, (UINTN)PageEntry, sizeof(*PageEntry));
return RETURN_SUCCESS;
} else {
return RETURN_UNSUPPORTED;
@@ -697,8 +716,11 @@ SplitSecondLevelPage (
for (Index = 0; Index < SIZE_4KB / sizeof(UINT64); Index++) {
NewPageEntry[Index] = (BaseAddress + SIZE_2MB * Index) | VTD_PG_PS | (PageEntry->Uint64 & PAGE_PROGATE_BITS);
}
+ FlushPageTableMemory (VtdIndex, (UINTN)NewPageEntry, SIZE_4KB);
+
PageEntry->Uint64 = (UINT64)(UINTN)NewPageEntry;
SetSecondLevelPagingEntryAttribute (PageEntry, EDKII_IOMMU_ACCESS_READ | EDKII_IOMMU_ACCESS_WRITE);
+ FlushPageTableMemory (VtdIndex, (UINTN)PageEntry, sizeof(*PageEntry));
return RETURN_SUCCESS;
} else {
return RETURN_UNSUPPORTED;
@@ -730,6 +752,7 @@ SplitSecondLevelPage (
EFI_STATUS
SetSecondLevelPagingAttribute (
IN UINTN VtdIndex,
+ IN UINT16 DomainIdentifier,
IN VTD_SECOND_LEVEL_PAGING_ENTRY *SecondLevelPagingEntry,
IN UINT64 BaseAddress,
IN UINT64 Length,
@@ -756,7 +779,7 @@ SetSecondLevelPagingAttribute (
}
while (Length != 0) {
- PageEntry = GetSecondLevelPageTableEntry (SecondLevelPagingEntry, BaseAddress, &PageAttribute);
+ PageEntry = GetSecondLevelPageTableEntry (VtdIndex, SecondLevelPagingEntry, BaseAddress, &PageAttribute);
if (PageEntry == NULL) {
DEBUG ((DEBUG_ERROR, "PageEntry - NULL\n"));
return RETURN_UNSUPPORTED;
@@ -764,7 +787,7 @@ SetSecondLevelPagingAttribute (
PageEntryLength = PageAttributeToLength (PageAttribute);
SplitAttribute = NeedSplitPage (BaseAddress, Length, PageAttribute);
if (SplitAttribute == PageNone) {
- ConvertSecondLevelPageEntryAttribute (PageEntry, IoMmuAccess, &IsEntryModified);
+ ConvertSecondLevelPageEntryAttribute (VtdIndex, PageEntry, IoMmuAccess, &IsEntryModified);
if (IsEntryModified) {
mVtdUnitInformation[VtdIndex].HasDirtyPages = TRUE;
}
@@ -774,7 +797,7 @@ SetSecondLevelPagingAttribute (
BaseAddress += PageEntryLength;
Length -= PageEntryLength;
} else {
- Status = SplitSecondLevelPage (PageEntry, PageAttribute, SplitAttribute);
+ Status = SplitSecondLevelPage (VtdIndex, PageEntry, PageAttribute, SplitAttribute);
if (RETURN_ERROR (Status)) {
DEBUG ((DEBUG_ERROR, "SplitSecondLevelPage - %r\n", Status));
return RETURN_UNSUPPORTED;
@@ -787,8 +810,6 @@ SetSecondLevelPagingAttribute (
}
}
- InvalidatePageEntry (VtdIndex);
-
return EFI_SUCCESS;
}
@@ -814,6 +835,7 @@ SetSecondLevelPagingAttribute (
EFI_STATUS
SetPageAttribute (
IN UINTN VtdIndex,
+ IN UINT16 DomainIdentifier,
IN VTD_SECOND_LEVEL_PAGING_ENTRY *SecondLevelPagingEntry,
IN UINT64 BaseAddress,
IN UINT64 Length,
@@ -823,7 +845,7 @@ SetPageAttribute (
EFI_STATUS Status;
Status = EFI_NOT_FOUND;
if (SecondLevelPagingEntry != NULL) {
- Status = SetSecondLevelPagingAttribute (VtdIndex, SecondLevelPagingEntry, BaseAddress, Length, IoMmuAccess);
+ Status = SetSecondLevelPagingAttribute (VtdIndex, DomainIdentifier, SecondLevelPagingEntry, BaseAddress, Length, IoMmuAccess);
}
return Status;
}
@@ -862,6 +884,8 @@ SetAccessAttribute (
VTD_CONTEXT_ENTRY *ContextEntry;
VTD_SECOND_LEVEL_PAGING_ENTRY *SecondLevelPagingEntry;
UINT64 Pt;
+ UINTN PciDescriptorIndex;
+ UINT16 DomainIdentifier;
SecondLevelPagingEntry = NULL;
@@ -873,6 +897,13 @@ SetAccessAttribute (
return EFI_DEVICE_ERROR;
}
+ PciDescriptorIndex = GetPciDescriptor (VtdIndex, Segment, SourceId);
+ mVtdUnitInformation[VtdIndex].PciDeviceInfo.AccessCount[PciDescriptorIndex]++;
+ //
+ // DomainId should not be 0.
+ //
+ DomainIdentifier = (UINT16)(PciDescriptorIndex + 1);
+
if (ExtContextEntry != NULL) {
if (ExtContextEntry->Bits.Present == 0) {
SecondLevelPagingEntry = CreateSecondLevelPagingEntry (VtdIndex, 0);
@@ -881,9 +912,11 @@ SetAccessAttribute (
ExtContextEntry->Bits.SecondLevelPageTranslationPointerLo = (UINT32) Pt;
ExtContextEntry->Bits.SecondLevelPageTranslationPointerHi = (UINT32) RShiftU64(Pt, 20);
- ExtContextEntry->Bits.DomainIdentifier = (UINT16) GetPciDescriptor (VtdIndex, Segment, SourceId);
+ ExtContextEntry->Bits.DomainIdentifier = DomainIdentifier;
ExtContextEntry->Bits.Present = 1;
+ FlushPageTableMemory (VtdIndex, (UINTN)ExtContextEntry, sizeof(*ExtContextEntry));
DumpDmarExtContextEntryTable (mVtdUnitInformation[VtdIndex].ExtRootEntryTable);
+ mVtdUnitInformation[VtdIndex].HasDirtyContext = TRUE;
} else {
SecondLevelPagingEntry = (VOID *)(UINTN)VTD_64BITS_ADDRESS(ExtContextEntry->Bits.SecondLevelPageTranslationPointerLo, ExtContextEntry->Bits.SecondLevelPageTranslationPointerHi);
DEBUG ((DEBUG_VERBOSE,"SecondLevelPagingEntry - 0x%x (S%04x B%02x D%02x F%02x)\n", SecondLevelPagingEntry, Segment, SourceId.Bits.Bus, SourceId.Bits.Device, SourceId.Bits.Function));
@@ -896,9 +929,11 @@ SetAccessAttribute (
ContextEntry->Bits.SecondLevelPageTranslationPointerLo = (UINT32) Pt;
ContextEntry->Bits.SecondLevelPageTranslationPointerHi = (UINT32) RShiftU64(Pt, 20);
- ContextEntry->Bits.DomainIdentifier = (UINT16) GetPciDescriptor (VtdIndex, Segment, SourceId);
+ ContextEntry->Bits.DomainIdentifier = DomainIdentifier;
ContextEntry->Bits.Present = 1;
+ FlushPageTableMemory (VtdIndex, (UINTN)ContextEntry, sizeof(*ContextEntry));
DumpDmarContextEntryTable (mVtdUnitInformation[VtdIndex].RootEntryTable);
+ mVtdUnitInformation[VtdIndex].HasDirtyContext = TRUE;
} else {
SecondLevelPagingEntry = (VOID *)(UINTN)VTD_64BITS_ADDRESS(ContextEntry->Bits.SecondLevelPageTranslationPointerLo, ContextEntry->Bits.SecondLevelPageTranslationPointerHi);
DEBUG ((DEBUG_VERBOSE,"SecondLevelPagingEntry - 0x%x (S%04x B%02x D%02x F%02x)\n", SecondLevelPagingEntry, Segment, SourceId.Bits.Bus, SourceId.Bits.Device, SourceId.Bits.Function));
@@ -911,6 +946,7 @@ SetAccessAttribute (
if (SecondLevelPagingEntry != mVtdUnitInformation[VtdIndex].FixedSecondLevelPagingEntry) {
Status = SetPageAttribute (
VtdIndex,
+ DomainIdentifier,
SecondLevelPagingEntry,
BaseAddress,
Length,
@@ -922,6 +958,8 @@ SetAccessAttribute (
}
}
+ InvalidatePageEntry (VtdIndex);
+
return EFI_SUCCESS;
}
@@ -965,11 +1003,13 @@ AlwaysEnablePageAttribute (
ExtContextEntry->Bits.SecondLevelPageTranslationPointerHi = (UINT32) RShiftU64(Pt, 20);
ExtContextEntry->Bits.DomainIdentifier = ((1 << (UINT8)((UINTN)mVtdUnitInformation[VtdIndex].CapReg.Bits.ND * 2 + 4)) - 1);
ExtContextEntry->Bits.Present = 1;
+ FlushPageTableMemory (VtdIndex, (UINTN)ExtContextEntry, sizeof(*ExtContextEntry));
} else if (ContextEntry != NULL) {
ContextEntry->Bits.SecondLevelPageTranslationPointerLo = (UINT32) Pt;
ContextEntry->Bits.SecondLevelPageTranslationPointerHi = (UINT32) RShiftU64(Pt, 20);
ContextEntry->Bits.DomainIdentifier = ((1 << (UINT8)((UINTN)mVtdUnitInformation[VtdIndex].CapReg.Bits.ND * 2 + 4)) - 1);
ContextEntry->Bits.Present = 1;
+ FlushPageTableMemory (VtdIndex, (UINTN)ContextEntry, sizeof(*ContextEntry));
}
return EFI_SUCCESS;
diff --git a/IntelSiliconPkg/IntelVTdDxe/TranslationTableEx.c b/IntelSiliconPkg/IntelVTdDxe/TranslationTableEx.c
index 65ed16e..9d4e6ea 100644
--- a/IntelSiliconPkg/IntelVTdDxe/TranslationTableEx.c
+++ b/IntelSiliconPkg/IntelVTdDxe/TranslationTableEx.c
@@ -73,6 +73,7 @@ CreateExtContextEntry (
ExtRootEntry->Bits.UpperContextTablePointerLo = (UINT32) RShiftU64 ((UINT64)(UINTN)Buffer, 12) + 1;
ExtRootEntry->Bits.UpperContextTablePointerHi = (UINT32) RShiftU64 (RShiftU64 ((UINT64)(UINTN)Buffer, 12) + 1, 20);
ExtRootEntry->Bits.UpperPresent = 1;
+ FlushPageTableMemory (VtdIndex, (UINTN)ExtRootEntry, sizeof(*ExtRootEntry));
Buffer = (UINT8 *)Buffer + EFI_PAGES_TO_SIZE (ContextPages);
}
@@ -92,6 +93,7 @@ CreateExtContextEntry (
ExtContextEntry->Bits.AddressWidth = 0x2;
break;
}
+ FlushPageTableMemory (VtdIndex, (UINTN)ExtContextEntry, sizeof(*ExtContextEntry));
}
return EFI_SUCCESS;
diff --git a/IntelSiliconPkg/IntelVTdDxe/VtdReg.c b/IntelSiliconPkg/IntelVTdDxe/VtdReg.c
index f36e3de..b1178b7 100644
--- a/IntelSiliconPkg/IntelVTdDxe/VtdReg.c
+++ b/IntelSiliconPkg/IntelVTdDxe/VtdReg.c
@@ -20,43 +20,62 @@ VTD_UNIT_INFORMATION *mVtdUnitInformation;
BOOLEAN mVtdEnabled;
/**
- Invalid VTd global IOTLB.
+ Flush VTD page table and context table memory.
- @param[in] VtdIndex The index of VTd engine.
+ This action is to make sure the IOMMU engine can get final data in memory.
- @retval EFI_SUCCESS VTd global IOTLB is invalidated.
- @retval EFI_DEVICE_ERROR VTd global IOTLB is not invalidated.
+ @param[in] VtdIndex The index used to identify a VTd engine.
+ @param[in] Base The base address of memory to be flushed.
+ @param[in] Size The size of memory in bytes to be flushed.
**/
-EFI_STATUS
-InvalidateVtdIOTLBGlobal (
+VOID
+FlushPageTableMemory (
+ IN UINTN VtdIndex,
+ IN UINTN Base,
+ IN UINTN Size
+ )
+{
+ if (mVtdUnitInformation[VtdIndex].ECapReg.Bits.C == 0) {
+ WriteBackDataCacheRange ((VOID *)Base, Size);
+ }
+}
+
+/**
+ Flush VTd engine write buffer.
+
+ @param[in] VtdIndex The index used to identify a VTd engine.
+**/
+VOID
+FlushWriteBuffer (
IN UINTN VtdIndex
)
{
- UINT64 Reg64;
UINT32 Reg32;
- if (!mVtdEnabled) {
- return EFI_SUCCESS;
+ if (mVtdUnitInformation[VtdIndex].CapReg.Bits.RWBF != 0) {
+ Reg32 = MmioRead32 (mVtdUnitInformation[VtdIndex].VtdUnitBaseAddress + R_GSTS_REG);
+ MmioWrite32 (mVtdUnitInformation[VtdIndex].VtdUnitBaseAddress + R_GCMD_REG, Reg32 | B_GMCD_REG_WBF);
+ do {
+ Reg32 = MmioRead32 (mVtdUnitInformation[VtdIndex].VtdUnitBaseAddress + R_GSTS_REG);
+ } while ((Reg32 & B_GSTS_REG_WBF) != 0);
}
+}
- DEBUG((DEBUG_VERBOSE, "InvalidateVtdIOTLBGlobal(%d)\n", VtdIndex));
-
- AsmWbinvd();
+/**
+ Invalidate VTd context cache.
- //
- // Write Buffer Flush before invalidation
- //
- Reg32 = MmioRead32 (mVtdUnitInformation[VtdIndex].VtdUnitBaseAddress + R_CAP_REG);
- if ((Reg32 & B_CAP_REG_RWBF) != 0) {
- MmioWrite32 (mVtdUnitInformation[VtdIndex].VtdUnitBaseAddress + R_GCMD_REG, B_GMCD_REG_WBF);
- }
+ @param[in] VtdIndex The index used to identify a VTd engine.
+**/
+EFI_STATUS
+InvalidateContextCache (
+ IN UINTN VtdIndex
+ )
+{
+ UINT64 Reg64;
- //
- // Invalidate the context cache
- //
Reg64 = MmioRead64 (mVtdUnitInformation[VtdIndex].VtdUnitBaseAddress + R_CCMD_REG);
if ((Reg64 & B_CCMD_REG_ICC) != 0) {
- DEBUG ((DEBUG_ERROR,"ERROR: InvalidateVtdIOTLBGlobal: B_CCMD_REG_ICC is set for VTD(%d)\n",VtdIndex));
+ DEBUG ((DEBUG_ERROR,"ERROR: InvalidateContextCache: B_CCMD_REG_ICC is set for VTD(%d)\n",VtdIndex));
return EFI_DEVICE_ERROR;
}
@@ -68,97 +87,29 @@ InvalidateVtdIOTLBGlobal (
Reg64 = MmioRead64 (mVtdUnitInformation[VtdIndex].VtdUnitBaseAddress + R_CCMD_REG);
} while ((Reg64 & B_CCMD_REG_ICC) != 0);
- //
- // Invalidate the IOTLB cache
- //
-
- Reg64 = MmioRead64 (mVtdUnitInformation[VtdIndex].VtdUnitBaseAddress + (mVtdUnitInformation[VtdIndex].ECapReg.Bits.IRO * 16) + R_IOTLB_REG);
- if ((Reg64 & B_IOTLB_REG_IVT) != 0) {
- DEBUG ((DEBUG_ERROR,"ERROR: InvalidateVtdIOTLBGlobal: B_IOTLB_REG_IVT is set for VTD(%d)\n", VtdIndex));
- return EFI_DEVICE_ERROR;
- }
-
- Reg64 &= ((~B_IOTLB_REG_IVT) & (~B_IOTLB_REG_IIRG_MASK));
- Reg64 |= (B_IOTLB_REG_IVT | V_IOTLB_REG_IIRG_GLOBAL);
- MmioWrite64 (mVtdUnitInformation[VtdIndex].VtdUnitBaseAddress + (mVtdUnitInformation[VtdIndex].ECapReg.Bits.IRO * 16) + R_IOTLB_REG, Reg64);
-
- do {
- Reg64 = MmioRead64 (mVtdUnitInformation[VtdIndex].VtdUnitBaseAddress + (mVtdUnitInformation[VtdIndex].ECapReg.Bits.IRO * 16) + R_IOTLB_REG);
- } while ((Reg64 & B_IOTLB_REG_IVT) != 0);
-
- //
- // Disable VTd
- //
- MmioWrite32 (mVtdUnitInformation[VtdIndex].VtdUnitBaseAddress + R_GCMD_REG, B_GMCD_REG_SRTP);
- do {
- Reg32 = MmioRead32 (mVtdUnitInformation[VtdIndex].VtdUnitBaseAddress + R_GSTS_REG);
- } while((Reg32 & B_GSTS_REG_RTPS) == 0);
-
- //
- // Enable VTd
- //
- MmioWrite32 (mVtdUnitInformation[VtdIndex].VtdUnitBaseAddress + R_GCMD_REG, B_GMCD_REG_TE);
- do {
- Reg32 = MmioRead32 (mVtdUnitInformation[VtdIndex].VtdUnitBaseAddress + R_GSTS_REG);
- } while ((Reg32 & B_GSTS_REG_TE) == 0);
-
return EFI_SUCCESS;
}
/**
- Invalid VTd IOTLB domain.
-
- @param[in] VtdIndex The index of VTd engine.
- @param[in] DomainIdentifier The domain ID of the source.
+ Invalidate VTd IOTLB.
- @retval EFI_SUCCESS VTd IOTLB domain is invalidated.
- @retval EFI_DEVICE_ERROR VTd IOTLB domain is not invalidated.
+ @param[in] VtdIndex The index used to identify a VTd engine.
**/
EFI_STATUS
-InvalidateVtdIOTLBDomain (
- IN UINTN VtdIndex,
- IN UINT16 DomainIdentifier
+InvalidateIOTLB (
+ IN UINTN VtdIndex
)
{
UINT64 Reg64;
- if (!mVtdEnabled) {
- return EFI_SUCCESS;
- }
-
- DEBUG((DEBUG_VERBOSE, "InvalidateVtdIOTLBDomain(%d): 0x%016lx (0x%04x)\n", VtdIndex, DomainIdentifier));
-
- //
- // Invalidate the context cache
- //
- Reg64 = MmioRead64 (mVtdUnitInformation[VtdIndex].VtdUnitBaseAddress + R_CCMD_REG);
- if ((Reg64 & B_CCMD_REG_ICC) != 0) {
- DEBUG ((DEBUG_ERROR,"ERROR: InvalidateVtdIOTLBDomain: B_CCMD_REG_ICC is set for VTD(%d)\n",VtdIndex));
- return EFI_DEVICE_ERROR;
- }
-
- Reg64 &= ((~B_CCMD_REG_ICC) & (~B_CCMD_REG_CIRG_MASK));
- Reg64 |= (B_CCMD_REG_ICC | V_CCMD_REG_CIRG_DOMAIN);
- Reg64 |= (B_CCMD_REG_ICC | V_CCMD_REG_CIRG_DOMAIN);
- MmioWrite64 (mVtdUnitInformation[VtdIndex].VtdUnitBaseAddress + R_CCMD_REG, Reg64);
-
- do {
- Reg64 = MmioRead64 (mVtdUnitInformation[VtdIndex].VtdUnitBaseAddress + R_CCMD_REG);
- } while ((Reg64 & B_CCMD_REG_ICC) != 0);
-
- //
- // Invalidate the IOTLB cache
- //
-
Reg64 = MmioRead64 (mVtdUnitInformation[VtdIndex].VtdUnitBaseAddress + (mVtdUnitInformation[VtdIndex].ECapReg.Bits.IRO * 16) + R_IOTLB_REG);
if ((Reg64 & B_IOTLB_REG_IVT) != 0) {
- DEBUG ((DEBUG_ERROR,"ERROR: InvalidateVtdIOTLBDomain: B_IOTLB_REG_IVT is set for VTD(%d)\n", VtdIndex));
+ DEBUG ((DEBUG_ERROR,"ERROR: InvalidateIOTLB: B_IOTLB_REG_IVT is set for VTD(%d)\n", VtdIndex));
return EFI_DEVICE_ERROR;
}
Reg64 &= ((~B_IOTLB_REG_IVT) & (~B_IOTLB_REG_IIRG_MASK));
- Reg64 |= (B_IOTLB_REG_IVT | V_IOTLB_REG_IIRG_DOMAIN);
- Reg64 |= LShiftU64 (DomainIdentifier, 32);
+ Reg64 |= (B_IOTLB_REG_IVT | V_IOTLB_REG_IIRG_GLOBAL);
MmioWrite64 (mVtdUnitInformation[VtdIndex].VtdUnitBaseAddress + (mVtdUnitInformation[VtdIndex].ECapReg.Bits.IRO * 16) + R_IOTLB_REG, Reg64);
do {
@@ -169,53 +120,41 @@ InvalidateVtdIOTLBDomain (
}
/**
- Invalid VTd IOTLB page.
+ Invalid VTd global IOTLB.
@param[in] VtdIndex The index of VTd engine.
- @param[in] Address The address of IOTLB page.
- @param[in] AddressMode The address mode of IOTLB page.
- @param[in] DomainIdentifier The domain ID of the source.
- @retval EFI_SUCCESS VTd IOTLB page is invalidated.
- @retval EFI_DEVICE_ERROR VTd IOTLB page is not invalidated.
+ @retval EFI_SUCCESS VTd global IOTLB is invalidated.
+ @retval EFI_DEVICE_ERROR VTd global IOTLB is not invalidated.
**/
EFI_STATUS
-InvalidateVtdIOTLBPage (
- IN UINTN VtdIndex,
- IN UINT64 Address,
- IN UINT8 AddressMode,
- IN UINT16 DomainIdentifier
+InvalidateVtdIOTLBGlobal (
+ IN UINTN VtdIndex
)
{
- UINT64 Reg64;
- UINT64 Data64;
-
if (!mVtdEnabled) {
return EFI_SUCCESS;
}
- DEBUG((DEBUG_VERBOSE, "InvalidateVtdIOTLBPage(%d): 0x%016lx (0x%02x)\n", VtdIndex, Address, AddressMode));
-
- if (mVtdUnitInformation[VtdIndex].CapReg.Bits.PSI != 0) {
- Reg64 = MmioRead64 (mVtdUnitInformation[VtdIndex].VtdUnitBaseAddress + (mVtdUnitInformation[VtdIndex].ECapReg.Bits.IRO * 16) + R_IOTLB_REG);
- if ((Reg64 & B_IOTLB_REG_IVT) != 0) {
- DEBUG ((DEBUG_ERROR,"ERROR: InvalidateVtdIOTLBPage: B_IOTLB_REG_IVT is set for VTD(%d)\n", VtdIndex));
- return EFI_DEVICE_ERROR;
- }
+ DEBUG((DEBUG_VERBOSE, "InvalidateVtdIOTLBGlobal(%d)\n", VtdIndex));
- Data64 = Address | AddressMode;
- MmioWrite64 (mVtdUnitInformation[VtdIndex].VtdUnitBaseAddress + (mVtdUnitInformation[VtdIndex].ECapReg.Bits.IRO * 16) + R_IVA_REG, Data64);
+ //
+ // Write Buffer Flush before invalidation
+ //
+ FlushWriteBuffer (VtdIndex);
- Reg64 &= ((~B_IOTLB_REG_IVT) & (~B_IOTLB_REG_IIRG_MASK));
- Reg64 |= (B_IOTLB_REG_IVT | V_IOTLB_REG_IIRG_PAGE);
- Reg64 |= LShiftU64 (DomainIdentifier, 32);
- MmioWrite64 (mVtdUnitInformation[VtdIndex].VtdUnitBaseAddress + (mVtdUnitInformation[VtdIndex].ECapReg.Bits.IRO * 16) + R_IOTLB_REG, Reg64);
+ //
+ // Invalidate the context cache
+ //
+ if (mVtdUnitInformation[VtdIndex].HasDirtyContext) {
+ InvalidateContextCache (VtdIndex);
+ }
- do {
- Reg64 = MmioRead64 (mVtdUnitInformation[VtdIndex].VtdUnitBaseAddress + (mVtdUnitInformation[VtdIndex].ECapReg.Bits.IRO * 16) + R_IOTLB_REG);
- } while ((Reg64 & B_IOTLB_REG_IVT) != 0);
- } else {
- InvalidateVtdIOTLBGlobal (VtdIndex);
+ //
+ // Invalidate the IOTLB cache
+ //
+ if (mVtdUnitInformation[VtdIndex].HasDirtyContext || mVtdUnitInformation[VtdIndex].HasDirtyPages) {
+ InvalidateIOTLB (VtdIndex);
}
return EFI_SUCCESS;
@@ -268,11 +207,8 @@ EnableDmar (
)
{
UINTN Index;
- UINT64 Reg64;
UINT32 Reg32;
- AsmWbinvd();
-
for (Index = 0; Index < mVtdUnitNumber; Index++) {
DEBUG((DEBUG_INFO, ">>>>>>EnableDmar() for engine [%d] \n", Index));
@@ -299,48 +235,17 @@ EnableDmar (
//
// Write Buffer Flush before invalidation
//
- Reg32 = MmioRead32 (mVtdUnitInformation[Index].VtdUnitBaseAddress + R_CAP_REG);
- if ((Reg32 & B_CAP_REG_RWBF) != 0) {
- MmioWrite32 (mVtdUnitInformation[Index].VtdUnitBaseAddress + R_GCMD_REG, B_GMCD_REG_WBF);
- }
+ FlushWriteBuffer (Index);
//
// Invalidate the context cache
//
- Reg64 = MmioRead64 (mVtdUnitInformation[Index].VtdUnitBaseAddress + R_CCMD_REG);
- if ((Reg64 & B_CCMD_REG_ICC) != 0) {
- DEBUG ((DEBUG_INFO,"ERROR: EnableDmar: B_CCMD_REG_ICC is set for VTD(%d)\n",Index));
- return EFI_DEVICE_ERROR;
- }
-
- Reg64 &= ((~B_CCMD_REG_ICC) & (~B_CCMD_REG_CIRG_MASK));
- Reg64 |= (B_CCMD_REG_ICC | V_CCMD_REG_CIRG_GLOBAL);
- MmioWrite64 (mVtdUnitInformation[Index].VtdUnitBaseAddress + R_CCMD_REG, Reg64);
-
- DEBUG((DEBUG_INFO, "EnableDmar: Waiting B_CCMD_REG_ICC ...\n"));
- do {
- Reg64 = MmioRead64 (mVtdUnitInformation[Index].VtdUnitBaseAddress + R_CCMD_REG);
- } while ((Reg64 & B_CCMD_REG_ICC) != 0);
+ InvalidateContextCache (Index);
//
// Invalidate the IOTLB cache
//
- DEBUG((DEBUG_INFO, "EnableDmar: IRO 0x%x\n", mVtdUnitInformation[Index].ECapReg.Bits.IRO));
-
- Reg64 = MmioRead64 (mVtdUnitInformation[Index].VtdUnitBaseAddress + (mVtdUnitInformation[Index].ECapReg.Bits.IRO * 16) + R_IOTLB_REG);
- if ((Reg64 & B_IOTLB_REG_IVT) != 0) {
- DEBUG ((DEBUG_INFO,"ERROR: EnableDmar: B_IOTLB_REG_IVT is set for VTD(%d)\n", Index));
- return EFI_DEVICE_ERROR;
- }
-
- Reg64 &= ((~B_IOTLB_REG_IVT) & (~B_IOTLB_REG_IIRG_MASK));
- Reg64 |= (B_IOTLB_REG_IVT | V_IOTLB_REG_IIRG_GLOBAL);
- MmioWrite64 (mVtdUnitInformation[Index].VtdUnitBaseAddress + (mVtdUnitInformation[Index].ECapReg.Bits.IRO * 16) + R_IOTLB_REG, Reg64);
-
- DEBUG((DEBUG_INFO, "EnableDmar: Waiting B_IOTLB_REG_IVT ...\n"));
- do {
- Reg64 = MmioRead64 (mVtdUnitInformation[Index].VtdUnitBaseAddress + (mVtdUnitInformation[Index].ECapReg.Bits.IRO * 16) + R_IOTLB_REG);
- } while ((Reg64 & B_IOTLB_REG_IVT) != 0);
+ InvalidateIOTLB (Index);
//
// Enable VTd
@@ -371,20 +276,16 @@ DisableDmar (
)
{
UINTN Index;
+ UINTN SubIndex;
UINT32 Reg32;
- AsmWbinvd();
-
for (Index = 0; Index < mVtdUnitNumber; Index++) {
DEBUG((DEBUG_INFO, ">>>>>>DisableDmar() for engine [%d] \n", Index));
//
// Write Buffer Flush before invalidation
//
- Reg32 = MmioRead32 (mVtdUnitInformation[Index].VtdUnitBaseAddress + R_CAP_REG);
- if ((Reg32 & B_CAP_REG_RWBF) != 0) {
- MmioWrite32 (mVtdUnitInformation[Index].VtdUnitBaseAddress + R_GCMD_REG, B_GMCD_REG_WBF);
- }
+ FlushWriteBuffer (Index);
//
// Disable VTd
@@ -402,6 +303,19 @@ DisableDmar (
mVtdEnabled = FALSE;
+ for (Index = 0; Index < mVtdUnitNumber; Index++) {
+ DEBUG((DEBUG_INFO, "engine [%d] access\n", Index));
+ for (SubIndex = 0; SubIndex < mVtdUnitInformation[Index].PciDeviceInfo.PciDescriptorNumber; SubIndex++) {
+ DEBUG ((DEBUG_INFO, " PCI S%04X B%02x D%02x F%02x - %d\n",
+ mVtdUnitInformation[Index].Segment,
+ mVtdUnitInformation[Index].PciDeviceInfo.PciDescriptors[SubIndex].Bits.Bus,
+ mVtdUnitInformation[Index].PciDeviceInfo.PciDescriptors[SubIndex].Bits.Device,
+ mVtdUnitInformation[Index].PciDeviceInfo.PciDescriptors[SubIndex].Bits.Function,
+ mVtdUnitInformation[Index].PciDeviceInfo.AccessCount[SubIndex]
+ ));
+ }
+ }
+
return EFI_SUCCESS;
}
--
2.7.4.windows.1
^ permalink raw reply related [flat|nested] 5+ messages in thread
* Re: [PATCH V2 2/2] IntelSiliconPkg/IntelVTdDxe: Improve performance.
2017-08-11 14:54 ` [PATCH V2 2/2] IntelSiliconPkg/IntelVTdDxe: Improve performance Jiewen Yao
@ 2017-08-14 3:53 ` Zeng, Star
2017-08-14 4:16 ` Yao, Jiewen
0 siblings, 1 reply; 5+ messages in thread
From: Zeng, Star @ 2017-08-14 3:53 UTC (permalink / raw)
To: Yao, Jiewen, edk2-devel@lists.01.org; +Cc: Zeng, Star
Two minor comments, others are good to me. Reviewed-by: Star Zeng <star.zeng@intel.com>
1. In commit log, add WriteBackDataCacheRange() and remove duplicated "to"?
CLFLUSH is used to to only flush the context table or second level page table if they are changed.
->
CLFLUSH by WriteBackDataCacheRange() is used to only flush the context table or second level page table if they are changed.
2. PERF_CODE can be used.
PERF_CODE_BEGIN();
AsciiSPrint (PerfToken, sizeof(PerfToken), "VTD(S%04x.B%02x.D%02x.F%02x)", Segment, SourceId.Bits.Bus, SourceId.Bits.Device, SourceId.Bits.Function);
PERF_START (gImageHandle, PerfToken, "IntelVTD", 0);
PERF_CODE_END();
->
PERF_CODE (
AsciiSPrint (PerfToken, sizeof(PerfToken), "VTD(S%04x.B%02x.D%02x.F%02x)", Segment, SourceId.Bits.Bus, SourceId.Bits.Device, SourceId.Bits.Function);
PERF_START (gImageHandle, PerfToken, "IntelVTD", 0);
);
Thanks,
Star
-----Original Message-----
From: Yao, Jiewen
Sent: Friday, August 11, 2017 10:54 PM
To: edk2-devel@lists.01.org
Cc: Zeng, Star <star.zeng@intel.com>
Subject: [PATCH V2 2/2] IntelSiliconPkg/IntelVTdDxe: Improve performance.
This patch is to improve IOMMU performance.
All WBINVD is removed due to performance issue.
CLFLUSH is used to to only flush the context table or second level page table if they are changed.
This patch also removed some unused functions.
Cc: Star Zeng <star.zeng@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.0
Signed-off-by: Jiewen Yao <jiewen.yao@intel.com>
---
IntelSiliconPkg/IntelVTdDxe/DmaProtection.h | 57 ++---
IntelSiliconPkg/IntelVTdDxe/IntelVTdDxe.c | 10 +
IntelSiliconPkg/IntelVTdDxe/IntelVTdDxe.inf | 3 +
IntelSiliconPkg/IntelVTdDxe/PciInfo.c | 12 +
IntelSiliconPkg/IntelVTdDxe/TranslationTable.c | 58 ++++-
IntelSiliconPkg/IntelVTdDxe/TranslationTableEx.c | 2 +
IntelSiliconPkg/IntelVTdDxe/VtdReg.c | 260 +++++++-------------
7 files changed, 186 insertions(+), 216 deletions(-)
diff --git a/IntelSiliconPkg/IntelVTdDxe/DmaProtection.h b/IntelSiliconPkg/IntelVTdDxe/DmaProtection.h
index 8cfa69c..c3b57a0 100644
--- a/IntelSiliconPkg/IntelVTdDxe/DmaProtection.h
+++ b/IntelSiliconPkg/IntelVTdDxe/DmaProtection.h
@@ -25,6 +25,9 @@
#include <Library/PciSegmentLib.h>
#include <Library/DebugLib.h>
#include <Library/UefiLib.h>
+#include <Library/CacheMaintenanceLib.h> #include
+<Library/PerformanceLib.h> #include <Library/PrintLib.h>
#include <Guid/EventGroup.h>
#include <Guid/Acpi.h>
@@ -58,6 +61,8 @@ typedef struct {
UINTN PciDescriptorMaxNumber;
BOOLEAN *IsRealPciDevice;
VTD_SOURCE_ID *PciDescriptors;
+ // for statistic analysis
+ UINTN *AccessCount;
} PCI_DEVICE_INFORMATION;
typedef struct {
@@ -68,6 +73,7 @@ typedef struct {
VTD_ROOT_ENTRY *RootEntryTable;
VTD_EXT_ROOT_ENTRY *ExtRootEntryTable;
VTD_SECOND_LEVEL_PAGING_ENTRY *FixedSecondLevelPagingEntry;
+ BOOLEAN HasDirtyContext;
BOOLEAN HasDirtyPages;
PCI_DEVICE_INFORMATION PciDeviceInfo;
} VTD_UNIT_INFORMATION;
@@ -125,40 +131,6 @@ DisableDmar (
);
/**
- Invalid VTd IOTLB page.
-
- @param[in] VtdIndex The index of VTd engine.
- @param[in] Address The address of IOTLB page.
- @param[in] AddressMode The address mode of IOTLB page.
- @param[in] DomainIdentifier The domain ID of the source.
-
- @retval EFI_SUCCESS VTd IOTLB page is invalidated.
- @retval EFI_DEVICE_ERROR VTd IOTLB page is not invalidated.
-**/
-EFI_STATUS
-InvalidateVtdIOTLBPage (
- IN UINTN VtdIndex,
- IN UINT64 Address,
- IN UINT8 AddressMode,
- IN UINT16 DomainIdentifier
- );
-
-/**
- Invalid VTd IOTLB domain.
-
- @param[in] VtdIndex The index of VTd engine.
- @param[in] DomainIdentifier The domain ID of the source.
-
- @retval EFI_SUCCESS VTd IOTLB domain is invalidated.
- @retval EFI_DEVICE_ERROR VTd IOTLB domain is not invalidated.
-**/
-EFI_STATUS
-InvalidateVtdIOTLBDomain (
- IN UINTN VtdIndex,
- IN UINT16 DomainIdentifier
- );
-
-/**
Invalid VTd global IOTLB.
@param[in] VtdIndex The index of VTd engine.
@@ -362,6 +334,7 @@ DumpSecondLevelPagingEntry ( EFI_STATUS SetPageAttribute (
IN UINTN VtdIndex,
+ IN UINT16 DomainIdentifier,
IN VTD_SECOND_LEVEL_PAGING_ENTRY *SecondLevelPagingEntry,
IN UINT64 BaseAddress,
IN UINT64 Length,
@@ -500,4 +473,20 @@ AllocateZeroPages (
IN UINTN Pages
);
+/**
+ Flush VTD page table and context table memory.
+
+ This action is to make sure the IOMMU engine can get final data in memory.
+
+ @param[in] VtdIndex The index used to identify a VTd engine.
+ @param[in] Base The base address of memory to be flushed.
+ @param[in] Size The size of memory in bytes to be flushed.
+**/
+VOID
+FlushPageTableMemory (
+ IN UINTN VtdIndex,
+ IN UINTN Base,
+ IN UINTN Size
+ );
+
#endif
diff --git a/IntelSiliconPkg/IntelVTdDxe/IntelVTdDxe.c b/IntelSiliconPkg/IntelVTdDxe/IntelVTdDxe.c
index d22222d..7feaaf5 100644
--- a/IntelSiliconPkg/IntelVTdDxe/IntelVTdDxe.c
+++ b/IntelSiliconPkg/IntelVTdDxe/IntelVTdDxe.c
@@ -227,6 +227,7 @@ VTdSetAttribute (
EFI_STATUS Status;
UINT16 Segment;
VTD_SOURCE_ID SourceId;
+ CHAR8 PerfToken[sizeof("VTD(S0000.B00.D00.F00)")];
DumpVtdIfError ();
@@ -239,8 +240,17 @@ VTdSetAttribute (
DEBUG ((DEBUG_VERBOSE, "PCI(S%x.B%x.D%x.F%x) ", Segment, SourceId.Bits.Bus, SourceId.Bits.Device, SourceId.Bits.Function));
DEBUG ((DEBUG_VERBOSE, "(0x%lx~0x%lx) - %lx\n", DeviceAddress, Length, IoMmuAccess));
+ PERF_CODE_BEGIN();
+ AsciiSPrint (PerfToken, sizeof(PerfToken), "VTD(S%04x.B%02x.D%02x.F%02x)", Segment, SourceId.Bits.Bus, SourceId.Bits.Device, SourceId.Bits.Function);
+ PERF_START (gImageHandle, PerfToken, "IntelVTD", 0);
+ PERF_CODE_END();
+
Status = SetAccessAttribute (Segment, SourceId, DeviceAddress, Length, IoMmuAccess);
+ PERF_CODE_BEGIN();
+ PERF_END (gImageHandle, PerfToken, "IntelVTD", 0);
+ PERF_CODE_END();
+
return Status;
}
diff --git a/IntelSiliconPkg/IntelVTdDxe/IntelVTdDxe.inf b/IntelSiliconPkg/IntelVTdDxe/IntelVTdDxe.inf
index 6a61c13..697932e 100644
--- a/IntelSiliconPkg/IntelVTdDxe/IntelVTdDxe.inf
+++ b/IntelSiliconPkg/IntelVTdDxe/IntelVTdDxe.inf
@@ -57,6 +57,9 @@
BaseMemoryLib
MemoryAllocationLib
UefiLib
+ CacheMaintenanceLib
+ PerformanceLib
+ PrintLib
[Guids]
gEfiEventExitBootServicesGuid ## CONSUMES ## Event
diff --git a/IntelSiliconPkg/IntelVTdDxe/PciInfo.c b/IntelSiliconPkg/IntelVTdDxe/PciInfo.c
index d5f096f..27e253d 100644
--- a/IntelSiliconPkg/IntelVTdDxe/PciInfo.c
+++ b/IntelSiliconPkg/IntelVTdDxe/PciInfo.c
@@ -77,6 +77,7 @@ RegisterPciDevice (
UINTN Index;
BOOLEAN *NewIsRealPciDevice;
VTD_SOURCE_ID *NewPciDescriptors;
+ UINTN *NewAccessCount;
PciDeviceInfo = &mVtdUnitInformation[VtdIndex].PciDeviceInfo;
@@ -112,6 +113,12 @@ RegisterPciDevice (
FreePool (NewIsRealPciDevice);
return EFI_OUT_OF_RESOURCES;
}
+ NewAccessCount = AllocateZeroPool (sizeof(*NewAccessCount) * (PciDeviceInfo->PciDescriptorMaxNumber + MAX_PCI_DESCRIPTORS));
+ if (NewAccessCount == NULL) {
+ FreePool (NewIsRealPciDevice);
+ FreePool (NewPciDescriptors);
+ return EFI_OUT_OF_RESOURCES;
+ }
PciDeviceInfo->PciDescriptorMaxNumber += MAX_PCI_DESCRIPTORS;
if (PciDeviceInfo->IsRealPciDevice != NULL) {
CopyMem (NewIsRealPciDevice, PciDeviceInfo->IsRealPciDevice, sizeof(*NewIsRealPciDevice) * PciDeviceInfo->PciDescriptorNumber);
@@ -123,6 +130,11 @@ RegisterPciDevice (
FreePool (PciDeviceInfo->PciDescriptors);
}
PciDeviceInfo->PciDescriptors = NewPciDescriptors;
+ if (PciDeviceInfo->AccessCount != NULL) {
+ CopyMem (NewAccessCount, PciDeviceInfo->AccessCount, sizeof(*NewAccessCount) * PciDeviceInfo->PciDescriptorNumber);
+ FreePool (PciDeviceInfo->AccessCount);
+ }
+ PciDeviceInfo->AccessCount = NewAccessCount;
}
ASSERT (PciDeviceInfo->PciDescriptorNumber < PciDeviceInfo->PciDescriptorMaxNumber);
diff --git a/IntelSiliconPkg/IntelVTdDxe/TranslationTable.c b/IntelSiliconPkg/IntelVTdDxe/TranslationTable.c
index 961d7ca..80fc823 100644
--- a/IntelSiliconPkg/IntelVTdDxe/TranslationTable.c
+++ b/IntelSiliconPkg/IntelVTdDxe/TranslationTable.c
@@ -124,6 +124,7 @@ CreateContextEntry (
RootEntry->Bits.ContextTablePointerHi = (UINT32) RShiftU64 ((UINT64)(UINTN)Buffer, 32);
RootEntry->Bits.Present = 1;
Buffer = (UINT8 *)Buffer + EFI_PAGES_TO_SIZE (ContextPages);
+ FlushPageTableMemory (VtdIndex, (UINTN)RootEntry,
+ sizeof(*RootEntry));
}
ContextEntryTable = (VTD_CONTEXT_ENTRY *)(UINTN)VTD_64BITS_ADDRESS(RootEntry->Bits.ContextTablePointerLo, RootEntry->Bits.ContextTablePointerHi) ; @@ -142,6 +143,7 @@ CreateContextEntry (
ContextEntry->Bits.AddressWidth = 0x2;
break;
}
+ FlushPageTableMemory (VtdIndex, (UINTN)ContextEntry,
+ sizeof(*ContextEntry));
}
return EFI_SUCCESS;
@@ -250,8 +252,11 @@ CreateSecondLevelPagingEntryTable (
goto Done;
}
}
+ FlushPageTableMemory (VtdIndex, (UINTN)Lvl2PtEntry, SIZE_4KB);
}
+ FlushPageTableMemory (VtdIndex, (UINTN)&Lvl3PtEntry[Lvl3Start],
+ (UINTN)&Lvl3PtEntry[Lvl3End + 1] - (UINTN)&Lvl3PtEntry[Lvl3Start]);
}
+ FlushPageTableMemory (VtdIndex, (UINTN)&Lvl4PtEntry[Lvl4Start],
+ (UINTN)&Lvl4PtEntry[Lvl4End + 1] - (UINTN)&Lvl4PtEntry[Lvl4Start]);
Done:
return SecondLevelPagingEntry;
@@ -429,9 +434,10 @@ InvalidatePageEntry (
IN UINTN VtdIndex
)
{
- if (mVtdUnitInformation[VtdIndex].HasDirtyPages) {
+ if (mVtdUnitInformation[VtdIndex].HasDirtyContext ||
+ mVtdUnitInformation[VtdIndex].HasDirtyPages) {
InvalidateVtdIOTLBGlobal (VtdIndex);
}
+ mVtdUnitInformation[VtdIndex].HasDirtyContext = FALSE;
mVtdUnitInformation[VtdIndex].HasDirtyPages = FALSE; }
@@ -498,6 +504,7 @@ PageAttributeToLength (
/**
Return page table entry to match the address.
+ @param[in] VtdIndex The index used to identify a VTd engine.
@param[in] SecondLevelPagingEntry The second level paging entry in VTd table for the device.
@param[in] Address The address to be checked.
@param[out] PageAttributes The page attribute of the page entry.
@@ -506,6 +513,7 @@ PageAttributeToLength ( **/ VOID * GetSecondLevelPageTableEntry (
+ IN UINTN VtdIndex,
IN VTD_SECOND_LEVEL_PAGING_ENTRY *SecondLevelPagingEntry,
IN PHYSICAL_ADDRESS Address,
OUT PAGE_ATTRIBUTE *PageAttribute
@@ -535,6 +543,7 @@ GetSecondLevelPageTableEntry (
return NULL;
}
SetSecondLevelPagingEntryAttribute ((VTD_SECOND_LEVEL_PAGING_ENTRY *)&L4PageTable[Index4], EDKII_IOMMU_ACCESS_READ | EDKII_IOMMU_ACCESS_WRITE);
+ FlushPageTableMemory (VtdIndex, (UINTN)&L4PageTable[Index4],
+ sizeof(L4PageTable[Index4]));
}
L3PageTable = (UINT64 *)(UINTN)(L4PageTable[Index4] & PAGING_4K_ADDRESS_MASK_64); @@ -547,6 +556,7 @@ GetSecondLevelPageTableEntry (
return NULL;
}
SetSecondLevelPagingEntryAttribute ((VTD_SECOND_LEVEL_PAGING_ENTRY *)&L3PageTable[Index3], EDKII_IOMMU_ACCESS_READ | EDKII_IOMMU_ACCESS_WRITE);
+ FlushPageTableMemory (VtdIndex, (UINTN)&L3PageTable[Index3],
+ sizeof(L3PageTable[Index3]));
}
if ((L3PageTable[Index3] & VTD_PG_PS) != 0) {
// 1G
@@ -559,6 +569,7 @@ GetSecondLevelPageTableEntry (
L2PageTable[Index2] = Address & PAGING_2M_ADDRESS_MASK_64;
SetSecondLevelPagingEntryAttribute ((VTD_SECOND_LEVEL_PAGING_ENTRY *)&L2PageTable[Index2], 0);
L2PageTable[Index2] |= VTD_PG_PS;
+ FlushPageTableMemory (VtdIndex, (UINTN)&L2PageTable[Index2],
+ sizeof(L2PageTable[Index2]));
}
if ((L2PageTable[Index2] & VTD_PG_PS) != 0) {
// 2M
@@ -579,12 +590,14 @@ GetSecondLevelPageTableEntry (
/**
Modify memory attributes of page entry.
+ @param[in] VtdIndex The index used to identify a VTd engine.
@param[in] PageEntry The page entry.
@param[in] IoMmuAccess The IOMMU access.
@param[out] IsModified TRUE means page table modified. FALSE means page table not modified.
**/
VOID
ConvertSecondLevelPageEntryAttribute (
+ IN UINTN VtdIndex,
IN VTD_SECOND_LEVEL_PAGING_ENTRY *PageEntry,
IN UINT64 IoMmuAccess,
OUT BOOLEAN *IsModified
@@ -595,6 +608,7 @@ ConvertSecondLevelPageEntryAttribute (
CurrentPageEntry = PageEntry->Uint64;
SetSecondLevelPagingEntryAttribute (PageEntry, IoMmuAccess);
+ FlushPageTableMemory (VtdIndex, (UINTN)PageEntry,
+ sizeof(*PageEntry));
NewPageEntry = PageEntry->Uint64;
if (CurrentPageEntry != NewPageEntry) {
*IsModified = TRUE;
@@ -639,6 +653,7 @@ NeedSplitPage (
/**
This function splits one page entry to small page entries.
+ @param[in] VtdIndex The index used to identify a VTd engine.
@param[in] PageEntry The page entry to be splitted.
@param[in] PageAttribute The page attribute of the page entry.
@param[in] SplitAttribute How to split the page entry.
@@ -649,6 +664,7 @@ NeedSplitPage (
**/
RETURN_STATUS
SplitSecondLevelPage (
+ IN UINTN VtdIndex,
IN VTD_SECOND_LEVEL_PAGING_ENTRY *PageEntry,
IN PAGE_ATTRIBUTE PageAttribute,
IN PAGE_ATTRIBUTE SplitAttribute
@@ -675,8 +691,11 @@ SplitSecondLevelPage (
for (Index = 0; Index < SIZE_4KB / sizeof(UINT64); Index++) {
NewPageEntry[Index] = (BaseAddress + SIZE_4KB * Index) | (PageEntry->Uint64 & PAGE_PROGATE_BITS);
}
+ FlushPageTableMemory (VtdIndex, (UINTN)NewPageEntry, SIZE_4KB);
+
PageEntry->Uint64 = (UINT64)(UINTN)NewPageEntry;
SetSecondLevelPagingEntryAttribute (PageEntry, EDKII_IOMMU_ACCESS_READ | EDKII_IOMMU_ACCESS_WRITE);
+ FlushPageTableMemory (VtdIndex, (UINTN)PageEntry,
+ sizeof(*PageEntry));
return RETURN_SUCCESS;
} else {
return RETURN_UNSUPPORTED;
@@ -697,8 +716,11 @@ SplitSecondLevelPage (
for (Index = 0; Index < SIZE_4KB / sizeof(UINT64); Index++) {
NewPageEntry[Index] = (BaseAddress + SIZE_2MB * Index) | VTD_PG_PS | (PageEntry->Uint64 & PAGE_PROGATE_BITS);
}
+ FlushPageTableMemory (VtdIndex, (UINTN)NewPageEntry, SIZE_4KB);
+
PageEntry->Uint64 = (UINT64)(UINTN)NewPageEntry;
SetSecondLevelPagingEntryAttribute (PageEntry, EDKII_IOMMU_ACCESS_READ | EDKII_IOMMU_ACCESS_WRITE);
+ FlushPageTableMemory (VtdIndex, (UINTN)PageEntry,
+ sizeof(*PageEntry));
return RETURN_SUCCESS;
} else {
return RETURN_UNSUPPORTED;
@@ -730,6 +752,7 @@ SplitSecondLevelPage ( EFI_STATUS SetSecondLevelPagingAttribute (
IN UINTN VtdIndex,
+ IN UINT16 DomainIdentifier,
IN VTD_SECOND_LEVEL_PAGING_ENTRY *SecondLevelPagingEntry,
IN UINT64 BaseAddress,
IN UINT64 Length,
@@ -756,7 +779,7 @@ SetSecondLevelPagingAttribute (
}
while (Length != 0) {
- PageEntry = GetSecondLevelPageTableEntry (SecondLevelPagingEntry, BaseAddress, &PageAttribute);
+ PageEntry = GetSecondLevelPageTableEntry (VtdIndex,
+ SecondLevelPagingEntry, BaseAddress, &PageAttribute);
if (PageEntry == NULL) {
DEBUG ((DEBUG_ERROR, "PageEntry - NULL\n"));
return RETURN_UNSUPPORTED;
@@ -764,7 +787,7 @@ SetSecondLevelPagingAttribute (
PageEntryLength = PageAttributeToLength (PageAttribute);
SplitAttribute = NeedSplitPage (BaseAddress, Length, PageAttribute);
if (SplitAttribute == PageNone) {
- ConvertSecondLevelPageEntryAttribute (PageEntry, IoMmuAccess, &IsEntryModified);
+ ConvertSecondLevelPageEntryAttribute (VtdIndex, PageEntry,
+ IoMmuAccess, &IsEntryModified);
if (IsEntryModified) {
mVtdUnitInformation[VtdIndex].HasDirtyPages = TRUE;
}
@@ -774,7 +797,7 @@ SetSecondLevelPagingAttribute (
BaseAddress += PageEntryLength;
Length -= PageEntryLength;
} else {
- Status = SplitSecondLevelPage (PageEntry, PageAttribute, SplitAttribute);
+ Status = SplitSecondLevelPage (VtdIndex, PageEntry,
+ PageAttribute, SplitAttribute);
if (RETURN_ERROR (Status)) {
DEBUG ((DEBUG_ERROR, "SplitSecondLevelPage - %r\n", Status));
return RETURN_UNSUPPORTED;
@@ -787,8 +810,6 @@ SetSecondLevelPagingAttribute (
}
}
- InvalidatePageEntry (VtdIndex);
-
return EFI_SUCCESS;
}
@@ -814,6 +835,7 @@ SetSecondLevelPagingAttribute ( EFI_STATUS SetPageAttribute (
IN UINTN VtdIndex,
+ IN UINT16 DomainIdentifier,
IN VTD_SECOND_LEVEL_PAGING_ENTRY *SecondLevelPagingEntry,
IN UINT64 BaseAddress,
IN UINT64 Length,
@@ -823,7 +845,7 @@ SetPageAttribute (
EFI_STATUS Status;
Status = EFI_NOT_FOUND;
if (SecondLevelPagingEntry != NULL) {
- Status = SetSecondLevelPagingAttribute (VtdIndex, SecondLevelPagingEntry, BaseAddress, Length, IoMmuAccess);
+ Status = SetSecondLevelPagingAttribute (VtdIndex, DomainIdentifier,
+ SecondLevelPagingEntry, BaseAddress, Length, IoMmuAccess);
}
return Status;
}
@@ -862,6 +884,8 @@ SetAccessAttribute (
VTD_CONTEXT_ENTRY *ContextEntry;
VTD_SECOND_LEVEL_PAGING_ENTRY *SecondLevelPagingEntry;
UINT64 Pt;
+ UINTN PciDescriptorIndex;
+ UINT16 DomainIdentifier;
SecondLevelPagingEntry = NULL;
@@ -873,6 +897,13 @@ SetAccessAttribute (
return EFI_DEVICE_ERROR;
}
+ PciDescriptorIndex = GetPciDescriptor (VtdIndex, Segment, SourceId);
+ mVtdUnitInformation[VtdIndex].PciDeviceInfo.AccessCount[PciDescriptorI
+ ndex]++;
+ //
+ // DomainId should not be 0.
+ //
+ DomainIdentifier = (UINT16)(PciDescriptorIndex + 1);
+
if (ExtContextEntry != NULL) {
if (ExtContextEntry->Bits.Present == 0) {
SecondLevelPagingEntry = CreateSecondLevelPagingEntry (VtdIndex, 0); @@ -881,9 +912,11 @@ SetAccessAttribute (
ExtContextEntry->Bits.SecondLevelPageTranslationPointerLo = (UINT32) Pt;
ExtContextEntry->Bits.SecondLevelPageTranslationPointerHi = (UINT32) RShiftU64(Pt, 20);
- ExtContextEntry->Bits.DomainIdentifier = (UINT16) GetPciDescriptor (VtdIndex, Segment, SourceId);
+ ExtContextEntry->Bits.DomainIdentifier = DomainIdentifier;
ExtContextEntry->Bits.Present = 1;
+ FlushPageTableMemory (VtdIndex, (UINTN)ExtContextEntry,
+ sizeof(*ExtContextEntry));
DumpDmarExtContextEntryTable (mVtdUnitInformation[VtdIndex].ExtRootEntryTable);
+ mVtdUnitInformation[VtdIndex].HasDirtyContext = TRUE;
} else {
SecondLevelPagingEntry = (VOID *)(UINTN)VTD_64BITS_ADDRESS(ExtContextEntry->Bits.SecondLevelPageTranslationPointerLo, ExtContextEntry->Bits.SecondLevelPageTranslationPointerHi);
DEBUG ((DEBUG_VERBOSE,"SecondLevelPagingEntry - 0x%x (S%04x B%02x D%02x F%02x)\n", SecondLevelPagingEntry, Segment, SourceId.Bits.Bus, SourceId.Bits.Device, SourceId.Bits.Function)); @@ -896,9 +929,11 @@ SetAccessAttribute (
ContextEntry->Bits.SecondLevelPageTranslationPointerLo = (UINT32) Pt;
ContextEntry->Bits.SecondLevelPageTranslationPointerHi = (UINT32) RShiftU64(Pt, 20);
- ContextEntry->Bits.DomainIdentifier = (UINT16) GetPciDescriptor (VtdIndex, Segment, SourceId);
+ ContextEntry->Bits.DomainIdentifier = DomainIdentifier;
ContextEntry->Bits.Present = 1;
+ FlushPageTableMemory (VtdIndex, (UINTN)ContextEntry,
+ sizeof(*ContextEntry));
DumpDmarContextEntryTable (mVtdUnitInformation[VtdIndex].RootEntryTable);
+ mVtdUnitInformation[VtdIndex].HasDirtyContext = TRUE;
} else {
SecondLevelPagingEntry = (VOID *)(UINTN)VTD_64BITS_ADDRESS(ContextEntry->Bits.SecondLevelPageTranslationPointerLo, ContextEntry->Bits.SecondLevelPageTranslationPointerHi);
DEBUG ((DEBUG_VERBOSE,"SecondLevelPagingEntry - 0x%x (S%04x B%02x D%02x F%02x)\n", SecondLevelPagingEntry, Segment, SourceId.Bits.Bus, SourceId.Bits.Device, SourceId.Bits.Function)); @@ -911,6 +946,7 @@ SetAccessAttribute (
if (SecondLevelPagingEntry != mVtdUnitInformation[VtdIndex].FixedSecondLevelPagingEntry) {
Status = SetPageAttribute (
VtdIndex,
+ DomainIdentifier,
SecondLevelPagingEntry,
BaseAddress,
Length,
@@ -922,6 +958,8 @@ SetAccessAttribute (
}
}
+ InvalidatePageEntry (VtdIndex);
+
return EFI_SUCCESS;
}
@@ -965,11 +1003,13 @@ AlwaysEnablePageAttribute (
ExtContextEntry->Bits.SecondLevelPageTranslationPointerHi = (UINT32) RShiftU64(Pt, 20);
ExtContextEntry->Bits.DomainIdentifier = ((1 << (UINT8)((UINTN)mVtdUnitInformation[VtdIndex].CapReg.Bits.ND * 2 + 4)) - 1);
ExtContextEntry->Bits.Present = 1;
+ FlushPageTableMemory (VtdIndex, (UINTN)ExtContextEntry,
+ sizeof(*ExtContextEntry));
} else if (ContextEntry != NULL) {
ContextEntry->Bits.SecondLevelPageTranslationPointerLo = (UINT32) Pt;
ContextEntry->Bits.SecondLevelPageTranslationPointerHi = (UINT32) RShiftU64(Pt, 20);
ContextEntry->Bits.DomainIdentifier = ((1 << (UINT8)((UINTN)mVtdUnitInformation[VtdIndex].CapReg.Bits.ND * 2 + 4)) - 1);
ContextEntry->Bits.Present = 1;
+ FlushPageTableMemory (VtdIndex, (UINTN)ContextEntry,
+ sizeof(*ContextEntry));
}
return EFI_SUCCESS;
diff --git a/IntelSiliconPkg/IntelVTdDxe/TranslationTableEx.c b/IntelSiliconPkg/IntelVTdDxe/TranslationTableEx.c
index 65ed16e..9d4e6ea 100644
--- a/IntelSiliconPkg/IntelVTdDxe/TranslationTableEx.c
+++ b/IntelSiliconPkg/IntelVTdDxe/TranslationTableEx.c
@@ -73,6 +73,7 @@ CreateExtContextEntry (
ExtRootEntry->Bits.UpperContextTablePointerLo = (UINT32) RShiftU64 ((UINT64)(UINTN)Buffer, 12) + 1;
ExtRootEntry->Bits.UpperContextTablePointerHi = (UINT32) RShiftU64 (RShiftU64 ((UINT64)(UINTN)Buffer, 12) + 1, 20);
ExtRootEntry->Bits.UpperPresent = 1;
+ FlushPageTableMemory (VtdIndex, (UINTN)ExtRootEntry,
+ sizeof(*ExtRootEntry));
Buffer = (UINT8 *)Buffer + EFI_PAGES_TO_SIZE (ContextPages);
}
@@ -92,6 +93,7 @@ CreateExtContextEntry (
ExtContextEntry->Bits.AddressWidth = 0x2;
break;
}
+ FlushPageTableMemory (VtdIndex, (UINTN)ExtContextEntry,
+ sizeof(*ExtContextEntry));
}
return EFI_SUCCESS;
diff --git a/IntelSiliconPkg/IntelVTdDxe/VtdReg.c b/IntelSiliconPkg/IntelVTdDxe/VtdReg.c
index f36e3de..b1178b7 100644
--- a/IntelSiliconPkg/IntelVTdDxe/VtdReg.c
+++ b/IntelSiliconPkg/IntelVTdDxe/VtdReg.c
@@ -20,43 +20,62 @@ VTD_UNIT_INFORMATION *mVtdUnitInformation;
BOOLEAN mVtdEnabled;
/**
- Invalid VTd global IOTLB.
+ Flush VTD page table and context table memory.
- @param[in] VtdIndex The index of VTd engine.
+ This action is to make sure the IOMMU engine can get final data in memory.
- @retval EFI_SUCCESS VTd global IOTLB is invalidated.
- @retval EFI_DEVICE_ERROR VTd global IOTLB is not invalidated.
+ @param[in] VtdIndex The index used to identify a VTd engine.
+ @param[in] Base The base address of memory to be flushed.
+ @param[in] Size The size of memory in bytes to be flushed.
**/
-EFI_STATUS
-InvalidateVtdIOTLBGlobal (
+VOID
+FlushPageTableMemory (
+ IN UINTN VtdIndex,
+ IN UINTN Base,
+ IN UINTN Size
+ )
+{
+ if (mVtdUnitInformation[VtdIndex].ECapReg.Bits.C == 0) {
+ WriteBackDataCacheRange ((VOID *)Base, Size);
+ }
+}
+
+/**
+ Flush VTd engine write buffer.
+
+ @param[in] VtdIndex The index used to identify a VTd engine.
+**/
+VOID
+FlushWriteBuffer (
IN UINTN VtdIndex
)
{
- UINT64 Reg64;
UINT32 Reg32;
- if (!mVtdEnabled) {
- return EFI_SUCCESS;
+ if (mVtdUnitInformation[VtdIndex].CapReg.Bits.RWBF != 0) {
+ Reg32 = MmioRead32 (mVtdUnitInformation[VtdIndex].VtdUnitBaseAddress + R_GSTS_REG);
+ MmioWrite32 (mVtdUnitInformation[VtdIndex].VtdUnitBaseAddress + R_GCMD_REG, Reg32 | B_GMCD_REG_WBF);
+ do {
+ Reg32 = MmioRead32 (mVtdUnitInformation[VtdIndex].VtdUnitBaseAddress + R_GSTS_REG);
+ } while ((Reg32 & B_GSTS_REG_WBF) != 0);
}
+}
- DEBUG((DEBUG_VERBOSE, "InvalidateVtdIOTLBGlobal(%d)\n", VtdIndex));
-
- AsmWbinvd();
+/**
+ Invalidate VTd context cache.
- //
- // Write Buffer Flush before invalidation
- //
- Reg32 = MmioRead32 (mVtdUnitInformation[VtdIndex].VtdUnitBaseAddress + R_CAP_REG);
- if ((Reg32 & B_CAP_REG_RWBF) != 0) {
- MmioWrite32 (mVtdUnitInformation[VtdIndex].VtdUnitBaseAddress + R_GCMD_REG, B_GMCD_REG_WBF);
- }
+ @param[in] VtdIndex The index used to identify a VTd engine.
+**/
+EFI_STATUS
+InvalidateContextCache (
+ IN UINTN VtdIndex
+ )
+{
+ UINT64 Reg64;
- //
- // Invalidate the context cache
- //
Reg64 = MmioRead64 (mVtdUnitInformation[VtdIndex].VtdUnitBaseAddress + R_CCMD_REG);
if ((Reg64 & B_CCMD_REG_ICC) != 0) {
- DEBUG ((DEBUG_ERROR,"ERROR: InvalidateVtdIOTLBGlobal: B_CCMD_REG_ICC is set for VTD(%d)\n",VtdIndex));
+ DEBUG ((DEBUG_ERROR,"ERROR: InvalidateContextCache: B_CCMD_REG_ICC
+ is set for VTD(%d)\n",VtdIndex));
return EFI_DEVICE_ERROR;
}
@@ -68,97 +87,29 @@ InvalidateVtdIOTLBGlobal (
Reg64 = MmioRead64 (mVtdUnitInformation[VtdIndex].VtdUnitBaseAddress + R_CCMD_REG);
} while ((Reg64 & B_CCMD_REG_ICC) != 0);
- //
- // Invalidate the IOTLB cache
- //
-
- Reg64 = MmioRead64 (mVtdUnitInformation[VtdIndex].VtdUnitBaseAddress + (mVtdUnitInformation[VtdIndex].ECapReg.Bits.IRO * 16) + R_IOTLB_REG);
- if ((Reg64 & B_IOTLB_REG_IVT) != 0) {
- DEBUG ((DEBUG_ERROR,"ERROR: InvalidateVtdIOTLBGlobal: B_IOTLB_REG_IVT is set for VTD(%d)\n", VtdIndex));
- return EFI_DEVICE_ERROR;
- }
-
- Reg64 &= ((~B_IOTLB_REG_IVT) & (~B_IOTLB_REG_IIRG_MASK));
- Reg64 |= (B_IOTLB_REG_IVT | V_IOTLB_REG_IIRG_GLOBAL);
- MmioWrite64 (mVtdUnitInformation[VtdIndex].VtdUnitBaseAddress + (mVtdUnitInformation[VtdIndex].ECapReg.Bits.IRO * 16) + R_IOTLB_REG, Reg64);
-
- do {
- Reg64 = MmioRead64 (mVtdUnitInformation[VtdIndex].VtdUnitBaseAddress + (mVtdUnitInformation[VtdIndex].ECapReg.Bits.IRO * 16) + R_IOTLB_REG);
- } while ((Reg64 & B_IOTLB_REG_IVT) != 0);
-
- //
- // Disable VTd
- //
- MmioWrite32 (mVtdUnitInformation[VtdIndex].VtdUnitBaseAddress + R_GCMD_REG, B_GMCD_REG_SRTP);
- do {
- Reg32 = MmioRead32 (mVtdUnitInformation[VtdIndex].VtdUnitBaseAddress + R_GSTS_REG);
- } while((Reg32 & B_GSTS_REG_RTPS) == 0);
-
- //
- // Enable VTd
- //
- MmioWrite32 (mVtdUnitInformation[VtdIndex].VtdUnitBaseAddress + R_GCMD_REG, B_GMCD_REG_TE);
- do {
- Reg32 = MmioRead32 (mVtdUnitInformation[VtdIndex].VtdUnitBaseAddress + R_GSTS_REG);
- } while ((Reg32 & B_GSTS_REG_TE) == 0);
-
return EFI_SUCCESS;
}
/**
- Invalid VTd IOTLB domain.
-
- @param[in] VtdIndex The index of VTd engine.
- @param[in] DomainIdentifier The domain ID of the source.
+ Invalidate VTd IOTLB.
- @retval EFI_SUCCESS VTd IOTLB domain is invalidated.
- @retval EFI_DEVICE_ERROR VTd IOTLB domain is not invalidated.
+ @param[in] VtdIndex The index used to identify a VTd engine.
**/
EFI_STATUS
-InvalidateVtdIOTLBDomain (
- IN UINTN VtdIndex,
- IN UINT16 DomainIdentifier
+InvalidateIOTLB (
+ IN UINTN VtdIndex
)
{
UINT64 Reg64;
- if (!mVtdEnabled) {
- return EFI_SUCCESS;
- }
-
- DEBUG((DEBUG_VERBOSE, "InvalidateVtdIOTLBDomain(%d): 0x%016lx (0x%04x)\n", VtdIndex, DomainIdentifier));
-
- //
- // Invalidate the context cache
- //
- Reg64 = MmioRead64 (mVtdUnitInformation[VtdIndex].VtdUnitBaseAddress + R_CCMD_REG);
- if ((Reg64 & B_CCMD_REG_ICC) != 0) {
- DEBUG ((DEBUG_ERROR,"ERROR: InvalidateVtdIOTLBDomain: B_CCMD_REG_ICC is set for VTD(%d)\n",VtdIndex));
- return EFI_DEVICE_ERROR;
- }
-
- Reg64 &= ((~B_CCMD_REG_ICC) & (~B_CCMD_REG_CIRG_MASK));
- Reg64 |= (B_CCMD_REG_ICC | V_CCMD_REG_CIRG_DOMAIN);
- Reg64 |= (B_CCMD_REG_ICC | V_CCMD_REG_CIRG_DOMAIN);
- MmioWrite64 (mVtdUnitInformation[VtdIndex].VtdUnitBaseAddress + R_CCMD_REG, Reg64);
-
- do {
- Reg64 = MmioRead64 (mVtdUnitInformation[VtdIndex].VtdUnitBaseAddress + R_CCMD_REG);
- } while ((Reg64 & B_CCMD_REG_ICC) != 0);
-
- //
- // Invalidate the IOTLB cache
- //
-
Reg64 = MmioRead64 (mVtdUnitInformation[VtdIndex].VtdUnitBaseAddress + (mVtdUnitInformation[VtdIndex].ECapReg.Bits.IRO * 16) + R_IOTLB_REG);
if ((Reg64 & B_IOTLB_REG_IVT) != 0) {
- DEBUG ((DEBUG_ERROR,"ERROR: InvalidateVtdIOTLBDomain: B_IOTLB_REG_IVT is set for VTD(%d)\n", VtdIndex));
+ DEBUG ((DEBUG_ERROR,"ERROR: InvalidateIOTLB: B_IOTLB_REG_IVT is set
+ for VTD(%d)\n", VtdIndex));
return EFI_DEVICE_ERROR;
}
Reg64 &= ((~B_IOTLB_REG_IVT) & (~B_IOTLB_REG_IIRG_MASK));
- Reg64 |= (B_IOTLB_REG_IVT | V_IOTLB_REG_IIRG_DOMAIN);
- Reg64 |= LShiftU64 (DomainIdentifier, 32);
+ Reg64 |= (B_IOTLB_REG_IVT | V_IOTLB_REG_IIRG_GLOBAL);
MmioWrite64 (mVtdUnitInformation[VtdIndex].VtdUnitBaseAddress + (mVtdUnitInformation[VtdIndex].ECapReg.Bits.IRO * 16) + R_IOTLB_REG, Reg64);
do {
@@ -169,53 +120,41 @@ InvalidateVtdIOTLBDomain ( }
/**
- Invalid VTd IOTLB page.
+ Invalid VTd global IOTLB.
@param[in] VtdIndex The index of VTd engine.
- @param[in] Address The address of IOTLB page.
- @param[in] AddressMode The address mode of IOTLB page.
- @param[in] DomainIdentifier The domain ID of the source.
- @retval EFI_SUCCESS VTd IOTLB page is invalidated.
- @retval EFI_DEVICE_ERROR VTd IOTLB page is not invalidated.
+ @retval EFI_SUCCESS VTd global IOTLB is invalidated.
+ @retval EFI_DEVICE_ERROR VTd global IOTLB is not invalidated.
**/
EFI_STATUS
-InvalidateVtdIOTLBPage (
- IN UINTN VtdIndex,
- IN UINT64 Address,
- IN UINT8 AddressMode,
- IN UINT16 DomainIdentifier
+InvalidateVtdIOTLBGlobal (
+ IN UINTN VtdIndex
)
{
- UINT64 Reg64;
- UINT64 Data64;
-
if (!mVtdEnabled) {
return EFI_SUCCESS;
}
- DEBUG((DEBUG_VERBOSE, "InvalidateVtdIOTLBPage(%d): 0x%016lx (0x%02x)\n", VtdIndex, Address, AddressMode));
-
- if (mVtdUnitInformation[VtdIndex].CapReg.Bits.PSI != 0) {
- Reg64 = MmioRead64 (mVtdUnitInformation[VtdIndex].VtdUnitBaseAddress + (mVtdUnitInformation[VtdIndex].ECapReg.Bits.IRO * 16) + R_IOTLB_REG);
- if ((Reg64 & B_IOTLB_REG_IVT) != 0) {
- DEBUG ((DEBUG_ERROR,"ERROR: InvalidateVtdIOTLBPage: B_IOTLB_REG_IVT is set for VTD(%d)\n", VtdIndex));
- return EFI_DEVICE_ERROR;
- }
+ DEBUG((DEBUG_VERBOSE, "InvalidateVtdIOTLBGlobal(%d)\n", VtdIndex));
- Data64 = Address | AddressMode;
- MmioWrite64 (mVtdUnitInformation[VtdIndex].VtdUnitBaseAddress + (mVtdUnitInformation[VtdIndex].ECapReg.Bits.IRO * 16) + R_IVA_REG, Data64);
+ //
+ // Write Buffer Flush before invalidation // FlushWriteBuffer
+ (VtdIndex);
- Reg64 &= ((~B_IOTLB_REG_IVT) & (~B_IOTLB_REG_IIRG_MASK));
- Reg64 |= (B_IOTLB_REG_IVT | V_IOTLB_REG_IIRG_PAGE);
- Reg64 |= LShiftU64 (DomainIdentifier, 32);
- MmioWrite64 (mVtdUnitInformation[VtdIndex].VtdUnitBaseAddress + (mVtdUnitInformation[VtdIndex].ECapReg.Bits.IRO * 16) + R_IOTLB_REG, Reg64);
+ //
+ // Invalidate the context cache
+ //
+ if (mVtdUnitInformation[VtdIndex].HasDirtyContext) {
+ InvalidateContextCache (VtdIndex);
+ }
- do {
- Reg64 = MmioRead64 (mVtdUnitInformation[VtdIndex].VtdUnitBaseAddress + (mVtdUnitInformation[VtdIndex].ECapReg.Bits.IRO * 16) + R_IOTLB_REG);
- } while ((Reg64 & B_IOTLB_REG_IVT) != 0);
- } else {
- InvalidateVtdIOTLBGlobal (VtdIndex);
+ //
+ // Invalidate the IOTLB cache
+ //
+ if (mVtdUnitInformation[VtdIndex].HasDirtyContext || mVtdUnitInformation[VtdIndex].HasDirtyPages) {
+ InvalidateIOTLB (VtdIndex);
}
return EFI_SUCCESS;
@@ -268,11 +207,8 @@ EnableDmar (
)
{
UINTN Index;
- UINT64 Reg64;
UINT32 Reg32;
- AsmWbinvd();
-
for (Index = 0; Index < mVtdUnitNumber; Index++) {
DEBUG((DEBUG_INFO, ">>>>>>EnableDmar() for engine [%d] \n", Index));
@@ -299,48 +235,17 @@ EnableDmar (
//
// Write Buffer Flush before invalidation
//
- Reg32 = MmioRead32 (mVtdUnitInformation[Index].VtdUnitBaseAddress + R_CAP_REG);
- if ((Reg32 & B_CAP_REG_RWBF) != 0) {
- MmioWrite32 (mVtdUnitInformation[Index].VtdUnitBaseAddress + R_GCMD_REG, B_GMCD_REG_WBF);
- }
+ FlushWriteBuffer (Index);
//
// Invalidate the context cache
//
- Reg64 = MmioRead64 (mVtdUnitInformation[Index].VtdUnitBaseAddress + R_CCMD_REG);
- if ((Reg64 & B_CCMD_REG_ICC) != 0) {
- DEBUG ((DEBUG_INFO,"ERROR: EnableDmar: B_CCMD_REG_ICC is set for VTD(%d)\n",Index));
- return EFI_DEVICE_ERROR;
- }
-
- Reg64 &= ((~B_CCMD_REG_ICC) & (~B_CCMD_REG_CIRG_MASK));
- Reg64 |= (B_CCMD_REG_ICC | V_CCMD_REG_CIRG_GLOBAL);
- MmioWrite64 (mVtdUnitInformation[Index].VtdUnitBaseAddress + R_CCMD_REG, Reg64);
-
- DEBUG((DEBUG_INFO, "EnableDmar: Waiting B_CCMD_REG_ICC ...\n"));
- do {
- Reg64 = MmioRead64 (mVtdUnitInformation[Index].VtdUnitBaseAddress + R_CCMD_REG);
- } while ((Reg64 & B_CCMD_REG_ICC) != 0);
+ InvalidateContextCache (Index);
//
// Invalidate the IOTLB cache
//
- DEBUG((DEBUG_INFO, "EnableDmar: IRO 0x%x\n", mVtdUnitInformation[Index].ECapReg.Bits.IRO));
-
- Reg64 = MmioRead64 (mVtdUnitInformation[Index].VtdUnitBaseAddress + (mVtdUnitInformation[Index].ECapReg.Bits.IRO * 16) + R_IOTLB_REG);
- if ((Reg64 & B_IOTLB_REG_IVT) != 0) {
- DEBUG ((DEBUG_INFO,"ERROR: EnableDmar: B_IOTLB_REG_IVT is set for VTD(%d)\n", Index));
- return EFI_DEVICE_ERROR;
- }
-
- Reg64 &= ((~B_IOTLB_REG_IVT) & (~B_IOTLB_REG_IIRG_MASK));
- Reg64 |= (B_IOTLB_REG_IVT | V_IOTLB_REG_IIRG_GLOBAL);
- MmioWrite64 (mVtdUnitInformation[Index].VtdUnitBaseAddress + (mVtdUnitInformation[Index].ECapReg.Bits.IRO * 16) + R_IOTLB_REG, Reg64);
-
- DEBUG((DEBUG_INFO, "EnableDmar: Waiting B_IOTLB_REG_IVT ...\n"));
- do {
- Reg64 = MmioRead64 (mVtdUnitInformation[Index].VtdUnitBaseAddress + (mVtdUnitInformation[Index].ECapReg.Bits.IRO * 16) + R_IOTLB_REG);
- } while ((Reg64 & B_IOTLB_REG_IVT) != 0);
+ InvalidateIOTLB (Index);
//
// Enable VTd
@@ -371,20 +276,16 @@ DisableDmar (
)
{
UINTN Index;
+ UINTN SubIndex;
UINT32 Reg32;
- AsmWbinvd();
-
for (Index = 0; Index < mVtdUnitNumber; Index++) {
DEBUG((DEBUG_INFO, ">>>>>>DisableDmar() for engine [%d] \n", Index));
//
// Write Buffer Flush before invalidation
//
- Reg32 = MmioRead32 (mVtdUnitInformation[Index].VtdUnitBaseAddress + R_CAP_REG);
- if ((Reg32 & B_CAP_REG_RWBF) != 0) {
- MmioWrite32 (mVtdUnitInformation[Index].VtdUnitBaseAddress + R_GCMD_REG, B_GMCD_REG_WBF);
- }
+ FlushWriteBuffer (Index);
//
// Disable VTd
@@ -402,6 +303,19 @@ DisableDmar (
mVtdEnabled = FALSE;
+ for (Index = 0; Index < mVtdUnitNumber; Index++) {
+ DEBUG((DEBUG_INFO, "engine [%d] access\n", Index));
+ for (SubIndex = 0; SubIndex < mVtdUnitInformation[Index].PciDeviceInfo.PciDescriptorNumber; SubIndex++) {
+ DEBUG ((DEBUG_INFO, " PCI S%04X B%02x D%02x F%02x - %d\n",
+ mVtdUnitInformation[Index].Segment,
+ mVtdUnitInformation[Index].PciDeviceInfo.PciDescriptors[SubIndex].Bits.Bus,
+ mVtdUnitInformation[Index].PciDeviceInfo.PciDescriptors[SubIndex].Bits.Device,
+ mVtdUnitInformation[Index].PciDeviceInfo.PciDescriptors[SubIndex].Bits.Function,
+ mVtdUnitInformation[Index].PciDeviceInfo.AccessCount[SubIndex]
+ ));
+ }
+ }
+
return EFI_SUCCESS;
}
--
2.7.4.windows.1
^ permalink raw reply related [flat|nested] 5+ messages in thread
* Re: [PATCH V2 2/2] IntelSiliconPkg/IntelVTdDxe: Improve performance.
2017-08-14 3:53 ` Zeng, Star
@ 2017-08-14 4:16 ` Yao, Jiewen
0 siblings, 0 replies; 5+ messages in thread
From: Yao, Jiewen @ 2017-08-14 4:16 UTC (permalink / raw)
To: Zeng, Star, edk2-devel@lists.01.org
Thanks. I will update that.
> -----Original Message-----
> From: Zeng, Star
> Sent: Monday, August 14, 2017 11:53 AM
> To: Yao, Jiewen <jiewen.yao@intel.com>; edk2-devel@lists.01.org
> Cc: Zeng, Star <star.zeng@intel.com>
> Subject: RE: [PATCH V2 2/2] IntelSiliconPkg/IntelVTdDxe: Improve performance.
>
> Two minor comments, others are good to me. Reviewed-by: Star Zeng
> <star.zeng@intel.com>
>
> 1. In commit log, add WriteBackDataCacheRange() and remove duplicated "to"?
> CLFLUSH is used to to only flush the context table or second level page table if
> they are changed.
> ->
> CLFLUSH by WriteBackDataCacheRange() is used to only flush the context table
> or second level page table if they are changed.
>
> 2. PERF_CODE can be used.
>
> PERF_CODE_BEGIN();
> AsciiSPrint (PerfToken, sizeof(PerfToken),
> "VTD(S%04x.B%02x.D%02x.F%02x)", Segment, SourceId.Bits.Bus,
> SourceId.Bits.Device, SourceId.Bits.Function);
> PERF_START (gImageHandle, PerfToken, "IntelVTD", 0);
> PERF_CODE_END();
>
> ->
>
> PERF_CODE (
> AsciiSPrint (PerfToken, sizeof(PerfToken),
> "VTD(S%04x.B%02x.D%02x.F%02x)", Segment, SourceId.Bits.Bus,
> SourceId.Bits.Device, SourceId.Bits.Function);
> PERF_START (gImageHandle, PerfToken, "IntelVTD", 0);
> );
>
>
> Thanks,
> Star
> -----Original Message-----
> From: Yao, Jiewen
> Sent: Friday, August 11, 2017 10:54 PM
> To: edk2-devel@lists.01.org
> Cc: Zeng, Star <star.zeng@intel.com>
> Subject: [PATCH V2 2/2] IntelSiliconPkg/IntelVTdDxe: Improve performance.
>
> This patch is to improve IOMMU performance.
> All WBINVD is removed due to performance issue.
> CLFLUSH is used to to only flush the context table or second level page table if
> they are changed.
>
> This patch also removed some unused functions.
>
> Cc: Star Zeng <star.zeng@intel.com>
> Contributed-under: TianoCore Contribution Agreement 1.0
> Signed-off-by: Jiewen Yao <jiewen.yao@intel.com>
> ---
> IntelSiliconPkg/IntelVTdDxe/DmaProtection.h | 57 ++---
> IntelSiliconPkg/IntelVTdDxe/IntelVTdDxe.c | 10 +
> IntelSiliconPkg/IntelVTdDxe/IntelVTdDxe.inf | 3 +
> IntelSiliconPkg/IntelVTdDxe/PciInfo.c | 12 +
> IntelSiliconPkg/IntelVTdDxe/TranslationTable.c | 58 ++++-
> IntelSiliconPkg/IntelVTdDxe/TranslationTableEx.c | 2 +
> IntelSiliconPkg/IntelVTdDxe/VtdReg.c | 260 +++++++-------------
> 7 files changed, 186 insertions(+), 216 deletions(-)
>
> diff --git a/IntelSiliconPkg/IntelVTdDxe/DmaProtection.h
> b/IntelSiliconPkg/IntelVTdDxe/DmaProtection.h
> index 8cfa69c..c3b57a0 100644
> --- a/IntelSiliconPkg/IntelVTdDxe/DmaProtection.h
> +++ b/IntelSiliconPkg/IntelVTdDxe/DmaProtection.h
> @@ -25,6 +25,9 @@
> #include <Library/PciSegmentLib.h>
> #include <Library/DebugLib.h>
> #include <Library/UefiLib.h>
> +#include <Library/CacheMaintenanceLib.h> #include
> +<Library/PerformanceLib.h> #include <Library/PrintLib.h>
>
> #include <Guid/EventGroup.h>
> #include <Guid/Acpi.h>
> @@ -58,6 +61,8 @@ typedef struct {
> UINTN PciDescriptorMaxNumber;
> BOOLEAN *IsRealPciDevice;
> VTD_SOURCE_ID *PciDescriptors;
> + // for statistic analysis
> + UINTN *AccessCount;
> } PCI_DEVICE_INFORMATION;
>
> typedef struct {
> @@ -68,6 +73,7 @@ typedef struct {
> VTD_ROOT_ENTRY *RootEntryTable;
> VTD_EXT_ROOT_ENTRY *ExtRootEntryTable;
> VTD_SECOND_LEVEL_PAGING_ENTRY *FixedSecondLevelPagingEntry;
> + BOOLEAN HasDirtyContext;
> BOOLEAN HasDirtyPages;
> PCI_DEVICE_INFORMATION PciDeviceInfo;
> } VTD_UNIT_INFORMATION;
> @@ -125,40 +131,6 @@ DisableDmar (
> );
>
> /**
> - Invalid VTd IOTLB page.
> -
> - @param[in] VtdIndex The index of VTd engine.
> - @param[in] Address The address of IOTLB page.
> - @param[in] AddressMode The address mode of IOTLB page.
> - @param[in] DomainIdentifier The domain ID of the source.
> -
> - @retval EFI_SUCCESS VTd IOTLB page is invalidated.
> - @retval EFI_DEVICE_ERROR VTd IOTLB page is not invalidated.
> -**/
> -EFI_STATUS
> -InvalidateVtdIOTLBPage (
> - IN UINTN VtdIndex,
> - IN UINT64 Address,
> - IN UINT8 AddressMode,
> - IN UINT16 DomainIdentifier
> - );
> -
> -/**
> - Invalid VTd IOTLB domain.
> -
> - @param[in] VtdIndex The index of VTd engine.
> - @param[in] DomainIdentifier The domain ID of the source.
> -
> - @retval EFI_SUCCESS VTd IOTLB domain is invalidated.
> - @retval EFI_DEVICE_ERROR VTd IOTLB domain is not invalidated.
> -**/
> -EFI_STATUS
> -InvalidateVtdIOTLBDomain (
> - IN UINTN VtdIndex,
> - IN UINT16 DomainIdentifier
> - );
> -
> -/**
> Invalid VTd global IOTLB.
>
> @param[in] VtdIndex The index of VTd engine.
> @@ -362,6 +334,7 @@ DumpSecondLevelPagingEntry ( EFI_STATUS
> SetPageAttribute (
> IN UINTN VtdIndex,
> + IN UINT16 DomainIdentifier,
> IN VTD_SECOND_LEVEL_PAGING_ENTRY *SecondLevelPagingEntry,
> IN UINT64 BaseAddress,
> IN UINT64 Length,
> @@ -500,4 +473,20 @@ AllocateZeroPages (
> IN UINTN Pages
> );
>
> +/**
> + Flush VTD page table and context table memory.
> +
> + This action is to make sure the IOMMU engine can get final data in memory.
> +
> + @param[in] VtdIndex The index used to identify a VTd engine.
> + @param[in] Base The base address of memory to be
> flushed.
> + @param[in] Size The size of memory in bytes to be flushed.
> +**/
> +VOID
> +FlushPageTableMemory (
> + IN UINTN VtdIndex,
> + IN UINTN Base,
> + IN UINTN Size
> + );
> +
> #endif
> diff --git a/IntelSiliconPkg/IntelVTdDxe/IntelVTdDxe.c
> b/IntelSiliconPkg/IntelVTdDxe/IntelVTdDxe.c
> index d22222d..7feaaf5 100644
> --- a/IntelSiliconPkg/IntelVTdDxe/IntelVTdDxe.c
> +++ b/IntelSiliconPkg/IntelVTdDxe/IntelVTdDxe.c
> @@ -227,6 +227,7 @@ VTdSetAttribute (
> EFI_STATUS Status;
> UINT16 Segment;
> VTD_SOURCE_ID SourceId;
> + CHAR8 PerfToken[sizeof("VTD(S0000.B00.D00.F00)")];
>
> DumpVtdIfError ();
>
> @@ -239,8 +240,17 @@ VTdSetAttribute (
> DEBUG ((DEBUG_VERBOSE, "PCI(S%x.B%x.D%x.F%x) ", Segment,
> SourceId.Bits.Bus, SourceId.Bits.Device, SourceId.Bits.Function));
> DEBUG ((DEBUG_VERBOSE, "(0x%lx~0x%lx) - %lx\n", DeviceAddress, Length,
> IoMmuAccess));
>
> + PERF_CODE_BEGIN();
> + AsciiSPrint (PerfToken, sizeof(PerfToken),
> "VTD(S%04x.B%02x.D%02x.F%02x)", Segment, SourceId.Bits.Bus,
> SourceId.Bits.Device, SourceId.Bits.Function);
> + PERF_START (gImageHandle, PerfToken, "IntelVTD", 0);
> + PERF_CODE_END();
> +
> Status = SetAccessAttribute (Segment, SourceId, DeviceAddress, Length,
> IoMmuAccess);
>
> + PERF_CODE_BEGIN();
> + PERF_END (gImageHandle, PerfToken, "IntelVTD", 0);
> + PERF_CODE_END();
> +
> return Status;
> }
>
> diff --git a/IntelSiliconPkg/IntelVTdDxe/IntelVTdDxe.inf
> b/IntelSiliconPkg/IntelVTdDxe/IntelVTdDxe.inf
> index 6a61c13..697932e 100644
> --- a/IntelSiliconPkg/IntelVTdDxe/IntelVTdDxe.inf
> +++ b/IntelSiliconPkg/IntelVTdDxe/IntelVTdDxe.inf
> @@ -57,6 +57,9 @@
> BaseMemoryLib
> MemoryAllocationLib
> UefiLib
> + CacheMaintenanceLib
> + PerformanceLib
> + PrintLib
>
> [Guids]
> gEfiEventExitBootServicesGuid ## CONSUMES ## Event
> diff --git a/IntelSiliconPkg/IntelVTdDxe/PciInfo.c
> b/IntelSiliconPkg/IntelVTdDxe/PciInfo.c
> index d5f096f..27e253d 100644
> --- a/IntelSiliconPkg/IntelVTdDxe/PciInfo.c
> +++ b/IntelSiliconPkg/IntelVTdDxe/PciInfo.c
> @@ -77,6 +77,7 @@ RegisterPciDevice (
> UINTN Index;
> BOOLEAN *NewIsRealPciDevice;
> VTD_SOURCE_ID *NewPciDescriptors;
> + UINTN *NewAccessCount;
>
> PciDeviceInfo = &mVtdUnitInformation[VtdIndex].PciDeviceInfo;
>
> @@ -112,6 +113,12 @@ RegisterPciDevice (
> FreePool (NewIsRealPciDevice);
> return EFI_OUT_OF_RESOURCES;
> }
> + NewAccessCount = AllocateZeroPool (sizeof(*NewAccessCount) *
> (PciDeviceInfo->PciDescriptorMaxNumber + MAX_PCI_DESCRIPTORS));
> + if (NewAccessCount == NULL) {
> + FreePool (NewIsRealPciDevice);
> + FreePool (NewPciDescriptors);
> + return EFI_OUT_OF_RESOURCES;
> + }
> PciDeviceInfo->PciDescriptorMaxNumber += MAX_PCI_DESCRIPTORS;
> if (PciDeviceInfo->IsRealPciDevice != NULL) {
> CopyMem (NewIsRealPciDevice, PciDeviceInfo->IsRealPciDevice,
> sizeof(*NewIsRealPciDevice) * PciDeviceInfo->PciDescriptorNumber);
> @@ -123,6 +130,11 @@ RegisterPciDevice (
> FreePool (PciDeviceInfo->PciDescriptors);
> }
> PciDeviceInfo->PciDescriptors = NewPciDescriptors;
> + if (PciDeviceInfo->AccessCount != NULL) {
> + CopyMem (NewAccessCount, PciDeviceInfo->AccessCount,
> sizeof(*NewAccessCount) * PciDeviceInfo->PciDescriptorNumber);
> + FreePool (PciDeviceInfo->AccessCount);
> + }
> + PciDeviceInfo->AccessCount = NewAccessCount;
> }
>
> ASSERT (PciDeviceInfo->PciDescriptorNumber <
> PciDeviceInfo->PciDescriptorMaxNumber);
> diff --git a/IntelSiliconPkg/IntelVTdDxe/TranslationTable.c
> b/IntelSiliconPkg/IntelVTdDxe/TranslationTable.c
> index 961d7ca..80fc823 100644
> --- a/IntelSiliconPkg/IntelVTdDxe/TranslationTable.c
> +++ b/IntelSiliconPkg/IntelVTdDxe/TranslationTable.c
> @@ -124,6 +124,7 @@ CreateContextEntry (
> RootEntry->Bits.ContextTablePointerHi = (UINT32) RShiftU64
> ((UINT64)(UINTN)Buffer, 32);
> RootEntry->Bits.Present = 1;
> Buffer = (UINT8 *)Buffer + EFI_PAGES_TO_SIZE (ContextPages);
> + FlushPageTableMemory (VtdIndex, (UINTN)RootEntry,
> + sizeof(*RootEntry));
> }
>
> ContextEntryTable = (VTD_CONTEXT_ENTRY
> *)(UINTN)VTD_64BITS_ADDRESS(RootEntry->Bits.ContextTablePointerLo,
> RootEntry->Bits.ContextTablePointerHi) ; @@ -142,6 +143,7 @@
> CreateContextEntry (
> ContextEntry->Bits.AddressWidth = 0x2;
> break;
> }
> + FlushPageTableMemory (VtdIndex, (UINTN)ContextEntry,
> + sizeof(*ContextEntry));
> }
>
> return EFI_SUCCESS;
> @@ -250,8 +252,11 @@ CreateSecondLevelPagingEntryTable (
> goto Done;
> }
> }
> + FlushPageTableMemory (VtdIndex, (UINTN)Lvl2PtEntry, SIZE_4KB);
> }
> + FlushPageTableMemory (VtdIndex, (UINTN)&Lvl3PtEntry[Lvl3Start],
> + (UINTN)&Lvl3PtEntry[Lvl3End + 1] - (UINTN)&Lvl3PtEntry[Lvl3Start]);
> }
> + FlushPageTableMemory (VtdIndex, (UINTN)&Lvl4PtEntry[Lvl4Start],
> + (UINTN)&Lvl4PtEntry[Lvl4End + 1] - (UINTN)&Lvl4PtEntry[Lvl4Start]);
>
> Done:
> return SecondLevelPagingEntry;
> @@ -429,9 +434,10 @@ InvalidatePageEntry (
> IN UINTN VtdIndex
> )
> {
> - if (mVtdUnitInformation[VtdIndex].HasDirtyPages) {
> + if (mVtdUnitInformation[VtdIndex].HasDirtyContext ||
> + mVtdUnitInformation[VtdIndex].HasDirtyPages) {
> InvalidateVtdIOTLBGlobal (VtdIndex);
> }
> + mVtdUnitInformation[VtdIndex].HasDirtyContext = FALSE;
> mVtdUnitInformation[VtdIndex].HasDirtyPages = FALSE; }
>
> @@ -498,6 +504,7 @@ PageAttributeToLength (
> /**
> Return page table entry to match the address.
>
> + @param[in] VtdIndex The index used to identify a VTd
> engine.
> @param[in] SecondLevelPagingEntry The second level paging entry in
> VTd table for the device.
> @param[in] Address The address to be checked.
> @param[out] PageAttributes The page attribute of the page
> entry.
> @@ -506,6 +513,7 @@ PageAttributeToLength ( **/ VOID *
> GetSecondLevelPageTableEntry (
> + IN UINTN VtdIndex,
> IN VTD_SECOND_LEVEL_PAGING_ENTRY *SecondLevelPagingEntry,
> IN PHYSICAL_ADDRESS Address,
> OUT PAGE_ATTRIBUTE *PageAttribute
> @@ -535,6 +543,7 @@ GetSecondLevelPageTableEntry (
> return NULL;
> }
> SetSecondLevelPagingEntryAttribute
> ((VTD_SECOND_LEVEL_PAGING_ENTRY *)&L4PageTable[Index4],
> EDKII_IOMMU_ACCESS_READ | EDKII_IOMMU_ACCESS_WRITE);
> + FlushPageTableMemory (VtdIndex, (UINTN)&L4PageTable[Index4],
> + sizeof(L4PageTable[Index4]));
> }
>
> L3PageTable = (UINT64 *)(UINTN)(L4PageTable[Index4] &
> PAGING_4K_ADDRESS_MASK_64); @@ -547,6 +556,7 @@
> GetSecondLevelPageTableEntry (
> return NULL;
> }
> SetSecondLevelPagingEntryAttribute
> ((VTD_SECOND_LEVEL_PAGING_ENTRY *)&L3PageTable[Index3],
> EDKII_IOMMU_ACCESS_READ | EDKII_IOMMU_ACCESS_WRITE);
> + FlushPageTableMemory (VtdIndex, (UINTN)&L3PageTable[Index3],
> + sizeof(L3PageTable[Index3]));
> }
> if ((L3PageTable[Index3] & VTD_PG_PS) != 0) {
> // 1G
> @@ -559,6 +569,7 @@ GetSecondLevelPageTableEntry (
> L2PageTable[Index2] = Address & PAGING_2M_ADDRESS_MASK_64;
> SetSecondLevelPagingEntryAttribute
> ((VTD_SECOND_LEVEL_PAGING_ENTRY *)&L2PageTable[Index2], 0);
> L2PageTable[Index2] |= VTD_PG_PS;
> + FlushPageTableMemory (VtdIndex, (UINTN)&L2PageTable[Index2],
> + sizeof(L2PageTable[Index2]));
> }
> if ((L2PageTable[Index2] & VTD_PG_PS) != 0) {
> // 2M
> @@ -579,12 +590,14 @@ GetSecondLevelPageTableEntry (
> /**
> Modify memory attributes of page entry.
>
> + @param[in] VtdIndex The index used to identify a VTd engine.
> @param[in] PageEntry The page entry.
> @param[in] IoMmuAccess The IOMMU access.
> @param[out] IsModified TRUE means page table modified. FALSE
> means page table not modified.
> **/
> VOID
> ConvertSecondLevelPageEntryAttribute (
> + IN UINTN VtdIndex,
> IN VTD_SECOND_LEVEL_PAGING_ENTRY *PageEntry,
> IN UINT64 IoMmuAccess,
> OUT BOOLEAN *IsModified
> @@ -595,6 +608,7 @@ ConvertSecondLevelPageEntryAttribute (
>
> CurrentPageEntry = PageEntry->Uint64;
> SetSecondLevelPagingEntryAttribute (PageEntry, IoMmuAccess);
> + FlushPageTableMemory (VtdIndex, (UINTN)PageEntry,
> + sizeof(*PageEntry));
> NewPageEntry = PageEntry->Uint64;
> if (CurrentPageEntry != NewPageEntry) {
> *IsModified = TRUE;
> @@ -639,6 +653,7 @@ NeedSplitPage (
> /**
> This function splits one page entry to small page entries.
>
> + @param[in] VtdIndex The index used to identify a VTd engine.
> @param[in] PageEntry The page entry to be splitted.
> @param[in] PageAttribute The page attribute of the page entry.
> @param[in] SplitAttribute How to split the page entry.
> @@ -649,6 +664,7 @@ NeedSplitPage (
> **/
> RETURN_STATUS
> SplitSecondLevelPage (
> + IN UINTN VtdIndex,
> IN VTD_SECOND_LEVEL_PAGING_ENTRY *PageEntry,
> IN PAGE_ATTRIBUTE PageAttribute,
> IN PAGE_ATTRIBUTE SplitAttribute
> @@ -675,8 +691,11 @@ SplitSecondLevelPage (
> for (Index = 0; Index < SIZE_4KB / sizeof(UINT64); Index++) {
> NewPageEntry[Index] = (BaseAddress + SIZE_4KB * Index) |
> (PageEntry->Uint64 & PAGE_PROGATE_BITS);
> }
> + FlushPageTableMemory (VtdIndex, (UINTN)NewPageEntry, SIZE_4KB);
> +
> PageEntry->Uint64 = (UINT64)(UINTN)NewPageEntry;
> SetSecondLevelPagingEntryAttribute (PageEntry,
> EDKII_IOMMU_ACCESS_READ | EDKII_IOMMU_ACCESS_WRITE);
> + FlushPageTableMemory (VtdIndex, (UINTN)PageEntry,
> + sizeof(*PageEntry));
> return RETURN_SUCCESS;
> } else {
> return RETURN_UNSUPPORTED;
> @@ -697,8 +716,11 @@ SplitSecondLevelPage (
> for (Index = 0; Index < SIZE_4KB / sizeof(UINT64); Index++) {
> NewPageEntry[Index] = (BaseAddress + SIZE_2MB * Index) |
> VTD_PG_PS | (PageEntry->Uint64 & PAGE_PROGATE_BITS);
> }
> + FlushPageTableMemory (VtdIndex, (UINTN)NewPageEntry, SIZE_4KB);
> +
> PageEntry->Uint64 = (UINT64)(UINTN)NewPageEntry;
> SetSecondLevelPagingEntryAttribute (PageEntry,
> EDKII_IOMMU_ACCESS_READ | EDKII_IOMMU_ACCESS_WRITE);
> + FlushPageTableMemory (VtdIndex, (UINTN)PageEntry,
> + sizeof(*PageEntry));
> return RETURN_SUCCESS;
> } else {
> return RETURN_UNSUPPORTED;
> @@ -730,6 +752,7 @@ SplitSecondLevelPage ( EFI_STATUS
> SetSecondLevelPagingAttribute (
> IN UINTN VtdIndex,
> + IN UINT16 DomainIdentifier,
> IN VTD_SECOND_LEVEL_PAGING_ENTRY *SecondLevelPagingEntry,
> IN UINT64 BaseAddress,
> IN UINT64 Length,
> @@ -756,7 +779,7 @@ SetSecondLevelPagingAttribute (
> }
>
> while (Length != 0) {
> - PageEntry = GetSecondLevelPageTableEntry (SecondLevelPagingEntry,
> BaseAddress, &PageAttribute);
> + PageEntry = GetSecondLevelPageTableEntry (VtdIndex,
> + SecondLevelPagingEntry, BaseAddress, &PageAttribute);
> if (PageEntry == NULL) {
> DEBUG ((DEBUG_ERROR, "PageEntry - NULL\n"));
> return RETURN_UNSUPPORTED;
> @@ -764,7 +787,7 @@ SetSecondLevelPagingAttribute (
> PageEntryLength = PageAttributeToLength (PageAttribute);
> SplitAttribute = NeedSplitPage (BaseAddress, Length, PageAttribute);
> if (SplitAttribute == PageNone) {
> - ConvertSecondLevelPageEntryAttribute (PageEntry, IoMmuAccess,
> &IsEntryModified);
> + ConvertSecondLevelPageEntryAttribute (VtdIndex, PageEntry,
> + IoMmuAccess, &IsEntryModified);
> if (IsEntryModified) {
> mVtdUnitInformation[VtdIndex].HasDirtyPages = TRUE;
> }
> @@ -774,7 +797,7 @@ SetSecondLevelPagingAttribute (
> BaseAddress += PageEntryLength;
> Length -= PageEntryLength;
> } else {
> - Status = SplitSecondLevelPage (PageEntry, PageAttribute, SplitAttribute);
> + Status = SplitSecondLevelPage (VtdIndex, PageEntry,
> + PageAttribute, SplitAttribute);
> if (RETURN_ERROR (Status)) {
> DEBUG ((DEBUG_ERROR, "SplitSecondLevelPage - %r\n", Status));
> return RETURN_UNSUPPORTED;
> @@ -787,8 +810,6 @@ SetSecondLevelPagingAttribute (
> }
> }
>
> - InvalidatePageEntry (VtdIndex);
> -
> return EFI_SUCCESS;
> }
>
> @@ -814,6 +835,7 @@ SetSecondLevelPagingAttribute ( EFI_STATUS
> SetPageAttribute (
> IN UINTN VtdIndex,
> + IN UINT16 DomainIdentifier,
> IN VTD_SECOND_LEVEL_PAGING_ENTRY *SecondLevelPagingEntry,
> IN UINT64 BaseAddress,
> IN UINT64 Length,
> @@ -823,7 +845,7 @@ SetPageAttribute (
> EFI_STATUS Status;
> Status = EFI_NOT_FOUND;
> if (SecondLevelPagingEntry != NULL) {
> - Status = SetSecondLevelPagingAttribute (VtdIndex,
> SecondLevelPagingEntry, BaseAddress, Length, IoMmuAccess);
> + Status = SetSecondLevelPagingAttribute (VtdIndex, DomainIdentifier,
> + SecondLevelPagingEntry, BaseAddress, Length, IoMmuAccess);
> }
> return Status;
> }
> @@ -862,6 +884,8 @@ SetAccessAttribute (
> VTD_CONTEXT_ENTRY *ContextEntry;
> VTD_SECOND_LEVEL_PAGING_ENTRY *SecondLevelPagingEntry;
> UINT64 Pt;
> + UINTN PciDescriptorIndex;
> + UINT16 DomainIdentifier;
>
> SecondLevelPagingEntry = NULL;
>
> @@ -873,6 +897,13 @@ SetAccessAttribute (
> return EFI_DEVICE_ERROR;
> }
>
> + PciDescriptorIndex = GetPciDescriptor (VtdIndex, Segment, SourceId);
> + mVtdUnitInformation[VtdIndex].PciDeviceInfo.AccessCount[PciDescriptorI
> + ndex]++;
> + //
> + // DomainId should not be 0.
> + //
> + DomainIdentifier = (UINT16)(PciDescriptorIndex + 1);
> +
> if (ExtContextEntry != NULL) {
> if (ExtContextEntry->Bits.Present == 0) {
> SecondLevelPagingEntry = CreateSecondLevelPagingEntry (VtdIndex, 0);
> @@ -881,9 +912,11 @@ SetAccessAttribute (
>
> ExtContextEntry->Bits.SecondLevelPageTranslationPointerLo = (UINT32)
> Pt;
> ExtContextEntry->Bits.SecondLevelPageTranslationPointerHi = (UINT32)
> RShiftU64(Pt, 20);
> - ExtContextEntry->Bits.DomainIdentifier = (UINT16) GetPciDescriptor
> (VtdIndex, Segment, SourceId);
> + ExtContextEntry->Bits.DomainIdentifier = DomainIdentifier;
> ExtContextEntry->Bits.Present = 1;
> + FlushPageTableMemory (VtdIndex, (UINTN)ExtContextEntry,
> + sizeof(*ExtContextEntry));
> DumpDmarExtContextEntryTable
> (mVtdUnitInformation[VtdIndex].ExtRootEntryTable);
> + mVtdUnitInformation[VtdIndex].HasDirtyContext = TRUE;
> } else {
> SecondLevelPagingEntry = (VOID
> *)(UINTN)VTD_64BITS_ADDRESS(ExtContextEntry->Bits.SecondLevelPageTransla
> tionPointerLo, ExtContextEntry->Bits.SecondLevelPageTranslationPointerHi);
> DEBUG ((DEBUG_VERBOSE,"SecondLevelPagingEntry - 0x%x (S%04x
> B%02x D%02x F%02x)\n", SecondLevelPagingEntry, Segment, SourceId.Bits.Bus,
> SourceId.Bits.Device, SourceId.Bits.Function)); @@ -896,9 +929,11 @@
> SetAccessAttribute (
>
> ContextEntry->Bits.SecondLevelPageTranslationPointerLo = (UINT32) Pt;
> ContextEntry->Bits.SecondLevelPageTranslationPointerHi = (UINT32)
> RShiftU64(Pt, 20);
> - ContextEntry->Bits.DomainIdentifier = (UINT16) GetPciDescriptor
> (VtdIndex, Segment, SourceId);
> + ContextEntry->Bits.DomainIdentifier = DomainIdentifier;
> ContextEntry->Bits.Present = 1;
> + FlushPageTableMemory (VtdIndex, (UINTN)ContextEntry,
> + sizeof(*ContextEntry));
> DumpDmarContextEntryTable
> (mVtdUnitInformation[VtdIndex].RootEntryTable);
> + mVtdUnitInformation[VtdIndex].HasDirtyContext = TRUE;
> } else {
> SecondLevelPagingEntry = (VOID
> *)(UINTN)VTD_64BITS_ADDRESS(ContextEntry->Bits.SecondLevelPageTranslatio
> nPointerLo, ContextEntry->Bits.SecondLevelPageTranslationPointerHi);
> DEBUG ((DEBUG_VERBOSE,"SecondLevelPagingEntry - 0x%x (S%04x
> B%02x D%02x F%02x)\n", SecondLevelPagingEntry, Segment, SourceId.Bits.Bus,
> SourceId.Bits.Device, SourceId.Bits.Function)); @@ -911,6 +946,7 @@
> SetAccessAttribute (
> if (SecondLevelPagingEntry !=
> mVtdUnitInformation[VtdIndex].FixedSecondLevelPagingEntry) {
> Status = SetPageAttribute (
> VtdIndex,
> + DomainIdentifier,
> SecondLevelPagingEntry,
> BaseAddress,
> Length,
> @@ -922,6 +958,8 @@ SetAccessAttribute (
> }
> }
>
> + InvalidatePageEntry (VtdIndex);
> +
> return EFI_SUCCESS;
> }
>
> @@ -965,11 +1003,13 @@ AlwaysEnablePageAttribute (
> ExtContextEntry->Bits.SecondLevelPageTranslationPointerHi = (UINT32)
> RShiftU64(Pt, 20);
> ExtContextEntry->Bits.DomainIdentifier = ((1 <<
> (UINT8)((UINTN)mVtdUnitInformation[VtdIndex].CapReg.Bits.ND * 2 + 4)) - 1);
> ExtContextEntry->Bits.Present = 1;
> + FlushPageTableMemory (VtdIndex, (UINTN)ExtContextEntry,
> + sizeof(*ExtContextEntry));
> } else if (ContextEntry != NULL) {
> ContextEntry->Bits.SecondLevelPageTranslationPointerLo = (UINT32) Pt;
> ContextEntry->Bits.SecondLevelPageTranslationPointerHi = (UINT32)
> RShiftU64(Pt, 20);
> ContextEntry->Bits.DomainIdentifier = ((1 <<
> (UINT8)((UINTN)mVtdUnitInformation[VtdIndex].CapReg.Bits.ND * 2 + 4)) - 1);
> ContextEntry->Bits.Present = 1;
> + FlushPageTableMemory (VtdIndex, (UINTN)ContextEntry,
> + sizeof(*ContextEntry));
> }
>
> return EFI_SUCCESS;
> diff --git a/IntelSiliconPkg/IntelVTdDxe/TranslationTableEx.c
> b/IntelSiliconPkg/IntelVTdDxe/TranslationTableEx.c
> index 65ed16e..9d4e6ea 100644
> --- a/IntelSiliconPkg/IntelVTdDxe/TranslationTableEx.c
> +++ b/IntelSiliconPkg/IntelVTdDxe/TranslationTableEx.c
> @@ -73,6 +73,7 @@ CreateExtContextEntry (
> ExtRootEntry->Bits.UpperContextTablePointerLo = (UINT32) RShiftU64
> ((UINT64)(UINTN)Buffer, 12) + 1;
> ExtRootEntry->Bits.UpperContextTablePointerHi = (UINT32) RShiftU64
> (RShiftU64 ((UINT64)(UINTN)Buffer, 12) + 1, 20);
> ExtRootEntry->Bits.UpperPresent = 1;
> + FlushPageTableMemory (VtdIndex, (UINTN)ExtRootEntry,
> + sizeof(*ExtRootEntry));
> Buffer = (UINT8 *)Buffer + EFI_PAGES_TO_SIZE (ContextPages);
> }
>
> @@ -92,6 +93,7 @@ CreateExtContextEntry (
> ExtContextEntry->Bits.AddressWidth = 0x2;
> break;
> }
> + FlushPageTableMemory (VtdIndex, (UINTN)ExtContextEntry,
> + sizeof(*ExtContextEntry));
> }
>
> return EFI_SUCCESS;
> diff --git a/IntelSiliconPkg/IntelVTdDxe/VtdReg.c
> b/IntelSiliconPkg/IntelVTdDxe/VtdReg.c
> index f36e3de..b1178b7 100644
> --- a/IntelSiliconPkg/IntelVTdDxe/VtdReg.c
> +++ b/IntelSiliconPkg/IntelVTdDxe/VtdReg.c
> @@ -20,43 +20,62 @@ VTD_UNIT_INFORMATION
> *mVtdUnitInformation;
> BOOLEAN mVtdEnabled;
>
> /**
> - Invalid VTd global IOTLB.
> + Flush VTD page table and context table memory.
>
> - @param[in] VtdIndex The index of VTd engine.
> + This action is to make sure the IOMMU engine can get final data in memory.
>
> - @retval EFI_SUCCESS VTd global IOTLB is invalidated.
> - @retval EFI_DEVICE_ERROR VTd global IOTLB is not invalidated.
> + @param[in] VtdIndex The index used to identify a VTd engine.
> + @param[in] Base The base address of memory to be
> flushed.
> + @param[in] Size The size of memory in bytes to be flushed.
> **/
> -EFI_STATUS
> -InvalidateVtdIOTLBGlobal (
> +VOID
> +FlushPageTableMemory (
> + IN UINTN VtdIndex,
> + IN UINTN Base,
> + IN UINTN Size
> + )
> +{
> + if (mVtdUnitInformation[VtdIndex].ECapReg.Bits.C == 0) {
> + WriteBackDataCacheRange ((VOID *)Base, Size);
> + }
> +}
> +
> +/**
> + Flush VTd engine write buffer.
> +
> + @param[in] VtdIndex The index used to identify a VTd engine.
> +**/
> +VOID
> +FlushWriteBuffer (
> IN UINTN VtdIndex
> )
> {
> - UINT64 Reg64;
> UINT32 Reg32;
>
> - if (!mVtdEnabled) {
> - return EFI_SUCCESS;
> + if (mVtdUnitInformation[VtdIndex].CapReg.Bits.RWBF != 0) {
> + Reg32 = MmioRead32
> (mVtdUnitInformation[VtdIndex].VtdUnitBaseAddress + R_GSTS_REG);
> + MmioWrite32 (mVtdUnitInformation[VtdIndex].VtdUnitBaseAddress +
> R_GCMD_REG, Reg32 | B_GMCD_REG_WBF);
> + do {
> + Reg32 = MmioRead32
> (mVtdUnitInformation[VtdIndex].VtdUnitBaseAddress + R_GSTS_REG);
> + } while ((Reg32 & B_GSTS_REG_WBF) != 0);
> }
> +}
>
> - DEBUG((DEBUG_VERBOSE, "InvalidateVtdIOTLBGlobal(%d)\n", VtdIndex));
> -
> - AsmWbinvd();
> +/**
> + Invalidate VTd context cache.
>
> - //
> - // Write Buffer Flush before invalidation
> - //
> - Reg32 = MmioRead32 (mVtdUnitInformation[VtdIndex].VtdUnitBaseAddress
> + R_CAP_REG);
> - if ((Reg32 & B_CAP_REG_RWBF) != 0) {
> - MmioWrite32 (mVtdUnitInformation[VtdIndex].VtdUnitBaseAddress +
> R_GCMD_REG, B_GMCD_REG_WBF);
> - }
> + @param[in] VtdIndex The index used to identify a VTd engine.
> +**/
> +EFI_STATUS
> +InvalidateContextCache (
> + IN UINTN VtdIndex
> + )
> +{
> + UINT64 Reg64;
>
> - //
> - // Invalidate the context cache
> - //
> Reg64 = MmioRead64 (mVtdUnitInformation[VtdIndex].VtdUnitBaseAddress
> + R_CCMD_REG);
> if ((Reg64 & B_CCMD_REG_ICC) != 0) {
> - DEBUG ((DEBUG_ERROR,"ERROR: InvalidateVtdIOTLBGlobal:
> B_CCMD_REG_ICC is set for VTD(%d)\n",VtdIndex));
> + DEBUG ((DEBUG_ERROR,"ERROR: InvalidateContextCache:
> B_CCMD_REG_ICC
> + is set for VTD(%d)\n",VtdIndex));
> return EFI_DEVICE_ERROR;
> }
>
> @@ -68,97 +87,29 @@ InvalidateVtdIOTLBGlobal (
> Reg64 = MmioRead64
> (mVtdUnitInformation[VtdIndex].VtdUnitBaseAddress + R_CCMD_REG);
> } while ((Reg64 & B_CCMD_REG_ICC) != 0);
>
> - //
> - // Invalidate the IOTLB cache
> - //
> -
> - Reg64 = MmioRead64 (mVtdUnitInformation[VtdIndex].VtdUnitBaseAddress
> + (mVtdUnitInformation[VtdIndex].ECapReg.Bits.IRO * 16) + R_IOTLB_REG);
> - if ((Reg64 & B_IOTLB_REG_IVT) != 0) {
> - DEBUG ((DEBUG_ERROR,"ERROR: InvalidateVtdIOTLBGlobal:
> B_IOTLB_REG_IVT is set for VTD(%d)\n", VtdIndex));
> - return EFI_DEVICE_ERROR;
> - }
> -
> - Reg64 &= ((~B_IOTLB_REG_IVT) & (~B_IOTLB_REG_IIRG_MASK));
> - Reg64 |= (B_IOTLB_REG_IVT | V_IOTLB_REG_IIRG_GLOBAL);
> - MmioWrite64 (mVtdUnitInformation[VtdIndex].VtdUnitBaseAddress +
> (mVtdUnitInformation[VtdIndex].ECapReg.Bits.IRO * 16) + R_IOTLB_REG,
> Reg64);
> -
> - do {
> - Reg64 = MmioRead64
> (mVtdUnitInformation[VtdIndex].VtdUnitBaseAddress +
> (mVtdUnitInformation[VtdIndex].ECapReg.Bits.IRO * 16) + R_IOTLB_REG);
> - } while ((Reg64 & B_IOTLB_REG_IVT) != 0);
> -
> - //
> - // Disable VTd
> - //
> - MmioWrite32 (mVtdUnitInformation[VtdIndex].VtdUnitBaseAddress +
> R_GCMD_REG, B_GMCD_REG_SRTP);
> - do {
> - Reg32 = MmioRead32
> (mVtdUnitInformation[VtdIndex].VtdUnitBaseAddress + R_GSTS_REG);
> - } while((Reg32 & B_GSTS_REG_RTPS) == 0);
> -
> - //
> - // Enable VTd
> - //
> - MmioWrite32 (mVtdUnitInformation[VtdIndex].VtdUnitBaseAddress +
> R_GCMD_REG, B_GMCD_REG_TE);
> - do {
> - Reg32 = MmioRead32
> (mVtdUnitInformation[VtdIndex].VtdUnitBaseAddress + R_GSTS_REG);
> - } while ((Reg32 & B_GSTS_REG_TE) == 0);
> -
> return EFI_SUCCESS;
> }
>
> /**
> - Invalid VTd IOTLB domain.
> -
> - @param[in] VtdIndex The index of VTd engine.
> - @param[in] DomainIdentifier The domain ID of the source.
> + Invalidate VTd IOTLB.
>
> - @retval EFI_SUCCESS VTd IOTLB domain is invalidated.
> - @retval EFI_DEVICE_ERROR VTd IOTLB domain is not invalidated.
> + @param[in] VtdIndex The index used to identify a VTd engine.
> **/
> EFI_STATUS
> -InvalidateVtdIOTLBDomain (
> - IN UINTN VtdIndex,
> - IN UINT16 DomainIdentifier
> +InvalidateIOTLB (
> + IN UINTN VtdIndex
> )
> {
> UINT64 Reg64;
>
> - if (!mVtdEnabled) {
> - return EFI_SUCCESS;
> - }
> -
> - DEBUG((DEBUG_VERBOSE, "InvalidateVtdIOTLBDomain(%d): 0x%016lx
> (0x%04x)\n", VtdIndex, DomainIdentifier));
> -
> - //
> - // Invalidate the context cache
> - //
> - Reg64 = MmioRead64 (mVtdUnitInformation[VtdIndex].VtdUnitBaseAddress
> + R_CCMD_REG);
> - if ((Reg64 & B_CCMD_REG_ICC) != 0) {
> - DEBUG ((DEBUG_ERROR,"ERROR: InvalidateVtdIOTLBDomain:
> B_CCMD_REG_ICC is set for VTD(%d)\n",VtdIndex));
> - return EFI_DEVICE_ERROR;
> - }
> -
> - Reg64 &= ((~B_CCMD_REG_ICC) & (~B_CCMD_REG_CIRG_MASK));
> - Reg64 |= (B_CCMD_REG_ICC | V_CCMD_REG_CIRG_DOMAIN);
> - Reg64 |= (B_CCMD_REG_ICC | V_CCMD_REG_CIRG_DOMAIN);
> - MmioWrite64 (mVtdUnitInformation[VtdIndex].VtdUnitBaseAddress +
> R_CCMD_REG, Reg64);
> -
> - do {
> - Reg64 = MmioRead64
> (mVtdUnitInformation[VtdIndex].VtdUnitBaseAddress + R_CCMD_REG);
> - } while ((Reg64 & B_CCMD_REG_ICC) != 0);
> -
> - //
> - // Invalidate the IOTLB cache
> - //
> -
> Reg64 = MmioRead64 (mVtdUnitInformation[VtdIndex].VtdUnitBaseAddress
> + (mVtdUnitInformation[VtdIndex].ECapReg.Bits.IRO * 16) + R_IOTLB_REG);
> if ((Reg64 & B_IOTLB_REG_IVT) != 0) {
> - DEBUG ((DEBUG_ERROR,"ERROR: InvalidateVtdIOTLBDomain:
> B_IOTLB_REG_IVT is set for VTD(%d)\n", VtdIndex));
> + DEBUG ((DEBUG_ERROR,"ERROR: InvalidateIOTLB: B_IOTLB_REG_IVT is set
> + for VTD(%d)\n", VtdIndex));
> return EFI_DEVICE_ERROR;
> }
>
> Reg64 &= ((~B_IOTLB_REG_IVT) & (~B_IOTLB_REG_IIRG_MASK));
> - Reg64 |= (B_IOTLB_REG_IVT | V_IOTLB_REG_IIRG_DOMAIN);
> - Reg64 |= LShiftU64 (DomainIdentifier, 32);
> + Reg64 |= (B_IOTLB_REG_IVT | V_IOTLB_REG_IIRG_GLOBAL);
> MmioWrite64 (mVtdUnitInformation[VtdIndex].VtdUnitBaseAddress +
> (mVtdUnitInformation[VtdIndex].ECapReg.Bits.IRO * 16) + R_IOTLB_REG,
> Reg64);
>
> do {
> @@ -169,53 +120,41 @@ InvalidateVtdIOTLBDomain ( }
>
> /**
> - Invalid VTd IOTLB page.
> + Invalid VTd global IOTLB.
>
> @param[in] VtdIndex The index of VTd engine.
> - @param[in] Address The address of IOTLB page.
> - @param[in] AddressMode The address mode of IOTLB page.
> - @param[in] DomainIdentifier The domain ID of the source.
>
> - @retval EFI_SUCCESS VTd IOTLB page is invalidated.
> - @retval EFI_DEVICE_ERROR VTd IOTLB page is not invalidated.
> + @retval EFI_SUCCESS VTd global IOTLB is invalidated.
> + @retval EFI_DEVICE_ERROR VTd global IOTLB is not invalidated.
> **/
> EFI_STATUS
> -InvalidateVtdIOTLBPage (
> - IN UINTN VtdIndex,
> - IN UINT64 Address,
> - IN UINT8 AddressMode,
> - IN UINT16 DomainIdentifier
> +InvalidateVtdIOTLBGlobal (
> + IN UINTN VtdIndex
> )
> {
> - UINT64 Reg64;
> - UINT64 Data64;
> -
> if (!mVtdEnabled) {
> return EFI_SUCCESS;
> }
>
> - DEBUG((DEBUG_VERBOSE, "InvalidateVtdIOTLBPage(%d): 0x%016lx
> (0x%02x)\n", VtdIndex, Address, AddressMode));
> -
> - if (mVtdUnitInformation[VtdIndex].CapReg.Bits.PSI != 0) {
> - Reg64 = MmioRead64
> (mVtdUnitInformation[VtdIndex].VtdUnitBaseAddress +
> (mVtdUnitInformation[VtdIndex].ECapReg.Bits.IRO * 16) + R_IOTLB_REG);
> - if ((Reg64 & B_IOTLB_REG_IVT) != 0) {
> - DEBUG ((DEBUG_ERROR,"ERROR: InvalidateVtdIOTLBPage:
> B_IOTLB_REG_IVT is set for VTD(%d)\n", VtdIndex));
> - return EFI_DEVICE_ERROR;
> - }
> + DEBUG((DEBUG_VERBOSE, "InvalidateVtdIOTLBGlobal(%d)\n", VtdIndex));
>
> - Data64 = Address | AddressMode;
> - MmioWrite64 (mVtdUnitInformation[VtdIndex].VtdUnitBaseAddress +
> (mVtdUnitInformation[VtdIndex].ECapReg.Bits.IRO * 16) + R_IVA_REG, Data64);
> + //
> + // Write Buffer Flush before invalidation // FlushWriteBuffer
> + (VtdIndex);
>
> - Reg64 &= ((~B_IOTLB_REG_IVT) & (~B_IOTLB_REG_IIRG_MASK));
> - Reg64 |= (B_IOTLB_REG_IVT | V_IOTLB_REG_IIRG_PAGE);
> - Reg64 |= LShiftU64 (DomainIdentifier, 32);
> - MmioWrite64 (mVtdUnitInformation[VtdIndex].VtdUnitBaseAddress +
> (mVtdUnitInformation[VtdIndex].ECapReg.Bits.IRO * 16) + R_IOTLB_REG,
> Reg64);
> + //
> + // Invalidate the context cache
> + //
> + if (mVtdUnitInformation[VtdIndex].HasDirtyContext) {
> + InvalidateContextCache (VtdIndex);
> + }
>
> - do {
> - Reg64 = MmioRead64
> (mVtdUnitInformation[VtdIndex].VtdUnitBaseAddress +
> (mVtdUnitInformation[VtdIndex].ECapReg.Bits.IRO * 16) + R_IOTLB_REG);
> - } while ((Reg64 & B_IOTLB_REG_IVT) != 0);
> - } else {
> - InvalidateVtdIOTLBGlobal (VtdIndex);
> + //
> + // Invalidate the IOTLB cache
> + //
> + if (mVtdUnitInformation[VtdIndex].HasDirtyContext ||
> mVtdUnitInformation[VtdIndex].HasDirtyPages) {
> + InvalidateIOTLB (VtdIndex);
> }
>
> return EFI_SUCCESS;
> @@ -268,11 +207,8 @@ EnableDmar (
> )
> {
> UINTN Index;
> - UINT64 Reg64;
> UINT32 Reg32;
>
> - AsmWbinvd();
> -
> for (Index = 0; Index < mVtdUnitNumber; Index++) {
> DEBUG((DEBUG_INFO, ">>>>>>EnableDmar() for engine [%d] \n", Index));
>
> @@ -299,48 +235,17 @@ EnableDmar (
> //
> // Write Buffer Flush before invalidation
> //
> - Reg32 = MmioRead32 (mVtdUnitInformation[Index].VtdUnitBaseAddress +
> R_CAP_REG);
> - if ((Reg32 & B_CAP_REG_RWBF) != 0) {
> - MmioWrite32 (mVtdUnitInformation[Index].VtdUnitBaseAddress +
> R_GCMD_REG, B_GMCD_REG_WBF);
> - }
> + FlushWriteBuffer (Index);
>
> //
> // Invalidate the context cache
> //
> - Reg64 = MmioRead64 (mVtdUnitInformation[Index].VtdUnitBaseAddress +
> R_CCMD_REG);
> - if ((Reg64 & B_CCMD_REG_ICC) != 0) {
> - DEBUG ((DEBUG_INFO,"ERROR: EnableDmar: B_CCMD_REG_ICC is set
> for VTD(%d)\n",Index));
> - return EFI_DEVICE_ERROR;
> - }
> -
> - Reg64 &= ((~B_CCMD_REG_ICC) & (~B_CCMD_REG_CIRG_MASK));
> - Reg64 |= (B_CCMD_REG_ICC | V_CCMD_REG_CIRG_GLOBAL);
> - MmioWrite64 (mVtdUnitInformation[Index].VtdUnitBaseAddress +
> R_CCMD_REG, Reg64);
> -
> - DEBUG((DEBUG_INFO, "EnableDmar: Waiting B_CCMD_REG_ICC ...\n"));
> - do {
> - Reg64 = MmioRead64 (mVtdUnitInformation[Index].VtdUnitBaseAddress
> + R_CCMD_REG);
> - } while ((Reg64 & B_CCMD_REG_ICC) != 0);
> + InvalidateContextCache (Index);
>
> //
> // Invalidate the IOTLB cache
> //
> - DEBUG((DEBUG_INFO, "EnableDmar: IRO 0x%x\n",
> mVtdUnitInformation[Index].ECapReg.Bits.IRO));
> -
> - Reg64 = MmioRead64 (mVtdUnitInformation[Index].VtdUnitBaseAddress +
> (mVtdUnitInformation[Index].ECapReg.Bits.IRO * 16) + R_IOTLB_REG);
> - if ((Reg64 & B_IOTLB_REG_IVT) != 0) {
> - DEBUG ((DEBUG_INFO,"ERROR: EnableDmar: B_IOTLB_REG_IVT is set for
> VTD(%d)\n", Index));
> - return EFI_DEVICE_ERROR;
> - }
> -
> - Reg64 &= ((~B_IOTLB_REG_IVT) & (~B_IOTLB_REG_IIRG_MASK));
> - Reg64 |= (B_IOTLB_REG_IVT | V_IOTLB_REG_IIRG_GLOBAL);
> - MmioWrite64 (mVtdUnitInformation[Index].VtdUnitBaseAddress +
> (mVtdUnitInformation[Index].ECapReg.Bits.IRO * 16) + R_IOTLB_REG, Reg64);
> -
> - DEBUG((DEBUG_INFO, "EnableDmar: Waiting B_IOTLB_REG_IVT ...\n"));
> - do {
> - Reg64 = MmioRead64 (mVtdUnitInformation[Index].VtdUnitBaseAddress
> + (mVtdUnitInformation[Index].ECapReg.Bits.IRO * 16) + R_IOTLB_REG);
> - } while ((Reg64 & B_IOTLB_REG_IVT) != 0);
> + InvalidateIOTLB (Index);
>
> //
> // Enable VTd
> @@ -371,20 +276,16 @@ DisableDmar (
> )
> {
> UINTN Index;
> + UINTN SubIndex;
> UINT32 Reg32;
>
> - AsmWbinvd();
> -
> for (Index = 0; Index < mVtdUnitNumber; Index++) {
> DEBUG((DEBUG_INFO, ">>>>>>DisableDmar() for engine [%d] \n",
> Index));
>
> //
> // Write Buffer Flush before invalidation
> //
> - Reg32 = MmioRead32 (mVtdUnitInformation[Index].VtdUnitBaseAddress +
> R_CAP_REG);
> - if ((Reg32 & B_CAP_REG_RWBF) != 0) {
> - MmioWrite32 (mVtdUnitInformation[Index].VtdUnitBaseAddress +
> R_GCMD_REG, B_GMCD_REG_WBF);
> - }
> + FlushWriteBuffer (Index);
>
> //
> // Disable VTd
> @@ -402,6 +303,19 @@ DisableDmar (
>
> mVtdEnabled = FALSE;
>
> + for (Index = 0; Index < mVtdUnitNumber; Index++) {
> + DEBUG((DEBUG_INFO, "engine [%d] access\n", Index));
> + for (SubIndex = 0; SubIndex <
> mVtdUnitInformation[Index].PciDeviceInfo.PciDescriptorNumber; SubIndex++) {
> + DEBUG ((DEBUG_INFO, " PCI S%04X B%02x D%02x F%02x - %d\n",
> + mVtdUnitInformation[Index].Segment,
> +
> mVtdUnitInformation[Index].PciDeviceInfo.PciDescriptors[SubIndex].Bits.Bus,
> +
> mVtdUnitInformation[Index].PciDeviceInfo.PciDescriptors[SubIndex].Bits.Device
> ,
> +
> mVtdUnitInformation[Index].PciDeviceInfo.PciDescriptors[SubIndex].Bits.Functi
> on,
> + mVtdUnitInformation[Index].PciDeviceInfo.AccessCount[SubIndex]
> + ));
> + }
> + }
> +
> return EFI_SUCCESS;
> }
>
> --
> 2.7.4.windows.1
^ permalink raw reply [flat|nested] 5+ messages in thread
end of thread, other threads:[~2017-08-14 4:14 UTC | newest]
Thread overview: 5+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2017-08-11 14:54 [PATCH V2 0/2] Improve IntelVTd performance Jiewen Yao
2017-08-11 14:54 ` [PATCH V2 1/2] IntelSiliconPkg/dsc: Add CacheMaintenanceLib Jiewen Yao
2017-08-11 14:54 ` [PATCH V2 2/2] IntelSiliconPkg/IntelVTdDxe: Improve performance Jiewen Yao
2017-08-14 3:53 ` Zeng, Star
2017-08-14 4:16 ` Yao, Jiewen
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox