* [PATCH] BaseTools: Fix --hash Package and Module hash value.
[not found] <20180509073836.6924-1-derek.lin2@hpe.com>
@ 2018-05-09 7:49 ` Lin, Derek (HPS UEFI Dev)
2018-05-09 8:35 ` Zhu, Yonghong
0 siblings, 1 reply; 7+ messages in thread
From: Lin, Derek (HPS UEFI Dev) @ 2018-05-09 7:49 UTC (permalink / raw)
To: edk2-devel@lists.01.org; +Cc: yonghong.zhu@intel.com, Lin, Derek (HPS UEFI Dev)
From: Lin, Derek (HPS UEFI Dev)
Sent: Wednesday, May 9, 2018 3:39 PM
To: Lin, Derek (HPS UEFI Dev) <derek.lin2@hpe.com>
Subject: [PATCH] BaseTools: Fix --hash Package and Module hash value.
The order of List enumeration is arbitrary.
Need to be sorted while calculating Package/Module hash, otherwise it generate different hash value even nothing changes.
---
BaseTools/Source/Python/AutoGen/AutoGen.py | 17 ++++++++++-------
1 file changed, 10 insertions(+), 7 deletions(-)
diff --git a/BaseTools/Source/Python/AutoGen/AutoGen.py b/BaseTools/Source/Python/AutoGen/AutoGen.py
index 54f6b1f173..90704dcae4 100644
--- a/BaseTools/Source/Python/AutoGen/AutoGen.py
+++ b/BaseTools/Source/Python/AutoGen/AutoGen.py
@@ -2,6 +2,7 @@
# Generate AutoGen.h, AutoGen.c and *.depex files # # Copyright (c) 2007 - 2018, Intel Corporation. All rights reserved.<BR>
+# Copyright (c) 2018, Hewlett Packard Enterprise Development, L.P.<BR>
# This program and the accompanying materials # are licensed and made available under the terms and conditions of the BSD License # which accompanies this distribution. The full text of the license may be found at @@ -670,6 +671,9 @@ class WorkspaceAutoGen(AutoGen):
return True
def _GenPkgLevelHash(self, Pkg):
+ if Pkg.PackageName in GlobalData.gPackageHash[Pkg.Arch]:
+ return
+
PkgDir = os.path.join(self.BuildDir, Pkg.Arch, Pkg.PackageName)
CreateDirectory(PkgDir)
HashFile = os.path.join(PkgDir, Pkg.PackageName + '.hash') @@ -681,17 +685,16 @@ class WorkspaceAutoGen(AutoGen):
m.update(Content)
# Get include files hash value
if Pkg.Includes:
- for inc in Pkg.Includes:
+ for inc in sorted(Pkg.Includes, key=lambda x: str(x)):
for Root, Dirs, Files in os.walk(str(inc)):
- for File in Files:
+ for File in sorted(Files):
File_Path = os.path.join(Root, File)
f = open(File_Path, 'r')
Content = f.read()
f.close()
m.update(Content)
SaveFileOnChange(HashFile, m.hexdigest(), True)
- if Pkg.PackageName not in GlobalData.gPackageHash[Pkg.Arch]:
- GlobalData.gPackageHash[Pkg.Arch][Pkg.PackageName] = m.hexdigest()
+ GlobalData.gPackageHash[Pkg.Arch][Pkg.PackageName] =
+ m.hexdigest()
def _GetMetaFiles(self, Target, Toolchain, Arch):
AllWorkSpaceMetaFiles = set()
@@ -4432,13 +4435,13 @@ class ModuleAutoGen(AutoGen):
m.update(GlobalData.gPlatformHash)
# Add Package level hash
if self.DependentPackageList:
- for Pkg in self.DependentPackageList:
+ for Pkg in sorted(self.DependentPackageList, key=lambda x: x.PackageName):
if Pkg.PackageName in GlobalData.gPackageHash[self.Arch]:
m.update(GlobalData.gPackageHash[self.Arch][Pkg.PackageName])
# Add Library hash
if self.LibraryAutoGenList:
- for Lib in self.LibraryAutoGenList:
+ for Lib in sorted(self.LibraryAutoGenList, key=lambda x: x.Name):
if Lib.Name not in GlobalData.gModuleHash[self.Arch]:
Lib.GenModuleHash()
m.update(GlobalData.gModuleHash[self.Arch][Lib.Name])
@@ -4450,7 +4453,7 @@ class ModuleAutoGen(AutoGen):
m.update(Content)
# Add Module's source files
if self.SourceFileList:
- for File in self.SourceFileList:
+ for File in sorted(self.SourceFileList, key=lambda x: str(x)):
f = open(str(File), 'r')
Content = f.read()
f.close()
--
2.15.1.windows.2
^ permalink raw reply related [flat|nested] 7+ messages in thread
* Re: [PATCH] BaseTools: Fix --hash Package and Module hash value.
2018-05-09 7:49 ` [PATCH] BaseTools: Fix --hash Package and Module hash value Lin, Derek (HPS UEFI Dev)
@ 2018-05-09 8:35 ` Zhu, Yonghong
2018-05-09 9:04 ` Lin, Derek (HPS UEFI Dev)
0 siblings, 1 reply; 7+ messages in thread
From: Zhu, Yonghong @ 2018-05-09 8:35 UTC (permalink / raw)
To: Lin, Derek (HPS UEFI Dev), edk2-devel@lists.01.org
Hi Derek,
Please refer below link to add Contributed-under and Signed-off-by info.
We can use BaseTools\Scripts\PatchCheck.py to check the patch format. Thanks.
https://github.com/tianocore/tianocore.github.io/wiki/Commit-Message-Format
Best Regards,
Zhu Yonghong
-----Original Message-----
From: Lin, Derek (HPS UEFI Dev) [mailto:derek.lin2@hpe.com]
Sent: Wednesday, May 09, 2018 3:49 PM
To: edk2-devel@lists.01.org
Cc: Zhu, Yonghong <yonghong.zhu@intel.com>; Lin, Derek (HPS UEFI Dev) <derek.lin2@hpe.com>
Subject: [PATCH] BaseTools: Fix --hash Package and Module hash value.
From: Lin, Derek (HPS UEFI Dev)
Sent: Wednesday, May 9, 2018 3:39 PM
To: Lin, Derek (HPS UEFI Dev) <derek.lin2@hpe.com>
Subject: [PATCH] BaseTools: Fix --hash Package and Module hash value.
The order of List enumeration is arbitrary.
Need to be sorted while calculating Package/Module hash, otherwise it generate different hash value even nothing changes.
---
BaseTools/Source/Python/AutoGen/AutoGen.py | 17 ++++++++++-------
1 file changed, 10 insertions(+), 7 deletions(-)
diff --git a/BaseTools/Source/Python/AutoGen/AutoGen.py b/BaseTools/Source/Python/AutoGen/AutoGen.py
index 54f6b1f173..90704dcae4 100644
--- a/BaseTools/Source/Python/AutoGen/AutoGen.py
+++ b/BaseTools/Source/Python/AutoGen/AutoGen.py
@@ -2,6 +2,7 @@
# Generate AutoGen.h, AutoGen.c and *.depex files # # Copyright (c) 2007 - 2018, Intel Corporation. All rights reserved.<BR>
+# Copyright (c) 2018, Hewlett Packard Enterprise Development, L.P.<BR>
# This program and the accompanying materials # are licensed and made available under the terms and conditions of the BSD License # which accompanies this distribution. The full text of the license may be found at @@ -670,6 +671,9 @@ class WorkspaceAutoGen(AutoGen):
return True
def _GenPkgLevelHash(self, Pkg):
+ if Pkg.PackageName in GlobalData.gPackageHash[Pkg.Arch]:
+ return
+
PkgDir = os.path.join(self.BuildDir, Pkg.Arch, Pkg.PackageName)
CreateDirectory(PkgDir)
HashFile = os.path.join(PkgDir, Pkg.PackageName + '.hash') @@ -681,17 +685,16 @@ class WorkspaceAutoGen(AutoGen):
m.update(Content)
# Get include files hash value
if Pkg.Includes:
- for inc in Pkg.Includes:
+ for inc in sorted(Pkg.Includes, key=lambda x: str(x)):
for Root, Dirs, Files in os.walk(str(inc)):
- for File in Files:
+ for File in sorted(Files):
File_Path = os.path.join(Root, File)
f = open(File_Path, 'r')
Content = f.read()
f.close()
m.update(Content)
SaveFileOnChange(HashFile, m.hexdigest(), True)
- if Pkg.PackageName not in GlobalData.gPackageHash[Pkg.Arch]:
- GlobalData.gPackageHash[Pkg.Arch][Pkg.PackageName] = m.hexdigest()
+ GlobalData.gPackageHash[Pkg.Arch][Pkg.PackageName] =
+ m.hexdigest()
def _GetMetaFiles(self, Target, Toolchain, Arch):
AllWorkSpaceMetaFiles = set()
@@ -4432,13 +4435,13 @@ class ModuleAutoGen(AutoGen):
m.update(GlobalData.gPlatformHash)
# Add Package level hash
if self.DependentPackageList:
- for Pkg in self.DependentPackageList:
+ for Pkg in sorted(self.DependentPackageList, key=lambda x: x.PackageName):
if Pkg.PackageName in GlobalData.gPackageHash[self.Arch]:
m.update(GlobalData.gPackageHash[self.Arch][Pkg.PackageName])
# Add Library hash
if self.LibraryAutoGenList:
- for Lib in self.LibraryAutoGenList:
+ for Lib in sorted(self.LibraryAutoGenList, key=lambda x: x.Name):
if Lib.Name not in GlobalData.gModuleHash[self.Arch]:
Lib.GenModuleHash()
m.update(GlobalData.gModuleHash[self.Arch][Lib.Name])
@@ -4450,7 +4453,7 @@ class ModuleAutoGen(AutoGen):
m.update(Content)
# Add Module's source files
if self.SourceFileList:
- for File in self.SourceFileList:
+ for File in sorted(self.SourceFileList, key=lambda x: str(x)):
f = open(str(File), 'r')
Content = f.read()
f.close()
--
2.15.1.windows.2
^ permalink raw reply related [flat|nested] 7+ messages in thread
* [PATCH] BaseTools: Fix --hash Package and Module hash value.
[not found] <20180509085924.13140-1-derek.lin2@hpe.com>
@ 2018-05-09 9:03 ` Lin, Derek (HPS UEFI Dev)
2018-05-16 7:12 ` Zhu, Yonghong
0 siblings, 1 reply; 7+ messages in thread
From: Lin, Derek (HPS UEFI Dev) @ 2018-05-09 9:03 UTC (permalink / raw)
To: edk2-devel@lists.01.org; +Cc: yonghong.zhu@intel.com, Lin, Derek (HPS UEFI Dev)
The order of List enumeration is arbitrary.
Need to be sorted while calculating Package/Module hash, otherwise it generate different hash value even nothing changes.
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Derek Lin <derek.lin2@hpe.com>
---
BaseTools/Source/Python/AutoGen/AutoGen.py | 17 ++++++++++-------
1 file changed, 10 insertions(+), 7 deletions(-)
diff --git a/BaseTools/Source/Python/AutoGen/AutoGen.py b/BaseTools/Source/Python/AutoGen/AutoGen.py
index 54f6b1f173..90704dcae4 100644
--- a/BaseTools/Source/Python/AutoGen/AutoGen.py
+++ b/BaseTools/Source/Python/AutoGen/AutoGen.py
@@ -2,6 +2,7 @@
# Generate AutoGen.h, AutoGen.c and *.depex files # # Copyright (c) 2007 - 2018, Intel Corporation. All rights reserved.<BR>
+# Copyright (c) 2018, Hewlett Packard Enterprise Development, L.P.<BR>
# This program and the accompanying materials # are licensed and made available under the terms and conditions of the BSD License # which accompanies this distribution. The full text of the license may be found at @@ -670,6 +671,9 @@ class WorkspaceAutoGen(AutoGen):
return True
def _GenPkgLevelHash(self, Pkg):
+ if Pkg.PackageName in GlobalData.gPackageHash[Pkg.Arch]:
+ return
+
PkgDir = os.path.join(self.BuildDir, Pkg.Arch, Pkg.PackageName)
CreateDirectory(PkgDir)
HashFile = os.path.join(PkgDir, Pkg.PackageName + '.hash') @@ -681,17 +685,16 @@ class WorkspaceAutoGen(AutoGen):
m.update(Content)
# Get include files hash value
if Pkg.Includes:
- for inc in Pkg.Includes:
+ for inc in sorted(Pkg.Includes, key=lambda x: str(x)):
for Root, Dirs, Files in os.walk(str(inc)):
- for File in Files:
+ for File in sorted(Files):
File_Path = os.path.join(Root, File)
f = open(File_Path, 'r')
Content = f.read()
f.close()
m.update(Content)
SaveFileOnChange(HashFile, m.hexdigest(), True)
- if Pkg.PackageName not in GlobalData.gPackageHash[Pkg.Arch]:
- GlobalData.gPackageHash[Pkg.Arch][Pkg.PackageName] = m.hexdigest()
+ GlobalData.gPackageHash[Pkg.Arch][Pkg.PackageName] =
+ m.hexdigest()
def _GetMetaFiles(self, Target, Toolchain, Arch):
AllWorkSpaceMetaFiles = set()
@@ -4432,13 +4435,13 @@ class ModuleAutoGen(AutoGen):
m.update(GlobalData.gPlatformHash)
# Add Package level hash
if self.DependentPackageList:
- for Pkg in self.DependentPackageList:
+ for Pkg in sorted(self.DependentPackageList, key=lambda x: x.PackageName):
if Pkg.PackageName in GlobalData.gPackageHash[self.Arch]:
m.update(GlobalData.gPackageHash[self.Arch][Pkg.PackageName])
# Add Library hash
if self.LibraryAutoGenList:
- for Lib in self.LibraryAutoGenList:
+ for Lib in sorted(self.LibraryAutoGenList, key=lambda x: x.Name):
if Lib.Name not in GlobalData.gModuleHash[self.Arch]:
Lib.GenModuleHash()
m.update(GlobalData.gModuleHash[self.Arch][Lib.Name])
@@ -4450,7 +4453,7 @@ class ModuleAutoGen(AutoGen):
m.update(Content)
# Add Module's source files
if self.SourceFileList:
- for File in self.SourceFileList:
+ for File in sorted(self.SourceFileList, key=lambda x: str(x)):
f = open(str(File), 'r')
Content = f.read()
f.close()
--
2.15.1.windows.2
^ permalink raw reply related [flat|nested] 7+ messages in thread
* Re: [PATCH] BaseTools: Fix --hash Package and Module hash value.
2018-05-09 8:35 ` Zhu, Yonghong
@ 2018-05-09 9:04 ` Lin, Derek (HPS UEFI Dev)
0 siblings, 0 replies; 7+ messages in thread
From: Lin, Derek (HPS UEFI Dev) @ 2018-05-09 9:04 UTC (permalink / raw)
To: Zhu, Yonghong, edk2-devel@lists.01.org
Hi Yonghong,
Thanks for notify about this.
I resend the patch.
Thanks,
Derek
-----Original Message-----
From: Zhu, Yonghong [mailto:yonghong.zhu@intel.com]
Sent: Wednesday, May 9, 2018 4:35 PM
To: Lin, Derek (HPS UEFI Dev) <derek.lin2@hpe.com>; edk2-devel@lists.01.org
Cc: Zhu, Yonghong <yonghong.zhu@intel.com>
Subject: RE: [PATCH] BaseTools: Fix --hash Package and Module hash value.
Hi Derek,
Please refer below link to add Contributed-under and Signed-off-by info.
We can use BaseTools\Scripts\PatchCheck.py to check the patch format. Thanks.
https://github.com/tianocore/tianocore.github.io/wiki/Commit-Message-Format
Best Regards,
Zhu Yonghong
-----Original Message-----
From: Lin, Derek (HPS UEFI Dev) [mailto:derek.lin2@hpe.com]
Sent: Wednesday, May 09, 2018 3:49 PM
To: edk2-devel@lists.01.org
Cc: Zhu, Yonghong <yonghong.zhu@intel.com>; Lin, Derek (HPS UEFI Dev) <derek.lin2@hpe.com>
Subject: [PATCH] BaseTools: Fix --hash Package and Module hash value.
From: Lin, Derek (HPS UEFI Dev)
Sent: Wednesday, May 9, 2018 3:39 PM
To: Lin, Derek (HPS UEFI Dev) <derek.lin2@hpe.com>
Subject: [PATCH] BaseTools: Fix --hash Package and Module hash value.
The order of List enumeration is arbitrary.
Need to be sorted while calculating Package/Module hash, otherwise it generate different hash value even nothing changes.
---
BaseTools/Source/Python/AutoGen/AutoGen.py | 17 ++++++++++-------
1 file changed, 10 insertions(+), 7 deletions(-)
diff --git a/BaseTools/Source/Python/AutoGen/AutoGen.py b/BaseTools/Source/Python/AutoGen/AutoGen.py
index 54f6b1f173..90704dcae4 100644
--- a/BaseTools/Source/Python/AutoGen/AutoGen.py
+++ b/BaseTools/Source/Python/AutoGen/AutoGen.py
@@ -2,6 +2,7 @@
# Generate AutoGen.h, AutoGen.c and *.depex files # # Copyright (c) 2007 - 2018, Intel Corporation. All rights reserved.<BR>
+# Copyright (c) 2018, Hewlett Packard Enterprise Development, L.P.<BR>
# This program and the accompanying materials # are licensed and made available under the terms and conditions of the BSD License # which accompanies this distribution. The full text of the license may be found at @@ -670,6 +671,9 @@ class WorkspaceAutoGen(AutoGen):
return True
def _GenPkgLevelHash(self, Pkg):
+ if Pkg.PackageName in GlobalData.gPackageHash[Pkg.Arch]:
+ return
+
PkgDir = os.path.join(self.BuildDir, Pkg.Arch, Pkg.PackageName)
CreateDirectory(PkgDir)
HashFile = os.path.join(PkgDir, Pkg.PackageName + '.hash') @@ -681,17 +685,16 @@ class WorkspaceAutoGen(AutoGen):
m.update(Content)
# Get include files hash value
if Pkg.Includes:
- for inc in Pkg.Includes:
+ for inc in sorted(Pkg.Includes, key=lambda x: str(x)):
for Root, Dirs, Files in os.walk(str(inc)):
- for File in Files:
+ for File in sorted(Files):
File_Path = os.path.join(Root, File)
f = open(File_Path, 'r')
Content = f.read()
f.close()
m.update(Content)
SaveFileOnChange(HashFile, m.hexdigest(), True)
- if Pkg.PackageName not in GlobalData.gPackageHash[Pkg.Arch]:
- GlobalData.gPackageHash[Pkg.Arch][Pkg.PackageName] = m.hexdigest()
+ GlobalData.gPackageHash[Pkg.Arch][Pkg.PackageName] =
+ m.hexdigest()
def _GetMetaFiles(self, Target, Toolchain, Arch):
AllWorkSpaceMetaFiles = set()
@@ -4432,13 +4435,13 @@ class ModuleAutoGen(AutoGen):
m.update(GlobalData.gPlatformHash)
# Add Package level hash
if self.DependentPackageList:
- for Pkg in self.DependentPackageList:
+ for Pkg in sorted(self.DependentPackageList, key=lambda x: x.PackageName):
if Pkg.PackageName in GlobalData.gPackageHash[self.Arch]:
m.update(GlobalData.gPackageHash[self.Arch][Pkg.PackageName])
# Add Library hash
if self.LibraryAutoGenList:
- for Lib in self.LibraryAutoGenList:
+ for Lib in sorted(self.LibraryAutoGenList, key=lambda x: x.Name):
if Lib.Name not in GlobalData.gModuleHash[self.Arch]:
Lib.GenModuleHash()
m.update(GlobalData.gModuleHash[self.Arch][Lib.Name])
@@ -4450,7 +4453,7 @@ class ModuleAutoGen(AutoGen):
m.update(Content)
# Add Module's source files
if self.SourceFileList:
- for File in self.SourceFileList:
+ for File in sorted(self.SourceFileList, key=lambda x: str(x)):
f = open(str(File), 'r')
Content = f.read()
f.close()
--
2.15.1.windows.2
^ permalink raw reply related [flat|nested] 7+ messages in thread
* Re: [PATCH] BaseTools: Fix --hash Package and Module hash value.
2018-05-09 9:03 ` Lin, Derek (HPS UEFI Dev)
@ 2018-05-16 7:12 ` Zhu, Yonghong
2018-05-16 16:20 ` Carsey, Jaben
0 siblings, 1 reply; 7+ messages in thread
From: Zhu, Yonghong @ 2018-05-16 7:12 UTC (permalink / raw)
To: Lin, Derek (HPS UEFI Dev), edk2-devel@lists.01.org
Reviewed-by: Yonghong Zhu <yonghong.zhu@intel.com>
Best Regards,
Zhu Yonghong
-----Original Message-----
From: Lin, Derek (HPS UEFI Dev) [mailto:derek.lin2@hpe.com]
Sent: Wednesday, May 09, 2018 5:03 PM
To: edk2-devel@lists.01.org
Cc: Zhu, Yonghong <yonghong.zhu@intel.com>; Lin, Derek (HPS UEFI Dev) <derek.lin2@hpe.com>
Subject: [PATCH] BaseTools: Fix --hash Package and Module hash value.
The order of List enumeration is arbitrary.
Need to be sorted while calculating Package/Module hash, otherwise it generate different hash value even nothing changes.
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Derek Lin <derek.lin2@hpe.com>
---
BaseTools/Source/Python/AutoGen/AutoGen.py | 17 ++++++++++-------
1 file changed, 10 insertions(+), 7 deletions(-)
diff --git a/BaseTools/Source/Python/AutoGen/AutoGen.py b/BaseTools/Source/Python/AutoGen/AutoGen.py
index 54f6b1f173..90704dcae4 100644
--- a/BaseTools/Source/Python/AutoGen/AutoGen.py
+++ b/BaseTools/Source/Python/AutoGen/AutoGen.py
@@ -2,6 +2,7 @@
# Generate AutoGen.h, AutoGen.c and *.depex files # # Copyright (c) 2007 - 2018, Intel Corporation. All rights reserved.<BR>
+# Copyright (c) 2018, Hewlett Packard Enterprise Development, L.P.<BR>
# This program and the accompanying materials # are licensed and made available under the terms and conditions of the BSD License # which accompanies this distribution. The full text of the license may be found at @@ -670,6 +671,9 @@ class WorkspaceAutoGen(AutoGen):
return True
def _GenPkgLevelHash(self, Pkg):
+ if Pkg.PackageName in GlobalData.gPackageHash[Pkg.Arch]:
+ return
+
PkgDir = os.path.join(self.BuildDir, Pkg.Arch, Pkg.PackageName)
CreateDirectory(PkgDir)
HashFile = os.path.join(PkgDir, Pkg.PackageName + '.hash') @@ -681,17 +685,16 @@ class WorkspaceAutoGen(AutoGen):
m.update(Content)
# Get include files hash value
if Pkg.Includes:
- for inc in Pkg.Includes:
+ for inc in sorted(Pkg.Includes, key=lambda x: str(x)):
for Root, Dirs, Files in os.walk(str(inc)):
- for File in Files:
+ for File in sorted(Files):
File_Path = os.path.join(Root, File)
f = open(File_Path, 'r')
Content = f.read()
f.close()
m.update(Content)
SaveFileOnChange(HashFile, m.hexdigest(), True)
- if Pkg.PackageName not in GlobalData.gPackageHash[Pkg.Arch]:
- GlobalData.gPackageHash[Pkg.Arch][Pkg.PackageName] = m.hexdigest()
+ GlobalData.gPackageHash[Pkg.Arch][Pkg.PackageName] =
+ m.hexdigest()
def _GetMetaFiles(self, Target, Toolchain, Arch):
AllWorkSpaceMetaFiles = set()
@@ -4432,13 +4435,13 @@ class ModuleAutoGen(AutoGen):
m.update(GlobalData.gPlatformHash)
# Add Package level hash
if self.DependentPackageList:
- for Pkg in self.DependentPackageList:
+ for Pkg in sorted(self.DependentPackageList, key=lambda x: x.PackageName):
if Pkg.PackageName in GlobalData.gPackageHash[self.Arch]:
m.update(GlobalData.gPackageHash[self.Arch][Pkg.PackageName])
# Add Library hash
if self.LibraryAutoGenList:
- for Lib in self.LibraryAutoGenList:
+ for Lib in sorted(self.LibraryAutoGenList, key=lambda x: x.Name):
if Lib.Name not in GlobalData.gModuleHash[self.Arch]:
Lib.GenModuleHash()
m.update(GlobalData.gModuleHash[self.Arch][Lib.Name])
@@ -4450,7 +4453,7 @@ class ModuleAutoGen(AutoGen):
m.update(Content)
# Add Module's source files
if self.SourceFileList:
- for File in self.SourceFileList:
+ for File in sorted(self.SourceFileList, key=lambda x: str(x)):
f = open(str(File), 'r')
Content = f.read()
f.close()
--
2.15.1.windows.2
^ permalink raw reply related [flat|nested] 7+ messages in thread
* Re: [PATCH] BaseTools: Fix --hash Package and Module hash value.
2018-05-16 7:12 ` Zhu, Yonghong
@ 2018-05-16 16:20 ` Carsey, Jaben
2018-05-17 2:14 ` Lin, Derek (HPS UEFI Dev)
0 siblings, 1 reply; 7+ messages in thread
From: Carsey, Jaben @ 2018-05-16 16:20 UTC (permalink / raw)
To: Zhu, Yonghong, Lin, Derek (HPS UEFI Dev), edk2-devel@lists.01.org
Derek,
The change is good. I have a comment/question on the message.
Reviewed-by: Jaben Carsey <Jaben.carsey@intel.com>
Order of list enumeration is not arbitrary, it's based on python lists being inherently insertion-ordered objects.
If insertion order is irrelevant, should we consider changing those to a different type? (set?)
-Jaben
> -----Original Message-----
> From: edk2-devel [mailto:edk2-devel-bounces@lists.01.org] On Behalf Of
> Zhu, Yonghong
> Sent: Wednesday, May 16, 2018 12:12 AM
> To: Lin, Derek (HPS UEFI Dev) <derek.lin2@hpe.com>; edk2-
> devel@lists.01.org
> Subject: Re: [edk2] [PATCH] BaseTools: Fix --hash Package and Module hash
> value.
>
> Reviewed-by: Yonghong Zhu <yonghong.zhu@intel.com>
>
> Best Regards,
> Zhu Yonghong
>
>
> -----Original Message-----
> From: Lin, Derek (HPS UEFI Dev) [mailto:derek.lin2@hpe.com]
> Sent: Wednesday, May 09, 2018 5:03 PM
> To: edk2-devel@lists.01.org
> Cc: Zhu, Yonghong <yonghong.zhu@intel.com>; Lin, Derek (HPS UEFI Dev)
> <derek.lin2@hpe.com>
> Subject: [PATCH] BaseTools: Fix --hash Package and Module hash value.
>
> The order of List enumeration is arbitrary.
> Need to be sorted while calculating Package/Module hash, otherwise it
> generate different hash value even nothing changes.
>
> Contributed-under: TianoCore Contribution Agreement 1.1
> Signed-off-by: Derek Lin <derek.lin2@hpe.com>
> ---
> BaseTools/Source/Python/AutoGen/AutoGen.py | 17 ++++++++++-------
> 1 file changed, 10 insertions(+), 7 deletions(-)
>
> diff --git a/BaseTools/Source/Python/AutoGen/AutoGen.py
> b/BaseTools/Source/Python/AutoGen/AutoGen.py
> index 54f6b1f173..90704dcae4 100644
> --- a/BaseTools/Source/Python/AutoGen/AutoGen.py
> +++ b/BaseTools/Source/Python/AutoGen/AutoGen.py
> @@ -2,6 +2,7 @@
> # Generate AutoGen.h, AutoGen.c and *.depex files # # Copyright (c) 2007
> - 2018, Intel Corporation. All rights reserved.<BR>
> +# Copyright (c) 2018, Hewlett Packard Enterprise Development, L.P.<BR>
> # This program and the accompanying materials # are licensed and made
> available under the terms and conditions of the BSD License # which
> accompanies this distribution. The full text of the license may be found at
> @@ -670,6 +671,9 @@ class WorkspaceAutoGen(AutoGen):
> return True
>
> def _GenPkgLevelHash(self, Pkg):
> + if Pkg.PackageName in GlobalData.gPackageHash[Pkg.Arch]:
> + return
> +
> PkgDir = os.path.join(self.BuildDir, Pkg.Arch, Pkg.PackageName)
> CreateDirectory(PkgDir)
> HashFile = os.path.join(PkgDir, Pkg.PackageName + '.hash') @@ -681,17
> +685,16 @@ class WorkspaceAutoGen(AutoGen):
> m.update(Content)
> # Get include files hash value
> if Pkg.Includes:
> - for inc in Pkg.Includes:
> + for inc in sorted(Pkg.Includes, key=lambda x: str(x)):
> for Root, Dirs, Files in os.walk(str(inc)):
> - for File in Files:
> + for File in sorted(Files):
> File_Path = os.path.join(Root, File)
> f = open(File_Path, 'r')
> Content = f.read()
> f.close()
> m.update(Content)
> SaveFileOnChange(HashFile, m.hexdigest(), True)
> - if Pkg.PackageName not in GlobalData.gPackageHash[Pkg.Arch]:
> - GlobalData.gPackageHash[Pkg.Arch][Pkg.PackageName] =
> m.hexdigest()
> + GlobalData.gPackageHash[Pkg.Arch][Pkg.PackageName] =
> + m.hexdigest()
>
> def _GetMetaFiles(self, Target, Toolchain, Arch):
> AllWorkSpaceMetaFiles = set()
> @@ -4432,13 +4435,13 @@ class ModuleAutoGen(AutoGen):
> m.update(GlobalData.gPlatformHash)
> # Add Package level hash
> if self.DependentPackageList:
> - for Pkg in self.DependentPackageList:
> + for Pkg in sorted(self.DependentPackageList, key=lambda x:
> x.PackageName):
> if Pkg.PackageName in GlobalData.gPackageHash[self.Arch]:
>
> m.update(GlobalData.gPackageHash[self.Arch][Pkg.PackageName])
>
> # Add Library hash
> if self.LibraryAutoGenList:
> - for Lib in self.LibraryAutoGenList:
> + for Lib in sorted(self.LibraryAutoGenList, key=lambda x: x.Name):
> if Lib.Name not in GlobalData.gModuleHash[self.Arch]:
> Lib.GenModuleHash()
> m.update(GlobalData.gModuleHash[self.Arch][Lib.Name])
> @@ -4450,7 +4453,7 @@ class ModuleAutoGen(AutoGen):
> m.update(Content)
> # Add Module's source files
> if self.SourceFileList:
> - for File in self.SourceFileList:
> + for File in sorted(self.SourceFileList, key=lambda x: str(x)):
> f = open(str(File), 'r')
> Content = f.read()
> f.close()
> --
> 2.15.1.windows.2
>
>
> _______________________________________________
> edk2-devel mailing list
> edk2-devel@lists.01.org
> https://lists.01.org/mailman/listinfo/edk2-devel
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [PATCH] BaseTools: Fix --hash Package and Module hash value.
2018-05-16 16:20 ` Carsey, Jaben
@ 2018-05-17 2:14 ` Lin, Derek (HPS UEFI Dev)
0 siblings, 0 replies; 7+ messages in thread
From: Lin, Derek (HPS UEFI Dev) @ 2018-05-17 2:14 UTC (permalink / raw)
To: Carsey, Jaben, Zhu, Yonghong, edk2-devel@lists.01.org
Jaben,
You're right, Python list is ordered.
So it's because of insertion order, hmm.. but the meta-file parser run in single thread, not sure why it generate different order in different build...never mind.
However, I like the idea that changing list to set. I should improve performance.
Thanks,
Derek
-----Original Message-----
From: Carsey, Jaben [mailto:jaben.carsey@intel.com]
Sent: Thursday, May 17, 2018 12:20 AM
To: Zhu, Yonghong <yonghong.zhu@intel.com>; Lin, Derek (HPS UEFI Dev) <derek.lin2@hpe.com>; edk2-devel@lists.01.org
Subject: RE: [PATCH] BaseTools: Fix --hash Package and Module hash value.
Derek,
The change is good. I have a comment/question on the message.
Reviewed-by: Jaben Carsey <Jaben.carsey@intel.com>
Order of list enumeration is not arbitrary, it's based on python lists being inherently insertion-ordered objects.
If insertion order is irrelevant, should we consider changing those to a different type? (set?)
-Jaben
> -----Original Message-----
> From: edk2-devel [mailto:edk2-devel-bounces@lists.01.org] On Behalf Of
> Zhu, Yonghong
> Sent: Wednesday, May 16, 2018 12:12 AM
> To: Lin, Derek (HPS UEFI Dev) <derek.lin2@hpe.com>; edk2-
> devel@lists.01.org
> Subject: Re: [edk2] [PATCH] BaseTools: Fix --hash Package and Module
> hash value.
>
> Reviewed-by: Yonghong Zhu <yonghong.zhu@intel.com>
>
> Best Regards,
> Zhu Yonghong
>
>
> -----Original Message-----
> From: Lin, Derek (HPS UEFI Dev) [mailto:derek.lin2@hpe.com]
> Sent: Wednesday, May 09, 2018 5:03 PM
> To: edk2-devel@lists.01.org
> Cc: Zhu, Yonghong <yonghong.zhu@intel.com>; Lin, Derek (HPS UEFI Dev)
> <derek.lin2@hpe.com>
> Subject: [PATCH] BaseTools: Fix --hash Package and Module hash value.
>
> The order of List enumeration is arbitrary.
> Need to be sorted while calculating Package/Module hash, otherwise it
> generate different hash value even nothing changes.
>
> Contributed-under: TianoCore Contribution Agreement 1.1
> Signed-off-by: Derek Lin <derek.lin2@hpe.com>
> ---
> BaseTools/Source/Python/AutoGen/AutoGen.py | 17 ++++++++++-------
> 1 file changed, 10 insertions(+), 7 deletions(-)
>
> diff --git a/BaseTools/Source/Python/AutoGen/AutoGen.py
> b/BaseTools/Source/Python/AutoGen/AutoGen.py
> index 54f6b1f173..90704dcae4 100644
> --- a/BaseTools/Source/Python/AutoGen/AutoGen.py
> +++ b/BaseTools/Source/Python/AutoGen/AutoGen.py
> @@ -2,6 +2,7 @@
> # Generate AutoGen.h, AutoGen.c and *.depex files # # Copyright (c)
> 2007
> - 2018, Intel Corporation. All rights reserved.<BR>
> +# Copyright (c) 2018, Hewlett Packard Enterprise Development,
> +L.P.<BR>
> # This program and the accompanying materials # are licensed and
> made available under the terms and conditions of the BSD License #
> which accompanies this distribution. The full text of the license may
> be found at @@ -670,6 +671,9 @@ class WorkspaceAutoGen(AutoGen):
> return True
>
> def _GenPkgLevelHash(self, Pkg):
> + if Pkg.PackageName in GlobalData.gPackageHash[Pkg.Arch]:
> + return
> +
> PkgDir = os.path.join(self.BuildDir, Pkg.Arch, Pkg.PackageName)
> CreateDirectory(PkgDir)
> HashFile = os.path.join(PkgDir, Pkg.PackageName + '.hash') @@
> -681,17
> +685,16 @@ class WorkspaceAutoGen(AutoGen):
> m.update(Content)
> # Get include files hash value
> if Pkg.Includes:
> - for inc in Pkg.Includes:
> + for inc in sorted(Pkg.Includes, key=lambda x: str(x)):
> for Root, Dirs, Files in os.walk(str(inc)):
> - for File in Files:
> + for File in sorted(Files):
> File_Path = os.path.join(Root, File)
> f = open(File_Path, 'r')
> Content = f.read()
> f.close()
> m.update(Content)
> SaveFileOnChange(HashFile, m.hexdigest(), True)
> - if Pkg.PackageName not in GlobalData.gPackageHash[Pkg.Arch]:
> - GlobalData.gPackageHash[Pkg.Arch][Pkg.PackageName] =
> m.hexdigest()
> + GlobalData.gPackageHash[Pkg.Arch][Pkg.PackageName] =
> + m.hexdigest()
>
> def _GetMetaFiles(self, Target, Toolchain, Arch):
> AllWorkSpaceMetaFiles = set() @@ -4432,13 +4435,13 @@ class
> ModuleAutoGen(AutoGen):
> m.update(GlobalData.gPlatformHash)
> # Add Package level hash
> if self.DependentPackageList:
> - for Pkg in self.DependentPackageList:
> + for Pkg in sorted(self.DependentPackageList, key=lambda x:
> x.PackageName):
> if Pkg.PackageName in GlobalData.gPackageHash[self.Arch]:
>
> m.update(GlobalData.gPackageHash[self.Arch][Pkg.PackageName])
>
> # Add Library hash
> if self.LibraryAutoGenList:
> - for Lib in self.LibraryAutoGenList:
> + for Lib in sorted(self.LibraryAutoGenList, key=lambda x: x.Name):
> if Lib.Name not in GlobalData.gModuleHash[self.Arch]:
> Lib.GenModuleHash()
> m.update(GlobalData.gModuleHash[self.Arch][Lib.Name])
> @@ -4450,7 +4453,7 @@ class ModuleAutoGen(AutoGen):
> m.update(Content)
> # Add Module's source files
> if self.SourceFileList:
> - for File in self.SourceFileList:
> + for File in sorted(self.SourceFileList, key=lambda x: str(x)):
> f = open(str(File), 'r')
> Content = f.read()
> f.close()
> --
> 2.15.1.windows.2
>
>
> _______________________________________________
> edk2-devel mailing list
> edk2-devel@lists.01.org
> https://lists.01.org/mailman/listinfo/edk2-devel
^ permalink raw reply [flat|nested] 7+ messages in thread
end of thread, other threads:[~2018-05-17 2:14 UTC | newest]
Thread overview: 7+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
[not found] <20180509073836.6924-1-derek.lin2@hpe.com>
2018-05-09 7:49 ` [PATCH] BaseTools: Fix --hash Package and Module hash value Lin, Derek (HPS UEFI Dev)
2018-05-09 8:35 ` Zhu, Yonghong
2018-05-09 9:04 ` Lin, Derek (HPS UEFI Dev)
[not found] <20180509085924.13140-1-derek.lin2@hpe.com>
2018-05-09 9:03 ` Lin, Derek (HPS UEFI Dev)
2018-05-16 7:12 ` Zhu, Yonghong
2018-05-16 16:20 ` Carsey, Jaben
2018-05-17 2:14 ` Lin, Derek (HPS UEFI Dev)
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox