[Patch 0/3] BaseTools: Fix some bugs about hash value enable

[Patch 0/3] BaseTools: Fix some bugs about hash value enable

[Yonghong Zhu]

This patches fix some bugs about hash value enable feature.

Yonghong Zhu (3):
  BaseTools: Not cache the .efi file location into build option
  BaseTools: back up the binary files when hash value is same
  BaseTools: enable hash value check for single module build

 BaseTools/Source/Python/AutoGen/AutoGen.py | 25 +++++++++++++++++--------
 BaseTools/Source/Python/build/build.py     | 10 +++++++++-
 2 files changed, 26 insertions(+), 9 deletions(-)

-- 
2.6.1.windows.1

[Patch 1/3] BaseTools: Not cache the .efi file location into build option

[Yonghong Zhu]

We don't need cache the .efi file location into build option, otherwise
when we change the --binary-destination location, it would cause the
hash value is different.

Cc: Liming Gao <liming.gao@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Yonghong Zhu <yonghong.zhu@intel.com>
---
 BaseTools/Source/Python/AutoGen/AutoGen.py | 3 ---
 1 file changed, 3 deletions(-)

diff --git a/BaseTools/Source/Python/AutoGen/AutoGen.py b/BaseTools/Source/Python/AutoGen/AutoGen.py
index 1c4c395..8ad385a 100644
--- a/BaseTools/Source/Python/AutoGen/AutoGen.py
+++ b/BaseTools/Source/Python/AutoGen/AutoGen.py
@@ -684,13 +684,10 @@ class WorkspaceAutoGen(AutoGen):
         content += os.linesep
         if self.FdfFile:
             content += 'Flash Image Definition: '
             content += str(self.FdfFile)
             content += os.linesep
-        if GlobalData.gBinCacheDest:
-            content += 'Cache of .efi location: '
-            content += str(GlobalData.gBinCacheDest)
         SaveFileOnChange(os.path.join(self.BuildDir, 'BuildOptions'), content, False)
 
         #
         # Create PcdToken Number file for Dynamic/DynamicEx Pcd.
         #
-- 
2.6.1.windows.1

[Patch 2/3] BaseTools: back up the binary files when hash value is same

[Yonghong Zhu]

We meet the case that first build with --hash option, then build it
again with --hash and --binary-destination option, since the hash value
is same, tool will not build the driver again, it cause the binary
files are not backed up.

Cc: Liming Gao <liming.gao@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Yonghong Zhu <yonghong.zhu@intel.com>
---
 BaseTools/Source/Python/AutoGen/AutoGen.py | 22 +++++++++++++++++-----
 BaseTools/Source/Python/build/build.py     |  5 +++++
 2 files changed, 22 insertions(+), 5 deletions(-)

diff --git a/BaseTools/Source/Python/AutoGen/AutoGen.py b/BaseTools/Source/Python/AutoGen/AutoGen.py
index 8ad385a..740bd4f 100644
--- a/BaseTools/Source/Python/AutoGen/AutoGen.py
+++ b/BaseTools/Source/Python/AutoGen/AutoGen.py
@@ -2706,21 +2706,21 @@ class ModuleAutoGen(AutoGen):
     #   @param      Target              Build target (DEBUG, RELEASE)
     #   @param      Toolchain           Name of tool chain
     #   @param      Arch                The arch the module supports
     #   @param      PlatformFile        Platform meta-file
     #
-    def _Init(self, Workspace, ModuleFile, Target, Toolchain, Arch, PlatformFile):
+    def _Init(self, Workspace, ModuleFile, Target, Toolchain, Arch, PlatformFile, Flag=False):
         EdkLogger.debug(EdkLogger.DEBUG_9, "AutoGen module [%s] [%s]" % (ModuleFile, Arch))
         GlobalData.gProcessingFile = "%s [%s, %s, %s]" % (ModuleFile, Arch, Toolchain, Target)
 
         self.Workspace = Workspace
         self.WorkspaceDir = Workspace.WorkspaceDir
-
+        self.Flag = Flag
         self.MetaFile = ModuleFile
         self.PlatformInfo = PlatformAutoGen(Workspace, PlatformFile, Target, Toolchain, Arch)
         # check if this module is employed by active platform
-        if not self.PlatformInfo.ValidModule(self.MetaFile):
+        if not self.Flag and not self.PlatformInfo.ValidModule(self.MetaFile):
             EdkLogger.verbose("Module [%s] for [%s] is not employed by active platform\n" \
                               % (self.MetaFile, Arch))
             return False
 
         self.SourceDir = self.MetaFile.SubDir
@@ -2828,10 +2828,13 @@ class ModuleAutoGen(AutoGen):
                 
         return self._FixedAtBuildPcds        
 
     def _GetUniqueBaseName(self):
         BaseName = self.Name
+        # when self.Flag is true, the module is Asbuilt inf in the build output
+        if self.Flag:
+            return BaseName
         for Module in self.PlatformInfo.ModuleAutoGenList:
             if Module.MetaFile == self.MetaFile:
                 continue
             if Module.Name == self.Name:
                 if uuid.UUID(Module.Guid) == uuid.UUID(self.Guid):
@@ -3836,11 +3839,17 @@ class ModuleAutoGen(AutoGen):
         fInputfile.close ()
         return OutputName
 
     ## Create AsBuilt INF file the module
     #
-    def CreateAsBuiltInf(self):
+    def CreateAsBuiltInf(self, IsOnlyCopy = False):
+        self.OutputFile = []
+        if IsOnlyCopy:
+            if GlobalData.gBinCacheDest:
+                self.CopyModuleToCache()
+                return
+
         if self.IsAsBuiltInfCreated:
             return
             
         # Skip the following code for EDK I inf
         if self.AutoGenVersion < 0x00010005:
@@ -3969,11 +3978,10 @@ class ModuleAutoGen(AutoGen):
             AsBuiltInfDict['module_uefi_specification_version'] += [self.Specification['UEFI_SPECIFICATION_VERSION']]
         if 'PI_SPECIFICATION_VERSION' in self.Specification:
             AsBuiltInfDict['module_pi_specification_version'] += [self.Specification['PI_SPECIFICATION_VERSION']]
 
         OutputDir = self.OutputDir.replace('\\', '/').strip('/')
-        self.OutputFile = []
         for Item in self.CodaTargetList:
             File = Item.Target.Path.replace('\\', '/').strip('/').replace(OutputDir, '').strip('/')
             if File not in self.OutputFile:
                 self.OutputFile.append(File)
             if Item.Target.Ext.lower() == '.aml':
@@ -4196,12 +4204,16 @@ class ModuleAutoGen(AutoGen):
         ModuleFile = path.join(self.OutputDir, self.Name + '.inf')
         if os.path.exists(HashFile):
             shutil.copy2(HashFile, FileDir)
         if os.path.exists(ModuleFile):
             shutil.copy2(ModuleFile, FileDir)
+        if not self.OutputFile:
+            Ma = ModuleAutoGen(self.Workspace, PathClass(ModuleFile), self.BuildTarget, self.ToolChain, self.Arch, self.Workspace.Platform.MetaFile, True)
+            self.OutputFile = Ma.BinaryFileList
         if self.OutputFile:
             for File in self.OutputFile:
+                File = str(File)
                 if not os.path.isabs(File):
                     File = os.path.join(self.OutputDir, File)
                 if os.path.exists(File):
                     shutil.copy2(File, FileDir)
 
diff --git a/BaseTools/Source/Python/build/build.py b/BaseTools/Source/Python/build/build.py
index f94285a..457f6c9 100644
--- a/BaseTools/Source/Python/build/build.py
+++ b/BaseTools/Source/Python/build/build.py
@@ -823,10 +823,11 @@ class Build():
         self.Platform = None
         self.ToolChainFamily = None
         self.LoadFixAddress = 0
         self.UniFlag        = BuildOptions.Flag
         self.BuildModules = []
+        self.HashSkipModules = []
         self.Db_Flag = False
         self.LaunchPrebuildFlag = False
         self.PlatformBuildPath = os.path.join(GlobalData.gConfDirectory,'.cache', '.PlatformBuild')
         if BuildOptions.CommandLength:
             GlobalData.gCommandMaxLength = BuildOptions.CommandLength
@@ -2014,10 +2015,11 @@ class Build():
                         Ma = ModuleAutoGen(Wa, Module, BuildTarget, ToolChain, Arch, self.PlatformFile)
                         
                         if Ma == None:
                             continue
                         if Ma.CanSkipbyHash():
+                            self.HashSkipModules.append(Ma)
                             continue
 
                         # Not to auto-gen for targets 'clean', 'cleanlib', 'cleanall', 'run', 'fds'
                         if self.Target not in ['clean', 'cleanlib', 'cleanall', 'run', 'fds']:
                             # for target which must generate AutoGen code and makefile
@@ -2212,11 +2214,14 @@ class Build():
             RemoveDirectory(os.path.dirname(GlobalData.gDatabasePath), True)
 
     def CreateAsBuiltInf(self):
         for Module in self.BuildModules:
             Module.CreateAsBuiltInf()
+        for Module in self.HashSkipModules:
+            Module.CreateAsBuiltInf(True)
         self.BuildModules = []
+        self.HashSkipModules = []
     ## Do some clean-up works when error occurred
     def Relinquish(self):
         OldLogLevel = EdkLogger.GetLevel()
         EdkLogger.SetLevel(EdkLogger.ERROR)
         #self.DumpBuildData()
-- 
2.6.1.windows.1

[Patch 3/3] BaseTools: enable hash value check for single module build

[Yonghong Zhu]

This patch enables hash value check for single module build to decide
whether we can skip to build this module.

Cc: Liming Gao <liming.gao@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Yonghong Zhu <yonghong.zhu@intel.com>
---
 BaseTools/Source/Python/build/build.py | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)

diff --git a/BaseTools/Source/Python/build/build.py b/BaseTools/Source/Python/build/build.py
index 457f6c9..8cf139c 100644
--- a/BaseTools/Source/Python/build/build.py
+++ b/BaseTools/Source/Python/build/build.py
@@ -1843,10 +1843,14 @@ class Build():
                     Pa = PlatformAutoGen(Wa, self.PlatformFile, BuildTarget, ToolChain, Arch)
                     for Module in Pa.Platform.Modules:
                         if self.ModuleFile.Dir == Module.Dir and self.ModuleFile.Name == Module.Name:
                             Ma = ModuleAutoGen(Wa, Module, BuildTarget, ToolChain, Arch, self.PlatformFile)
                             if Ma == None: continue
+                            MaList.append(Ma)
+                            if Ma.CanSkipbyHash():
+                                self.HashSkipModules.append(Ma)
+                                continue
                             # Not to auto-gen for targets 'clean', 'cleanlib', 'cleanall', 'run', 'fds'
                             if self.Target not in ['clean', 'cleanlib', 'cleanall', 'run', 'fds']:
                                 # for target which must generate AutoGen code and makefile
                                 if not self.SkipAutoGen or self.Target == 'genc':
                                     Ma.CreateCodeFile(True)
@@ -1854,11 +1858,10 @@ class Build():
                                     if CmdListDict and self.Fdf and (Module.File, Arch) in CmdListDict:
                                         Ma.CreateMakeFile(True, CmdListDict[Module.File, Arch])
                                         del CmdListDict[Module.File, Arch]
                                     else:
                                         Ma.CreateMakeFile(True)
-                            MaList.append(Ma)
                             self.BuildModules.append(Ma)
                     self.AutoGenTime += int(round((time.time() - AutoGenStart)))
                     MakeStart = time.time()
                     for Ma in self.BuildModules:
                         if not Ma.IsBinaryModule:
-- 
2.6.1.windows.1

Re: [Patch 0/3] BaseTools: Fix some bugs about hash value enable

[Gao, Liming]

Reviewed-by: Liming Gao <liming.gao@intel.com>

>-----Original Message-----
>From: edk2-devel [mailto:edk2-devel-bounces@lists.01.org] On Behalf Of
>Yonghong Zhu
>Sent: Wednesday, December 06, 2017 6:13 PM
>To: edk2-devel@lists.01.org
>Subject: [edk2] [Patch 0/3] BaseTools: Fix some bugs about hash value enable
>
>This patches fix some bugs about hash value enable feature.
>
>Yonghong Zhu (3):
>  BaseTools: Not cache the .efi file location into build option
>  BaseTools: back up the binary files when hash value is same
>  BaseTools: enable hash value check for single module build
>
> BaseTools/Source/Python/AutoGen/AutoGen.py | 25 +++++++++++++++++-
>-------
> BaseTools/Source/Python/build/build.py     | 10 +++++++++-
> 2 files changed, 26 insertions(+), 9 deletions(-)
>
>--
>2.6.1.windows.1
>
>_______________________________________________
>edk2-devel mailing list
>edk2-devel@lists.01.org
>https://lists.01.org/mailman/listinfo/edk2-devel