[PATCH v3 0/1] BaseTools: Cannot store library cache of different arch together

[PATCH v3 0/1] BaseTools: Cannot store library cache of different arch together

[Steven Shi]

V3:
WorkspaceAutoGen does not have the Arch attribute and current Meta-file
of active platform already can make sure the __hash__ value is unique for
WorkspaceAutoGen object. So, only add adds the arch string into the
PlatformAutoGen and ModuleAutoGen __hash_ definitions.

V2:
Follow the Christian suggestion, Still keep the set() usage, but instead to
enahnce the AutoGen object __hash__ definition to be unique by including
the AutoGen object arch string into the __hash_ values.

V1:
Avoid to use the set() as the container to save the library and module objects

Steven Shi (1):
  BaseTools: Cannot store library cache of different arch together

 BaseTools/Source/Python/AutoGen/AutoGen.py | 21 +++++++++++++++++++++
 1 file changed, 21 insertions(+)

-- 
2.17.1.windows.2

[PATCH v3 1/1] BaseTools: Cannot store library cache of different arch together

[Steven Shi]

https://bugzilla.tianocore.org/show_bug.cgi?id=1895

Build cache cannot store cache for the same library modules
in different arch together. E.g. Both the below IA32 and X64
arch BaseLib caches should exist after build Ovmf3264, but now
only the one in X64 arch exist.
The reason is the current Basetool use a set() to same all
library AutoGen objects, but the different arch lib AutoGen
objects have same __hash_ value which comes from the lib
MetaFile(The path of module file):
    def __hash__(self):
        return hash(self.MetaFile)

So the different arch lib AutoGen objects are duplicated one
to the set() and only one can exist. This is why the Basetool
can only store one arch cache for library.

This patch adds the arch string into the PlatformAutoGen and
ModuleAutoGen __hash_ definitions and ensure the different
platform and module AutoGen objects have different __hash_ values.

Cc: Liming Gao <liming.gao@intel.com>
Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Christian Rodriguez <christian.rodriguez@intel.com>
Signed-off-by: Steven Shi <steven.shi@intel.com>
---
 BaseTools/Source/Python/AutoGen/AutoGen.py | 21 +++++++++++++++++++++
 1 file changed, 21 insertions(+)

diff --git a/BaseTools/Source/Python/AutoGen/AutoGen.py b/BaseTools/Source/Python/AutoGen/AutoGen.py
index 3f41fbb507..79203d8105 100644
--- a/BaseTools/Source/Python/AutoGen/AutoGen.py
+++ b/BaseTools/Source/Python/AutoGen/AutoGen.py
@@ -1166,6 +1166,17 @@ class PlatformAutoGen(AutoGen):
 
         return True
 
+    ## hash() operator of PlatformAutoGen
+    #
+    #  The platform file path and arch string will be used to represent
+    #  hash value of this object
+    #
+    #   @retval   int Hash value of the platform file path and arch
+    #
+    @cached_class_function
+    def __hash__(self):
+        return hash((self.MetaFile, self.Arch))
+
     @cached_class_function
     def __repr__(self):
         return "%s [%s]" % (self.MetaFile, self.Arch)
@@ -2579,6 +2590,16 @@ class ModuleAutoGen(AutoGen):
         self.ReferenceModules = []
         self.ConstPcd                  = {}
 
+    ## hash() operator of ModuleAutoGen
+    #
+    #  The module file path and arch string will be used to represent
+    #  hash value of this object
+    #
+    #   @retval   int Hash value of the module file path and arch
+    #
+    @cached_class_function
+    def __hash__(self):
+        return hash((self.MetaFile, self.Arch))
 
     def __repr__(self):
         return "%s [%s]" % (self.MetaFile, self.Arch)
-- 
2.17.1.windows.2

Re: [PATCH v3 1/1] BaseTools: Cannot store library cache of different arch together

[Bob Feng]

Reviewed-by: Bob Feng <bob.c.feng@intel.com> 

-----Original Message-----
From: Shi, Steven 
Sent: Monday, June 17, 2019 4:16 PM
To: devel@edk2.groups.io
Cc: Gao, Liming <liming.gao@intel.com>; Feng, Bob C <bob.c.feng@intel.com>; Rodriguez, Christian <christian.rodriguez@intel.com>
Subject: [PATCH v3 1/1] BaseTools: Cannot store library cache of different arch together

https://bugzilla.tianocore.org/show_bug.cgi?id=1895

Build cache cannot store cache for the same library modules in different arch together. E.g. Both the below IA32 and X64 arch BaseLib caches should exist after build Ovmf3264, but now only the one in X64 arch exist.
The reason is the current Basetool use a set() to same all library AutoGen objects, but the different arch lib AutoGen objects have same __hash_ value which comes from the lib MetaFile(The path of module file):
    def __hash__(self):
        return hash(self.MetaFile)

So the different arch lib AutoGen objects are duplicated one to the set() and only one can exist. This is why the Basetool can only store one arch cache for library.

This patch adds the arch string into the PlatformAutoGen and ModuleAutoGen __hash_ definitions and ensure the different platform and module AutoGen objects have different __hash_ values.

Cc: Liming Gao <liming.gao@intel.com>
Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Christian Rodriguez <christian.rodriguez@intel.com>
Signed-off-by: Steven Shi <steven.shi@intel.com>
---
 BaseTools/Source/Python/AutoGen/AutoGen.py | 21 +++++++++++++++++++++
 1 file changed, 21 insertions(+)

diff --git a/BaseTools/Source/Python/AutoGen/AutoGen.py b/BaseTools/Source/Python/AutoGen/AutoGen.py
index 3f41fbb507..79203d8105 100644
--- a/BaseTools/Source/Python/AutoGen/AutoGen.py
+++ b/BaseTools/Source/Python/AutoGen/AutoGen.py
@@ -1166,6 +1166,17 @@ class PlatformAutoGen(AutoGen):
 
         return True
 
+    ## hash() operator of PlatformAutoGen
+    #
+    #  The platform file path and arch string will be used to represent
+    #  hash value of this object
+    #
+    #   @retval   int Hash value of the platform file path and arch
+    #
+    @cached_class_function
+    def __hash__(self):
+        return hash((self.MetaFile, self.Arch))
+
     @cached_class_function
     def __repr__(self):
         return "%s [%s]" % (self.MetaFile, self.Arch) @@ -2579,6 +2590,16 @@ class ModuleAutoGen(AutoGen):
         self.ReferenceModules = []
         self.ConstPcd                  = {}
 
+    ## hash() operator of ModuleAutoGen
+    #
+    #  The module file path and arch string will be used to represent
+    #  hash value of this object
+    #
+    #   @retval   int Hash value of the module file path and arch
+    #
+    @cached_class_function
+    def __hash__(self):
+        return hash((self.MetaFile, self.Arch))
 
     def __repr__(self):
         return "%s [%s]" % (self.MetaFile, self.Arch)
--
2.17.1.windows.2