[PATCH v3 0/1] BaseTools:Extend the binary cache to support library

[PATCH v3 0/1] BaseTools:Extend the binary cache to support library

[Steven Shi]

V3 change:
Update the commit message and move the change notices in cover letter

V2 change:
Simplify the library cache implementation and remove
CacheRestored flag based filter. Use the current
CanSkipbyHash() filter in AddDependency() instead

V1: Initial version to enable the library cache

Steven Shi (1):
  BaseTools:Extend the binary cache to support library cache

 BaseTools/Source/Python/AutoGen/AutoGen.py | 6 ++++++
 1 file changed, 6 insertions(+)

-- 
2.17.1.windows.2

[PATCH v3 1/1] BaseTools:Extend the binary cache to support library cache

[Steven Shi]

https://bugzilla.tianocore.org/show_bug.cgi?id=1797

Current binary cache doesn't support to save and restore
the library module. If a driver module cache miss happen,
all its dependency library modules need rebuild which
is very time-consuming. This patch is to entend the binary
cache to support library.

Cc: Liming Gao <liming.gao@intel.com>
Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Christian Rodriguez <christian.rodriguez@intel.com>
Signed-off-by: Steven Shi <steven.shi@intel.com>
---
 BaseTools/Source/Python/AutoGen/AutoGen.py | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/BaseTools/Source/Python/AutoGen/AutoGen.py b/BaseTools/Source/Python/AutoGen/AutoGen.py
index a5bef4f7c6..7b35f837f5 100644
--- a/BaseTools/Source/Python/AutoGen/AutoGen.py
+++ b/BaseTools/Source/Python/AutoGen/AutoGen.py
@@ -3906,6 +3906,12 @@ class ModuleAutoGen(AutoGen):
             ModuleFile = path.join(self.OutputDir, self.Name + '.inf')
             if os.path.exists(ModuleFile):
                 shutil.copy2(ModuleFile, FileDir)
+        else:
+            OutputDir = self.OutputDir.replace('\\', '/').strip('/')
+            DebugDir = self.DebugDir.replace('\\', '/').strip('/')
+            for Item in self.CodaTargetList:
+                File = Item.Target.Path.replace('\\', '/').strip('/').replace(DebugDir, '').replace(OutputDir, '').strip('/')
+                self.OutputFile.add(File)
         if not self.OutputFile:
             Ma = self.BuildDatabase[self.MetaFile, self.Arch, self.BuildTarget, self.ToolChain]
             self.OutputFile = Ma.Binaries
-- 
2.17.1.windows.2

Re: [edk2-devel] [PATCH v3 1/1] BaseTools:Extend the binary cache to support library cache

[Bob Feng]

The "entend" should be "extend", right? I can fix it when I push this patch.

Reviewed-by: Bob Feng <bob.c.feng@intel.com>

-----Original Message-----
From: devel@edk2.groups.io [mailto:devel@edk2.groups.io] On Behalf Of Steven Shi
Sent: Thursday, May 30, 2019 10:31 AM
To: devel@edk2.groups.io
Cc: Gao, Liming <liming.gao@intel.com>; Feng, Bob C <bob.c.feng@intel.com>; Rodriguez, Christian <christian.rodriguez@intel.com>; Fan, ZhijuX <zhijux.fan@intel.com>
Subject: [edk2-devel] [PATCH v3 1/1] BaseTools:Extend the binary cache to support library cache

https://bugzilla.tianocore.org/show_bug.cgi?id=1797

Current binary cache doesn't support to save and restore the library module. If a driver module cache miss happen, all its dependency library modules need rebuild which is very time-consuming. This patch is to entend the binary cache to support library.

Cc: Liming Gao <liming.gao@intel.com>
Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Christian Rodriguez <christian.rodriguez@intel.com>
Signed-off-by: Steven Shi <steven.shi@intel.com>
---
 BaseTools/Source/Python/AutoGen/AutoGen.py | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/BaseTools/Source/Python/AutoGen/AutoGen.py b/BaseTools/Source/Python/AutoGen/AutoGen.py
index a5bef4f7c6..7b35f837f5 100644
--- a/BaseTools/Source/Python/AutoGen/AutoGen.py
+++ b/BaseTools/Source/Python/AutoGen/AutoGen.py
@@ -3906,6 +3906,12 @@ class ModuleAutoGen(AutoGen):
             ModuleFile = path.join(self.OutputDir, self.Name + '.inf')
             if os.path.exists(ModuleFile):
                 shutil.copy2(ModuleFile, FileDir)
+        else:
+            OutputDir = self.OutputDir.replace('\\', '/').strip('/')
+            DebugDir = self.DebugDir.replace('\\', '/').strip('/')
+            for Item in self.CodaTargetList:
+                File = Item.Target.Path.replace('\\', '/').strip('/').replace(DebugDir, '').replace(OutputDir, '').strip('/')
+                self.OutputFile.add(File)
         if not self.OutputFile:
             Ma = self.BuildDatabase[self.MetaFile, self.Arch, self.BuildTarget, self.ToolChain]
             self.OutputFile = Ma.Binaries
--
2.17.1.windows.2

Re: [edk2-devel] [PATCH v3 1/1] BaseTools:Extend the binary cache to support library cache

[Steven Shi]

Yes. The "entend" should be "extend", please help to fix it when pushing. Thanks!

Steven Shi


> -----Original Message-----
> From: Feng, Bob C
> Sent: Thursday, May 30, 2019 10:44 AM
> To: devel@edk2.groups.io; Shi, Steven <steven.shi@intel.com>
> Cc: Gao, Liming <liming.gao@intel.com>; Rodriguez, Christian
> <christian.rodriguez@intel.com>; Fan, ZhijuX <zhijux.fan@intel.com>
> Subject: RE: [edk2-devel] [PATCH v3 1/1] BaseTools:Extend the binary cache to
> support library cache
> 
> The "entend" should be "extend", right? I can fix it when I push this patch.
> 
> Reviewed-by: Bob Feng <bob.c.feng@intel.com>
> 
> -----Original Message-----
> From: devel@edk2.groups.io [mailto:devel@edk2.groups.io] On Behalf Of
> Steven Shi
> Sent: Thursday, May 30, 2019 10:31 AM
> To: devel@edk2.groups.io
> Cc: Gao, Liming <liming.gao@intel.com>; Feng, Bob C <bob.c.feng@intel.com>;
> Rodriguez, Christian <christian.rodriguez@intel.com>; Fan, ZhijuX
> <zhijux.fan@intel.com>
> Subject: [edk2-devel] [PATCH v3 1/1] BaseTools:Extend the binary cache to
> support library cache
> 
> https://bugzilla.tianocore.org/show_bug.cgi?id=1797
> 
> Current binary cache doesn't support to save and restore the library module.
> If a driver module cache miss happen, all its dependency library modules
> need rebuild which is very time-consuming. This patch is to entend the binary
> cache to support library.
> 
> Cc: Liming Gao <liming.gao@intel.com>
> Cc: Bob Feng <bob.c.feng@intel.com>
> Cc: Christian Rodriguez <christian.rodriguez@intel.com>
> Signed-off-by: Steven Shi <steven.shi@intel.com>
> ---
>  BaseTools/Source/Python/AutoGen/AutoGen.py | 6 ++++++
>  1 file changed, 6 insertions(+)
> 
> diff --git a/BaseTools/Source/Python/AutoGen/AutoGen.py
> b/BaseTools/Source/Python/AutoGen/AutoGen.py
> index a5bef4f7c6..7b35f837f5 100644
> --- a/BaseTools/Source/Python/AutoGen/AutoGen.py
> +++ b/BaseTools/Source/Python/AutoGen/AutoGen.py
> @@ -3906,6 +3906,12 @@ class ModuleAutoGen(AutoGen):
>              ModuleFile = path.join(self.OutputDir, self.Name + '.inf')
>              if os.path.exists(ModuleFile):
>                  shutil.copy2(ModuleFile, FileDir)
> +        else:
> +            OutputDir = self.OutputDir.replace('\\', '/').strip('/')
> +            DebugDir = self.DebugDir.replace('\\', '/').strip('/')
> +            for Item in self.CodaTargetList:
> +                File = Item.Target.Path.replace('\\', '/').strip('/').replace(DebugDir,
> '').replace(OutputDir, '').strip('/')
> +                self.OutputFile.add(File)
>          if not self.OutputFile:
>              Ma = self.BuildDatabase[self.MetaFile, self.Arch, self.BuildTarget,
> self.ToolChain]
>              self.OutputFile = Ma.Binaries
> --
> 2.17.1.windows.2
> 
> 
>