[PATCH v2] BaseTools:Extend the binary cache to support library cache

[PATCH v2] BaseTools:Extend the binary cache to support library cache

[Steven Shi]

V2 change:
Simplify the library cache implementation and remove
CacheRestored flag based filter. Use the current
CanSkipbyHash() filter in AddDependency() instead.

V1 change:

https://bugzilla.tianocore.org/show_bug.cgi?id=1797

Current binary cache doesn't support to save and restore
the library module. If a driver module cache miss happen,
all its dependency library modules need rebuild which
is very time-consuming. This patch is to entend the binary
cache to support library.

Cc: Liming Gao <liming.gao@intel.com>
Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Christian Rodriguez <christian.rodriguez@intel.com>
Signed-off-by: Steven Shi <steven.shi@intel.com>
---
 BaseTools/Source/Python/AutoGen/AutoGen.py | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/BaseTools/Source/Python/AutoGen/AutoGen.py b/BaseTools/Source/Python/AutoGen/AutoGen.py
index a5bef4f7c6..7b35f837f5 100644
--- a/BaseTools/Source/Python/AutoGen/AutoGen.py
+++ b/BaseTools/Source/Python/AutoGen/AutoGen.py
@@ -3906,6 +3906,12 @@ class ModuleAutoGen(AutoGen):
             ModuleFile = path.join(self.OutputDir, self.Name + '.inf')
             if os.path.exists(ModuleFile):
                 shutil.copy2(ModuleFile, FileDir)
+        else:
+            OutputDir = self.OutputDir.replace('\\', '/').strip('/')
+            DebugDir = self.DebugDir.replace('\\', '/').strip('/')
+            for Item in self.CodaTargetList:
+                File = Item.Target.Path.replace('\\', '/').strip('/').replace(DebugDir, '').replace(OutputDir, '').strip('/')
+                self.OutputFile.add(File)
         if not self.OutputFile:
             Ma = self.BuildDatabase[self.MetaFile, self.Arch, self.BuildTarget, self.ToolChain]
             self.OutputFile = Ma.Binaries
-- 
2.17.1.windows.2

Re: [edk2-devel] [PATCH v2] BaseTools:Extend the binary cache to support library cache

[Leif Lindholm]

On Wed, May 29, 2019 at 09:46:54AM +0800, Steven Shi wrote:
> V2 change:
> Simplify the library cache implementation and remove
> CacheRestored flag based filter. Use the current
> CanSkipbyHash() filter in AddDependency() instead.
> 
> V1 change:

Please don't put change notices in the commit message. These go below
---, or in a cover letter.

/
    Leif

> 
> https://bugzilla.tianocore.org/show_bug.cgi?id=1797
> 
> Current binary cache doesn't support to save and restore
> the library module. If a driver module cache miss happen,
> all its dependency library modules need rebuild which
> is very time-consuming. This patch is to entend the binary
> cache to support library.
> 
> Cc: Liming Gao <liming.gao@intel.com>
> Cc: Bob Feng <bob.c.feng@intel.com>
> Cc: Christian Rodriguez <christian.rodriguez@intel.com>
> Signed-off-by: Steven Shi <steven.shi@intel.com>
> ---
>  BaseTools/Source/Python/AutoGen/AutoGen.py | 6 ++++++
>  1 file changed, 6 insertions(+)
> 
> diff --git a/BaseTools/Source/Python/AutoGen/AutoGen.py b/BaseTools/Source/Python/AutoGen/AutoGen.py
> index a5bef4f7c6..7b35f837f5 100644
> --- a/BaseTools/Source/Python/AutoGen/AutoGen.py
> +++ b/BaseTools/Source/Python/AutoGen/AutoGen.py
> @@ -3906,6 +3906,12 @@ class ModuleAutoGen(AutoGen):
>              ModuleFile = path.join(self.OutputDir, self.Name + '.inf')
>              if os.path.exists(ModuleFile):
>                  shutil.copy2(ModuleFile, FileDir)
> +        else:
> +            OutputDir = self.OutputDir.replace('\\', '/').strip('/')
> +            DebugDir = self.DebugDir.replace('\\', '/').strip('/')
> +            for Item in self.CodaTargetList:
> +                File = Item.Target.Path.replace('\\', '/').strip('/').replace(DebugDir, '').replace(OutputDir, '').strip('/')
> +                self.OutputFile.add(File)
>          if not self.OutputFile:
>              Ma = self.BuildDatabase[self.MetaFile, self.Arch, self.BuildTarget, self.ToolChain]
>              self.OutputFile = Ma.Binaries
> -- 
> 2.17.1.windows.2
> 
> 
> 
> 

Re: [edk2-devel] [PATCH v2] BaseTools:Extend the binary cache to support library cache

[Steven Shi]

OK, will move the change notices from commit message to cover letter.


Thanks
Steven Shi

> -----Original Message-----
> From: Leif Lindholm [mailto:leif.lindholm@linaro.org]
> Sent: Wednesday, May 29, 2019 7:14 PM
> To: devel@edk2.groups.io; Shi, Steven <steven.shi@intel.com>
> Cc: Gao, Liming <liming.gao@intel.com>; Feng, Bob C <bob.c.feng@intel.com>;
> Rodriguez, Christian <christian.rodriguez@intel.com>; Fan, ZhijuX
> <zhijux.fan@intel.com>
> Subject: Re: [edk2-devel] [PATCH v2] BaseTools:Extend the binary cache to
> support library cache
> 
> On Wed, May 29, 2019 at 09:46:54AM +0800, Steven Shi wrote:
> > V2 change:
> > Simplify the library cache implementation and remove
> > CacheRestored flag based filter. Use the current
> > CanSkipbyHash() filter in AddDependency() instead.
> >
> > V1 change:
> 
> Please don't put change notices in the commit message. These go below
> ---, or in a cover letter.
> 
> /
>     Leif
> 
> >
> > https://bugzilla.tianocore.org/show_bug.cgi?id=1797
> >
> > Current binary cache doesn't support to save and restore
> > the library module. If a driver module cache miss happen,
> > all its dependency library modules need rebuild which
> > is very time-consuming. This patch is to entend the binary
> > cache to support library.
> >
> > Cc: Liming Gao <liming.gao@intel.com>
> > Cc: Bob Feng <bob.c.feng@intel.com>
> > Cc: Christian Rodriguez <christian.rodriguez@intel.com>
> > Signed-off-by: Steven Shi <steven.shi@intel.com>
> > ---
> >  BaseTools/Source/Python/AutoGen/AutoGen.py | 6 ++++++
> >  1 file changed, 6 insertions(+)
> >
> > diff --git a/BaseTools/Source/Python/AutoGen/AutoGen.py
> b/BaseTools/Source/Python/AutoGen/AutoGen.py
> > index a5bef4f7c6..7b35f837f5 100644
> > --- a/BaseTools/Source/Python/AutoGen/AutoGen.py
> > +++ b/BaseTools/Source/Python/AutoGen/AutoGen.py
> > @@ -3906,6 +3906,12 @@ class ModuleAutoGen(AutoGen):
> >              ModuleFile = path.join(self.OutputDir, self.Name + '.inf')
> >              if os.path.exists(ModuleFile):
> >                  shutil.copy2(ModuleFile, FileDir)
> > +        else:
> > +            OutputDir = self.OutputDir.replace('\\', '/').strip('/')
> > +            DebugDir = self.DebugDir.replace('\\', '/').strip('/')
> > +            for Item in self.CodaTargetList:
> > +                File = Item.Target.Path.replace('\\', '/').strip('/').replace(DebugDir,
> '').replace(OutputDir, '').strip('/')
> > +                self.OutputFile.add(File)
> >          if not self.OutputFile:
> >              Ma = self.BuildDatabase[self.MetaFile, self.Arch, self.BuildTarget,
> self.ToolChain]
> >              self.OutputFile = Ma.Binaries
> > --
> > 2.17.1.windows.2
> >
> >
> > 
> >