Re: [PATCH V6] BaseTools:Add the spare space FV image size checker

["Liming Gao" <liming.gao@intel.com>]

Zhiju:
  With this improvement, Reviewed-by: Liming Gao <liming.gao@intel.com>

> -----Original Message-----
> From: Fan, ZhijuX <zhijux.fan@intel.com>
> Sent: Monday, April 20, 2020 9:17 AM
> To: Gao, Liming <liming.gao@intel.com>; devel@edk2.groups.io
> Cc: Feng, Bob C <bob.c.feng@intel.com>
> Subject: RE: [PATCH V6] BaseTools:Add the spare space FV image size checker
> 
> Hi LiMing,
> 
> I agree with this change and will improve it
> 
> 
> 
> Any question, please let me know. Thanks.
> 
> Best Regards
> Fan Zhiju
> 
> 
> 
> > -----Original Message-----
> > From: Gao, Liming <liming.gao@intel.com>
> > Sent: Wednesday, April 15, 2020 9:21 PM
> > To: Fan, ZhijuX <zhijux.fan@intel.com>; devel@edk2.groups.io
> > Cc: Feng, Bob C <bob.c.feng@intel.com>
> > Subject: RE: [PATCH V6] BaseTools:Add the spare space FV image size checker
> >
> > Zhiju:
> >   Thanks for your update. The change is good to me. For the report error
> > message, I suggest as below
> >
> > xx FV free space xxxx is not enough to meet with the required spare space xxxx
> > set by -D FV_SPARE_SPACE_THRESHOLD option.
> >
> > Thanks
> > Liming
> > > -----Original Message-----
> > > From: Fan, ZhijuX <zhijux.fan@intel.com>
> > > Sent: Wednesday, April 15, 2020 7:49 PM
> > > To: devel@edk2.groups.io
> > > Cc: Gao, Liming <liming.gao@intel.com>; Feng, Bob C
> > > <bob.c.feng@intel.com>
> > > Subject: [PATCH V6] BaseTools:Add the spare space FV image size
> > > checker
> > >
> > > REF:https://bugzilla.tianocore.org/show_bug.cgi?id=2654
> > >
> > > If FV is placed in FD region, its FV image size is fixed.
> > > When FV image size exceeds it, it will trig the build break.
> > > To alert the developer to adjust FV image size earlier, I request to
> > > add new checker for the the spare FV space.
> > > When the spare FV space is less than the specified threshold, build
> > > tool will report the error.
> > >
> > > This checker is the optional.
> > > It can be enabled by -D FV_SPARE_SPACE_THRESHOLD=10000.
> > > Macro is the value of the spare space threshold size.
> > > It can be decimal or hex format. If it is enabled, BaseTools will
> > > check every FV with the fixed size.
> > > If FV doesn't meet with the size requirement, Build tool will report
> > > error message to say there is no enough spare space.
> > >
> > > Cc: Liming Gao <liming.gao@intel.com>
> > > Cc: Bob Feng <bob.c.feng@intel.com>
> > > Signed-off-by: Zhiju.Fan <zhijux.fan@intel.com>
> > > ---
> > > changed the error message
> > >
> > >  BaseTools/Source/Python/Common/BuildToolError.py |  2 +
> > >  BaseTools/Source/Python/build/build.py           | 47 ++++++++++
> > >  2 files changed, 49 insertions(+)
> > >
> > > diff --git a/BaseTools/Source/Python/Common/BuildToolError.py
> > > b/BaseTools/Source/Python/Common/BuildToolError.py
> > > index ecc83d0f48bd..21549683cd19 100644
> > > --- a/BaseTools/Source/Python/Common/BuildToolError.py
> > > +++ b/BaseTools/Source/Python/Common/BuildToolError.py
> > > @@ -64,6 +64,8 @@ COMMAND_FAILURE = 0x7000
> > >
> > >  PERMISSION_FAILURE = 0x8000
> > >
> > > +FV_FREESIZE_ERROR = 0x9000
> > > +
> > >  CODE_ERROR = 0xC0DE
> > >
> > >  AUTOGEN_ERROR = 0xF000
> > > diff --git a/BaseTools/Source/Python/build/build.py
> > > b/BaseTools/Source/Python/build/build.py
> > > index bec848a7b2e3..68f5b8cabea3 100755
> > > --- a/BaseTools/Source/Python/build/build.py
> > > +++ b/BaseTools/Source/Python/build/build.py
> > > @@ -25,6 +25,7 @@ import traceback
> > >  import multiprocessing
> > >  from threading import Thread,Event,BoundedSemaphore  import threading
> > > +from linecache import getlines
> > >  from subprocess import Popen,PIPE, STDOUT  from collections import
> > > OrderedDict, defaultdict
> > >
> > > @@ -1413,6 +1414,9 @@ class Build():
> > >          if Target == 'fds':
> > >              if GenFdsApi(AutoGenObject.GenFdsCommandDict, self.Db):
> > >                  EdkLogger.error("build", COMMAND_FAILURE)
> > > +            Threshold = self.GetFreeSizeThreshold()
> > > +            if Threshold:
> > > +                self.CheckFreeSizeThreshold(Threshold,
> > > + AutoGenObject.FvDir)
> > >              return True
> > >
> > >          # run
> > > @@ -2311,6 +2315,9 @@ class Build():
> > >                          GenFdsStart = time.time()
> > >                          if GenFdsApi(Wa.GenFdsCommandDict, self.Db):
> > >                              EdkLogger.error("build", COMMAND_FAILURE)
> > > +                        Threshold = self.GetFreeSizeThreshold()
> > > +                        if Threshold:
> > > +                            self.CheckFreeSizeThreshold(Threshold,
> > > + Wa.FvDir)
> > >
> > >                          #
> > >                          # Create MAP file for all platform FVs after GenFds.
> > > @@ -2322,6 +2329,46 @@ class Build():
> > >                      #
> > >                      self._SaveMapFile(MapBuffer, Wa)
> > >                  self.CreateGuidedSectionToolsFile(Wa)
> > > +
> > > +    ## GetFreeSizeThreshold()
> > > +    #
> > > +    #   @retval int             Threshold value
> > > +    #
> > > +    def GetFreeSizeThreshold(self):
> > > +        Threshold = None
> > > +        Threshold_Str =
> > GlobalData.gCommandLineDefines.get('FV_SPARE_SPACE_THRESHOLD')
> > > +        if Threshold_Str:
> > > +            try:
> > > +                if Threshold_Str.lower().startswith('0x'):
> > > +                    Threshold = int(Threshold_Str, 16)
> > > +                else:
> > > +                    Threshold = int(Threshold_Str)
> > > +            except:
> > > +                EdkLogger.warn("build", 'incorrect value for
> > > + FV_SPARE_SPACE_THRESHOLD %s.Only decimal or hex format is allowed.'
> > > + %
> > > Threshold_Str)
> > > +        return Threshold
> > > +
> > > +    def CheckFreeSizeThreshold(self, Threshold=None, FvDir=None):
> > > +        if not isinstance(Threshold, int):
> > > +            return
> > > +        if not isinstance(FvDir, str) or not FvDir:
> > > +            return
> > > +        FdfParserObject = GlobalData.gFdfParser
> > > +        FvRegionNameList = [FvName for FvName in
> > > + FdfParserObject.Profile.FvDict if
> > > FdfParserObject.Profile.FvDict[FvName].FvRegionInFD]
> > > +        for FvName in FdfParserObject.Profile.FvDict:
> > > +            if FvName in FvRegionNameList:
> > > +                FvSpaceInfoFileName = os.path.join(FvDir, FvName.upper() +
> > '.Fv.map')
> > > +                if os.path.exists(FvSpaceInfoFileName):
> > > +                    FileLinesList = getlines(FvSpaceInfoFileName)
> > > +                    for Line in FileLinesList:
> > > +                        NameValue = Line.split('=')
> > > +                        if len(NameValue) == 2 and NameValue[0].strip() ==
> > 'EFI_FV_SPACE_SIZE':
> > > +                            FreeSizeValue = int(NameValue[1].strip(), 0)
> > > +                            if FreeSizeValue < Threshold:
> > > +                                EdkLogger.error("build", FV_FREESIZE_ERROR,
> > > +                                                'the required spare
> > > + space in fv image size %d of %s FV exceeds the set spare space in fv
> > > + image size %d' %
> > > (
> > > +                                                    FreeSizeValue, FvName, Threshold))
> > > +                            break
> > > +
> > >      ## Generate GuidedSectionTools.txt in the FV directories.
> > >      #
> > >      def CreateGuidedSectionToolsFile(self,Wa):
> > > --
> > > 2.14.1.windows.1