[PATCH v1 0/3] BaseTools: reduce RegExp compile times...

[PATCH v1 0/3] BaseTools: reduce RegExp compile times...

[Jaben Carsey]

make a new shareable RegEx in FdfParserLite, then use it

Jaben Carsey (3):
  BaseTools: use single RegExp for token matching
  BaseTools: use new RegEx from FdfParserLite
  BaseTools: fix comment cut/paste error

 BaseTools/Source/Python/Common/FdfParserLite.py | 20 +++++++++-----------
 BaseTools/Source/Python/GenFds/FdfParser.py     | 12 +++++-------
 2 files changed, 14 insertions(+), 18 deletions(-)

-- 
2.16.2.windows.1

[PATCH v1 1/3] BaseTools: use single RegExp for token matching

[Jaben Carsey]

same pattern was compiled 3 places in the file.  just compile once and share.

Cc: Liming Gao <liming.gao@intel.com>
Cc: Yonghong Zhu <yonghong.zhu@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Jaben Carsey <jaben.carsey@intel.com>
---
 BaseTools/Source/Python/Common/FdfParserLite.py | 16 +++++++---------
 1 file changed, 7 insertions(+), 9 deletions(-)

diff --git a/BaseTools/Source/Python/Common/FdfParserLite.py b/BaseTools/Source/Python/Common/FdfParserLite.py
index b7b5e21a9e3f..c22711603233 100644
--- a/BaseTools/Source/Python/Common/FdfParserLite.py
+++ b/BaseTools/Source/Python/Common/FdfParserLite.py
@@ -49,6 +49,7 @@ InputMacroDict = {}
 AllMacroList = []
 
 FileExtensionPattern = re.compile(r'([a-zA-Z][a-zA-Z0-9]*)')
+TokenFindPattern = re.compile(r'([a-zA-Z0-9\-]+|\$\(TARGET\)|\*)_([a-zA-Z0-9\-]+|\$\(TOOL_CHAIN_TAG\)|\*)_([a-zA-Z0-9\-]+|\$\(ARCH\)|\*)')
 
 def GetRealFileLine (File, Line):
     
@@ -2094,8 +2095,7 @@ class FdfParser(object):
 
                 
         if self.__GetNextToken():
-            p = re.compile(r'([a-zA-Z0-9\-]+|\$\(TARGET\)|\*)_([a-zA-Z0-9\-]+|\$\(TOOL_CHAIN_TAG\)|\*)_([a-zA-Z0-9\-]+|\$\(ARCH\)|\*)')
-            if p.match(self.__Token):
+            if TokenFindPattern.match(self.__Token):
                 FfsInfObj.KeyStringList.append(self.__Token)
                 if not self.__IsToken(","):
                     return
@@ -2104,7 +2104,7 @@ class FdfParser(object):
                 return
                 
             while self.__GetNextToken():
-                if not p.match(self.__Token):
+                if not TokenFindPattern.match(self.__Token):
                     raise Warning("expected KeyString \"Target_Tag_Arch\" At Line ", self.FileName, self.CurrentLineNumber)
                 FfsInfObj.KeyStringList.append(self.__Token)
     
@@ -2252,12 +2252,11 @@ class FdfParser(object):
     def __GetFileOpts(self, FfsFileObj):
         
         if self.__GetNextToken():
-            Pattern = re.compile(r'([a-zA-Z0-9\-]+|\$\(TARGET\)|\*)_([a-zA-Z0-9\-]+|\$\(TOOL_CHAIN_TAG\)|\*)_([a-zA-Z0-9\-]+|\$\(ARCH\)|\*)')
-            if Pattern.match(self.__Token):
+            if TokenFindPattern.match(self.__Token):
                 FfsFileObj.KeyStringList.append(self.__Token)
                 if self.__IsToken(","):
                     while self.__GetNextToken():
-                        if not Pattern.match(self.__Token):
+                        if not TokenFindPattern.match(self.__Token):
                             raise Warning("expected KeyString \"Target_Tag_Arch\" At Line ", self.FileName, self.CurrentLineNumber)
                         FfsFileObj.KeyStringList.append(self.__Token)
 
@@ -2902,12 +2901,11 @@ class FdfParser(object):
         
         KeyStringList = []
         if self.__GetNextToken():
-            Pattern = re.compile(r'([a-zA-Z0-9\-]+|\$\(TARGET\)|\*)_([a-zA-Z0-9\-]+|\$\(TOOL_CHAIN_TAG\)|\*)_([a-zA-Z0-9\-]+|\$\(ARCH\)|\*)')
-            if Pattern.match(self.__Token):
+            if TokenFindPattern.match(self.__Token):
                 KeyStringList.append(self.__Token)
                 if self.__IsToken(","):
                     while self.__GetNextToken():
-                        if not Pattern.match(self.__Token):
+                        if not TokenFindPattern.match(self.__Token):
                             raise Warning("expected KeyString \"Target_Tag_Arch\" At Line ", self.FileName, self.CurrentLineNumber)
                         KeyStringList.append(self.__Token)
 
-- 
2.16.2.windows.1

[PATCH v1 2/3] BaseTools: use new RegEx from FdfParserLite

[Jaben Carsey]

FdfParser has identical one.  import and share.

Cc: Liming Gao <liming.gao@intel.com>
Cc: Yonghong Zhu <yonghong.zhu@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Jaben Carsey <jaben.carsey@intel.com>
---
 BaseTools/Source/Python/GenFds/FdfParser.py | 12 +++++-------
 1 file changed, 5 insertions(+), 7 deletions(-)

diff --git a/BaseTools/Source/Python/GenFds/FdfParser.py b/BaseTools/Source/Python/GenFds/FdfParser.py
index 9903e9570cf9..4c911789fa62 100644
--- a/BaseTools/Source/Python/GenFds/FdfParser.py
+++ b/BaseTools/Source/Python/GenFds/FdfParser.py
@@ -60,7 +60,7 @@ from Common.LongFilePathSupport import OpenLongFilePath as open
 from Capsule import EFI_CERT_TYPE_PKCS7_GUID
 from Capsule import EFI_CERT_TYPE_RSA2048_SHA256_GUID
 from Common.RangeExpression import RangeExpression
-from Common.FdfParserLite import FileExtensionPattern
+from Common.FdfParserLite import FileExtensionPattern,TokenFindPattern
 
 ##define T_CHAR_SPACE                ' '
 ##define T_CHAR_NULL                 '\0'
@@ -2800,12 +2800,11 @@ class FdfParser:
     def __GetFileOpts(self, FfsFileObj):
 
         if self.__GetNextToken():
-            Pattern = re.compile(r'([a-zA-Z0-9\-]+|\$\(TARGET\)|\*)_([a-zA-Z0-9\-]+|\$\(TOOL_CHAIN_TAG\)|\*)_([a-zA-Z0-9\-]+|\$\(ARCH\)|\*)')
-            if Pattern.match(self.__Token):
+            if TokenFindPattern.match(self.__Token):
                 FfsFileObj.KeyStringList.append(self.__Token)
                 if self.__IsToken(","):
                     while self.__GetNextToken():
-                        if not Pattern.match(self.__Token):
+                        if not TokenFindPattern.match(self.__Token):
                             raise Warning("expected KeyString \"Target_Tag_Arch\"", self.FileName, self.CurrentLineNumber)
                         FfsFileObj.KeyStringList.append(self.__Token)
 
@@ -3749,12 +3748,11 @@ class FdfParser:
 
         KeyStringList = []
         if self.__GetNextToken():
-            Pattern = re.compile(r'([a-zA-Z0-9\-]+|\$\(TARGET\)|\*)_([a-zA-Z0-9\-]+|\$\(TOOL_CHAIN_TAG\)|\*)_([a-zA-Z0-9\-]+|\$\(ARCH\)|\*)')
-            if Pattern.match(self.__Token):
+            if TokenFindPattern.match(self.__Token):
                 KeyStringList.append(self.__Token)
                 if self.__IsToken(","):
                     while self.__GetNextToken():
-                        if not Pattern.match(self.__Token):
+                        if not TokenFindPattern.match(self.__Token):
                             raise Warning("expected KeyString \"Target_Tag_Arch\"", self.FileName, self.CurrentLineNumber)
                         KeyStringList.append(self.__Token)
 
-- 
2.16.2.windows.1

[PATCH v1 3/3] BaseTools: fix comment cut/paste error

[Jaben Carsey]

comment was duplicated from other function.

Cc: Liming Gao <liming.gao@intel.com>
Cc: Yonghong Zhu <yonghong.zhu@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Jaben Carsey <jaben.carsey@intel.com>
---
 BaseTools/Source/Python/Common/FdfParserLite.py | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/BaseTools/Source/Python/Common/FdfParserLite.py b/BaseTools/Source/Python/Common/FdfParserLite.py
index c22711603233..323a260661d1 100644
--- a/BaseTools/Source/Python/Common/FdfParserLite.py
+++ b/BaseTools/Source/Python/Common/FdfParserLite.py
@@ -538,9 +538,9 @@ class FdfParser(object):
                 
         self.Rewind()
 
-    ## PreprocessIncludeFile() method
+    ## PreprocessConditionalStatement() method
     #
-    #   Preprocess file contents, replace !include statements with file contents.
+    #   Preprocess conditional statement.
     #   In the end, rewind the file buffer pointer to the beginning
     #
     #   @param  self        The object pointer
-- 
2.16.2.windows.1

Re: [PATCH v1 0/3] BaseTools: reduce RegExp compile times...

[Zhu, Yonghong]

Reviewed-by: Yonghong Zhu <yonghong.zhu@intel.com> 

Best Regards,
Zhu Yonghong


-----Original Message-----
From: edk2-devel [mailto:edk2-devel-bounces@lists.01.org] On Behalf Of Jaben Carsey
Sent: Friday, March 30, 2018 5:38 AM
To: edk2-devel@lists.01.org
Subject: [edk2] [PATCH v1 0/3] BaseTools: reduce RegExp compile times...

make a new shareable RegEx in FdfParserLite, then use it

Jaben Carsey (3):
  BaseTools: use single RegExp for token matching
  BaseTools: use new RegEx from FdfParserLite
  BaseTools: fix comment cut/paste error

 BaseTools/Source/Python/Common/FdfParserLite.py | 20 +++++++++-----------
 BaseTools/Source/Python/GenFds/FdfParser.py     | 12 +++++-------
 2 files changed, 14 insertions(+), 18 deletions(-)

-- 
2.16.2.windows.1

_______________________________________________
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel