[PATCH v1 0/9] BaseTools: refactor variables and regular expressions

[PATCH v1 0/9] BaseTools: refactor variables and regular expressions

[Jaben Carsey]

change how regular expressions and variables are used to maximize sharing 
during build

Jaben Carsey (9):
  BaseTools: FdfParser and FdfParserLite share reg exp
  BaseTools: GlobalData share same MACRO name definition
  BaseTools: add GUID pattern to global data
  BaseTools: use new shared GUID regular expressions
  BaseTools: Regular Expressions refactor out the hex char for later
    reuse
  BaseTools: Add new RegExp for future use
  BaseTools: Use precompiled RegExp
  BaseTools: GlobalData Add a regular expression for a hex number
  BaseTools: remove local hex number regular expression

 BaseTools/Source/Python/AutoGen/UniClassObject.py |  4 ++--
 BaseTools/Source/Python/Common/Expression.py      |  6 ++----
 BaseTools/Source/Python/Common/FdfParserLite.py   | 10 +++++-----
 BaseTools/Source/Python/Common/GlobalData.py      | 21 +++++++++++++++++---
 BaseTools/Source/Python/Common/RangeExpression.py | 19 ++++++++----------
 BaseTools/Source/Python/GenFds/FdfParser.py       |  5 ++---
 6 files changed, 37 insertions(+), 28 deletions(-)

-- 
2.16.2.windows.1

[PATCH v1 1/9] BaseTools: FdfParser and FdfParserLite share reg exp

[Jaben Carsey]

FdfParser can share regular expression from FdfParserLite.
reduce overlap and reduce recompile of the same expression.

Cc: Yonghong Zhu <yonghong.zhu@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Jaben Carsey <jaben.carsey@intel.com>
---
 BaseTools/Source/Python/Common/FdfParserLite.py | 5 +++--
 BaseTools/Source/Python/GenFds/FdfParser.py     | 2 +-
 2 files changed, 4 insertions(+), 3 deletions(-)

diff --git a/BaseTools/Source/Python/Common/FdfParserLite.py b/BaseTools/Source/Python/Common/FdfParserLite.py
index 7d129bfcab59..330422ad59d2 100644
--- a/BaseTools/Source/Python/Common/FdfParserLite.py
+++ b/BaseTools/Source/Python/Common/FdfParserLite.py
@@ -46,6 +46,8 @@ InputMacroDict = {}
 # All Macro values when parsing file, not replace existing Macro
 AllMacroList = []
 
+FileExtensionPattern = re.compile(r'([a-zA-Z][a-zA-Z0-9]*)')
+
 def GetRealFileLine (File, Line):
     
     InsertedLines = 0
@@ -2842,8 +2844,7 @@ class FdfParser(object):
             
         Ext = ""
         if self.__GetNextToken():
-            Pattern = re.compile(r'([a-zA-Z][a-zA-Z0-9]*)')
-            if Pattern.match(self.__Token):
+            if FileExtensionPattern.match(self.__Token):
                 Ext = self.__Token                            
                 return '.' + Ext    
             else:
diff --git a/BaseTools/Source/Python/GenFds/FdfParser.py b/BaseTools/Source/Python/GenFds/FdfParser.py
index dda7ed4ce798..03183e398bdb 100644
--- a/BaseTools/Source/Python/GenFds/FdfParser.py
+++ b/BaseTools/Source/Python/GenFds/FdfParser.py
@@ -59,6 +59,7 @@ import Common.LongFilePathOs as os
 from Common.LongFilePathSupport import OpenLongFilePath as open
 from Capsule import EFI_CERT_TYPE_PKCS7_GUID
 from Capsule import EFI_CERT_TYPE_RSA2048_SHA256_GUID
+from Common.FdfParserLite import FileExtensionPattern
 
 ##define T_CHAR_SPACE                ' '
 ##define T_CHAR_NULL                 '\0'
@@ -3706,7 +3707,6 @@ class FdfParser:
 
         Ext = ""
         if self.__GetNextToken():
-            Pattern = re.compile(r'([a-zA-Z][a-zA-Z0-9]*)')
             if Pattern.match(self.__Token):
                 Ext = self.__Token
                 return '.' + Ext
-- 
2.16.2.windows.1

[PATCH v1 2/9] BaseTools: GlobalData share same MACRO name definition

[Jaben Carsey]

use the same MACRO name definition across shared regular expression patterns.

Cc: Yonghong Zhu <yonghong.zhu@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Jaben Carsey <jaben.carsey@intel.com>
---
 BaseTools/Source/Python/Common/GlobalData.py | 9 ++++++---
 1 file changed, 6 insertions(+), 3 deletions(-)

diff --git a/BaseTools/Source/Python/Common/GlobalData.py b/BaseTools/Source/Python/Common/GlobalData.py
index 8b36a1b77366..83ba3d19961a 100644
--- a/BaseTools/Source/Python/Common/GlobalData.py
+++ b/BaseTools/Source/Python/Common/GlobalData.py
@@ -1,7 +1,7 @@
 ## @file
 # This file is used to define common static strings used by INF/DEC/DSC files
 #
-# Copyright (c) 2007 - 2016, Intel Corporation. All rights reserved.<BR>
+# Copyright (c) 2007 - 2018, Intel Corporation. All rights reserved.<BR>
 # This program and the accompanying materials
 # are licensed and made available under the terms and conditions of the BSD License
 # which accompanies this distribution.  The full text of the license may be found at
@@ -43,10 +43,13 @@ gBuildingModule = ''
 gSkuids = []
 gDefaultStores = []
 
+# definition for a MACRO name.  used to create regular expressions below.
+_MacroNamePattern = "[A-Z][A-Z0-9_]*"
+
 ## Regular expression for matching macro used in DSC/DEC/INF file inclusion
-gMacroRefPattern = re.compile("\$\(([A-Z][_A-Z0-9]*)\)", re.UNICODE)
+gMacroRefPattern = re.compile("\$\(({})\)".format(_MacroNamePattern), re.UNICODE)
 gMacroDefPattern = re.compile("^(DEFINE|EDK_GLOBAL)[ \t]+")
-gMacroNamePattern = re.compile("^[A-Z][A-Z0-9_]*$")
+gMacroNamePattern = re.compile("^{}$".format(_MacroNamePattern))
 
 #
 # A global variable for whether current build in AutoGen phase or not.
-- 
2.16.2.windows.1

[PATCH v1 3/9] BaseTools: add GUID pattern to global data

[Jaben Carsey]

add a shared global regular expression for GUID matching

Cc: Yonghong Zhu <yonghong.zhu@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Jaben Carsey <jaben.carsey@intel.com>
---
 BaseTools/Source/Python/Common/GlobalData.py | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/BaseTools/Source/Python/Common/GlobalData.py b/BaseTools/Source/Python/Common/GlobalData.py
index 83ba3d19961a..4a85a8e0db63 100644
--- a/BaseTools/Source/Python/Common/GlobalData.py
+++ b/BaseTools/Source/Python/Common/GlobalData.py
@@ -51,6 +51,13 @@ gMacroRefPattern = re.compile("\$\(({})\)".format(_MacroNamePattern), re.UNICODE
 gMacroDefPattern = re.compile("^(DEFINE|EDK_GLOBAL)[ \t]+")
 gMacroNamePattern = re.compile("^{}$".format(_MacroNamePattern))
 
+# definition for a GUID.  used to create regular expressions below.
+_GuidPattern = r"[0-9a-fA-F]{8}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{12}"
+
+## Regular expressions for GUID matching
+gGuidPattern = re.compile(r'{}'.format(_GuidPattern))
+gGuidPatternEnd = re.compile(r'{}$'.format(_GuidPattern))
+
 #
 # A global variable for whether current build in AutoGen phase or not.
 #
-- 
2.16.2.windows.1

[PATCH v1 4/9] BaseTools: use new shared GUID regular expressions

[Jaben Carsey]

remove local variables that are GUID matching and replace with shared
expression.

Cc: Yonghong Zhu <yonghong.zhu@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Jaben Carsey <jaben.carsey@intel.com>
---
 BaseTools/Source/Python/Common/Expression.py      |  3 +--
 BaseTools/Source/Python/Common/FdfParserLite.py   |  5 ++---
 BaseTools/Source/Python/Common/RangeExpression.py | 14 ++++++--------
 BaseTools/Source/Python/GenFds/FdfParser.py       |  3 +--
 4 files changed, 10 insertions(+), 15 deletions(-)

diff --git a/BaseTools/Source/Python/Common/Expression.py b/BaseTools/Source/Python/Common/Expression.py
index e76f09c367c1..c69c3cb1e84a 100644
--- a/BaseTools/Source/Python/Common/Expression.py
+++ b/BaseTools/Source/Python/Common/Expression.py
@@ -220,7 +220,6 @@ class ValueExpression(object):
 
     PcdPattern = re.compile(r'[_a-zA-Z][0-9A-Za-z_]*\.[_a-zA-Z][0-9A-Za-z_]*$')
     HexPattern = re.compile(r'0[xX][0-9a-fA-F]+$')
-    RegGuidPattern = re.compile(r'[0-9a-fA-F]{8}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{12}')
 
     SymbolPattern = re.compile("("
                                  "\$\([A-Z][A-Z0-9_]*\)|\$\(\w+\.\w+\)|\w+\.\w+|"
@@ -731,7 +730,7 @@ class ValueExpression(object):
         self._Token = ''
         if Expr:
             Ch = Expr[0]
-            Match = self.RegGuidPattern.match(Expr)
+            Match = gGuidPattern.match(Expr)
             if Match and not Expr[Match.end():Match.end()+1].isalnum() \
                 and Expr[Match.end():Match.end()+1] != '_':
                 self._Idx += Match.end()
diff --git a/BaseTools/Source/Python/Common/FdfParserLite.py b/BaseTools/Source/Python/Common/FdfParserLite.py
index 330422ad59d2..773f3b1a708f 100644
--- a/BaseTools/Source/Python/Common/FdfParserLite.py
+++ b/BaseTools/Source/Python/Common/FdfParserLite.py
@@ -1,7 +1,7 @@
 ## @file
 # parse FDF file
 #
-#  Copyright (c) 2007 - 2017, Intel Corporation. All rights reserved.<BR>
+#  Copyright (c) 2007 - 2018, Intel Corporation. All rights reserved.<BR>
 #
 #  This program and the accompanying materials
 #  are licensed and made available under the terms and conditions of the BSD License
@@ -933,8 +933,7 @@ class FdfParser(object):
         
         if not self.__GetNextToken():
             return False
-        p = re.compile('[a-fA-F0-9]{8}-[a-fA-F0-9]{4}-[a-fA-F0-9]{4}-[a-fA-F0-9]{4}-[a-fA-F0-9]{12}')
-        if p.match(self.__Token) != None:
+        if gGuidPattern.match(self.__Token) != None:
             return True
         else:
             self.__UndoToken()
diff --git a/BaseTools/Source/Python/Common/RangeExpression.py b/BaseTools/Source/Python/Common/RangeExpression.py
index b6c929fd885b..bbcbd6b767e0 100644
--- a/BaseTools/Source/Python/Common/RangeExpression.py
+++ b/BaseTools/Source/Python/Common/RangeExpression.py
@@ -1,7 +1,7 @@
 # # @file
 # This file is used to parse and evaluate range expression in Pcd declaration.
 #
-# Copyright (c) 2015, Intel Corporation. All rights reserved.<BR>
+# Copyright (c) 2015 - 2018, Intel Corporation. All rights reserved.<BR>
 # This program and the accompanying materials
 # are licensed and made available under the terms and conditions of the BSD License
 # which accompanies this distribution.    The full text of the license may be found at
@@ -211,8 +211,6 @@ class RangeExpression(object):
 
     PcdPattern = re.compile(r'[_a-zA-Z][0-9A-Za-z_]*\.[_a-zA-Z][0-9A-Za-z_]*$')
     HexPattern = re.compile(r'0[xX][0-9a-fA-F]+')
-    RegGuidPattern = re.compile(r'[0-9a-fA-F]{8}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{12}')
-    ExRegGuidPattern = re.compile(r'[0-9a-fA-F]{8}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{12}$')
     
     SymbolPattern = re.compile("("
                                  "\$\([A-Z][A-Z0-9_]*\)|\$\(\w+\.\w+\)|\w+\.\w+|"
@@ -348,18 +346,18 @@ class RangeExpression(object):
     def Eval(self, Operator, Oprand1, Oprand2 = None):
         
         if Operator in ["!", "NOT", "not"]:
-            if not self.RegGuidPattern.match(Oprand1.strip()):
+            if not gGuidPattern.match(Oprand1.strip()):
                 raise BadExpression(ERR_STRING_EXPR % Operator)
             return self.NegtiveRange(Oprand1)
         else:
             if Operator in ["==", ">=", "<=", ">", "<", '^']:
                 return self.EvalRange(Operator, Oprand1)
             elif Operator == 'and' :
-                if not self.ExRegGuidPattern.match(Oprand1.strip()) or not self.ExRegGuidPattern.match(Oprand2.strip()):
+                if not gGuidPatternEnd.match(Oprand1.strip()) or not gGuidPatternEnd.match(Oprand2.strip()):
                     raise BadExpression(ERR_STRING_EXPR % Operator)
                 return self.Rangeintersection(Oprand1, Oprand2)    
             elif Operator == 'or':
-                if not self.ExRegGuidPattern.match(Oprand1.strip()) or not self.ExRegGuidPattern.match(Oprand2.strip()):
+                if not gGuidPatternEnd.match(Oprand1.strip()) or not gGuidPatternEnd.match(Oprand2.strip()):
                     raise BadExpression(ERR_STRING_EXPR % Operator)
                 return self.Rangecollections(Oprand1, Oprand2)
             else:
@@ -417,7 +415,7 @@ class RangeExpression(object):
         # check if the expression does not need to evaluate
         if RealValue and Depth == 0:
             self._Token = self._Expr
-            if self.ExRegGuidPattern.match(self._Expr):
+            if gGuidPatternEnd.match(self._Expr):
                 return [self.operanddict[self._Expr] ]
 
             self._Idx = 0
@@ -664,7 +662,7 @@ class RangeExpression(object):
         self._Token = ''
         if Expr:
             Ch = Expr[0]
-            Match = self.RegGuidPattern.match(Expr)
+            Match = gGuidPattern.match(Expr)
             if Match and not Expr[Match.end():Match.end() + 1].isalnum() \
                 and Expr[Match.end():Match.end() + 1] != '_':
                 self._Idx += Match.end()
diff --git a/BaseTools/Source/Python/GenFds/FdfParser.py b/BaseTools/Source/Python/GenFds/FdfParser.py
index 03183e398bdb..b8be1251ef52 100644
--- a/BaseTools/Source/Python/GenFds/FdfParser.py
+++ b/BaseTools/Source/Python/GenFds/FdfParser.py
@@ -1150,8 +1150,7 @@ class FdfParser:
 
         if not self.__GetNextToken():
             return False
-        p = re.compile('[a-fA-F0-9]{8}-[a-fA-F0-9]{4}-[a-fA-F0-9]{4}-[a-fA-F0-9]{4}-[a-fA-F0-9]{12}')
-        if p.match(self.__Token) != None:
+        if gGuidPattern.match(self.__Token) != None:
             return True
         else:
             self.__UndoToken()
-- 
2.16.2.windows.1

[PATCH v1 5/9] BaseTools: Regular Expressions refactor out the hex char for later reuse

[Jaben Carsey]

move hex character info from GUID expressions into seperate variable to facilitate reuse.

Cc: Yonghong Zhu <yonghong.zhu@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Jaben Carsey <jaben.carsey@intel.com>
---
 BaseTools/Source/Python/Common/GlobalData.py | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/BaseTools/Source/Python/Common/GlobalData.py b/BaseTools/Source/Python/Common/GlobalData.py
index 4a85a8e0db63..4d0833a55b52 100644
--- a/BaseTools/Source/Python/Common/GlobalData.py
+++ b/BaseTools/Source/Python/Common/GlobalData.py
@@ -52,7 +52,8 @@ gMacroDefPattern = re.compile("^(DEFINE|EDK_GLOBAL)[ \t]+")
 gMacroNamePattern = re.compile("^{}$".format(_MacroNamePattern))
 
 # definition for a GUID.  used to create regular expressions below.
-_GuidPattern = r"[0-9a-fA-F]{8}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{12}"
+_HexChar = r"[0-9a-fA-F]"
+_GuidPattern = r"{Hex}{8}-{Hex}{4}-{Hex}{4}-{Hex}{4}-{Hex}{12}".format(Hex=_HexChar)
 
 ## Regular expressions for GUID matching
 gGuidPattern = re.compile(r'{}'.format(_GuidPattern))
-- 
2.16.2.windows.1

[PATCH v1 6/9] BaseTools: Add new RegExp for future use

[Jaben Carsey]

Add a precompiled RegExp for 4 hex chars.

Cc: Yonghong Zhu <yonghong.zhu@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Jaben Carsey <jaben.carsey@intel.com>
---
 BaseTools/Source/Python/Common/GlobalData.py | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/BaseTools/Source/Python/Common/GlobalData.py b/BaseTools/Source/Python/Common/GlobalData.py
index 4d0833a55b52..972ae3e2253c 100644
--- a/BaseTools/Source/Python/Common/GlobalData.py
+++ b/BaseTools/Source/Python/Common/GlobalData.py
@@ -59,6 +59,8 @@ _GuidPattern = r"{Hex}{8}-{Hex}{4}-{Hex}{4}-{Hex}{4}-{Hex}{12}".format(Hex=_HexC
 gGuidPattern = re.compile(r'{}'.format(_GuidPattern))
 gGuidPatternEnd = re.compile(r'{}$'.format(_GuidPattern))
 
+g4HexChar = re.compile(r'{}{4}'.format(_HexChar))
+
 #
 # A global variable for whether current build in AutoGen phase or not.
 #
-- 
2.16.2.windows.1

[PATCH v1 7/9] BaseTools: Use precompiled RegExp

[Jaben Carsey]

avoid recompiling the regular expression for each use in a while loop

Cc: Yonghong Zhu <yonghong.zhu@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Jaben Carsey <jaben.carsey@intel.com>
---
 BaseTools/Source/Python/AutoGen/UniClassObject.py | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/BaseTools/Source/Python/AutoGen/UniClassObject.py b/BaseTools/Source/Python/AutoGen/UniClassObject.py
index 856d19cda270..5c653f58e4ff 100644
--- a/BaseTools/Source/Python/AutoGen/UniClassObject.py
+++ b/BaseTools/Source/Python/AutoGen/UniClassObject.py
@@ -436,11 +436,11 @@ class UniFileClassObject(object):
             while (StartPos != -1):
                 EndPos = Line.find(u'\\', StartPos + 1, StartPos + 7)
                 if EndPos != -1 and EndPos - StartPos == 6 :
-                    if re.match('[a-fA-F0-9]{4}', Line[StartPos + 2 : EndPos], re.UNICODE):
+                    if g4HexChar.match(Line[StartPos + 2 : EndPos], re.UNICODE):
                         EndStr = Line[EndPos: ]
                         UniStr = ('\u' + (Line[StartPos + 2 : EndPos])).decode('unicode_escape')
                         if EndStr.startswith(u'\\x') and len(EndStr) >= 7:
-                            if EndStr[6] == u'\\' and re.match('[a-fA-F0-9]{4}', EndStr[2 : 6], re.UNICODE):
+                            if EndStr[6] == u'\\' and g4HexChar.match(EndStr[2 : 6], re.UNICODE):
                                 Line = Line[0 : StartPos] + UniStr + EndStr
                         else:
                             Line = Line[0 : StartPos] + UniStr + EndStr[1:]
-- 
2.16.2.windows.1

[PATCH v1 8/9] BaseTools: GlobalData Add a regular expression for a hex number

[Jaben Carsey]

add a shared precompiled regular expression

Cc: Yonghong Zhu <yonghong.zhu@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Jaben Carsey <jaben.carsey@intel.com>
---
 BaseTools/Source/Python/Common/GlobalData.py | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/BaseTools/Source/Python/Common/GlobalData.py b/BaseTools/Source/Python/Common/GlobalData.py
index 972ae3e2253c..eb1626bc151b 100644
--- a/BaseTools/Source/Python/Common/GlobalData.py
+++ b/BaseTools/Source/Python/Common/GlobalData.py
@@ -59,7 +59,9 @@ _GuidPattern = r"{Hex}{8}-{Hex}{4}-{Hex}{4}-{Hex}{4}-{Hex}{12}".format(Hex=_HexC
 gGuidPattern = re.compile(r'{}'.format(_GuidPattern))
 gGuidPatternEnd = re.compile(r'{}$'.format(_GuidPattern))
 
+## Regular expressions for HEX matching
 g4HexChar = re.compile(r'{}{4}'.format(_HexChar))
+gHexPattern = re.compile(r'0[xX]{}+'.format(_HexChar))
 
 #
 # A global variable for whether current build in AutoGen phase or not.
-- 
2.16.2.windows.1

[PATCH v1 9/9] BaseTools: remove local hex number regular expression

[Jaben Carsey]

Change to using the new shared hex number regular expression

Cc: Yonghong Zhu <yonghong.zhu@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Jaben Carsey <jaben.carsey@intel.com>
---
 BaseTools/Source/Python/Common/Expression.py      | 3 +--
 BaseTools/Source/Python/Common/RangeExpression.py | 5 ++---
 2 files changed, 3 insertions(+), 5 deletions(-)

diff --git a/BaseTools/Source/Python/Common/Expression.py b/BaseTools/Source/Python/Common/Expression.py
index c69c3cb1e84a..43ef586050e7 100644
--- a/BaseTools/Source/Python/Common/Expression.py
+++ b/BaseTools/Source/Python/Common/Expression.py
@@ -219,7 +219,6 @@ class ValueExpression(object):
     NonLetterOpLst = ['+', '-', '*', '/', '%', '&', '|', '^', '~', '<<', '>>', '!', '=', '>', '<', '?', ':']
 
     PcdPattern = re.compile(r'[_a-zA-Z][0-9A-Za-z_]*\.[_a-zA-Z][0-9A-Za-z_]*$')
-    HexPattern = re.compile(r'0[xX][0-9a-fA-F]+$')
 
     SymbolPattern = re.compile("("
                                  "\$\([A-Z][A-Z0-9_]*\)|\$\(\w+\.\w+\)|\w+\.\w+|"
@@ -671,7 +670,7 @@ class ValueExpression(object):
             self._LiteralToken.endswith('}'):
             return True
 
-        if self.HexPattern.match(self._LiteralToken):
+        if gHexPattern.match(self._LiteralToken):
             Token = self._LiteralToken[2:]
             if not Token:
                 self._LiteralToken = '0x0'
diff --git a/BaseTools/Source/Python/Common/RangeExpression.py b/BaseTools/Source/Python/Common/RangeExpression.py
index bbcbd6b767e0..2fdb99688557 100644
--- a/BaseTools/Source/Python/Common/RangeExpression.py
+++ b/BaseTools/Source/Python/Common/RangeExpression.py
@@ -210,7 +210,6 @@ class RangeExpression(object):
     NonLetterOpLst = ['+', '-', '&', '|', '^', '!', '=', '>', '<']
 
     PcdPattern = re.compile(r'[_a-zA-Z][0-9A-Za-z_]*\.[_a-zA-Z][0-9A-Za-z_]*$')
-    HexPattern = re.compile(r'0[xX][0-9a-fA-F]+')
     
     SymbolPattern = re.compile("("
                                  "\$\([A-Z][A-Z0-9_]*\)|\$\(\w+\.\w+\)|\w+\.\w+|"
@@ -226,7 +225,7 @@ class RangeExpression(object):
         # convert interval to object index. ex. 1 - 10 to a GUID
         expr = expr.strip()
         NumberDict = {}
-        for HexNumber in self.HexPattern.findall(expr):
+        for HexNumber in gHexPattern.findall(expr):
             Number = str(int(HexNumber, 16))
             NumberDict[HexNumber] = Number
         for HexNum in NumberDict:
@@ -631,7 +630,7 @@ class RangeExpression(object):
             self._LiteralToken.endswith('}'):
             return True
 
-        if self.HexPattern.match(self._LiteralToken):
+        if gHexPattern.match(self._LiteralToken):
             Token = self._LiteralToken[2:]
             Token = Token.lstrip('0')
             if not Token:
-- 
2.16.2.windows.1

Re: [PATCH v1 1/9] BaseTools: FdfParser and FdfParserLite share reg exp

[Zhu, Yonghong]

Hi Jaben,

For this series patch, I have two comment:
1.  1/9 patch: after remove the 'Pattern' variable in FdfParser.py file, this statement " if Pattern.match(self.__Token): " is incorrect now since there missing  'Pattern' variable.
2.  6/9 patch: need additional {} ({{4}}) for the statement "g4HexChar = re.compile(r'{}{4}'.format(_HexChar))"
Others are good to me. thanks.

Best Regards,
Zhu Yonghong


-----Original Message-----
From: Carsey, Jaben 
Sent: Saturday, March 17, 2018 7:28 AM
To: edk2-devel@lists.01.org
Cc: Zhu, Yonghong <yonghong.zhu@intel.com>; Gao, Liming <liming.gao@intel.com>
Subject: [PATCH v1 1/9] BaseTools: FdfParser and FdfParserLite share reg exp

FdfParser can share regular expression from FdfParserLite.
reduce overlap and reduce recompile of the same expression.

Cc: Yonghong Zhu <yonghong.zhu@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Jaben Carsey <jaben.carsey@intel.com>
---
 BaseTools/Source/Python/Common/FdfParserLite.py | 5 +++--
 BaseTools/Source/Python/GenFds/FdfParser.py     | 2 +-
 2 files changed, 4 insertions(+), 3 deletions(-)

diff --git a/BaseTools/Source/Python/Common/FdfParserLite.py b/BaseTools/Source/Python/Common/FdfParserLite.py
index 7d129bfcab59..330422ad59d2 100644
--- a/BaseTools/Source/Python/Common/FdfParserLite.py
+++ b/BaseTools/Source/Python/Common/FdfParserLite.py
@@ -46,6 +46,8 @@ InputMacroDict = {}
 # All Macro values when parsing file, not replace existing Macro  AllMacroList = []
 
+FileExtensionPattern = re.compile(r'([a-zA-Z][a-zA-Z0-9]*)')
+
 def GetRealFileLine (File, Line):
     
     InsertedLines = 0
@@ -2842,8 +2844,7 @@ class FdfParser(object):
             
         Ext = ""
         if self.__GetNextToken():
-            Pattern = re.compile(r'([a-zA-Z][a-zA-Z0-9]*)')
-            if Pattern.match(self.__Token):
+            if FileExtensionPattern.match(self.__Token):
                 Ext = self.__Token                            
                 return '.' + Ext    
             else:
diff --git a/BaseTools/Source/Python/GenFds/FdfParser.py b/BaseTools/Source/Python/GenFds/FdfParser.py
index dda7ed4ce798..03183e398bdb 100644
--- a/BaseTools/Source/Python/GenFds/FdfParser.py
+++ b/BaseTools/Source/Python/GenFds/FdfParser.py
@@ -59,6 +59,7 @@ import Common.LongFilePathOs as os  from Common.LongFilePathSupport import OpenLongFilePath as open  from Capsule import EFI_CERT_TYPE_PKCS7_GUID  from Capsule import EFI_CERT_TYPE_RSA2048_SHA256_GUID
+from Common.FdfParserLite import FileExtensionPattern
 
 ##define T_CHAR_SPACE                ' '
 ##define T_CHAR_NULL                 '\0'
@@ -3706,7 +3707,6 @@ class FdfParser:
 
         Ext = ""
         if self.__GetNextToken():
-            Pattern = re.compile(r'([a-zA-Z][a-zA-Z0-9]*)')
             if Pattern.match(self.__Token):
                 Ext = self.__Token
                 return '.' + Ext
--
2.16.2.windows.1