* [PATCH v1 0/5] BaseTools: RegEx refactoring
@ 2018-04-03 22:34 Jaben Carsey
2018-04-03 22:34 ` [PATCH v1 1/5] BaseTools: Autogen - move RegEx compile Jaben Carsey
` (5 more replies)
0 siblings, 6 replies; 7+ messages in thread
From: Jaben Carsey @ 2018-04-03 22:34 UTC (permalink / raw)
To: edk2-devel
move RegEx compile out of functions/loops
share RegEx when have identical data
Jaben Carsey (5):
BaseTools: Autogen - move RegEx compile
BaseTools: GenFds - move RegEx compile
BaseTools: Add new RegEx pattern to GlobalData
BaseTools: AutoGen - use the new shared RegEx
BaseTools: remove redundant check
BaseTools/Source/Python/AutoGen/AutoGen.py | 22 +++++++++++++-------
BaseTools/Source/Python/AutoGen/IdfClassObject.py | 5 +++--
BaseTools/Source/Python/AutoGen/UniClassObject.py | 8 +++----
BaseTools/Source/Python/Common/GlobalData.py | 3 +++
BaseTools/Source/Python/GenFds/FdfParser.py | 5 ++---
5 files changed, 27 insertions(+), 16 deletions(-)
--
2.16.2.windows.1
^ permalink raw reply [flat|nested] 7+ messages in thread
* [PATCH v1 1/5] BaseTools: Autogen - move RegEx compile
2018-04-03 22:34 [PATCH v1 0/5] BaseTools: RegEx refactoring Jaben Carsey
@ 2018-04-03 22:34 ` Jaben Carsey
2018-04-03 22:34 ` [PATCH v1 2/5] BaseTools: GenFds " Jaben Carsey
` (4 subsequent siblings)
5 siblings, 0 replies; 7+ messages in thread
From: Jaben Carsey @ 2018-04-03 22:34 UTC (permalink / raw)
To: edk2-devel; +Cc: Liming Gao, Yonghong Zhu
compile each RegEx once not in loops/functions
Cc: Liming Gao <liming.gao@intel.com>
Cc: Yonghong Zhu <yonghong.zhu@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Jaben Carsey <jaben.carsey@intel.com>
---
BaseTools/Source/Python/AutoGen/AutoGen.py | 22 +++++++++++++-------
1 file changed, 15 insertions(+), 7 deletions(-)
diff --git a/BaseTools/Source/Python/AutoGen/AutoGen.py b/BaseTools/Source/Python/AutoGen/AutoGen.py
index 3384fdb70b7e..7dd7c35e6685 100644
--- a/BaseTools/Source/Python/AutoGen/AutoGen.py
+++ b/BaseTools/Source/Python/AutoGen/AutoGen.py
@@ -49,6 +49,16 @@ from GenVar import VariableMgr,var_info
## Regular expression for splitting Dependency Expression string into tokens
gDepexTokenPattern = re.compile("(\(|\)|\w+| \S+\.inf)")
+## Regular expression for match: PCD(xxxx.yyy)
+gPCDAsGuidPattern = re.compile(r"^PCD\(.+\..+\)$")
+
+#
+# Regular expression for finding Include Directories, the difference between MSFT and INTEL/GCC/RVCT
+# is the former use /I , the Latter used -I to specify include directories
+#
+gBuildOptIncludePatternMsft = re.compile(r"(?:.*?)/I[ \t]*([^ ]*)", re.MULTILINE | re.DOTALL)
+gBuildOptIncludePatternOther = re.compile(r"(?:.*?)-I[ \t]*([^ ]*)", re.MULTILINE | re.DOTALL)
+
#
# Match name = variable
#
@@ -818,13 +828,11 @@ class WorkspaceAutoGen(AutoGen):
InfFoundFlag = False
if FfsFile.NameGuid is not None:
- _CheckPCDAsGuidPattern = re.compile("^PCD\(.+\..+\)$")
-
#
# If the NameGuid reference a PCD name.
# The style must match: PCD(xxxx.yyy)
#
- if _CheckPCDAsGuidPattern.match(FfsFile.NameGuid):
+ if gPCDAsGuidPattern.match(FfsFile.NameGuid):
#
# Replace the PCD value.
#
@@ -3315,9 +3323,9 @@ class ModuleAutoGen(AutoGen):
# is the former use /I , the Latter used -I to specify include directories
#
if self.PlatformInfo.ToolChainFamily in ('MSFT'):
- gBuildOptIncludePattern = re.compile(r"(?:.*?)/I[ \t]*([^ ]*)", re.MULTILINE | re.DOTALL)
+ BuildOptIncludeRegEx = gBuildOptIncludePatternMsft
elif self.PlatformInfo.ToolChainFamily in ('INTEL', 'GCC', 'RVCT'):
- gBuildOptIncludePattern = re.compile(r"(?:.*?)-I[ \t]*([^ ]*)", re.MULTILINE | re.DOTALL)
+ BuildOptIncludeRegEx = gBuildOptIncludePatternOther
else:
#
# New ToolChainFamily, don't known whether there is option to specify include directories
@@ -3334,13 +3342,13 @@ class ModuleAutoGen(AutoGen):
FlagOption = ''
if self.PlatformInfo.ToolChainFamily != 'RVCT':
- IncPathList = [NormPath(Path, self.Macros) for Path in gBuildOptIncludePattern.findall(FlagOption)]
+ IncPathList = [NormPath(Path, self.Macros) for Path in BuildOptIncludeRegEx.findall(FlagOption)]
else:
#
# RVCT may specify a list of directory seperated by commas
#
IncPathList = []
- for Path in gBuildOptIncludePattern.findall(FlagOption):
+ for Path in BuildOptIncludeRegEx.findall(FlagOption):
PathList = GetSplitList(Path, TAB_COMMA_SPLIT)
IncPathList += [NormPath(PathEntry, self.Macros) for PathEntry in PathList]
--
2.16.2.windows.1
^ permalink raw reply related [flat|nested] 7+ messages in thread
* [PATCH v1 2/5] BaseTools: GenFds - move RegEx compile
2018-04-03 22:34 [PATCH v1 0/5] BaseTools: RegEx refactoring Jaben Carsey
2018-04-03 22:34 ` [PATCH v1 1/5] BaseTools: Autogen - move RegEx compile Jaben Carsey
@ 2018-04-03 22:34 ` Jaben Carsey
2018-04-03 22:34 ` [PATCH v1 3/5] BaseTools: Add new RegEx pattern to GlobalData Jaben Carsey
` (3 subsequent siblings)
5 siblings, 0 replies; 7+ messages in thread
From: Jaben Carsey @ 2018-04-03 22:34 UTC (permalink / raw)
To: edk2-devel; +Cc: Liming Gao, Yonghong Zhu
compile each RegEx once not in loops/functions
Cc: Liming Gao <liming.gao@intel.com>
Cc: Yonghong Zhu <yonghong.zhu@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Jaben Carsey <jaben.carsey@intel.com>
---
BaseTools/Source/Python/GenFds/FdfParser.py | 5 ++---
1 file changed, 2 insertions(+), 3 deletions(-)
diff --git a/BaseTools/Source/Python/GenFds/FdfParser.py b/BaseTools/Source/Python/GenFds/FdfParser.py
index 9b7e898570a3..00e03446421d 100644
--- a/BaseTools/Source/Python/GenFds/FdfParser.py
+++ b/BaseTools/Source/Python/GenFds/FdfParser.py
@@ -85,6 +85,7 @@ RegionSizePattern = re.compile("\s*(?P<base>(?:0x|0X)?[a-fA-F0-9]+)\s*\|\s*(?P<s
RegionSizeGuidPattern = re.compile("\s*(?P<base>\w+\.\w+)\s*\|\s*(?P<size>\w+\.\w+)\s*")
RegionOffsetPcdPattern = re.compile("\s*(?P<base>\w+\.\w+)\s*$")
ShortcutPcdPattern = re.compile("\s*\w+\s*=\s*(?P<value>(?:0x|0X)?[a-fA-F0-9]+)\s*\|\s*(?P<name>\w+\.\w+)\s*")
+BaseAddrValuePattern = re.compile('^0[xX][0-9a-fA-F]+')
AllIncludeFileList = []
@@ -2211,9 +2212,7 @@ class FdfParser:
if not self.__GetNextToken():
raise Warning("expected FV base address value", self.FileName, self.CurrentLineNumber)
- IsValidBaseAddrValue = re.compile('^0[x|X][0-9a-fA-F]+')
-
- if not IsValidBaseAddrValue.match(self.__Token.upper()):
+ if not BaseAddrValuePattern.match(self.__Token.upper()):
raise Warning("Unknown FV base address value '%s'" % self.__Token, self.FileName, self.CurrentLineNumber)
Obj.FvBaseAddress = self.__Token
return True
--
2.16.2.windows.1
^ permalink raw reply related [flat|nested] 7+ messages in thread
* [PATCH v1 3/5] BaseTools: Add new RegEx pattern to GlobalData
2018-04-03 22:34 [PATCH v1 0/5] BaseTools: RegEx refactoring Jaben Carsey
2018-04-03 22:34 ` [PATCH v1 1/5] BaseTools: Autogen - move RegEx compile Jaben Carsey
2018-04-03 22:34 ` [PATCH v1 2/5] BaseTools: GenFds " Jaben Carsey
@ 2018-04-03 22:34 ` Jaben Carsey
2018-04-03 22:34 ` [PATCH v1 4/5] BaseTools: AutoGen - use the new shared RegEx Jaben Carsey
` (2 subsequent siblings)
5 siblings, 0 replies; 7+ messages in thread
From: Jaben Carsey @ 2018-04-03 22:34 UTC (permalink / raw)
To: edk2-devel; +Cc: Liming Gao, Yonghong Zhu
Cc: Liming Gao <liming.gao@intel.com>
Cc: Yonghong Zhu <yonghong.zhu@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Jaben Carsey <jaben.carsey@intel.com>
---
BaseTools/Source/Python/Common/GlobalData.py | 3 +++
1 file changed, 3 insertions(+)
diff --git a/BaseTools/Source/Python/Common/GlobalData.py b/BaseTools/Source/Python/Common/GlobalData.py
index fab67601a9e5..f58dc5a8dda2 100644
--- a/BaseTools/Source/Python/Common/GlobalData.py
+++ b/BaseTools/Source/Python/Common/GlobalData.py
@@ -63,6 +63,9 @@ gGuidPatternEnd = re.compile(r'{}$'.format(_GuidPattern))
g4HexChar = re.compile(r'{}{{4}}'.format(_HexChar))
gHexPattern = re.compile(r'0[xX]{}+'.format(_HexChar))
+## Regular expressions for string identifier checking
+gIdentifierPattern = re.compile('^[a-zA-Z][a-zA-Z0-9_]*$', re.UNICODE)
+
#
# A global variable for whether current build in AutoGen phase or not.
#
--
2.16.2.windows.1
^ permalink raw reply related [flat|nested] 7+ messages in thread
* [PATCH v1 4/5] BaseTools: AutoGen - use the new shared RegEx
2018-04-03 22:34 [PATCH v1 0/5] BaseTools: RegEx refactoring Jaben Carsey
` (2 preceding siblings ...)
2018-04-03 22:34 ` [PATCH v1 3/5] BaseTools: Add new RegEx pattern to GlobalData Jaben Carsey
@ 2018-04-03 22:34 ` Jaben Carsey
2018-04-03 22:34 ` [PATCH v1 5/5] BaseTools: remove redundant check Jaben Carsey
2018-04-08 7:05 ` [PATCH v1 0/5] BaseTools: RegEx refactoring Zhu, Yonghong
5 siblings, 0 replies; 7+ messages in thread
From: Jaben Carsey @ 2018-04-03 22:34 UTC (permalink / raw)
To: edk2-devel; +Cc: Liming Gao, Yonghong Zhu
Cc: Liming Gao <liming.gao@intel.com>
Cc: Yonghong Zhu <yonghong.zhu@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Jaben Carsey <jaben.carsey@intel.com>
---
BaseTools/Source/Python/AutoGen/IdfClassObject.py | 3 ++-
BaseTools/Source/Python/AutoGen/UniClassObject.py | 4 ++--
2 files changed, 4 insertions(+), 3 deletions(-)
diff --git a/BaseTools/Source/Python/AutoGen/IdfClassObject.py b/BaseTools/Source/Python/AutoGen/IdfClassObject.py
index cb72219b40d5..a028aceee927 100644
--- a/BaseTools/Source/Python/AutoGen/IdfClassObject.py
+++ b/BaseTools/Source/Python/AutoGen/IdfClassObject.py
@@ -21,6 +21,7 @@ from Common.Misc import PathClass
from Common.LongFilePathSupport import LongFilePath
import re
import os
+from Common.GlobalData import gIdentifierPattern
IMAGE_TOKEN = re.compile('IMAGE_TOKEN *\(([A-Z0-9_]+) *\)', re.MULTILINE | re.UNICODE)
@@ -105,7 +106,7 @@ class IdfFileClassObject(object):
EdkLogger.error("Image Definition File Parser", PARSER_ERROR, 'The format is not match #image IMAGE_ID [TRANSPARENT] ImageFileName in Line %s of File %s.' % (LineNo, File.Path))
if Len == 4 and LineDetails[2] != 'TRANSPARENT':
EdkLogger.error("Image Definition File Parser", PARSER_ERROR, 'Please use the keyword "TRANSPARENT" to describe the transparency setting in Line %s of File %s.' % (LineNo, File.Path))
- MatchString = re.match('^[a-zA-Z][a-zA-Z0-9_]*$', LineDetails[1], re.UNICODE)
+ MatchString = gIdentifierPattern.match(LineDetails[1])
if MatchString is None or MatchString.end(0) != len(LineDetails[1]):
EdkLogger.error('Image Definition File Parser', FORMAT_INVALID, 'The Image token name %s defined in Idf file %s contains the invalid character.' % (LineDetails[1], File.Path))
if LineDetails[1] not in self.ImageIDList:
diff --git a/BaseTools/Source/Python/AutoGen/UniClassObject.py b/BaseTools/Source/Python/AutoGen/UniClassObject.py
index 242402dfaeeb..8b0c563a8c88 100644
--- a/BaseTools/Source/Python/AutoGen/UniClassObject.py
+++ b/BaseTools/Source/Python/AutoGen/UniClassObject.py
@@ -351,7 +351,7 @@ class UniFileClassObject(object):
Name = Item.split()[1]
# Check the string name
if Name != '':
- MatchString = re.match('^[a-zA-Z][a-zA-Z0-9_]*$', Name, re.UNICODE)
+ MatchString = gIdentifierPattern.match(Name)
if MatchString is None or MatchString.end(0) != len(Name):
EdkLogger.error('Unicode File Parser', FORMAT_INVALID, 'The string token name %s defined in UNI file %s contains the invalid character.' % (Name, self.File))
LanguageList = Item.split(u'#language ')
@@ -521,7 +521,7 @@ class UniFileClassObject(object):
Language = GetLanguageCode(Language, self.IsCompatibleMode, self.File)
# Check the string name
if not self.IsCompatibleMode and Name != '':
- MatchString = re.match('^[a-zA-Z][a-zA-Z0-9_]*$', Name, re.UNICODE)
+ MatchString = gIdentifierPattern.match(Name)
if MatchString is None or MatchString.end(0) != len(Name):
EdkLogger.error('Unicode File Parser', FORMAT_INVALID, 'The string token name %s defined in UNI file %s contains the invalid character.' % (Name, self.File))
self.AddStringToList(Name, Language, Value)
--
2.16.2.windows.1
^ permalink raw reply related [flat|nested] 7+ messages in thread
* [PATCH v1 5/5] BaseTools: remove redundant check
2018-04-03 22:34 [PATCH v1 0/5] BaseTools: RegEx refactoring Jaben Carsey
` (3 preceding siblings ...)
2018-04-03 22:34 ` [PATCH v1 4/5] BaseTools: AutoGen - use the new shared RegEx Jaben Carsey
@ 2018-04-03 22:34 ` Jaben Carsey
2018-04-08 7:05 ` [PATCH v1 0/5] BaseTools: RegEx refactoring Zhu, Yonghong
5 siblings, 0 replies; 7+ messages in thread
From: Jaben Carsey @ 2018-04-03 22:34 UTC (permalink / raw)
To: edk2-devel; +Cc: Liming Gao, Yonghong Zhu
The RegEx matches begining and end of the string, dont then check length.
Cc: Liming Gao <liming.gao@intel.com>
Cc: Yonghong Zhu <yonghong.zhu@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Jaben Carsey <jaben.carsey@intel.com>
---
BaseTools/Source/Python/AutoGen/IdfClassObject.py | 2 +-
BaseTools/Source/Python/AutoGen/UniClassObject.py | 4 ++--
2 files changed, 3 insertions(+), 3 deletions(-)
diff --git a/BaseTools/Source/Python/AutoGen/IdfClassObject.py b/BaseTools/Source/Python/AutoGen/IdfClassObject.py
index a028aceee927..6953854a5247 100644
--- a/BaseTools/Source/Python/AutoGen/IdfClassObject.py
+++ b/BaseTools/Source/Python/AutoGen/IdfClassObject.py
@@ -107,7 +107,7 @@ class IdfFileClassObject(object):
if Len == 4 and LineDetails[2] != 'TRANSPARENT':
EdkLogger.error("Image Definition File Parser", PARSER_ERROR, 'Please use the keyword "TRANSPARENT" to describe the transparency setting in Line %s of File %s.' % (LineNo, File.Path))
MatchString = gIdentifierPattern.match(LineDetails[1])
- if MatchString is None or MatchString.end(0) != len(LineDetails[1]):
+ if MatchString is None:
EdkLogger.error('Image Definition File Parser', FORMAT_INVALID, 'The Image token name %s defined in Idf file %s contains the invalid character.' % (LineDetails[1], File.Path))
if LineDetails[1] not in self.ImageIDList:
self.ImageIDList.append(LineDetails[1])
diff --git a/BaseTools/Source/Python/AutoGen/UniClassObject.py b/BaseTools/Source/Python/AutoGen/UniClassObject.py
index 8b0c563a8c88..5b879d784d9c 100644
--- a/BaseTools/Source/Python/AutoGen/UniClassObject.py
+++ b/BaseTools/Source/Python/AutoGen/UniClassObject.py
@@ -352,7 +352,7 @@ class UniFileClassObject(object):
# Check the string name
if Name != '':
MatchString = gIdentifierPattern.match(Name)
- if MatchString is None or MatchString.end(0) != len(Name):
+ if MatchString is None:
EdkLogger.error('Unicode File Parser', FORMAT_INVALID, 'The string token name %s defined in UNI file %s contains the invalid character.' % (Name, self.File))
LanguageList = Item.split(u'#language ')
for IndexI in range(len(LanguageList)):
@@ -522,7 +522,7 @@ class UniFileClassObject(object):
# Check the string name
if not self.IsCompatibleMode and Name != '':
MatchString = gIdentifierPattern.match(Name)
- if MatchString is None or MatchString.end(0) != len(Name):
+ if MatchString is None:
EdkLogger.error('Unicode File Parser', FORMAT_INVALID, 'The string token name %s defined in UNI file %s contains the invalid character.' % (Name, self.File))
self.AddStringToList(Name, Language, Value)
continue
--
2.16.2.windows.1
^ permalink raw reply related [flat|nested] 7+ messages in thread
* Re: [PATCH v1 0/5] BaseTools: RegEx refactoring
2018-04-03 22:34 [PATCH v1 0/5] BaseTools: RegEx refactoring Jaben Carsey
` (4 preceding siblings ...)
2018-04-03 22:34 ` [PATCH v1 5/5] BaseTools: remove redundant check Jaben Carsey
@ 2018-04-08 7:05 ` Zhu, Yonghong
5 siblings, 0 replies; 7+ messages in thread
From: Zhu, Yonghong @ 2018-04-08 7:05 UTC (permalink / raw)
To: Carsey, Jaben, edk2-devel@lists.01.org
The patches are good to me.
Reviewed-by: Yonghong Zhu <yonghong.zhu@intel.com>
Best Regards,
Zhu Yonghong
-----Original Message-----
From: edk2-devel [mailto:edk2-devel-bounces@lists.01.org] On Behalf Of Jaben Carsey
Sent: Wednesday, April 04, 2018 6:34 AM
To: edk2-devel@lists.01.org
Subject: [edk2] [PATCH v1 0/5] BaseTools: RegEx refactoring
move RegEx compile out of functions/loops share RegEx when have identical data
Jaben Carsey (5):
BaseTools: Autogen - move RegEx compile
BaseTools: GenFds - move RegEx compile
BaseTools: Add new RegEx pattern to GlobalData
BaseTools: AutoGen - use the new shared RegEx
BaseTools: remove redundant check
BaseTools/Source/Python/AutoGen/AutoGen.py | 22 +++++++++++++-------
BaseTools/Source/Python/AutoGen/IdfClassObject.py | 5 +++-- BaseTools/Source/Python/AutoGen/UniClassObject.py | 8 +++----
BaseTools/Source/Python/Common/GlobalData.py | 3 +++
BaseTools/Source/Python/GenFds/FdfParser.py | 5 ++---
5 files changed, 27 insertions(+), 16 deletions(-)
--
2.16.2.windows.1
_______________________________________________
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel
^ permalink raw reply [flat|nested] 7+ messages in thread
end of thread, other threads:[~2018-04-08 7:05 UTC | newest]
Thread overview: 7+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2018-04-03 22:34 [PATCH v1 0/5] BaseTools: RegEx refactoring Jaben Carsey
2018-04-03 22:34 ` [PATCH v1 1/5] BaseTools: Autogen - move RegEx compile Jaben Carsey
2018-04-03 22:34 ` [PATCH v1 2/5] BaseTools: GenFds " Jaben Carsey
2018-04-03 22:34 ` [PATCH v1 3/5] BaseTools: Add new RegEx pattern to GlobalData Jaben Carsey
2018-04-03 22:34 ` [PATCH v1 4/5] BaseTools: AutoGen - use the new shared RegEx Jaben Carsey
2018-04-03 22:34 ` [PATCH v1 5/5] BaseTools: remove redundant check Jaben Carsey
2018-04-08 7:05 ` [PATCH v1 0/5] BaseTools: RegEx refactoring Zhu, Yonghong
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox