* [edk2-devel] [PATCH v1 0/1] Fix raw strings containing valid escape characters
@ 2023-12-28 16:47 Joey Vagedes via groups.io
2023-12-28 16:47 ` [edk2-devel] [PATCH v1 1/1] BaseTools: " Joey Vagedes via groups.io
0 siblings, 1 reply; 12+ messages in thread
From: Joey Vagedes via groups.io @ 2023-12-28 16:47 UTC (permalink / raw)
To: devel; +Cc: Rebecca Cran, Liming Gao, Bob Feng, Yuwei Chen
Commit 9f0061a03b61d282fbc0ba5be22155d06a5e64a1 was intended to fix regex
strings using invalid escape characters. The regex module suggested way to
fix this is to use the raw string modifiler (`r'`). However in doing this,
any strings that also contained valid escape characters that were being used
in the regex strings (mainly \t and \n) were broken. This is because the raw
string modifier converted them from the tab character and newline characters to
the literal character components.
This patch series updates any of the raw strings that contained valid escape
characters to resolve the issue.
Cc: Rebecca Cran <rebecca@bsdio.com>
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Yuwei Chen <yuwei.chen@intel.com>
Joey Vagedes (1):
BaseTools: Fix raw strings containing valid escape characters
BaseTools/Source/Python/AutoGen/GenMake.py | 2 +-
BaseTools/Source/Python/Common/Misc.py | 2 +-
BaseTools/Source/Python/Workspace/DscBuildData.py | 4 ++--
3 files changed, 4 insertions(+), 4 deletions(-)
--
2.43.0.windows.1
-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.
View/Reply Online (#112985): https://edk2.groups.io/g/devel/message/112985
Mute This Topic: https://groups.io/mt/103403581/7686176
Group Owner: devel+owner@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub [rebecca@openfw.io]
-=-=-=-=-=-=-=-=-=-=-=-
^ permalink raw reply [flat|nested] 12+ messages in thread
* [edk2-devel] [PATCH v1 1/1] BaseTools: Fix raw strings containing valid escape characters
2023-12-28 16:47 [edk2-devel] [PATCH v1 0/1] Fix raw strings containing valid escape characters Joey Vagedes via groups.io
@ 2023-12-28 16:47 ` Joey Vagedes via groups.io
2024-01-08 1:15 ` Yuwei Chen
[not found] ` <17A83A4D186CC8F7.1387@groups.io>
0 siblings, 2 replies; 12+ messages in thread
From: Joey Vagedes via groups.io @ 2023-12-28 16:47 UTC (permalink / raw)
To: devel; +Cc: Rebecca Cran, Liming Gao, Bob Feng, Yuwei Chen
Fixes raw regex strings that contain valid (and purposeful) escape
characters as they are being treated as individual characters rather
than the single escaped character they represent (i.e. '\t' is being
treated as a '\' and a 't' rather than a single tab character).
Signed-off-by: Joey Vagedes <joey.vagedes@gmail.com>
Cc: Rebecca Cran <rebecca@bsdio.com>
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Yuwei Chen <yuwei.chen@intel.com>
---
BaseTools/Source/Python/AutoGen/GenMake.py | 2 +-
BaseTools/Source/Python/Common/Misc.py | 2 +-
BaseTools/Source/Python/Workspace/DscBuildData.py | 4 ++--
3 files changed, 4 insertions(+), 4 deletions(-)
diff --git a/BaseTools/Source/Python/AutoGen/GenMake.py b/BaseTools/Source/Python/AutoGen/GenMake.py
index c416fe172fe5..fbd35d498923 100755
--- a/BaseTools/Source/Python/AutoGen/GenMake.py
+++ b/BaseTools/Source/Python/AutoGen/GenMake.py
@@ -28,7 +28,7 @@ from Common.DataType import TAB_COMPILER_MSFT
gIncludePattern = re.compile(r"^[ \t]*[#%]?[ \t]*include(?:[ \t]*(?:\\(?:\r\n|\r|\n))*[ \t]*)*(?:\(?[\"<]?[ \t]*)([-\w.\\/() \t]+)(?:[ \t]*[\">]?\)?)", re.MULTILINE | re.UNICODE | re.IGNORECASE)
## Regular expression for matching macro used in header file inclusion
-gMacroPattern = re.compile(r"([_A-Z][_A-Z0-9]*)[ \t]*\((.+)\)", re.UNICODE)
+gMacroPattern = re.compile("([_A-Z][_A-Z0-9]*)[ \t]*\\((.+)\\)", re.UNICODE)
gIsFileMap = {}
diff --git a/BaseTools/Source/Python/Common/Misc.py b/BaseTools/Source/Python/Common/Misc.py
index f87d9dbdba39..34bfc90abd94 100755
--- a/BaseTools/Source/Python/Common/Misc.py
+++ b/BaseTools/Source/Python/Common/Misc.py
@@ -1926,4 +1926,4 @@ def CopyDict(ori_dict):
# Remove the c/c++ comments: // and /* */
#
def RemoveCComments(ctext):
- return re.sub(r'//.*?\n|/\*.*?\*/', '\n', ctext, flags=re.S)
+ return re.sub('//.*?\n|/\\*.*?\\*/', '\n', ctext, flags=re.S)
diff --git a/BaseTools/Source/Python/Workspace/DscBuildData.py b/BaseTools/Source/Python/Workspace/DscBuildData.py
index 817cdbe5f19c..4768099343e5 100644
--- a/BaseTools/Source/Python/Workspace/DscBuildData.py
+++ b/BaseTools/Source/Python/Workspace/DscBuildData.py
@@ -2840,7 +2840,7 @@ class DscBuildData(PlatformBuildClassObject):
# start generating makefile
MakeApp = PcdMakefileHeader
if sys.platform == "win32":
- MakeApp = MakeApp + r'APPFILE = %s\%s.exe\n' % (self.OutputPath, PcdValueInitName) + r'APPNAME = %s\n' % (PcdValueInitName) + r'OBJECTS = %s\%s.obj %s.obj\n' % (self.OutputPath, PcdValueInitName, os.path.join(self.OutputPath, PcdValueCommonName)) + 'INC = '
+ MakeApp = MakeApp + 'APPFILE = %s\\%s.exe\n' % (self.OutputPath, PcdValueInitName) + 'APPNAME = %s\n' % (PcdValueInitName) + 'OBJECTS = %s\\%s.obj %s.obj\n' % (self.OutputPath, PcdValueInitName, os.path.join(self.OutputPath, PcdValueCommonName)) + 'INC = '
else:
MakeApp = MakeApp + PcdGccMakefile
MakeApp = MakeApp + 'APPFILE = %s/%s\n' % (self.OutputPath, PcdValueInitName) + 'APPNAME = %s\n' % (PcdValueInitName) + 'OBJECTS = %s/%s.o %s.o\n' % (self.OutputPath, PcdValueInitName, os.path.join(self.OutputPath, PcdValueCommonName)) + \
@@ -2950,7 +2950,7 @@ class DscBuildData(PlatformBuildClassObject):
MakeApp += "$(OBJECTS) : %s\n" % include_file
if sys.platform == "win32":
PcdValueCommonPath = os.path.normpath(mws.join(GlobalData.gGlobalDefines["EDK_TOOLS_PATH"], "Source\C\Common\PcdValueCommon.c"))
- MakeApp = MakeApp + r'%s\PcdValueCommon.c : %s\n' % (self.OutputPath, PcdValueCommonPath)
+ MakeApp = MakeApp + '%s\\PcdValueCommon.c : %s\n' % (self.OutputPath, PcdValueCommonPath)
MakeApp = MakeApp + '\tcopy /y %s $@\n' % (PcdValueCommonPath)
else:
PcdValueCommonPath = os.path.normpath(mws.join(GlobalData.gGlobalDefines["EDK_TOOLS_PATH"], "Source/C/Common/PcdValueCommon.c"))
--
2.43.0.windows.1
-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.
View/Reply Online (#112986): https://edk2.groups.io/g/devel/message/112986
Mute This Topic: https://groups.io/mt/103403582/7686176
Group Owner: devel+owner@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub [rebecca@openfw.io]
-=-=-=-=-=-=-=-=-=-=-=-
^ permalink raw reply related [flat|nested] 12+ messages in thread
* Re: [edk2-devel] [PATCH v1 1/1] BaseTools: Fix raw strings containing valid escape characters
2023-12-28 16:47 ` [edk2-devel] [PATCH v1 1/1] BaseTools: " Joey Vagedes via groups.io
@ 2024-01-08 1:15 ` Yuwei Chen
[not found] ` <17A83A4D186CC8F7.1387@groups.io>
1 sibling, 0 replies; 12+ messages in thread
From: Yuwei Chen @ 2024-01-08 1:15 UTC (permalink / raw)
To: Joey Vagedes, devel@edk2.groups.io; +Cc: Rebecca Cran, Gao, Liming, Feng, Bob C
Hi Joey,
Thanks for fixing.
Reviewed-by: Yuwei Chen <yuwei.chen@intel.com>
> -----Original Message-----
> From: Joey Vagedes <joey.vagedes@gmail.com>
> Sent: Friday, December 29, 2023 12:48 AM
> To: devel@edk2.groups.io
> Cc: Rebecca Cran <rebecca@bsdio.com>; Gao, Liming
> <gaoliming@byosoft.com.cn>; Feng, Bob C <bob.c.feng@intel.com>; Chen,
> Christine <yuwei.chen@intel.com>
> Subject: [PATCH v1 1/1] BaseTools: Fix raw strings containing valid escape
> characters
>
> Fixes raw regex strings that contain valid (and purposeful) escape characters
> as they are being treated as individual characters rather than the single
> escaped character they represent (i.e. '\t' is being treated as a '\' and a 't'
> rather than a single tab character).
>
> Signed-off-by: Joey Vagedes <joey.vagedes@gmail.com>
> Cc: Rebecca Cran <rebecca@bsdio.com>
> Cc: Liming Gao <gaoliming@byosoft.com.cn>
> Cc: Bob Feng <bob.c.feng@intel.com>
> Cc: Yuwei Chen <yuwei.chen@intel.com>
> ---
> BaseTools/Source/Python/AutoGen/GenMake.py | 2 +-
> BaseTools/Source/Python/Common/Misc.py | 2 +-
> BaseTools/Source/Python/Workspace/DscBuildData.py | 4 ++--
> 3 files changed, 4 insertions(+), 4 deletions(-)
>
> diff --git a/BaseTools/Source/Python/AutoGen/GenMake.py
> b/BaseTools/Source/Python/AutoGen/GenMake.py
> index c416fe172fe5..fbd35d498923 100755
> --- a/BaseTools/Source/Python/AutoGen/GenMake.py
> +++ b/BaseTools/Source/Python/AutoGen/GenMake.py
> @@ -28,7 +28,7 @@ from Common.DataType import TAB_COMPILER_MSFT
> gIncludePattern =
> re.compile(r"^[ \t]*[#%]?[ \t]*include(?:[ \t]*(?:\\(?:\r\n|\r|\n))*[ \t]*)*(?:\(?[
> \"<]?[ \t]*)([-\w.\\/() \t]+)(?:[ \t]*[\">]?\)?)", re.MULTILINE | re.UNICODE |
> re.IGNORECASE) ## Regular expression for matching macro used in header
> file inclusion-gMacroPattern = re.compile(r"([_A-Z][_A-Z0-9]*)[ \t]*\((.+)\)",
> re.UNICODE)+gMacroPattern = re.compile("([_A-Z][_A-Z0-9]*)[ \t]*\\((.+)\\)",
> re.UNICODE) gIsFileMap = {} diff --git
> a/BaseTools/Source/Python/Common/Misc.py
> b/BaseTools/Source/Python/Common/Misc.py
> index f87d9dbdba39..34bfc90abd94 100755
> --- a/BaseTools/Source/Python/Common/Misc.py
> +++ b/BaseTools/Source/Python/Common/Misc.py
> @@ -1926,4 +1926,4 @@ def CopyDict(ori_dict):
> # Remove the c/c++ comments: // and /* */ # def
> RemoveCComments(ctext):- return re.sub(r'//.*?\n|/\*.*?\*/', '\n', ctext,
> flags=re.S)+ return re.sub('//.*?\n|/\\*.*?\\*/', '\n', ctext, flags=re.S)diff --git
> a/BaseTools/Source/Python/Workspace/DscBuildData.py
> b/BaseTools/Source/Python/Workspace/DscBuildData.py
> index 817cdbe5f19c..4768099343e5 100644
> --- a/BaseTools/Source/Python/Workspace/DscBuildData.py
> +++ b/BaseTools/Source/Python/Workspace/DscBuildData.py
> @@ -2840,7 +2840,7 @@ class DscBuildData(PlatformBuildClassObject):
> # start generating makefile MakeApp = PcdMakefileHeader if
> sys.platform == "win32":- MakeApp = MakeApp + r'APPFILE
> = %s\%s.exe\n' % (self.OutputPath, PcdValueInitName) + r'APPNAME
> = %s\n' % (PcdValueInitName) + r'OBJECTS = %s\%s.obj %s.obj\n' %
> (self.OutputPath, PcdValueInitName, os.path.join(self.OutputPath,
> PcdValueCommonName)) + 'INC = '+ MakeApp = MakeApp + 'APPFILE
> = %s\\%s.exe\n' % (self.OutputPath, PcdValueInitName) + 'APPNAME
> = %s\n' % (PcdValueInitName) + 'OBJECTS = %s\\%s.obj %s.obj\n' %
> (self.OutputPath, PcdValueInitName, os.path.join(self.OutputPath,
> PcdValueCommonName)) + 'INC = ' else: MakeApp = MakeApp +
> PcdGccMakefile MakeApp = MakeApp + 'APPFILE = %s/%s\n' %
> (self.OutputPath, PcdValueInitName) + 'APPNAME = %s\n' %
> (PcdValueInitName) + 'OBJECTS = %s/%s.o %s.o\n' % (self.OutputPath,
> PcdValueInitName, os.path.join(self.OutputPath, PcdValueCommonName)) +
> \@@ -2950,7 +2950,7 @@ class DscBuildData(PlatformBuildClassObject):
> MakeApp += "$(OBJECTS) : %s\n" % include_file if sys.platform ==
> "win32": PcdValueCommonPath =
> os.path.normpath(mws.join(GlobalData.gGlobalDefines["EDK_TOOLS_PATH"]
> , "Source\C\Common\PcdValueCommon.c"))- MakeApp = MakeApp +
> r'%s\PcdValueCommon.c : %s\n' % (self.OutputPath, PcdValueCommonPath)+
> MakeApp = MakeApp + '%s\\PcdValueCommon.c : %s\n' % (self.OutputPath,
> PcdValueCommonPath) MakeApp = MakeApp + '\tcopy /y %s $@\n' %
> (PcdValueCommonPath) else: PcdValueCommonPath =
> os.path.normpath(mws.join(GlobalData.gGlobalDefines["EDK_TOOLS_PATH"]
> , "Source/C/Common/PcdValueCommon.c"))--
> 2.43.0.windows.1
-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.
View/Reply Online (#113338): https://edk2.groups.io/g/devel/message/113338
Mute This Topic: https://groups.io/mt/103403582/7686176
Group Owner: devel+owner@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub [rebecca@openfw.io]
-=-=-=-=-=-=-=-=-=-=-=-
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [edk2-devel] [PATCH v1 1/1] BaseTools: Fix raw strings containing valid escape characters
[not found] ` <17A83A4D186CC8F7.1387@groups.io>
@ 2024-01-08 5:37 ` Yuwei Chen
2024-01-08 6:06 ` Ni, Ray
2024-01-08 13:25 ` 回复: " gaoliming via groups.io
0 siblings, 2 replies; 12+ messages in thread
From: Yuwei Chen @ 2024-01-08 5:37 UTC (permalink / raw)
To: devel@edk2.groups.io, Chen, Christine, Joey Vagedes
Cc: Rebecca Cran, Gao, Liming, Feng, Bob C, Huang, Yanbo,
Kinney, Michael D
Hi Liming and Cran,
We should merge this fixing patch asap as it influences BaseTool's usage.
Thanks,
Christine
> -----Original Message-----
> From: devel@edk2.groups.io <devel@edk2.groups.io> On Behalf Of Yuwei
> Chen
> Sent: Monday, January 8, 2024 9:16 AM
> To: Joey Vagedes <joey.vagedes@gmail.com>; devel@edk2.groups.io
> Cc: Rebecca Cran <rebecca@bsdio.com>; Gao, Liming
> <gaoliming@byosoft.com.cn>; Feng, Bob C <bob.c.feng@intel.com>
> Subject: Re: [edk2-devel] [PATCH v1 1/1] BaseTools: Fix raw strings containing
> valid escape characters
>
> Hi Joey,
>
> Thanks for fixing.
>
> Reviewed-by: Yuwei Chen <yuwei.chen@intel.com>
>
> > -----Original Message-----
> > From: Joey Vagedes <joey.vagedes@gmail.com>
> > Sent: Friday, December 29, 2023 12:48 AM
> > To: devel@edk2.groups.io
> > Cc: Rebecca Cran <rebecca@bsdio.com>; Gao, Liming
> > <gaoliming@byosoft.com.cn>; Feng, Bob C <bob.c.feng@intel.com>; Chen,
> > Christine <yuwei.chen@intel.com>
> > Subject: [PATCH v1 1/1] BaseTools: Fix raw strings containing valid
> > escape characters
> >
> > Fixes raw regex strings that contain valid (and purposeful) escape
> > characters as they are being treated as individual characters rather
> > than the single escaped character they represent (i.e. '\t' is being treated as
> a '\' and a 't'
> > rather than a single tab character).
> >
> > Signed-off-by: Joey Vagedes <joey.vagedes@gmail.com>
> > Cc: Rebecca Cran <rebecca@bsdio.com>
> > Cc: Liming Gao <gaoliming@byosoft.com.cn>
> > Cc: Bob Feng <bob.c.feng@intel.com>
> > Cc: Yuwei Chen <yuwei.chen@intel.com>
> > ---
> > BaseTools/Source/Python/AutoGen/GenMake.py | 2 +-
> > BaseTools/Source/Python/Common/Misc.py | 2 +-
> > BaseTools/Source/Python/Workspace/DscBuildData.py | 4 ++--
> > 3 files changed, 4 insertions(+), 4 deletions(-)
> >
> > diff --git a/BaseTools/Source/Python/AutoGen/GenMake.py
> > b/BaseTools/Source/Python/AutoGen/GenMake.py
> > index c416fe172fe5..fbd35d498923 100755
> > --- a/BaseTools/Source/Python/AutoGen/GenMake.py
> > +++ b/BaseTools/Source/Python/AutoGen/GenMake.py
> > @@ -28,7 +28,7 @@ from Common.DataType import
> TAB_COMPILER_MSFT
> > gIncludePattern = re.compile(r"^[ \t]*[#%]?[ \t]*include(?:[
> > \t]*(?:\\(?:\r\n|\r|\n))*[ \t]*)*(?:\(?[ \"<]?[ \t]*)([-\w.\\/()
> > \t]+)(?:[ \t]*[\">]?\)?)", re.MULTILINE | re.UNICODE |
> > re.IGNORECASE) ## Regular expression for matching macro used in
> > header file inclusion-gMacroPattern = re.compile(r"([_A-Z][_A-Z0-9]*)[
> > \t]*\((.+)\)", re.UNICODE)+gMacroPattern =
> > re.compile("([_A-Z][_A-Z0-9]*)[ \t]*\\((.+)\\)",
> > re.UNICODE) gIsFileMap = {} diff --git
> > a/BaseTools/Source/Python/Common/Misc.py
> > b/BaseTools/Source/Python/Common/Misc.py
> > index f87d9dbdba39..34bfc90abd94 100755
> > --- a/BaseTools/Source/Python/Common/Misc.py
> > +++ b/BaseTools/Source/Python/Common/Misc.py
> > @@ -1926,4 +1926,4 @@ def CopyDict(ori_dict):
> > # Remove the c/c++ comments: // and /* */ # def
> > RemoveCComments(ctext):- return re.sub(r'//.*?\n|/\*.*?\*/', '\n', ctext,
> > flags=re.S)+ return re.sub('//.*?\n|/\\*.*?\\*/', '\n', ctext, flags=re.S)diff --
> git
> > a/BaseTools/Source/Python/Workspace/DscBuildData.py
> > b/BaseTools/Source/Python/Workspace/DscBuildData.py
> > index 817cdbe5f19c..4768099343e5 100644
> > --- a/BaseTools/Source/Python/Workspace/DscBuildData.py
> > +++ b/BaseTools/Source/Python/Workspace/DscBuildData.py
> > @@ -2840,7 +2840,7 @@ class DscBuildData(PlatformBuildClassObject):
> > # start generating makefile MakeApp = PcdMakefileHeader if
> > sys.platform == "win32":- MakeApp = MakeApp + r'APPFILE
> > = %s\%s.exe\n' % (self.OutputPath, PcdValueInitName) + r'APPNAME =
> > %s\n' % (PcdValueInitName) + r'OBJECTS = %s\%s.obj %s.obj\n' %
> > (self.OutputPath, PcdValueInitName, os.path.join(self.OutputPath,
> > PcdValueCommonName)) + 'INC = '+ MakeApp = MakeApp + 'APPFILE
> > = %s\\%s.exe\n' % (self.OutputPath, PcdValueInitName) + 'APPNAME =
> > %s\n' % (PcdValueInitName) + 'OBJECTS = %s\\%s.obj %s.obj\n' %
> > (self.OutputPath, PcdValueInitName, os.path.join(self.OutputPath,
> > PcdValueCommonName)) + 'INC = ' else: MakeApp = MakeApp +
> > PcdGccMakefile MakeApp = MakeApp + 'APPFILE = %s/%s\n' %
> > (self.OutputPath, PcdValueInitName) + 'APPNAME = %s\n' %
> > (PcdValueInitName) + 'OBJECTS = %s/%s.o %s.o\n' % (self.OutputPath,
> > PcdValueInitName, os.path.join(self.OutputPath, PcdValueCommonName))
> +
> > \@@ -2950,7 +2950,7 @@ class DscBuildData(PlatformBuildClassObject):
> > MakeApp += "$(OBJECTS) : %s\n" % include_file if sys.platform
> ==
> > "win32": PcdValueCommonPath =
> >
> os.path.normpath(mws.join(GlobalData.gGlobalDefines["EDK_TOOLS_PATH"]
> > , "Source\C\Common\PcdValueCommon.c"))- MakeApp = MakeApp +
> > r'%s\PcdValueCommon.c : %s\n' % (self.OutputPath,
> PcdValueCommonPath)+
> > MakeApp = MakeApp + '%s\\PcdValueCommon.c : %s\n' % (self.OutputPath,
> > PcdValueCommonPath) MakeApp = MakeApp + '\tcopy /y %s
> $@\n' %
> > (PcdValueCommonPath) else: PcdValueCommonPath =
> >
> os.path.normpath(mws.join(GlobalData.gGlobalDefines["EDK_TOOLS_PATH"]
> > , "Source/C/Common/PcdValueCommon.c"))--
> > 2.43.0.windows.1
>
>
>
>
>
-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.
View/Reply Online (#113358): https://edk2.groups.io/g/devel/message/113358
Mute This Topic: https://groups.io/mt/103403582/7686176
Group Owner: devel+owner@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub [rebecca@openfw.io]
-=-=-=-=-=-=-=-=-=-=-=-
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [edk2-devel] [PATCH v1 1/1] BaseTools: Fix raw strings containing valid escape characters
2024-01-08 5:37 ` Yuwei Chen
@ 2024-01-08 6:06 ` Ni, Ray
2024-01-08 13:25 ` 回复: " gaoliming via groups.io
1 sibling, 0 replies; 12+ messages in thread
From: Ni, Ray @ 2024-01-08 6:06 UTC (permalink / raw)
To: devel@edk2.groups.io, Chen, Christine, Joey Vagedes
Cc: Rebecca Cran, Gao, Liming, Feng, Bob C, Huang, Yanbo,
Kinney, Michael D
Yes. Please!
My local build cannot pass without this patch.
Thanks,
Ray
> -----Original Message-----
> From: devel@edk2.groups.io <devel@edk2.groups.io> On Behalf Of Yuwei
> Chen
> Sent: Monday, January 8, 2024 1:37 PM
> To: devel@edk2.groups.io; Chen, Christine <yuwei.chen@intel.com>; Joey
> Vagedes <joey.vagedes@gmail.com>
> Cc: Rebecca Cran <rebecca@bsdio.com>; Gao, Liming
> <gaoliming@byosoft.com.cn>; Feng, Bob C <bob.c.feng@intel.com>; Huang,
> Yanbo <yanbo.huang@intel.com>; Kinney, Michael D
> <michael.d.kinney@intel.com>
> Subject: Re: [edk2-devel] [PATCH v1 1/1] BaseTools: Fix raw strings containing
> valid escape characters
>
> Hi Liming and Cran,
>
> We should merge this fixing patch asap as it influences BaseTool's usage.
>
> Thanks,
> Christine
>
> > -----Original Message-----
> > From: devel@edk2.groups.io <devel@edk2.groups.io> On Behalf Of Yuwei
> > Chen
> > Sent: Monday, January 8, 2024 9:16 AM
> > To: Joey Vagedes <joey.vagedes@gmail.com>; devel@edk2.groups.io
> > Cc: Rebecca Cran <rebecca@bsdio.com>; Gao, Liming
> > <gaoliming@byosoft.com.cn>; Feng, Bob C <bob.c.feng@intel.com>
> > Subject: Re: [edk2-devel] [PATCH v1 1/1] BaseTools: Fix raw strings
> containing
> > valid escape characters
> >
> > Hi Joey,
> >
> > Thanks for fixing.
> >
> > Reviewed-by: Yuwei Chen <yuwei.chen@intel.com>
> >
> > > -----Original Message-----
> > > From: Joey Vagedes <joey.vagedes@gmail.com>
> > > Sent: Friday, December 29, 2023 12:48 AM
> > > To: devel@edk2.groups.io
> > > Cc: Rebecca Cran <rebecca@bsdio.com>; Gao, Liming
> > > <gaoliming@byosoft.com.cn>; Feng, Bob C <bob.c.feng@intel.com>; Chen,
> > > Christine <yuwei.chen@intel.com>
> > > Subject: [PATCH v1 1/1] BaseTools: Fix raw strings containing valid
> > > escape characters
> > >
> > > Fixes raw regex strings that contain valid (and purposeful) escape
> > > characters as they are being treated as individual characters rather
> > > than the single escaped character they represent (i.e. '\t' is being treated as
> > a '\' and a 't'
> > > rather than a single tab character).
> > >
> > > Signed-off-by: Joey Vagedes <joey.vagedes@gmail.com>
> > > Cc: Rebecca Cran <rebecca@bsdio.com>
> > > Cc: Liming Gao <gaoliming@byosoft.com.cn>
> > > Cc: Bob Feng <bob.c.feng@intel.com>
> > > Cc: Yuwei Chen <yuwei.chen@intel.com>
> > > ---
> > > BaseTools/Source/Python/AutoGen/GenMake.py | 2 +-
> > > BaseTools/Source/Python/Common/Misc.py | 2 +-
> > > BaseTools/Source/Python/Workspace/DscBuildData.py | 4 ++--
> > > 3 files changed, 4 insertions(+), 4 deletions(-)
> > >
> > > diff --git a/BaseTools/Source/Python/AutoGen/GenMake.py
> > > b/BaseTools/Source/Python/AutoGen/GenMake.py
> > > index c416fe172fe5..fbd35d498923 100755
> > > --- a/BaseTools/Source/Python/AutoGen/GenMake.py
> > > +++ b/BaseTools/Source/Python/AutoGen/GenMake.py
> > > @@ -28,7 +28,7 @@ from Common.DataType import
> > TAB_COMPILER_MSFT
> > > gIncludePattern = re.compile(r"^[ \t]*[#%]?[ \t]*include(?:[
> > > \t]*(?:\\(?:\r\n|\r|\n))*[ \t]*)*(?:\(?[ \"<]?[ \t]*)([-\w.\\/()
> > > \t]+)(?:[ \t]*[\">]?\)?)", re.MULTILINE | re.UNICODE |
> > > re.IGNORECASE) ## Regular expression for matching macro used in
> > > header file inclusion-gMacroPattern = re.compile(r"([_A-Z][_A-Z0-9]*)[
> > > \t]*\((.+)\)", re.UNICODE)+gMacroPattern =
> > > re.compile("([_A-Z][_A-Z0-9]*)[ \t]*\\((.+)\\)",
> > > re.UNICODE) gIsFileMap = {} diff --git
> > > a/BaseTools/Source/Python/Common/Misc.py
> > > b/BaseTools/Source/Python/Common/Misc.py
> > > index f87d9dbdba39..34bfc90abd94 100755
> > > --- a/BaseTools/Source/Python/Common/Misc.py
> > > +++ b/BaseTools/Source/Python/Common/Misc.py
> > > @@ -1926,4 +1926,4 @@ def CopyDict(ori_dict):
> > > # Remove the c/c++ comments: // and /* */ # def
> > > RemoveCComments(ctext):- return re.sub(r'//.*?\n|/\*.*?\*/', '\n',
> ctext,
> > > flags=re.S)+ return re.sub('//.*?\n|/\\*.*?\\*/', '\n', ctext, flags=re.S)diff
> --
> > git
> > > a/BaseTools/Source/Python/Workspace/DscBuildData.py
> > > b/BaseTools/Source/Python/Workspace/DscBuildData.py
> > > index 817cdbe5f19c..4768099343e5 100644
> > > --- a/BaseTools/Source/Python/Workspace/DscBuildData.py
> > > +++ b/BaseTools/Source/Python/Workspace/DscBuildData.py
> > > @@ -2840,7 +2840,7 @@ class DscBuildData(PlatformBuildClassObject):
> > > # start generating makefile MakeApp = PcdMakefileHeader if
> > > sys.platform == "win32":- MakeApp = MakeApp + r'APPFILE
> > > = %s\%s.exe\n' % (self.OutputPath, PcdValueInitName) + r'APPNAME =
> > > %s\n' % (PcdValueInitName) + r'OBJECTS = %s\%s.obj %s.obj\n' %
> > > (self.OutputPath, PcdValueInitName, os.path.join(self.OutputPath,
> > > PcdValueCommonName)) + 'INC = '+ MakeApp = MakeApp + 'APPFILE
> > > = %s\\%s.exe\n' % (self.OutputPath, PcdValueInitName) + 'APPNAME =
> > > %s\n' % (PcdValueInitName) + 'OBJECTS = %s\\%s.obj %s.obj\n' %
> > > (self.OutputPath, PcdValueInitName, os.path.join(self.OutputPath,
> > > PcdValueCommonName)) + 'INC = ' else: MakeApp = MakeApp +
> > > PcdGccMakefile MakeApp = MakeApp + 'APPFILE = %s/%s\n' %
> > > (self.OutputPath, PcdValueInitName) + 'APPNAME = %s\n' %
> > > (PcdValueInitName) + 'OBJECTS = %s/%s.o %s.o\n' % (self.OutputPath,
> > > PcdValueInitName, os.path.join(self.OutputPath,
> PcdValueCommonName))
> > +
> > > \@@ -2950,7 +2950,7 @@ class DscBuildData(PlatformBuildClassObject):
> > > MakeApp += "$(OBJECTS) : %s\n" % include_file if sys.platform
> > ==
> > > "win32": PcdValueCommonPath =
> > >
> >
> os.path.normpath(mws.join(GlobalData.gGlobalDefines["EDK_TOOLS_PATH"]
> > > , "Source\C\Common\PcdValueCommon.c"))- MakeApp = MakeApp
> +
> > > r'%s\PcdValueCommon.c : %s\n' % (self.OutputPath,
> > PcdValueCommonPath)+
> > > MakeApp = MakeApp + '%s\\PcdValueCommon.c : %s\n' %
> (self.OutputPath,
> > > PcdValueCommonPath) MakeApp = MakeApp + '\tcopy /y %s
> > $@\n' %
> > > (PcdValueCommonPath) else: PcdValueCommonPath =
> > >
> >
> os.path.normpath(mws.join(GlobalData.gGlobalDefines["EDK_TOOLS_PATH"]
> > > , "Source/C/Common/PcdValueCommon.c"))--
> > > 2.43.0.windows.1
> >
> >
> >
> >
> >
>
>
>
>
>
-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.
View/Reply Online (#113359): https://edk2.groups.io/g/devel/message/113359
Mute This Topic: https://groups.io/mt/103403582/7686176
Group Owner: devel+owner@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/leave/12367111/7686176/1913456212/xyzzy [rebecca@openfw.io]
-=-=-=-=-=-=-=-=-=-=-=-
^ permalink raw reply [flat|nested] 12+ messages in thread
* 回复: [edk2-devel] [PATCH v1 1/1] BaseTools: Fix raw strings containing valid escape characters
2024-01-08 5:37 ` Yuwei Chen
2024-01-08 6:06 ` Ni, Ray
@ 2024-01-08 13:25 ` gaoliming via groups.io
2024-01-08 13:59 ` Huang, Yanbo
1 sibling, 1 reply; 12+ messages in thread
From: gaoliming via groups.io @ 2024-01-08 13:25 UTC (permalink / raw)
To: devel, yuwei.chen, 'Joey Vagedes'
Cc: 'Rebecca Cran', 'Feng, Bob C',
'Huang, Yanbo', 'Kinney, Michael D'
Christine:
Is this a regression issue? Or does it resolve new usage? I am curious why
it is not reported before.
Thanks
Liming
> -----邮件原件-----
> 发件人: devel@edk2.groups.io <devel@edk2.groups.io> 代表 Yuwei Chen
> 发送时间: 2024年1月8日 13:37
> 收件人: devel@edk2.groups.io; Chen, Christine <yuwei.chen@intel.com>;
> Joey Vagedes <joey.vagedes@gmail.com>
> 抄送: Rebecca Cran <rebecca@bsdio.com>; Gao, Liming
> <gaoliming@byosoft.com.cn>; Feng, Bob C <bob.c.feng@intel.com>; Huang,
> Yanbo <yanbo.huang@intel.com>; Kinney, Michael D
> <michael.d.kinney@intel.com>
> 主题: Re: [edk2-devel] [PATCH v1 1/1] BaseTools: Fix raw strings
containing
> valid escape characters
>
> Hi Liming and Cran,
>
> We should merge this fixing patch asap as it influences BaseTool's usage.
>
> Thanks,
> Christine
>
> > -----Original Message-----
> > From: devel@edk2.groups.io <devel@edk2.groups.io> On Behalf Of Yuwei
> > Chen
> > Sent: Monday, January 8, 2024 9:16 AM
> > To: Joey Vagedes <joey.vagedes@gmail.com>; devel@edk2.groups.io
> > Cc: Rebecca Cran <rebecca@bsdio.com>; Gao, Liming
> > <gaoliming@byosoft.com.cn>; Feng, Bob C <bob.c.feng@intel.com>
> > Subject: Re: [edk2-devel] [PATCH v1 1/1] BaseTools: Fix raw strings
> containing
> > valid escape characters
> >
> > Hi Joey,
> >
> > Thanks for fixing.
> >
> > Reviewed-by: Yuwei Chen <yuwei.chen@intel.com>
> >
> > > -----Original Message-----
> > > From: Joey Vagedes <joey.vagedes@gmail.com>
> > > Sent: Friday, December 29, 2023 12:48 AM
> > > To: devel@edk2.groups.io
> > > Cc: Rebecca Cran <rebecca@bsdio.com>; Gao, Liming
> > > <gaoliming@byosoft.com.cn>; Feng, Bob C <bob.c.feng@intel.com>;
> Chen,
> > > Christine <yuwei.chen@intel.com>
> > > Subject: [PATCH v1 1/1] BaseTools: Fix raw strings containing valid
> > > escape characters
> > >
> > > Fixes raw regex strings that contain valid (and purposeful) escape
> > > characters as they are being treated as individual characters rather
> > > than the single escaped character they represent (i.e. '\t' is being
treated
> as
> > a '\' and a 't'
> > > rather than a single tab character).
> > >
> > > Signed-off-by: Joey Vagedes <joey.vagedes@gmail.com>
> > > Cc: Rebecca Cran <rebecca@bsdio.com>
> > > Cc: Liming Gao <gaoliming@byosoft.com.cn>
> > > Cc: Bob Feng <bob.c.feng@intel.com>
> > > Cc: Yuwei Chen <yuwei.chen@intel.com>
> > > ---
> > > BaseTools/Source/Python/AutoGen/GenMake.py | 2 +-
> > > BaseTools/Source/Python/Common/Misc.py | 2 +-
> > > BaseTools/Source/Python/Workspace/DscBuildData.py | 4 ++--
> > > 3 files changed, 4 insertions(+), 4 deletions(-)
> > >
> > > diff --git a/BaseTools/Source/Python/AutoGen/GenMake.py
> > > b/BaseTools/Source/Python/AutoGen/GenMake.py
> > > index c416fe172fe5..fbd35d498923 100755
> > > --- a/BaseTools/Source/Python/AutoGen/GenMake.py
> > > +++ b/BaseTools/Source/Python/AutoGen/GenMake.py
> > > @@ -28,7 +28,7 @@ from Common.DataType import
> > TAB_COMPILER_MSFT
> > > gIncludePattern = re.compile(r"^[ \t]*[#%]?[ \t]*include(?:[
> > > \t]*(?:\\(?:\r\n|\r|\n))*[ \t]*)*(?:\(?[ \"<]?[ \t]*)([-\w.\\/()
> > > \t]+)(?:[ \t]*[\">]?\)?)", re.MULTILINE | re.UNICODE |
> > > re.IGNORECASE) ## Regular expression for matching macro used in
> > > header file inclusion-gMacroPattern = re.compile(r"([_A-Z][_A-Z0-9]*)[
> > > \t]*\((.+)\)", re.UNICODE)+gMacroPattern =
> > > re.compile("([_A-Z][_A-Z0-9]*)[ \t]*\\((.+)\\)",
> > > re.UNICODE) gIsFileMap = {} diff --git
> > > a/BaseTools/Source/Python/Common/Misc.py
> > > b/BaseTools/Source/Python/Common/Misc.py
> > > index f87d9dbdba39..34bfc90abd94 100755
> > > --- a/BaseTools/Source/Python/Common/Misc.py
> > > +++ b/BaseTools/Source/Python/Common/Misc.py
> > > @@ -1926,4 +1926,4 @@ def CopyDict(ori_dict):
> > > # Remove the c/c++ comments: // and /* */ # def
> > > RemoveCComments(ctext):- return re.sub(r'//.*?\n|/\*.*?\*/', '\n',
> ctext,
> > > flags=re.S)+ return re.sub('//.*?\n|/\\*.*?\\*/', '\n', ctext,
> flags=re.S)diff --
> > git
> > > a/BaseTools/Source/Python/Workspace/DscBuildData.py
> > > b/BaseTools/Source/Python/Workspace/DscBuildData.py
> > > index 817cdbe5f19c..4768099343e5 100644
> > > --- a/BaseTools/Source/Python/Workspace/DscBuildData.py
> > > +++ b/BaseTools/Source/Python/Workspace/DscBuildData.py
> > > @@ -2840,7 +2840,7 @@ class DscBuildData(PlatformBuildClassObject):
> > > # start generating makefile MakeApp =
> PcdMakefileHeader if
> > > sys.platform == "win32":- MakeApp = MakeApp + r'APPFILE
> > > = %s\%s.exe\n' % (self.OutputPath, PcdValueInitName) + r'APPNAME =
> > > %s\n' % (PcdValueInitName) + r'OBJECTS = %s\%s.obj %s.obj\n' %
> > > (self.OutputPath, PcdValueInitName, os.path.join(self.OutputPath,
> > > PcdValueCommonName)) + 'INC = '+ MakeApp = MakeApp
> + 'APPFILE
> > > = %s\\%s.exe\n' % (self.OutputPath, PcdValueInitName) + 'APPNAME =
> > > %s\n' % (PcdValueInitName) + 'OBJECTS = %s\\%s.obj %s.obj\n' %
> > > (self.OutputPath, PcdValueInitName, os.path.join(self.OutputPath,
> > > PcdValueCommonName)) + 'INC = ' else:
> MakeApp = MakeApp +
> > > PcdGccMakefile MakeApp = MakeApp + 'APPFILE
> = %s/%s\n' %
> > > (self.OutputPath, PcdValueInitName) + 'APPNAME = %s\n' %
> > > (PcdValueInitName) + 'OBJECTS = %s/%s.o %s.o\n' % (self.OutputPath,
> > > PcdValueInitName, os.path.join(self.OutputPath, PcdValueCommonName))
> > +
> > > \@@ -2950,7 +2950,7 @@ class DscBuildData(PlatformBuildClassObject):
> > > MakeApp += "$(OBJECTS) : %s\n" % include_file
> if sys.platform
> > ==
> > > "win32": PcdValueCommonPath =
> > >
> >
> os.path.normpath(mws.join(GlobalData.gGlobalDefines["EDK_TOOLS_PATH"]
> > > , "Source\C\Common\PcdValueCommon.c"))- MakeApp =
> MakeApp +
> > > r'%s\PcdValueCommon.c : %s\n' % (self.OutputPath,
> > PcdValueCommonPath)+
> > > MakeApp = MakeApp + '%s\\PcdValueCommon.c : %s\n' %
> (self.OutputPath,
> > > PcdValueCommonPath) MakeApp = MakeApp + '\tcopy
> /y %s
> > $@\n' %
> > > (PcdValueCommonPath) else:
> PcdValueCommonPath =
> > >
> >
> os.path.normpath(mws.join(GlobalData.gGlobalDefines["EDK_TOOLS_PATH"]
> > > , "Source/C/Common/PcdValueCommon.c"))--
> > > 2.43.0.windows.1
> >
> >
> >
> >
> >
>
>
>
>
>
-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.
View/Reply Online (#113391): https://edk2.groups.io/g/devel/message/113391
Mute This Topic: https://groups.io/mt/103596531/7686176
Group Owner: devel+owner@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub [rebecca@openfw.io]
-=-=-=-=-=-=-=-=-=-=-=-
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [edk2-devel] [PATCH v1 1/1] BaseTools: Fix raw strings containing valid escape characters
2024-01-08 13:25 ` 回复: " gaoliming via groups.io
@ 2024-01-08 13:59 ` Huang, Yanbo
2024-01-08 15:08 ` Joey Vagedes via groups.io
0 siblings, 1 reply; 12+ messages in thread
From: Huang, Yanbo @ 2024-01-08 13:59 UTC (permalink / raw)
To: Gao, Liming, devel@edk2.groups.io, Chen, Christine,
'Joey Vagedes'
Cc: 'Rebecca Cran', Feng, Bob C, Kinney, Michael D
Hi Liming,
This issue is introduced by this commit:
SHA-1: 9f0061a03b61d282fbc0ba5be22155d06a5e64a1
* BaseTools: Resolve regex syntax warnings
Switches regex patterns to raw text to resolve python 3.12 syntax
warnings in regards to invalid escape sequences, as is suggested by the
re (regex) module in python.
Best Regards,
Yanbo Huang
-----Original Message-----
From: gaoliming <gaoliming@byosoft.com.cn>
Sent: Monday, January 8, 2024 9:25 PM
To: devel@edk2.groups.io; Chen, Christine <yuwei.chen@intel.com>; 'Joey Vagedes' <joey.vagedes@gmail.com>
Cc: 'Rebecca Cran' <rebecca@bsdio.com>; Feng, Bob C <bob.c.feng@intel.com>; Huang, Yanbo <yanbo.huang@intel.com>; Kinney, Michael D <michael.d.kinney@intel.com>
Subject: 回复: [edk2-devel] [PATCH v1 1/1] BaseTools: Fix raw strings containing valid escape characters
Christine:
Is this a regression issue? Or does it resolve new usage? I am curious why it is not reported before.
Thanks
Liming
> -----邮件原件-----
> 发件人: devel@edk2.groups.io <devel@edk2.groups.io> 代表 Yuwei Chen
> 发送时间: 2024年1月8日 13:37
> 收件人: devel@edk2.groups.io; Chen, Christine <yuwei.chen@intel.com>;
> Joey Vagedes <joey.vagedes@gmail.com>
> 抄送: Rebecca Cran <rebecca@bsdio.com>; Gao, Liming
> <gaoliming@byosoft.com.cn>; Feng, Bob C <bob.c.feng@intel.com>; Huang,
> Yanbo <yanbo.huang@intel.com>; Kinney, Michael D
> <michael.d.kinney@intel.com>
> 主题: Re: [edk2-devel] [PATCH v1 1/1] BaseTools: Fix raw strings
containing
> valid escape characters
>
> Hi Liming and Cran,
>
> We should merge this fixing patch asap as it influences BaseTool's usage.
>
> Thanks,
> Christine
>
> > -----Original Message-----
> > From: devel@edk2.groups.io <devel@edk2.groups.io> On Behalf Of Yuwei
> > Chen
> > Sent: Monday, January 8, 2024 9:16 AM
> > To: Joey Vagedes <joey.vagedes@gmail.com>; devel@edk2.groups.io
> > Cc: Rebecca Cran <rebecca@bsdio.com>; Gao, Liming
> > <gaoliming@byosoft.com.cn>; Feng, Bob C <bob.c.feng@intel.com>
> > Subject: Re: [edk2-devel] [PATCH v1 1/1] BaseTools: Fix raw strings
> containing
> > valid escape characters
> >
> > Hi Joey,
> >
> > Thanks for fixing.
> >
> > Reviewed-by: Yuwei Chen <yuwei.chen@intel.com>
> >
> > > -----Original Message-----
> > > From: Joey Vagedes <joey.vagedes@gmail.com>
> > > Sent: Friday, December 29, 2023 12:48 AM
> > > To: devel@edk2.groups.io
> > > Cc: Rebecca Cran <rebecca@bsdio.com>; Gao, Liming
> > > <gaoliming@byosoft.com.cn>; Feng, Bob C <bob.c.feng@intel.com>;
> Chen,
> > > Christine <yuwei.chen@intel.com>
> > > Subject: [PATCH v1 1/1] BaseTools: Fix raw strings containing
> > > valid escape characters
> > >
> > > Fixes raw regex strings that contain valid (and purposeful) escape
> > > characters as they are being treated as individual characters
> > > rather than the single escaped character they represent (i.e. '\t'
> > > is being
treated
> as
> > a '\' and a 't'
> > > rather than a single tab character).
> > >
> > > Signed-off-by: Joey Vagedes <joey.vagedes@gmail.com>
> > > Cc: Rebecca Cran <rebecca@bsdio.com>
> > > Cc: Liming Gao <gaoliming@byosoft.com.cn>
> > > Cc: Bob Feng <bob.c.feng@intel.com>
> > > Cc: Yuwei Chen <yuwei.chen@intel.com>
> > > ---
> > > BaseTools/Source/Python/AutoGen/GenMake.py | 2 +-
> > > BaseTools/Source/Python/Common/Misc.py | 2 +-
> > > BaseTools/Source/Python/Workspace/DscBuildData.py | 4 ++--
> > > 3 files changed, 4 insertions(+), 4 deletions(-)
> > >
> > > diff --git a/BaseTools/Source/Python/AutoGen/GenMake.py
> > > b/BaseTools/Source/Python/AutoGen/GenMake.py
> > > index c416fe172fe5..fbd35d498923 100755
> > > --- a/BaseTools/Source/Python/AutoGen/GenMake.py
> > > +++ b/BaseTools/Source/Python/AutoGen/GenMake.py
> > > @@ -28,7 +28,7 @@ from Common.DataType import
> > TAB_COMPILER_MSFT
> > > gIncludePattern = re.compile(r"^[ \t]*[#%]?[ \t]*include(?:[
> > > \t]*(?:\\(?:\r\n|\r|\n))*[ \t]*)*(?:\(?[ \"<]?[ \t]*)([-\w.\\/()
> > > \t]+)(?:[ \t]*[\">]?\)?)", re.MULTILINE | re.UNICODE |
> > > re.IGNORECASE) ## Regular expression for matching macro used in
> > > header file inclusion-gMacroPattern =
> > > re.compile(r"([_A-Z][_A-Z0-9]*)[ \t]*\((.+)\)",
> > > re.UNICODE)+gMacroPattern = re.compile("([_A-Z][_A-Z0-9]*)[
> > > \t]*\\((.+)\\)",
> > > re.UNICODE) gIsFileMap = {} diff --git
> > > a/BaseTools/Source/Python/Common/Misc.py
> > > b/BaseTools/Source/Python/Common/Misc.py
> > > index f87d9dbdba39..34bfc90abd94 100755
> > > --- a/BaseTools/Source/Python/Common/Misc.py
> > > +++ b/BaseTools/Source/Python/Common/Misc.py
> > > @@ -1926,4 +1926,4 @@ def CopyDict(ori_dict):
> > > # Remove the c/c++ comments: // and /* */ # def
> > > RemoveCComments(ctext):- return re.sub(r'//.*?\n|/\*.*?\*/', '\n',
> ctext,
> > > flags=re.S)+ return re.sub('//.*?\n|/\\*.*?\\*/', '\n', ctext,
> flags=re.S)diff --
> > git
> > > a/BaseTools/Source/Python/Workspace/DscBuildData.py
> > > b/BaseTools/Source/Python/Workspace/DscBuildData.py
> > > index 817cdbe5f19c..4768099343e5 100644
> > > --- a/BaseTools/Source/Python/Workspace/DscBuildData.py
> > > +++ b/BaseTools/Source/Python/Workspace/DscBuildData.py
> > > @@ -2840,7 +2840,7 @@ class DscBuildData(PlatformBuildClassObject):
> > > # start generating makefile MakeApp =
> PcdMakefileHeader if
> > > sys.platform == "win32":- MakeApp = MakeApp + r'APPFILE
> > > = %s\%s.exe\n' % (self.OutputPath, PcdValueInitName) + r'APPNAME =
> > > %s\n' % (PcdValueInitName) + r'OBJECTS = %s\%s.obj %s.obj\n' %
> > > (self.OutputPath, PcdValueInitName, os.path.join(self.OutputPath,
> > > PcdValueCommonName)) + 'INC = '+ MakeApp = MakeApp
> + 'APPFILE
> > > = %s\\%s.exe\n' % (self.OutputPath, PcdValueInitName) + 'APPNAME =
> > > %s\n' % (PcdValueInitName) + 'OBJECTS = %s\\%s.obj %s.obj\n' %
> > > (self.OutputPath, PcdValueInitName, os.path.join(self.OutputPath,
> > > PcdValueCommonName)) + 'INC = ' else:
> MakeApp = MakeApp +
> > > PcdGccMakefile MakeApp = MakeApp + 'APPFILE
> = %s/%s\n' %
> > > (self.OutputPath, PcdValueInitName) + 'APPNAME = %s\n' %
> > > (PcdValueInitName) + 'OBJECTS = %s/%s.o %s.o\n' %
> > > (self.OutputPath, PcdValueInitName, os.path.join(self.OutputPath,
> > > PcdValueCommonName))
> > +
> > > \@@ -2950,7 +2950,7 @@ class DscBuildData(PlatformBuildClassObject):
> > > MakeApp += "$(OBJECTS) : %s\n" % include_file
> if sys.platform
> > ==
> > > "win32": PcdValueCommonPath =
> > >
> >
> os.path.normpath(mws.join(GlobalData.gGlobalDefines["EDK_TOOLS_PATH"]
> > > , "Source\C\Common\PcdValueCommon.c"))- MakeApp =
> MakeApp +
> > > r'%s\PcdValueCommon.c : %s\n' % (self.OutputPath,
> > PcdValueCommonPath)+
> > > MakeApp = MakeApp + '%s\\PcdValueCommon.c : %s\n' %
> (self.OutputPath,
> > > PcdValueCommonPath) MakeApp = MakeApp + '\tcopy
> /y %s
> > $@\n' %
> > > (PcdValueCommonPath) else:
> PcdValueCommonPath =
> > >
> >
> os.path.normpath(mws.join(GlobalData.gGlobalDefines["EDK_TOOLS_PATH"]
> > > , "Source/C/Common/PcdValueCommon.c"))--
> > > 2.43.0.windows.1
> >
> >
> >
> >
> >
>
>
>
>
>
-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.
View/Reply Online (#113393): https://edk2.groups.io/g/devel/message/113393
Mute This Topic: https://groups.io/mt/103597129/7686176
Group Owner: devel+owner@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub [rebecca@openfw.io]
-=-=-=-=-=-=-=-=-=-=-=-
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [edk2-devel] [PATCH v1 1/1] BaseTools: Fix raw strings containing valid escape characters
2024-01-08 13:59 ` Huang, Yanbo
@ 2024-01-08 15:08 ` Joey Vagedes via groups.io
2024-01-09 2:01 ` 回复: " gaoliming via groups.io
0 siblings, 1 reply; 12+ messages in thread
From: Joey Vagedes via groups.io @ 2024-01-08 15:08 UTC (permalink / raw)
To: Huang, Yanbo
Cc: Chen, Christine, Feng, Bob C, Gao, Liming, Kinney, Michael D,
Rebecca Cran, devel@edk2.groups.io
[-- Attachment #1: Type: text/plain, Size: 8705 bytes --]
Hi Liming,
It was introduced due to the fact that a few of the regex strings with
invalid escape sequences (due to regex using their own escape sequences)
also included valid escape sequences - for handling new lines. I did not
see the valid escape characters in the strings, and the virtual platforms
(Ovmf, etc) did not have the circumstances to catch this, so it slipped
through the cracks.
Some platform DSCs do, and cause the break that the patch is resolving.
I’ve already verified this fix resolves the issue on numerous platforms.
thanks,
Joey
On Mon, Jan 8, 2024 at 5:59 AM Huang, Yanbo <yanbo.huang@intel.com> wrote:
> Hi Liming,
>
> This issue is introduced by this commit:
> SHA-1: 9f0061a03b61d282fbc0ba5be22155d06a5e64a1
>
> * BaseTools: Resolve regex syntax warnings
>
> Switches regex patterns to raw text to resolve python 3.12 syntax
> warnings in regards to invalid escape sequences, as is suggested by the
> re (regex) module in python.
>
> Best Regards,
> Yanbo Huang
>
> -----Original Message-----
> From: gaoliming <gaoliming@byosoft.com.cn>
> Sent: Monday, January 8, 2024 9:25 PM
> To: devel@edk2.groups.io; Chen, Christine <yuwei.chen@intel.com>; 'Joey
> Vagedes' <joey.vagedes@gmail.com>
> Cc: 'Rebecca Cran' <rebecca@bsdio.com>; Feng, Bob C <bob.c.feng@intel.com>;
> Huang, Yanbo <yanbo.huang@intel.com>; Kinney, Michael D <
> michael.d.kinney@intel.com>
> Subject: 回复: [edk2-devel] [PATCH v1 1/1] BaseTools: Fix raw strings
> containing valid escape characters
>
> Christine:
> Is this a regression issue? Or does it resolve new usage? I am curious
> why it is not reported before.
>
> Thanks
> Liming
> > -----邮件原件-----
> > 发件人: devel@edk2.groups.io <devel@edk2.groups.io> 代表 Yuwei Chen
> > 发送时间: 2024年1月8日 13:37
> > 收件人: devel@edk2.groups.io; Chen, Christine <yuwei.chen@intel.com>;
> > Joey Vagedes <joey.vagedes@gmail.com>
> > 抄送: Rebecca Cran <rebecca@bsdio.com>; Gao, Liming
> > <gaoliming@byosoft.com.cn>; Feng, Bob C <bob.c.feng@intel.com>; Huang,
> > Yanbo <yanbo.huang@intel.com>; Kinney, Michael D
> > <michael.d.kinney@intel.com>
> > 主题: Re: [edk2-devel] [PATCH v1 1/1] BaseTools: Fix raw strings
> containing
> > valid escape characters
> >
> > Hi Liming and Cran,
> >
> > We should merge this fixing patch asap as it influences BaseTool's usage.
> >
> > Thanks,
> > Christine
> >
> > > -----Original Message-----
> > > From: devel@edk2.groups.io <devel@edk2.groups.io> On Behalf Of Yuwei
> > > Chen
> > > Sent: Monday, January 8, 2024 9:16 AM
> > > To: Joey Vagedes <joey.vagedes@gmail.com>; devel@edk2.groups.io
> > > Cc: Rebecca Cran <rebecca@bsdio.com>; Gao, Liming
> > > <gaoliming@byosoft.com.cn>; Feng, Bob C <bob.c.feng@intel.com>
> > > Subject: Re: [edk2-devel] [PATCH v1 1/1] BaseTools: Fix raw strings
> > containing
> > > valid escape characters
> > >
> > > Hi Joey,
> > >
> > > Thanks for fixing.
> > >
> > > Reviewed-by: Yuwei Chen <yuwei.chen@intel.com>
> > >
> > > > -----Original Message-----
> > > > From: Joey Vagedes <joey.vagedes@gmail.com>
> > > > Sent: Friday, December 29, 2023 12:48 AM
> > > > To: devel@edk2.groups.io
> > > > Cc: Rebecca Cran <rebecca@bsdio.com>; Gao, Liming
> > > > <gaoliming@byosoft.com.cn>; Feng, Bob C <bob.c.feng@intel.com>;
> > Chen,
> > > > Christine <yuwei.chen@intel.com>
> > > > Subject: [PATCH v1 1/1] BaseTools: Fix raw strings containing
> > > > valid escape characters
> > > >
> > > > Fixes raw regex strings that contain valid (and purposeful) escape
> > > > characters as they are being treated as individual characters
> > > > rather than the single escaped character they represent (i.e. '\t'
> > > > is being
> treated
> > as
> > > a '\' and a 't'
> > > > rather than a single tab character).
> > > >
> > > > Signed-off-by: Joey Vagedes <joey.vagedes@gmail.com>
> > > > Cc: Rebecca Cran <rebecca@bsdio.com>
> > > > Cc: Liming Gao <gaoliming@byosoft.com.cn>
> > > > Cc: Bob Feng <bob.c.feng@intel.com>
> > > > Cc: Yuwei Chen <yuwei.chen@intel.com>
> > > > ---
> > > > BaseTools/Source/Python/AutoGen/GenMake.py | 2 +-
> > > > BaseTools/Source/Python/Common/Misc.py | 2 +-
> > > > BaseTools/Source/Python/Workspace/DscBuildData.py | 4 ++--
> > > > 3 files changed, 4 insertions(+), 4 deletions(-)
> > > >
> > > > diff --git a/BaseTools/Source/Python/AutoGen/GenMake.py
> > > > b/BaseTools/Source/Python/AutoGen/GenMake.py
> > > > index c416fe172fe5..fbd35d498923 100755
> > > > --- a/BaseTools/Source/Python/AutoGen/GenMake.py
> > > > +++ b/BaseTools/Source/Python/AutoGen/GenMake.py
> > > > @@ -28,7 +28,7 @@ from Common.DataType import
> > > TAB_COMPILER_MSFT
> > > > gIncludePattern = re.compile(r"^[ \t]*[#%]?[ \t]*include(?:[
> > > > \t]*(?:\\(?:\r\n|\r|\n))*[ \t]*)*(?:\(?[ \"<]?[ \t]*)([-\w.\\/()
> > > > \t]+)(?:[ \t]*[\">]?\)?)", re.MULTILINE | re.UNICODE |
> > > > re.IGNORECASE) ## Regular expression for matching macro used in
> > > > header file inclusion-gMacroPattern =
> > > > re.compile(r"([_A-Z][_A-Z0-9]*)[ \t]*\((.+)\)",
> > > > re.UNICODE)+gMacroPattern = re.compile("([_A-Z][_A-Z0-9]*)[
> > > > \t]*\\((.+)\\)",
> > > > re.UNICODE) gIsFileMap = {} diff --git
> > > > a/BaseTools/Source/Python/Common/Misc.py
> > > > b/BaseTools/Source/Python/Common/Misc.py
> > > > index f87d9dbdba39..34bfc90abd94 100755
> > > > --- a/BaseTools/Source/Python/Common/Misc.py
> > > > +++ b/BaseTools/Source/Python/Common/Misc.py
> > > > @@ -1926,4 +1926,4 @@ def CopyDict(ori_dict):
> > > > # Remove the c/c++ comments: // and /* */ # def
> > > > RemoveCComments(ctext):- return re.sub(r'//.*?\n|/\*.*?\*/', '\n',
> > ctext,
> > > > flags=re.S)+ return re.sub('//.*?\n|/\\*.*?\\*/', '\n', ctext,
> > flags=re.S)diff --
> > > git
> > > > a/BaseTools/Source/Python/Workspace/DscBuildData.py
> > > > b/BaseTools/Source/Python/Workspace/DscBuildData.py
> > > > index 817cdbe5f19c..4768099343e5 100644
> > > > --- a/BaseTools/Source/Python/Workspace/DscBuildData.py
> > > > +++ b/BaseTools/Source/Python/Workspace/DscBuildData.py
> > > > @@ -2840,7 +2840,7 @@ class DscBuildData(PlatformBuildClassObject):
> > > > # start generating makefile MakeApp =
> > PcdMakefileHeader if
> > > > sys.platform == "win32":- MakeApp = MakeApp + r'APPFILE
> > > > = %s\%s.exe\n' % (self.OutputPath, PcdValueInitName) + r'APPNAME =
> > > > %s\n' % (PcdValueInitName) + r'OBJECTS = %s\%s.obj %s.obj\n' %
> > > > (self.OutputPath, PcdValueInitName, os.path.join(self.OutputPath,
> > > > PcdValueCommonName)) + 'INC = '+ MakeApp = MakeApp
> > + 'APPFILE
> > > > = %s\\%s.exe\n' % (self.OutputPath, PcdValueInitName) + 'APPNAME =
> > > > %s\n' % (PcdValueInitName) + 'OBJECTS = %s\\%s.obj %s.obj\n' %
> > > > (self.OutputPath, PcdValueInitName, os.path.join(self.OutputPath,
> > > > PcdValueCommonName)) + 'INC = ' else:
> > MakeApp = MakeApp +
> > > > PcdGccMakefile MakeApp = MakeApp + 'APPFILE
> > = %s/%s\n' %
> > > > (self.OutputPath, PcdValueInitName) + 'APPNAME = %s\n' %
> > > > (PcdValueInitName) + 'OBJECTS = %s/%s.o %s.o\n' %
> > > > (self.OutputPath, PcdValueInitName, os.path.join(self.OutputPath,
> > > > PcdValueCommonName))
> > > +
> > > > \@@ -2950,7 +2950,7 @@ class DscBuildData(PlatformBuildClassObject):
> > > > MakeApp += "$(OBJECTS) : %s\n" % include_file
> > if sys.platform
> > > ==
> > > > "win32": PcdValueCommonPath =
> > > >
> > >
> > os.path.normpath(mws.join(GlobalData.gGlobalDefines["EDK_TOOLS_PATH"]
> > > > , "Source\C\Common\PcdValueCommon.c"))- MakeApp =
> > MakeApp +
> > > > r'%s\PcdValueCommon.c : %s\n' % (self.OutputPath,
> > > PcdValueCommonPath)+
> > > > MakeApp = MakeApp + '%s\\PcdValueCommon.c : %s\n' %
> > (self.OutputPath,
> > > > PcdValueCommonPath) MakeApp = MakeApp + '\tcopy
> > /y %s
> > > $@\n' %
> > > > (PcdValueCommonPath) else:
> > PcdValueCommonPath =
> > > >
> > >
> > os.path.normpath(mws.join(GlobalData.gGlobalDefines["EDK_TOOLS_PATH"]
> > > > , "Source/C/Common/PcdValueCommon.c"))--
> > > > 2.43.0.windows.1
> > >
> > >
> > >
> > >
> > >
> >
> >
> >
> >
> >
>
>
>
>
-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.
View/Reply Online (#113402): https://edk2.groups.io/g/devel/message/113402
Mute This Topic: https://groups.io/mt/103597129/7686176
Group Owner: devel+owner@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub [rebecca@openfw.io]
-=-=-=-=-=-=-=-=-=-=-=-
[-- Attachment #2: Type: text/html, Size: 13807 bytes --]
^ permalink raw reply [flat|nested] 12+ messages in thread
* 回复: [edk2-devel] [PATCH v1 1/1] BaseTools: Fix raw strings containing valid escape characters
2024-01-08 15:08 ` Joey Vagedes via groups.io
@ 2024-01-09 2:01 ` gaoliming via groups.io
2024-01-09 8:14 ` Yuwei Chen
0 siblings, 1 reply; 12+ messages in thread
From: gaoliming via groups.io @ 2024-01-09 2:01 UTC (permalink / raw)
To: devel, joeyvagedes, 'Huang, Yanbo'
Cc: 'Chen, Christine', 'Feng, Bob C',
'Kinney, Michael D', 'Rebecca Cran'
[-- Attachment #1: Type: text/plain, Size: 10309 bytes --]
Joey and Yanbo:
Thanks for your update. So, this one is new introduced issue. I agree to resolve it asap.
Christine:
Can you create Pull Request to merge it?
Thanks
Liming
发件人: devel@edk2.groups.io <devel@edk2.groups.io> 代表 Joey Vagedes via groups.io
发送时间: 2024年1月8日 23:09
收件人: Huang, Yanbo <yanbo.huang@intel.com>
抄送: Chen, Christine <yuwei.chen@intel.com>; Feng, Bob C <bob.c.feng@intel.com>; Gao, Liming <gaoliming@byosoft.com.cn>; Kinney, Michael D <michael.d.kinney@intel.com>; Rebecca Cran <rebecca@bsdio.com>; devel@edk2.groups.io
主题: Re: [edk2-devel] [PATCH v1 1/1] BaseTools: Fix raw strings containing valid escape characters
Hi Liming,
It was introduced due to the fact that a few of the regex strings with invalid escape sequences (due to regex using their own escape sequences) also included valid escape sequences - for handling new lines. I did not see the valid escape characters in the strings, and the virtual platforms (Ovmf, etc) did not have the circumstances to catch this, so it slipped through the cracks.
Some platform DSCs do, and cause the break that the patch is resolving. I’ve already verified this fix resolves the issue on numerous platforms.
thanks,
Joey
On Mon, Jan 8, 2024 at 5:59 AM Huang, Yanbo <yanbo.huang@intel.com <mailto:yanbo.huang@intel.com> > wrote:
Hi Liming,
This issue is introduced by this commit:
SHA-1: 9f0061a03b61d282fbc0ba5be22155d06a5e64a1
* BaseTools: Resolve regex syntax warnings
Switches regex patterns to raw text to resolve python 3.12 syntax
warnings in regards to invalid escape sequences, as is suggested by the
re (regex) module in python.
Best Regards,
Yanbo Huang
-----Original Message-----
From: gaoliming <gaoliming@byosoft.com.cn <mailto:gaoliming@byosoft.com.cn> >
Sent: Monday, January 8, 2024 9:25 PM
To: devel@edk2.groups.io <mailto:devel@edk2.groups.io> ; Chen, Christine <yuwei.chen@intel.com <mailto:yuwei.chen@intel.com> >; 'Joey Vagedes' <joey.vagedes@gmail.com <mailto:joey.vagedes@gmail.com> >
Cc: 'Rebecca Cran' <rebecca@bsdio.com <mailto:rebecca@bsdio.com> >; Feng, Bob C <bob.c.feng@intel.com <mailto:bob.c.feng@intel.com> >; Huang, Yanbo <yanbo.huang@intel.com <mailto:yanbo.huang@intel.com> >; Kinney, Michael D <michael.d.kinney@intel.com <mailto:michael.d.kinney@intel.com> >
Subject: 回复: [edk2-devel] [PATCH v1 1/1] BaseTools: Fix raw strings containing valid escape characters
Christine:
Is this a regression issue? Or does it resolve new usage? I am curious why it is not reported before.
Thanks
Liming
> -----邮件原件-----
> 发件人: devel@edk2.groups.io <mailto:devel@edk2.groups.io> <devel@edk2.groups.io <mailto:devel@edk2.groups.io> > 代表 Yuwei Chen
> 发送时间: 2024年1月8日 13:37
> 收件人: devel@edk2.groups.io <mailto:devel@edk2.groups.io> ; Chen, Christine <yuwei.chen@intel.com <mailto:yuwei.chen@intel.com> >;
> Joey Vagedes <joey.vagedes@gmail.com <mailto:joey.vagedes@gmail.com> >
> 抄送: Rebecca Cran <rebecca@bsdio.com <mailto:rebecca@bsdio.com> >; Gao, Liming
> <gaoliming@byosoft.com.cn <mailto:gaoliming@byosoft.com.cn> >; Feng, Bob C <bob.c.feng@intel.com <mailto:bob.c.feng@intel.com> >; Huang,
> Yanbo <yanbo.huang@intel.com <mailto:yanbo.huang@intel.com> >; Kinney, Michael D
> <michael.d.kinney@intel.com <mailto:michael.d.kinney@intel.com> >
> 主题: Re: [edk2-devel] [PATCH v1 1/1] BaseTools: Fix raw strings
containing
> valid escape characters
>
> Hi Liming and Cran,
>
> We should merge this fixing patch asap as it influences BaseTool's usage.
>
> Thanks,
> Christine
>
> > -----Original Message-----
> > From: devel@edk2.groups.io <mailto:devel@edk2.groups.io> <devel@edk2.groups.io <mailto:devel@edk2.groups.io> > On Behalf Of Yuwei
> > Chen
> > Sent: Monday, January 8, 2024 9:16 AM
> > To: Joey Vagedes <joey.vagedes@gmail.com <mailto:joey.vagedes@gmail.com> >; devel@edk2.groups.io <mailto:devel@edk2.groups.io>
> > Cc: Rebecca Cran <rebecca@bsdio.com <mailto:rebecca@bsdio.com> >; Gao, Liming
> > <gaoliming@byosoft.com.cn <mailto:gaoliming@byosoft.com.cn> >; Feng, Bob C <bob.c.feng@intel.com <mailto:bob.c.feng@intel.com> >
> > Subject: Re: [edk2-devel] [PATCH v1 1/1] BaseTools: Fix raw strings
> containing
> > valid escape characters
> >
> > Hi Joey,
> >
> > Thanks for fixing.
> >
> > Reviewed-by: Yuwei Chen <yuwei.chen@intel.com <mailto:yuwei.chen@intel.com> >
> >
> > > -----Original Message-----
> > > From: Joey Vagedes <joey.vagedes@gmail.com <mailto:joey.vagedes@gmail.com> >
> > > Sent: Friday, December 29, 2023 12:48 AM
> > > To: devel@edk2.groups.io <mailto:devel@edk2.groups.io>
> > > Cc: Rebecca Cran <rebecca@bsdio.com <mailto:rebecca@bsdio.com> >; Gao, Liming
> > > <gaoliming@byosoft.com.cn <mailto:gaoliming@byosoft.com.cn> >; Feng, Bob C <bob.c.feng@intel.com <mailto:bob.c.feng@intel.com> >;
> Chen,
> > > Christine <yuwei.chen@intel.com <mailto:yuwei.chen@intel.com> >
> > > Subject: [PATCH v1 1/1] BaseTools: Fix raw strings containing
> > > valid escape characters
> > >
> > > Fixes raw regex strings that contain valid (and purposeful) escape
> > > characters as they are being treated as individual characters
> > > rather than the single escaped character they represent (i.e. '\t'
> > > is being
treated
> as
> > a '\' and a 't'
> > > rather than a single tab character).
> > >
> > > Signed-off-by: Joey Vagedes <joey.vagedes@gmail.com <mailto:joey.vagedes@gmail.com> >
> > > Cc: Rebecca Cran <rebecca@bsdio.com <mailto:rebecca@bsdio.com> >
> > > Cc: Liming Gao <gaoliming@byosoft.com.cn <mailto:gaoliming@byosoft.com.cn> >
> > > Cc: Bob Feng <bob.c.feng@intel.com <mailto:bob.c.feng@intel.com> >
> > > Cc: Yuwei Chen <yuwei.chen@intel.com <mailto:yuwei.chen@intel.com> >
> > > ---
> > > BaseTools/Source/Python/AutoGen/GenMake.py | 2 +-
> > > BaseTools/Source/Python/Common/Misc.py | 2 +-
> > > BaseTools/Source/Python/Workspace/DscBuildData.py | 4 ++--
> > > 3 files changed, 4 insertions(+), 4 deletions(-)
> > >
> > > diff --git a/BaseTools/Source/Python/AutoGen/GenMake.py
> > > b/BaseTools/Source/Python/AutoGen/GenMake.py
> > > index c416fe172fe5..fbd35d498923 100755
> > > --- a/BaseTools/Source/Python/AutoGen/GenMake.py
> > > +++ b/BaseTools/Source/Python/AutoGen/GenMake.py
> > > @@ -28,7 +28,7 @@ from Common.DataType import
> > TAB_COMPILER_MSFT
> > > gIncludePattern = re.compile(r"^[ \t]*[#%]?[ \t]*include(?:[
> > > \t]*(?:\\(?:\r\n|\r|\n))*[ \t]*)*(?:\(?[ \"<]?[ \t]*)([-\w.\\/()
> > > \t]+)(?:[ \t]*[\">]?\)?)", re.MULTILINE | re.UNICODE |
> > > re.IGNORECASE) ## Regular expression for matching macro used in
> > > header file inclusion-gMacroPattern =
> > > re.compile(r"([_A-Z][_A-Z0-9]*)[ \t]*\((.+)\)",
> > > re.UNICODE)+gMacroPattern = re.compile("([_A-Z][_A-Z0-9]*)[
> > > \t]*\\((.+)\\)",
> > > re.UNICODE) gIsFileMap = {} diff --git
> > > a/BaseTools/Source/Python/Common/Misc.py
> > > b/BaseTools/Source/Python/Common/Misc.py
> > > index f87d9dbdba39..34bfc90abd94 100755
> > > --- a/BaseTools/Source/Python/Common/Misc.py
> > > +++ b/BaseTools/Source/Python/Common/Misc.py
> > > @@ -1926,4 +1926,4 @@ def CopyDict(ori_dict):
> > > # Remove the c/c++ comments: // and /* */ # def
> > > RemoveCComments(ctext):- return re.sub(r'//.*?\n|/\*.*?\*/', '\n',
> ctext,
> > > flags=re.S)+ return re.sub('//.*?\n|/\\*.*?\\*/', '\n', ctext,
> flags=re.S)diff --
> > git
> > > a/BaseTools/Source/Python/Workspace/DscBuildData.py
> > > b/BaseTools/Source/Python/Workspace/DscBuildData.py
> > > index 817cdbe5f19c..4768099343e5 100644
> > > --- a/BaseTools/Source/Python/Workspace/DscBuildData.py
> > > +++ b/BaseTools/Source/Python/Workspace/DscBuildData.py
> > > @@ -2840,7 +2840,7 @@ class DscBuildData(PlatformBuildClassObject):
> > > # start generating makefile MakeApp =
> PcdMakefileHeader if
> > > sys.platform == "win32":- MakeApp = MakeApp + r'APPFILE
> > > = %s\%s.exe\n' % (self.OutputPath, PcdValueInitName) + r'APPNAME =
> > > %s\n' % (PcdValueInitName) + r'OBJECTS = %s\%s.obj %s.obj\n' %
> > > (self.OutputPath, PcdValueInitName, os.path.join(self.OutputPath,
> > > PcdValueCommonName)) + 'INC = '+ MakeApp = MakeApp
> + 'APPFILE
> > > = %s\\%s.exe\n' % (self.OutputPath, PcdValueInitName) + 'APPNAME =
> > > %s\n' % (PcdValueInitName) + 'OBJECTS = %s\\%s.obj %s.obj\n' %
> > > (self.OutputPath, PcdValueInitName, os.path.join(self.OutputPath,
> > > PcdValueCommonName)) + 'INC = ' else:
> MakeApp = MakeApp +
> > > PcdGccMakefile MakeApp = MakeApp + 'APPFILE
> = %s/%s\n' %
> > > (self.OutputPath, PcdValueInitName) + 'APPNAME = %s\n' %
> > > (PcdValueInitName) + 'OBJECTS = %s/%s.o %s.o\n' %
> > > (self.OutputPath, PcdValueInitName, os.path.join(self.OutputPath,
> > > PcdValueCommonName))
> > +
> > > \@@ -2950,7 +2950,7 @@ class DscBuildData(PlatformBuildClassObject):
> > > MakeApp += "$(OBJECTS) : %s\n" % include_file
> if sys.platform
> > ==
> > > "win32": PcdValueCommonPath =
> > >
> >
> os.path.normpath(mws.join(GlobalData.gGlobalDefines["EDK_TOOLS_PATH"]
> > > , "Source\C\Common\PcdValueCommon.c"))- MakeApp =
> MakeApp +
> > > r'%s\PcdValueCommon.c : %s\n' % (self.OutputPath,
> > PcdValueCommonPath)+
> > > MakeApp = MakeApp + '%s\\PcdValueCommon.c : %s\n' %
> (self.OutputPath,
> > > PcdValueCommonPath) MakeApp = MakeApp + '\tcopy
> /y %s
> > $@\n' %
> > > (PcdValueCommonPath) else:
> PcdValueCommonPath =
> > >
> >
> os.path.normpath(mws.join(GlobalData.gGlobalDefines["EDK_TOOLS_PATH"]
> > > , "Source/C/Common/PcdValueCommon.c"))--
> > > 2.43.0.windows.1
> >
> >
> >
> >
> >
>
>
>
>
>
-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.
View/Reply Online (#113415): https://edk2.groups.io/g/devel/message/113415
Mute This Topic: https://groups.io/mt/103612970/7686176
Group Owner: devel+owner@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub [rebecca@openfw.io]
-=-=-=-=-=-=-=-=-=-=-=-
[-- Attachment #2: Type: text/html, Size: 18662 bytes --]
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [edk2-devel] [PATCH v1 1/1] BaseTools: Fix raw strings containing valid escape characters
2024-01-09 2:01 ` 回复: " gaoliming via groups.io
@ 2024-01-09 8:14 ` Yuwei Chen
2024-01-10 6:52 ` Huang, Yanbo
0 siblings, 1 reply; 12+ messages in thread
From: Yuwei Chen @ 2024-01-09 8:14 UTC (permalink / raw)
To: Gao, Liming, devel@edk2.groups.io, joeyvagedes@microsoft.com,
Huang, Yanbo
Cc: Feng, Bob C, Kinney, Michael D, 'Rebecca Cran'
[-- Attachment #1: Type: text/plain, Size: 11103 bytes --]
Liming:
PR has been created: BaseTools: Fix raw strings containing valid escape characters by YuweiChen1110 · Pull Request #5238 · tianocore/edk2 (github.com)<https://github.com/tianocore/edk2/pull/5238>
Thanks for reviewing.
Thanks,
Christine
From: gaoliming <gaoliming@byosoft.com.cn>
Sent: Tuesday, January 9, 2024 10:01 AM
To: devel@edk2.groups.io; joeyvagedes@microsoft.com; Huang, Yanbo <yanbo.huang@intel.com>
Cc: Chen, Christine <yuwei.chen@intel.com>; Feng, Bob C <bob.c.feng@intel.com>; Kinney, Michael D <michael.d.kinney@intel.com>; 'Rebecca Cran' <rebecca@bsdio.com>
Subject: 回复: [edk2-devel] [PATCH v1 1/1] BaseTools: Fix raw strings containing valid escape characters
Joey and Yanbo:
Thanks for your update. So, this one is new introduced issue. I agree to resolve it asap.
Christine:
Can you create Pull Request to merge it?
Thanks
Liming
发件人: devel@edk2.groups.io<mailto:devel@edk2.groups.io> <devel@edk2.groups.io<mailto:devel@edk2.groups.io>> 代表 Joey Vagedes via groups.io
发送时间: 2024年1月8日 23:09
收件人: Huang, Yanbo <yanbo.huang@intel.com<mailto:yanbo.huang@intel.com>>
抄送: Chen, Christine <yuwei.chen@intel.com<mailto:yuwei.chen@intel.com>>; Feng, Bob C <bob.c.feng@intel.com<mailto:bob.c.feng@intel.com>>; Gao, Liming <gaoliming@byosoft.com.cn<mailto:gaoliming@byosoft.com.cn>>; Kinney, Michael D <michael.d.kinney@intel.com<mailto:michael.d.kinney@intel.com>>; Rebecca Cran <rebecca@bsdio.com<mailto:rebecca@bsdio.com>>; devel@edk2.groups.io<mailto:devel@edk2.groups.io>
主题: Re: [edk2-devel] [PATCH v1 1/1] BaseTools: Fix raw strings containing valid escape characters
Hi Liming,
It was introduced due to the fact that a few of the regex strings with invalid escape sequences (due to regex using their own escape sequences) also included valid escape sequences - for handling new lines. I did not see the valid escape characters in the strings, and the virtual platforms (Ovmf, etc) did not have the circumstances to catch this, so it slipped through the cracks.
Some platform DSCs do, and cause the break that the patch is resolving. I’ve already verified this fix resolves the issue on numerous platforms.
thanks,
Joey
On Mon, Jan 8, 2024 at 5:59 AM Huang, Yanbo <yanbo.huang@intel.com<mailto:yanbo.huang@intel.com>> wrote:
Hi Liming,
This issue is introduced by this commit:
SHA-1: 9f0061a03b61d282fbc0ba5be22155d06a5e64a1
* BaseTools: Resolve regex syntax warnings
Switches regex patterns to raw text to resolve python 3.12 syntax
warnings in regards to invalid escape sequences, as is suggested by the
re (regex) module in python.
Best Regards,
Yanbo Huang
-----Original Message-----
From: gaoliming <gaoliming@byosoft.com.cn<mailto:gaoliming@byosoft.com.cn>>
Sent: Monday, January 8, 2024 9:25 PM
To: devel@edk2.groups.io<mailto:devel@edk2.groups.io>; Chen, Christine <yuwei.chen@intel.com<mailto:yuwei.chen@intel.com>>; 'Joey Vagedes' <joey.vagedes@gmail.com<mailto:joey.vagedes@gmail.com>>
Cc: 'Rebecca Cran' <rebecca@bsdio.com<mailto:rebecca@bsdio.com>>; Feng, Bob C <bob.c.feng@intel.com<mailto:bob.c.feng@intel.com>>; Huang, Yanbo <yanbo.huang@intel.com<mailto:yanbo.huang@intel.com>>; Kinney, Michael D <michael.d.kinney@intel.com<mailto:michael.d.kinney@intel.com>>
Subject: 回复: [edk2-devel] [PATCH v1 1/1] BaseTools: Fix raw strings containing valid escape characters
Christine:
Is this a regression issue? Or does it resolve new usage? I am curious why it is not reported before.
Thanks
Liming
> -----邮件原件-----
> 发件人: devel@edk2.groups.io<mailto:devel@edk2.groups.io> <devel@edk2.groups.io<mailto:devel@edk2.groups.io>> 代表 Yuwei Chen
> 发送时间: 2024年1月8日 13:37
> 收件人: devel@edk2.groups.io<mailto:devel@edk2.groups.io>; Chen, Christine <yuwei.chen@intel.com<mailto:yuwei.chen@intel.com>>;
> Joey Vagedes <joey.vagedes@gmail.com<mailto:joey.vagedes@gmail.com>>
> 抄送: Rebecca Cran <rebecca@bsdio.com<mailto:rebecca@bsdio.com>>; Gao, Liming
> <gaoliming@byosoft.com.cn<mailto:gaoliming@byosoft.com.cn>>; Feng, Bob C <bob.c.feng@intel.com<mailto:bob.c.feng@intel.com>>; Huang,
> Yanbo <yanbo.huang@intel.com<mailto:yanbo.huang@intel.com>>; Kinney, Michael D
> <michael.d.kinney@intel.com<mailto:michael.d.kinney@intel.com>>
> 主题: Re: [edk2-devel] [PATCH v1 1/1] BaseTools: Fix raw strings
containing
> valid escape characters
>
> Hi Liming and Cran,
>
> We should merge this fixing patch asap as it influences BaseTool's usage.
>
> Thanks,
> Christine
>
> > -----Original Message-----
> > From: devel@edk2.groups.io<mailto:devel@edk2.groups.io> <devel@edk2.groups.io<mailto:devel@edk2.groups.io>> On Behalf Of Yuwei
> > Chen
> > Sent: Monday, January 8, 2024 9:16 AM
> > To: Joey Vagedes <joey.vagedes@gmail.com<mailto:joey.vagedes@gmail.com>>; devel@edk2.groups.io<mailto:devel@edk2.groups.io>
> > Cc: Rebecca Cran <rebecca@bsdio.com<mailto:rebecca@bsdio.com>>; Gao, Liming
> > <gaoliming@byosoft.com.cn<mailto:gaoliming@byosoft.com.cn>>; Feng, Bob C <bob.c.feng@intel.com<mailto:bob.c.feng@intel.com>>
> > Subject: Re: [edk2-devel] [PATCH v1 1/1] BaseTools: Fix raw strings
> containing
> > valid escape characters
> >
> > Hi Joey,
> >
> > Thanks for fixing.
> >
> > Reviewed-by: Yuwei Chen <yuwei.chen@intel.com<mailto:yuwei.chen@intel.com>>
> >
> > > -----Original Message-----
> > > From: Joey Vagedes <joey.vagedes@gmail.com<mailto:joey.vagedes@gmail.com>>
> > > Sent: Friday, December 29, 2023 12:48 AM
> > > To: devel@edk2.groups.io<mailto:devel@edk2.groups.io>
> > > Cc: Rebecca Cran <rebecca@bsdio.com<mailto:rebecca@bsdio.com>>; Gao, Liming
> > > <gaoliming@byosoft.com.cn<mailto:gaoliming@byosoft.com.cn>>; Feng, Bob C <bob.c.feng@intel.com<mailto:bob.c.feng@intel.com>>;
> Chen,
> > > Christine <yuwei.chen@intel.com<mailto:yuwei.chen@intel.com>>
> > > Subject: [PATCH v1 1/1] BaseTools: Fix raw strings containing
> > > valid escape characters
> > >
> > > Fixes raw regex strings that contain valid (and purposeful) escape
> > > characters as they are being treated as individual characters
> > > rather than the single escaped character they represent (i.e. '\t'
> > > is being
treated
> as
> > a '\' and a 't'
> > > rather than a single tab character).
> > >
> > > Signed-off-by: Joey Vagedes <joey.vagedes@gmail.com<mailto:joey.vagedes@gmail.com>>
> > > Cc: Rebecca Cran <rebecca@bsdio.com<mailto:rebecca@bsdio.com>>
> > > Cc: Liming Gao <gaoliming@byosoft.com.cn<mailto:gaoliming@byosoft.com.cn>>
> > > Cc: Bob Feng <bob.c.feng@intel.com<mailto:bob.c.feng@intel.com>>
> > > Cc: Yuwei Chen <yuwei.chen@intel.com<mailto:yuwei.chen@intel.com>>
> > > ---
> > > BaseTools/Source/Python/AutoGen/GenMake.py | 2 +-
> > > BaseTools/Source/Python/Common/Misc.py | 2 +-
> > > BaseTools/Source/Python/Workspace/DscBuildData.py | 4 ++--
> > > 3 files changed, 4 insertions(+), 4 deletions(-)
> > >
> > > diff --git a/BaseTools/Source/Python/AutoGen/GenMake.py
> > > b/BaseTools/Source/Python/AutoGen/GenMake.py
> > > index c416fe172fe5..fbd35d498923 100755
> > > --- a/BaseTools/Source/Python/AutoGen/GenMake.py
> > > +++ b/BaseTools/Source/Python/AutoGen/GenMake.py
> > > @@ -28,7 +28,7 @@ from Common.DataType import
> > TAB_COMPILER_MSFT
> > > gIncludePattern = re.compile(r"^[ \t]*[#%]?[ \t]*include(?:[
> > > \t]*(?:\\(?:\r\n|\r|\n))*[ \t]*)*(?:\(?[ \"<]?[ \t]*)([-\w.\\/()
> > > \t]+)(?:[ \t]*[\">]?\)?)", re.MULTILINE | re.UNICODE |
> > > re.IGNORECASE) ## Regular expression for matching macro used in
> > > header file inclusion-gMacroPattern =
> > > re.compile(r"([_A-Z][_A-Z0-9]*)[ \t]*\((.+)\)",
> > > re.UNICODE)+gMacroPattern = re.compile("([_A-Z][_A-Z0-9]*)[
> > > \t]*\\((.+)\\)",
> > > re.UNICODE) gIsFileMap = {} diff --git
> > > a/BaseTools/Source/Python/Common/Misc.py
> > > b/BaseTools/Source/Python/Common/Misc.py
> > > index f87d9dbdba39..34bfc90abd94 100755
> > > --- a/BaseTools/Source/Python/Common/Misc.py
> > > +++ b/BaseTools/Source/Python/Common/Misc.py
> > > @@ -1926,4 +1926,4 @@ def CopyDict(ori_dict):
> > > # Remove the c/c++ comments: // and /* */ # def
> > > RemoveCComments(ctext):- return re.sub(r'//.*?\n|/\*.*?\*/', '\n',
> ctext,
> > > flags=re.S)+ return re.sub('//.*?\n|/\\*.*?\\*/', '\n', ctext,
> flags=re.S)diff --
> > git
> > > a/BaseTools/Source/Python/Workspace/DscBuildData.py
> > > b/BaseTools/Source/Python/Workspace/DscBuildData.py
> > > index 817cdbe5f19c..4768099343e5 100644
> > > --- a/BaseTools/Source/Python/Workspace/DscBuildData.py
> > > +++ b/BaseTools/Source/Python/Workspace/DscBuildData.py
> > > @@ -2840,7 +2840,7 @@ class DscBuildData(PlatformBuildClassObject):
> > > # start generating makefile MakeApp =
> PcdMakefileHeader if
> > > sys.platform == "win32":- MakeApp = MakeApp + r'APPFILE
> > > = %s\%s.exe\n' % (self.OutputPath, PcdValueInitName) + r'APPNAME =
> > > %s\n' % (PcdValueInitName) + r'OBJECTS = %s\%s.obj %s.obj\n' %
> > > (self.OutputPath, PcdValueInitName, os.path.join(self.OutputPath,
> > > PcdValueCommonName)) + 'INC = '+ MakeApp = MakeApp
> + 'APPFILE
> > > = %s\\%s.exe\n' % (self.OutputPath, PcdValueInitName) + 'APPNAME =
> > > %s\n' % (PcdValueInitName) + 'OBJECTS = %s\\%s.obj %s.obj\n' %
> > > (self.OutputPath, PcdValueInitName, os.path.join(self.OutputPath,
> > > PcdValueCommonName)) + 'INC = ' else:
> MakeApp = MakeApp +
> > > PcdGccMakefile MakeApp = MakeApp + 'APPFILE
> = %s/%s\n' %
> > > (self.OutputPath, PcdValueInitName) + 'APPNAME = %s\n' %
> > > (PcdValueInitName) + 'OBJECTS = %s/%s.o %s.o\n' %
> > > (self.OutputPath, PcdValueInitName, os.path.join(self.OutputPath,
> > > PcdValueCommonName))
> > +
> > > \@@ -2950,7 +2950,7 @@ class DscBuildData(PlatformBuildClassObject):
> > > MakeApp += "$(OBJECTS) : %s\n" % include_file
> if sys.platform
> > ==
> > > "win32": PcdValueCommonPath =
> > >
> >
> os.path.normpath(mws.join(GlobalData.gGlobalDefines["EDK_TOOLS_PATH"]
> > > , "Source\C\Common\PcdValueCommon.c"))- MakeApp =
> MakeApp +
> > > r'%s\PcdValueCommon.c : %s\n' % (self.OutputPath,
> > PcdValueCommonPath)+
> > > MakeApp = MakeApp + '%s\\PcdValueCommon.c : %s\n' %
> (self.OutputPath,
> > > PcdValueCommonPath) MakeApp = MakeApp + '\tcopy
> /y %s
> > $@\n' %
> > > (PcdValueCommonPath) else:
> PcdValueCommonPath =
> > >
> >
> os.path.normpath(mws.join(GlobalData.gGlobalDefines["EDK_TOOLS_PATH"]
> > > , "Source/C/Common/PcdValueCommon.c"))--
> > > 2.43.0.windows.1
> >
> >
> >
> >
> >
>
>
>
>
>
-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.
View/Reply Online (#113428): https://edk2.groups.io/g/devel/message/113428
Mute This Topic: https://groups.io/mt/103616495/7686176
Group Owner: devel+owner@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub [rebecca@openfw.io]
-=-=-=-=-=-=-=-=-=-=-=-
[-- Attachment #2: Type: text/html, Size: 21634 bytes --]
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [edk2-devel] [PATCH v1 1/1] BaseTools: Fix raw strings containing valid escape characters
2024-01-09 8:14 ` Yuwei Chen
@ 2024-01-10 6:52 ` Huang, Yanbo
2024-01-10 18:34 ` Rebecca Cran
0 siblings, 1 reply; 12+ messages in thread
From: Huang, Yanbo @ 2024-01-10 6:52 UTC (permalink / raw)
To: Chen, Christine, Gao, Liming, devel@edk2.groups.io,
joeyvagedes@microsoft.com, Kinney, Michael D,
'Rebecca Cran'
Cc: Feng, Bob C
[-- Attachment #1: Type: text/plain, Size: 12153 bytes --]
@Gao, Liming<mailto:gaoliming@byosoft.com.cn> @'Rebecca Cran'<mailto:rebecca@bsdio.com>
Could you please help to merge this PR?
BaseTools: Fix raw strings containing valid escape characters by YuweiChen1110 · Pull Request #5238 · tianocore/edk2 (github.com)<https://github.com/tianocore/edk2/pull/5238>
Thanks in advance!
Best Regards,
Yanbo Huang
From: Chen, Christine <yuwei.chen@intel.com>
Sent: Tuesday, January 9, 2024 4:15 PM
To: Gao, Liming <gaoliming@byosoft.com.cn>; devel@edk2.groups.io; joeyvagedes@microsoft.com; Huang, Yanbo <yanbo.huang@intel.com>
Cc: Feng, Bob C <bob.c.feng@intel.com>; Kinney, Michael D <michael.d.kinney@intel.com>; 'Rebecca Cran' <rebecca@bsdio.com>
Subject: RE: [edk2-devel] [PATCH v1 1/1] BaseTools: Fix raw strings containing valid escape characters
Liming:
PR has been created: BaseTools: Fix raw strings containing valid escape characters by YuweiChen1110 · Pull Request #5238 · tianocore/edk2 (github.com)<https://github.com/tianocore/edk2/pull/5238>
Thanks for reviewing.
Thanks,
Christine
From: gaoliming <gaoliming@byosoft.com.cn<mailto:gaoliming@byosoft.com.cn>>
Sent: Tuesday, January 9, 2024 10:01 AM
To: devel@edk2.groups.io<mailto:devel@edk2.groups.io>; joeyvagedes@microsoft.com<mailto:joeyvagedes@microsoft.com>; Huang, Yanbo <yanbo.huang@intel.com<mailto:yanbo.huang@intel.com>>
Cc: Chen, Christine <yuwei.chen@intel.com<mailto:yuwei.chen@intel.com>>; Feng, Bob C <bob.c.feng@intel.com<mailto:bob.c.feng@intel.com>>; Kinney, Michael D <michael.d.kinney@intel.com<mailto:michael.d.kinney@intel.com>>; 'Rebecca Cran' <rebecca@bsdio.com<mailto:rebecca@bsdio.com>>
Subject: 回复: [edk2-devel] [PATCH v1 1/1] BaseTools: Fix raw strings containing valid escape characters
Joey and Yanbo:
Thanks for your update. So, this one is new introduced issue. I agree to resolve it asap.
Christine:
Can you create Pull Request to merge it?
Thanks
Liming
发件人: devel@edk2.groups.io<mailto:devel@edk2.groups.io> <devel@edk2.groups.io<mailto:devel@edk2.groups.io>> 代表 Joey Vagedes via groups.io
发送时间: 2024年1月8日 23:09
收件人: Huang, Yanbo <yanbo.huang@intel.com<mailto:yanbo.huang@intel.com>>
抄送: Chen, Christine <yuwei.chen@intel.com<mailto:yuwei.chen@intel.com>>; Feng, Bob C <bob.c.feng@intel.com<mailto:bob.c.feng@intel.com>>; Gao, Liming <gaoliming@byosoft.com.cn<mailto:gaoliming@byosoft.com.cn>>; Kinney, Michael D <michael.d.kinney@intel.com<mailto:michael.d.kinney@intel.com>>; Rebecca Cran <rebecca@bsdio.com<mailto:rebecca@bsdio.com>>; devel@edk2.groups.io<mailto:devel@edk2.groups.io>
主题: Re: [edk2-devel] [PATCH v1 1/1] BaseTools: Fix raw strings containing valid escape characters
Hi Liming,
It was introduced due to the fact that a few of the regex strings with invalid escape sequences (due to regex using their own escape sequences) also included valid escape sequences - for handling new lines. I did not see the valid escape characters in the strings, and the virtual platforms (Ovmf, etc) did not have the circumstances to catch this, so it slipped through the cracks.
Some platform DSCs do, and cause the break that the patch is resolving. I’ve already verified this fix resolves the issue on numerous platforms.
thanks,
Joey
On Mon, Jan 8, 2024 at 5:59 AM Huang, Yanbo <yanbo.huang@intel.com<mailto:yanbo.huang@intel.com>> wrote:
Hi Liming,
This issue is introduced by this commit:
SHA-1: 9f0061a03b61d282fbc0ba5be22155d06a5e64a1
* BaseTools: Resolve regex syntax warnings
Switches regex patterns to raw text to resolve python 3.12 syntax
warnings in regards to invalid escape sequences, as is suggested by the
re (regex) module in python.
Best Regards,
Yanbo Huang
-----Original Message-----
From: gaoliming <gaoliming@byosoft.com.cn<mailto:gaoliming@byosoft.com.cn>>
Sent: Monday, January 8, 2024 9:25 PM
To: devel@edk2.groups.io<mailto:devel@edk2.groups.io>; Chen, Christine <yuwei.chen@intel.com<mailto:yuwei.chen@intel.com>>; 'Joey Vagedes' <joey.vagedes@gmail.com<mailto:joey.vagedes@gmail.com>>
Cc: 'Rebecca Cran' <rebecca@bsdio.com<mailto:rebecca@bsdio.com>>; Feng, Bob C <bob.c.feng@intel.com<mailto:bob.c.feng@intel.com>>; Huang, Yanbo <yanbo.huang@intel.com<mailto:yanbo.huang@intel.com>>; Kinney, Michael D <michael.d.kinney@intel.com<mailto:michael.d.kinney@intel.com>>
Subject: 回复: [edk2-devel] [PATCH v1 1/1] BaseTools: Fix raw strings containing valid escape characters
Christine:
Is this a regression issue? Or does it resolve new usage? I am curious why it is not reported before.
Thanks
Liming
> -----邮件原件-----
> 发件人: devel@edk2.groups.io<mailto:devel@edk2.groups.io> <devel@edk2.groups.io<mailto:devel@edk2.groups.io>> 代表 Yuwei Chen
> 发送时间: 2024年1月8日 13:37
> 收件人: devel@edk2.groups.io<mailto:devel@edk2.groups.io>; Chen, Christine <yuwei.chen@intel.com<mailto:yuwei.chen@intel.com>>;
> Joey Vagedes <joey.vagedes@gmail.com<mailto:joey.vagedes@gmail.com>>
> 抄送: Rebecca Cran <rebecca@bsdio.com<mailto:rebecca@bsdio.com>>; Gao, Liming
> <gaoliming@byosoft.com.cn<mailto:gaoliming@byosoft.com.cn>>; Feng, Bob C <bob.c.feng@intel.com<mailto:bob.c.feng@intel.com>>; Huang,
> Yanbo <yanbo.huang@intel.com<mailto:yanbo.huang@intel.com>>; Kinney, Michael D
> <michael.d.kinney@intel.com<mailto:michael.d.kinney@intel.com>>
> 主题: Re: [edk2-devel] [PATCH v1 1/1] BaseTools: Fix raw strings
containing
> valid escape characters
>
> Hi Liming and Cran,
>
> We should merge this fixing patch asap as it influences BaseTool's usage.
>
> Thanks,
> Christine
>
> > -----Original Message-----
> > From: devel@edk2.groups.io<mailto:devel@edk2.groups.io> <devel@edk2.groups.io<mailto:devel@edk2.groups.io>> On Behalf Of Yuwei
> > Chen
> > Sent: Monday, January 8, 2024 9:16 AM
> > To: Joey Vagedes <joey.vagedes@gmail.com<mailto:joey.vagedes@gmail.com>>; devel@edk2.groups.io<mailto:devel@edk2.groups.io>
> > Cc: Rebecca Cran <rebecca@bsdio.com<mailto:rebecca@bsdio.com>>; Gao, Liming
> > <gaoliming@byosoft.com.cn<mailto:gaoliming@byosoft.com.cn>>; Feng, Bob C <bob.c.feng@intel.com<mailto:bob.c.feng@intel.com>>
> > Subject: Re: [edk2-devel] [PATCH v1 1/1] BaseTools: Fix raw strings
> containing
> > valid escape characters
> >
> > Hi Joey,
> >
> > Thanks for fixing.
> >
> > Reviewed-by: Yuwei Chen <yuwei.chen@intel.com<mailto:yuwei.chen@intel.com>>
> >
> > > -----Original Message-----
> > > From: Joey Vagedes <joey.vagedes@gmail.com<mailto:joey.vagedes@gmail.com>>
> > > Sent: Friday, December 29, 2023 12:48 AM
> > > To: devel@edk2.groups.io<mailto:devel@edk2.groups.io>
> > > Cc: Rebecca Cran <rebecca@bsdio.com<mailto:rebecca@bsdio.com>>; Gao, Liming
> > > <gaoliming@byosoft.com.cn<mailto:gaoliming@byosoft.com.cn>>; Feng, Bob C <bob.c.feng@intel.com<mailto:bob.c.feng@intel.com>>;
> Chen,
> > > Christine <yuwei.chen@intel.com<mailto:yuwei.chen@intel.com>>
> > > Subject: [PATCH v1 1/1] BaseTools: Fix raw strings containing
> > > valid escape characters
> > >
> > > Fixes raw regex strings that contain valid (and purposeful) escape
> > > characters as they are being treated as individual characters
> > > rather than the single escaped character they represent (i.e. '\t'
> > > is being
treated
> as
> > a '\' and a 't'
> > > rather than a single tab character).
> > >
> > > Signed-off-by: Joey Vagedes <joey.vagedes@gmail.com<mailto:joey.vagedes@gmail.com>>
> > > Cc: Rebecca Cran <rebecca@bsdio.com<mailto:rebecca@bsdio.com>>
> > > Cc: Liming Gao <gaoliming@byosoft.com.cn<mailto:gaoliming@byosoft.com.cn>>
> > > Cc: Bob Feng <bob.c.feng@intel.com<mailto:bob.c.feng@intel.com>>
> > > Cc: Yuwei Chen <yuwei.chen@intel.com<mailto:yuwei.chen@intel.com>>
> > > ---
> > > BaseTools/Source/Python/AutoGen/GenMake.py | 2 +-
> > > BaseTools/Source/Python/Common/Misc.py | 2 +-
> > > BaseTools/Source/Python/Workspace/DscBuildData.py | 4 ++--
> > > 3 files changed, 4 insertions(+), 4 deletions(-)
> > >
> > > diff --git a/BaseTools/Source/Python/AutoGen/GenMake.py
> > > b/BaseTools/Source/Python/AutoGen/GenMake.py
> > > index c416fe172fe5..fbd35d498923 100755
> > > --- a/BaseTools/Source/Python/AutoGen/GenMake.py
> > > +++ b/BaseTools/Source/Python/AutoGen/GenMake.py
> > > @@ -28,7 +28,7 @@ from Common.DataType import
> > TAB_COMPILER_MSFT
> > > gIncludePattern = re.compile(r"^[ \t]*[#%]?[ \t]*include(?:[
> > > \t]*(?:\\(?:\r\n|\r|\n))*[ \t]*)*(?:\(?[ \"<]?[ \t]*)([-\w.\\/()
> > > \t]+)(?:[ \t]*[\">]?\)?)", re.MULTILINE | re.UNICODE |
> > > re.IGNORECASE) ## Regular expression for matching macro used in
> > > header file inclusion-gMacroPattern =
> > > re.compile(r"([_A-Z][_A-Z0-9]*)[ \t]*\((.+)\)",
> > > re.UNICODE)+gMacroPattern = re.compile("([_A-Z][_A-Z0-9]*)[
> > > \t]*\\((.+)\\)",
> > > re.UNICODE) gIsFileMap = {} diff --git
> > > a/BaseTools/Source/Python/Common/Misc.py
> > > b/BaseTools/Source/Python/Common/Misc.py
> > > index f87d9dbdba39..34bfc90abd94 100755
> > > --- a/BaseTools/Source/Python/Common/Misc.py
> > > +++ b/BaseTools/Source/Python/Common/Misc.py
> > > @@ -1926,4 +1926,4 @@ def CopyDict(ori_dict):
> > > # Remove the c/c++ comments: // and /* */ # def
> > > RemoveCComments(ctext):- return re.sub(r'//.*?\n|/\*.*?\*/', '\n',
> ctext,
> > > flags=re.S)+ return re.sub('//.*?\n|/\\*.*?\\*/', '\n', ctext,
> flags=re.S)diff --
> > git
> > > a/BaseTools/Source/Python/Workspace/DscBuildData.py
> > > b/BaseTools/Source/Python/Workspace/DscBuildData.py
> > > index 817cdbe5f19c..4768099343e5 100644
> > > --- a/BaseTools/Source/Python/Workspace/DscBuildData.py
> > > +++ b/BaseTools/Source/Python/Workspace/DscBuildData.py
> > > @@ -2840,7 +2840,7 @@ class DscBuildData(PlatformBuildClassObject):
> > > # start generating makefile MakeApp =
> PcdMakefileHeader if
> > > sys.platform == "win32":- MakeApp = MakeApp + r'APPFILE
> > > = %s\%s.exe\n' % (self.OutputPath, PcdValueInitName) + r'APPNAME =
> > > %s\n' % (PcdValueInitName) + r'OBJECTS = %s\%s.obj %s.obj\n' %
> > > (self.OutputPath, PcdValueInitName, os.path.join(self.OutputPath,
> > > PcdValueCommonName)) + 'INC = '+ MakeApp = MakeApp
> + 'APPFILE
> > > = %s\\%s.exe\n' % (self.OutputPath, PcdValueInitName) + 'APPNAME =
> > > %s\n' % (PcdValueInitName) + 'OBJECTS = %s\\%s.obj %s.obj\n' %
> > > (self.OutputPath, PcdValueInitName, os.path.join(self.OutputPath,
> > > PcdValueCommonName)) + 'INC = ' else:
> MakeApp = MakeApp +
> > > PcdGccMakefile MakeApp = MakeApp + 'APPFILE
> = %s/%s\n' %
> > > (self.OutputPath, PcdValueInitName) + 'APPNAME = %s\n' %
> > > (PcdValueInitName) + 'OBJECTS = %s/%s.o %s.o\n' %
> > > (self.OutputPath, PcdValueInitName, os.path.join(self.OutputPath,
> > > PcdValueCommonName))
> > +
> > > \@@ -2950,7 +2950,7 @@ class DscBuildData(PlatformBuildClassObject):
> > > MakeApp += "$(OBJECTS) : %s\n" % include_file
> if sys.platform
> > ==
> > > "win32": PcdValueCommonPath =
> > >
> >
> os.path.normpath(mws.join(GlobalData.gGlobalDefines["EDK_TOOLS_PATH"]
> > > , "Source\C\Common\PcdValueCommon.c"))- MakeApp =
> MakeApp +
> > > r'%s\PcdValueCommon.c : %s\n' % (self.OutputPath,
> > PcdValueCommonPath)+
> > > MakeApp = MakeApp + '%s\\PcdValueCommon.c : %s\n' %
> (self.OutputPath,
> > > PcdValueCommonPath) MakeApp = MakeApp + '\tcopy
> /y %s
> > $@\n' %
> > > (PcdValueCommonPath) else:
> PcdValueCommonPath =
> > >
> >
> os.path.normpath(mws.join(GlobalData.gGlobalDefines["EDK_TOOLS_PATH"]
> > > , "Source/C/Common/PcdValueCommon.c"))--
> > > 2.43.0.windows.1
> >
> >
> >
> >
> >
>
>
>
>
>
-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.
View/Reply Online (#113516): https://edk2.groups.io/g/devel/message/113516
Mute This Topic: https://groups.io/mt/103616495/7686176
Group Owner: devel+owner@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub [rebecca@openfw.io]
-=-=-=-=-=-=-=-=-=-=-=-
[-- Attachment #2: Type: text/html, Size: 24085 bytes --]
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [edk2-devel] [PATCH v1 1/1] BaseTools: Fix raw strings containing valid escape characters
2024-01-10 6:52 ` Huang, Yanbo
@ 2024-01-10 18:34 ` Rebecca Cran
0 siblings, 0 replies; 12+ messages in thread
From: Rebecca Cran @ 2024-01-10 18:34 UTC (permalink / raw)
To: Huang, Yanbo, Chen, Christine, Gao, Liming, devel@edk2.groups.io,
joeyvagedes@microsoft.com, Kinney, Michael D
Cc: Feng, Bob C
On 1/9/2024 11:52 PM, Huang, Yanbo wrote:
>
> @Gao, Liming <mailto:gaoliming@byosoft.com.cn> @'Rebecca Cran'
> <mailto:rebecca@bsdio.com>
>
> Could you please help to merge this PR?
>
> BaseTools: Fix raw strings containing valid escape characters by
> YuweiChen1110 · Pull Request #5238 · tianocore/edk2 (github.com)
> <https://github.com/tianocore/edk2/pull/5238>
>
> Thanks in advance!
>
It's been merged.
--
Rebecca Cran
-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.
View/Reply Online (#113538): https://edk2.groups.io/g/devel/message/113538
Mute This Topic: https://groups.io/mt/103616495/7686176
Group Owner: devel+owner@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub [rebecca@openfw.io]
-=-=-=-=-=-=-=-=-=-=-=-
^ permalink raw reply [flat|nested] 12+ messages in thread
end of thread, other threads:[~2024-01-10 18:35 UTC | newest]
Thread overview: 12+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2023-12-28 16:47 [edk2-devel] [PATCH v1 0/1] Fix raw strings containing valid escape characters Joey Vagedes via groups.io
2023-12-28 16:47 ` [edk2-devel] [PATCH v1 1/1] BaseTools: " Joey Vagedes via groups.io
2024-01-08 1:15 ` Yuwei Chen
[not found] ` <17A83A4D186CC8F7.1387@groups.io>
2024-01-08 5:37 ` Yuwei Chen
2024-01-08 6:06 ` Ni, Ray
2024-01-08 13:25 ` 回复: " gaoliming via groups.io
2024-01-08 13:59 ` Huang, Yanbo
2024-01-08 15:08 ` Joey Vagedes via groups.io
2024-01-09 2:01 ` 回复: " gaoliming via groups.io
2024-01-09 8:14 ` Yuwei Chen
2024-01-10 6:52 ` Huang, Yanbo
2024-01-10 18:34 ` Rebecca Cran
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox