Re: [edk2-devel] [PATCH v1 1/1] BaseTools: Fix raw strings containing valid escape characters

["Joey Vagedes via groups.io" <joeyvagedes=microsoft.com@groups.io>]

[-- Attachment #1: Type: text/plain, Size: 8705 bytes --]

Hi Liming,

It was introduced due to the fact that a few of the regex strings with
invalid escape sequences (due to regex using their own escape sequences)
also included valid escape sequences - for handling new lines. I did not
see the valid escape characters in the strings, and the virtual platforms
(Ovmf, etc) did not have the circumstances to catch this, so it slipped
through the cracks.

Some platform DSCs do, and cause the break that the patch is resolving.
I’ve already verified this fix resolves the issue on numerous platforms.

thanks,
Joey

On Mon, Jan 8, 2024 at 5:59 AM Huang, Yanbo <yanbo.huang@intel.com> wrote:

> Hi Liming,
>
> This issue is introduced by this commit:
> SHA-1: 9f0061a03b61d282fbc0ba5be22155d06a5e64a1
>
> * BaseTools: Resolve regex syntax warnings
>
> Switches regex patterns to raw text to resolve python 3.12 syntax
> warnings in regards to invalid escape sequences, as is suggested by the
> re (regex) module in python.
>
> Best Regards,
> Yanbo Huang
>
> -----Original Message-----
> From: gaoliming <gaoliming@byosoft.com.cn>
> Sent: Monday, January 8, 2024 9:25 PM
> To: devel@edk2.groups.io; Chen, Christine <yuwei.chen@intel.com>; 'Joey
> Vagedes' <joey.vagedes@gmail.com>
> Cc: 'Rebecca Cran' <rebecca@bsdio.com>; Feng, Bob C <bob.c.feng@intel.com>;
> Huang, Yanbo <yanbo.huang@intel.com>; Kinney, Michael D <
> michael.d.kinney@intel.com>
> Subject: 回复: [edk2-devel] [PATCH v1 1/1] BaseTools: Fix raw strings
> containing valid escape characters
>
> Christine:
>   Is this a regression issue? Or does it resolve new usage? I am curious
> why it is not reported before.
>
> Thanks
> Liming
> > -----邮件原件-----
> > 发件人: devel@edk2.groups.io <devel@edk2.groups.io> 代表 Yuwei Chen
> > 发送时间: 2024年1月8日 13:37
> > 收件人: devel@edk2.groups.io; Chen, Christine <yuwei.chen@intel.com>;
> > Joey Vagedes <joey.vagedes@gmail.com>
> > 抄送: Rebecca Cran <rebecca@bsdio.com>; Gao, Liming
> > <gaoliming@byosoft.com.cn>; Feng, Bob C <bob.c.feng@intel.com>; Huang,
> > Yanbo <yanbo.huang@intel.com>; Kinney, Michael D
> > <michael.d.kinney@intel.com>
> > 主题: Re: [edk2-devel] [PATCH v1 1/1] BaseTools: Fix raw strings
> containing
> > valid escape characters
> >
> > Hi Liming and Cran,
> >
> > We should merge this fixing patch asap as it influences BaseTool's usage.
> >
> > Thanks,
> > Christine
> >
> > > -----Original Message-----
> > > From: devel@edk2.groups.io <devel@edk2.groups.io> On Behalf Of Yuwei
> > > Chen
> > > Sent: Monday, January 8, 2024 9:16 AM
> > > To: Joey Vagedes <joey.vagedes@gmail.com>; devel@edk2.groups.io
> > > Cc: Rebecca Cran <rebecca@bsdio.com>; Gao, Liming
> > > <gaoliming@byosoft.com.cn>; Feng, Bob C <bob.c.feng@intel.com>
> > > Subject: Re: [edk2-devel] [PATCH v1 1/1] BaseTools: Fix raw strings
> > containing
> > > valid escape characters
> > >
> > > Hi Joey,
> > >
> > > Thanks for fixing.
> > >
> > > Reviewed-by: Yuwei Chen <yuwei.chen@intel.com>
> > >
> > > > -----Original Message-----
> > > > From: Joey Vagedes <joey.vagedes@gmail.com>
> > > > Sent: Friday, December 29, 2023 12:48 AM
> > > > To: devel@edk2.groups.io
> > > > Cc: Rebecca Cran <rebecca@bsdio.com>; Gao, Liming
> > > > <gaoliming@byosoft.com.cn>; Feng, Bob C <bob.c.feng@intel.com>;
> > Chen,
> > > > Christine <yuwei.chen@intel.com>
> > > > Subject: [PATCH v1 1/1] BaseTools: Fix raw strings containing
> > > > valid escape characters
> > > >
> > > > Fixes raw regex strings that contain valid (and purposeful) escape
> > > > characters as they are being treated as individual characters
> > > > rather than the single escaped character they represent (i.e. '\t'
> > > > is being
> treated
> > as
> > > a '\' and a 't'
> > > > rather than a single tab character).
> > > >
> > > > Signed-off-by: Joey Vagedes <joey.vagedes@gmail.com>
> > > > Cc: Rebecca Cran <rebecca@bsdio.com>
> > > > Cc: Liming Gao <gaoliming@byosoft.com.cn>
> > > > Cc: Bob Feng <bob.c.feng@intel.com>
> > > > Cc: Yuwei Chen <yuwei.chen@intel.com>
> > > > ---
> > > >  BaseTools/Source/Python/AutoGen/GenMake.py        | 2 +-
> > > >  BaseTools/Source/Python/Common/Misc.py            | 2 +-
> > > >  BaseTools/Source/Python/Workspace/DscBuildData.py | 4 ++--
> > > >  3 files changed, 4 insertions(+), 4 deletions(-)
> > > >
> > > > diff --git a/BaseTools/Source/Python/AutoGen/GenMake.py
> > > > b/BaseTools/Source/Python/AutoGen/GenMake.py
> > > > index c416fe172fe5..fbd35d498923 100755
> > > > --- a/BaseTools/Source/Python/AutoGen/GenMake.py
> > > > +++ b/BaseTools/Source/Python/AutoGen/GenMake.py
> > > > @@ -28,7 +28,7 @@ from Common.DataType import
> > > TAB_COMPILER_MSFT
> > > > gIncludePattern = re.compile(r"^[ \t]*[#%]?[ \t]*include(?:[
> > > > \t]*(?:\\(?:\r\n|\r|\n))*[ \t]*)*(?:\(?[ \"<]?[ \t]*)([-\w.\\/()
> > > > \t]+)(?:[ \t]*[\">]?\)?)", re.MULTILINE | re.UNICODE |
> > > > re.IGNORECASE)  ## Regular expression for matching macro used in
> > > > header file inclusion-gMacroPattern =
> > > > re.compile(r"([_A-Z][_A-Z0-9]*)[ \t]*\((.+)\)",
> > > > re.UNICODE)+gMacroPattern = re.compile("([_A-Z][_A-Z0-9]*)[
> > > > \t]*\\((.+)\\)",
> > > > re.UNICODE)  gIsFileMap = {} diff --git
> > > > a/BaseTools/Source/Python/Common/Misc.py
> > > > b/BaseTools/Source/Python/Common/Misc.py
> > > > index f87d9dbdba39..34bfc90abd94 100755
> > > > --- a/BaseTools/Source/Python/Common/Misc.py
> > > > +++ b/BaseTools/Source/Python/Common/Misc.py
> > > > @@ -1926,4 +1926,4 @@ def CopyDict(ori_dict):
> > > >  # Remove the c/c++ comments: // and /* */ # def
> > > > RemoveCComments(ctext):-    return re.sub(r'//.*?\n|/\*.*?\*/', '\n',
> > ctext,
> > > > flags=re.S)+    return re.sub('//.*?\n|/\\*.*?\\*/', '\n', ctext,
> > flags=re.S)diff --
> > > git
> > > > a/BaseTools/Source/Python/Workspace/DscBuildData.py
> > > > b/BaseTools/Source/Python/Workspace/DscBuildData.py
> > > > index 817cdbe5f19c..4768099343e5 100644
> > > > --- a/BaseTools/Source/Python/Workspace/DscBuildData.py
> > > > +++ b/BaseTools/Source/Python/Workspace/DscBuildData.py
> > > > @@ -2840,7 +2840,7 @@ class DscBuildData(PlatformBuildClassObject):
> > > >          # start generating makefile         MakeApp =
> > PcdMakefileHeader         if
> > > > sys.platform == "win32":-            MakeApp = MakeApp + r'APPFILE
> > > > = %s\%s.exe\n' % (self.OutputPath, PcdValueInitName) + r'APPNAME =
> > > > %s\n' % (PcdValueInitName) + r'OBJECTS = %s\%s.obj %s.obj\n' %
> > > > (self.OutputPath, PcdValueInitName, os.path.join(self.OutputPath,
> > > > PcdValueCommonName)) + 'INC = '+            MakeApp = MakeApp
> > + 'APPFILE
> > > > = %s\\%s.exe\n' % (self.OutputPath, PcdValueInitName) + 'APPNAME =
> > > > %s\n' % (PcdValueInitName) + 'OBJECTS = %s\\%s.obj %s.obj\n' %
> > > > (self.OutputPath, PcdValueInitName, os.path.join(self.OutputPath,
> > > > PcdValueCommonName)) + 'INC = '         else:
> > MakeApp = MakeApp +
> > > > PcdGccMakefile             MakeApp = MakeApp + 'APPFILE
> > = %s/%s\n' %
> > > > (self.OutputPath, PcdValueInitName) + 'APPNAME = %s\n' %
> > > > (PcdValueInitName) + 'OBJECTS = %s/%s.o %s.o\n' %
> > > > (self.OutputPath, PcdValueInitName, os.path.join(self.OutputPath,
> > > > PcdValueCommonName))
> > > +
> > > > \@@ -2950,7 +2950,7 @@ class DscBuildData(PlatformBuildClassObject):
> > > >              MakeApp += "$(OBJECTS) : %s\n" % include_file
> > if sys.platform
> > > ==
> > > > "win32":             PcdValueCommonPath =
> > > >
> > >
> > os.path.normpath(mws.join(GlobalData.gGlobalDefines["EDK_TOOLS_PATH"]
> > > > , "Source\C\Common\PcdValueCommon.c"))-            MakeApp =
> > MakeApp +
> > > > r'%s\PcdValueCommon.c : %s\n' % (self.OutputPath,
> > > PcdValueCommonPath)+
> > > > MakeApp = MakeApp + '%s\\PcdValueCommon.c : %s\n' %
> > (self.OutputPath,
> > > > PcdValueCommonPath)             MakeApp = MakeApp + '\tcopy
> > /y %s
> > > $@\n' %
> > > > (PcdValueCommonPath)         else:
> > PcdValueCommonPath =
> > > >
> > >
> > os.path.normpath(mws.join(GlobalData.gGlobalDefines["EDK_TOOLS_PATH"]
> > > > , "Source/C/Common/PcdValueCommon.c"))--
> > > > 2.43.0.windows.1
> > >
> > >
> > >
> > >
> > >
> >
> >
> >
> > 
> >
>
>
>
>


-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.
View/Reply Online (#113402): https://edk2.groups.io/g/devel/message/113402
Mute This Topic: https://groups.io/mt/103597129/7686176
Group Owner: devel+owner@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub [rebecca@openfw.io]
-=-=-=-=-=-=-=-=-=-=-=-



[-- Attachment #2: Type: text/html, Size: 13807 bytes --]