Re: [edk2-devel] [PATCH v1 1/1] BaseTools: Fix raw strings containing valid escape characters

["Huang, Yanbo" <yanbo.huang@intel.com>]

[-- Attachment #1: Type: text/plain, Size: 12153 bytes --]

@Gao, Liming<mailto:gaoliming@byosoft.com.cn> @'Rebecca Cran'<mailto:rebecca@bsdio.com>
Could you please help to merge this PR?
BaseTools: Fix raw strings containing valid escape characters by YuweiChen1110 · Pull Request #5238 · tianocore/edk2 (github.com)<https://github.com/tianocore/edk2/pull/5238>
Thanks in advance!

Best Regards,
Yanbo Huang

From: Chen, Christine <yuwei.chen@intel.com>
Sent: Tuesday, January 9, 2024 4:15 PM
To: Gao, Liming <gaoliming@byosoft.com.cn>; devel@edk2.groups.io; joeyvagedes@microsoft.com; Huang, Yanbo <yanbo.huang@intel.com>
Cc: Feng, Bob C <bob.c.feng@intel.com>; Kinney, Michael D <michael.d.kinney@intel.com>; 'Rebecca Cran' <rebecca@bsdio.com>
Subject: RE: [edk2-devel] [PATCH v1 1/1] BaseTools: Fix raw strings containing valid escape characters

Liming:
PR has been created: BaseTools: Fix raw strings containing valid escape characters by YuweiChen1110 · Pull Request #5238 · tianocore/edk2 (github.com)<https://github.com/tianocore/edk2/pull/5238>
Thanks for reviewing.

Thanks,
Christine
From: gaoliming <gaoliming@byosoft.com.cn<mailto:gaoliming@byosoft.com.cn>>
Sent: Tuesday, January 9, 2024 10:01 AM
To: devel@edk2.groups.io<mailto:devel@edk2.groups.io>; joeyvagedes@microsoft.com<mailto:joeyvagedes@microsoft.com>; Huang, Yanbo <yanbo.huang@intel.com<mailto:yanbo.huang@intel.com>>
Cc: Chen, Christine <yuwei.chen@intel.com<mailto:yuwei.chen@intel.com>>; Feng, Bob C <bob.c.feng@intel.com<mailto:bob.c.feng@intel.com>>; Kinney, Michael D <michael.d.kinney@intel.com<mailto:michael.d.kinney@intel.com>>; 'Rebecca Cran' <rebecca@bsdio.com<mailto:rebecca@bsdio.com>>
Subject: 回复: [edk2-devel] [PATCH v1 1/1] BaseTools: Fix raw strings containing valid escape characters

Joey and Yanbo:
 Thanks for your update. So, this one is new introduced issue. I agree to resolve it asap.

Christine:
 Can you create Pull Request to merge it?

Thanks
Liming
发件人: devel@edk2.groups.io<mailto:devel@edk2.groups.io> <devel@edk2.groups.io<mailto:devel@edk2.groups.io>> 代表 Joey Vagedes via groups.io
发送时间: 2024年1月8日 23:09
收件人: Huang, Yanbo <yanbo.huang@intel.com<mailto:yanbo.huang@intel.com>>
抄送: Chen, Christine <yuwei.chen@intel.com<mailto:yuwei.chen@intel.com>>; Feng, Bob C <bob.c.feng@intel.com<mailto:bob.c.feng@intel.com>>; Gao, Liming <gaoliming@byosoft.com.cn<mailto:gaoliming@byosoft.com.cn>>; Kinney, Michael D <michael.d.kinney@intel.com<mailto:michael.d.kinney@intel.com>>; Rebecca Cran <rebecca@bsdio.com<mailto:rebecca@bsdio.com>>; devel@edk2.groups.io<mailto:devel@edk2.groups.io>
主题: Re: [edk2-devel] [PATCH v1 1/1] BaseTools: Fix raw strings containing valid escape characters

Hi Liming,

It was introduced due to the fact that a few of the regex strings with invalid escape sequences (due to regex using their own escape sequences) also included valid escape sequences - for handling new lines. I did not see the valid escape characters in the strings, and the virtual platforms (Ovmf, etc) did not have the circumstances to catch this, so it slipped through the cracks.

Some platform DSCs do, and cause the break that the patch is resolving. I’ve already verified this fix resolves the issue on numerous platforms.

thanks,
Joey

On Mon, Jan 8, 2024 at 5:59 AM Huang, Yanbo <yanbo.huang@intel.com<mailto:yanbo.huang@intel.com>> wrote:
Hi Liming,

This issue is introduced by this commit:
SHA-1: 9f0061a03b61d282fbc0ba5be22155d06a5e64a1

* BaseTools: Resolve regex syntax warnings

Switches regex patterns to raw text to resolve python 3.12 syntax
warnings in regards to invalid escape sequences, as is suggested by the
re (regex) module in python.

Best Regards,
Yanbo Huang

-----Original Message-----
From: gaoliming <gaoliming@byosoft.com.cn<mailto:gaoliming@byosoft.com.cn>>
Sent: Monday, January 8, 2024 9:25 PM
To: devel@edk2.groups.io<mailto:devel@edk2.groups.io>; Chen, Christine <yuwei.chen@intel.com<mailto:yuwei.chen@intel.com>>; 'Joey Vagedes' <joey.vagedes@gmail.com<mailto:joey.vagedes@gmail.com>>
Cc: 'Rebecca Cran' <rebecca@bsdio.com<mailto:rebecca@bsdio.com>>; Feng, Bob C <bob.c.feng@intel.com<mailto:bob.c.feng@intel.com>>; Huang, Yanbo <yanbo.huang@intel.com<mailto:yanbo.huang@intel.com>>; Kinney, Michael D <michael.d.kinney@intel.com<mailto:michael.d.kinney@intel.com>>
Subject: 回复: [edk2-devel] [PATCH v1 1/1] BaseTools: Fix raw strings containing valid escape characters

Christine:
  Is this a regression issue? Or does it resolve new usage? I am curious why it is not reported before.

Thanks
Liming
> -----邮件原件-----
> 发件人: devel@edk2.groups.io<mailto:devel@edk2.groups.io> <devel@edk2.groups.io<mailto:devel@edk2.groups.io>> 代表 Yuwei Chen
> 发送时间: 2024年1月8日 13:37
> 收件人: devel@edk2.groups.io<mailto:devel@edk2.groups.io>; Chen, Christine <yuwei.chen@intel.com<mailto:yuwei.chen@intel.com>>;
> Joey Vagedes <joey.vagedes@gmail.com<mailto:joey.vagedes@gmail.com>>
> 抄送: Rebecca Cran <rebecca@bsdio.com<mailto:rebecca@bsdio.com>>; Gao, Liming
> <gaoliming@byosoft.com.cn<mailto:gaoliming@byosoft.com.cn>>; Feng, Bob C <bob.c.feng@intel.com<mailto:bob.c.feng@intel.com>>; Huang,
> Yanbo <yanbo.huang@intel.com<mailto:yanbo.huang@intel.com>>; Kinney, Michael D
> <michael.d.kinney@intel.com<mailto:michael.d.kinney@intel.com>>
> 主题: Re: [edk2-devel] [PATCH v1 1/1] BaseTools: Fix raw strings
containing
> valid escape characters
>
> Hi Liming and Cran,
>
> We should merge this fixing patch asap as it influences BaseTool's usage.
>
> Thanks,
> Christine
>
> > -----Original Message-----
> > From: devel@edk2.groups.io<mailto:devel@edk2.groups.io> <devel@edk2.groups.io<mailto:devel@edk2.groups.io>> On Behalf Of Yuwei
> > Chen
> > Sent: Monday, January 8, 2024 9:16 AM
> > To: Joey Vagedes <joey.vagedes@gmail.com<mailto:joey.vagedes@gmail.com>>; devel@edk2.groups.io<mailto:devel@edk2.groups.io>
> > Cc: Rebecca Cran <rebecca@bsdio.com<mailto:rebecca@bsdio.com>>; Gao, Liming
> > <gaoliming@byosoft.com.cn<mailto:gaoliming@byosoft.com.cn>>; Feng, Bob C <bob.c.feng@intel.com<mailto:bob.c.feng@intel.com>>
> > Subject: Re: [edk2-devel] [PATCH v1 1/1] BaseTools: Fix raw strings
> containing
> > valid escape characters
> >
> > Hi Joey,
> >
> > Thanks for fixing.
> >
> > Reviewed-by: Yuwei Chen <yuwei.chen@intel.com<mailto:yuwei.chen@intel.com>>
> >
> > > -----Original Message-----
> > > From: Joey Vagedes <joey.vagedes@gmail.com<mailto:joey.vagedes@gmail.com>>
> > > Sent: Friday, December 29, 2023 12:48 AM
> > > To: devel@edk2.groups.io<mailto:devel@edk2.groups.io>
> > > Cc: Rebecca Cran <rebecca@bsdio.com<mailto:rebecca@bsdio.com>>; Gao, Liming
> > > <gaoliming@byosoft.com.cn<mailto:gaoliming@byosoft.com.cn>>; Feng, Bob C <bob.c.feng@intel.com<mailto:bob.c.feng@intel.com>>;
> Chen,
> > > Christine <yuwei.chen@intel.com<mailto:yuwei.chen@intel.com>>
> > > Subject: [PATCH v1 1/1] BaseTools: Fix raw strings containing
> > > valid escape characters
> > >
> > > Fixes raw regex strings that contain valid (and purposeful) escape
> > > characters as they are being treated as individual characters
> > > rather than the single escaped character they represent (i.e. '\t'
> > > is being
treated
> as
> > a '\' and a 't'
> > > rather than a single tab character).
> > >
> > > Signed-off-by: Joey Vagedes <joey.vagedes@gmail.com<mailto:joey.vagedes@gmail.com>>
> > > Cc: Rebecca Cran <rebecca@bsdio.com<mailto:rebecca@bsdio.com>>
> > > Cc: Liming Gao <gaoliming@byosoft.com.cn<mailto:gaoliming@byosoft.com.cn>>
> > > Cc: Bob Feng <bob.c.feng@intel.com<mailto:bob.c.feng@intel.com>>
> > > Cc: Yuwei Chen <yuwei.chen@intel.com<mailto:yuwei.chen@intel.com>>
> > > ---
> > >  BaseTools/Source/Python/AutoGen/GenMake.py        | 2 +-
> > >  BaseTools/Source/Python/Common/Misc.py            | 2 +-
> > >  BaseTools/Source/Python/Workspace/DscBuildData.py | 4 ++--
> > >  3 files changed, 4 insertions(+), 4 deletions(-)
> > >
> > > diff --git a/BaseTools/Source/Python/AutoGen/GenMake.py
> > > b/BaseTools/Source/Python/AutoGen/GenMake.py
> > > index c416fe172fe5..fbd35d498923 100755
> > > --- a/BaseTools/Source/Python/AutoGen/GenMake.py
> > > +++ b/BaseTools/Source/Python/AutoGen/GenMake.py
> > > @@ -28,7 +28,7 @@ from Common.DataType import
> > TAB_COMPILER_MSFT
> > > gIncludePattern = re.compile(r"^[ \t]*[#%]?[ \t]*include(?:[
> > > \t]*(?:\\(?:\r\n|\r|\n))*[ \t]*)*(?:\(?[ \"<]?[ \t]*)([-\w.\\/()
> > > \t]+)(?:[ \t]*[\">]?\)?)", re.MULTILINE | re.UNICODE |
> > > re.IGNORECASE)  ## Regular expression for matching macro used in
> > > header file inclusion-gMacroPattern =
> > > re.compile(r"([_A-Z][_A-Z0-9]*)[ \t]*\((.+)\)",
> > > re.UNICODE)+gMacroPattern = re.compile("([_A-Z][_A-Z0-9]*)[
> > > \t]*\\((.+)\\)",
> > > re.UNICODE)  gIsFileMap = {} diff --git
> > > a/BaseTools/Source/Python/Common/Misc.py
> > > b/BaseTools/Source/Python/Common/Misc.py
> > > index f87d9dbdba39..34bfc90abd94 100755
> > > --- a/BaseTools/Source/Python/Common/Misc.py
> > > +++ b/BaseTools/Source/Python/Common/Misc.py
> > > @@ -1926,4 +1926,4 @@ def CopyDict(ori_dict):
> > >  # Remove the c/c++ comments: // and /* */ # def
> > > RemoveCComments(ctext):-    return re.sub(r'//.*?\n|/\*.*?\*/', '\n',
> ctext,
> > > flags=re.S)+    return re.sub('//.*?\n|/\\*.*?\\*/', '\n', ctext,
> flags=re.S)diff --
> > git
> > > a/BaseTools/Source/Python/Workspace/DscBuildData.py
> > > b/BaseTools/Source/Python/Workspace/DscBuildData.py
> > > index 817cdbe5f19c..4768099343e5 100644
> > > --- a/BaseTools/Source/Python/Workspace/DscBuildData.py
> > > +++ b/BaseTools/Source/Python/Workspace/DscBuildData.py
> > > @@ -2840,7 +2840,7 @@ class DscBuildData(PlatformBuildClassObject):
> > >          # start generating makefile         MakeApp =
> PcdMakefileHeader         if
> > > sys.platform == "win32":-            MakeApp = MakeApp + r'APPFILE
> > > = %s\%s.exe\n' % (self.OutputPath, PcdValueInitName) + r'APPNAME =
> > > %s\n' % (PcdValueInitName) + r'OBJECTS = %s\%s.obj %s.obj\n' %
> > > (self.OutputPath, PcdValueInitName, os.path.join(self.OutputPath,
> > > PcdValueCommonName)) + 'INC = '+            MakeApp = MakeApp
> + 'APPFILE
> > > = %s\\%s.exe\n' % (self.OutputPath, PcdValueInitName) + 'APPNAME =
> > > %s\n' % (PcdValueInitName) + 'OBJECTS = %s\\%s.obj %s.obj\n' %
> > > (self.OutputPath, PcdValueInitName, os.path.join(self.OutputPath,
> > > PcdValueCommonName)) + 'INC = '         else:
> MakeApp = MakeApp +
> > > PcdGccMakefile             MakeApp = MakeApp + 'APPFILE
> = %s/%s\n' %
> > > (self.OutputPath, PcdValueInitName) + 'APPNAME = %s\n' %
> > > (PcdValueInitName) + 'OBJECTS = %s/%s.o %s.o\n' %
> > > (self.OutputPath, PcdValueInitName, os.path.join(self.OutputPath,
> > > PcdValueCommonName))
> > +
> > > \@@ -2950,7 +2950,7 @@ class DscBuildData(PlatformBuildClassObject):
> > >              MakeApp += "$(OBJECTS) : %s\n" % include_file
> if sys.platform
> > ==
> > > "win32":             PcdValueCommonPath =
> > >
> >
> os.path.normpath(mws.join(GlobalData.gGlobalDefines["EDK_TOOLS_PATH"]
> > > , "Source\C\Common\PcdValueCommon.c"))-            MakeApp =
> MakeApp +
> > > r'%s\PcdValueCommon.c : %s\n' % (self.OutputPath,
> > PcdValueCommonPath)+
> > > MakeApp = MakeApp + '%s\\PcdValueCommon.c : %s\n' %
> (self.OutputPath,
> > > PcdValueCommonPath)             MakeApp = MakeApp + '\tcopy
> /y %s
> > $@\n' %
> > > (PcdValueCommonPath)         else:
> PcdValueCommonPath =
> > >
> >
> os.path.normpath(mws.join(GlobalData.gGlobalDefines["EDK_TOOLS_PATH"]
> > > , "Source/C/Common/PcdValueCommon.c"))--
> > > 2.43.0.windows.1
> >
> >
> >
> >
> >
>
>
>
>
>



-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.
View/Reply Online (#113516): https://edk2.groups.io/g/devel/message/113516
Mute This Topic: https://groups.io/mt/103616495/7686176
Group Owner: devel+owner@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub [rebecca@openfw.io]
-=-=-=-=-=-=-=-=-=-=-=-



[-- Attachment #2: Type: text/html, Size: 24085 bytes --]