* [PATCH] BaseTools: Normalize case of pathname when evaluating Macros. @ 2020-09-23 10:57 Bob Feng 2020-09-23 14:23 ` [edk2-devel] " Andrew Fish 2020-09-24 6:29 ` Yuwei Chen 0 siblings, 2 replies; 6+ messages in thread From: Bob Feng @ 2020-09-23 10:57 UTC (permalink / raw) To: devel; +Cc: Mingyue Liang, Liming Gao, Yuwei Chen From: Mingyue Liang <mingyuex.liang@intel.com> REF: https://bugzilla.tianocore.org/show_bug.cgi?id=2880 Currently, When doing the Incremental build, the directory macros extended to absolute path in output Makefile, which is inconsistent with the output of Clean build. When we do macro replacement, we can't replace macro due to inconsistent path case, which results in inconsistent display of incremental build and clean build in makefile.Therefore, the path is converted to achieve the correct macro replacement. Signed-off-by: Mingyue Liang <mingyuex.liang@intel.com> Cc: Bob Feng <bob.c.feng@intel.com> Cc: Liming Gao <gaoliming@byosoft.com.cn> Cc: Yuwei Chen <yuwei.chen@intel.com> --- BaseTools/Source/Python/AutoGen/GenMake.py | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/BaseTools/Source/Python/AutoGen/GenMake.py b/BaseTools/Source/Python/AutoGen/GenMake.py index 0314d0ea34..b04d3f5436 100755 --- a/BaseTools/Source/Python/AutoGen/GenMake.py +++ b/BaseTools/Source/Python/AutoGen/GenMake.py @@ -786,8 +786,10 @@ cleanlib: def ReplaceMacro(self, str): for Macro in self.MacroList: - if self._AutoGenObject.Macros[Macro] and self._AutoGenObject.Macros[Macro] in str: - str = str.replace(self._AutoGenObject.Macros[Macro], '$(' + Macro + ')') + if self._AutoGenObject.Macros[Macro] and os.path.normcase(self._AutoGenObject.Macros[Macro]) in os.path.normcase(str): + replace_dir = str[os.path.normcase(str).index(os.path.normcase(self._AutoGenObject.Macros[Macro])): os.path.normcase(str).index( + os.path.normcase(self._AutoGenObject.Macros[Macro])) + len(self._AutoGenObject.Macros[Macro])] + str = str.replace(replace_dir, '$(' + Macro + ')') return str def CommandExceedLimit(self): -- 2.28.0.windows.1 ^ permalink raw reply related [flat|nested] 6+ messages in thread
* Re: [edk2-devel] [PATCH] BaseTools: Normalize case of pathname when evaluating Macros. 2020-09-23 10:57 [PATCH] BaseTools: Normalize case of pathname when evaluating Macros Bob Feng @ 2020-09-23 14:23 ` Andrew Fish 2020-09-23 14:59 ` Bob Feng 2020-09-24 6:29 ` Yuwei Chen 1 sibling, 1 reply; 6+ messages in thread From: Andrew Fish @ 2020-09-23 14:23 UTC (permalink / raw) To: devel, bob.c.feng; +Cc: Mingyue Liang, Liming Gao, Yuwei Chen Does this work on case sensitive file systems? > On Sep 23, 2020, at 3:58 AM, Bob Feng <bob.c.feng@intel.com> wrote: > > From: Mingyue Liang <mingyuex.liang@intel.com> > > REF: https://bugzilla.tianocore.org/show_bug.cgi?id=2880 > > Currently, When doing the Incremental build, the directory > macros extended to absolute path in output Makefile, which > is inconsistent with the output of Clean build. > > When we do macro replacement, we can't replace macro due to > inconsistent path case, which results in inconsistent display > of incremental build and clean build in makefile.Therefore, > the path is converted to achieve the correct macro replacement. > > Signed-off-by: Mingyue Liang <mingyuex.liang@intel.com> > Cc: Bob Feng <bob.c.feng@intel.com> > Cc: Liming Gao <gaoliming@byosoft.com.cn> > Cc: Yuwei Chen <yuwei.chen@intel.com> > --- > BaseTools/Source/Python/AutoGen/GenMake.py | 6 ++++-- > 1 file changed, 4 insertions(+), 2 deletions(-) > > diff --git a/BaseTools/Source/Python/AutoGen/GenMake.py b/BaseTools/Source/Python/AutoGen/GenMake.py > index 0314d0ea34..b04d3f5436 100755 > --- a/BaseTools/Source/Python/AutoGen/GenMake.py > +++ b/BaseTools/Source/Python/AutoGen/GenMake.py > @@ -786,8 +786,10 @@ cleanlib: > > def ReplaceMacro(self, str): > for Macro in self.MacroList: > - if self._AutoGenObject.Macros[Macro] and self._AutoGenObject.Macros[Macro] in str: > - str = str.replace(self._AutoGenObject.Macros[Macro], '$(' + Macro + ')') > + if self._AutoGenObject.Macros[Macro] and os.path.normcase(self._AutoGenObject.Macros[Macro]) in os.path.normcase(str): > + replace_dir = str[os.path.normcase(str).index(os.path.normcase(self._AutoGenObject.Macros[Macro])): os.path.normcase(str).index( > + os.path.normcase(self._AutoGenObject.Macros[Macro])) + len(self._AutoGenObject.Macros[Macro])] > + str = str.replace(replace_dir, '$(' + Macro + ')') > return str > > def CommandExceedLimit(self): > -- > 2.28.0.windows.1 > > > > > > ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [edk2-devel] [PATCH] BaseTools: Normalize case of pathname when evaluating Macros. 2020-09-23 14:23 ` [edk2-devel] " Andrew Fish @ 2020-09-23 14:59 ` Bob Feng 2020-09-23 15:31 ` Andrew Fish 0 siblings, 1 reply; 6+ messages in thread From: Bob Feng @ 2020-09-23 14:59 UTC (permalink / raw) To: Andrew Fish, devel@edk2.groups.io Cc: Liang, MingyueX, Liming Gao, Chen, Christine Yes. we did test on Windows and Linux. From the https://docs.python.org/3/library/os.path.html os.path.normcase(path) Normalize the case of a pathname. On Windows, convert all characters in the pathname to lowercase, and also convert forward slashes to backward slashes. On other operating systems, return the path unchanged. Thanks, Bob -----Original Message----- From: Andrew Fish <afish@apple.com> Sent: Wednesday, September 23, 2020 10:24 PM To: devel@edk2.groups.io; Feng, Bob C <bob.c.feng@intel.com> Cc: Liang, MingyueX <mingyuex.liang@intel.com>; Liming Gao <gaoliming@byosoft.com.cn>; Chen, Christine <yuwei.chen@intel.com> Subject: Re: [edk2-devel] [PATCH] BaseTools: Normalize case of pathname when evaluating Macros. Does this work on case sensitive file systems? > On Sep 23, 2020, at 3:58 AM, Bob Feng <bob.c.feng@intel.com> wrote: > > From: Mingyue Liang <mingyuex.liang@intel.com> > > REF: https://bugzilla.tianocore.org/show_bug.cgi?id=2880 > > Currently, When doing the Incremental build, the directory macros > extended to absolute path in output Makefile, which is inconsistent > with the output of Clean build. > > When we do macro replacement, we can't replace macro due to > inconsistent path case, which results in inconsistent display of > incremental build and clean build in makefile.Therefore, the path is > converted to achieve the correct macro replacement. > > Signed-off-by: Mingyue Liang <mingyuex.liang@intel.com> > Cc: Bob Feng <bob.c.feng@intel.com> > Cc: Liming Gao <gaoliming@byosoft.com.cn> > Cc: Yuwei Chen <yuwei.chen@intel.com> > --- > BaseTools/Source/Python/AutoGen/GenMake.py | 6 ++++-- > 1 file changed, 4 insertions(+), 2 deletions(-) > > diff --git a/BaseTools/Source/Python/AutoGen/GenMake.py > b/BaseTools/Source/Python/AutoGen/GenMake.py > index 0314d0ea34..b04d3f5436 100755 > --- a/BaseTools/Source/Python/AutoGen/GenMake.py > +++ b/BaseTools/Source/Python/AutoGen/GenMake.py > @@ -786,8 +786,10 @@ cleanlib: > > def ReplaceMacro(self, str): > for Macro in self.MacroList: > - if self._AutoGenObject.Macros[Macro] and self._AutoGenObject.Macros[Macro] in str: > - str = str.replace(self._AutoGenObject.Macros[Macro], '$(' + Macro + ')') > + if self._AutoGenObject.Macros[Macro] and os.path.normcase(self._AutoGenObject.Macros[Macro]) in os.path.normcase(str): > + replace_dir = str[os.path.normcase(str).index(os.path.normcase(self._AutoGenObject.Macros[Macro])): os.path.normcase(str).index( > + os.path.normcase(self._AutoGenObject.Macros[Macro])) + len(self._AutoGenObject.Macros[Macro])] > + str = str.replace(replace_dir, '$(' + Macro + ')') > return str > > def CommandExceedLimit(self): > -- > 2.28.0.windows.1 > > > > > > ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [edk2-devel] [PATCH] BaseTools: Normalize case of pathname when evaluating Macros. 2020-09-23 14:59 ` Bob Feng @ 2020-09-23 15:31 ` Andrew Fish 2020-09-23 15:45 ` Bob Feng 0 siblings, 1 reply; 6+ messages in thread From: Andrew Fish @ 2020-09-23 15:31 UTC (permalink / raw) To: edk2-devel-groups-io, bob.c.feng Cc: Liang, MingyueX, Liming Gao, Chen, Christine [-- Attachment #1: Type: text/plain, Size: 3823 bytes --] Bob, Sorry I was confused by the commit comment `inconsistent path case`. So this was only a bug on case insensitive file systems? I’m paranoid as macOS support case and case insensitive file systems and we have had a lot of strange bugs in the past. Thanks, Andrew Fish > On Sep 23, 2020, at 7:59 AM, Bob Feng <bob.c.feng@intel.com> wrote: > > Yes. we did test on Windows and Linux. > > From the https://docs.python.org/3/library/os.path.html <https://docs.python.org/3/library/os.path.html> > os.path.normcase(path) > Normalize the case of a pathname. On Windows, convert all characters in the pathname to lowercase, and also convert forward slashes to backward slashes. On other operating systems, return the path unchanged. > > Thanks, > Bob > -----Original Message----- > From: Andrew Fish <afish@apple.com <mailto:afish@apple.com>> > Sent: Wednesday, September 23, 2020 10:24 PM > To: devel@edk2.groups.io <mailto:devel@edk2.groups.io>; Feng, Bob C <bob.c.feng@intel.com <mailto:bob.c.feng@intel.com>> > Cc: Liang, MingyueX <mingyuex.liang@intel.com <mailto:mingyuex.liang@intel.com>>; Liming Gao <gaoliming@byosoft.com.cn <mailto:gaoliming@byosoft.com.cn>>; Chen, Christine <yuwei.chen@intel.com <mailto:yuwei.chen@intel.com>> > Subject: Re: [edk2-devel] [PATCH] BaseTools: Normalize case of pathname when evaluating Macros. > > Does this work on case sensitive file systems? >> On Sep 23, 2020, at 3:58 AM, Bob Feng <bob.c.feng@intel.com <mailto:bob.c.feng@intel.com>> wrote: >> >> From: Mingyue Liang <mingyuex.liang@intel.com <mailto:mingyuex.liang@intel.com>> >> >> REF: https://bugzilla.tianocore.org/show_bug.cgi?id=2880 <https://bugzilla.tianocore.org/show_bug.cgi?id=2880> >> >> Currently, When doing the Incremental build, the directory macros >> extended to absolute path in output Makefile, which is inconsistent >> with the output of Clean build. >> >> When we do macro replacement, we can't replace macro due to >> inconsistent path case, which results in inconsistent display of >> incremental build and clean build in makefile.Therefore, the path is >> converted to achieve the correct macro replacement. >> >> Signed-off-by: Mingyue Liang <mingyuex.liang@intel.com <mailto:mingyuex.liang@intel.com>> >> Cc: Bob Feng <bob.c.feng@intel.com <mailto:bob.c.feng@intel.com>> >> Cc: Liming Gao <gaoliming@byosoft.com.cn <mailto:gaoliming@byosoft.com.cn>> >> Cc: Yuwei Chen <yuwei.chen@intel.com <mailto:yuwei.chen@intel.com>> >> --- >> BaseTools/Source/Python/AutoGen/GenMake.py | 6 ++++-- >> 1 file changed, 4 insertions(+), 2 deletions(-) >> >> diff --git a/BaseTools/Source/Python/AutoGen/GenMake.py >> b/BaseTools/Source/Python/AutoGen/GenMake.py >> index 0314d0ea34..b04d3f5436 100755 >> --- a/BaseTools/Source/Python/AutoGen/GenMake.py >> +++ b/BaseTools/Source/Python/AutoGen/GenMake.py >> @@ -786,8 +786,10 @@ cleanlib: >> >> def ReplaceMacro(self, str): >> for Macro in self.MacroList: >> - if self._AutoGenObject.Macros[Macro] and self._AutoGenObject.Macros[Macro] in str: >> - str = str.replace(self._AutoGenObject.Macros[Macro], '$(' + Macro + ')') >> + if self._AutoGenObject.Macros[Macro] and os.path.normcase(self._AutoGenObject.Macros[Macro]) in os.path.normcase(str): >> + replace_dir = str[os.path.normcase(str).index(os.path.normcase(self._AutoGenObject.Macros[Macro])): os.path.normcase(str).index( >> + os.path.normcase(self._AutoGenObject.Macros[Macro])) + len(self._AutoGenObject.Macros[Macro])] >> + str = str.replace(replace_dir, '$(' + Macro + ')') >> return str >> >> def CommandExceedLimit(self): >> -- >> 2.28.0.windows.1 >> >> >> >> >> >> > > > [-- Attachment #2: Type: text/html, Size: 22620 bytes --] ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [edk2-devel] [PATCH] BaseTools: Normalize case of pathname when evaluating Macros. 2020-09-23 15:31 ` Andrew Fish @ 2020-09-23 15:45 ` Bob Feng 0 siblings, 0 replies; 6+ messages in thread From: Bob Feng @ 2020-09-23 15:45 UTC (permalink / raw) To: Andrew Fish, edk2-devel-groups-io Cc: Liang, MingyueX, Liming Gao, Chen, Christine [-- Attachment #1: Type: text/plain, Size: 4015 bytes --] Yes. this bug only happens on case insensitive file systems. Thanks, Bob From: Andrew Fish <afish@apple.com> Sent: Wednesday, September 23, 2020 11:31 PM To: edk2-devel-groups-io <devel@edk2.groups.io>; Feng, Bob C <bob.c.feng@intel.com> Cc: Liang, MingyueX <mingyuex.liang@intel.com>; Liming Gao <gaoliming@byosoft.com.cn>; Chen, Christine <yuwei.chen@intel.com> Subject: Re: [edk2-devel] [PATCH] BaseTools: Normalize case of pathname when evaluating Macros. Bob, Sorry I was confused by the commit comment `inconsistent path case`. So this was only a bug on case insensitive file systems? I’m paranoid as macOS support case and case insensitive file systems and we have had a lot of strange bugs in the past. Thanks, Andrew Fish On Sep 23, 2020, at 7:59 AM, Bob Feng <bob.c.feng@intel.com<mailto:bob.c.feng@intel.com>> wrote: Yes. we did test on Windows and Linux. From the https://docs.python.org/3/library/os.path.html os.path.normcase(path) Normalize the case of a pathname. On Windows, convert all characters in the pathname to lowercase, and also convert forward slashes to backward slashes. On other operating systems, return the path unchanged. Thanks, Bob -----Original Message----- From: Andrew Fish <afish@apple.com<mailto:afish@apple.com>> Sent: Wednesday, September 23, 2020 10:24 PM To: devel@edk2.groups.io<mailto:devel@edk2.groups.io>; Feng, Bob C <bob.c.feng@intel.com<mailto:bob.c.feng@intel.com>> Cc: Liang, MingyueX <mingyuex.liang@intel.com<mailto:mingyuex.liang@intel.com>>; Liming Gao <gaoliming@byosoft.com.cn<mailto:gaoliming@byosoft.com.cn>>; Chen, Christine <yuwei.chen@intel.com<mailto:yuwei.chen@intel.com>> Subject: Re: [edk2-devel] [PATCH] BaseTools: Normalize case of pathname when evaluating Macros. Does this work on case sensitive file systems? On Sep 23, 2020, at 3:58 AM, Bob Feng <bob.c.feng@intel.com<mailto:bob.c.feng@intel.com>> wrote: From: Mingyue Liang <mingyuex.liang@intel.com<mailto:mingyuex.liang@intel.com>> REF: https://bugzilla.tianocore.org/show_bug.cgi?id=2880 Currently, When doing the Incremental build, the directory macros extended to absolute path in output Makefile, which is inconsistent with the output of Clean build. When we do macro replacement, we can't replace macro due to inconsistent path case, which results in inconsistent display of incremental build and clean build in makefile.Therefore, the path is converted to achieve the correct macro replacement. Signed-off-by: Mingyue Liang <mingyuex.liang@intel.com<mailto:mingyuex.liang@intel.com>> Cc: Bob Feng <bob.c.feng@intel.com<mailto:bob.c.feng@intel.com>> Cc: Liming Gao <gaoliming@byosoft.com.cn<mailto:gaoliming@byosoft.com.cn>> Cc: Yuwei Chen <yuwei.chen@intel.com<mailto:yuwei.chen@intel.com>> --- BaseTools/Source/Python/AutoGen/GenMake.py | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/BaseTools/Source/Python/AutoGen/GenMake.py b/BaseTools/Source/Python/AutoGen/GenMake.py index 0314d0ea34..b04d3f5436 100755 --- a/BaseTools/Source/Python/AutoGen/GenMake.py +++ b/BaseTools/Source/Python/AutoGen/GenMake.py @@ -786,8 +786,10 @@ cleanlib: def ReplaceMacro(self, str): for Macro in self.MacroList: - if self._AutoGenObject.Macros[Macro] and self._AutoGenObject.Macros[Macro] in str: - str = str.replace(self._AutoGenObject.Macros[Macro], '$(' + Macro + ')') + if self._AutoGenObject.Macros[Macro] and os.path.normcase(self._AutoGenObject.Macros[Macro]) in os.path.normcase(str): + replace_dir = str[os.path.normcase(str).index(os.path.normcase(self._AutoGenObject.Macros[Macro])): os.path.normcase(str).index( + os.path.normcase(self._AutoGenObject.Macros[Macro])) + len(self._AutoGenObject.Macros[Macro])] + str = str.replace(replace_dir, '$(' + Macro + ')') return str def CommandExceedLimit(self): -- 2.28.0.windows.1 [-- Attachment #2: Type: text/html, Size: 10645 bytes --] ^ permalink raw reply related [flat|nested] 6+ messages in thread
* Re: [PATCH] BaseTools: Normalize case of pathname when evaluating Macros. 2020-09-23 10:57 [PATCH] BaseTools: Normalize case of pathname when evaluating Macros Bob Feng 2020-09-23 14:23 ` [edk2-devel] " Andrew Fish @ 2020-09-24 6:29 ` Yuwei Chen 1 sibling, 0 replies; 6+ messages in thread From: Yuwei Chen @ 2020-09-24 6:29 UTC (permalink / raw) To: Feng, Bob C, devel@edk2.groups.io; +Cc: Liang, MingyueX, Liming Gao Reviewed-by: Yuwei Chen<yuwei.chen@intel.com> > -----Original Message----- > From: Feng, Bob C <bob.c.feng@intel.com> > Sent: Wednesday, September 23, 2020 6:58 PM > To: devel@edk2.groups.io > Cc: Liang, MingyueX <mingyuex.liang@intel.com>; Liming Gao > <gaoliming@byosoft.com.cn>; Chen, Christine <yuwei.chen@intel.com> > Subject: [PATCH] BaseTools: Normalize case of pathname when evaluating > Macros. > > From: Mingyue Liang <mingyuex.liang@intel.com> > > REF: https://bugzilla.tianocore.org/show_bug.cgi?id=2880 > > Currently, When doing the Incremental build, the directory macros extended > to absolute path in output Makefile, which is inconsistent with the output of > Clean build. > > When we do macro replacement, we can't replace macro due to inconsistent > path case, which results in inconsistent display of incremental build and clean > build in makefile.Therefore, the path is converted to achieve the correct > macro replacement. > > Signed-off-by: Mingyue Liang <mingyuex.liang@intel.com> > Cc: Bob Feng <bob.c.feng@intel.com> > Cc: Liming Gao <gaoliming@byosoft.com.cn> > Cc: Yuwei Chen <yuwei.chen@intel.com> > --- > BaseTools/Source/Python/AutoGen/GenMake.py | 6 ++++-- > 1 file changed, 4 insertions(+), 2 deletions(-) > > diff --git a/BaseTools/Source/Python/AutoGen/GenMake.py > b/BaseTools/Source/Python/AutoGen/GenMake.py > index 0314d0ea34..b04d3f5436 100755 > --- a/BaseTools/Source/Python/AutoGen/GenMake.py > +++ b/BaseTools/Source/Python/AutoGen/GenMake.py > @@ -786,8 +786,10 @@ cleanlib: > > def ReplaceMacro(self, str): > for Macro in self.MacroList: > - if self._AutoGenObject.Macros[Macro] and > self._AutoGenObject.Macros[Macro] in str: > - str = str.replace(self._AutoGenObject.Macros[Macro], '$(' + Macro > + ')') > + if self._AutoGenObject.Macros[Macro] and > os.path.normcase(self._AutoGenObject.Macros[Macro]) in > os.path.normcase(str): > + replace_dir = > str[os.path.normcase(str).index(os.path.normcase(self._AutoGenObject.Ma > cros[Macro])): os.path.normcase(str).index( > + os.path.normcase(self._AutoGenObject.Macros[Macro])) + > len(self._AutoGenObject.Macros[Macro])] > + str = str.replace(replace_dir, '$(' + Macro + ')') > return str > > def CommandExceedLimit(self): > -- > 2.28.0.windows.1 ^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2020-09-24 6:29 UTC | newest] Thread overview: 6+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2020-09-23 10:57 [PATCH] BaseTools: Normalize case of pathname when evaluating Macros Bob Feng 2020-09-23 14:23 ` [edk2-devel] " Andrew Fish 2020-09-23 14:59 ` Bob Feng 2020-09-23 15:31 ` Andrew Fish 2020-09-23 15:45 ` Bob Feng 2020-09-24 6:29 ` Yuwei Chen
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox