* [PATCH v2 0/1] BaseTools/Trim: Normalize filepaths to fix comparisons on Windows @ 2018-06-28 19:31 Chris Co 2018-06-28 19:31 ` [PATCH v2 1/1] " Chris Co 0 siblings, 1 reply; 3+ messages in thread From: Chris Co @ 2018-06-28 19:31 UTC (permalink / raw) To: edk2-devel@lists.01.org Cc: Leif Lindholm, Yonghong Zhu, Liming Gao, Evan Lloyd REF: https://github.com/christopherco/edk2/tree/trim_gcc_v2 v1: https://lists.01.org/pipermail/edk2-devel/2018-June/026562.html new in v2: Use os module to normalize filepaths instead of direct string manipulation Contributed-under: TianoCore Contribution Agreement 1.1 Signed-off-by: Christopher Co <christopher.co@microsoft.com> Cc: Leif Lindholm <leif.lindholm@linaro.org> Cc: Yonghong Zhu <yonghong.zhu@intel.com> Cc: Liming Gao <liming.gao@intel.com> Cc: Evan Lloyd <Evan.Lloyd@arm.com> Christopher Co (1): BaseTools/Trim: Normalize filepaths to fix comparisons on Windows BaseTools/Source/Python/Trim/Trim.py | 2 ++ 1 file changed, 2 insertions(+) -- 2.16.2.gvfs.1.33.gf5370f1 ^ permalink raw reply [flat|nested] 3+ messages in thread
* [PATCH v2 1/1] BaseTools/Trim: Normalize filepaths to fix comparisons on Windows 2018-06-28 19:31 [PATCH v2 0/1] BaseTools/Trim: Normalize filepaths to fix comparisons on Windows Chris Co @ 2018-06-28 19:31 ` Chris Co 2018-07-03 5:04 ` Gao, Liming 0 siblings, 1 reply; 3+ messages in thread From: Chris Co @ 2018-06-28 19:31 UTC (permalink / raw) To: edk2-devel@lists.01.org; +Cc: Leif Lindholm, Yonghong Zhu, Liming Gao When using Linaro GCC5+ arm-eabi toolchain on Windows, the generated DSDT.iii contains a canonicalized ("\.\" removed and lower case) filepath for the preprocessed DSDT.i file in the first line. Trim.exe is called on DSDT.iii to generate DSDT.iiii, which does a line for line comparison of filepaths encountered to the preprocessed DSDT.i filepath found in the first line to determine what lines to place in DSDT.iiii. Since the DSDT.i filepath is canonicalized and all later filepaths in DSDT.iii are not canonicalized, all comparisons fail and the result is in an empty DSDT.iiii. Issue was first reported to Linaro here: https://bugs.linaro.org/show_bug.cgi?id=2909 where the recommendation was to address the issue in Trim.exe. This patch normalizes the case and pathname of all filepaths encountered during Trim.exe execution on preprocessed files. This fixes comparisons of filepaths that contain mismatching case on case-insensitive filesystems, redundant separators, and uplevel references. Contributed-under: TianoCore Contribution Agreement 1.1 Signed-off-by: Christopher Co <christopher.co@microsoft.com> Cc: Leif Lindholm <leif.lindholm@linaro.org> Cc: Yonghong Zhu <yonghong.zhu@intel.com> Cc: Liming Gao <liming.gao@intel.com> --- BaseTools/Source/Python/Trim/Trim.py | 2 ++ 1 file changed, 2 insertions(+) diff --git a/BaseTools/Source/Python/Trim/Trim.py b/BaseTools/Source/Python/Trim/Trim.py index 76944c0e25b3..b46d507b4e55 100644 --- a/BaseTools/Source/Python/Trim/Trim.py +++ b/BaseTools/Source/Python/Trim/Trim.py @@ -166,6 +166,8 @@ def TrimPreprocessedFile(Source, Target, ConvertHex, TrimLong): if len(MatchList) == 2: LineNumber = int(MatchList[0], 0) InjectedFile = MatchList[1] + InjectedFile = os.path.normpath(InjectedFile) + InjectedFile = os.path.normcase(InjectedFile) # The first injetcted file must be the preprocessed file itself if PreprocessedFile == "": PreprocessedFile = InjectedFile -- 2.16.2.gvfs.1.33.gf5370f1 ^ permalink raw reply related [flat|nested] 3+ messages in thread
* Re: [PATCH v2 1/1] BaseTools/Trim: Normalize filepaths to fix comparisons on Windows 2018-06-28 19:31 ` [PATCH v2 1/1] " Chris Co @ 2018-07-03 5:04 ` Gao, Liming 0 siblings, 0 replies; 3+ messages in thread From: Gao, Liming @ 2018-07-03 5:04 UTC (permalink / raw) To: Chris Co, edk2-devel@lists.01.org Reviewed-by: Liming Gao <liming.gao@intel.com> If no other comment, I will help push this change. >-----Original Message----- >From: Chris Co [mailto:Christopher.Co@microsoft.com] >Sent: Friday, June 29, 2018 3:31 AM >To: edk2-devel@lists.01.org >Cc: Leif Lindholm <leif.lindholm@linaro.org>; Zhu, Yonghong ><yonghong.zhu@intel.com>; Gao, Liming <liming.gao@intel.com> >Subject: [PATCH v2 1/1] BaseTools/Trim: Normalize filepaths to fix >comparisons on Windows > >When using Linaro GCC5+ arm-eabi toolchain on Windows, the generated >DSDT.iii contains a canonicalized ("\.\" removed and lower case) >filepath for the preprocessed DSDT.i file in the first line. >Trim.exe is called on DSDT.iii to generate DSDT.iiii, which does a >line for line comparison of filepaths encountered to the preprocessed >DSDT.i filepath found in the first line to determine what lines to >place in DSDT.iiii. Since the DSDT.i filepath is canonicalized and >all later filepaths in DSDT.iii are not canonicalized, all comparisons >fail and the result is in an empty DSDT.iiii. > >Issue was first reported to Linaro here: >https://bugs.linaro.org/show_bug.cgi?id=2909 >where the recommendation was to address the issue in Trim.exe. > >This patch normalizes the case and pathname of all filepaths >encountered during Trim.exe execution on preprocessed files. This >fixes comparisons of filepaths that contain mismatching case on >case-insensitive filesystems, redundant separators, and uplevel >references. > >Contributed-under: TianoCore Contribution Agreement 1.1 >Signed-off-by: Christopher Co <christopher.co@microsoft.com> >Cc: Leif Lindholm <leif.lindholm@linaro.org> >Cc: Yonghong Zhu <yonghong.zhu@intel.com> >Cc: Liming Gao <liming.gao@intel.com> >--- > BaseTools/Source/Python/Trim/Trim.py | 2 ++ > 1 file changed, 2 insertions(+) > >diff --git a/BaseTools/Source/Python/Trim/Trim.py >b/BaseTools/Source/Python/Trim/Trim.py >index 76944c0e25b3..b46d507b4e55 100644 >--- a/BaseTools/Source/Python/Trim/Trim.py >+++ b/BaseTools/Source/Python/Trim/Trim.py >@@ -166,6 +166,8 @@ def TrimPreprocessedFile(Source, Target, ConvertHex, >TrimLong): > if len(MatchList) == 2: > LineNumber = int(MatchList[0], 0) > InjectedFile = MatchList[1] >+ InjectedFile = os.path.normpath(InjectedFile) >+ InjectedFile = os.path.normcase(InjectedFile) > # The first injetcted file must be the preprocessed file itself > if PreprocessedFile == "": > PreprocessedFile = InjectedFile >-- >2.16.2.gvfs.1.33.gf5370f1 ^ permalink raw reply [flat|nested] 3+ messages in thread
end of thread, other threads:[~2018-07-03 5:04 UTC | newest] Thread overview: 3+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2018-06-28 19:31 [PATCH v2 0/1] BaseTools/Trim: Normalize filepaths to fix comparisons on Windows Chris Co 2018-06-28 19:31 ` [PATCH v2 1/1] " Chris Co 2018-07-03 5:04 ` Gao, Liming
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox