* [Patch 1/3] BaseTools: Improve GetDependencyList function
@ 2019-09-10 9:16 Bob Feng
2019-09-20 5:39 ` Liming Gao
0 siblings, 1 reply; 2+ messages in thread
From: Bob Feng @ 2019-09-10 9:16 UTC (permalink / raw)
To: devel; +Cc: Liming Gao, Bob Feng
REF: https://bugzilla.tianocore.org/show_bug.cgi?id=2102
GetDependencyList get the header file via
re.findall in the whole header file.
This patch is to pre-process the header file and
to feed the shorter string to re.findall.
This patch is to improve GetDependencyList() efficiency
Cc: Liming Gao <liming.gao@intel.com>
Signed-off-by: Bob Feng <bob.c.feng@intel.com>
---
BaseTools/Source/Python/AutoGen/GenMake.py | 29 ++++++++++++----------
1 file changed, 16 insertions(+), 13 deletions(-)
diff --git a/BaseTools/Source/Python/AutoGen/GenMake.py b/BaseTools/Source/Python/AutoGen/GenMake.py
index 47dae82e1aeb..5d02d9a05694 100755
--- a/BaseTools/Source/Python/AutoGen/GenMake.py
+++ b/BaseTools/Source/Python/AutoGen/GenMake.py
@@ -1690,26 +1690,29 @@ def GetDependencyList(AutoGenObject, FileCache, File, ForceList, SearchPathList)
CurrentFileDependencyList = []
if F in DepDb:
CurrentFileDependencyList = DepDb[F]
else:
try:
- Fd = open(F.Path, 'rb')
- FileContent = Fd.read()
- Fd.close()
+ with open(F.Path, 'rb') as Fd:
+ FileContent = Fd.read(1)
+ Fd.seek(0)
+ if not FileContent:
+ continue
+ if FileContent[0] == 0xff or FileContent[0] == 0xfe:
+ FileContent2 = Fd.read()
+ FileContent2 = FileContent2.decode('utf-16')
+ IncludedFileList = gIncludePattern.findall(FileContent2)
+ else:
+ FileLines = Fd.readlines()
+ FileContent2 = [line for line in FileLines if str(line).lstrip("#\t ")[:8] == "include "]
+ simpleFileContent="".join(FileContent2)
+
+ IncludedFileList = gIncludePattern.findall(simpleFileContent)
except BaseException as X:
EdkLogger.error("build", FILE_OPEN_FAILURE, ExtraData=F.Path + "\n\t" + str(X))
- if len(FileContent) == 0:
+ if not FileContent:
continue
- try:
- if FileContent[0] == 0xff or FileContent[0] == 0xfe:
- FileContent = FileContent.decode('utf-16')
- else:
- FileContent = FileContent.decode()
- except:
- # The file is not txt file. for example .mcb file
- continue
- IncludedFileList = gIncludePattern.findall(FileContent)
for Inc in IncludedFileList:
Inc = Inc.strip()
# if there's macro used to reference header file, expand it
HeaderList = gMacroPattern.findall(Inc)
--
2.20.1.windows.1
^ permalink raw reply related [flat|nested] 2+ messages in thread* Re: [Patch 1/3] BaseTools: Improve GetDependencyList function
2019-09-10 9:16 [Patch 1/3] BaseTools: Improve GetDependencyList function Bob Feng
@ 2019-09-20 5:39 ` Liming Gao
0 siblings, 0 replies; 2+ messages in thread
From: Liming Gao @ 2019-09-20 5:39 UTC (permalink / raw)
To: Feng, Bob C, devel@edk2.groups.io
Bob:
I find this patch will cause the dependent header files not be listed in module Makefile.
Then, it will cause the incremental build issue. I don't think this is a good fix.
I will send the patch to revert this change.
Thanks
Liming
> -----Original Message-----
> From: Feng, Bob C
> Sent: Tuesday, September 10, 2019 5:17 PM
> To: devel@edk2.groups.io
> Cc: Gao, Liming <liming.gao@intel.com>; Feng, Bob C <bob.c.feng@intel.com>
> Subject: [Patch 1/3] BaseTools: Improve GetDependencyList function
>
> REF: https://bugzilla.tianocore.org/show_bug.cgi?id=2102
>
> GetDependencyList get the header file via
> re.findall in the whole header file.
>
> This patch is to pre-process the header file and
> to feed the shorter string to re.findall.
>
> This patch is to improve GetDependencyList() efficiency
>
> Cc: Liming Gao <liming.gao@intel.com>
> Signed-off-by: Bob Feng <bob.c.feng@intel.com>
> ---
> BaseTools/Source/Python/AutoGen/GenMake.py | 29 ++++++++++++----------
> 1 file changed, 16 insertions(+), 13 deletions(-)
>
> diff --git a/BaseTools/Source/Python/AutoGen/GenMake.py b/BaseTools/Source/Python/AutoGen/GenMake.py
> index 47dae82e1aeb..5d02d9a05694 100755
> --- a/BaseTools/Source/Python/AutoGen/GenMake.py
> +++ b/BaseTools/Source/Python/AutoGen/GenMake.py
> @@ -1690,26 +1690,29 @@ def GetDependencyList(AutoGenObject, FileCache, File, ForceList, SearchPathList)
> CurrentFileDependencyList = []
> if F in DepDb:
> CurrentFileDependencyList = DepDb[F]
> else:
> try:
> - Fd = open(F.Path, 'rb')
> - FileContent = Fd.read()
> - Fd.close()
> + with open(F.Path, 'rb') as Fd:
> + FileContent = Fd.read(1)
> + Fd.seek(0)
> + if not FileContent:
> + continue
> + if FileContent[0] == 0xff or FileContent[0] == 0xfe:
> + FileContent2 = Fd.read()
> + FileContent2 = FileContent2.decode('utf-16')
> + IncludedFileList = gIncludePattern.findall(FileContent2)
> + else:
> + FileLines = Fd.readlines()
> + FileContent2 = [line for line in FileLines if str(line).lstrip("#\t ")[:8] == "include "]
> + simpleFileContent="".join(FileContent2)
> +
> + IncludedFileList = gIncludePattern.findall(simpleFileContent)
> except BaseException as X:
> EdkLogger.error("build", FILE_OPEN_FAILURE, ExtraData=F.Path + "\n\t" + str(X))
> - if len(FileContent) == 0:
> + if not FileContent:
> continue
> - try:
> - if FileContent[0] == 0xff or FileContent[0] == 0xfe:
> - FileContent = FileContent.decode('utf-16')
> - else:
> - FileContent = FileContent.decode()
> - except:
> - # The file is not txt file. for example .mcb file
> - continue
> - IncludedFileList = gIncludePattern.findall(FileContent)
>
> for Inc in IncludedFileList:
> Inc = Inc.strip()
> # if there's macro used to reference header file, expand it
> HeaderList = gMacroPattern.findall(Inc)
> --
> 2.20.1.windows.1
^ permalink raw reply [flat|nested] 2+ messages in thread
end of thread, other threads:[~2019-09-20 5:39 UTC | newest]
Thread overview: 2+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2019-09-10 9:16 [Patch 1/3] BaseTools: Improve GetDependencyList function Bob Feng
2019-09-20 5:39 ` Liming Gao
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox