* File in the build question. I'm working on a smart git grep
@ 2020-07-16 1:41 Andrew Fish
2020-07-16 13:18 ` [edk2-devel] " Laszlo Ersek
0 siblings, 1 reply; 7+ messages in thread
From: Andrew Fish @ 2020-07-16 1:41 UTC (permalink / raw)
To: edk2-devel-groups-io
[-- Attachment #1: Type: text/plain, Size: 8796 bytes --]
I think I mentioned before on the list that I think a smart git grep that only scopes to a give target would be useful. For our internal repo we have lots of targets and lots of generations of platforms so lots of code that hits in a git grep that is not in the platform.
In thinking about this it looks to me like the Build/**/AutoGenTimeStamp (module build files) files and the Build/**/AutoGen (global build files) contain a list of all the files for a project.
I wrote some python that takes the build target (OvmfX64 for example) parses all the AutoGenTimeStamp and AutoGen files merges then together and removes the workspace path. All the other arguments are passed to `git grep` and only lines that match the file set are displayed.
My questions are:
1) Did I extract all the files?
2) Are people interested in this feature?
3) Any ideas on how to hook it in? I just dropped in the root of the git tree for development.
4) Any other cool ideas of what we could do with a file list of the project?
Example (to give people a feel for how it works):
$ time ./ggrep.py OvmfX64 CC_FLAGS | wc -l
42
real 0m0.318s
user 0m0.243s
sys 0m0.576s
$ time git grep CC_FLAGS | wc -l
479
real 0m0.110s
user 0m0.148s
sys 0m0.525s
$ ./ggrep.py OvmfX64 CC_FLAGS
OvmfX64 only: git grep CC_FLAGS
CryptoPkg/Library/BaseCryptLib/BaseCryptLib.inf:96: MSFT:*_*_*_CC_FLAGS = /wd4090
CryptoPkg/Library/BaseCryptLib/BaseCryptLib.inf:100: RVCT:*_*_ARM_CC_FLAGS = -JCryptoPkg/Include --diag_remark=1
CryptoPkg/Library/BaseCryptLib/BaseCryptLib.inf:102: GCC:*_CLANG35_*_CC_FLAGS = -std=c99
CryptoPkg/Library/BaseCryptLib/BaseCryptLib.inf:103: GCC:*_CLANG38_*_CC_FLAGS = -std=c99
CryptoPkg/Library/BaseCryptLib/BaseCryptLib.inf:104: GCC:*_CLANGPDB_*_CC_FLAGS = -std=c99 -Wno-error=incompatible-pointer-types
CryptoPkg/Library/BaseCryptLib/BaseCryptLib.inf:106: XCODE:*_*_*_CC_FLAGS = -std=c99
CryptoPkg/Library/IntrinsicLib/IntrinsicLib.inf:55: MSFT:DEBUG_VS2003_IA32_CC_FLAGS == /nologo /c /WX /W4 /Gs32768 /Gy /D UNICODE /O1b2 /FIAutoGen.h /EHs-c- /GR- /GF /GX- /Zi /Gm
CryptoPkg/Library/IntrinsicLib/IntrinsicLib.inf:56: MSFT:RELEASE_VS2003_IA32_CC_FLAGS == /nologo /c /WX /W4 /Gs32768 /Gy /D UNICODE /O1b2 /FIAutoGen.h /EHs-c- /GR- /GF /GX-
CryptoPkg/Library/IntrinsicLib/IntrinsicLib.inf:57: MSFT:DEBUG_VS2003xASL_IA32_CC_FLAGS == /nologo /c /WX /W4 /Gs32768 /Gy /D UNICODE /O1b2 /FIAutoGen.h /EHs-c- /GR- /GF /GX- /Zi /Gm
CryptoPkg/Library/IntrinsicLib/IntrinsicLib.inf:58: MSFT:RELEASE_VS2003xASL_IA32_CC_FLAGS == /nologo /c /WX /W4 /Gs32768 /Gy /D UNICODE /O1b2 /FIAutoGen.h /EHs-c- /GR- /GF /GX-
CryptoPkg/Library/IntrinsicLib/IntrinsicLib.inf:59: MSFT:DEBUG_DDK3790_IA32_CC_FLAGS == /nologo /c /WX /Gy /Gs32768 /D UNICODE /O1b2 /FIAutoGen.h /EHs-c- /GR- /GF /Zi /Gm
CryptoPkg/Library/IntrinsicLib/IntrinsicLib.inf:60: MSFT:RELEASE_DDK3790_IA32_CC_FLAGS == /nologo /c /WX /Gy /Gs32768 /D UNICODE /O1b2 /FIAutoGen.h /EHs-c- /GR- /GF
CryptoPkg/Library/IntrinsicLib/IntrinsicLib.inf:61: MSFT:DEBUG_DDK3790xASL_IA32_CC_FLAGS == /nologo /c /WX /Gy /Gs32768 /D UNICODE /O1b2 /FIAutoGen.h /EHs-c- /GR- /GF /Zi /Gm
CryptoPkg/Library/IntrinsicLib/IntrinsicLib.inf:62: MSFT:RELEASE_DDK3790xASL_IA32_CC_FLAGS == /nologo /c /WX /Gy /Gs32768 /D UNICODE /O1b2 /FIAutoGen.h /EHs-c- /GR- /GF
CryptoPkg/Library/IntrinsicLib/IntrinsicLib.inf:63: MSFT:DEBUG_*_IA32_CC_FLAGS == /nologo /c /WX /GS- /W4 /Gs32768 /D UNICODE /O1b2 /FIAutoGen.h /EHs-c- /GR- /GF /Gy /Zi /Gm
CryptoPkg/Library/IntrinsicLib/IntrinsicLib.inf:64: MSFT:RELEASE_*_IA32_CC_FLAGS == /nologo /c /WX /GS- /W4 /Gs32768 /D UNICODE /O1b2 /FIAutoGen.h /EHs-c- /GR- /GF
CryptoPkg/Library/IntrinsicLib/IntrinsicLib.inf:65: MSFT:DEBUG_*_X64_CC_FLAGS == /nologo /c /WX /GS- /X /W4 /Gs32768 /D UNICODE /O1b2s /Gy /FIAutoGen.h /EHs-c- /GR- /GF /Zi /Gm
CryptoPkg/Library/IntrinsicLib/IntrinsicLib.inf:66: MSFT:RELEASE_*_X64_CC_FLAGS == /nologo /c /WX /GS- /X /W4 /Gs32768 /D UNICODE /O1b2s /Gy /FIAutoGen.h /EHs-c- /GR- /GF
CryptoPkg/Library/IntrinsicLib/IntrinsicLib.inf:67: INTEL:*_*_*_CC_FLAGS = /Oi-
CryptoPkg/Library/OpensslLib/OpensslLibCrypto.inf:569: MSFT:*_*_IA32_CC_FLAGS = -U_WIN32 -U_WIN64 -U_MSC_VER $(OPENSSL_FLAGS) /wd4090 /wd4132 /wd4244 /wd4245 /wd4267 /wd4310 /wd4389 /wd4700 /wd4702 /wd4706 /wd4819
CryptoPkg/Library/OpensslLib/OpensslLibCrypto.inf:570: MSFT:*_*_X64_CC_FLAGS = -U_WIN32 -U_WIN64 -U_MSC_VER $(OPENSSL_FLAGS) /wd4090 /wd4132 /wd4244 /wd4245 /wd4267 /wd4306 /wd4310 /wd4700 /wd4389 /wd4702 /wd4706 /wd4819
CryptoPkg/Library/OpensslLib/OpensslLibCrypto.inf:572: INTEL:*_*_IA32_CC_FLAGS = -U_WIN32 -U_WIN64 -U_MSC_VER -U__ICC $(OPENSSL_FLAGS) /w
CryptoPkg/Library/OpensslLib/OpensslLibCrypto.inf:573: INTEL:*_*_X64_CC_FLAGS = -U_WIN32 -U_WIN64 -U_MSC_VER -U__ICC $(OPENSSL_FLAGS) /w
CryptoPkg/Library/OpensslLib/OpensslLibCrypto.inf:582: GCC:*_*_IA32_CC_FLAGS = -U_WIN32 -U_WIN64 $(OPENSSL_FLAGS) -Wno-error=maybe-uninitialized -Wno-error=unused-but-set-variable
CryptoPkg/Library/OpensslLib/OpensslLibCrypto.inf:583: GCC:*_*_X64_CC_FLAGS = -U_WIN32 -U_WIN64 $(OPENSSL_FLAGS) -Wno-error=maybe-uninitialized -Wno-error=format -Wno-format -Wno-error=unused-but-set-variable -DNO_MSABI_VA_FUNCS
CryptoPkg/Library/OpensslLib/OpensslLibCrypto.inf:584: GCC:*_*_ARM_CC_FLAGS = $(OPENSSL_FLAGS) -Wno-error=maybe-uninitialized -Wno-error=unused-but-set-variable
CryptoPkg/Library/OpensslLib/OpensslLibCrypto.inf:585: GCC:*_*_AARCH64_CC_FLAGS = $(OPENSSL_FLAGS) -Wno-error=maybe-uninitialized -Wno-format -Wno-error=unused-but-set-variable
CryptoPkg/Library/OpensslLib/OpensslLibCrypto.inf:586: GCC:*_*_RISCV64_CC_FLAGS = $(OPENSSL_FLAGS) -Wno-error=maybe-uninitialized -Wno-format -Wno-error=unused-but-set-variable
CryptoPkg/Library/OpensslLib/OpensslLibCrypto.inf:587: GCC:*_CLANG35_*_CC_FLAGS = -std=c99 -Wno-error=uninitialized
CryptoPkg/Library/OpensslLib/OpensslLibCrypto.inf:588: GCC:*_CLANG38_*_CC_FLAGS = -std=c99 -Wno-error=uninitialized
CryptoPkg/Library/OpensslLib/OpensslLibCrypto.inf:589: GCC:*_CLANGPDB_*_CC_FLAGS = -std=c99 -Wno-error=uninitialized -Wno-error=incompatible-pointer-types -Wno-error=pointer-sign -Wno-error=implicit-function-declaration -Wno-error=ignored-pragma-optimize
CryptoPkg/Library/OpensslLib/OpensslLibCrypto.inf:610: RVCT:*_*_ARM_CC_FLAGS = $(OPENSSL_FLAGS) --library_interface=aeabi_clib99 --diag_suppress=1296,1295,550,1293,111,68,177,223,144,513,188,128,546,1,3017 -JCryptoPkg/Include
CryptoPkg/Library/OpensslLib/OpensslLibCrypto.inf:611: XCODE:*_*_IA32_CC_FLAGS = -mmmx -msse -U_WIN32 -U_WIN64 $(OPENSSL_FLAGS) -w -std=c99 -Wno-error=uninitialized
CryptoPkg/Library/OpensslLib/OpensslLibCrypto.inf:612: XCODE:*_*_X64_CC_FLAGS = -mmmx -msse -U_WIN32 -U_WIN64 $(OPENSSL_FLAGS) -w -std=c99 -Wno-error=uninitialized
OvmfPkg/OvmfPkgX64.dsc:73: GCC:RELEASE_*_*_CC_FLAGS = -DMDEPKG_NDEBUG
OvmfPkg/OvmfPkgX64.dsc:74: INTEL:RELEASE_*_*_CC_FLAGS = /D MDEPKG_NDEBUG
OvmfPkg/OvmfPkgX64.dsc:75: MSFT:RELEASE_*_*_CC_FLAGS = /D MDEPKG_NDEBUG
OvmfPkg/OvmfPkgX64.dsc:77: GCC:*_*_*_CC_FLAGS = -mno-mmx -mno-sse
OvmfPkg/OvmfPkgX64.dsc:88: MSFT:*_*_*_CC_FLAGS = /D DISABLE_NEW_DEPRECATED_INTERFACES
OvmfPkg/OvmfPkgX64.dsc:89: INTEL:*_*_*_CC_FLAGS = /D DISABLE_NEW_DEPRECATED_INTERFACES
OvmfPkg/OvmfPkgX64.dsc:90: GCC:*_*_*_CC_FLAGS = -D DISABLE_NEW_DEPRECATED_INTERFACES
$./ggrep.py OvmfX64 CC_FLAG -- *.dsc
OvmfX64 only: git grep CC_FLAG -- *.dsc
OvmfPkg/OvmfPkgX64.dsc:73: GCC:RELEASE_*_*_CC_FLAGS = -DMDEPKG_NDEBUG
OvmfPkg/OvmfPkgX64.dsc:74: INTEL:RELEASE_*_*_CC_FLAGS = /D MDEPKG_NDEBUG
OvmfPkg/OvmfPkgX64.dsc:75: MSFT:RELEASE_*_*_CC_FLAGS = /D MDEPKG_NDEBUG
OvmfPkg/OvmfPkgX64.dsc:77: GCC:*_*_*_CC_FLAGS = -mno-mmx -mno-sse
OvmfPkg/OvmfPkgX64.dsc:88: MSFT:*_*_*_CC_FLAGS = /D DISABLE_NEW_DEPRECATED_INTERFACES
OvmfPkg/OvmfPkgX64.dsc:89: INTEL:*_*_*_CC_FLAGS = /D DISABLE_NEW_DEPRECATED_INTERFACES
OvmfPkg/OvmfPkgX64.dsc:90: GCC:*_*_*_CC_FLAGS = -D DISABLE_NEW_DEPRECATED_INTERFACES
$
Note if we cache the combined AutoGenTimeStamp and AutoGen files into a single file in the Build direction and use that file on the next run the time goes down to:
real 0m0.177s
user 0m0.185s
sys 0m0.522s
If you don’t pass any `git grep` commands you get the file list back:
$./ggrep.py OvmfX64
Conf/build_rule.txt
Conf/tools_def.txt
CryptoPkg/CryptoPkg.dec
CryptoPkg/Library/BaseCryptLib/BaseCryptLib.inf
CryptoPkg/Library/BaseCryptLib/Cipher/CryptAes.c
…
My prototype is around 135 lines of Python.
Thanks,
Andrew Fish
[-- Attachment #2: Type: text/html, Size: 27282 bytes --]
^ permalink raw reply [flat|nested] 7+ messages in thread* Re: [edk2-devel] File in the build question. I'm working on a smart git grep 2020-07-16 1:41 File in the build question. I'm working on a smart git grep Andrew Fish @ 2020-07-16 13:18 ` Laszlo Ersek 2020-07-16 15:50 ` Andrew Fish [not found] ` <1622466A6176DB37.5128@groups.io> 0 siblings, 2 replies; 7+ messages in thread From: Laszlo Ersek @ 2020-07-16 13:18 UTC (permalink / raw) To: devel, afish On 07/16/20 03:41, Andrew Fish via groups.io wrote: > I think I mentioned before on the list that I think a smart git grep that only scopes to a give target would be useful. For our internal repo we have lots of targets and lots of generations of platforms so lots of code that hits in a git grep that is not in the platform. > > In thinking about this it looks to me like the Build/**/AutoGenTimeStamp (module build files) files and the Build/**/AutoGen (global build files) contain a list of all the files for a project. > > I wrote some python that takes the build target (OvmfX64 for example) parses all the AutoGenTimeStamp and AutoGen files merges then together and removes the workspace path. All the other arguments are passed to `git grep` and only lines that match the file set are displayed. > > My questions are: > 1) Did I extract all the files? I don't know enough of BaseTools to answer the question :) > 2) Are people interested in this feature? It seems useful. :) > 3) Any ideas on how to hook it in? I just dropped in the root of the git tree for development. We usually place such utilities under BaseTools/Scripts/, I think (e.g. PatchCheck.py, SetupGit.py, GetMaintainer.py). > 4) Any other cool ideas of what we could do with a file list of the project? Perhaps introduce a more focused utility that only produces the file list (on stdout) so people could pass the list to whatever commands they desire? Thanks! Laszlo > > Example (to give people a feel for how it works): > $ time ./ggrep.py OvmfX64 CC_FLAGS | wc -l > 42 > > real 0m0.318s > user 0m0.243s > sys 0m0.576s > $ time git grep CC_FLAGS | wc -l > 479 > > real 0m0.110s > user 0m0.148s > sys 0m0.525s > $ ./ggrep.py OvmfX64 CC_FLAGS > OvmfX64 only: git grep CC_FLAGS > CryptoPkg/Library/BaseCryptLib/BaseCryptLib.inf:96: MSFT:*_*_*_CC_FLAGS = /wd4090 > CryptoPkg/Library/BaseCryptLib/BaseCryptLib.inf:100: RVCT:*_*_ARM_CC_FLAGS = -JCryptoPkg/Include --diag_remark=1 > CryptoPkg/Library/BaseCryptLib/BaseCryptLib.inf:102: GCC:*_CLANG35_*_CC_FLAGS = -std=c99 > CryptoPkg/Library/BaseCryptLib/BaseCryptLib.inf:103: GCC:*_CLANG38_*_CC_FLAGS = -std=c99 > CryptoPkg/Library/BaseCryptLib/BaseCryptLib.inf:104: GCC:*_CLANGPDB_*_CC_FLAGS = -std=c99 -Wno-error=incompatible-pointer-types > CryptoPkg/Library/BaseCryptLib/BaseCryptLib.inf:106: XCODE:*_*_*_CC_FLAGS = -std=c99 > CryptoPkg/Library/IntrinsicLib/IntrinsicLib.inf:55: MSFT:DEBUG_VS2003_IA32_CC_FLAGS == /nologo /c /WX /W4 /Gs32768 /Gy /D UNICODE /O1b2 /FIAutoGen.h /EHs-c- /GR- /GF /GX- /Zi /Gm > CryptoPkg/Library/IntrinsicLib/IntrinsicLib.inf:56: MSFT:RELEASE_VS2003_IA32_CC_FLAGS == /nologo /c /WX /W4 /Gs32768 /Gy /D UNICODE /O1b2 /FIAutoGen.h /EHs-c- /GR- /GF /GX- > CryptoPkg/Library/IntrinsicLib/IntrinsicLib.inf:57: MSFT:DEBUG_VS2003xASL_IA32_CC_FLAGS == /nologo /c /WX /W4 /Gs32768 /Gy /D UNICODE /O1b2 /FIAutoGen.h /EHs-c- /GR- /GF /GX- /Zi /Gm > CryptoPkg/Library/IntrinsicLib/IntrinsicLib.inf:58: MSFT:RELEASE_VS2003xASL_IA32_CC_FLAGS == /nologo /c /WX /W4 /Gs32768 /Gy /D UNICODE /O1b2 /FIAutoGen.h /EHs-c- /GR- /GF /GX- > CryptoPkg/Library/IntrinsicLib/IntrinsicLib.inf:59: MSFT:DEBUG_DDK3790_IA32_CC_FLAGS == /nologo /c /WX /Gy /Gs32768 /D UNICODE /O1b2 /FIAutoGen.h /EHs-c- /GR- /GF /Zi /Gm > CryptoPkg/Library/IntrinsicLib/IntrinsicLib.inf:60: MSFT:RELEASE_DDK3790_IA32_CC_FLAGS == /nologo /c /WX /Gy /Gs32768 /D UNICODE /O1b2 /FIAutoGen.h /EHs-c- /GR- /GF > CryptoPkg/Library/IntrinsicLib/IntrinsicLib.inf:61: MSFT:DEBUG_DDK3790xASL_IA32_CC_FLAGS == /nologo /c /WX /Gy /Gs32768 /D UNICODE /O1b2 /FIAutoGen.h /EHs-c- /GR- /GF /Zi /Gm > CryptoPkg/Library/IntrinsicLib/IntrinsicLib.inf:62: MSFT:RELEASE_DDK3790xASL_IA32_CC_FLAGS == /nologo /c /WX /Gy /Gs32768 /D UNICODE /O1b2 /FIAutoGen.h /EHs-c- /GR- /GF > CryptoPkg/Library/IntrinsicLib/IntrinsicLib.inf:63: MSFT:DEBUG_*_IA32_CC_FLAGS == /nologo /c /WX /GS- /W4 /Gs32768 /D UNICODE /O1b2 /FIAutoGen.h /EHs-c- /GR- /GF /Gy /Zi /Gm > CryptoPkg/Library/IntrinsicLib/IntrinsicLib.inf:64: MSFT:RELEASE_*_IA32_CC_FLAGS == /nologo /c /WX /GS- /W4 /Gs32768 /D UNICODE /O1b2 /FIAutoGen.h /EHs-c- /GR- /GF > CryptoPkg/Library/IntrinsicLib/IntrinsicLib.inf:65: MSFT:DEBUG_*_X64_CC_FLAGS == /nologo /c /WX /GS- /X /W4 /Gs32768 /D UNICODE /O1b2s /Gy /FIAutoGen.h /EHs-c- /GR- /GF /Zi /Gm > CryptoPkg/Library/IntrinsicLib/IntrinsicLib.inf:66: MSFT:RELEASE_*_X64_CC_FLAGS == /nologo /c /WX /GS- /X /W4 /Gs32768 /D UNICODE /O1b2s /Gy /FIAutoGen.h /EHs-c- /GR- /GF > CryptoPkg/Library/IntrinsicLib/IntrinsicLib.inf:67: INTEL:*_*_*_CC_FLAGS = /Oi- > CryptoPkg/Library/OpensslLib/OpensslLibCrypto.inf:569: MSFT:*_*_IA32_CC_FLAGS = -U_WIN32 -U_WIN64 -U_MSC_VER $(OPENSSL_FLAGS) /wd4090 /wd4132 /wd4244 /wd4245 /wd4267 /wd4310 /wd4389 /wd4700 /wd4702 /wd4706 /wd4819 > CryptoPkg/Library/OpensslLib/OpensslLibCrypto.inf:570: MSFT:*_*_X64_CC_FLAGS = -U_WIN32 -U_WIN64 -U_MSC_VER $(OPENSSL_FLAGS) /wd4090 /wd4132 /wd4244 /wd4245 /wd4267 /wd4306 /wd4310 /wd4700 /wd4389 /wd4702 /wd4706 /wd4819 > CryptoPkg/Library/OpensslLib/OpensslLibCrypto.inf:572: INTEL:*_*_IA32_CC_FLAGS = -U_WIN32 -U_WIN64 -U_MSC_VER -U__ICC $(OPENSSL_FLAGS) /w > CryptoPkg/Library/OpensslLib/OpensslLibCrypto.inf:573: INTEL:*_*_X64_CC_FLAGS = -U_WIN32 -U_WIN64 -U_MSC_VER -U__ICC $(OPENSSL_FLAGS) /w > CryptoPkg/Library/OpensslLib/OpensslLibCrypto.inf:582: GCC:*_*_IA32_CC_FLAGS = -U_WIN32 -U_WIN64 $(OPENSSL_FLAGS) -Wno-error=maybe-uninitialized -Wno-error=unused-but-set-variable > CryptoPkg/Library/OpensslLib/OpensslLibCrypto.inf:583: GCC:*_*_X64_CC_FLAGS = -U_WIN32 -U_WIN64 $(OPENSSL_FLAGS) -Wno-error=maybe-uninitialized -Wno-error=format -Wno-format -Wno-error=unused-but-set-variable -DNO_MSABI_VA_FUNCS > CryptoPkg/Library/OpensslLib/OpensslLibCrypto.inf:584: GCC:*_*_ARM_CC_FLAGS = $(OPENSSL_FLAGS) -Wno-error=maybe-uninitialized -Wno-error=unused-but-set-variable > CryptoPkg/Library/OpensslLib/OpensslLibCrypto.inf:585: GCC:*_*_AARCH64_CC_FLAGS = $(OPENSSL_FLAGS) -Wno-error=maybe-uninitialized -Wno-format -Wno-error=unused-but-set-variable > CryptoPkg/Library/OpensslLib/OpensslLibCrypto.inf:586: GCC:*_*_RISCV64_CC_FLAGS = $(OPENSSL_FLAGS) -Wno-error=maybe-uninitialized -Wno-format -Wno-error=unused-but-set-variable > CryptoPkg/Library/OpensslLib/OpensslLibCrypto.inf:587: GCC:*_CLANG35_*_CC_FLAGS = -std=c99 -Wno-error=uninitialized > CryptoPkg/Library/OpensslLib/OpensslLibCrypto.inf:588: GCC:*_CLANG38_*_CC_FLAGS = -std=c99 -Wno-error=uninitialized > CryptoPkg/Library/OpensslLib/OpensslLibCrypto.inf:589: GCC:*_CLANGPDB_*_CC_FLAGS = -std=c99 -Wno-error=uninitialized -Wno-error=incompatible-pointer-types -Wno-error=pointer-sign -Wno-error=implicit-function-declaration -Wno-error=ignored-pragma-optimize > CryptoPkg/Library/OpensslLib/OpensslLibCrypto.inf:610: RVCT:*_*_ARM_CC_FLAGS = $(OPENSSL_FLAGS) --library_interface=aeabi_clib99 --diag_suppress=1296,1295,550,1293,111,68,177,223,144,513,188,128,546,1,3017 -JCryptoPkg/Include > CryptoPkg/Library/OpensslLib/OpensslLibCrypto.inf:611: XCODE:*_*_IA32_CC_FLAGS = -mmmx -msse -U_WIN32 -U_WIN64 $(OPENSSL_FLAGS) -w -std=c99 -Wno-error=uninitialized > CryptoPkg/Library/OpensslLib/OpensslLibCrypto.inf:612: XCODE:*_*_X64_CC_FLAGS = -mmmx -msse -U_WIN32 -U_WIN64 $(OPENSSL_FLAGS) -w -std=c99 -Wno-error=uninitialized > OvmfPkg/OvmfPkgX64.dsc:73: GCC:RELEASE_*_*_CC_FLAGS = -DMDEPKG_NDEBUG > OvmfPkg/OvmfPkgX64.dsc:74: INTEL:RELEASE_*_*_CC_FLAGS = /D MDEPKG_NDEBUG > OvmfPkg/OvmfPkgX64.dsc:75: MSFT:RELEASE_*_*_CC_FLAGS = /D MDEPKG_NDEBUG > OvmfPkg/OvmfPkgX64.dsc:77: GCC:*_*_*_CC_FLAGS = -mno-mmx -mno-sse > OvmfPkg/OvmfPkgX64.dsc:88: MSFT:*_*_*_CC_FLAGS = /D DISABLE_NEW_DEPRECATED_INTERFACES > OvmfPkg/OvmfPkgX64.dsc:89: INTEL:*_*_*_CC_FLAGS = /D DISABLE_NEW_DEPRECATED_INTERFACES > OvmfPkg/OvmfPkgX64.dsc:90: GCC:*_*_*_CC_FLAGS = -D DISABLE_NEW_DEPRECATED_INTERFACES > $./ggrep.py OvmfX64 CC_FLAG -- *.dsc > OvmfX64 only: git grep CC_FLAG -- *.dsc > OvmfPkg/OvmfPkgX64.dsc:73: GCC:RELEASE_*_*_CC_FLAGS = -DMDEPKG_NDEBUG > OvmfPkg/OvmfPkgX64.dsc:74: INTEL:RELEASE_*_*_CC_FLAGS = /D MDEPKG_NDEBUG > OvmfPkg/OvmfPkgX64.dsc:75: MSFT:RELEASE_*_*_CC_FLAGS = /D MDEPKG_NDEBUG > OvmfPkg/OvmfPkgX64.dsc:77: GCC:*_*_*_CC_FLAGS = -mno-mmx -mno-sse > OvmfPkg/OvmfPkgX64.dsc:88: MSFT:*_*_*_CC_FLAGS = /D DISABLE_NEW_DEPRECATED_INTERFACES > OvmfPkg/OvmfPkgX64.dsc:89: INTEL:*_*_*_CC_FLAGS = /D DISABLE_NEW_DEPRECATED_INTERFACES > OvmfPkg/OvmfPkgX64.dsc:90: GCC:*_*_*_CC_FLAGS = -D DISABLE_NEW_DEPRECATED_INTERFACES > $ > > Note if we cache the combined AutoGenTimeStamp and AutoGen files into a single file in the Build direction and use that file on the next run the time goes down to: > real 0m0.177s > user 0m0.185s > sys 0m0.522s > > If you don’t pass any `git grep` commands you get the file list back: > $./ggrep.py OvmfX64 > Conf/build_rule.txt > Conf/tools_def.txt > CryptoPkg/CryptoPkg.dec > CryptoPkg/Library/BaseCryptLib/BaseCryptLib.inf > CryptoPkg/Library/BaseCryptLib/Cipher/CryptAes.c > … > > > My prototype is around 135 lines of Python. > > Thanks, > > Andrew Fish > > > > > ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [edk2-devel] File in the build question. I'm working on a smart git grep 2020-07-16 13:18 ` [edk2-devel] " Laszlo Ersek @ 2020-07-16 15:50 ` Andrew Fish [not found] ` <1622466A6176DB37.5128@groups.io> 1 sibling, 0 replies; 7+ messages in thread From: Andrew Fish @ 2020-07-16 15:50 UTC (permalink / raw) To: Laszlo Ersek; +Cc: devel [-- Attachment #1: Type: text/plain, Size: 10967 bytes --] > On Jul 16, 2020, at 6:18 AM, Laszlo Ersek <lersek@redhat.com> wrote: > > On 07/16/20 03:41, Andrew Fish via groups.io <http://groups.io/> wrote: >> I think I mentioned before on the list that I think a smart git grep that only scopes to a give target would be useful. For our internal repo we have lots of targets and lots of generations of platforms so lots of code that hits in a git grep that is not in the platform. >> >> In thinking about this it looks to me like the Build/**/AutoGenTimeStamp (module build files) files and the Build/**/AutoGen (global build files) contain a list of all the files for a project. >> >> I wrote some python that takes the build target (OvmfX64 for example) parses all the AutoGenTimeStamp and AutoGen files merges then together and removes the workspace path. All the other arguments are passed to `git grep` and only lines that match the file set are displayed. >> >> My questions are: >> 1) Did I extract all the files? > > I don't know enough of BaseTools to answer the question :) > >> 2) Are people interested in this feature? > > It seems useful. :) > >> 3) Any ideas on how to hook it in? I just dropped in the root of the git tree for development. > > We usually place such utilities under BaseTools/Scripts/, I think (e.g. > PatchCheck.py, SetupGit.py, GetMaintainer.py). I figured out if I rename my script to git-pgrep and move it to BaseTools/BinWrappers/PosixLike it becomes a new git command. $ git pgrep OvmfX64 CC_FLAGS -- *.dsc OvmfX64 only: git grep CC_FLAGS -- *.dsc OvmfPkg/OvmfPkgX64.dsc:73: GCC:RELEASE_*_*_CC_FLAGS = -DMDEPKG_NDEBUG OvmfPkg/OvmfPkgX64.dsc:74: INTEL:RELEASE_*_*_CC_FLAGS = /D MDEPKG_NDEBUG OvmfPkg/OvmfPkgX64.dsc:75: MSFT:RELEASE_*_*_CC_FLAGS = /D MDEPKG_NDEBUG OvmfPkg/OvmfPkgX64.dsc:77: GCC:*_*_*_CC_FLAGS = -mno-mmx -mno-sse OvmfPkg/OvmfPkgX64.dsc:88: MSFT:*_*_*_CC_FLAGS = /D DISABLE_NEW_DEPRECATED_INTERFACES OvmfPkg/OvmfPkgX64.dsc:89: INTEL:*_*_*_CC_FLAGS = /D DISABLE_NEW_DEPRECATED_INTERFACES OvmfPkg/OvmfPkgX64.dsc:90: GCC:*_*_*_CC_FLAGS = -D DISABLE_NEW_DEPRECATED_INTERFACES I picked pgrep for platform grep. I guess we could add $EDK_TOOLS_PATH/Scripts to the $PATH and then custom git commands could get added in $EDK_TOOLS_PATH/Scripts > >> 4) Any other cool ideas of what we could do with a file list of the project? > > Perhaps introduce a more focused utility that only produces the file > list (on stdout) so people could pass the list to whatever commands they > desire? > I added that feature. If you don’t pass any git-grep arguments you get the file list for the passed in platform. $ git pgrep OvmfX64 Conf/build_rule.txt Conf/tools_def.txt CryptoPkg/CryptoPkg.dec CryptoPkg/Library/BaseCryptLib/BaseCryptLib.inf … Thanks, Andrew Fish > Thanks! > Laszlo > >> >> Example (to give people a feel for how it works): >> $ time ./ggrep.py OvmfX64 CC_FLAGS | wc -l >> 42 >> >> real 0m0.318s >> user 0m0.243s >> sys 0m0.576s >> $ time git grep CC_FLAGS | wc -l >> 479 >> >> real 0m0.110s >> user 0m0.148s >> sys 0m0.525s >> $ ./ggrep.py OvmfX64 CC_FLAGS >> OvmfX64 only: git grep CC_FLAGS >> CryptoPkg/Library/BaseCryptLib/BaseCryptLib.inf:96: MSFT:*_*_*_CC_FLAGS = /wd4090 >> CryptoPkg/Library/BaseCryptLib/BaseCryptLib.inf:100: RVCT:*_*_ARM_CC_FLAGS = -JCryptoPkg/Include --diag_remark=1 >> CryptoPkg/Library/BaseCryptLib/BaseCryptLib.inf:102: GCC:*_CLANG35_*_CC_FLAGS = -std=c99 >> CryptoPkg/Library/BaseCryptLib/BaseCryptLib.inf:103: GCC:*_CLANG38_*_CC_FLAGS = -std=c99 >> CryptoPkg/Library/BaseCryptLib/BaseCryptLib.inf:104: GCC:*_CLANGPDB_*_CC_FLAGS = -std=c99 -Wno-error=incompatible-pointer-types >> CryptoPkg/Library/BaseCryptLib/BaseCryptLib.inf:106: XCODE:*_*_*_CC_FLAGS = -std=c99 >> CryptoPkg/Library/IntrinsicLib/IntrinsicLib.inf:55: MSFT:DEBUG_VS2003_IA32_CC_FLAGS == /nologo /c /WX /W4 /Gs32768 /Gy /D UNICODE /O1b2 /FIAutoGen.h /EHs-c- /GR- /GF /GX- /Zi /Gm >> CryptoPkg/Library/IntrinsicLib/IntrinsicLib.inf:56: MSFT:RELEASE_VS2003_IA32_CC_FLAGS == /nologo /c /WX /W4 /Gs32768 /Gy /D UNICODE /O1b2 /FIAutoGen.h /EHs-c- /GR- /GF /GX- >> CryptoPkg/Library/IntrinsicLib/IntrinsicLib.inf:57: MSFT:DEBUG_VS2003xASL_IA32_CC_FLAGS == /nologo /c /WX /W4 /Gs32768 /Gy /D UNICODE /O1b2 /FIAutoGen.h /EHs-c- /GR- /GF /GX- /Zi /Gm >> CryptoPkg/Library/IntrinsicLib/IntrinsicLib.inf:58: MSFT:RELEASE_VS2003xASL_IA32_CC_FLAGS == /nologo /c /WX /W4 /Gs32768 /Gy /D UNICODE /O1b2 /FIAutoGen.h /EHs-c- /GR- /GF /GX- >> CryptoPkg/Library/IntrinsicLib/IntrinsicLib.inf:59: MSFT:DEBUG_DDK3790_IA32_CC_FLAGS == /nologo /c /WX /Gy /Gs32768 /D UNICODE /O1b2 /FIAutoGen.h /EHs-c- /GR- /GF /Zi /Gm >> CryptoPkg/Library/IntrinsicLib/IntrinsicLib.inf:60: MSFT:RELEASE_DDK3790_IA32_CC_FLAGS == /nologo /c /WX /Gy /Gs32768 /D UNICODE /O1b2 /FIAutoGen.h /EHs-c- /GR- /GF >> CryptoPkg/Library/IntrinsicLib/IntrinsicLib.inf:61: MSFT:DEBUG_DDK3790xASL_IA32_CC_FLAGS == /nologo /c /WX /Gy /Gs32768 /D UNICODE /O1b2 /FIAutoGen.h /EHs-c- /GR- /GF /Zi /Gm >> CryptoPkg/Library/IntrinsicLib/IntrinsicLib.inf:62: MSFT:RELEASE_DDK3790xASL_IA32_CC_FLAGS == /nologo /c /WX /Gy /Gs32768 /D UNICODE /O1b2 /FIAutoGen.h /EHs-c- /GR- /GF >> CryptoPkg/Library/IntrinsicLib/IntrinsicLib.inf:63: MSFT:DEBUG_*_IA32_CC_FLAGS == /nologo /c /WX /GS- /W4 /Gs32768 /D UNICODE /O1b2 /FIAutoGen.h /EHs-c- /GR- /GF /Gy /Zi /Gm >> CryptoPkg/Library/IntrinsicLib/IntrinsicLib.inf:64: MSFT:RELEASE_*_IA32_CC_FLAGS == /nologo /c /WX /GS- /W4 /Gs32768 /D UNICODE /O1b2 /FIAutoGen.h /EHs-c- /GR- /GF >> CryptoPkg/Library/IntrinsicLib/IntrinsicLib.inf:65: MSFT:DEBUG_*_X64_CC_FLAGS == /nologo /c /WX /GS- /X /W4 /Gs32768 /D UNICODE /O1b2s /Gy /FIAutoGen.h /EHs-c- /GR- /GF /Zi /Gm >> CryptoPkg/Library/IntrinsicLib/IntrinsicLib.inf:66: MSFT:RELEASE_*_X64_CC_FLAGS == /nologo /c /WX /GS- /X /W4 /Gs32768 /D UNICODE /O1b2s /Gy /FIAutoGen.h /EHs-c- /GR- /GF >> CryptoPkg/Library/IntrinsicLib/IntrinsicLib.inf:67: INTEL:*_*_*_CC_FLAGS = /Oi- >> CryptoPkg/Library/OpensslLib/OpensslLibCrypto.inf:569: MSFT:*_*_IA32_CC_FLAGS = -U_WIN32 -U_WIN64 -U_MSC_VER $(OPENSSL_FLAGS) /wd4090 /wd4132 /wd4244 /wd4245 /wd4267 /wd4310 /wd4389 /wd4700 /wd4702 /wd4706 /wd4819 >> CryptoPkg/Library/OpensslLib/OpensslLibCrypto.inf:570: MSFT:*_*_X64_CC_FLAGS = -U_WIN32 -U_WIN64 -U_MSC_VER $(OPENSSL_FLAGS) /wd4090 /wd4132 /wd4244 /wd4245 /wd4267 /wd4306 /wd4310 /wd4700 /wd4389 /wd4702 /wd4706 /wd4819 >> CryptoPkg/Library/OpensslLib/OpensslLibCrypto.inf:572: INTEL:*_*_IA32_CC_FLAGS = -U_WIN32 -U_WIN64 -U_MSC_VER -U__ICC $(OPENSSL_FLAGS) /w >> CryptoPkg/Library/OpensslLib/OpensslLibCrypto.inf:573: INTEL:*_*_X64_CC_FLAGS = -U_WIN32 -U_WIN64 -U_MSC_VER -U__ICC $(OPENSSL_FLAGS) /w >> CryptoPkg/Library/OpensslLib/OpensslLibCrypto.inf:582: GCC:*_*_IA32_CC_FLAGS = -U_WIN32 -U_WIN64 $(OPENSSL_FLAGS) -Wno-error=maybe-uninitialized -Wno-error=unused-but-set-variable >> CryptoPkg/Library/OpensslLib/OpensslLibCrypto.inf:583: GCC:*_*_X64_CC_FLAGS = -U_WIN32 -U_WIN64 $(OPENSSL_FLAGS) -Wno-error=maybe-uninitialized -Wno-error=format -Wno-format -Wno-error=unused-but-set-variable -DNO_MSABI_VA_FUNCS >> CryptoPkg/Library/OpensslLib/OpensslLibCrypto.inf:584: GCC:*_*_ARM_CC_FLAGS = $(OPENSSL_FLAGS) -Wno-error=maybe-uninitialized -Wno-error=unused-but-set-variable >> CryptoPkg/Library/OpensslLib/OpensslLibCrypto.inf:585: GCC:*_*_AARCH64_CC_FLAGS = $(OPENSSL_FLAGS) -Wno-error=maybe-uninitialized -Wno-format -Wno-error=unused-but-set-variable >> CryptoPkg/Library/OpensslLib/OpensslLibCrypto.inf:586: GCC:*_*_RISCV64_CC_FLAGS = $(OPENSSL_FLAGS) -Wno-error=maybe-uninitialized -Wno-format -Wno-error=unused-but-set-variable >> CryptoPkg/Library/OpensslLib/OpensslLibCrypto.inf:587: GCC:*_CLANG35_*_CC_FLAGS = -std=c99 -Wno-error=uninitialized >> CryptoPkg/Library/OpensslLib/OpensslLibCrypto.inf:588: GCC:*_CLANG38_*_CC_FLAGS = -std=c99 -Wno-error=uninitialized >> CryptoPkg/Library/OpensslLib/OpensslLibCrypto.inf:589: GCC:*_CLANGPDB_*_CC_FLAGS = -std=c99 -Wno-error=uninitialized -Wno-error=incompatible-pointer-types -Wno-error=pointer-sign -Wno-error=implicit-function-declaration -Wno-error=ignored-pragma-optimize >> CryptoPkg/Library/OpensslLib/OpensslLibCrypto.inf:610: RVCT:*_*_ARM_CC_FLAGS = $(OPENSSL_FLAGS) --library_interface=aeabi_clib99 --diag_suppress=1296,1295,550,1293,111,68,177,223,144,513,188,128,546,1,3017 -JCryptoPkg/Include >> CryptoPkg/Library/OpensslLib/OpensslLibCrypto.inf:611: XCODE:*_*_IA32_CC_FLAGS = -mmmx -msse -U_WIN32 -U_WIN64 $(OPENSSL_FLAGS) -w -std=c99 -Wno-error=uninitialized >> CryptoPkg/Library/OpensslLib/OpensslLibCrypto.inf:612: XCODE:*_*_X64_CC_FLAGS = -mmmx -msse -U_WIN32 -U_WIN64 $(OPENSSL_FLAGS) -w -std=c99 -Wno-error=uninitialized >> OvmfPkg/OvmfPkgX64.dsc:73: GCC:RELEASE_*_*_CC_FLAGS = -DMDEPKG_NDEBUG >> OvmfPkg/OvmfPkgX64.dsc:74: INTEL:RELEASE_*_*_CC_FLAGS = /D MDEPKG_NDEBUG >> OvmfPkg/OvmfPkgX64.dsc:75: MSFT:RELEASE_*_*_CC_FLAGS = /D MDEPKG_NDEBUG >> OvmfPkg/OvmfPkgX64.dsc:77: GCC:*_*_*_CC_FLAGS = -mno-mmx -mno-sse >> OvmfPkg/OvmfPkgX64.dsc:88: MSFT:*_*_*_CC_FLAGS = /D DISABLE_NEW_DEPRECATED_INTERFACES >> OvmfPkg/OvmfPkgX64.dsc:89: INTEL:*_*_*_CC_FLAGS = /D DISABLE_NEW_DEPRECATED_INTERFACES >> OvmfPkg/OvmfPkgX64.dsc:90: GCC:*_*_*_CC_FLAGS = -D DISABLE_NEW_DEPRECATED_INTERFACES >> $./ggrep.py OvmfX64 CC_FLAG -- *.dsc >> OvmfX64 only: git grep CC_FLAG -- *.dsc >> OvmfPkg/OvmfPkgX64.dsc:73: GCC:RELEASE_*_*_CC_FLAGS = -DMDEPKG_NDEBUG >> OvmfPkg/OvmfPkgX64.dsc:74: INTEL:RELEASE_*_*_CC_FLAGS = /D MDEPKG_NDEBUG >> OvmfPkg/OvmfPkgX64.dsc:75: MSFT:RELEASE_*_*_CC_FLAGS = /D MDEPKG_NDEBUG >> OvmfPkg/OvmfPkgX64.dsc:77: GCC:*_*_*_CC_FLAGS = -mno-mmx -mno-sse >> OvmfPkg/OvmfPkgX64.dsc:88: MSFT:*_*_*_CC_FLAGS = /D DISABLE_NEW_DEPRECATED_INTERFACES >> OvmfPkg/OvmfPkgX64.dsc:89: INTEL:*_*_*_CC_FLAGS = /D DISABLE_NEW_DEPRECATED_INTERFACES >> OvmfPkg/OvmfPkgX64.dsc:90: GCC:*_*_*_CC_FLAGS = -D DISABLE_NEW_DEPRECATED_INTERFACES >> $ >> >> Note if we cache the combined AutoGenTimeStamp and AutoGen files into a single file in the Build direction and use that file on the next run the time goes down to: >> real 0m0.177s >> user 0m0.185s >> sys 0m0.522s >> >> If you don’t pass any `git grep` commands you get the file list back: >> $./ggrep.py OvmfX64 >> Conf/build_rule.txt >> Conf/tools_def.txt >> CryptoPkg/CryptoPkg.dec >> CryptoPkg/Library/BaseCryptLib/BaseCryptLib.inf >> CryptoPkg/Library/BaseCryptLib/Cipher/CryptAes.c >> … >> >> >> My prototype is around 135 lines of Python. >> >> Thanks, >> >> Andrew Fish >> >> >> [-- Attachment #2: Type: text/html, Size: 30956 bytes --] ^ permalink raw reply [flat|nested] 7+ messages in thread
[parent not found: <1622466A6176DB37.5128@groups.io>]
* Re: [edk2-devel] File in the build question. I'm working on a smart git grep [not found] ` <1622466A6176DB37.5128@groups.io> @ 2020-07-18 20:23 ` Andrew Fish 2020-07-20 19:55 ` Laszlo Ersek 0 siblings, 1 reply; 7+ messages in thread From: Andrew Fish @ 2020-07-18 20:23 UTC (permalink / raw) To: edk2-devel-groups-io, Andrew Fish; +Cc: Laszlo Ersek [-- Attachment #1: Type: text/plain, Size: 11812 bytes --] Wanted to see if people are OK with adding BaseTools/Scripts/ to the path when you run `. edksetuip.sh` or edksetup.bat? If we do that and I add BaseTools/Scripts/git-pgrep then it will show up as an extended git command. I was thinking over time we could add git-edk2* git commands to help automate the process or maintainer workflow. If people are OK with adding the path I can make a patch set to contribute git-pgrep, but I need some help with how to add the new path to edksetup.bat. Thanks, Andrew Fish > On Jul 16, 2020, at 8:50 AM, Andrew Fish via groups.io <afish=apple.com@groups.io> wrote: > > > >> On Jul 16, 2020, at 6:18 AM, Laszlo Ersek <lersek@redhat.com <mailto:lersek@redhat.com>> wrote: >> >> On 07/16/20 03:41, Andrew Fish via groups.io <http://groups.io/> wrote: >>> I think I mentioned before on the list that I think a smart git grep that only scopes to a give target would be useful. For our internal repo we have lots of targets and lots of generations of platforms so lots of code that hits in a git grep that is not in the platform. >>> >>> In thinking about this it looks to me like the Build/**/AutoGenTimeStamp (module build files) files and the Build/**/AutoGen (global build files) contain a list of all the files for a project. >>> >>> I wrote some python that takes the build target (OvmfX64 for example) parses all the AutoGenTimeStamp and AutoGen files merges then together and removes the workspace path. All the other arguments are passed to `git grep` and only lines that match the file set are displayed. >>> >>> My questions are: >>> 1) Did I extract all the files? >> >> I don't know enough of BaseTools to answer the question :) >> >>> 2) Are people interested in this feature? >> >> It seems useful. :) >> >>> 3) Any ideas on how to hook it in? I just dropped in the root of the git tree for development. >> >> We usually place such utilities under BaseTools/Scripts/, I think (e.g. >> PatchCheck.py, SetupGit.py, GetMaintainer.py). > > I figured out if I rename my script to git-pgrep and move it to BaseTools/BinWrappers/PosixLike it becomes a new git command. > > $ git pgrep OvmfX64 CC_FLAGS -- *.dsc > OvmfX64 only: git grep CC_FLAGS -- *.dsc > OvmfPkg/OvmfPkgX64.dsc:73: GCC:RELEASE_*_*_CC_FLAGS = -DMDEPKG_NDEBUG > OvmfPkg/OvmfPkgX64.dsc:74: INTEL:RELEASE_*_*_CC_FLAGS = /D MDEPKG_NDEBUG > OvmfPkg/OvmfPkgX64.dsc:75: MSFT:RELEASE_*_*_CC_FLAGS = /D MDEPKG_NDEBUG > OvmfPkg/OvmfPkgX64.dsc:77: GCC:*_*_*_CC_FLAGS = -mno-mmx -mno-sse > OvmfPkg/OvmfPkgX64.dsc:88: MSFT:*_*_*_CC_FLAGS = /D DISABLE_NEW_DEPRECATED_INTERFACES > OvmfPkg/OvmfPkgX64.dsc:89: INTEL:*_*_*_CC_FLAGS = /D DISABLE_NEW_DEPRECATED_INTERFACES > OvmfPkg/OvmfPkgX64.dsc:90: GCC:*_*_*_CC_FLAGS = -D DISABLE_NEW_DEPRECATED_INTERFACES > > I picked pgrep for platform grep. > > I guess we could add $EDK_TOOLS_PATH/Scripts to the $PATH and then custom git commands could get added in $EDK_TOOLS_PATH/Scripts > >> >>> 4) Any other cool ideas of what we could do with a file list of the project? >> >> Perhaps introduce a more focused utility that only produces the file >> list (on stdout) so people could pass the list to whatever commands they >> desire? >> > > I added that feature. If you don’t pass any git-grep arguments you get the file list for the passed in platform. > > $ git pgrep OvmfX64 > Conf/build_rule.txt > Conf/tools_def.txt > CryptoPkg/CryptoPkg.dec > CryptoPkg/Library/BaseCryptLib/BaseCryptLib.inf > … > > Thanks, > > Andrew Fish > >> Thanks! >> Laszlo >> >>> >>> Example (to give people a feel for how it works): >>> $ time ./ggrep.py OvmfX64 CC_FLAGS | wc -l >>> 42 >>> >>> real 0m0.318s >>> user 0m0.243s >>> sys 0m0.576s >>> $ time git grep CC_FLAGS | wc -l >>> 479 >>> >>> real 0m0.110s >>> user 0m0.148s >>> sys 0m0.525s >>> $ ./ggrep.py OvmfX64 CC_FLAGS >>> OvmfX64 only: git grep CC_FLAGS >>> CryptoPkg/Library/BaseCryptLib/BaseCryptLib.inf:96: MSFT:*_*_*_CC_FLAGS = /wd4090 >>> CryptoPkg/Library/BaseCryptLib/BaseCryptLib.inf:100: RVCT:*_*_ARM_CC_FLAGS = -JCryptoPkg/Include --diag_remark=1 >>> CryptoPkg/Library/BaseCryptLib/BaseCryptLib.inf:102: GCC:*_CLANG35_*_CC_FLAGS = -std=c99 >>> CryptoPkg/Library/BaseCryptLib/BaseCryptLib.inf:103: GCC:*_CLANG38_*_CC_FLAGS = -std=c99 >>> CryptoPkg/Library/BaseCryptLib/BaseCryptLib.inf:104: GCC:*_CLANGPDB_*_CC_FLAGS = -std=c99 -Wno-error=incompatible-pointer-types >>> CryptoPkg/Library/BaseCryptLib/BaseCryptLib.inf:106: XCODE:*_*_*_CC_FLAGS = -std=c99 >>> CryptoPkg/Library/IntrinsicLib/IntrinsicLib.inf:55: MSFT:DEBUG_VS2003_IA32_CC_FLAGS == /nologo /c /WX /W4 /Gs32768 /Gy /D UNICODE /O1b2 /FIAutoGen.h /EHs-c- /GR- /GF /GX- /Zi /Gm >>> CryptoPkg/Library/IntrinsicLib/IntrinsicLib.inf:56: MSFT:RELEASE_VS2003_IA32_CC_FLAGS == /nologo /c /WX /W4 /Gs32768 /Gy /D UNICODE /O1b2 /FIAutoGen.h /EHs-c- /GR- /GF /GX- >>> CryptoPkg/Library/IntrinsicLib/IntrinsicLib.inf:57: MSFT:DEBUG_VS2003xASL_IA32_CC_FLAGS == /nologo /c /WX /W4 /Gs32768 /Gy /D UNICODE /O1b2 /FIAutoGen.h /EHs-c- /GR- /GF /GX- /Zi /Gm >>> CryptoPkg/Library/IntrinsicLib/IntrinsicLib.inf:58: MSFT:RELEASE_VS2003xASL_IA32_CC_FLAGS == /nologo /c /WX /W4 /Gs32768 /Gy /D UNICODE /O1b2 /FIAutoGen.h /EHs-c- /GR- /GF /GX- >>> CryptoPkg/Library/IntrinsicLib/IntrinsicLib.inf:59: MSFT:DEBUG_DDK3790_IA32_CC_FLAGS == /nologo /c /WX /Gy /Gs32768 /D UNICODE /O1b2 /FIAutoGen.h /EHs-c- /GR- /GF /Zi /Gm >>> CryptoPkg/Library/IntrinsicLib/IntrinsicLib.inf:60: MSFT:RELEASE_DDK3790_IA32_CC_FLAGS == /nologo /c /WX /Gy /Gs32768 /D UNICODE /O1b2 /FIAutoGen.h /EHs-c- /GR- /GF >>> CryptoPkg/Library/IntrinsicLib/IntrinsicLib.inf:61: MSFT:DEBUG_DDK3790xASL_IA32_CC_FLAGS == /nologo /c /WX /Gy /Gs32768 /D UNICODE /O1b2 /FIAutoGen.h /EHs-c- /GR- /GF /Zi /Gm >>> CryptoPkg/Library/IntrinsicLib/IntrinsicLib.inf:62: MSFT:RELEASE_DDK3790xASL_IA32_CC_FLAGS == /nologo /c /WX /Gy /Gs32768 /D UNICODE /O1b2 /FIAutoGen.h /EHs-c- /GR- /GF >>> CryptoPkg/Library/IntrinsicLib/IntrinsicLib.inf:63: MSFT:DEBUG_*_IA32_CC_FLAGS == /nologo /c /WX /GS- /W4 /Gs32768 /D UNICODE /O1b2 /FIAutoGen.h /EHs-c- /GR- /GF /Gy /Zi /Gm >>> CryptoPkg/Library/IntrinsicLib/IntrinsicLib.inf:64: MSFT:RELEASE_*_IA32_CC_FLAGS == /nologo /c /WX /GS- /W4 /Gs32768 /D UNICODE /O1b2 /FIAutoGen.h /EHs-c- /GR- /GF >>> CryptoPkg/Library/IntrinsicLib/IntrinsicLib.inf:65: MSFT:DEBUG_*_X64_CC_FLAGS == /nologo /c /WX /GS- /X /W4 /Gs32768 /D UNICODE /O1b2s /Gy /FIAutoGen.h /EHs-c- /GR- /GF /Zi /Gm >>> CryptoPkg/Library/IntrinsicLib/IntrinsicLib.inf:66: MSFT:RELEASE_*_X64_CC_FLAGS == /nologo /c /WX /GS- /X /W4 /Gs32768 /D UNICODE /O1b2s /Gy /FIAutoGen.h /EHs-c- /GR- /GF >>> CryptoPkg/Library/IntrinsicLib/IntrinsicLib.inf:67: INTEL:*_*_*_CC_FLAGS = /Oi- >>> CryptoPkg/Library/OpensslLib/OpensslLibCrypto.inf:569: MSFT:*_*_IA32_CC_FLAGS = -U_WIN32 -U_WIN64 -U_MSC_VER $(OPENSSL_FLAGS) /wd4090 /wd4132 /wd4244 /wd4245 /wd4267 /wd4310 /wd4389 /wd4700 /wd4702 /wd4706 /wd4819 >>> CryptoPkg/Library/OpensslLib/OpensslLibCrypto.inf:570: MSFT:*_*_X64_CC_FLAGS = -U_WIN32 -U_WIN64 -U_MSC_VER $(OPENSSL_FLAGS) /wd4090 /wd4132 /wd4244 /wd4245 /wd4267 /wd4306 /wd4310 /wd4700 /wd4389 /wd4702 /wd4706 /wd4819 >>> CryptoPkg/Library/OpensslLib/OpensslLibCrypto.inf:572: INTEL:*_*_IA32_CC_FLAGS = -U_WIN32 -U_WIN64 -U_MSC_VER -U__ICC $(OPENSSL_FLAGS) /w >>> CryptoPkg/Library/OpensslLib/OpensslLibCrypto.inf:573: INTEL:*_*_X64_CC_FLAGS = -U_WIN32 -U_WIN64 -U_MSC_VER -U__ICC $(OPENSSL_FLAGS) /w >>> CryptoPkg/Library/OpensslLib/OpensslLibCrypto.inf:582: GCC:*_*_IA32_CC_FLAGS = -U_WIN32 -U_WIN64 $(OPENSSL_FLAGS) -Wno-error=maybe-uninitialized -Wno-error=unused-but-set-variable >>> CryptoPkg/Library/OpensslLib/OpensslLibCrypto.inf:583: GCC:*_*_X64_CC_FLAGS = -U_WIN32 -U_WIN64 $(OPENSSL_FLAGS) -Wno-error=maybe-uninitialized -Wno-error=format -Wno-format -Wno-error=unused-but-set-variable -DNO_MSABI_VA_FUNCS >>> CryptoPkg/Library/OpensslLib/OpensslLibCrypto.inf:584: GCC:*_*_ARM_CC_FLAGS = $(OPENSSL_FLAGS) -Wno-error=maybe-uninitialized -Wno-error=unused-but-set-variable >>> CryptoPkg/Library/OpensslLib/OpensslLibCrypto.inf:585: GCC:*_*_AARCH64_CC_FLAGS = $(OPENSSL_FLAGS) -Wno-error=maybe-uninitialized -Wno-format -Wno-error=unused-but-set-variable >>> CryptoPkg/Library/OpensslLib/OpensslLibCrypto.inf:586: GCC:*_*_RISCV64_CC_FLAGS = $(OPENSSL_FLAGS) -Wno-error=maybe-uninitialized -Wno-format -Wno-error=unused-but-set-variable >>> CryptoPkg/Library/OpensslLib/OpensslLibCrypto.inf:587: GCC:*_CLANG35_*_CC_FLAGS = -std=c99 -Wno-error=uninitialized >>> CryptoPkg/Library/OpensslLib/OpensslLibCrypto.inf:588: GCC:*_CLANG38_*_CC_FLAGS = -std=c99 -Wno-error=uninitialized >>> CryptoPkg/Library/OpensslLib/OpensslLibCrypto.inf:589: GCC:*_CLANGPDB_*_CC_FLAGS = -std=c99 -Wno-error=uninitialized -Wno-error=incompatible-pointer-types -Wno-error=pointer-sign -Wno-error=implicit-function-declaration -Wno-error=ignored-pragma-optimize >>> CryptoPkg/Library/OpensslLib/OpensslLibCrypto.inf:610: RVCT:*_*_ARM_CC_FLAGS = $(OPENSSL_FLAGS) --library_interface=aeabi_clib99 --diag_suppress=1296,1295,550,1293,111,68,177,223,144,513,188,128,546,1,3017 -JCryptoPkg/Include >>> CryptoPkg/Library/OpensslLib/OpensslLibCrypto.inf:611: XCODE:*_*_IA32_CC_FLAGS = -mmmx -msse -U_WIN32 -U_WIN64 $(OPENSSL_FLAGS) -w -std=c99 -Wno-error=uninitialized >>> CryptoPkg/Library/OpensslLib/OpensslLibCrypto.inf:612: XCODE:*_*_X64_CC_FLAGS = -mmmx -msse -U_WIN32 -U_WIN64 $(OPENSSL_FLAGS) -w -std=c99 -Wno-error=uninitialized >>> OvmfPkg/OvmfPkgX64.dsc:73: GCC:RELEASE_*_*_CC_FLAGS = -DMDEPKG_NDEBUG >>> OvmfPkg/OvmfPkgX64.dsc:74: INTEL:RELEASE_*_*_CC_FLAGS = /D MDEPKG_NDEBUG >>> OvmfPkg/OvmfPkgX64.dsc:75: MSFT:RELEASE_*_*_CC_FLAGS = /D MDEPKG_NDEBUG >>> OvmfPkg/OvmfPkgX64.dsc:77: GCC:*_*_*_CC_FLAGS = -mno-mmx -mno-sse >>> OvmfPkg/OvmfPkgX64.dsc:88: MSFT:*_*_*_CC_FLAGS = /D DISABLE_NEW_DEPRECATED_INTERFACES >>> OvmfPkg/OvmfPkgX64.dsc:89: INTEL:*_*_*_CC_FLAGS = /D DISABLE_NEW_DEPRECATED_INTERFACES >>> OvmfPkg/OvmfPkgX64.dsc:90: GCC:*_*_*_CC_FLAGS = -D DISABLE_NEW_DEPRECATED_INTERFACES >>> $./ggrep.py OvmfX64 CC_FLAG -- *.dsc >>> OvmfX64 only: git grep CC_FLAG -- *.dsc >>> OvmfPkg/OvmfPkgX64.dsc:73: GCC:RELEASE_*_*_CC_FLAGS = -DMDEPKG_NDEBUG >>> OvmfPkg/OvmfPkgX64.dsc:74: INTEL:RELEASE_*_*_CC_FLAGS = /D MDEPKG_NDEBUG >>> OvmfPkg/OvmfPkgX64.dsc:75: MSFT:RELEASE_*_*_CC_FLAGS = /D MDEPKG_NDEBUG >>> OvmfPkg/OvmfPkgX64.dsc:77: GCC:*_*_*_CC_FLAGS = -mno-mmx -mno-sse >>> OvmfPkg/OvmfPkgX64.dsc:88: MSFT:*_*_*_CC_FLAGS = /D DISABLE_NEW_DEPRECATED_INTERFACES >>> OvmfPkg/OvmfPkgX64.dsc:89: INTEL:*_*_*_CC_FLAGS = /D DISABLE_NEW_DEPRECATED_INTERFACES >>> OvmfPkg/OvmfPkgX64.dsc:90: GCC:*_*_*_CC_FLAGS = -D DISABLE_NEW_DEPRECATED_INTERFACES >>> $ >>> >>> Note if we cache the combined AutoGenTimeStamp and AutoGen files into a single file in the Build direction and use that file on the next run the time goes down to: >>> real 0m0.177s >>> user 0m0.185s >>> sys 0m0.522s >>> >>> If you don’t pass any `git grep` commands you get the file list back: >>> $./ggrep.py OvmfX64 >>> Conf/build_rule.txt >>> Conf/tools_def.txt >>> CryptoPkg/CryptoPkg.dec >>> CryptoPkg/Library/BaseCryptLib/BaseCryptLib.inf >>> CryptoPkg/Library/BaseCryptLib/Cipher/CryptAes.c >>> … >>> >>> >>> My prototype is around 135 lines of Python. >>> >>> Thanks, >>> >>> Andrew Fish >>> >>> > > [-- Attachment #2: Type: text/html, Size: 34207 bytes --] ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [edk2-devel] File in the build question. I'm working on a smart git grep 2020-07-18 20:23 ` Andrew Fish @ 2020-07-20 19:55 ` Laszlo Ersek 2020-07-21 1:18 ` Guomin Jiang 0 siblings, 1 reply; 7+ messages in thread From: Laszlo Ersek @ 2020-07-20 19:55 UTC (permalink / raw) To: Andrew Fish, edk2-devel-groups-io On 07/18/20 22:23, Andrew Fish wrote: > Wanted to see if people are OK with adding BaseTools/Scripts/ to the path when you run `. edksetuip.sh` or edksetup.bat? > > If we do that and I add BaseTools/Scripts/git-pgrep then it will show up as an extended git command. > > I was thinking over time we could add git-edk2* git commands to help automate the process or maintainer workflow. > > If people are OK with adding the path I can make a patch set to contribute git-pgrep, but I need some help with how to add the new path to edksetup.bat. To be honest, I think edksetup.sh should add BaseTools/Scripts/ to the PATH environment variable anyway. I find it a bit awkward to run python BaseTools/Scripts/GetMaintainer.py -l ... rather than just GetMaintainer.py -l ... Thanks Laszlo ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [edk2-devel] File in the build question. I'm working on a smart git grep 2020-07-20 19:55 ` Laszlo Ersek @ 2020-07-21 1:18 ` Guomin Jiang 2020-07-21 1:53 ` Andrew Fish 0 siblings, 1 reply; 7+ messages in thread From: Guomin Jiang @ 2020-07-21 1:18 UTC (permalink / raw) To: devel@edk2.groups.io, lersek@redhat.com, Andrew Fish I would like that add BaseTols/Scripts/ to the PATH. Usually, I use ```set PATH=%PATH%;%WORKSPACE%\BaseTools\Scripts``` to add new directory to the environment variable. But I suggest that we should judge if the last character in PATH variable is ';' character and then add new directory. Let me know what I can help you. > -----Original Message----- > From: devel@edk2.groups.io <devel@edk2.groups.io> On Behalf Of Laszlo > Ersek > Sent: Tuesday, July 21, 2020 3:55 AM > To: Andrew Fish <afish@apple.com>; edk2-devel-groups-io > <devel@edk2.groups.io> > Subject: Re: [edk2-devel] File in the build question. I'm working on a smart git > grep > > On 07/18/20 22:23, Andrew Fish wrote: > > Wanted to see if people are OK with adding BaseTools/Scripts/ to the path > when you run `. edksetuip.sh` or edksetup.bat? > > > > If we do that and I add BaseTools/Scripts/git-pgrep then it will show up as > an extended git command. > > > > I was thinking over time we could add git-edk2* git commands to help > automate the process or maintainer workflow. > > > > If people are OK with adding the path I can make a patch set to contribute > git-pgrep, but I need some help with how to add the new path to > edksetup.bat. > > To be honest, I think edksetup.sh should add BaseTools/Scripts/ to the PATH > environment variable anyway. I find it a bit awkward to run > > python BaseTools/Scripts/GetMaintainer.py -l ... > > rather than just > > GetMaintainer.py -l ... > > Thanks > Laszlo > > > ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [edk2-devel] File in the build question. I'm working on a smart git grep 2020-07-21 1:18 ` Guomin Jiang @ 2020-07-21 1:53 ` Andrew Fish 0 siblings, 0 replies; 7+ messages in thread From: Andrew Fish @ 2020-07-21 1:53 UTC (permalink / raw) To: Jiang, Guomin; +Cc: devel@edk2.groups.io, lersek@redhat.com [-- Attachment #1: Type: text/plain, Size: 2158 bytes --] > On Jul 20, 2020, at 6:18 PM, Jiang, Guomin <guomin.jiang@intel.com> wrote: > > I would like that add BaseTols/Scripts/ to the PATH. > > Usually, I use ```set PATH=%PATH%;%WORKSPACE%\BaseTools\Scripts``` to add new directory to the environment variable. > > But I suggest that we should judge if the last character in PATH variable is ';' character and then add new directory. > > Let me know what I can help you. I’m thinking I could file a bugzilla to add BaseTools/Scripts/ to the path for edksetup.sh/edksetup.bat and then I can just contribute my Python git extension to BaseTools/Scripts As Laszlo pointed out there is generic value having BaseTools/Scripts/ in the path so I filed: https://bugzilla.tianocore.org/show_bug.cgi?id=2859 Thanks, Andrew Fish > >> -----Original Message----- >> From: devel@edk2.groups.io <mailto:devel@edk2.groups.io> <devel@edk2.groups.io <mailto:devel@edk2.groups.io>> On Behalf Of Laszlo >> Ersek >> Sent: Tuesday, July 21, 2020 3:55 AM >> To: Andrew Fish <afish@apple.com <mailto:afish@apple.com>>; edk2-devel-groups-io >> <devel@edk2.groups.io <mailto:devel@edk2.groups.io>> >> Subject: Re: [edk2-devel] File in the build question. I'm working on a smart git >> grep >> >> On 07/18/20 22:23, Andrew Fish wrote: >>> Wanted to see if people are OK with adding BaseTools/Scripts/ to the path >> when you run `. edksetuip.sh` or edksetup.bat? >>> >>> If we do that and I add BaseTools/Scripts/git-pgrep then it will show up as >> an extended git command. >>> >>> I was thinking over time we could add git-edk2* git commands to help >> automate the process or maintainer workflow. >>> >>> If people are OK with adding the path I can make a patch set to contribute >> git-pgrep, but I need some help with how to add the new path to >> edksetup.bat. >> >> To be honest, I think edksetup.sh should add BaseTools/Scripts/ to the PATH >> environment variable anyway. I find it a bit awkward to run >> >> python BaseTools/Scripts/GetMaintainer.py -l ... >> >> rather than just >> >> GetMaintainer.py -l ... >> >> Thanks >> Laszlo >> >> >> [-- Attachment #2: Type: text/html, Size: 8126 bytes --] ^ permalink raw reply [flat|nested] 7+ messages in thread
end of thread, other threads:[~2020-07-21 1:53 UTC | newest]
Thread overview: 7+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2020-07-16 1:41 File in the build question. I'm working on a smart git grep Andrew Fish
2020-07-16 13:18 ` [edk2-devel] " Laszlo Ersek
2020-07-16 15:50 ` Andrew Fish
[not found] ` <1622466A6176DB37.5128@groups.io>
2020-07-18 20:23 ` Andrew Fish
2020-07-20 19:55 ` Laszlo Ersek
2020-07-21 1:18 ` Guomin Jiang
2020-07-21 1:53 ` Andrew Fish
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox