public inbox for devel@edk2.groups.io
 help / color / mirror / Atom feed
* [Patch 0/1] BaseTools: Using compiler to generate source code dependency files.
@ 2019-10-28 10:47 Bob Feng
  2019-10-28 10:47 ` [Patch 1/1] BaseTools: Generate source file dependency in Make phase Bob Feng
  2019-10-28 12:23 ` [edk2-devel] [Patch 0/1] BaseTools: Using compiler to generate source code dependency files Ryszard Knop
  0 siblings, 2 replies; 9+ messages in thread
From: Bob Feng @ 2019-10-28 10:47 UTC (permalink / raw)
  To: devel

BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=2311

To support incremental build, build tool generates the dependent header
file for each of source file. This procedure is done in AutoGen phase.
The build tool goes through all the source file and header file and
use regular expression to find out all the dependent files for a source
file. This procedure is much time-consuming. And this method can't handle
the MACRO in #include, for example #include PATH(xxx.h).

This patch is going to use compiler to generate dependent files. This method
will be faster and more accurate.

The basic idea is:
1. In AutoGen phase, build tool add "!Include deps.txt" into Makefile instead
of defining COMMON_DEPS list.
2. During the Make phase, the compilers, Trim and C preprocessor generate 
dependent files, .d file, for each source file.
3. After Make, The build tool combines the .d files and generate a file deps.txt 
which list all the included files for a module.
4. Each source file will depends on the Module's includes files. The difference
with orignial behavior is that if the user
change the source file, build tool will only build that source file in 
incremental build; while if the user change a module's header file, build tool
will build the whole module in incremental build.

In this way, the time of AutoGen phase will be reduced much. And since we
will use c preprocessor to handle #include, the MACRO will be handled well
and the final dependent files will be more accurate.

Feng, Bob C (1):
  BaseTools: Using compiler to generate source code dependency files.

 BaseTools/Conf/build_rule.template            |  89 ++++++-----
 BaseTools/Conf/tools_def.template             | 138 +++++++++---------
 BaseTools/Source/Python/AutoGen/GenMake.py    |  73 +++------
 .../Source/Python/AutoGen/IncludesAutoGen.py  |  99 +++++++++++++
 BaseTools/Source/Python/Trim/Trim.py          | 113 +++++++++++---
 BaseTools/Source/Python/build/build.py        |  58 ++++++--
 6 files changed, 378 insertions(+), 192 deletions(-)
 create mode 100644 BaseTools/Source/Python/AutoGen/IncludesAutoGen.py

-- 
2.20.1.windows.1


^ permalink raw reply	[flat|nested] 9+ messages in thread

* [Patch 1/1] BaseTools: Generate source file dependency in Make phase
  2019-10-28 10:47 [Patch 0/1] BaseTools: Using compiler to generate source code dependency files Bob Feng
@ 2019-10-28 10:47 ` Bob Feng
  2019-10-28 12:23 ` [edk2-devel] [Patch 0/1] BaseTools: Using compiler to generate source code dependency files Ryszard Knop
  1 sibling, 0 replies; 9+ messages in thread
From: Bob Feng @ 2019-10-28 10:47 UTC (permalink / raw)
  To: devel; +Cc: Liming Gao, Steven Shi, Bob Feng

BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=2311
Current method to get source file dependency is to use regular expression
to match "include" pattern in each source file during AutoGen phase,
it's slow and can't get accurate include files, for example when
there is Macro using in #include.

This patch is to use compiler, C preprocessor and Trim
to generate included file list in Make phase.

This method will reduce AutoGen phase time and generate accurate
include files.

Cc: Liming Gao <liming.gao@intel.com>
Cc: Steven Shi <steven.shi@intel.com>
Signed-off-by: Bob Feng <bob.c.feng@intel.com>
---
 BaseTools/Conf/build_rule.template            |  89 ++++++-----
 BaseTools/Conf/tools_def.template             | 138 +++++++++---------
 BaseTools/Source/Python/AutoGen/GenMake.py    |  73 +++------
 .../Source/Python/AutoGen/IncludesAutoGen.py  |  99 +++++++++++++
 BaseTools/Source/Python/Trim/Trim.py          | 113 +++++++++++---
 BaseTools/Source/Python/build/build.py        |  58 ++++++--
 6 files changed, 378 insertions(+), 192 deletions(-)
 create mode 100644 BaseTools/Source/Python/AutoGen/IncludesAutoGen.py

diff --git a/BaseTools/Conf/build_rule.template b/BaseTools/Conf/build_rule.template
index 3a58ac8015..de9144ffda 100755
--- a/BaseTools/Conf/build_rule.template
+++ b/BaseTools/Conf/build_rule.template
@@ -121,18 +121,18 @@
 
     <OutputFile>
         $(OUTPUT_DIR)(+)${s_dir}(+)${s_base}.obj
 
     <Command.MSFT, Command.INTEL>
-        "$(CC)" /Fo${dst} $(CC_FLAGS) $(INC) ${src}
+        "$(CC)" /showIncludes /Fo${dst} $(CC_FLAGS) $(INC) ${src}
 
     <Command.GCC, Command.RVCT>
         # For RVCTCYGWIN CC_FLAGS must be first to work around pathing issues
-        "$(CC)" $(CC_FLAGS) -c -o ${dst} $(INC) ${src}
+        "$(CC)" -MMD $(CC_FLAGS) -c -o ${dst} $(INC) ${src}
 
     <Command.XCODE>
-        "$(CC)" $(CC_FLAGS) -o ${dst} $(INC) ${src}
+        "$(CC)" -MMD $(CC_FLAGS) -o ${dst} $(INC) ${src}
 
 [C-Code-File.BASE.AARCH64,C-Code-File.SEC.AARCH64,C-Code-File.PEI_CORE.AARCH64,C-Code-File.PEIM.AARCH64,C-Code-File.BASE.ARM,C-Code-File.SEC.ARM,C-Code-File.PEI_CORE.ARM,C-Code-File.PEIM.ARM]
     <InputFile>
         ?.c
 
@@ -141,11 +141,11 @@
 
     <OutputFile>
         $(OUTPUT_DIR)(+)${s_dir}(+)${s_base}.obj
 
     <Command.GCC, Command.RVCT>
-        "$(CC)" $(CC_FLAGS) $(CC_XIPFLAGS) -c -o ${dst} $(INC) ${src}
+        "$(CC)" -MMD $(CC_FLAGS) $(CC_XIPFLAGS) -c -o ${dst} $(INC) ${src}
 
 [C-Header-File]
     <InputFile>
         *.h, *.H
 
@@ -165,19 +165,21 @@
 
     <OutputFile>
         $(OUTPUT_DIR)(+)${s_dir}(+)${s_base}.obj
 
     <Command.MSFT, Command.INTEL>
-        "$(PP)" $(PP_FLAGS) $(INC) ${src} > ${d_path}(+)${s_base}.i
-        Trim --source-code --convert-hex --trim-long -o ${d_path}(+)${s_base}.iii ${d_path}(+)${s_base}.i
-        "$(ASM)" /Fo${dst} $(ASM_FLAGS) /I${s_path} $(INC) ${d_path}(+)${s_base}.iii
+        Trim --asm-file -o $(OUTPUT_DIR)(+)${s_dir}(+)${s_base}.i -i $(INC_LIST) ${src}
+        "$(PP)" /showIncludes $(PP_FLAGS) $(INC) $(OUTPUT_DIR)(+)${s_dir}(+)${s_base}.i > ${d_path}(+)${s_base}.ii
+        Trim --source-code --convert-hex --trim-long -o ${d_path}(+)${s_base}.iiii ${d_path}(+)${s_base}.ii
+        "$(ASM)" /Fo${dst} $(ASM_FLAGS) /I${s_path} $(INC) ${d_path}(+)${s_base}.iiii
 
     <Command.GCC, Command.RVCT>
-        "$(PP)" $(PP_FLAGS) $(INC) ${src} > ${d_path}(+)${s_base}.i
-        Trim --trim-long --source-code -o ${d_path}(+)${s_base}.iii ${d_path}(+)${s_base}.i
+        Trim --asm-file -o $(OUTPUT_DIR)(+)${s_dir}(+)${s_base}.i -i $(INC_LIST) ${src}
+        "$(PP)" -MMD $(PP_FLAGS) $(INC) $(OUTPUT_DIR)(+)${s_dir}(+)${s_base}.i > ${d_path}(+)${s_base}.ii
+        Trim --trim-long --source-code -o ${d_path}(+)${s_base}.iiii ${d_path}(+)${s_base}.ii
         # For RVCTCYGWIN ASM_FLAGS must be first to work around pathing issues
-        "$(ASM)" $(ASM_FLAGS) -o ${dst} $(INC) ${d_path}(+)${s_base}.iii
+        "$(ASM)" $(ASM_FLAGS) -o ${dst} $(INC) ${d_path}(+)${s_base}.iiii
 
 [Assembly-Code-File.COMMON.ARM,Assembly-Code-File.COMMON.AARCH64]
     # Remove --convert-hex for ARM as it breaks MSFT assemblers
     <InputFile.MSFT, InputFile.INTEL, InputFile.RVCT>
         ?.asm, ?.Asm, ?.ASM
@@ -190,24 +192,27 @@
 
     <OutputFile>
         $(OUTPUT_DIR)(+)${s_dir}(+)${s_base}.obj
 
     <Command.INTEL>
-        "$(PP)" $(PP_FLAGS) $(INC) ${src} > ${d_path}(+)${s_base}.i
-        Trim --source-code --convert-hex --trim-long -o ${d_path}(+)${s_base}.iii ${d_path}(+)${s_base}.i
-        "$(ASM)" /Fo${dst} $(ASM_FLAGS) /I${s_path} $(INC) ${d_path}(+)${s_base}.iii
+        Trim --asm-file -o $(OUTPUT_DIR)(+)${s_dir}(+)${s_base}.i -i $(INC_LIST) ${src}
+        "$(PP)" /showIncludes $(PP_FLAGS) $(INC) $(OUTPUT_DIR)(+)${s_dir}(+)${s_base}.i > ${d_path}(+)${s_base}.ii
+        Trim --source-code --convert-hex --trim-long -o ${d_path}(+)${s_base}.iiii ${d_path}(+)${s_base}.ii
+        "$(ASM)" /Fo${dst} $(ASM_FLAGS) /I${s_path} $(INC) ${d_path}(+)${s_base}.iiii
 
     <Command.MSFT>
-        "$(PP)" $(PP_FLAGS) $(INC) ${src} > ${d_path}(+)${s_base}.i
-        Trim --source-code --trim-long -o ${d_path}(+)${s_base}.iii ${d_path}(+)${s_base}.i
-        "$(ASM)" /Fo${dst} $(ASM_FLAGS) /I${s_path} $(INC) ${d_path}(+)${s_base}.iii
+        Trim --asm-file  -o $(OUTPUT_DIR)(+)${s_dir}(+)${s_base}.i -i $(INC_LIST) ${src}
+        "$(PP)" /showIncludes $(PP_FLAGS) $(INC) $(OUTPUT_DIR)(+)${s_dir}(+)${s_base}.i > ${d_path}(+)${s_base}.ii
+        Trim --source-code --trim-long -o ${d_path}(+)${s_base}.iiii ${d_path}(+)${s_base}.ii
+        "$(ASM)" /Fo${dst} $(ASM_FLAGS) /I${s_path} $(INC) ${d_path}(+)${s_base}.iiii
 
     <Command.GCC, Command.RVCT>
-        "$(PP)" $(PP_FLAGS) $(INC) ${src} > ${d_path}(+)${s_base}.i
-        Trim --trim-long --source-code -o ${d_path}(+)${s_base}.iii ${d_path}(+)${s_base}.i
+        Trim --asm-file -o $(OUTPUT_DIR)(+)${s_dir}(+)${s_base}.i -i $(INC_LIST) ${src}
+        "$(PP)" -MMD $(PP_FLAGS) $(INC) $(OUTPUT_DIR)(+)${s_dir}(+)${s_base}.i > ${d_path}(+)${s_base}.ii
+        Trim --trim-long --source-code -o ${d_path}(+)${s_base}.iiii ${d_path}(+)${s_base}.ii
         # For RVCTCYGWIN ASM_FLAGS must be first to work around pathing issues
-        "$(ASM)" $(ASM_FLAGS) -o ${dst} $(INC) ${d_path}(+)${s_base}.iii
+        "$(ASM)" $(ASM_FLAGS) -o ${dst} $(INC) ${d_path}(+)${s_base}.iiii
 
 [Nasm-Assembly-Code-File.COMMON.COMMON]
     <InputFile>
         ?.nasm
 
@@ -215,15 +220,18 @@
         $(MAKE_FILE)
 
     <OutputFile>
         $(OUTPUT_DIR)(+)${s_dir}(+)${s_base}.obj
 
-    <Command>
-        "$(PP)" $(PP_FLAGS) $(INC) ${src} > ${d_path}(+)${s_base}.i
+    <Command.GCC, Command.RVCT, Command.XCODE>
+        "$(PP)" -MMD $(PP_FLAGS) $(INC) ${src} > ${d_path}(+)${s_base}.i
         Trim --trim-long --source-code -o ${d_path}(+)${s_base}.iii ${d_path}(+)${s_base}.i
-        "$(NASM)" -I${s_path}(+) $(NASM_INC) $(NASM_FLAGS) -o $dst ${d_path}(+)${s_base}.iii
-
+        "$(NASM)" -I${s_path}(+) $(NASM_INC) $(NASM_FLAGS) -o $dst ${d_path}(+)${s_base}.iii -MD ${d_path}(+)${s_base}.iii.d
+    <Command.MSFT, Command.INTEL>
+        "$(PP)" /showIncludes $(PP_FLAGS) $(INC) ${src} > ${d_path}(+)${s_base}.i
+        Trim --trim-long --source-code -o ${d_path}(+)${s_base}.iii ${d_path}(+)${s_base}.i
+        "$(NASM)" -I${s_path}(+) $(NASM_INC) $(NASM_FLAGS) -o $dst ${d_path}(+)${s_base}.iii -MD ${d_path}(+)${s_base}.iii.d
 [Device-Tree-Source-File]
     <InputFile>
         ?.dts
 
     <ExtraDependency>
@@ -246,12 +254,16 @@
         $(MAKE_FILE)
 
     <OutputFile>
         $(DEBUG_DIR)(+)${s_dir}(+)${s_base}.c
 
-    <Command>
-        "$(VFRPP)" $(VFRPP_FLAGS) $(INC) ${src} > $(OUTPUT_DIR)(+)${s_base}.i
+    <Command.GCC, Command.RVCT>
+        "$(VFRPP)" -MMD $(VFRPP_FLAGS) $(INC) ${src} > $(OUTPUT_DIR)(+)${s_base}.i
+        "$(VFR)" $(VFR_FLAGS) --string-db $(OUTPUT_DIR)(+)$(MODULE_NAME)StrDefs.hpk --output-directory ${d_path} $(OUTPUT_DIR)(+)${s_base}.i
+
+    <Command.MSFT, Command.INTEL>
+        "$(VFRPP)" /showIncludes $(VFRPP_FLAGS) $(INC) ${src} > $(OUTPUT_DIR)(+)${s_base}.i
         "$(VFR)" $(VFR_FLAGS) --string-db $(OUTPUT_DIR)(+)$(MODULE_NAME)StrDefs.hpk --output-directory ${d_path} $(OUTPUT_DIR)(+)${s_base}.i
 
 [Object-File]
     <InputFile>
         *.obj
@@ -413,17 +425,17 @@
 
     <ExtraDependency>
         $(MAKE_FILE)
 
     <Command.MSFT, Command.INTEL>
-        Trim --asl-file -o $(OUTPUT_DIR)(+)${s_dir}(+)${s_base}.i -i $(INC_LIST) ${src}
+        Trim --asl-file --asl-deps -o $(OUTPUT_DIR)(+)${s_dir}(+)${s_base}.i -i $(INC_LIST) ${src}
         "$(ASLPP)" $(ASLPP_FLAGS) $(INC) /I${s_path} $(OUTPUT_DIR)(+)${s_dir}(+)${s_base}.i > $(OUTPUT_DIR)(+)${s_dir}(+)${s_base}.iii
         Trim --source-code -l -o $(OUTPUT_DIR)(+)${s_dir}(+)${s_base}.iiii $(OUTPUT_DIR)(+)${s_dir}(+)${s_base}.iii 
         "$(ASL)" $(ASL_FLAGS) $(ASL_OUTFLAGS)${dst} $(OUTPUT_DIR)(+)${s_dir}(+)${s_base}.iiii
 
     <Command.GCC>
-        Trim --asl-file -o $(OUTPUT_DIR)(+)${s_dir}(+)${s_base}.i -i $(INC_LIST) ${src}
+        Trim --asl-file --asl-deps -o $(OUTPUT_DIR)(+)${s_dir}(+)${s_base}.i -i $(INC_LIST) ${src}
         "$(ASLPP)" $(ASLPP_FLAGS) $(INC) -I${s_path} $(OUTPUT_DIR)(+)${s_dir}(+)${s_base}.i > $(OUTPUT_DIR)(+)${s_dir}(+)${s_base}.iii
         Trim --source-code -l -o $(OUTPUT_DIR)(+)${s_dir}(+)${s_base}.iiii $(OUTPUT_DIR)(+)${s_dir}(+)${s_base}.iii 
         "$(ASL)" $(ASL_FLAGS) $(ASL_OUTFLAGS)${dst} $(OUTPUT_DIR)(+)${s_dir}(+)${s_base}.iiii
 
 [C-Code-File.AcpiTable]
@@ -435,16 +447,16 @@
 
     <ExtraDependency>
         $(MAKE_FILE)
 
     <Command.MSFT, Command.INTEL>
-        "$(ASLCC)" /Fo$(OUTPUT_DIR)(+)${s_dir}(+)${s_base}.obj $(ASLCC_FLAGS) $(INC) ${src}
+        "$(ASLCC)" /showIncludes /Fo$(OUTPUT_DIR)(+)${s_dir}(+)${s_base}.obj $(ASLCC_FLAGS) $(INC) ${src}
         "$(ASLDLINK)" /OUT:$(OUTPUT_DIR)(+)${s_dir}(+)${s_base}.dll $(ASLDLINK_FLAGS) $(OUTPUT_DIR)(+)${s_dir}(+)${s_base}.obj
         "$(GENFW)" -o ${dst} -c $(OUTPUT_DIR)(+)${s_dir}(+)${s_base}.dll $(GENFW_FLAGS)
 
     <Command.GCC>
-        "$(ASLCC)" -c -o $(OUTPUT_DIR)(+)${s_dir}(+)${s_base}.obj $(CC_FLAGS) $(ASLCC_FLAGS) $(INC) ${src}
+        "$(ASLCC)" -MMD -c -o $(OUTPUT_DIR)(+)${s_dir}(+)${s_base}.obj $(CC_FLAGS) $(ASLCC_FLAGS) $(INC) ${src}
         "$(ASLDLINK)" -o $(OUTPUT_DIR)(+)${s_dir}(+)${s_base}.dll $(ASLDLINK_FLAGS) $(OUTPUT_DIR)(+)${s_dir}(+)${s_base}.obj
         "$(GENFW)" -o ${dst} -c $(OUTPUT_DIR)(+)${s_dir}(+)${s_base}.dll $(GENFW_FLAGS)
 
 [Acpi-Table-Code-File]
     <InputFile>
@@ -455,26 +467,26 @@
 
     <ExtraDependency>
         $(MAKE_FILE)
 
     <Command.MSFT, Command.INTEL>
-        "$(ASLCC)" /Fo$(OUTPUT_DIR)(+)${s_dir}(+)${s_base}.obj $(ASLCC_FLAGS) $(INC) ${src}
+        "$(ASLCC)" /showIncludes /Fo$(OUTPUT_DIR)(+)${s_dir}(+)${s_base}.obj $(ASLCC_FLAGS) $(INC) ${src}
         "$(ASLDLINK)" /OUT:$(OUTPUT_DIR)(+)${s_dir}(+)${s_base}.dll $(ASLDLINK_FLAGS) $(OUTPUT_DIR)(+)${s_dir}(+)${s_base}.obj
         "$(GENFW)" -o ${dst} -c $(OUTPUT_DIR)(+)${s_dir}(+)${s_base}.dll $(GENFW_FLAGS)
 
     <Command.GCC>
-        "$(ASLCC)" -c -o $(OUTPUT_DIR)(+)${s_dir}(+)${s_base}.obj $(CC_FLAGS) $(ASLCC_FLAGS) $(INC) ${src}
+        "$(ASLCC)" -MMD -c -o $(OUTPUT_DIR)(+)${s_dir}(+)${s_base}.obj $(CC_FLAGS) $(ASLCC_FLAGS) $(INC) ${src}
         "$(ASLDLINK)" -o $(OUTPUT_DIR)(+)${s_dir}(+)${s_base}.dll $(ASLDLINK_FLAGS) $(OUTPUT_DIR)(+)${s_dir}(+)${s_base}.obj $(CC_FLAGS) $(ASLCC_FLAGS)
         "$(GENFW)" -o ${dst} -c $(OUTPUT_DIR)(+)${s_dir}(+)${s_base}.dll $(GENFW_FLAGS)
 
     <Command.CLANGPE>
         "$(ASLCC)" -c -o $(OUTPUT_DIR)(+)${s_dir}(+)${s_base}.obj $(CC_FLAGS) $(ASLCC_FLAGS) $(INC) ${src}
         "$(ASLDLINK)" /OUT:$(OUTPUT_DIR)(+)${s_dir}(+)${s_base}.dll $(ASLDLINK_FLAGS) $(OUTPUT_DIR)(+)${s_dir}(+)${s_base}.obj
         "$(GENFW)" -o ${dst} -c $(OUTPUT_DIR)(+)${s_dir}(+)${s_base}.dll $(GENFW_FLAGS)
 
     <Command.XCODE>        
-        "$(ASLCC)" -o $(OUTPUT_DIR)(+)${s_dir}(+)${s_base}.obj  $(ASLCC_FLAGS) $(INC) ${src}
+        "$(ASLCC)" -MMD -o $(OUTPUT_DIR)(+)${s_dir}(+)${s_base}.obj  $(ASLCC_FLAGS) $(INC) ${src}
         "$(ASLDLINK)" -o $(OUTPUT_DIR)(+)${s_dir}(+)${s_base}.dll $(ASLDLINK_FLAGS) $(OUTPUT_DIR)(+)${s_dir}(+)${s_base}.obj
         "$(MTOC)" -subsystem $(MODULE_TYPE)  $(MTOC_FLAGS) $(OUTPUT_DIR)(+)${s_dir}(+)${s_base}.dll $(OUTPUT_DIR)(+)${s_dir}(+)${s_base}.pecoff
         "$(GENFW)" -o ${dst} -c $(OUTPUT_DIR)(+)${s_dir}(+)${s_base}.pecoff $(GENFW_FLAGS)
       
       
@@ -517,14 +529,21 @@
         $(MAKE_FILE)
 
     <OutputFile>
         $(OUTPUT_DIR)(+)${s_base}.bin
 
-    <Command>
-        "$(PP)" $(PP_FLAGS) $(INC) ${src} > ${d_path}(+)${s_base}.i
+    <Command.GCC, Command.RVCT, Command.XCODE>
+        "$(PP)" -MMD $(PP_FLAGS) $(INC) ${src} > ${d_path}(+)${s_base}.i
+        Trim --source-code --convert-hex -o ${d_path}(+)${s_base}.iii ${d_path}(+)${s_base}.i
+        "$(NASM)" -I${s_path}(+) -l ${d_path}(+)${s_base}.lst $(NASMB_FLAGS) -o $dst ${d_path}(+)${s_base}.iii -MD ${d_path}(+)${s_base}.iii.d
+        # copy the output file with .com postfix that be same to the output file of .asm16
+        $(CP) ${dst} $(OUTPUT_DIR)(+)${s_base}.com
+
+    <Command.MSFT, Command.INTEL>
+        "$(PP)" /showIncludes $(PP_FLAGS) $(INC) ${src} > ${d_path}(+)${s_base}.i
         Trim --source-code --convert-hex -o ${d_path}(+)${s_base}.iii ${d_path}(+)${s_base}.i
-        "$(NASM)" -I${s_path}(+) -l ${d_path}(+)${s_base}.lst $(NASMB_FLAGS) -o $dst ${d_path}(+)${s_base}.iii
+        "$(NASM)" -I${s_path}(+) -l ${d_path}(+)${s_base}.lst $(NASMB_FLAGS) -o $dst ${d_path}(+)${s_base}.iii -MD ${d_path}(+)${s_base}.iii.d
         # copy the output file with .com postfix that be same to the output file of .asm16
         $(CP) ${dst} $(OUTPUT_DIR)(+)${s_base}.com
 
 [Microcode-File.USER_DEFINED, Microcode-File.Microcode]
     <InputFile>
diff --git a/BaseTools/Conf/tools_def.template b/BaseTools/Conf/tools_def.template
index dce6c5875a..28b5ccde86 100755
--- a/BaseTools/Conf/tools_def.template
+++ b/BaseTools/Conf/tools_def.template
@@ -462,13 +462,13 @@ NOOPT_VS2008_IA32_DLINK_FLAGS     = /NOLOGO /NODEFAULTLIB /IGNORE:4001 /OPT:REF
 *_VS2008_X64_DLINK_PATH    = DEF(VS2008_BINX64)\link.exe
 *_VS2008_X64_ASLCC_PATH    = DEF(VS2008_BINX64)\cl.exe
 *_VS2008_X64_ASLPP_PATH    = DEF(VS2008_BINX64)\cl.exe
 *_VS2008_X64_ASLDLINK_PATH = DEF(VS2008_BINX64)\link.exe
 
-  DEBUG_VS2008_X64_CC_FLAGS     = /nologo /c /WX /GS- /W4 /Gs32768 /D UNICODE /O1b2s /GL /Gy /FIAutoGen.h /EHs-c- /GR- /GF /Z7 /MP
-RELEASE_VS2008_X64_CC_FLAGS     = /nologo /c /WX /GS- /W4 /Gs32768 /D UNICODE /O1b2s /GL /Gy /FIAutoGen.h /EHs-c- /GR- /GF /MP
-NOOPT_VS2008_X64_CC_FLAGS       = /nologo /c /WX /GS- /W4 /Gs32768 /D UNICODE /Gy /FIAutoGen.h /EHs-c- /GR- /GF /Z7 /Od /MP
+  DEBUG_VS2008_X64_CC_FLAGS     = /nologo /c /WX /GS- /W4 /Gs32768 /D UNICODE /O1b2s /GL /Gy /FIAutoGen.h /EHs-c- /GR- /GF /Z7
+RELEASE_VS2008_X64_CC_FLAGS     = /nologo /c /WX /GS- /W4 /Gs32768 /D UNICODE /O1b2s /GL /Gy /FIAutoGen.h /EHs-c- /GR- /GF
+NOOPT_VS2008_X64_CC_FLAGS       = /nologo /c /WX /GS- /W4 /Gs32768 /D UNICODE /Gy /FIAutoGen.h /EHs-c- /GR- /GF /Z7 /Od
 
   DEBUG_VS2008_X64_ASM_FLAGS    = /nologo /c /WX /W3 /Cx /Zd /Zi
 RELEASE_VS2008_X64_ASM_FLAGS    = /nologo /c /WX /W3 /Cx /Zd
 NOOPT_VS2008_X64_ASM_FLAGS    = /nologo /c /WX /W3 /Cx /Zd /Zi
 
@@ -546,13 +546,13 @@ NOOPT_VS2008_X64_DLINK_FLAGS  = /NOLOGO /NODEFAULTLIB /IGNORE:4001 /OPT:REF /OPT
 *_VS2008x86_IA32_APP_PATH     = DEF(VS2008x86_BIN)\cl.exe
 *_VS2008x86_IA32_PP_PATH      = DEF(VS2008x86_BIN)\cl.exe
 *_VS2008x86_IA32_ASM_PATH     = DEF(VS2008x86_BIN)\ml.exe
 
       *_VS2008x86_IA32_MAKE_FLAGS  = /nologo
-  DEBUG_VS2008x86_IA32_CC_FLAGS    = /nologo /c /WX /GS- /W4 /Gs32768 /D UNICODE /O1b2 /GL /FIAutoGen.h /EHs-c- /GR- /GF /Gy /Z7 /MP
-RELEASE_VS2008x86_IA32_CC_FLAGS    = /nologo /c /WX /GS- /W4 /Gs32768 /D UNICODE /O1b2 /GL /FIAutoGen.h /EHs-c- /GR- /GF /MP
-NOOPT_VS2008x86_IA32_CC_FLAGS      = /nologo /c /WX /GS- /W4 /Gs32768 /D UNICODE /FIAutoGen.h /EHs-c- /GR- /GF /Gy /Z7 /Od /MP
+  DEBUG_VS2008x86_IA32_CC_FLAGS    = /nologo /c /WX /GS- /W4 /Gs32768 /D UNICODE /O1b2 /GL /FIAutoGen.h /EHs-c- /GR- /GF /Gy /Z7
+RELEASE_VS2008x86_IA32_CC_FLAGS    = /nologo /c /WX /GS- /W4 /Gs32768 /D UNICODE /O1b2 /GL /FIAutoGen.h /EHs-c- /GR- /GF
+NOOPT_VS2008x86_IA32_CC_FLAGS      = /nologo /c /WX /GS- /W4 /Gs32768 /D UNICODE /FIAutoGen.h /EHs-c- /GR- /GF /Gy /Z7 /Od
 
   DEBUG_VS2008x86_IA32_ASM_FLAGS   = /nologo /c /WX /W3 /Cx /coff /Zd /Zi
 RELEASE_VS2008x86_IA32_ASM_FLAGS   = /nologo /c /WX /W3 /Cx /coff /Zd
 NOOPT_VS2008x86_IA32_ASM_FLAGS     = /nologo /c /WX /W3 /Cx /coff /Zd /Zi
 
@@ -578,13 +578,13 @@ NOOPT_VS2008x86_IA32_DLINK_FLAGS   = /NOLOGO /NODEFAULTLIB /IGNORE:4001 /OPT:REF
 *_VS2008x86_X64_ASM_PATH      = DEF(VS2008x86_BINX64)\ml64.exe
 *_VS2008x86_X64_SLINK_PATH    = DEF(VS2008x86_BINX64)\lib.exe
 *_VS2008x86_X64_DLINK_PATH    = DEF(VS2008x86_BINX64)\link.exe
 *_VS2008x86_X64_ASLDLINK_PATH = DEF(VS2008x86_BINX64)\link.exe
 
-  DEBUG_VS2008x86_X64_CC_FLAGS     = /nologo /c /WX /GS- /W4 /Gs32768 /D UNICODE /O1b2s /GL /Gy /FIAutoGen.h /EHs-c- /GR- /GF /Z7 /MP
-RELEASE_VS2008x86_X64_CC_FLAGS     = /nologo /c /WX /GS- /W4 /Gs32768 /D UNICODE /O1b2s /GL /Gy /FIAutoGen.h /EHs-c- /GR- /GF /MP
-NOOPT_VS2008x86_X64_CC_FLAGS       = /nologo /c /WX /GS- /W4 /Gs32768 /D UNICODE /Gy /FIAutoGen.h /EHs-c- /GR- /GF /Z7 /Od /MP
+  DEBUG_VS2008x86_X64_CC_FLAGS     = /nologo /c /WX /GS- /W4 /Gs32768 /D UNICODE /O1b2s /GL /Gy /FIAutoGen.h /EHs-c- /GR- /GF /Z7
+RELEASE_VS2008x86_X64_CC_FLAGS     = /nologo /c /WX /GS- /W4 /Gs32768 /D UNICODE /O1b2s /GL /Gy /FIAutoGen.h /EHs-c- /GR- /GF
+NOOPT_VS2008x86_X64_CC_FLAGS       = /nologo /c /WX /GS- /W4 /Gs32768 /D UNICODE /Gy /FIAutoGen.h /EHs-c- /GR- /GF /Z7 /Od
 
   DEBUG_VS2008x86_X64_ASM_FLAGS    = /nologo /c /WX /W3 /Cx /Zd /Zi
 RELEASE_VS2008x86_X64_ASM_FLAGS    = /nologo /c /WX /W3 /Cx /Zd
 NOOPT_VS2008x86_X64_ASM_FLAGS      = /nologo /c /WX /W3 /Cx /Zd /Zi
 
@@ -665,13 +665,13 @@ NOOPT_VS2008x86_X64_DLINK_FLAGS    = /NOLOGO /NODEFAULTLIB /IGNORE:4001 /OPT:REF
 *_VS2010_IA32_ASLCC_PATH          = DEF(VS2010_BIN)\cl.exe
 *_VS2010_IA32_ASLPP_PATH          = DEF(VS2010_BIN)\cl.exe
 *_VS2010_IA32_ASLDLINK_PATH       = DEF(VS2010_BIN)\link.exe
 
       *_VS2010_IA32_MAKE_FLAGS    = /nologo
-  DEBUG_VS2010_IA32_CC_FLAGS      = /nologo /c /WX /GS- /W4 /Gs32768 /D UNICODE /O1b2 /GL /FIAutoGen.h /EHs-c- /GR- /GF /Gy /Z7 /MP
-RELEASE_VS2010_IA32_CC_FLAGS      = /nologo /c /WX /GS- /W4 /Gs32768 /D UNICODE /O1b2 /GL /FIAutoGen.h /EHs-c- /GR- /GF /MP
-NOOPT_VS2010_IA32_CC_FLAGS        = /nologo /c /WX /GS- /W4 /Gs32768 /D UNICODE /FIAutoGen.h /EHs-c- /GR- /GF /Gy /Zi /Gm /Od /MP
+  DEBUG_VS2010_IA32_CC_FLAGS      = /nologo /c /WX /GS- /W4 /Gs32768 /D UNICODE /O1b2 /GL /FIAutoGen.h /EHs-c- /GR- /GF /Gy /Z7
+RELEASE_VS2010_IA32_CC_FLAGS      = /nologo /c /WX /GS- /W4 /Gs32768 /D UNICODE /O1b2 /GL /FIAutoGen.h /EHs-c- /GR- /GF
+NOOPT_VS2010_IA32_CC_FLAGS        = /nologo /c /WX /GS- /W4 /Gs32768 /D UNICODE /FIAutoGen.h /EHs-c- /GR- /GF /Gy /Zi /Gm /Od
 
   DEBUG_VS2010_IA32_ASM_FLAGS     = /nologo /c /WX /W3 /Cx /coff /Zd /Zi
 RELEASE_VS2010_IA32_ASM_FLAGS     = /nologo /c /WX /W3 /Cx /coff /Zd
 NOOPT_VS2010_IA32_ASM_FLAGS       = /nologo /c /WX /W3 /Cx /coff /Zd /Zi
 
@@ -697,13 +697,13 @@ NOOPT_VS2010_IA32_DLINK_FLAGS     = /NOLOGO /NODEFAULTLIB /IGNORE:4001 /OPT:REF
 *_VS2010_X64_DLINK_PATH    = DEF(VS2010_BINX64)\link.exe
 *_VS2010_X64_ASLCC_PATH    = DEF(VS2010_BINX64)\cl.exe
 *_VS2010_X64_ASLPP_PATH    = DEF(VS2010_BINX64)\cl.exe
 *_VS2010_X64_ASLDLINK_PATH = DEF(VS2010_BINX64)\link.exe
 
-  DEBUG_VS2010_X64_CC_FLAGS     = /nologo /c /WX /GS- /W4 /Gs32768 /D UNICODE /O1b2s /GL /Gy /FIAutoGen.h /EHs-c- /GR- /GF /Z7 /MP
-RELEASE_VS2010_X64_CC_FLAGS     = /nologo /c /WX /GS- /W4 /Gs32768 /D UNICODE /O1b2s /GL /Gy /FIAutoGen.h /EHs-c- /GR- /GF /MP
-NOOPT_VS2010_X64_CC_FLAGS       = /nologo /c /WX /GS- /W4 /Gs32768 /D UNICODE /Gy /FIAutoGen.h /EHs-c- /GR- /GF /Z7 /Od /MP
+  DEBUG_VS2010_X64_CC_FLAGS     = /nologo /c /WX /GS- /W4 /Gs32768 /D UNICODE /O1b2s /GL /Gy /FIAutoGen.h /EHs-c- /GR- /GF /Z7
+RELEASE_VS2010_X64_CC_FLAGS     = /nologo /c /WX /GS- /W4 /Gs32768 /D UNICODE /O1b2s /GL /Gy /FIAutoGen.h /EHs-c- /GR- /GF
+NOOPT_VS2010_X64_CC_FLAGS       = /nologo /c /WX /GS- /W4 /Gs32768 /D UNICODE /Gy /FIAutoGen.h /EHs-c- /GR- /GF /Z7 /Od
 
   DEBUG_VS2010_X64_ASM_FLAGS    = /nologo /c /WX /W3 /Cx /Zd /Zi
 RELEASE_VS2010_X64_ASM_FLAGS    = /nologo /c /WX /W3 /Cx /Zd
 NOOPT_VS2010_X64_ASM_FLAGS    = /nologo /c /WX /W3 /Cx /Zd /Zi
 
@@ -781,13 +781,13 @@ NOOPT_VS2010_X64_DLINK_FLAGS  = /NOLOGO /NODEFAULTLIB /IGNORE:4001 /OPT:REF /OPT
 *_VS2010x86_IA32_APP_PATH     = DEF(VS2010x86_BIN)\cl.exe
 *_VS2010x86_IA32_PP_PATH      = DEF(VS2010x86_BIN)\cl.exe
 *_VS2010x86_IA32_ASM_PATH     = DEF(VS2010x86_BIN)\ml.exe
 
       *_VS2010x86_IA32_MAKE_FLAGS  = /nologo
-  DEBUG_VS2010x86_IA32_CC_FLAGS    = /nologo /c /WX /GS- /W4 /Gs32768 /D UNICODE /O1b2 /GL /FIAutoGen.h /EHs-c- /GR- /GF /Gy /Z7 /MP
-RELEASE_VS2010x86_IA32_CC_FLAGS    = /nologo /c /WX /GS- /W4 /Gs32768 /D UNICODE /O1b2 /GL /FIAutoGen.h /EHs-c- /GR- /GF /MP
-NOOPT_VS2010x86_IA32_CC_FLAGS      = /nologo /c /WX /GS- /W4 /Gs32768 /D UNICODE /FIAutoGen.h /EHs-c- /GR- /GF /Gy /Z7 /Od /MP
+  DEBUG_VS2010x86_IA32_CC_FLAGS    = /nologo /c /WX /GS- /W4 /Gs32768 /D UNICODE /O1b2 /GL /FIAutoGen.h /EHs-c- /GR- /GF /Gy /Z7
+RELEASE_VS2010x86_IA32_CC_FLAGS    = /nologo /c /WX /GS- /W4 /Gs32768 /D UNICODE /O1b2 /GL /FIAutoGen.h /EHs-c- /GR- /GF
+NOOPT_VS2010x86_IA32_CC_FLAGS      = /nologo /c /WX /GS- /W4 /Gs32768 /D UNICODE /FIAutoGen.h /EHs-c- /GR- /GF /Gy /Z7 /Od
 
   DEBUG_VS2010x86_IA32_ASM_FLAGS   = /nologo /c /WX /W3 /Cx /coff /Zd /Zi
 RELEASE_VS2010x86_IA32_ASM_FLAGS   = /nologo /c /WX /W3 /Cx /coff /Zd
 NOOPT_VS2010x86_IA32_ASM_FLAGS     = /nologo /c /WX /W3 /Cx /coff /Zd /Zi
 
@@ -813,13 +813,13 @@ NOOPT_VS2010x86_IA32_DLINK_FLAGS   = /NOLOGO /NODEFAULTLIB /IGNORE:4001 /OPT:REF
 *_VS2010x86_X64_ASM_PATH      = DEF(VS2010x86_BINX64)\ml64.exe
 *_VS2010x86_X64_SLINK_PATH    = DEF(VS2010x86_BINX64)\lib.exe
 *_VS2010x86_X64_DLINK_PATH    = DEF(VS2010x86_BINX64)\link.exe
 *_VS2010x86_X64_ASLDLINK_PATH = DEF(VS2010x86_BINX64)\link.exe
 
-  DEBUG_VS2010x86_X64_CC_FLAGS     = /nologo /c /WX /GS- /W4 /Gs32768 /D UNICODE /O1b2s /GL /Gy /FIAutoGen.h /EHs-c- /GR- /GF /Z7 /MP
-RELEASE_VS2010x86_X64_CC_FLAGS     = /nologo /c /WX /GS- /W4 /Gs32768 /D UNICODE /O1b2s /GL /Gy /FIAutoGen.h /EHs-c- /GR- /GF /MP
-NOOPT_VS2010x86_X64_CC_FLAGS       = /nologo /c /WX /GS- /W4 /Gs32768 /D UNICODE /Gy /FIAutoGen.h /EHs-c- /GR- /GF /Z7 /Od /MP
+  DEBUG_VS2010x86_X64_CC_FLAGS     = /nologo /c /WX /GS- /W4 /Gs32768 /D UNICODE /O1b2s /GL /Gy /FIAutoGen.h /EHs-c- /GR- /GF /Z7
+RELEASE_VS2010x86_X64_CC_FLAGS     = /nologo /c /WX /GS- /W4 /Gs32768 /D UNICODE /O1b2s /GL /Gy /FIAutoGen.h /EHs-c- /GR- /GF
+NOOPT_VS2010x86_X64_CC_FLAGS       = /nologo /c /WX /GS- /W4 /Gs32768 /D UNICODE /Gy /FIAutoGen.h /EHs-c- /GR- /GF /Z7 /Od
 
   DEBUG_VS2010x86_X64_ASM_FLAGS    = /nologo /c /WX /W3 /Cx /Zd /Zi
 RELEASE_VS2010x86_X64_ASM_FLAGS    = /nologo /c /WX /W3 /Cx /Zd
 NOOPT_VS2010x86_X64_ASM_FLAGS      = /nologo /c /WX /W3 /Cx /Zd /Zi
 
@@ -899,13 +899,13 @@ NOOPT_VS2010x86_X64_DLINK_FLAGS    = /NOLOGO /NODEFAULTLIB /IGNORE:4001 /OPT:REF
 *_VS2012_IA32_ASLCC_PATH          = DEF(VS2012_BIN)\cl.exe
 *_VS2012_IA32_ASLPP_PATH          = DEF(VS2012_BIN)\cl.exe
 *_VS2012_IA32_ASLDLINK_PATH       = DEF(VS2012_BIN)\link.exe
 
       *_VS2012_IA32_MAKE_FLAGS    = /nologo
-  DEBUG_VS2012_IA32_CC_FLAGS      = /nologo /arch:IA32 /c /WX /GS- /W4 /Gs32768 /D UNICODE /O1b2 /GL /FIAutoGen.h /EHs-c- /GR- /GF /Gy /Z7 /MP
-RELEASE_VS2012_IA32_CC_FLAGS      = /nologo /arch:IA32 /c /WX /GS- /W4 /Gs32768 /D UNICODE /O1b2 /GL /FIAutoGen.h /EHs-c- /GR- /GF /MP
-NOOPT_VS2012_IA32_CC_FLAGS        = /nologo /arch:IA32 /c /WX /GS- /W4 /Gs32768 /D UNICODE /FIAutoGen.h /EHs-c- /GR- /GF /Gy /Z7 /Od /MP
+  DEBUG_VS2012_IA32_CC_FLAGS      = /nologo /arch:IA32 /c /WX /GS- /W4 /Gs32768 /D UNICODE /O1b2 /GL /FIAutoGen.h /EHs-c- /GR- /GF /Gy /Z7
+RELEASE_VS2012_IA32_CC_FLAGS      = /nologo /arch:IA32 /c /WX /GS- /W4 /Gs32768 /D UNICODE /O1b2 /GL /FIAutoGen.h /EHs-c- /GR- /GF
+NOOPT_VS2012_IA32_CC_FLAGS        = /nologo /arch:IA32 /c /WX /GS- /W4 /Gs32768 /D UNICODE /FIAutoGen.h /EHs-c- /GR- /GF /Gy /Z7 /Od
 
   DEBUG_VS2012_IA32_ASM_FLAGS     = /nologo /c /WX /W3 /Cx /coff /Zd /Zi
 RELEASE_VS2012_IA32_ASM_FLAGS     = /nologo /c /WX /W3 /Cx /coff /Zd
 NOOPT_VS2012_IA32_ASM_FLAGS       = /nologo /c /WX /W3 /Cx /coff /Zd /Zi
 
@@ -931,13 +931,13 @@ NOOPT_VS2012_IA32_DLINK_FLAGS     = /NOLOGO /NODEFAULTLIB /IGNORE:4001 /OPT:REF
 *_VS2012_X64_DLINK_PATH    = DEF(VS2012_BINX64)\link.exe
 *_VS2012_X64_ASLCC_PATH    = DEF(VS2012_BINX64)\cl.exe
 *_VS2012_X64_ASLPP_PATH    = DEF(VS2012_BINX64)\cl.exe
 *_VS2012_X64_ASLDLINK_PATH = DEF(VS2012_BINX64)\link.exe
 
-  DEBUG_VS2012_X64_CC_FLAGS     = /nologo /c /WX /GS- /W4 /Gs32768 /D UNICODE /O1b2s /GL /Gy /FIAutoGen.h /EHs-c- /GR- /GF /Z7 /MP
-RELEASE_VS2012_X64_CC_FLAGS     = /nologo /c /WX /GS- /W4 /Gs32768 /D UNICODE /O1b2s /GL /Gy /FIAutoGen.h /EHs-c- /GR- /GF /MP
-NOOPT_VS2012_X64_CC_FLAGS       = /nologo /c /WX /GS- /W4 /Gs32768 /D UNICODE /Gy /FIAutoGen.h /EHs-c- /GR- /GF /Z7 /Od /MP
+  DEBUG_VS2012_X64_CC_FLAGS     = /nologo /c /WX /GS- /W4 /Gs32768 /D UNICODE /O1b2s /GL /Gy /FIAutoGen.h /EHs-c- /GR- /GF /Z7
+RELEASE_VS2012_X64_CC_FLAGS     = /nologo /c /WX /GS- /W4 /Gs32768 /D UNICODE /O1b2s /GL /Gy /FIAutoGen.h /EHs-c- /GR- /GF
+NOOPT_VS2012_X64_CC_FLAGS       = /nologo /c /WX /GS- /W4 /Gs32768 /D UNICODE /Gy /FIAutoGen.h /EHs-c- /GR- /GF /Z7 /Od
 
   DEBUG_VS2012_X64_ASM_FLAGS    = /nologo /c /WX /W3 /Cx /Zd /Zi
 RELEASE_VS2012_X64_ASM_FLAGS    = /nologo /c /WX /W3 /Cx /Zd
 NOOPT_VS2012_X64_ASM_FLAGS    = /nologo /c /WX /W3 /Cx /Zd /Zi
 
@@ -1015,13 +1015,13 @@ NOOPT_VS2012_X64_DLINK_FLAGS  = /NOLOGO /NODEFAULTLIB /IGNORE:4001 /OPT:REF /OPT
 *_VS2012x86_IA32_APP_PATH     = DEF(VS2012x86_BIN)\cl.exe
 *_VS2012x86_IA32_PP_PATH      = DEF(VS2012x86_BIN)\cl.exe
 *_VS2012x86_IA32_ASM_PATH     = DEF(VS2012x86_BIN)\ml.exe
 
       *_VS2012x86_IA32_MAKE_FLAGS  = /nologo
-  DEBUG_VS2012x86_IA32_CC_FLAGS    = /nologo /arch:IA32 /c /WX /GS- /W4 /Gs32768 /D UNICODE /O1b2 /GL /FIAutoGen.h /EHs-c- /GR- /GF /Gy /Z7 /MP
-RELEASE_VS2012x86_IA32_CC_FLAGS    = /nologo /arch:IA32 /c /WX /GS- /W4 /Gs32768 /D UNICODE /O1b2 /GL /FIAutoGen.h /EHs-c- /GR- /GF /MP
-NOOPT_VS2012x86_IA32_CC_FLAGS      = /nologo /arch:IA32 /c /WX /GS- /W4 /Gs32768 /D UNICODE /FIAutoGen.h /EHs-c- /GR- /GF /Gy /Z7 /Od /MP
+  DEBUG_VS2012x86_IA32_CC_FLAGS    = /nologo /arch:IA32 /c /WX /GS- /W4 /Gs32768 /D UNICODE /O1b2 /GL /FIAutoGen.h /EHs-c- /GR- /GF /Gy /Z7
+RELEASE_VS2012x86_IA32_CC_FLAGS    = /nologo /arch:IA32 /c /WX /GS- /W4 /Gs32768 /D UNICODE /O1b2 /GL /FIAutoGen.h /EHs-c- /GR- /GF
+NOOPT_VS2012x86_IA32_CC_FLAGS      = /nologo /arch:IA32 /c /WX /GS- /W4 /Gs32768 /D UNICODE /FIAutoGen.h /EHs-c- /GR- /GF /Gy /Z7 /Od
 
   DEBUG_VS2012x86_IA32_ASM_FLAGS   = /nologo /c /WX /W3 /Cx /coff /Zd /Zi
 RELEASE_VS2012x86_IA32_ASM_FLAGS   = /nologo /c /WX /W3 /Cx /coff /Zd
 NOOPT_VS2012x86_IA32_ASM_FLAGS     = /nologo /c /WX /W3 /Cx /coff /Zd /Zi
 
@@ -1047,13 +1047,13 @@ NOOPT_VS2012x86_IA32_DLINK_FLAGS   = /NOLOGO /NODEFAULTLIB /IGNORE:4001 /OPT:REF
 *_VS2012x86_X64_ASM_PATH      = DEF(VS2012x86_BINX64)\ml64.exe
 *_VS2012x86_X64_SLINK_PATH    = DEF(VS2012x86_BINX64)\lib.exe
 *_VS2012x86_X64_DLINK_PATH    = DEF(VS2012x86_BINX64)\link.exe
 *_VS2012x86_X64_ASLDLINK_PATH = DEF(VS2012x86_BINX64)\link.exe
 
-  DEBUG_VS2012x86_X64_CC_FLAGS     = /nologo /c /WX /GS- /W4 /Gs32768 /D UNICODE /O1b2s /GL /Gy /FIAutoGen.h /EHs-c- /GR- /GF /Z7 /MP
-RELEASE_VS2012x86_X64_CC_FLAGS     = /nologo /c /WX /GS- /W4 /Gs32768 /D UNICODE /O1b2s /GL /Gy /FIAutoGen.h /EHs-c- /GR- /GF /MP
-NOOPT_VS2012x86_X64_CC_FLAGS       = /nologo /c /WX /GS- /W4 /Gs32768 /D UNICODE /Gy /FIAutoGen.h /EHs-c- /GR- /GF /Z7 /Od /MP
+  DEBUG_VS2012x86_X64_CC_FLAGS     = /nologo /c /WX /GS- /W4 /Gs32768 /D UNICODE /O1b2s /GL /Gy /FIAutoGen.h /EHs-c- /GR- /GF /Z7
+RELEASE_VS2012x86_X64_CC_FLAGS     = /nologo /c /WX /GS- /W4 /Gs32768 /D UNICODE /O1b2s /GL /Gy /FIAutoGen.h /EHs-c- /GR- /GF
+NOOPT_VS2012x86_X64_CC_FLAGS       = /nologo /c /WX /GS- /W4 /Gs32768 /D UNICODE /Gy /FIAutoGen.h /EHs-c- /GR- /GF /Z7 /Od
 
   DEBUG_VS2012x86_X64_ASM_FLAGS    = /nologo /c /WX /W3 /Cx /Zd /Zi
 RELEASE_VS2012x86_X64_ASM_FLAGS    = /nologo /c /WX /W3 /Cx /Zd
 NOOPT_VS2012x86_X64_ASM_FLAGS      = /nologo /c /WX /W3 /Cx /Zd /Zi
 
@@ -1133,13 +1133,13 @@ NOOPT_VS2012x86_X64_DLINK_FLAGS    = /NOLOGO /NODEFAULTLIB /IGNORE:4001 /OPT:REF
 *_VS2013_IA32_ASLCC_PATH          = DEF(VS2013_BIN)\cl.exe
 *_VS2013_IA32_ASLPP_PATH          = DEF(VS2013_BIN)\cl.exe
 *_VS2013_IA32_ASLDLINK_PATH       = DEF(VS2013_BIN)\link.exe
 
       *_VS2013_IA32_MAKE_FLAGS    = /nologo
-  DEBUG_VS2013_IA32_CC_FLAGS      = /nologo /arch:IA32 /c /WX /GS- /W4 /Gs32768 /D UNICODE /O1b2 /GL /FIAutoGen.h /EHs-c- /GR- /GF /Gy /Z7 /Gw /MP
-RELEASE_VS2013_IA32_CC_FLAGS      = /nologo /arch:IA32 /c /WX /GS- /W4 /Gs32768 /D UNICODE /O1b2 /GL /FIAutoGen.h /EHs-c- /GR- /GF /Gw /MP
-NOOPT_VS2013_IA32_CC_FLAGS        = /nologo /arch:IA32 /c /WX /GS- /W4 /Gs32768 /D UNICODE /FIAutoGen.h /EHs-c- /GR- /GF /Gy /Z7 /Od /MP
+  DEBUG_VS2013_IA32_CC_FLAGS      = /nologo /arch:IA32 /c /WX /GS- /W4 /Gs32768 /D UNICODE /O1b2 /GL /FIAutoGen.h /EHs-c- /GR- /GF /Gy /Z7 /Gw
+RELEASE_VS2013_IA32_CC_FLAGS      = /nologo /arch:IA32 /c /WX /GS- /W4 /Gs32768 /D UNICODE /O1b2 /GL /FIAutoGen.h /EHs-c- /GR- /GF /Gw
+NOOPT_VS2013_IA32_CC_FLAGS        = /nologo /arch:IA32 /c /WX /GS- /W4 /Gs32768 /D UNICODE /FIAutoGen.h /EHs-c- /GR- /GF /Gy /Z7 /Od
 
   DEBUG_VS2013_IA32_ASM_FLAGS     = /nologo /c /WX /W3 /Cx /coff /Zd /Zi
 RELEASE_VS2013_IA32_ASM_FLAGS     = /nologo /c /WX /W3 /Cx /coff /Zd
 NOOPT_VS2013_IA32_ASM_FLAGS       = /nologo /c /WX /W3 /Cx /coff /Zd /Zi
 
@@ -1165,13 +1165,13 @@ NOOPT_VS2013_IA32_DLINK_FLAGS     = /NOLOGO /NODEFAULTLIB /IGNORE:4001 /OPT:REF
 *_VS2013_X64_DLINK_PATH    = DEF(VS2013_BINX64)\link.exe
 *_VS2013_X64_ASLCC_PATH    = DEF(VS2013_BINX64)\cl.exe
 *_VS2013_X64_ASLPP_PATH    = DEF(VS2013_BINX64)\cl.exe
 *_VS2013_X64_ASLDLINK_PATH = DEF(VS2013_BINX64)\link.exe
 
-  DEBUG_VS2013_X64_CC_FLAGS     = /nologo /c /WX /GS- /W4 /Gs32768 /D UNICODE /O1b2s /GL /Gy /FIAutoGen.h /EHs-c- /GR- /GF /Z7 /Gw /MP
-RELEASE_VS2013_X64_CC_FLAGS     = /nologo /c /WX /GS- /W4 /Gs32768 /D UNICODE /O1b2s /GL /Gy /FIAutoGen.h /EHs-c- /GR- /GF /Gw /MP
-NOOPT_VS2013_X64_CC_FLAGS       = /nologo /c /WX /GS- /W4 /Gs32768 /D UNICODE /Gy /FIAutoGen.h /EHs-c- /GR- /GF /Z7 /Od /MP
+  DEBUG_VS2013_X64_CC_FLAGS     = /nologo /c /WX /GS- /W4 /Gs32768 /D UNICODE /O1b2s /GL /Gy /FIAutoGen.h /EHs-c- /GR- /GF /Z7 /Gw
+RELEASE_VS2013_X64_CC_FLAGS     = /nologo /c /WX /GS- /W4 /Gs32768 /D UNICODE /O1b2s /GL /Gy /FIAutoGen.h /EHs-c- /GR- /GF /Gw
+NOOPT_VS2013_X64_CC_FLAGS       = /nologo /c /WX /GS- /W4 /Gs32768 /D UNICODE /Gy /FIAutoGen.h /EHs-c- /GR- /GF /Z7 /Od
 
   DEBUG_VS2013_X64_ASM_FLAGS    = /nologo /c /WX /W3 /Cx /Zd /Zi
 RELEASE_VS2013_X64_ASM_FLAGS    = /nologo /c /WX /W3 /Cx /Zd
 NOOPT_VS2013_X64_ASM_FLAGS    = /nologo /c /WX /W3 /Cx /Zd /Zi
 
@@ -1249,13 +1249,13 @@ NOOPT_VS2013_X64_DLINK_FLAGS  = /NOLOGO /NODEFAULTLIB /IGNORE:4001 /OPT:REF /OPT
 *_VS2013x86_IA32_APP_PATH     = DEF(VS2013x86_BIN)\cl.exe
 *_VS2013x86_IA32_PP_PATH      = DEF(VS2013x86_BIN)\cl.exe
 *_VS2013x86_IA32_ASM_PATH     = DEF(VS2013x86_BIN)\ml.exe
 
       *_VS2013x86_IA32_MAKE_FLAGS  = /nologo
-  DEBUG_VS2013x86_IA32_CC_FLAGS    = /nologo /arch:IA32 /c /WX /GS- /W4 /Gs32768 /D UNICODE /O1b2 /GL /FIAutoGen.h /EHs-c- /GR- /GF /Gy /Z7 /Gw /MP
-RELEASE_VS2013x86_IA32_CC_FLAGS    = /nologo /arch:IA32 /c /WX /GS- /W4 /Gs32768 /D UNICODE /O1b2 /GL /FIAutoGen.h /EHs-c- /GR- /GF /Gw /MP
-NOOPT_VS2013x86_IA32_CC_FLAGS      = /nologo /arch:IA32 /c /WX /GS- /W4 /Gs32768 /D UNICODE /FIAutoGen.h /EHs-c- /GR- /GF /Gy /Z7 /Od /MP
+  DEBUG_VS2013x86_IA32_CC_FLAGS    = /nologo /arch:IA32 /c /WX /GS- /W4 /Gs32768 /D UNICODE /O1b2 /GL /FIAutoGen.h /EHs-c- /GR- /GF /Gy /Z7 /Gw
+RELEASE_VS2013x86_IA32_CC_FLAGS    = /nologo /arch:IA32 /c /WX /GS- /W4 /Gs32768 /D UNICODE /O1b2 /GL /FIAutoGen.h /EHs-c- /GR- /GF /Gw
+NOOPT_VS2013x86_IA32_CC_FLAGS      = /nologo /arch:IA32 /c /WX /GS- /W4 /Gs32768 /D UNICODE /FIAutoGen.h /EHs-c- /GR- /GF /Gy /Z7 /Od
 
   DEBUG_VS2013x86_IA32_ASM_FLAGS   = /nologo /c /WX /W3 /Cx /coff /Zd /Zi
 RELEASE_VS2013x86_IA32_ASM_FLAGS   = /nologo /c /WX /W3 /Cx /coff /Zd
 NOOPT_VS2013x86_IA32_ASM_FLAGS     = /nologo /c /WX /W3 /Cx /coff /Zd /Zi
 
@@ -1281,13 +1281,13 @@ NOOPT_VS2013x86_IA32_DLINK_FLAGS   = /NOLOGO /NODEFAULTLIB /IGNORE:4001 /OPT:REF
 *_VS2013x86_X64_ASM_PATH      = DEF(VS2013x86_BINX64)\ml64.exe
 *_VS2013x86_X64_SLINK_PATH    = DEF(VS2013x86_BINX64)\lib.exe
 *_VS2013x86_X64_DLINK_PATH    = DEF(VS2013x86_BINX64)\link.exe
 *_VS2013x86_X64_ASLDLINK_PATH = DEF(VS2013x86_BINX64)\link.exe
 
-  DEBUG_VS2013x86_X64_CC_FLAGS     = /nologo /c /WX /GS- /W4 /Gs32768 /D UNICODE /O1b2s /GL /Gy /FIAutoGen.h /EHs-c- /GR- /GF /Z7 /Gw /MP
-RELEASE_VS2013x86_X64_CC_FLAGS     = /nologo /c /WX /GS- /W4 /Gs32768 /D UNICODE /O1b2s /GL /Gy /FIAutoGen.h /EHs-c- /GR- /GF /Gw /MP
-NOOPT_VS2013x86_X64_CC_FLAGS       = /nologo /c /WX /GS- /W4 /Gs32768 /D UNICODE /Gy /FIAutoGen.h /EHs-c- /GR- /GF /Z7 /Od /MP
+  DEBUG_VS2013x86_X64_CC_FLAGS     = /nologo /c /WX /GS- /W4 /Gs32768 /D UNICODE /O1b2s /GL /Gy /FIAutoGen.h /EHs-c- /GR- /GF /Z7 /Gw
+RELEASE_VS2013x86_X64_CC_FLAGS     = /nologo /c /WX /GS- /W4 /Gs32768 /D UNICODE /O1b2s /GL /Gy /FIAutoGen.h /EHs-c- /GR- /GF /Gw
+NOOPT_VS2013x86_X64_CC_FLAGS       = /nologo /c /WX /GS- /W4 /Gs32768 /D UNICODE /Gy /FIAutoGen.h /EHs-c- /GR- /GF /Z7 /Od
 
   DEBUG_VS2013x86_X64_ASM_FLAGS    = /nologo /c /WX /W3 /Cx /Zd /Zi
 RELEASE_VS2013x86_X64_ASM_FLAGS    = /nologo /c /WX /W3 /Cx /Zd
 NOOPT_VS2013x86_X64_ASM_FLAGS      = /nologo /c /WX /W3 /Cx /Zd /Zi
 
@@ -1368,13 +1368,13 @@ NOOPT_VS2013x86_X64_DLINK_FLAGS    = /NOLOGO /NODEFAULTLIB /IGNORE:4001 /OPT:REF
 *_VS2015_IA32_ASLCC_PATH          = DEF(VS2015_BIN)\cl.exe
 *_VS2015_IA32_ASLPP_PATH          = DEF(VS2015_BIN)\cl.exe
 *_VS2015_IA32_ASLDLINK_PATH       = DEF(VS2015_BIN)\link.exe
 
       *_VS2015_IA32_MAKE_FLAGS    = /nologo
-  DEBUG_VS2015_IA32_CC_FLAGS      = /nologo /arch:IA32 /c /WX /GS- /W4 /Gs32768 /D UNICODE /O1b2 /GL /FIAutoGen.h /EHs-c- /GR- /GF /Gy /Z7 /Gw /MP
-RELEASE_VS2015_IA32_CC_FLAGS      = /nologo /arch:IA32 /c /WX /GS- /W4 /Gs32768 /D UNICODE /O1b2 /GL /FIAutoGen.h /EHs-c- /GR- /GF /Gw /MP
-NOOPT_VS2015_IA32_CC_FLAGS        = /nologo /arch:IA32 /c /WX /GS- /W4 /Gs32768 /D UNICODE /FIAutoGen.h /EHs-c- /GR- /GF /Gy /Z7 /Od /MP
+  DEBUG_VS2015_IA32_CC_FLAGS      = /nologo /arch:IA32 /c /WX /GS- /W4 /Gs32768 /D UNICODE /O1b2 /GL /FIAutoGen.h /EHs-c- /GR- /GF /Gy /Z7 /Gw
+RELEASE_VS2015_IA32_CC_FLAGS      = /nologo /arch:IA32 /c /WX /GS- /W4 /Gs32768 /D UNICODE /O1b2 /GL /FIAutoGen.h /EHs-c- /GR- /GF /Gw
+NOOPT_VS2015_IA32_CC_FLAGS        = /nologo /arch:IA32 /c /WX /GS- /W4 /Gs32768 /D UNICODE /FIAutoGen.h /EHs-c- /GR- /GF /Gy /Z7 /Od
 
   DEBUG_VS2015_IA32_ASM_FLAGS     = /nologo /c /WX /W3 /Cx /coff /Zd /Zi
 RELEASE_VS2015_IA32_ASM_FLAGS     = /nologo /c /WX /W3 /Cx /coff /Zd
 NOOPT_VS2015_IA32_ASM_FLAGS       = /nologo /c /WX /W3 /Cx /coff /Zd /Zi
 
@@ -1400,13 +1400,13 @@ NOOPT_VS2015_IA32_DLINK_FLAGS     = /NOLOGO /NODEFAULTLIB /IGNORE:4001 /OPT:REF
 *_VS2015_X64_DLINK_PATH    = DEF(VS2015_BINX64)\link.exe
 *_VS2015_X64_ASLCC_PATH    = DEF(VS2015_BINX64)\cl.exe
 *_VS2015_X64_ASLPP_PATH    = DEF(VS2015_BINX64)\cl.exe
 *_VS2015_X64_ASLDLINK_PATH = DEF(VS2015_BINX64)\link.exe
 
-  DEBUG_VS2015_X64_CC_FLAGS     = /nologo /c /WX /GS- /W4 /Gs32768 /D UNICODE /O1b2s /GL /Gy /FIAutoGen.h /EHs-c- /GR- /GF /Z7 /Gw /MP
-RELEASE_VS2015_X64_CC_FLAGS     = /nologo /c /WX /GS- /W4 /Gs32768 /D UNICODE /O1b2s /GL /Gy /FIAutoGen.h /EHs-c- /GR- /GF /Gw /MP
-NOOPT_VS2015_X64_CC_FLAGS       = /nologo /c /WX /GS- /W4 /Gs32768 /D UNICODE /Gy /FIAutoGen.h /EHs-c- /GR- /GF /Z7 /Od /MP
+  DEBUG_VS2015_X64_CC_FLAGS     = /nologo /c /WX /GS- /W4 /Gs32768 /D UNICODE /O1b2s /GL /Gy /FIAutoGen.h /EHs-c- /GR- /GF /Z7 /Gw
+RELEASE_VS2015_X64_CC_FLAGS     = /nologo /c /WX /GS- /W4 /Gs32768 /D UNICODE /O1b2s /GL /Gy /FIAutoGen.h /EHs-c- /GR- /GF /Gw
+NOOPT_VS2015_X64_CC_FLAGS       = /nologo /c /WX /GS- /W4 /Gs32768 /D UNICODE /Gy /FIAutoGen.h /EHs-c- /GR- /GF /Z7 /Od
 
   DEBUG_VS2015_X64_ASM_FLAGS    = /nologo /c /WX /W3 /Cx /Zd /Zi
 RELEASE_VS2015_X64_ASM_FLAGS    = /nologo /c /WX /W3 /Cx /Zd
 NOOPT_VS2015_X64_ASM_FLAGS    = /nologo /c /WX /W3 /Cx /Zd /Zi
 
@@ -1485,13 +1485,13 @@ NOOPT_VS2015_X64_DLINK_FLAGS  = /NOLOGO /NODEFAULTLIB /IGNORE:4001 /OPT:REF /OPT
 *_VS2015x86_IA32_APP_PATH     = DEF(VS2015x86_BIN)\cl.exe
 *_VS2015x86_IA32_PP_PATH      = DEF(VS2015x86_BIN)\cl.exe
 *_VS2015x86_IA32_ASM_PATH     = DEF(VS2015x86_BIN)\ml.exe
 
       *_VS2015x86_IA32_MAKE_FLAGS  = /nologo
-  DEBUG_VS2015x86_IA32_CC_FLAGS    = /nologo /arch:IA32 /c /WX /GS- /W4 /Gs32768 /D UNICODE /O1b2 /GL /FIAutoGen.h /EHs-c- /GR- /GF /Gy /Z7 /Gw /MP
-RELEASE_VS2015x86_IA32_CC_FLAGS    = /nologo /arch:IA32 /c /WX /GS- /W4 /Gs32768 /D UNICODE /O1b2 /GL /FIAutoGen.h /EHs-c- /GR- /GF /Gw /MP
-NOOPT_VS2015x86_IA32_CC_FLAGS      = /nologo /arch:IA32 /c /WX /GS- /W4 /Gs32768 /D UNICODE /FIAutoGen.h /EHs-c- /GR- /GF /Gy /Z7 /Od /MP
+  DEBUG_VS2015x86_IA32_CC_FLAGS    = /nologo /arch:IA32 /c /WX /GS- /W4 /Gs32768 /D UNICODE /O1b2 /GL /FIAutoGen.h /EHs-c- /GR- /GF /Gy /Z7 /Gw
+RELEASE_VS2015x86_IA32_CC_FLAGS    = /nologo /arch:IA32 /c /WX /GS- /W4 /Gs32768 /D UNICODE /O1b2 /GL /FIAutoGen.h /EHs-c- /GR- /GF /Gw
+NOOPT_VS2015x86_IA32_CC_FLAGS      = /nologo /arch:IA32 /c /WX /GS- /W4 /Gs32768 /D UNICODE /FIAutoGen.h /EHs-c- /GR- /GF /Gy /Z7 /Od
 
   DEBUG_VS2015x86_IA32_ASM_FLAGS   = /nologo /c /WX /W3 /Cx /coff /Zd /Zi
 RELEASE_VS2015x86_IA32_ASM_FLAGS   = /nologo /c /WX /W3 /Cx /coff /Zd
 NOOPT_VS2015x86_IA32_ASM_FLAGS     = /nologo /c /WX /W3 /Cx /coff /Zd /Zi
 
@@ -1517,13 +1517,13 @@ NOOPT_VS2015x86_IA32_DLINK_FLAGS   = /NOLOGO /NODEFAULTLIB /IGNORE:4001 /OPT:REF
 *_VS2015x86_X64_ASM_PATH      = DEF(VS2015x86_BINX64)\ml64.exe
 *_VS2015x86_X64_SLINK_PATH    = DEF(VS2015x86_BINX64)\lib.exe
 *_VS2015x86_X64_DLINK_PATH    = DEF(VS2015x86_BINX64)\link.exe
 *_VS2015x86_X64_ASLDLINK_PATH = DEF(VS2015x86_BINX64)\link.exe
 
-  DEBUG_VS2015x86_X64_CC_FLAGS     = /nologo /c /WX /GS- /W4 /Gs32768 /D UNICODE /O1b2s /GL /Gy /FIAutoGen.h /EHs-c- /GR- /GF /Z7 /Gw /MP
-RELEASE_VS2015x86_X64_CC_FLAGS     = /nologo /c /WX /GS- /W4 /Gs32768 /D UNICODE /O1b2s /GL /Gy /FIAutoGen.h /EHs-c- /GR- /GF /Gw /MP
-NOOPT_VS2015x86_X64_CC_FLAGS       = /nologo /c /WX /GS- /W4 /Gs32768 /D UNICODE /Gy /FIAutoGen.h /EHs-c- /GR- /GF /Z7 /Od /MP
+  DEBUG_VS2015x86_X64_CC_FLAGS     = /nologo /c /WX /GS- /W4 /Gs32768 /D UNICODE /O1b2s /GL /Gy /FIAutoGen.h /EHs-c- /GR- /GF /Z7 /Gw
+RELEASE_VS2015x86_X64_CC_FLAGS     = /nologo /c /WX /GS- /W4 /Gs32768 /D UNICODE /O1b2s /GL /Gy /FIAutoGen.h /EHs-c- /GR- /GF /Gw
+NOOPT_VS2015x86_X64_CC_FLAGS       = /nologo /c /WX /GS- /W4 /Gs32768 /D UNICODE /Gy /FIAutoGen.h /EHs-c- /GR- /GF /Z7 /Od
 
   DEBUG_VS2015x86_X64_ASM_FLAGS    = /nologo /c /WX /W3 /Cx /Zd /Zi
 RELEASE_VS2015x86_X64_ASM_FLAGS    = /nologo /c /WX /W3 /Cx /Zd
 NOOPT_VS2015x86_X64_ASM_FLAGS      = /nologo /c /WX /W3 /Cx /Zd /Zi
 
@@ -1599,13 +1599,13 @@ NOOPT_VS2015x86_X64_DLINK_FLAGS    = /NOLOGO /NODEFAULTLIB /IGNORE:4001 /OPT:REF
 *_VS2017_IA32_APP_PATH     = DEF(VS2017_BIN_IA32)\cl.exe
 *_VS2017_IA32_PP_PATH      = DEF(VS2017_BIN_IA32)\cl.exe
 *_VS2017_IA32_ASM_PATH     = DEF(VS2017_BIN_IA32)\ml.exe
 
       *_VS2017_IA32_MAKE_FLAGS  = /nologo
-  DEBUG_VS2017_IA32_CC_FLAGS    = /nologo /arch:IA32 /c /WX /GS- /W4 /Gs32768 /D UNICODE /O1b2 /GL /FIAutoGen.h /EHs-c- /GR- /GF /Gy /Z7 /Gw /MP
-RELEASE_VS2017_IA32_CC_FLAGS    = /nologo /arch:IA32 /c /WX /GS- /W4 /Gs32768 /D UNICODE /O1b2 /GL /FIAutoGen.h /EHs-c- /GR- /GF /Gw /MP
-NOOPT_VS2017_IA32_CC_FLAGS      = /nologo /arch:IA32 /c /WX /GS- /W4 /Gs32768 /D UNICODE /FIAutoGen.h /EHs-c- /GR- /GF /Gy /Z7 /Od /MP
+  DEBUG_VS2017_IA32_CC_FLAGS    = /nologo /arch:IA32 /c /WX /GS- /W4 /Gs32768 /D UNICODE /O1b2 /GL /FIAutoGen.h /EHs-c- /GR- /GF /Gy /Z7 /Gw
+RELEASE_VS2017_IA32_CC_FLAGS    = /nologo /arch:IA32 /c /WX /GS- /W4 /Gs32768 /D UNICODE /O1b2 /GL /FIAutoGen.h /EHs-c- /GR- /GF /Gw
+NOOPT_VS2017_IA32_CC_FLAGS      = /nologo /arch:IA32 /c /WX /GS- /W4 /Gs32768 /D UNICODE /FIAutoGen.h /EHs-c- /GR- /GF /Gy /Z7 /Od
 
   DEBUG_VS2017_IA32_ASM_FLAGS   = /nologo /c /WX /W3 /Cx /coff /Zd /Zi
 RELEASE_VS2017_IA32_ASM_FLAGS   = /nologo /c /WX /W3 /Cx /coff /Zd
 NOOPT_VS2017_IA32_ASM_FLAGS     = /nologo /c /WX /W3 /Cx /coff /Zd /Zi
 
@@ -1629,13 +1629,13 @@ NOOPT_VS2017_IA32_DLINK_FLAGS   = /NOLOGO /NODEFAULTLIB /IGNORE:4001 /OPT:REF /O
 *_VS2017_X64_ASM_PATH      = DEF(VS2017_BIN_X64)\ml64.exe
 *_VS2017_X64_SLINK_PATH    = DEF(VS2017_BIN_X64)\lib.exe
 *_VS2017_X64_DLINK_PATH    = DEF(VS2017_BIN_X64)\link.exe
 *_VS2017_X64_ASLDLINK_PATH = DEF(VS2017_BIN_X64)\link.exe
 
-  DEBUG_VS2017_X64_CC_FLAGS     = /nologo /c /WX /GS- /W4 /Gs32768 /D UNICODE /O1b2s /GL /Gy /FIAutoGen.h /EHs-c- /GR- /GF /Z7 /Gw /MP
-RELEASE_VS2017_X64_CC_FLAGS     = /nologo /c /WX /GS- /W4 /Gs32768 /D UNICODE /O1b2s /GL /Gy /FIAutoGen.h /EHs-c- /GR- /GF /Gw /MP
-NOOPT_VS2017_X64_CC_FLAGS       = /nologo /c /WX /GS- /W4 /Gs32768 /D UNICODE /Gy /FIAutoGen.h /EHs-c- /GR- /GF /Z7 /Od /MP
+  DEBUG_VS2017_X64_CC_FLAGS     = /nologo /c /WX /GS- /W4 /Gs32768 /D UNICODE /O1b2s /GL /Gy /FIAutoGen.h /EHs-c- /GR- /GF /Z7 /Gw
+RELEASE_VS2017_X64_CC_FLAGS     = /nologo /c /WX /GS- /W4 /Gs32768 /D UNICODE /O1b2s /GL /Gy /FIAutoGen.h /EHs-c- /GR- /GF /Gw
+NOOPT_VS2017_X64_CC_FLAGS       = /nologo /c /WX /GS- /W4 /Gs32768 /D UNICODE /Gy /FIAutoGen.h /EHs-c- /GR- /GF /Z7 /Od
 
   DEBUG_VS2017_X64_ASM_FLAGS    = /nologo /c /WX /W3 /Cx /Zd /Zi
 RELEASE_VS2017_X64_ASM_FLAGS    = /nologo /c /WX /W3 /Cx /Zd
 NOOPT_VS2017_X64_ASM_FLAGS      = /nologo /c /WX /W3 /Cx /Zd /Zi
 
@@ -1762,13 +1762,13 @@ NOOPT_VS2017_AARCH64_DLINK_FLAGS   = /NOLOGO /NODEFAULTLIB /IGNORE:4001 /OPT:REF
 *_VS2019_IA32_APP_PATH     = DEF(VS2019_BIN_IA32)\cl.exe
 *_VS2019_IA32_PP_PATH      = DEF(VS2019_BIN_IA32)\cl.exe
 *_VS2019_IA32_ASM_PATH     = DEF(VS2019_BIN_IA32)\ml.exe
 
       *_VS2019_IA32_MAKE_FLAGS  = /nologo
-  DEBUG_VS2019_IA32_CC_FLAGS    = /nologo /arch:IA32 /c /WX /GS- /W4 /Gs32768 /D UNICODE /O1b2 /GL /FIAutoGen.h /EHs-c- /GR- /GF /Gy /Z7 /Gw /MP
-RELEASE_VS2019_IA32_CC_FLAGS    = /nologo /arch:IA32 /c /WX /GS- /W4 /Gs32768 /D UNICODE /O1b2 /GL /FIAutoGen.h /EHs-c- /GR- /GF /Gw /MP
-NOOPT_VS2019_IA32_CC_FLAGS      = /nologo /arch:IA32 /c /WX /GS- /W4 /Gs32768 /D UNICODE /FIAutoGen.h /EHs-c- /GR- /GF /Gy /Z7 /Od /MP
+  DEBUG_VS2019_IA32_CC_FLAGS    = /nologo /arch:IA32 /c /WX /GS- /W4 /Gs32768 /D UNICODE /O1b2 /GL /FIAutoGen.h /EHs-c- /GR- /GF /Gy /Z7 /Gw
+RELEASE_VS2019_IA32_CC_FLAGS    = /nologo /arch:IA32 /c /WX /GS- /W4 /Gs32768 /D UNICODE /O1b2 /GL /FIAutoGen.h /EHs-c- /GR- /GF /Gw
+NOOPT_VS2019_IA32_CC_FLAGS      = /nologo /arch:IA32 /c /WX /GS- /W4 /Gs32768 /D UNICODE /FIAutoGen.h /EHs-c- /GR- /GF /Gy /Z7 /Od
 
   DEBUG_VS2019_IA32_ASM_FLAGS   = /nologo /c /WX /W3 /Cx /coff /Zd /Zi
 RELEASE_VS2019_IA32_ASM_FLAGS   = /nologo /c /WX /W3 /Cx /coff /Zd
 NOOPT_VS2019_IA32_ASM_FLAGS     = /nologo /c /WX /W3 /Cx /coff /Zd /Zi
 
@@ -1792,13 +1792,13 @@ NOOPT_VS2019_IA32_DLINK_FLAGS   = /NOLOGO /NODEFAULTLIB /IGNORE:4001 /OPT:REF /O
 *_VS2019_X64_ASM_PATH      = DEF(VS2019_BIN_X64)\ml64.exe
 *_VS2019_X64_SLINK_PATH    = DEF(VS2019_BIN_X64)\lib.exe
 *_VS2019_X64_DLINK_PATH    = DEF(VS2019_BIN_X64)\link.exe
 *_VS2019_X64_ASLDLINK_PATH = DEF(VS2019_BIN_X64)\link.exe
 
-  DEBUG_VS2019_X64_CC_FLAGS     = /nologo /c /WX /GS- /W4 /Gs32768 /D UNICODE /O1b2s /GL /Gy /FIAutoGen.h /EHs-c- /GR- /GF /Z7 /Gw /MP
-RELEASE_VS2019_X64_CC_FLAGS     = /nologo /c /WX /GS- /W4 /Gs32768 /D UNICODE /O1b2s /GL /Gy /FIAutoGen.h /EHs-c- /GR- /GF /Gw /MP
-NOOPT_VS2019_X64_CC_FLAGS       = /nologo /c /WX /GS- /W4 /Gs32768 /D UNICODE /Gy /FIAutoGen.h /EHs-c- /GR- /GF /Z7 /Od /MP
+  DEBUG_VS2019_X64_CC_FLAGS     = /nologo /c /WX /GS- /W4 /Gs32768 /D UNICODE /O1b2s /GL /Gy /FIAutoGen.h /EHs-c- /GR- /GF /Z7 /Gw
+RELEASE_VS2019_X64_CC_FLAGS     = /nologo /c /WX /GS- /W4 /Gs32768 /D UNICODE /O1b2s /GL /Gy /FIAutoGen.h /EHs-c- /GR- /GF /Gw
+NOOPT_VS2019_X64_CC_FLAGS       = /nologo /c /WX /GS- /W4 /Gs32768 /D UNICODE /Gy /FIAutoGen.h /EHs-c- /GR- /GF /Z7 /Od
 
   DEBUG_VS2019_X64_ASM_FLAGS    = /nologo /c /WX /W3 /Cx /Zd /Zi
 RELEASE_VS2019_X64_ASM_FLAGS    = /nologo /c /WX /W3 /Cx /Zd
 NOOPT_VS2019_X64_ASM_FLAGS      = /nologo /c /WX /W3 /Cx /Zd /Zi
 
diff --git a/BaseTools/Source/Python/AutoGen/GenMake.py b/BaseTools/Source/Python/AutoGen/GenMake.py
index 59a01a7f24..b4cd68c989 100755
--- a/BaseTools/Source/Python/AutoGen/GenMake.py
+++ b/BaseTools/Source/Python/AutoGen/GenMake.py
@@ -183,10 +183,13 @@ class BuildFile(object):
             EdkLogger.error("build", PARAMETER_INVALID, "Invalid build type [%s]" % FileType,
                             ExtraData="[%s]" % str(self._AutoGenObject))
         self._FileType = FileType
         FileContent = self._TEMPLATE_.Replace(self._TemplateDict)
         FileName = self._FILE_NAME_[FileType]
+        if not os.path.exists(os.path.join(self._AutoGenObject.MakeFileDir, "deps.txt")):
+            with open(os.path.join(self._AutoGenObject.MakeFileDir, "deps.txt"),"w+") as fd:
+                fd.write("COMMON_DEPS = \n")
         return SaveFileOnChange(os.path.join(self._AutoGenObject.MakeFileDir, FileName), FileContent, False)
 
     ## Return a list of directory creation command string
     #
     #   @param      DirList     The list of directory to be created
@@ -302,12 +305,11 @@ MAKE_FILE = ${makefile_path}
 # Build Macro
 #
 ${BEGIN}${file_macro}
 ${END}
 
-COMMON_DEPS = ${BEGIN}${common_dependency_file} \\
-              ${END}
+${INCLUDETAG}
 
 #
 # Overridable Target Macro Definitions
 #
 FORCE_REBUILD = force_build
@@ -588,14 +590,13 @@ cleanlib:
                                                     "source_file" : IncludePathList
                                                 }
                                                 )
         FileMacroList.append(FileMacro)
         # Add support when compiling .nasm source files
-        for File in self.FileCache.keys():
-            if not str(File).endswith('.nasm'):
-                continue
-            IncludePathList = []
+        IncludePathList = []
+        asmsource = [item for item in MyAgo.SourceFileList if item.File.upper().endswith((".NASM",".ASM"))]
+        if asmsource:
             for P in  MyAgo.IncludePathList:
                 IncludePath = self._INC_FLAG_['NASM'] + self.PlaceMacro(P, self.Macros)
                 if IncludePath.endswith(os.sep):
                     IncludePath = IncludePath.rstrip(os.sep)
                 # When compiling .nasm files, need to add a literal backslash at each path
@@ -604,11 +605,10 @@ cleanlib:
                     IncludePath = ''.join([IncludePath, '^', os.sep])
                 else:
                     IncludePath = os.path.join(IncludePath, '')
                 IncludePathList.append(IncludePath)
             FileMacroList.append(self._FILE_MACRO_TEMPLATE.Replace({"macro_name": "NASM_INC", "source_file": IncludePathList}))
-            break
 
         # Generate macros used to represent files containing list of input files
         for ListFileMacro in self.ListFileMacros:
             ListFileName = os.path.join(MyAgo.OutputDir, "%s.lst" % ListFileMacro.lower()[:len(ListFileMacro) - 5])
             FileMacroList.append("%s = %s" % (ListFileMacro, ListFileName))
@@ -694,10 +694,11 @@ cleanlib:
             "dependent_library_build_directory" : self.LibraryBuildDirectoryList,
             "library_build_command"     : LibraryMakeCommandList,
             "file_macro"                : FileMacroList,
             "file_build_target"         : self.BuildTargetList,
             "backward_compatible_target": BcTargetList,
+            "INCLUDETAG"                   : self._INCLUDE_CMD_[self._FileType] + " " + os.path.join("$(MODULE_BUILD_DIR)","deps.txt")
         }
 
         return MakefileTemplateDict
 
     def ParserGenerateFfsCmd(self):
@@ -901,20 +902,14 @@ cleanlib:
         if OutPutFileList:
             for Item in OutPutFileList:
                 if Item in SourceFileList:
                     SourceFileList.remove(Item)
 
-        FileDependencyDict = self.GetFileDependency(
-                                    SourceFileList,
-                                    ForceIncludedFile,
-                                    self._AutoGenObject.IncludePathList + self._AutoGenObject.BuildOptionIncPathList
-                                    )
+        FileDependencyDict = {item:ForceIncludedFile for item in SourceFileList}
 
-
-        if FileDependencyDict:
-            for Dependency in FileDependencyDict.values():
-                self.DependencyHeaderFileSet.update(set(Dependency))
+        for Dependency in FileDependencyDict.values():
+            self.DependencyHeaderFileSet.update(set(Dependency))
 
         # Get a set of unique package includes from MetaFile
         parentMetaFileIncludes = set()
         for aInclude in self._AutoGenObject.PackageIncludePathList:
             aIncludeName = str(aInclude)
@@ -970,46 +965,22 @@ cleanlib:
                 GlobalData.gModuleBuildTracking[self._AutoGenObject] = 'FAIL_METAFILE'
             EdkLogger.warn("build","Module MetaFile [Sources] is missing local header!",
                         ExtraData = "Local Header: " + aFile + " not found in " + self._AutoGenObject.MetaFile.Path
                         )
 
-        DepSet = None
         for File,Dependency in FileDependencyDict.items():
             if not Dependency:
-                FileDependencyDict[File] = ['$(FORCE_REBUILD)']
+                FileDependencyDict[File] = ["$(COMMON_DEPS)"]
                 continue
+            FileDependencyDict[File] = ["$(COMMON_DEPS)"]
 
             self._AutoGenObject.AutoGenDepSet |= set(Dependency)
 
-            # skip non-C files
-            if File.Ext not in [".c", ".C"] or File.Name == "AutoGen.c":
-                continue
-            elif DepSet is None:
-                DepSet = set(Dependency)
-            else:
-                DepSet &= set(Dependency)
-        # in case nothing in SourceFileList
-        if DepSet is None:
-            DepSet = set()
-        #
-        # Extract common files list in the dependency files
-        #
-        for File in DepSet:
-            self.CommonFileDependency.append(self.PlaceMacro(File.Path, self.Macros))
-
         CmdSumDict = {}
         CmdTargetDict = {}
         CmdCppDict = {}
         DependencyDict = FileDependencyDict.copy()
-        for File in FileDependencyDict:
-            # skip non-C files
-            if File.Ext not in [".c", ".C"] or File.Name == "AutoGen.c":
-                continue
-            NewDepSet = set(FileDependencyDict[File])
-            NewDepSet -= DepSet
-            FileDependencyDict[File] = ["$(COMMON_DEPS)"] + list(NewDepSet)
-            DependencyDict[File] = list(NewDepSet)
 
         # Convert target description object to target string in makefile
         if self._AutoGenObject.BuildRuleFamily == TAB_COMPILER_MSFT and TAB_C_CODE_FILE in self._AutoGenObject.Targets:
             for T in self._AutoGenObject.Targets[TAB_C_CODE_FILE]:
                 NewFile = self.PlaceMacro(str(T), self.Macros)
@@ -1078,21 +1049,17 @@ cleanlib:
                     if CmdCppDict.get(item.Target.SubDir):
                         CmdCppDict[item.Target.SubDir].append(Path)
                     else:
                         CmdCppDict[item.Target.SubDir] = ['$(MAKE_FILE)', Path]
                     if CppPath.Path in DependencyDict:
-                        if '$(FORCE_REBUILD)' in DependencyDict[CppPath.Path]:
-                            if '$(FORCE_REBUILD)' not in (self.CommonFileDependency + CmdCppDict[item.Target.SubDir]):
-                                CmdCppDict[item.Target.SubDir].append('$(FORCE_REBUILD)')
-                        else:
-                            for Temp in DependencyDict[CppPath.Path]:
-                                try:
-                                    Path = self.PlaceMacro(Temp.Path, self.Macros)
-                                except:
-                                    continue
-                                if Path not in (self.CommonFileDependency + CmdCppDict[item.Target.SubDir]):
-                                    CmdCppDict[item.Target.SubDir].append(Path)
+                        for Temp in DependencyDict[CppPath.Path]:
+                            try:
+                                Path = self.PlaceMacro(Temp.Path, self.Macros)
+                            except:
+                                continue
+                            if Path not in (self.CommonFileDependency + CmdCppDict[item.Target.SubDir]):
+                                CmdCppDict[item.Target.SubDir].append(Path)
         if T.Commands:
             CommandList = T.Commands[:]
             for Item in CommandList[:]:
                 SingleCommandList = Item.split()
                 if len(SingleCommandList) > 0 and self.CheckCCCmd(SingleCommandList):
diff --git a/BaseTools/Source/Python/AutoGen/IncludesAutoGen.py b/BaseTools/Source/Python/AutoGen/IncludesAutoGen.py
new file mode 100644
index 0000000000..c64a75e8e4
--- /dev/null
+++ b/BaseTools/Source/Python/AutoGen/IncludesAutoGen.py
@@ -0,0 +1,99 @@
+## @file
+# Build cache intermediate result and state
+#
+# Copyright (c) 2019, Intel Corporation. All rights reserved.<BR>
+# SPDX-License-Identifier: BSD-2-Clause-Patent
+#
+from Common.caching import cached_property
+import Common.EdkLogger as EdkLogger
+import Common.LongFilePathOs as os
+from Common.BuildToolError import *
+from Common.Misc import SaveFileOnChange
+
+gIsFileMap = {}
+class IncludesAutoGen():
+    def __init__(self, makefile_folder, ModuleAuto):
+        self.d_folder = makefile_folder
+        self.makefile_folder = makefile_folder
+        self.module_autogen = ModuleAuto
+        self.ToolChainFamily = ModuleAuto.ToolChainFamily
+        self.workspace = ModuleAuto.WorkspaceDir
+    def CreateCommenDeps(self):
+        SaveFileOnChange(os.path.join(self.makefile_folder,"deps.txt"),"\n".join(self.DepsCollection),False)
+
+    @cached_property
+    def DepsCollection(self):
+        includes = set()
+
+        for root, _, files in os.walk(self.d_folder, topdown=False):
+            for name in files:
+                if not name.endswith(".d"):
+                    continue
+                abspath = os.path.join(root, name)
+                try:
+                    with open(abspath,"r") as fd:
+                        lines = fd.readlines()
+                    for item in lines:
+                        linestr = item.strip("\\  : \n")
+                        if not linestr:
+                            continue
+                        if os.path.normpath(linestr +".d") == abspath:
+                            continue
+                        filename = os.path.basename(linestr).strip()
+                        if filename in self.SourceFileList:
+                            continue
+                        if filename in self.TargetFileList:
+                            continue
+                        if filename.endswith(".o") and filename+"bj" in self.TargetFileList:
+                            continue
+                        includes.add(linestr.strip())
+                except Exception as e:
+                    EdkLogger.error("build",FILE_NOT_FOUND, "%s doesn't exist" % abspath, ExtraData=str(e), RaiseError=False)
+                    continue
+        rt = sorted(list(set(["      "+item.strip("\\ \n") + " \\" for item in includes])))
+        if rt:
+            rt[-1] = rt[-1].rstrip(" \\")
+            rt.insert(0,'COMMON_DEPS = \\')
+        return rt
+    @cached_property
+    def SourceFileList(self):
+        source = [os.path.basename(item.File) for item in self.module_autogen.SourceFileList]
+        middle_file = []
+        for afile in source:
+            if afile.upper().endswith(".VFR"):
+                middle_file.append(afile.split(".")[0]+".c")
+            if afile.upper().endswith((".S","ASM")):
+                middle_file.append(afile.split(".")[0]+".i")
+            if afile.upper().endswith(".ASL"):
+                middle_file.append(afile.split(".")[0]+".i")
+        source.append("AutoGen.c")
+        source.extend(middle_file)
+        return source
+    @cached_property
+    def TargetFileList(self):
+        targets = set()
+        targets.add("AutoGen.obj")
+        for item in self.module_autogen.Targets.values():
+            for block in item:
+                targets.add(block.Target.Name)
+        return targets
+    def CreateDepsFileForWin(self, DepList):
+        if not DepList:
+            return
+        ModuleDepDict = {}
+        current_source = ""
+        for line in DepList:
+            if line.strip() in self.SourceFileList:
+                current_source = line.strip()
+                ModuleDepDict[current_source] = []
+            elif "Note: including file:" ==  line.lstrip()[:21]:
+                if not current_source:
+                    EdkLogger.error("build",BUILD_ERROR, "Parse /showIncludes output failed. line: %s. \n" % line, RaiseError=False)
+                else:
+                    ModuleDepDict[current_source].append(line.lstrip()[22:].strip())
+            else:
+                current_source = ""
+
+        for source in ModuleDepDict:
+            dep_file_name = source + ".d"
+            SaveFileOnChange(os.path.join(self.makefile_folder,dep_file_name)," \\\n".join(ModuleDepDict[source]),False)
diff --git a/BaseTools/Source/Python/Trim/Trim.py b/BaseTools/Source/Python/Trim/Trim.py
index 24c3fafa76..593de12d39 100644
--- a/BaseTools/Source/Python/Trim/Trim.py
+++ b/BaseTools/Source/Python/Trim/Trim.py
@@ -54,10 +54,14 @@ gLongNumberPattern = re.compile("(?<=[^a-zA-Z0-9_])(0[xX][0-9a-fA-F]+|[0-9]+)U?L
 gAslIncludePattern = re.compile("^(\s*)[iI]nclude\s*\(\"?([^\"\(\)]+)\"\)", re.MULTILINE)
 ## Regular expression for matching C style #include "XXX.asl" in asl file
 gAslCIncludePattern = re.compile(r'^(\s*)#include\s*[<"]\s*([-\\/\w.]+)\s*([>"])', re.MULTILINE)
 ## Patterns used to convert EDK conventions to EDK2 ECP conventions
 
+## Regular expression for finding header file inclusions
+gIncludePattern = re.compile(r"^[ \t]*[#%]?[ \t]*include(?:[ \t]*(?:\\(?:\r\n|\r|\n))*[ \t]*)*(?:\(?[\"<]?[ \t]*)([-\w.\\/() \t]+)(?:[ \t]*[\">]?\)?)", re.MULTILINE | re.UNICODE | re.IGNORECASE)
+
+
 ## file cache to avoid circular include in ASL file
 gIncludedAslFile = []
 
 ## Trim preprocessed source code
 #
@@ -251,13 +255,14 @@ def TrimPreprocessedVfr(Source, Target):
 # @param  IncludePathList   The list of external include file
 # @param  LocalSearchPath   If LocalSearchPath is specified, this path will be searched
 #                           first for the included file; otherwise, only the path specified
 #                           in the IncludePathList will be searched.
 #
-def DoInclude(Source, Indent='', IncludePathList=[], LocalSearchPath=None):
+def DoInclude(Source, Indent='', IncludePathList=[], LocalSearchPath=None, IncludeFileList = None, filetype=None):
     NewFileContent = []
-
+    if IncludeFileList is None:
+        IncludeFileList = []
     try:
         #
         # Search LocalSearchPath first if it is specified.
         #
         if LocalSearchPath:
@@ -286,28 +291,41 @@ def DoInclude(Source, Indent='', IncludePathList=[], LocalSearchPath=None):
     if IncludeFile in gIncludedAslFile:
         EdkLogger.warn("Trim", "Circular include",
                        ExtraData= "%s -> %s" % (" -> ".join(gIncludedAslFile), IncludeFile))
         return []
     gIncludedAslFile.append(IncludeFile)
-
+    IncludeFileList.append(IncludeFile.strip())
     for Line in F:
         LocalSearchPath = None
-        Result = gAslIncludePattern.findall(Line)
-        if len(Result) == 0:
-            Result = gAslCIncludePattern.findall(Line)
-            if len(Result) == 0 or os.path.splitext(Result[0][1])[1].lower() not in [".asl", ".asi"]:
+        if filetype == "ASL":
+            Result = gAslIncludePattern.findall(Line)
+            if len(Result) == 0:
+                Result = gAslCIncludePattern.findall(Line)
+                if len(Result) == 0 or os.path.splitext(Result[0][1])[1].lower() not in [".asl", ".asi"]:
+                    NewFileContent.append("%s%s" % (Indent, Line))
+                    continue
+                #
+                # We should first search the local directory if current file are using pattern #include "XXX"
+                #
+                if Result[0][2] == '"':
+                    LocalSearchPath = os.path.dirname(IncludeFile)
+            CurrentIndent = Indent + Result[0][0]
+            IncludedFile = Result[0][1]
+            NewFileContent.extend(DoInclude(IncludedFile, CurrentIndent, IncludePathList, LocalSearchPath,IncludeFileList,filetype))
+            NewFileContent.append("\n")
+        elif filetype == "ASM":
+            Result = gIncludePattern.findall(Line)
+            if len(Result) == 0:
                 NewFileContent.append("%s%s" % (Indent, Line))
                 continue
-            #
-            # We should first search the local directory if current file are using pattern #include "XXX"
-            #
-            if Result[0][2] == '"':
-                LocalSearchPath = os.path.dirname(IncludeFile)
-        CurrentIndent = Indent + Result[0][0]
-        IncludedFile = Result[0][1]
-        NewFileContent.extend(DoInclude(IncludedFile, CurrentIndent, IncludePathList, LocalSearchPath))
-        NewFileContent.append("\n")
+
+            IncludedFile = Result[0]
+
+            IncludedFile = IncludedFile.strip()
+            IncludedFile = os.path.normpath(IncludedFile)
+            NewFileContent.extend(DoInclude(IncludedFile, '', IncludePathList, LocalSearchPath,IncludeFileList,filetype))
+            NewFileContent.append("\n")
 
     gIncludedAslFile.pop()
 
     return NewFileContent
 
@@ -318,22 +336,21 @@ def DoInclude(Source, Indent='', IncludePathList=[], LocalSearchPath=None):
 #
 # @param  Source          File to be trimmed
 # @param  Target          File to store the trimmed content
 # @param  IncludePathFile The file to log the external include path
 #
-def TrimAslFile(Source, Target, IncludePathFile):
+def TrimAslFile(Source, Target, IncludePathFile,AslDeps = False):
     CreateDirectory(os.path.dirname(Target))
 
     SourceDir = os.path.dirname(Source)
     if SourceDir == '':
         SourceDir = '.'
 
     #
     # Add source directory as the first search directory
     #
     IncludePathList = [SourceDir]
-
     #
     # If additional include path file is specified, append them all
     # to the search directory list.
     #
     if IncludePathFile:
@@ -347,12 +364,14 @@ def TrimAslFile(Source, Target, IncludePathFile):
                     IncludePathList.append(Line[2:].strip())
                 else:
                     EdkLogger.warn("Trim", "Invalid include line in include list file.", IncludePathFile, LineNum)
         except:
             EdkLogger.error("Trim", FILE_OPEN_FAILURE, ExtraData=IncludePathFile)
-
-    Lines = DoInclude(Source, '', IncludePathList)
+    AslIncludes = []
+    Lines = DoInclude(Source, '', IncludePathList,IncludeFileList=AslIncludes,filetype='ASL')
+    if AslDeps:
+        SaveFileOnChange(Target+".d", "\n".join(AslIncludes),False)
 
     #
     # Undef MIN and MAX to avoid collision in ASL source code
     #
     Lines.insert(0, "#undef MIN\n#undef MAX\n")
@@ -362,10 +381,56 @@ def TrimAslFile(Source, Target, IncludePathFile):
         with open(Target, 'w') as File:
             File.writelines(Lines)
     except:
         EdkLogger.error("Trim", FILE_OPEN_FAILURE, ExtraData=Target)
 
+## Trim ASM file
+#
+# Output ASM include statement with the content the included file
+#
+# @param  Source          File to be trimmed
+# @param  Target          File to store the trimmed content
+# @param  IncludePathFile The file to log the external include path
+#
+def TrimAsmFile(Source, Target, IncludePathFile):
+    CreateDirectory(os.path.dirname(Target))
+
+    SourceDir = os.path.dirname(Source)
+    if SourceDir == '':
+        SourceDir = '.'
+
+    #
+    # Add source directory as the first search directory
+    #
+    IncludePathList = [SourceDir]
+    #
+    # If additional include path file is specified, append them all
+    # to the search directory list.
+    #
+    if IncludePathFile:
+        try:
+            LineNum = 0
+            with open(IncludePathFile, 'r') as File:
+                FileLines = File.readlines()
+            for Line in FileLines:
+                LineNum += 1
+                if Line.startswith("/I") or Line.startswith ("-I"):
+                    IncludePathList.append(Line[2:].strip())
+                else:
+                    EdkLogger.warn("Trim", "Invalid include line in include list file.", IncludePathFile, LineNum)
+        except:
+            EdkLogger.error("Trim", FILE_OPEN_FAILURE, ExtraData=IncludePathFile)
+    AsmIncludes = []
+    Lines = DoInclude(Source, '', IncludePathList,IncludeFileList=AsmIncludes,filetype='ASM')
+    SaveFileOnChange(Target+".d", "\n".join(AsmIncludes),False)
+    # save all lines trimmed
+    try:
+        with open(Target, 'w') as File:
+            File.writelines(Lines)
+    except:
+        EdkLogger.error("Trim", FILE_OPEN_FAILURE, ExtraData=Target)
+
 def GenerateVfrBinSec(ModuleName, DebugDir, OutputFile):
     VfrNameList = []
     if os.path.isdir(DebugDir):
         for CurrentDir, Dirs, Files in os.walk(DebugDir):
             for FileName in Files:
@@ -438,12 +503,16 @@ def Options():
                           help="The input file is preprocessed source code, including C or assembly code"),
         make_option("-r", "--vfr-file", dest="FileType", const="Vfr", action="store_const",
                           help="The input file is preprocessed VFR file"),
         make_option("--Vfr-Uni-Offset", dest="FileType", const="VfrOffsetBin", action="store_const",
                           help="The input file is EFI image"),
+        make_option("--asl-deps", dest="AslDeps", const="True", action="store_const",
+                          help="Generate Asl dependent files."),
         make_option("-a", "--asl-file", dest="FileType", const="Asl", action="store_const",
                           help="The input file is ASL file"),
+        make_option( "--asm-file", dest="FileType", const="Asm", action="store_const",
+                          help="The input file is asm file"),
         make_option("-c", "--convert-hex", dest="ConvertHex", action="store_true",
                           help="Convert standard hex format (0xabcd) to MASM format (abcdh)"),
 
         make_option("-l", "--trim-long", dest="TrimLong", action="store_true",
                           help="Remove postfix of long number"),
@@ -513,13 +582,15 @@ def Main():
                 CommandOptions.OutputFile = os.path.splitext(InputFile)[0] + '.iii'
             TrimPreprocessedVfr(InputFile, CommandOptions.OutputFile)
         elif CommandOptions.FileType == "Asl":
             if CommandOptions.OutputFile is None:
                 CommandOptions.OutputFile = os.path.splitext(InputFile)[0] + '.iii'
-            TrimAslFile(InputFile, CommandOptions.OutputFile, CommandOptions.IncludePathFile)
+            TrimAslFile(InputFile, CommandOptions.OutputFile, CommandOptions.IncludePathFile,CommandOptions.AslDeps)
         elif CommandOptions.FileType == "VfrOffsetBin":
             GenerateVfrBinSec(CommandOptions.ModuleName, CommandOptions.DebugDir, CommandOptions.OutputFile)
+        elif CommandOptions.FileType == "Asm":
+            TrimAsmFile(InputFile, CommandOptions.OutputFile, CommandOptions.IncludePathFile)
         else :
             if CommandOptions.OutputFile is None:
                 CommandOptions.OutputFile = os.path.splitext(InputFile)[0] + '.iii'
             TrimPreprocessedFile(InputFile, CommandOptions.OutputFile, CommandOptions.ConvertHex, CommandOptions.TrimLong)
     except FatalError as X:
diff --git a/BaseTools/Source/Python/build/build.py b/BaseTools/Source/Python/build/build.py
index bcd832c525..8668aa315e 100755
--- a/BaseTools/Source/Python/build/build.py
+++ b/BaseTools/Source/Python/build/build.py
@@ -55,11 +55,11 @@ from GenFds.GenFds import GenFds, GenFdsApi
 import multiprocessing as mp
 from multiprocessing import Manager
 from AutoGen.DataPipe import MemoryDataPipe
 from AutoGen.ModuleAutoGenHelper import WorkSpaceInfo, PlatformInfo
 from GenFds.FdfParser import FdfParser
-
+from AutoGen.IncludesAutoGen import IncludesAutoGen
 
 ## standard targets of build command
 gSupportedTarget = ['all', 'genc', 'genmake', 'modules', 'libraries', 'fds', 'clean', 'cleanall', 'cleanlib', 'run']
 
 ## build configuration file
@@ -173,33 +173,46 @@ def NormFile(FilePath, Workspace):
 #
 # @param  From      The stream message read from
 # @param  To        The stream message put on
 # @param  ExitFlag  The flag used to indicate stopping reading
 #
-def ReadMessage(From, To, ExitFlag):
+def ReadMessage(From, To, ExitFlag,MemTo=None):
     while True:
         # read one line a time
         Line = From.readline()
         # empty string means "end"
         if Line is not None and Line != b"":
-            To(Line.rstrip().decode(encoding='utf-8', errors='ignore'))
+            LineStr = Line.rstrip().decode(encoding='utf-8', errors='ignore')
+            if MemTo is not None:
+                if "Note: including file:" ==  LineStr.lstrip()[:21]:
+                    MemTo.append(LineStr)
+                else:
+                    To(LineStr)
+                    MemTo.append(LineStr)
+            else:
+                To(LineStr)
         else:
             break
         if ExitFlag.isSet():
             break
 
+class MakeSubProc(Popen):
+    def __init__(self,*args, **argv):
+        super(MakeSubProc,self).__init__(*args, **argv)
+        self.ProcOut = []
+
 ## Launch an external program
 #
 # This method will call subprocess.Popen to execute an external program with
 # given options in specified directory. Because of the dead-lock issue during
 # redirecting output of the external program, threads are used to to do the
 # redirection work.
 #
 # @param  Command               A list or string containing the call of the program
 # @param  WorkingDir            The directory in which the program will be running
 #
-def LaunchCommand(Command, WorkingDir):
+def LaunchCommand(Command, WorkingDir,ModuleAuto = None):
     BeginTime = time.time()
     # if working directory doesn't exist, Popen() will raise an exception
     if not os.path.isdir(WorkingDir):
         EdkLogger.error("build", FILE_NOT_FOUND, ExtraData=WorkingDir)
 
@@ -214,23 +227,23 @@ def LaunchCommand(Command, WorkingDir):
 
     Proc = None
     EndOfProcedure = None
     try:
         # launch the command
-        Proc = Popen(Command, stdout=PIPE, stderr=PIPE, env=os.environ, cwd=WorkingDir, bufsize=-1, shell=True)
+        Proc = MakeSubProc(Command, stdout=PIPE, stderr=PIPE, env=os.environ, cwd=WorkingDir, bufsize=-1, shell=True)
 
         # launch two threads to read the STDOUT and STDERR
         EndOfProcedure = Event()
         EndOfProcedure.clear()
         if Proc.stdout:
-            StdOutThread = Thread(target=ReadMessage, args=(Proc.stdout, EdkLogger.info, EndOfProcedure))
+            StdOutThread = Thread(target=ReadMessage, args=(Proc.stdout, EdkLogger.info, EndOfProcedure,Proc.ProcOut))
             StdOutThread.setName("STDOUT-Redirector")
             StdOutThread.setDaemon(False)
             StdOutThread.start()
 
         if Proc.stderr:
-            StdErrThread = Thread(target=ReadMessage, args=(Proc.stderr, EdkLogger.quiet, EndOfProcedure))
+            StdErrThread = Thread(target=ReadMessage, args=(Proc.stderr, EdkLogger.quiet, EndOfProcedure,Proc.ProcOut))
             StdErrThread.setName("STDERR-Redirector")
             StdErrThread.setDaemon(False)
             StdErrThread.start()
 
         # waiting for program exit
@@ -261,10 +274,14 @@ def LaunchCommand(Command, WorkingDir):
             RespContent = f.read()
             f.close()
             EdkLogger.info(RespContent)
 
         EdkLogger.error("build", COMMAND_FAILURE, ExtraData="%s [%s]" % (Command, WorkingDir))
+    if ModuleAuto:
+        iau = IncludesAutoGen(WorkingDir,ModuleAuto)
+        iau.CreateDepsFileForWin(Proc.ProcOut)
+        iau.CreateCommenDeps()
     return "%dms" % (int(round((time.time() - BeginTime) * 1000)))
 
 ## The smallest unit that can be built in multi-thread build mode
 #
 # This is the base class of build unit. The "Obj" parameter must provide
@@ -606,11 +623,11 @@ class BuildTask:
     # @param  Command               A list or string contains the call of the command
     # @param  WorkingDir            The directory in which the program will be running
     #
     def _CommandThread(self, Command, WorkingDir):
         try:
-            self.BuildItem.BuildObject.BuildTime = LaunchCommand(Command, WorkingDir)
+            self.BuildItem.BuildObject.BuildTime = LaunchCommand(Command, WorkingDir,self.BuildItem.BuildObject)
             self.CompleteFlag = True
 
             # Run hash operation post dependency, to account for libs
             if GlobalData.gUseHashCache and self.BuildItem.BuildObject.IsLibrary:
                 HashFile = path.join(self.BuildItem.BuildObject.BuildDir, self.BuildItem.BuildObject.Name + ".hash")
@@ -1274,23 +1291,36 @@ class Build():
             self.BuildModules = []
             return True
 
         # build library
         if Target == 'libraries':
-            for Lib in AutoGenObject.LibraryBuildDirectoryList:
+            DirList = []
+            for Lib in AutoGenObject.LibraryAutoGenList:
+                if not Lib.IsBinaryModule:
+                    DirList.append((os.path.join(AutoGenObject.BuildDir, Lib.BuildDir),Lib))
+            for Lib, LibAutoGen in DirList:
                 NewBuildCommand = BuildCommand + ['-f', os.path.normpath(os.path.join(Lib, makefile)), 'pbuild']
-                LaunchCommand(NewBuildCommand, AutoGenObject.MakeFileDir)
+                LaunchCommand(NewBuildCommand, AutoGenObject.MakeFileDir,LibAutoGen)
             return True
 
         # build module
         if Target == 'modules':
-            for Lib in AutoGenObject.LibraryBuildDirectoryList:
+            DirList = []
+            for Lib in AutoGenObject.LibraryAutoGenList:
+                if not Lib.IsBinaryModule:
+                    DirList.append((os.path.join(AutoGenObject.BuildDir, Lib.BuildDir),Lib))
+            for Lib, LibAutoGen in DirList:
                 NewBuildCommand = BuildCommand + ['-f', os.path.normpath(os.path.join(Lib, makefile)), 'pbuild']
-                LaunchCommand(NewBuildCommand, AutoGenObject.MakeFileDir)
-            for Mod in AutoGenObject.ModuleBuildDirectoryList:
+                LaunchCommand(NewBuildCommand, AutoGenObject.MakeFileDir,LibAutoGen)
+
+            DirList = []
+            for ModuleAutoGen in AutoGenObject.ModuleAutoGenList:
+                if not ModuleAutoGen.IsBinaryModule:
+                    DirList.append((os.path.join(AutoGenObject.BuildDir, ModuleAutoGen.BuildDir),ModuleAutoGen))
+            for Mod,ModAutoGen in DirList:
                 NewBuildCommand = BuildCommand + ['-f', os.path.normpath(os.path.join(Mod, makefile)), 'pbuild']
-                LaunchCommand(NewBuildCommand, AutoGenObject.MakeFileDir)
+                LaunchCommand(NewBuildCommand, AutoGenObject.MakeFileDir,ModAutoGen)
             self.CreateAsBuiltInf()
             if GlobalData.gBinCacheDest:
                 self.UpdateBuildCache()
             self.BuildModules = []
             return True
-- 
2.20.1.windows.1


^ permalink raw reply related	[flat|nested] 9+ messages in thread

* Re: [edk2-devel] [Patch 0/1] BaseTools: Using compiler to generate source code dependency files.
  2019-10-28 10:47 [Patch 0/1] BaseTools: Using compiler to generate source code dependency files Bob Feng
  2019-10-28 10:47 ` [Patch 1/1] BaseTools: Generate source file dependency in Make phase Bob Feng
@ 2019-10-28 12:23 ` Ryszard Knop
  2019-10-28 14:56   ` Bob Feng
  1 sibling, 1 reply; 9+ messages in thread
From: Ryszard Knop @ 2019-10-28 12:23 UTC (permalink / raw)
  To: devel, bob.c.feng

Just a quick note: .d files are used by the D language. You might want 
to use an extension like .deps instead.

On 2019-10-28 11:47, Bob Feng wrote:
> BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=2311
>
> To support incremental build, build tool generates the dependent header
> file for each of source file. This procedure is done in AutoGen phase.
> The build tool goes through all the source file and header file and
> use regular expression to find out all the dependent files for a source
> file. This procedure is much time-consuming. And this method can't handle
> the MACRO in #include, for example #include PATH(xxx.h).
>
> This patch is going to use compiler to generate dependent files. This method
> will be faster and more accurate.
>
> The basic idea is:
> 1. In AutoGen phase, build tool add "!Include deps.txt" into Makefile instead
> of defining COMMON_DEPS list.
> 2. During the Make phase, the compilers, Trim and C preprocessor generate
> dependent files, .d file, for each source file.
> 3. After Make, The build tool combines the .d files and generate a file deps.txt
> which list all the included files for a module.
> 4. Each source file will depends on the Module's includes files. The difference
> with orignial behavior is that if the user
> change the source file, build tool will only build that source file in
> incremental build; while if the user change a module's header file, build tool
> will build the whole module in incremental build.
>
> In this way, the time of AutoGen phase will be reduced much. And since we
> will use c preprocessor to handle #include, the MACRO will be handled well
> and the final dependent files will be more accurate.
>
> Feng, Bob C (1):
>    BaseTools: Using compiler to generate source code dependency files.
>
>   BaseTools/Conf/build_rule.template            |  89 ++++++-----
>   BaseTools/Conf/tools_def.template             | 138 +++++++++---------
>   BaseTools/Source/Python/AutoGen/GenMake.py    |  73 +++------
>   .../Source/Python/AutoGen/IncludesAutoGen.py  |  99 +++++++++++++
>   BaseTools/Source/Python/Trim/Trim.py          | 113 +++++++++++---
>   BaseTools/Source/Python/build/build.py        |  58 ++++++--
>   6 files changed, 378 insertions(+), 192 deletions(-)
>   create mode 100644 BaseTools/Source/Python/AutoGen/IncludesAutoGen.py
>

^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: [edk2-devel] [Patch 0/1] BaseTools: Using compiler to generate source code dependency files.
  2019-10-28 12:23 ` [edk2-devel] [Patch 0/1] BaseTools: Using compiler to generate source code dependency files Ryszard Knop
@ 2019-10-28 14:56   ` Bob Feng
  2019-10-28 15:03     ` Yao, Jiewen
  0 siblings, 1 reply; 9+ messages in thread
From: Bob Feng @ 2019-10-28 14:56 UTC (permalink / raw)
  To: Ryszard Knop, devel@edk2.groups.io

Thanks for your comment. I think .d file should be fine since edk2 does not support D language.

Thanks,
Bob

-----Original Message-----
From: Ryszard Knop <ryszard.knop@linux.intel.com> 
Sent: Monday, October 28, 2019 8:24 PM
To: devel@edk2.groups.io; Feng, Bob C <bob.c.feng@intel.com>
Subject: Re: [edk2-devel] [Patch 0/1] BaseTools: Using compiler to generate source code dependency files.

Just a quick note: .d files are used by the D language. You might want to use an extension like .deps instead.

On 2019-10-28 11:47, Bob Feng wrote:
> BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=2311
>
> To support incremental build, build tool generates the dependent 
> header file for each of source file. This procedure is done in AutoGen phase.
> The build tool goes through all the source file and header file and 
> use regular expression to find out all the dependent files for a 
> source file. This procedure is much time-consuming. And this method 
> can't handle the MACRO in #include, for example #include PATH(xxx.h).
>
> This patch is going to use compiler to generate dependent files. This 
> method will be faster and more accurate.
>
> The basic idea is:
> 1. In AutoGen phase, build tool add "!Include deps.txt" into Makefile 
> instead of defining COMMON_DEPS list.
> 2. During the Make phase, the compilers, Trim and C preprocessor 
> generate dependent files, .d file, for each source file.
> 3. After Make, The build tool combines the .d files and generate a 
> file deps.txt which list all the included files for a module.
> 4. Each source file will depends on the Module's includes files. The 
> difference with orignial behavior is that if the user change the 
> source file, build tool will only build that source file in 
> incremental build; while if the user change a module's header file, 
> build tool will build the whole module in incremental build.
>
> In this way, the time of AutoGen phase will be reduced much. And since 
> we will use c preprocessor to handle #include, the MACRO will be 
> handled well and the final dependent files will be more accurate.
>
> Feng, Bob C (1):
>    BaseTools: Using compiler to generate source code dependency files.
>
>   BaseTools/Conf/build_rule.template            |  89 ++++++-----
>   BaseTools/Conf/tools_def.template             | 138 +++++++++---------
>   BaseTools/Source/Python/AutoGen/GenMake.py    |  73 +++------
>   .../Source/Python/AutoGen/IncludesAutoGen.py  |  99 +++++++++++++
>   BaseTools/Source/Python/Trim/Trim.py          | 113 +++++++++++---
>   BaseTools/Source/Python/build/build.py        |  58 ++++++--
>   6 files changed, 378 insertions(+), 192 deletions(-)
>   create mode 100644 
> BaseTools/Source/Python/AutoGen/IncludesAutoGen.py
>

^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: [edk2-devel] [Patch 0/1] BaseTools: Using compiler to generate source code dependency files.
  2019-10-28 14:56   ` Bob Feng
@ 2019-10-28 15:03     ` Yao, Jiewen
  2019-10-28 15:25       ` Bob Feng
  2019-10-28 17:18       ` Andrew Fish
  0 siblings, 2 replies; 9+ messages in thread
From: Yao, Jiewen @ 2019-10-28 15:03 UTC (permalink / raw)
  To: devel@edk2.groups.io, Feng, Bob C, Ryszard Knop

I think we need avoid confusing for future.
I don’t believe .d is good choice, since it is a known conflict.

Thank you
Yao Jiewen


> -----Original Message-----
> From: devel@edk2.groups.io <devel@edk2.groups.io> On Behalf Of Bob Feng
> Sent: Monday, October 28, 2019 10:57 PM
> To: Ryszard Knop <ryszard.knop@linux.intel.com>; devel@edk2.groups.io
> Subject: Re: [edk2-devel] [Patch 0/1] BaseTools: Using compiler to generate
> source code dependency files.
> 
> Thanks for your comment. I think .d file should be fine since edk2 does not
> support D language.
> 
> Thanks,
> Bob
> 
> -----Original Message-----
> From: Ryszard Knop <ryszard.knop@linux.intel.com>
> Sent: Monday, October 28, 2019 8:24 PM
> To: devel@edk2.groups.io; Feng, Bob C <bob.c.feng@intel.com>
> Subject: Re: [edk2-devel] [Patch 0/1] BaseTools: Using compiler to generate
> source code dependency files.
> 
> Just a quick note: .d files are used by the D language. You might want to use an
> extension like .deps instead.
> 
> On 2019-10-28 11:47, Bob Feng wrote:
> > BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=2311
> >
> > To support incremental build, build tool generates the dependent
> > header file for each of source file. This procedure is done in AutoGen phase.
> > The build tool goes through all the source file and header file and
> > use regular expression to find out all the dependent files for a
> > source file. This procedure is much time-consuming. And this method
> > can't handle the MACRO in #include, for example #include PATH(xxx.h).
> >
> > This patch is going to use compiler to generate dependent files. This
> > method will be faster and more accurate.
> >
> > The basic idea is:
> > 1. In AutoGen phase, build tool add "!Include deps.txt" into Makefile
> > instead of defining COMMON_DEPS list.
> > 2. During the Make phase, the compilers, Trim and C preprocessor
> > generate dependent files, .d file, for each source file.
> > 3. After Make, The build tool combines the .d files and generate a
> > file deps.txt which list all the included files for a module.
> > 4. Each source file will depends on the Module's includes files. The
> > difference with orignial behavior is that if the user change the
> > source file, build tool will only build that source file in
> > incremental build; while if the user change a module's header file,
> > build tool will build the whole module in incremental build.
> >
> > In this way, the time of AutoGen phase will be reduced much. And since
> > we will use c preprocessor to handle #include, the MACRO will be
> > handled well and the final dependent files will be more accurate.
> >
> > Feng, Bob C (1):
> >    BaseTools: Using compiler to generate source code dependency files.
> >
> >   BaseTools/Conf/build_rule.template            |  89 ++++++-----
> >   BaseTools/Conf/tools_def.template             | 138 +++++++++---------
> >   BaseTools/Source/Python/AutoGen/GenMake.py    |  73 +++------
> >   .../Source/Python/AutoGen/IncludesAutoGen.py  |  99 +++++++++++++
> >   BaseTools/Source/Python/Trim/Trim.py          | 113 +++++++++++---
> >   BaseTools/Source/Python/build/build.py        |  58 ++++++--
> >   6 files changed, 378 insertions(+), 192 deletions(-)
> >   create mode 100644
> > BaseTools/Source/Python/AutoGen/IncludesAutoGen.py
> >
> 
> 


^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: [edk2-devel] [Patch 0/1] BaseTools: Using compiler to generate source code dependency files.
  2019-10-28 15:03     ` Yao, Jiewen
@ 2019-10-28 15:25       ` Bob Feng
  2019-10-28 17:18       ` Andrew Fish
  1 sibling, 0 replies; 9+ messages in thread
From: Bob Feng @ 2019-10-28 15:25 UTC (permalink / raw)
  To: Yao, Jiewen, devel@edk2.groups.io, Ryszard Knop

OK. I'll change the .d in V2.

-----Original Message-----
From: Yao, Jiewen <jiewen.yao@intel.com> 
Sent: Monday, October 28, 2019 11:04 PM
To: devel@edk2.groups.io; Feng, Bob C <bob.c.feng@intel.com>; Ryszard Knop <ryszard.knop@linux.intel.com>
Subject: RE: [edk2-devel] [Patch 0/1] BaseTools: Using compiler to generate source code dependency files.

I think we need avoid confusing for future.
I don’t believe .d is good choice, since it is a known conflict.

Thank you
Yao Jiewen


> -----Original Message-----
> From: devel@edk2.groups.io <devel@edk2.groups.io> On Behalf Of Bob 
> Feng
> Sent: Monday, October 28, 2019 10:57 PM
> To: Ryszard Knop <ryszard.knop@linux.intel.com>; devel@edk2.groups.io
> Subject: Re: [edk2-devel] [Patch 0/1] BaseTools: Using compiler to 
> generate source code dependency files.
> 
> Thanks for your comment. I think .d file should be fine since edk2 
> does not support D language.
> 
> Thanks,
> Bob
> 
> -----Original Message-----
> From: Ryszard Knop <ryszard.knop@linux.intel.com>
> Sent: Monday, October 28, 2019 8:24 PM
> To: devel@edk2.groups.io; Feng, Bob C <bob.c.feng@intel.com>
> Subject: Re: [edk2-devel] [Patch 0/1] BaseTools: Using compiler to 
> generate source code dependency files.
> 
> Just a quick note: .d files are used by the D language. You might want 
> to use an extension like .deps instead.
> 
> On 2019-10-28 11:47, Bob Feng wrote:
> > BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=2311
> >
> > To support incremental build, build tool generates the dependent 
> > header file for each of source file. This procedure is done in AutoGen phase.
> > The build tool goes through all the source file and header file and 
> > use regular expression to find out all the dependent files for a 
> > source file. This procedure is much time-consuming. And this method 
> > can't handle the MACRO in #include, for example #include PATH(xxx.h).
> >
> > This patch is going to use compiler to generate dependent files. 
> > This method will be faster and more accurate.
> >
> > The basic idea is:
> > 1. In AutoGen phase, build tool add "!Include deps.txt" into 
> > Makefile instead of defining COMMON_DEPS list.
> > 2. During the Make phase, the compilers, Trim and C preprocessor 
> > generate dependent files, .d file, for each source file.
> > 3. After Make, The build tool combines the .d files and generate a 
> > file deps.txt which list all the included files for a module.
> > 4. Each source file will depends on the Module's includes files. The 
> > difference with orignial behavior is that if the user change the 
> > source file, build tool will only build that source file in 
> > incremental build; while if the user change a module's header file, 
> > build tool will build the whole module in incremental build.
> >
> > In this way, the time of AutoGen phase will be reduced much. And 
> > since we will use c preprocessor to handle #include, the MACRO will 
> > be handled well and the final dependent files will be more accurate.
> >
> > Feng, Bob C (1):
> >    BaseTools: Using compiler to generate source code dependency files.
> >
> >   BaseTools/Conf/build_rule.template            |  89 ++++++-----
> >   BaseTools/Conf/tools_def.template             | 138 +++++++++---------
> >   BaseTools/Source/Python/AutoGen/GenMake.py    |  73 +++------
> >   .../Source/Python/AutoGen/IncludesAutoGen.py  |  99 +++++++++++++
> >   BaseTools/Source/Python/Trim/Trim.py          | 113 +++++++++++---
> >   BaseTools/Source/Python/build/build.py        |  58 ++++++--
> >   6 files changed, 378 insertions(+), 192 deletions(-)
> >   create mode 100644
> > BaseTools/Source/Python/AutoGen/IncludesAutoGen.py
> >
> 
> 


^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: [edk2-devel] [Patch 0/1] BaseTools: Using compiler to generate source code dependency files.
  2019-10-28 15:03     ` Yao, Jiewen
  2019-10-28 15:25       ` Bob Feng
@ 2019-10-28 17:18       ` Andrew Fish
  2019-10-29  1:11         ` Bob Feng
  1 sibling, 1 reply; 9+ messages in thread
From: Andrew Fish @ 2019-10-28 17:18 UTC (permalink / raw)
  To: devel, Yao, Jiewen; +Cc: Feng, Bob C, Ryszard Knop

[-- Attachment #1: Type: text/plain, Size: 5588 bytes --]

The .d is the default file name extension used by GCC for dependency files. Given the dependency files are in the build output and the makefiles reference them explicitly I'm not sure there is going to be lots of confusion. 

But I think it is likely the default dependency file name is mostly just a convenience in constructing the makefiles, since if you use a custom name each invocation has to do string operations to create a file with a custom extension. Given our makefiles are generated by a tool this is probably not really an issue for us. 

Does Visual Studio have the concept of a dependency file? If yes what suffix does that use, as I guess we could use that? I guess the dependency handling could just be some XML data in some larger Visual Studio meta data....

GCC/clang flags around dependencies. 

-MD
 <>-MD is equivalent to -M -MF file, except that -E is not implied. The driver determines file based on whether an -o option is given. If it is, the driver uses its argument but with a suffix of .d, otherwise it takes the name of the input file, removes any directory components and suffix, and applies a .d suffix.

If -MD is used in conjunction with -E, any -o switch is understood to specify the dependency output file (see -MF <https://gcc.gnu.org/onlinedocs/gcc/Preprocessor-Options.html#dashMF>), but if used without -E, each -o is understood to specify a target object file.

Since -E is not implied, -MD can be used to generate a dependency output file as a side effect of the compilation process



-MF file
 <>When used with -M or -MM, specifies a file to write the dependencies to. If no -MF switch is given the preprocessor sends the rules to the same place it would send preprocessed output.

When used with the driver options -MD or -MMD, -MF overrides the default dependency output file.

If file is -, then the dependencies are written to stdout.


Thanks,

Andrew Fish

> On Oct 28, 2019, at 8:03 AM, Yao, Jiewen <jiewen.yao@intel.com> wrote:
> 
> I think we need avoid confusing for future.
> I don’t believe .d is good choice, since it is a known conflict.
> 
> Thank you
> Yao Jiewen
> 
> 
>> -----Original Message-----
>> From: devel@edk2.groups.io <mailto:devel@edk2.groups.io> <devel@edk2.groups.io <mailto:devel@edk2.groups.io>> On Behalf Of Bob Feng
>> Sent: Monday, October 28, 2019 10:57 PM
>> To: Ryszard Knop <ryszard.knop@linux.intel.com <mailto:ryszard.knop@linux.intel.com>>; devel@edk2.groups.io <mailto:devel@edk2.groups.io>
>> Subject: Re: [edk2-devel] [Patch 0/1] BaseTools: Using compiler to generate
>> source code dependency files.
>> 
>> Thanks for your comment. I think .d file should be fine since edk2 does not
>> support D language.
>> 
>> Thanks,
>> Bob
>> 
>> -----Original Message-----
>> From: Ryszard Knop <ryszard.knop@linux.intel.com>
>> Sent: Monday, October 28, 2019 8:24 PM
>> To: devel@edk2.groups.io; Feng, Bob C <bob.c.feng@intel.com>
>> Subject: Re: [edk2-devel] [Patch 0/1] BaseTools: Using compiler to generate
>> source code dependency files.
>> 
>> Just a quick note: .d files are used by the D language. You might want to use an
>> extension like .deps instead.
>> 
>> On 2019-10-28 11:47, Bob Feng wrote:
>>> BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=2311
>>> 
>>> To support incremental build, build tool generates the dependent
>>> header file for each of source file. This procedure is done in AutoGen phase.
>>> The build tool goes through all the source file and header file and
>>> use regular expression to find out all the dependent files for a
>>> source file. This procedure is much time-consuming. And this method
>>> can't handle the MACRO in #include, for example #include PATH(xxx.h).
>>> 
>>> This patch is going to use compiler to generate dependent files. This
>>> method will be faster and more accurate.
>>> 
>>> The basic idea is:
>>> 1. In AutoGen phase, build tool add "!Include deps.txt" into Makefile
>>> instead of defining COMMON_DEPS list.
>>> 2. During the Make phase, the compilers, Trim and C preprocessor
>>> generate dependent files, .d file, for each source file.
>>> 3. After Make, The build tool combines the .d files and generate a
>>> file deps.txt which list all the included files for a module.
>>> 4. Each source file will depends on the Module's includes files. The
>>> difference with orignial behavior is that if the user change the
>>> source file, build tool will only build that source file in
>>> incremental build; while if the user change a module's header file,
>>> build tool will build the whole module in incremental build.
>>> 
>>> In this way, the time of AutoGen phase will be reduced much. And since
>>> we will use c preprocessor to handle #include, the MACRO will be
>>> handled well and the final dependent files will be more accurate.
>>> 
>>> Feng, Bob C (1):
>>>   BaseTools: Using compiler to generate source code dependency files.
>>> 
>>>  BaseTools/Conf/build_rule.template            |  89 ++++++-----
>>>  BaseTools/Conf/tools_def.template             | 138 +++++++++---------
>>>  BaseTools/Source/Python/AutoGen/GenMake.py    |  73 +++------
>>>  .../Source/Python/AutoGen/IncludesAutoGen.py  |  99 +++++++++++++
>>>  BaseTools/Source/Python/Trim/Trim.py          | 113 +++++++++++---
>>>  BaseTools/Source/Python/build/build.py        |  58 ++++++--
>>>  6 files changed, 378 insertions(+), 192 deletions(-)
>>>  create mode 100644
>>> BaseTools/Source/Python/AutoGen/IncludesAutoGen.py
>>> 
>> 
>> 
> 
> 
> 


[-- Attachment #2: Type: text/html, Size: 14387 bytes --]

^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: [edk2-devel] [Patch 0/1] BaseTools: Using compiler to generate source code dependency files.
  2019-10-28 17:18       ` Andrew Fish
@ 2019-10-29  1:11         ` Bob Feng
  2019-10-29  4:11           ` Andrew Fish
  0 siblings, 1 reply; 9+ messages in thread
From: Bob Feng @ 2019-10-29  1:11 UTC (permalink / raw)
  To: afish@apple.com, devel@edk2.groups.io, Yao, Jiewen; +Cc: Ryszard Knop

[-- Attachment #1: Type: text/plain, Size: 6239 bytes --]

Yes. For Gcc and Clang, we can use –MMD –MF <filename.deps> to generate custom name dependency file for each source file. For example, gcc main.c  –MMD –MF main.deps.

For MSVC and Intel compiler, there is a build option /showIncludes which makes compiler print the dependency files on stdout but not a file. The build tool will capture the message from stdout and generate .deps files on file system. The format will be the same as GCC generate.

Thanks,
Bob

From: afish@apple.com [mailto:afish@apple.com]
Sent: Tuesday, October 29, 2019 1:19 AM
To: devel@edk2.groups.io; Yao, Jiewen <jiewen.yao@intel.com>
Cc: Feng, Bob C <bob.c.feng@intel.com>; Ryszard Knop <ryszard.knop@linux.intel.com>
Subject: Re: [edk2-devel] [Patch 0/1] BaseTools: Using compiler to generate source code dependency files.

The .d is the default file name extension used by GCC for dependency files. Given the dependency files are in the build output and the makefiles reference them explicitly I'm not sure there is going to be lots of confusion.

But I think it is likely the default dependency file name is mostly just a convenience in constructing the makefiles, since if you use a custom name each invocation has to do string operations to create a file with a custom extension. Given our makefiles are generated by a tool this is probably not really an issue for us.

Does Visual Studio have the concept of a dependency file? If yes what suffix does that use, as I guess we could use that? I guess the dependency handling could just be some XML data in some larger Visual Studio meta data....

GCC/clang flags around dependencies.

-MD
-MD is equivalent to -M -MF file, except that -E is not implied. The driver determines file based on whether an -o option is given. If it is, the driver uses its argument but with a suffix of .d, otherwise it takes the name of the input file, removes any directory components and suffix, and applies a .d suffix.
If -MD is used in conjunction with -E, any -o switch is understood to specify the dependency output file (see -MF<https://gcc.gnu.org/onlinedocs/gcc/Preprocessor-Options.html#dashMF>), but if used without -E, each -o is understood to specify a target object file.
Since -E is not implied, -MD can be used to generate a dependency output file as a side effect of the compilation process


-MF file
When used with -M or -MM, specifies a file to write the dependencies to. If no -MF switch is given the preprocessor sends the rules to the same place it would send preprocessed output.
When used with the driver options -MD or -MMD, -MF overrides the default dependency output file.
If file is -, then the dependencies are written to stdout.

Thanks,

Andrew Fish


On Oct 28, 2019, at 8:03 AM, Yao, Jiewen <jiewen.yao@intel.com<mailto:jiewen.yao@intel.com>> wrote:

I think we need avoid confusing for future.
I don’t believe .d is good choice, since it is a known conflict.

Thank you
Yao Jiewen



-----Original Message-----
From: devel@edk2.groups.io<mailto:devel@edk2.groups.io> <devel@edk2.groups.io<mailto:devel@edk2.groups.io>> On Behalf Of Bob Feng
Sent: Monday, October 28, 2019 10:57 PM
To: Ryszard Knop <ryszard.knop@linux.intel.com<mailto:ryszard.knop@linux.intel.com>>; devel@edk2.groups.io<mailto:devel@edk2.groups.io>
Subject: Re: [edk2-devel] [Patch 0/1] BaseTools: Using compiler to generate
source code dependency files.

Thanks for your comment. I think .d file should be fine since edk2 does not
support D language.

Thanks,
Bob

-----Original Message-----
From: Ryszard Knop <ryszard.knop@linux.intel.com<mailto:ryszard.knop@linux.intel.com>>
Sent: Monday, October 28, 2019 8:24 PM
To: devel@edk2.groups.io<mailto:devel@edk2.groups.io>; Feng, Bob C <bob.c.feng@intel.com<mailto:bob.c.feng@intel.com>>
Subject: Re: [edk2-devel] [Patch 0/1] BaseTools: Using compiler to generate
source code dependency files.

Just a quick note: .d files are used by the D language. You might want to use an
extension like .deps instead.

On 2019-10-28 11:47, Bob Feng wrote:

BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=2311

To support incremental build, build tool generates the dependent
header file for each of source file. This procedure is done in AutoGen phase.
The build tool goes through all the source file and header file and
use regular expression to find out all the dependent files for a
source file. This procedure is much time-consuming. And this method
can't handle the MACRO in #include, for example #include PATH(xxx.h).

This patch is going to use compiler to generate dependent files. This
method will be faster and more accurate.

The basic idea is:
1. In AutoGen phase, build tool add "!Include deps.txt" into Makefile
instead of defining COMMON_DEPS list.
2. During the Make phase, the compilers, Trim and C preprocessor
generate dependent files, .d file, for each source file.
3. After Make, The build tool combines the .d files and generate a
file deps.txt which list all the included files for a module.
4. Each source file will depends on the Module's includes files. The
difference with orignial behavior is that if the user change the
source file, build tool will only build that source file in
incremental build; while if the user change a module's header file,
build tool will build the whole module in incremental build.

In this way, the time of AutoGen phase will be reduced much. And since
we will use c preprocessor to handle #include, the MACRO will be
handled well and the final dependent files will be more accurate.

Feng, Bob C (1):
  BaseTools: Using compiler to generate source code dependency files.

 BaseTools/Conf/build_rule.template            |  89 ++++++-----
 BaseTools/Conf/tools_def.template             | 138 +++++++++---------
 BaseTools/Source/Python/AutoGen/GenMake.py    |  73 +++------
 .../Source/Python/AutoGen/IncludesAutoGen.py  |  99 +++++++++++++
 BaseTools/Source/Python/Trim/Trim.py          | 113 +++++++++++---
 BaseTools/Source/Python/build/build.py        |  58 ++++++--
 6 files changed, 378 insertions(+), 192 deletions(-)
 create mode 100644
BaseTools/Source/Python/AutoGen/IncludesAutoGen.py






[-- Attachment #2: Type: text/html, Size: 17648 bytes --]

^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: [edk2-devel] [Patch 0/1] BaseTools: Using compiler to generate source code dependency files.
  2019-10-29  1:11         ` Bob Feng
@ 2019-10-29  4:11           ` Andrew Fish
  0 siblings, 0 replies; 9+ messages in thread
From: Andrew Fish @ 2019-10-29  4:11 UTC (permalink / raw)
  To: Feng, Bob C; +Cc: devel@edk2.groups.io, Yao, Jiewen, Ryszard Knop

[-- Attachment #1: Type: text/plain, Size: 6409 bytes --]

Bob,

I was trying to point out the .d file extension also implies a C dependency file in Gcc world.  So to me I would have picked .d. But other choices are OK too.
> On Oct 28, 2019, at 6:12 PM, Feng, Bob C <bob.c.feng@intel.com> wrote:
> 
> 
> Yes. For Gcc and Clang, we can use –MMD –MF <filename.deps> to generate custom name dependency file for each source file. For example, gcc main.c  –MMD –MF main.deps. 
>
> For MSVC and Intel compiler, there is a build option /showIncludes which makes compiler print the dependency files on stdout but not a file. The build tool will capture the message from stdout and generate .deps files on file system. The format will be the same as GCC generate.
>
> Thanks,
> Bob
>
> From: afish@apple.com [mailto:afish@apple.com] 
> Sent: Tuesday, October 29, 2019 1:19 AM
> To: devel@edk2.groups.io; Yao, Jiewen <jiewen.yao@intel.com>
> Cc: Feng, Bob C <bob.c.feng@intel.com>; Ryszard Knop <ryszard.knop@linux.intel.com>
> Subject: Re: [edk2-devel] [Patch 0/1] BaseTools: Using compiler to generate source code dependency files.
>
> The .d is the default file name extension used by GCC for dependency files. Given the dependency files are in the build output and the makefiles reference them explicitly I'm not sure there is going to be lots of confusion. 
>
> But I think it is likely the default dependency file name is mostly just a convenience in constructing the makefiles, since if you use a custom name each invocation has to do string operations to create a file with a custom extension. Given our makefiles are generated by a tool this is probably not really an issue for us. 
>
> Does Visual Studio have the concept of a dependency file? If yes what suffix does that use, as I guess we could use that? I guess the dependency handling could just be some XML data in some larger Visual Studio meta data....
>
> GCC/clang flags around dependencies. 
>
> -MD
> -MD is equivalent to -M -MF file, except that -E is not implied. The driver determines file based on whether an -o option is given. If it is, the driver uses its argument but with a suffix of .d, otherwise it takes the name of the input file, removes any directory components and suffix, and applies a .d suffix.
> If -MD is used in conjunction with -E, any -o switch is understood to specify the dependency output file (see -MF), but if used without -E, each -o is understood to specify a target object file.
> Since -E is not implied, -MD can be used to generate a dependency output file as a side effect of the compilation process
>
>
> -MF file
> When used with -M or -MM, specifies a file to write the dependencies to. If no -MF switch is given the preprocessor sends the rules to the same place it would send preprocessed output.
> When used with the driver options -MD or -MMD, -MF overrides the default dependency output file.
> If file is -, then the dependencies are written to stdout.
>
> Thanks,
>
> Andrew Fish
> 
> 
> On Oct 28, 2019, at 8:03 AM, Yao, Jiewen <jiewen.yao@intel.com> wrote:
>
> I think we need avoid confusing for future.
> I don’t believe .d is good choice, since it is a known conflict.
> 
> Thank you
> Yao Jiewen
> 
> 
> 
> -----Original Message-----
> From: devel@edk2.groups.io <devel@edk2.groups.io> On Behalf Of Bob Feng
> Sent: Monday, October 28, 2019 10:57 PM
> To: Ryszard Knop <ryszard.knop@linux.intel.com>; devel@edk2.groups.io
> Subject: Re: [edk2-devel] [Patch 0/1] BaseTools: Using compiler to generate
> source code dependency files.
> 
> Thanks for your comment. I think .d file should be fine since edk2 does not
> support D language.
> 
> Thanks,
> Bob
> 
> -----Original Message-----
> From: Ryszard Knop <ryszard.knop@linux.intel.com>
> Sent: Monday, October 28, 2019 8:24 PM
> To: devel@edk2.groups.io; Feng, Bob C <bob.c.feng@intel.com>
> Subject: Re: [edk2-devel] [Patch 0/1] BaseTools: Using compiler to generate
> source code dependency files.
> 
> Just a quick note: .d files are used by the D language. You might want to use an
> extension like .deps instead.
> 
> On 2019-10-28 11:47, Bob Feng wrote:
> 
> BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=2311
> 
> To support incremental build, build tool generates the dependent
> header file for each of source file. This procedure is done in AutoGen phase.
> The build tool goes through all the source file and header file and
> use regular expression to find out all the dependent files for a
> source file. This procedure is much time-consuming. And this method
> can't handle the MACRO in #include, for example #include PATH(xxx.h).
> 
> This patch is going to use compiler to generate dependent files. This
> method will be faster and more accurate.
> 
> The basic idea is:
> 1. In AutoGen phase, build tool add "!Include deps.txt" into Makefile
> instead of defining COMMON_DEPS list.
> 2. During the Make phase, the compilers, Trim and C preprocessor
> generate dependent files, .d file, for each source file.
> 3. After Make, The build tool combines the .d files and generate a
> file deps.txt which list all the included files for a module.
> 4. Each source file will depends on the Module's includes files. The
> difference with orignial behavior is that if the user change the
> source file, build tool will only build that source file in
> incremental build; while if the user change a module's header file,
> build tool will build the whole module in incremental build.
> 
> In this way, the time of AutoGen phase will be reduced much. And since
> we will use c preprocessor to handle #include, the MACRO will be
> handled well and the final dependent files will be more accurate.
> 
> Feng, Bob C (1):
>   BaseTools: Using compiler to generate source code dependency files.
> 
>  BaseTools/Conf/build_rule.template            |  89 ++++++-----
>  BaseTools/Conf/tools_def.template             | 138 +++++++++---------
>  BaseTools/Source/Python/AutoGen/GenMake.py    |  73 +++------
>  .../Source/Python/AutoGen/IncludesAutoGen.py  |  99 +++++++++++++
>  BaseTools/Source/Python/Trim/Trim.py          | 113 +++++++++++---
>  BaseTools/Source/Python/build/build.py        |  58 ++++++--
>  6 files changed, 378 insertions(+), 192 deletions(-)
>  create mode 100644
> BaseTools/Source/Python/AutoGen/IncludesAutoGen.py
> 
>
> 
> 
> 
> 
>

[-- Attachment #2: Type: text/html, Size: 17674 bytes --]

^ permalink raw reply	[flat|nested] 9+ messages in thread

end of thread, other threads:[~2019-10-29  4:11 UTC | newest]

Thread overview: 9+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2019-10-28 10:47 [Patch 0/1] BaseTools: Using compiler to generate source code dependency files Bob Feng
2019-10-28 10:47 ` [Patch 1/1] BaseTools: Generate source file dependency in Make phase Bob Feng
2019-10-28 12:23 ` [edk2-devel] [Patch 0/1] BaseTools: Using compiler to generate source code dependency files Ryszard Knop
2019-10-28 14:56   ` Bob Feng
2019-10-28 15:03     ` Yao, Jiewen
2019-10-28 15:25       ` Bob Feng
2019-10-28 17:18       ` Andrew Fish
2019-10-29  1:11         ` Bob Feng
2019-10-29  4:11           ` Andrew Fish

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox