Re: [Patch V2] Build spec: add description for build with binary cache

["Gao, Liming" <liming.gao@intel.com>]

Laszlo:
  Hash way may improve the incremental build performance. If hash value is not changed, module AutoGen and Make will be skipped. 
  Time stamp way bases on Makefile. This way will run AutoGen every time. If no change, Makefile will not be updated.  

Thanks
Liming
>-----Original Message-----
>From: Laszlo Ersek [mailto:lersek@redhat.com]
>Sent: Thursday, September 28, 2017 4:19 PM
>To: Zhu, Yonghong <yonghong.zhu@intel.com>; edk2-devel@lists.01.org
>Cc: Kinney, Michael D <michael.d.kinney@intel.com>; Shaw, Kevin W
><kevin.w.shaw@intel.com>; Gao, Liming <liming.gao@intel.com>
>Subject: Re: [edk2] [Patch V2] Build spec: add description for build with binary
>cache
>
>On 09/19/17 08:48, Yonghong Zhu wrote:
>> V2:
>> change the option name to --binary-destination and --binary-source.
>>
>> fixes:https://bugzilla.tianocore.org/show_bug.cgi?id=689
>
>What are the benefits of a hash-based incremental build over a
>timestamp-based incremental build?
>
>Thank you,
>Laszlo
>
>> Cc: Liming Gao <liming.gao@intel.com>
>> Cc: Michael Kinney <michael.d.kinney@intel.com>
>> Cc: Kevin W Shaw <kevin.w.shaw@intel.com>
>> Contributed-under: TianoCore Contribution Agreement 1.1
>> Signed-off-by: Yonghong Zhu <yonghong.zhu@intel.com>
>> ---
>>  .../82_auto-generation_process.md                    | 20
>++++++++++++++++++++
>>  README.md                                            |  1 +
>>  appendix_d_buildexe_command/d4_usage.md              | 19
>+++++++++++++++----
>>  3 files changed, 36 insertions(+), 4 deletions(-)
>>
>> diff --git a/8_pre-build_autogen_stage/82_auto-generation_process.md
>b/8_pre-build_autogen_stage/82_auto-generation_process.md
>> index 671a7d5..f2ddf32 100644
>> --- a/8_pre-build_autogen_stage/82_auto-generation_process.md
>> +++ b/8_pre-build_autogen_stage/82_auto-generation_process.md
>> @@ -1031,10 +1031,30 @@ maximum command line length.  The default
>value is 4096.
>>  **Note:** The following `FLAGS` options are included in the response file:
>>  `PP_FLAGS`, `CC_FLAGS`, `VFRPP_FLAGS`, `APP_FLAGS`, `ASLPP_FLAGS`,
>`ASLCC_FLAGS`,
>>  and `ASM_FLAGS`.
>>  **********
>>
>> +#### 8.2.4.15 Build with Binary Cache
>> +
>> +**build** tool provides three new options for binary cache feature.
>> +--hash enables hash-based caching during build process. when --hash is
>enabled,
>> +build tool will base on the module hash value to do the incremental build,
>without
>> +--hash, build tool will base on the timestamp to do the incremental build. --
>hash
>> +option use md5 method to get every hash value, DSC/FDF, tools_def.txt,
>build_rule.txt
>> +and build command are calculated as global hash value, Package DEC and its
>include
>> +header files are calculated as package hash value, Module source files and
>its INF
>> +file are calculated as module hash value. Library hash value will combine
>the global
>> +hash value and its dependent package hash value. Driver hash value will
>combine the
>> +global hash value, its dependent package hash value and its linked library
>hash value.
>> +When --hash and --binary-destination are specified, build tool will copy the
>generated
>> +binary files for each module into the directory specified by binary-
>destination at the
>> +build phase. Binary-destination directory caches all the generated binary
>files.
>> +When --hash and --binary-source are specified, build tool will try to get the
>binary
>> +files from the binary source directory at the build phase. If the cached
>binary has
>> +the same hash value, it will be directly used. Otherwise, build tool will
>compile the
>> +source files and generate the binary files.
>> +
>>  ### 8.2.5 Post processing
>>
>>  Once all files are parsed, the build tools will do following work for each EDK
>>  II module:
>>
>> diff --git a/README.md b/README.md
>> index 52abb6a..ca59a35 100644
>> --- a/README.md
>> +++ b/README.md
>> @@ -215,5 +215,6 @@ Copyright (c) 2008-2017, Intel Corporation. All rights
>reserved.
>>  |            | [#523](https://bugzilla.tianocore.org/show_bug.cgi?id=523) Build
>spec: add EBNF for the --pcd syntax in the Section D.4
>|               |
>>  |            | [#517](https://bugzilla.tianocore.org/show_bug.cgi?id=517) Build
>spec: chapter 5.2.2 Guided Tools add description for Pkcs7Sign tool and
>BrotliCompress tool
>|               |
>>  |            | [#481](https://bugzilla.tianocore.org/show_bug.cgi?id=481) Build
>Spec: add clarification for not used Pcd that build tool will not do additional
>checks on its value
>|               |
>>  |            | [#518](https://bugzilla.tianocore.org/show_bug.cgi?id=518) Build
>Spec: Update Precedence of PCD Values
>|               |
>>  |            | [#669](https://bugzilla.tianocore.org/show_bug.cgi?id=669) Build
>Spec: Add multi-arg support to PREBUILD/POSTBUILD
>|               |
>> +|            | [#689](https://bugzilla.tianocore.org/show_bug.cgi?id=689) Build
>spec: add description for build with binary cache
>|               |
>> diff --git a/appendix_d_buildexe_command/d4_usage.md
>b/appendix_d_buildexe_command/d4_usage.md
>> index b71f2d0..c901266 100644
>> --- a/appendix_d_buildexe_command/d4_usage.md
>> +++ b/appendix_d_buildexe_command/d4_usage.md
>> @@ -32,19 +32,20 @@
>>  ## D.4 Usage
>>
>>  ```ini
>>  Usage: build.exe [options]
>>  [all|fds|genc|genmake|clean|cleanall|cleanlib|modules|libraries|run]
>> -Copyright (c) 2007 - 2014, Intel Corporation All rights reserved.
>> +Copyright (c) 2007 - 2017, Intel Corporation All rights reserved.
>>
>>  Options:
>>    --version             show program's version number and exit
>>    -h, --help            show this help message and exit
>>    -a TARGETARCH, --arch=TARGETARCH
>> -                        ARCHS is one of list: IA32, X64, IPF, ARM, or EBC,
>> -                        which overrides target.txt's TARGET_ARCH definition.
>> -                        To specify more archs, please repeat this option.
>> +                        ARCHS is one of list: IA32, X64, IPF, ARM, AARCH64 or
>> +                        EBC, which overrides target.txt's TARGET_ARCH
>> +                        definition. To specify more archs, please repeat this
>> +                        option.
>>    -p PLATFORMFILE, --platform=PLATFORMFILE
>>                          Build the platform specified by the DSC file name
>>                          argument, overriding target.txt's ACTIVE_PLATFORM
>>                          definition.
>>    -m MODULEFILE, --module=MODULEFILE
>> @@ -112,10 +113,20 @@ Options:
>>    -N, --no-cache        Disable build cache mechanism
>>    --conf=CONFDIRECTORY  Specify the customized Conf directory.
>>    --check-usage         Check usage content of entries listed in INF file.
>>    --ignore-sources      Focus to a binary build and ignore all source files
>>    --pcd=OPTIONPCD       Set PCD value by command line. Format:
>"PcdName=Value"
>> +  -l COMMANDLENGTH, --cmd-len=COMMANDLENGTH
>> +                        Specify the maximum line length of build command.
>> +                        Default is 4096.
>> +  --hash                Enable hash-based caching during build process.
>> +  --binary-destination=BINCACHEDEST
>> +                        Generate a cache of binary files in the specified
>> +                        directory.
>> +  --binary-source=BINCACHESOURCE
>> +                        Consume a cache of binary files from the specified
>> +                        directory.
>>  ```
>>
>>  ### D.4.1 Debug Levels
>>
>>  The numeric debug levels are defined as integer values 0-9.
>>