Re: [Patch V2] Build spec: add description for build with binary cache

[Laszlo Ersek <lersek@redhat.com>]

On 09/19/17 08:48, Yonghong Zhu wrote:
> V2:
> change the option name to --binary-destination and --binary-source.
> 
> fixes:https://bugzilla.tianocore.org/show_bug.cgi?id=689

What are the benefits of a hash-based incremental build over a
timestamp-based incremental build?

Thank you,
Laszlo

> Cc: Liming Gao <liming.gao@intel.com>
> Cc: Michael Kinney <michael.d.kinney@intel.com>
> Cc: Kevin W Shaw <kevin.w.shaw@intel.com>
> Contributed-under: TianoCore Contribution Agreement 1.1
> Signed-off-by: Yonghong Zhu <yonghong.zhu@intel.com>
> ---
>  .../82_auto-generation_process.md                    | 20 ++++++++++++++++++++
>  README.md                                            |  1 +
>  appendix_d_buildexe_command/d4_usage.md              | 19 +++++++++++++++----
>  3 files changed, 36 insertions(+), 4 deletions(-)
> 
> diff --git a/8_pre-build_autogen_stage/82_auto-generation_process.md b/8_pre-build_autogen_stage/82_auto-generation_process.md
> index 671a7d5..f2ddf32 100644
> --- a/8_pre-build_autogen_stage/82_auto-generation_process.md
> +++ b/8_pre-build_autogen_stage/82_auto-generation_process.md
> @@ -1031,10 +1031,30 @@ maximum command line length.  The default value is 4096.
>  **Note:** The following `FLAGS` options are included in the response file:
>  `PP_FLAGS`, `CC_FLAGS`, `VFRPP_FLAGS`, `APP_FLAGS`, `ASLPP_FLAGS`, `ASLCC_FLAGS`,
>  and `ASM_FLAGS`.
>  **********
>  
> +#### 8.2.4.15 Build with Binary Cache
> +
> +**build** tool provides three new options for binary cache feature.
> +--hash enables hash-based caching during build process. when --hash is enabled,
> +build tool will base on the module hash value to do the incremental build, without
> +--hash, build tool will base on the timestamp to do the incremental build. --hash
> +option use md5 method to get every hash value, DSC/FDF, tools_def.txt, build_rule.txt
> +and build command are calculated as global hash value, Package DEC and its include
> +header files are calculated as package hash value, Module source files and its INF
> +file are calculated as module hash value. Library hash value will combine the global
> +hash value and its dependent package hash value. Driver hash value will combine the
> +global hash value, its dependent package hash value and its linked library hash value.
> +When --hash and --binary-destination are specified, build tool will copy the generated
> +binary files for each module into the directory specified by binary-destination at the
> +build phase. Binary-destination directory caches all the generated binary files.
> +When --hash and --binary-source are specified, build tool will try to get the binary
> +files from the binary source directory at the build phase. If the cached binary has
> +the same hash value, it will be directly used. Otherwise, build tool will compile the
> +source files and generate the binary files.
> +
>  ### 8.2.5 Post processing
>  
>  Once all files are parsed, the build tools will do following work for each EDK
>  II module:
>  
> diff --git a/README.md b/README.md
> index 52abb6a..ca59a35 100644
> --- a/README.md
> +++ b/README.md
> @@ -215,5 +215,6 @@ Copyright (c) 2008-2017, Intel Corporation. All rights reserved.
>  |            | [#523](https://bugzilla.tianocore.org/show_bug.cgi?id=523) Build spec: add EBNF for the --pcd syntax in the Section D.4                                                                                                                                                                                                                                             |               |
>  |            | [#517](https://bugzilla.tianocore.org/show_bug.cgi?id=517) Build spec: chapter 5.2.2 Guided Tools add description for Pkcs7Sign tool and BrotliCompress tool                                                                                                                                                                                                        |               |
>  |            | [#481](https://bugzilla.tianocore.org/show_bug.cgi?id=481) Build Spec: add clarification for not used Pcd that build tool will not do additional checks on its value                                                                                                                                                                                                |               |
>  |            | [#518](https://bugzilla.tianocore.org/show_bug.cgi?id=518) Build Spec: Update Precedence of PCD Values                                                                                                                                                                                                                                                              |               |
>  |            | [#669](https://bugzilla.tianocore.org/show_bug.cgi?id=669) Build Spec: Add multi-arg support to PREBUILD/POSTBUILD                                                                                                                                                                                                                                                  |               |
> +|            | [#689](https://bugzilla.tianocore.org/show_bug.cgi?id=689) Build spec: add description for build with binary cache                                                                                                                                                                                                                                                  |               |
> diff --git a/appendix_d_buildexe_command/d4_usage.md b/appendix_d_buildexe_command/d4_usage.md
> index b71f2d0..c901266 100644
> --- a/appendix_d_buildexe_command/d4_usage.md
> +++ b/appendix_d_buildexe_command/d4_usage.md
> @@ -32,19 +32,20 @@
>  ## D.4 Usage
>  
>  ```ini
>  Usage: build.exe [options]
>  [all|fds|genc|genmake|clean|cleanall|cleanlib|modules|libraries|run]
> -Copyright (c) 2007 - 2014, Intel Corporation All rights reserved.
> +Copyright (c) 2007 - 2017, Intel Corporation All rights reserved.
>  
>  Options:
>    --version             show program's version number and exit
>    -h, --help            show this help message and exit
>    -a TARGETARCH, --arch=TARGETARCH
> -                        ARCHS is one of list: IA32, X64, IPF, ARM, or EBC,
> -                        which overrides target.txt's TARGET_ARCH definition.
> -                        To specify more archs, please repeat this option.
> +                        ARCHS is one of list: IA32, X64, IPF, ARM, AARCH64 or
> +                        EBC, which overrides target.txt's TARGET_ARCH
> +                        definition. To specify more archs, please repeat this
> +                        option.
>    -p PLATFORMFILE, --platform=PLATFORMFILE
>                          Build the platform specified by the DSC file name
>                          argument, overriding target.txt's ACTIVE_PLATFORM
>                          definition.
>    -m MODULEFILE, --module=MODULEFILE
> @@ -112,10 +113,20 @@ Options:
>    -N, --no-cache        Disable build cache mechanism
>    --conf=CONFDIRECTORY  Specify the customized Conf directory.
>    --check-usage         Check usage content of entries listed in INF file.
>    --ignore-sources      Focus to a binary build and ignore all source files
>    --pcd=OPTIONPCD       Set PCD value by command line. Format: "PcdName=Value"
> +  -l COMMANDLENGTH, --cmd-len=COMMANDLENGTH
> +                        Specify the maximum line length of build command.
> +                        Default is 4096.
> +  --hash                Enable hash-based caching during build process.
> +  --binary-destination=BINCACHEDEST
> +                        Generate a cache of binary files in the specified
> +                        directory.
> +  --binary-source=BINCACHESOURCE
> +                        Consume a cache of binary files from the specified
> +                        directory.
>  ```
>  
>  ### D.4.1 Debug Levels
>  
>  The numeric debug levels are defined as integer values 0-9.
>