Re: [PATCH v5 2/4] BaseTools-Conf:Introduce CLANG38 new toolchain for x86

["Gao, Liming" <liming.gao@intel.com>]

Reviewed-by: Liming Gao <liming.gao@intel.com>

> -----Original Message-----
> From: Shi, Steven
> Sent: Wednesday, August 03, 2016 5:43 PM
> To: edk2-devel@lists.01.org; Gao, Liming <liming.gao@intel.com>; Carsey,
> Jaben <jaben.carsey@intel.com>
> Cc: afish@apple.com; Kinney, Michael D <michael.d.kinney@intel.com>;
> ard.biesheuvel@linaro.org; Shi, Steven <steven.shi@intel.com>
> Subject: [PATCH v5 2/4] BaseTools-Conf:Introduce CLANG38 new toolchain
> for x86
> 
> This adds support for LLVM 3.8.x in LTO mode for IA32 and X64.
> CLANG38 enable LLVM Link Time Optimization (LTO) and code size
> optimization flag (-Oz) by default for aggressive code size
> improvement. CLANG38 X64 code is small code model + PIE.
> 
> CLANG LTO needs PIE in link flags to generate PIE code correctly,
> otherwise the PIE is not really enabled. (e.g. OvmfPkgX64 will
> hang in 64bits SEC at high address because of small model code
> displacement overflow).
> 
> Test pass platforms: OVMF (OvmfPkgIa32.dsc, OvmfPkgX64.dsc and
> OvmfPkgIa32X64.dsc).
> Test compiler and linker version: LLVM 3.8, GNU ld 2.26.
> 
> Example steps to use the CLANG38 tool chain to build OVMF platform:
> 1. Download and extract the llvm 3.8.0 Pre-Built Binaries from
> http://www.llvm.org/releases/ (e.g. http://www.llvm.org/releases/
> 3.8.0/clang+llvm-3.8.0-x86_64-linux-gnu-ubuntu-16.04.tar.xz and
> extract it as ~/clang38).
> 2. Copy LLVMgold.so from https://github.com/shijunjing/edk2/blob/
> llvm/BaseTools/Bin/LLVMgold.so to above clang lib folder (e.g.
> ~/clang38/lib/LLVMgold.so)
> 3. Install new version linker with plugin support (e.g. ld 2.26 in
> GNU Binutils 2.26 or Ubuntu16.04)
> $ cd edk2
> $ git checkout llvm
> $ export CLANG38_BIN=path/to/your/clang38/
> (e.g. export CLANG38_BIN=~/clang38/bin/)
> $ source edksetup.sh
> $ make -C BaseTools/Source/C
> $ build -t CLANG38 -a X64 -p OvmfPkg/OvmfPkgX64.dsc -n 5 -b DEBUG
> -DDEBUG_ON_SERIAL_PORT
> $ cd edk2/Build/OvmfX64/DEBUG_CLANG38/FV
> $ qemu-system-x86_64.exe -bios OVMF.fd -serial file:serial.log -m 4096
>  -hda fat:.
> 
> If you want, you can build and install GNU Binutils 2.26 as below steps
> in Ubuntu:
> Download binutils-2.26 source code from http://ftp.gnu.org/gnu/binutils/
>  and extract it to ~/binutils-2.26
> $sudo apt-get install bison
> $sudo apt-get install flex
> Install other necessary binutils build tools if missing
> $ mkdir build
> $ cd build
> $ ../binutils-2.26/configure --enable-gold --enable-plugins
> --disable-werror --prefix=/usr
> $ make -j 5
> $ sudo make install
> 
> If you want, you can build LLVMgold.so as below steps
> Download llvm-3.8.0 source code from http://www.llvm.org/releases/
> 3.8.0/llvm-3.8.0.src.tar.xz and extract it to ~/llvm-3.8.0.src
> Download clang3.8.0 source code from http://www.llvm.org/releases/
> 3.8.0/cfe-3.8.0.src.tar.xz and extract it to ~/llvm-3.8.0.src/tools/clang
> Refer http://clang.llvm.org/get_started.html to Install other necessary
> clang build tools if missing
> $ mkdir llvm38build
> $ cd llvm38build
> If your GNU Binutils 2.26 is in /home/jshi19/binutils-2.26,
> $ cmake ../llvm-3.8.0.src -G "Unix Makefiles" -
> DCMAKE_BUILD_TYPE="Release"
> -DLLVM_TARGETS_TO_BUILD="X86" -DCMAKE_VERBOSE_MAKEFILE=ON
> -DCMAKE_CXX_COMPILER="/usr/bin/g++" -
> DCMAKE_C_COMPILER="/usr/bin/gcc"
> -DLLVM_BINUTILS_INCDIR=/home/jshi19/binutils-2.26/include
> $ make -j 5 LLVMgold The LLVMgold.so is in ~/llvm38build/lib/LLVMgold.so
> 
> Contributed-under: TianoCore Contribution Agreement 1.0
> Signed-off-by: Steven Shi <steven.shi@intel.com>
> ---
>  BaseTools/Conf/tools_def.template | 87
> +++++++++++++++++++++++++++++++++++++++
>  1 file changed, 87 insertions(+)
>  mode change 100644 => 100755 BaseTools/Conf/tools_def.template
> 
> diff --git a/BaseTools/Conf/tools_def.template
> b/BaseTools/Conf/tools_def.template
> old mode 100644
> new mode 100755
> index fd9eccb..6489039
> --- a/BaseTools/Conf/tools_def.template
> +++ b/BaseTools/Conf/tools_def.template
> @@ -381,6 +381,12 @@ DEFINE SOURCERY_CYGWIN_TOOLS =
> /cygdrive/c/Program Files/CodeSourcery/Sourcery G
>  #                             Required to build platforms or ACPI tables:
>  #                               Intel(r) ACPI Compiler from
>  #                               https://acpica.org/downloads
> +#   CLANG38  -Linux-  Requires:
> +#                             Clang v3.8 or later, LLVMgold plugin and GNU binutils 2.26
> targeting x86_64-linux-gnu
> +#                        Optional:
> +#                             Required to build platforms or ACPI tables:
> +#                               Intel(r) ACPI Compiler from
> +#                               https://acpica.org/downloads
>  #   ELFGCC      -Linux-  Requires:
>  #                             GCC(this tool chain uses whatever version of gcc and binutils
> that is installed in /usr/bin)
>  #                        Optional:
> @@ -5428,6 +5434,87 @@ RELEASE_CLANG35_AARCH64_CC_FLAGS =
> DEF(CLANG35_AARCH64_CC_FLAGS) $(ARCHCC_FLAGS)
> 
> 
> ##########################################################
> ##########################
>  #
> +# Clang 3.8 - This configuration is used to compile under Linux to produce
> +#  PE/COFF binaries using LLVM/Clang 3.8 with Link Time Optimization
> enabled
> +#
> +#########################################################
> ###########################
> +*_CLANG38_*_*_FAMILY                = GCC
> +*_CLANG38_*_MAKE_PATH               = make
> +*_CLANG38_*_*_DLL                   = ENV(CLANG38_DLL)
> +*_CLANG38_*_ASL_PATH                = DEF(UNIX_IASL_BIN)
> +
> +*_CLANG38_*_APP_FLAGS               =
> +*_CLANG38_*_ASL_FLAGS               = DEF(IASL_FLAGS)
> +*_CLANG38_*_ASL_OUTFLAGS            = DEF(IASL_OUTFLAGS)
> +
> +DEFINE CLANG38_IA32_PREFIX          = ENV(CLANG38_BIN)
> +DEFINE CLANG38_X64_PREFIX           = ENV(CLANG38_BIN)
> +
> +DEFINE CLANG38_IA32_TARGET          = -target i686-pc-linux-gnu
> +DEFINE CLANG38_X64_TARGET           = -target x86_64-pc-linux-gnu
> +
> +DEFINE CLANG38_ALL_CC_FLAGS         = DEF(GCC44_ALL_CC_FLAGS) -Wno-
> empty-body -fno-stack-protector -fno-builtin -mms-bitfields -Wno-address -
> Wno-shift-negative-value -Wno-parentheses-equality -Wno-unknown-
> pragmas -Wno-tautological-constant-out-of-range-compare -Wno-
> incompatible-library-redeclaration -fno-asynchronous-unwind-tables -mno-
> sse -mno-mmx -msoft-float -mno-implicit-float  -ftrap-
> function=undefined_behavior_has_been_optimized_away_by_clang -
> funsigned-char -fno-ms-extensions -Wno-null-dereference -Wno-
> tautological-compare
> +
> +###########################
> +# CLANG38 IA32 definitions
> +###########################
> +*_CLANG38_IA32_OBJCOPY_PATH         = objcopy
> +*_CLANG38_IA32_CC_PATH              = DEF(CLANG38_IA32_PREFIX)clang
> +*_CLANG38_IA32_SLINK_PATH           = DEF(CLANG38_IA32_PREFIX)llvm-ar
> +*_CLANG38_IA32_DLINK_PATH           = DEF(CLANG38_IA32_PREFIX)clang
> +*_CLANG38_IA32_ASLDLINK_PATH        = DEF(CLANG38_IA32_PREFIX)clang
> +*_CLANG38_IA32_ASM_PATH             = DEF(CLANG38_IA32_PREFIX)clang
> +*_CLANG38_IA32_PP_PATH              = DEF(CLANG38_IA32_PREFIX)clang
> +*_CLANG38_IA32_VFRPP_PATH           = DEF(CLANG38_IA32_PREFIX)clang
> +*_CLANG38_IA32_ASLCC_PATH           = DEF(CLANG38_IA32_PREFIX)clang
> +*_CLANG38_IA32_ASLPP_PATH           = DEF(CLANG38_IA32_PREFIX)clang
> +*_CLANG38_IA32_RC_PATH              = objcopy
> +
> +*_CLANG38_IA32_ASLCC_FLAGS          = DEF(GCC_ASLCC_FLAGS) -m32 -fno-
> lto DEF(CLANG38_IA32_TARGET)
> +*_CLANG38_IA32_ASLDLINK_FLAGS       =
> DEF(GCC5_IA32_X64_ASLDLINK_FLAGS) -Wl,-m,elf_i386
> +*_CLANG38_IA32_ASM_FLAGS            = DEF(GCC5_ASM_FLAGS) -m32 -
> march=i386 DEF(CLANG38_IA32_TARGET)
> +DEBUG_CLANG38_IA32_CC_FLAGS         = DEF(CLANG38_ALL_CC_FLAGS) -
> m32 -Oz -flto -march=i586 DEF(CLANG38_IA32_TARGET) -g
> +RELEASE_CLANG38_IA32_CC_FLAGS       = DEF(CLANG38_ALL_CC_FLAGS) -
> m32 -Oz -flto -march=i586 DEF(CLANG38_IA32_TARGET)
> +*_CLANG38_IA32_DLINK_FLAGS          = DEF(GCC5_IA32_X64_DLINK_FLAGS)
> -Wl,-Oz -Wl,-melf_i386 -Wl,--oformat=elf32-i386
> +*_CLANG38_IA32_DLINK2_FLAGS         = DEF(GCC5_IA32_DLINK2_FLAGS)
> +*_CLANG38_IA32_RC_FLAGS             = DEF(GCC_IA32_RC_FLAGS)
> +*_CLANG38_IA32_OBJCOPY_FLAGS        =
> +*_CLANG38_IA32_NASM_FLAGS           = -f elf32
> +*_CLANG38_IA32_PP_FLAGS             = DEF(GCC_PP_FLAGS)
> DEF(CLANG38_IA32_TARGET)
> +*_CLANG38_IA32_ASLPP_FLAGS          = DEF(GCC_ASLPP_FLAGS)
> DEF(CLANG38_IA32_TARGET)
> +*_CLANG38_IA32_VFRPP_FLAGS          = DEF(GCC_VFRPP_FLAGS)
> DEF(CLANG38_IA32_TARGET)
> +
> +##########################
> +# CLANG38 X64 definitions
> +##########################
> +*_CLANG38_X64_OBJCOPY_PATH         = objcopy
> +*_CLANG38_X64_CC_PATH              = DEF(CLANG38_X64_PREFIX)clang
> +*_CLANG38_X64_SLINK_PATH           = DEF(CLANG38_X64_PREFIX)llvm-ar
> +*_CLANG38_X64_DLINK_PATH           = DEF(CLANG38_X64_PREFIX)clang
> +*_CLANG38_X64_ASLDLINK_PATH        = DEF(CLANG38_X64_PREFIX)clang
> +*_CLANG38_X64_ASM_PATH             = DEF(CLANG38_X64_PREFIX)clang
> +*_CLANG38_X64_PP_PATH              = DEF(CLANG38_X64_PREFIX)clang
> +*_CLANG38_X64_VFRPP_PATH           = DEF(CLANG38_X64_PREFIX)clang
> +*_CLANG38_X64_ASLCC_PATH           = DEF(CLANG38_X64_PREFIX)clang
> +*_CLANG38_X64_ASLPP_PATH           = DEF(CLANG38_X64_PREFIX)clang
> +*_CLANG38_X64_RC_PATH              = objcopy
> +
> +*_CLANG38_X64_ASLCC_FLAGS          = DEF(GCC_ASLCC_FLAGS) -m64 -fno-
> lto DEF(CLANG38_X64_TARGET)
> +*_CLANG38_X64_ASLDLINK_FLAGS       =
> DEF(GCC5_IA32_X64_ASLDLINK_FLAGS) -Wl,-m,elf_x86_64
> +*_CLANG38_X64_ASM_FLAGS            = DEF(GCC5_ASM_FLAGS) -m64
> DEF(CLANG38_X64_TARGET)
> +DEBUG_CLANG38_X64_CC_FLAGS         = DEF(CLANG38_ALL_CC_FLAGS) -
> m64 "-DEFIAPI=__attribute__((ms_abi))" -mno-red-zone -mcmodel=small -
> fpie -Oz -flto DEF(CLANG38_X64_TARGET) -g
> +RELEASE_CLANG38_X64_CC_FLAGS       = DEF(CLANG38_ALL_CC_FLAGS) -
> m64 "-DEFIAPI=__attribute__((ms_abi))" -mno-red-zone -mcmodel=small -
> fpie -Oz -flto DEF(CLANG38_X64_TARGET)
> +*_CLANG38_X64_DLINK_FLAGS          = DEF(GCC5_IA32_X64_DLINK_FLAGS)
> -Wl,-Oz -Wl,-melf_x86_64 -Wl,--oformat=elf64-x86-64 -Wl,-pie -
> mcmodel=small
> +*_CLANG38_X64_DLINK2_FLAGS         = DEF(GCC5_X64_DLINK2_FLAGS)
> +*_CLANG38_X64_RC_FLAGS             = DEF(GCC_X64_RC_FLAGS)
> +*_CLANG38_X64_OBJCOPY_FLAGS        =
> +*_CLANG38_X64_NASM_FLAGS           = -f elf64
> +*_CLANG38_X64_PP_FLAGS             = DEF(GCC_PP_FLAGS)
> DEF(CLANG38_X64_TARGET)
> +*_CLANG38_X64_ASLPP_FLAGS          = DEF(GCC_ASLPP_FLAGS)
> DEF(CLANG38_X64_TARGET)
> +*_CLANG38_X64_VFRPP_FLAGS          = DEF(GCC_VFRPP_FLAGS)
> DEF(CLANG38_X64_TARGET)
> +
> +#########################################################
> ###########################
> +#
>  # Cygwin GCC And Intel ACPI Compiler
>  #
> 
> ##########################################################
> ##########################
> --
> 2.7.4