Re: [PATCH 2/4] BaseTools-Conf:Introduce CLANG38 new toolchain for x86

[Ard Biesheuvel <ard.biesheuvel@linaro.org>]

Hello Steven,

On 3 August 2016 at 08:48, Shi, Steven <steven.shi@intel.com> wrote:
> This adds support for LLVM 3.8.x in LTO mode for IA32 and X64.
> CLANG38 enable LLVM Link Time Optimization (LTO) and code size
> optimization flag (-Oz) by default for aggressive code size
> improvement. CLANG38 X64 code is small code model + PIE.
>
> CLANG LTO needs PIE in link flags to generate PIE code correctly,
> otherwise the PIE is not really enabled. (e.g. OvmfPkgX64 will
> hang in 64bits SEC at high address because of small model code
> displacement overflow).
>

This is probably caused by the same issue I am addressing with the
series I sent out yesterday, to pass the CC flags to the DLINK
command.

The reason is that code is generated by the link pass, so it needs to
see the same -fpie -mcmodel=small options that we passed to the
compiler as wel.

Could you check whether replacing '-Wl,-pie' with -fpie does the trick
as well? As I mentioned before, creating a PIE executable at link time
is not the same as generatic position independent code at compile time
(whether it is via $(CC) or via $(DLINK)). The PIE executable will
contain a .rela section that partially overlaps with other absolute
relocations, so it is best to avoid it.

Some more comments below

> Test pass platforms: OVMF (OvmfPkgIa32.dsc, OvmfPkgX64.dsc and
> OvmfPkgIa32X64.dsc).
> Test compiler and linker version: LLVM 3.8, GNU ld 2.26.
>
> Example steps to use the CLANG38 tool chain to build OVMF platform:
> 1. Download and extract the llvm 3.8.0 Pre-Built Binaries from
> http://www.llvm.org/releases/ (e.g. http://www.llvm.org/releases/
> 3.8.0/clang+llvm-3.8.0-x86_64-linux-gnu-ubuntu-16.04.tar.xz and
> extract it as ~/clang38).
> 2. Copy LLVMgold.so from https://github.com/shijunjing/edk2/blob/
> llvm/BaseTools/Bin/LLVMgold.so to above clang lib folder (e.g.
> ~/clang38/lib/LLVMgold.so)
> 3. Install new version linker with plugin support (e.g. ld 2.26 in
> GNU Binutils 2.26 or Ubuntu16.04)
> $ cd edk2
> $ git checkout llvm
> $ export CLANG38_BIN=path/to/your/clang38/
> (e.g. export CLANG38_BIN=~/clang38/bin/)
> $ source edksetup.sh
> $ make -C BaseTools/Source/C
> $ build -t CLANG38 -a X64 -p OvmfPkg/OvmfPkgX64.dsc -n 5 -b DEBUG
> -DDEBUG_ON_SERIAL_PORT
> $ cd edk2/Build/OvmfX64/DEBUG_CLANG38/FV
> $ qemu-system-x86_64.exe -bios OVMF.fd -serial file:serial.log -m 4096
>  -hda fat:.
>
> If you want, you can build and install GNU Binutils 2.26 as below steps
> in Ubuntu:
> Download binutils-2.26 source code from http://ftp.gnu.org/gnu/binutils/
>  and extract it to ~/binutils-2.26
> $sudo apt-get install bison
> $sudo apt-get install flex
> Install other necessary binutils build tools if missing
> $ mkdir build
> $ cd build
> $ ../binutils-2.26/configure --enable-gold --enable-plugins
> --disable-werror --prefix=/usr
> $ make -j 5
> $ sudo make install
>
> If you want, you can build LLVMgold.so as below steps
> Download llvm-3.8.0 source code from http://www.llvm.org/releases/
> 3.8.0/llvm-3.8.0.src.tar.xz and extract it to ~/llvm-3.8.0.src
> Download clang3.8.0 source code from http://www.llvm.org/releases/
> 3.8.0/cfe-3.8.0.src.tar.xz and extract it to ~/llvm-3.8.0.src/tools/clang
> Refer http://clang.llvm.org/get_started.html to Install other necessary
> clang build tools if missing
> $ mkdir llvm38build
> $ cd llvm38build
> If your GNU Binutils 2.26 is in /home/jshi19/binutils-2.26,
> $ cmake ../llvm-3.8.0.src -G "Unix Makefiles" -DCMAKE_BUILD_TYPE="Release"
> -DLLVM_TARGETS_TO_BUILD="X86" -DCMAKE_VERBOSE_MAKEFILE=ON
> -DCMAKE_CXX_COMPILER="/usr/bin/g++" -DCMAKE_C_COMPILER="/usr/bin/gcc"
> -DLLVM_BINUTILS_INCDIR=/home/jshi19/binutils-2.26/include
> $ make -j 5 LLVMgold The LLVMgold.so is in ~/llvm38build/lib/LLVMgold.so
>
> Contributed-under: TianoCore Contribution Agreement 1.0
> Signed-off-by: Steven Shi <steven.shi@intel.com>
> ---
>  BaseTools/Conf/tools_def.template | 81 +++++++++++++++++++++++++++++++++++++++
>  1 file changed, 81 insertions(+)
>  mode change 100644 => 100755 BaseTools/Conf/tools_def.template
>
> diff --git a/BaseTools/Conf/tools_def.template b/BaseTools/Conf/tools_def.template
> old mode 100644
> new mode 100755
> index fd9eccb..6d964f1
> --- a/BaseTools/Conf/tools_def.template
> +++ b/BaseTools/Conf/tools_def.template
> @@ -5428,6 +5428,87 @@ RELEASE_CLANG35_AARCH64_CC_FLAGS = DEF(CLANG35_AARCH64_CC_FLAGS) $(ARCHCC_FLAGS)
>
>  ####################################################################################
>  #
> +# Clang 3.8 - This configuration is used to compile under Linux to produce
> +#  PE/COFF binaries using LLVM/Clang 3.8 with Link Time Optimization enabled
> +#
> +####################################################################################
> +*_CLANG38_*_*_FAMILY                = GCC
> +*_CLANG38_*_MAKE_PATH               = DEF(GCC5_IA32_PREFIX)make
> +*_CLANG38_*_*_DLL                   = ENV(CLANG38_DLL)
> +*_CLANG38_*_ASL_PATH                = DEF(UNIX_IASL_BIN)
> +
> +*_CLANG38_*_APP_FLAGS               =
> +*_CLANG38_*_ASL_FLAGS               = DEF(IASL_FLAGS)
> +*_CLANG38_*_ASL_OUTFLAGS            = DEF(IASL_OUTFLAGS)
> +
> +DEFINE CLANG38_IA32_PREFIX          = ENV(CLANG38_BIN)
> +DEFINE CLANG38_X64_PREFIX           = ENV(CLANG38_BIN)
> +
> +DEFINE CLANG38_IA32_TARGET          = -target i686-pc-linux-gnu
> +DEFINE CLANG38_X64_TARGET           = -target x86_64-pc-linux-gnu
> +
> +DEFINE CLANG38_ALL_CC_FLAGS         = DEF(GCC44_ALL_CC_FLAGS) -Wno-empty-body -fno-stack-protector -fno-builtin -mms-bitfields -Wno-address -Wno-shift-negative-value -Wno-parentheses-equality -Wno-unknown-pragmas -Wno-tautological-constant-out-of-range-compare -Wno-incompatible-library-redeclaration -fno-asynchronous-unwind-tables -mno-sse -mno-mmx -msoft-float -mno-implicit-float  -ftrap-function=undefined_behavior_has_been_optimized_away_by_clang -funsigned-char -fno-ms-extensions -Wno-null-dereference -Wno-tautological-compare
> +
> +###########################
> +# CLANG38 IA32 definitions
> +###########################
> +*_CLANG38_IA32_OBJCOPY_PATH         = DEF(GCC5_IA32_PREFIX)objcopy

Why are you using the GCC5 prefix here? A clang user may not set GCC5_BIN

> +*_CLANG38_IA32_CC_PATH              = DEF(CLANG38_IA32_PREFIX)clang
> +*_CLANG38_IA32_SLINK_PATH           = DEF(CLANG38_IA32_PREFIX)llvm-ar
> +*_CLANG38_IA32_DLINK_PATH           = DEF(CLANG38_IA32_PREFIX)clang
> +*_CLANG38_IA32_ASLDLINK_PATH        = DEF(CLANG38_IA32_PREFIX)clang
> +*_CLANG38_IA32_ASM_PATH             = DEF(CLANG38_IA32_PREFIX)clang
> +*_CLANG38_IA32_PP_PATH              = DEF(CLANG38_IA32_PREFIX)clang
> +*_CLANG38_IA32_VFRPP_PATH           = DEF(CLANG38_IA32_PREFIX)clang
> +*_CLANG38_IA32_ASLCC_PATH           = DEF(CLANG38_IA32_PREFIX)clang
> +*_CLANG38_IA32_ASLPP_PATH           = DEF(CLANG38_IA32_PREFIX)clang
> +*_CLANG38_IA32_RC_PATH              = DEF(GCC5_IA32_PREFIX)objcopy
> +
> +*_CLANG38_IA32_ASLCC_FLAGS          = -x c -Os -m32 DEF(CLANG38_IA32_TARGET) -flto

Does LTO make any sense for ACPI tables?

> +*_CLANG38_IA32_ASLDLINK_FLAGS       = DEF(GCC5_IA32_X64_DLINK_FLAGS) -Wl,-Oz -Wl,-melf_i386 -Wl,--entry,ReferenceAcpiTable -Wl,-u,ReferenceAcpiTable
> +*_CLANG38_IA32_ASM_FLAGS            = DEF(GCC5_ASM_FLAGS) -m32 -march=i386 DEF(CLANG38_IA32_TARGET)
> +DEBUG_CLANG38_IA32_CC_FLAGS         = DEF(CLANG38_ALL_CC_FLAGS) -m32 -Oz -flto -march=i586 DEF(CLANG38_IA32_TARGET) -g
> +RELEASE_CLANG38_IA32_CC_FLAGS       = DEF(CLANG38_ALL_CC_FLAGS) -m32 -Oz -flto -march=i586 DEF(CLANG38_IA32_TARGET)
> +*_CLANG38_IA32_DLINK_FLAGS          = DEF(GCC5_IA32_X64_DLINK_FLAGS) -Wl,-Oz -Wl,-melf_i386 -Wl,--oformat=elf32-i386
> +*_CLANG38_IA32_DLINK2_FLAGS         = DEF(GCC5_IA32_DLINK2_FLAGS)
> +*_CLANG38_IA32_RC_FLAGS             = DEF(GCC_IA32_RC_FLAGS)
> +*_CLANG38_IA32_OBJCOPY_FLAGS        =
> +*_CLANG38_IA32_NASM_FLAGS           = -f elf32
> +*_CLANG38_IA32_PP_FLAGS             = DEF(GCC_PP_FLAGS) DEF(CLANG38_IA32_TARGET)
> +*_CLANG38_IA32_ASLPP_FLAGS          = DEF(GCC_ASLPP_FLAGS) DEF(CLANG38_IA32_TARGET)
> +*_CLANG38_IA32_VFRPP_FLAGS          = DEF(GCC_VFRPP_FLAGS) DEF(CLANG38_IA32_TARGET)
> +
> +##########################
> +# CLANG38 X64 definitions
> +##########################
> +*_CLANG38_X64_OBJCOPY_PATH         = DEF(GCC5_X64_PREFIX)objcopy
> +*_CLANG38_X64_CC_PATH              = DEF(CLANG38_X64_PREFIX)clang
> +*_CLANG38_X64_SLINK_PATH           = DEF(CLANG38_X64_PREFIX)llvm-ar
> +*_CLANG38_X64_DLINK_PATH           = DEF(CLANG38_X64_PREFIX)clang
> +*_CLANG38_X64_ASLDLINK_PATH        = DEF(CLANG38_X64_PREFIX)clang
> +*_CLANG38_X64_ASM_PATH             = DEF(CLANG38_X64_PREFIX)clang
> +*_CLANG38_X64_PP_PATH              = DEF(CLANG38_X64_PREFIX)clang
> +*_CLANG38_X64_VFRPP_PATH           = DEF(CLANG38_X64_PREFIX)clang
> +*_CLANG38_X64_ASLCC_PATH           = DEF(CLANG38_X64_PREFIX)clang
> +*_CLANG38_X64_ASLPP_PATH           = DEF(CLANG38_X64_PREFIX)clang
> +*_CLANG38_X64_RC_PATH              = DEF(GCC5_X64_PREFIX)objcopy
> +
> +*_CLANG38_X64_ASLCC_FLAGS          = -x c -Os -m64 -flto DEF(CLANG38_X64_TARGET)
> +*_CLANG38_X64_ASLDLINK_FLAGS       = DEF(GCC5_IA32_X64_DLINK_FLAGS) -Wl,-Oz -Wl,-melf_x86_64 -Wl,--entry,ReferenceAcpiTable -Wl,-u,ReferenceAcpiTable -Wl,-pie -mcmodel=small
> +*_CLANG38_X64_ASM_FLAGS            = DEF(GCC5_ASM_FLAGS) -m64 DEF(CLANG38_X64_TARGET)
> +DEBUG_CLANG38_X64_CC_FLAGS         = DEF(CLANG38_ALL_CC_FLAGS) -m64 "-DEFIAPI=__attribute__((ms_abi))" -mno-red-zone -mcmodel=small -fpie -Oz -flto DEF(CLANG38_X64_TARGET) -g
> +RELEASE_CLANG38_X64_CC_FLAGS       = DEF(CLANG38_ALL_CC_FLAGS) -m64 "-DEFIAPI=__attribute__((ms_abi))" -mno-red-zone -mcmodel=small -fpie -Oz -flto DEF(CLANG38_X64_TARGET)
> +*_CLANG38_X64_DLINK_FLAGS          = DEF(GCC5_IA32_X64_DLINK_FLAGS) -Wl,-Oz -Wl,-melf_x86_64 -Wl,--oformat=elf64-x86-64 -Wl,-pie -mcmodel=small
> +*_CLANG38_X64_DLINK2_FLAGS         = DEF(GCC5_X64_DLINK2_FLAGS)
> +*_CLANG38_X64_RC_FLAGS             = DEF(GCC_X64_RC_FLAGS)
> +*_CLANG38_X64_OBJCOPY_FLAGS        =
> +*_CLANG38_X64_NASM_FLAGS           = -f elf64
> +*_CLANG38_X64_PP_FLAGS             = DEF(GCC_PP_FLAGS) DEF(CLANG38_X64_TARGET)
> +*_CLANG38_X64_ASLPP_FLAGS          = DEF(GCC_ASLPP_FLAGS) DEF(CLANG38_X64_TARGET)
> +*_CLANG38_X64_VFRPP_FLAGS          = DEF(GCC_VFRPP_FLAGS) DEF(CLANG38_X64_TARGET)
> +
> +####################################################################################
> +#
>  # Cygwin GCC And Intel ACPI Compiler
>  #
>  ####################################################################################
> --
> 2.7.4
>
> _______________________________________________
> edk2-devel mailing list
> edk2-devel@lists.01.org
> https://lists.01.org/mailman/listinfo/edk2-devel