[PATCH 0/4] Introduce CLANG38 toolchain in edk2

[PATCH 0/4] Introduce CLANG38 toolchain in edk2

[Shi, Steven]

Please review my new commits in public branch:
https://github.com/shijunjing/edk2/commits/llvm_v4

The difference from last V2(https://github.com/shijunjing/edk2/commits/llvm_v2):
1. Seperate the CLANG38 from the other two toolchains (GCC5, CLANGSCAN38), and only focus on CLANG38 in this serial.
2. Base on current GCC5 in edk2 to enhance the CLANG38.
3. Seperate the GenFW new relocation types support patch from this serial, and I will send it later in new serial.
4. Add EFIAPI for UefiShellCommandLib function.

Test pass platforms: OVMF (OvmfPkgIa32.dsc, OvmfPkgX64.dsc and
OvmfPkgIa32X64.dsc).
Test compiler and linker version: LLVM 3.8, GNU ld 2.26.

Example steps to use the CLANG38 tool chain to build OVMF platform:
1. Download and extract the llvm 3.8.0 Pre-Built Binaries from
http://www.llvm.org/releases/ (e.g. http://www.llvm.org/releases/
3.8.0/clang+llvm-3.8.0-x86_64-linux-gnu-ubuntu-16.04.tar.xz and
extract it as ~/clang38).
2. Copy LLVMgold.so from https://github.com/shijunjing/edk2/blob/
llvm/BaseTools/Bin/LLVMgold.so to above clang lib folder (e.g.
~/clang38/lib/LLVMgold.so)
3. Install new version linker with plugin support (e.g. ld 2.26 in
GNU Binutils 2.26 or Ubuntu16.04)
$ cd edk2
$ git checkout llvm
$ export CLANG38_BIN=path/to/your/clang38/
(e.g. export CLANG38_BIN=~/clang38/bin/)
$ source edksetup.sh
$ make -C BaseTools/Source/C
$ build -t CLANG38 -a X64 -p OvmfPkg/OvmfPkgX64.dsc -n 5 -b DEBUG
-DDEBUG_ON_SERIAL_PORT
$ cd edk2/Build/OvmfX64/DEBUG_CLANG38/FV
$ qemu-system-x86_64.exe -bios OVMF.fd -serial file:serial.log -m 4096
 -hda fat:.


Shi, Steven (4):
  BaseTools-Conf:Remove short dash in ar flag for LLVM
  BaseTools-Conf:Introduce CLANG38 new toolchain for x86
  ShellPkg-UefiShellTftpCommandLib: Replace compiler builtin
  ShellPkg-UefiShellCommandLib: Add EFIAPI in VA_List library function

 BaseTools/Conf/build_rule.template                 |  2 +-
 BaseTools/Conf/tools_def.template                  | 81 ++++++++++++++++++++++
 .../Library/UefiShellCommandLib/ConsistMapping.c   |  1 +
 ShellPkg/Library/UefiShellTftpCommandLib/Tftp.c    |  2 +-
 4 files changed, 84 insertions(+), 2 deletions(-)
 mode change 100644 => 100755 BaseTools/Conf/build_rule.template
 mode change 100644 => 100755 BaseTools/Conf/tools_def.template
 mode change 100644 => 100755 ShellPkg/Library/UefiShellCommandLib/ConsistMapping.c
 mode change 100644 => 100755 ShellPkg/Library/UefiShellTftpCommandLib/Tftp.c

-- 
2.7.4

[PATCH 1/4] BaseTools-Conf:Remove short dash in ar flag for LLVM

[Shi, Steven]

Both binutils ar and LLVM ar support "cr", but LLVM ar doens't
support add "-" in the flags, and llvm-ar cannot accept "-cr".
So remove the short dash "-" to make llvm archives work.

Contributed-under: TianoCore Contribution Agreement 1.0
Signed-off-by: Steven Shi <steven.shi@intel.com>
---
 BaseTools/Conf/build_rule.template | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
 mode change 100644 => 100755 BaseTools/Conf/build_rule.template

diff --git a/BaseTools/Conf/build_rule.template b/BaseTools/Conf/build_rule.template
old mode 100644
new mode 100755
index 9adf391..6e7b7d4
--- a/BaseTools/Conf/build_rule.template
+++ b/BaseTools/Conf/build_rule.template
@@ -266,7 +266,7 @@
         "$(SLINK)" $(SLINK_FLAGS) /OUT:${dst} @$(OBJECT_FILES_LIST)
 
     <Command.GCC, Command.GCCLD>
-        "$(SLINK)" -cr ${dst} $(SLINK_FLAGS) @$(OBJECT_FILES_LIST)
+        "$(SLINK)" cr ${dst} $(SLINK_FLAGS) @$(OBJECT_FILES_LIST)
     
     <Command.RVCT>
         "$(SLINK)" $(SLINK_FLAGS) ${dst} --via $(OBJECT_FILES_LIST)
-- 
2.7.4

[PATCH 2/4] BaseTools-Conf:Introduce CLANG38 new toolchain for x86

[Shi, Steven]

This adds support for LLVM 3.8.x in LTO mode for IA32 and X64.
CLANG38 enable LLVM Link Time Optimization (LTO) and code size
optimization flag (-Oz) by default for aggressive code size
improvement. CLANG38 X64 code is small code model + PIE.

CLANG LTO needs PIE in link flags to generate PIE code correctly,
otherwise the PIE is not really enabled. (e.g. OvmfPkgX64 will
hang in 64bits SEC at high address because of small model code
displacement overflow).

Test pass platforms: OVMF (OvmfPkgIa32.dsc, OvmfPkgX64.dsc and
OvmfPkgIa32X64.dsc).
Test compiler and linker version: LLVM 3.8, GNU ld 2.26.

Example steps to use the CLANG38 tool chain to build OVMF platform:
1. Download and extract the llvm 3.8.0 Pre-Built Binaries from
http://www.llvm.org/releases/ (e.g. http://www.llvm.org/releases/
3.8.0/clang+llvm-3.8.0-x86_64-linux-gnu-ubuntu-16.04.tar.xz and
extract it as ~/clang38).
2. Copy LLVMgold.so from https://github.com/shijunjing/edk2/blob/
llvm/BaseTools/Bin/LLVMgold.so to above clang lib folder (e.g.
~/clang38/lib/LLVMgold.so)
3. Install new version linker with plugin support (e.g. ld 2.26 in
GNU Binutils 2.26 or Ubuntu16.04)
$ cd edk2
$ git checkout llvm
$ export CLANG38_BIN=path/to/your/clang38/
(e.g. export CLANG38_BIN=~/clang38/bin/)
$ source edksetup.sh
$ make -C BaseTools/Source/C
$ build -t CLANG38 -a X64 -p OvmfPkg/OvmfPkgX64.dsc -n 5 -b DEBUG
-DDEBUG_ON_SERIAL_PORT
$ cd edk2/Build/OvmfX64/DEBUG_CLANG38/FV
$ qemu-system-x86_64.exe -bios OVMF.fd -serial file:serial.log -m 4096
 -hda fat:.

If you want, you can build and install GNU Binutils 2.26 as below steps
in Ubuntu:
Download binutils-2.26 source code from http://ftp.gnu.org/gnu/binutils/
 and extract it to ~/binutils-2.26
$sudo apt-get install bison
$sudo apt-get install flex
Install other necessary binutils build tools if missing
$ mkdir build
$ cd build
$ ../binutils-2.26/configure --enable-gold --enable-plugins
--disable-werror --prefix=/usr
$ make -j 5
$ sudo make install

If you want, you can build LLVMgold.so as below steps
Download llvm-3.8.0 source code from http://www.llvm.org/releases/
3.8.0/llvm-3.8.0.src.tar.xz and extract it to ~/llvm-3.8.0.src
Download clang3.8.0 source code from http://www.llvm.org/releases/
3.8.0/cfe-3.8.0.src.tar.xz and extract it to ~/llvm-3.8.0.src/tools/clang
Refer http://clang.llvm.org/get_started.html to Install other necessary
clang build tools if missing
$ mkdir llvm38build
$ cd llvm38build
If your GNU Binutils 2.26 is in /home/jshi19/binutils-2.26,
$ cmake ../llvm-3.8.0.src -G "Unix Makefiles" -DCMAKE_BUILD_TYPE="Release"
-DLLVM_TARGETS_TO_BUILD="X86" -DCMAKE_VERBOSE_MAKEFILE=ON
-DCMAKE_CXX_COMPILER="/usr/bin/g++" -DCMAKE_C_COMPILER="/usr/bin/gcc"
-DLLVM_BINUTILS_INCDIR=/home/jshi19/binutils-2.26/include
$ make -j 5 LLVMgold The LLVMgold.so is in ~/llvm38build/lib/LLVMgold.so

Contributed-under: TianoCore Contribution Agreement 1.0
Signed-off-by: Steven Shi <steven.shi@intel.com>
---
 BaseTools/Conf/tools_def.template | 81 +++++++++++++++++++++++++++++++++++++++
 1 file changed, 81 insertions(+)
 mode change 100644 => 100755 BaseTools/Conf/tools_def.template

diff --git a/BaseTools/Conf/tools_def.template b/BaseTools/Conf/tools_def.template
old mode 100644
new mode 100755
index fd9eccb..6d964f1
--- a/BaseTools/Conf/tools_def.template
+++ b/BaseTools/Conf/tools_def.template
@@ -5428,6 +5428,87 @@ RELEASE_CLANG35_AARCH64_CC_FLAGS = DEF(CLANG35_AARCH64_CC_FLAGS) $(ARCHCC_FLAGS)
 
 ####################################################################################
 #
+# Clang 3.8 - This configuration is used to compile under Linux to produce
+#  PE/COFF binaries using LLVM/Clang 3.8 with Link Time Optimization enabled
+#
+####################################################################################
+*_CLANG38_*_*_FAMILY                = GCC
+*_CLANG38_*_MAKE_PATH               = DEF(GCC5_IA32_PREFIX)make
+*_CLANG38_*_*_DLL                   = ENV(CLANG38_DLL)
+*_CLANG38_*_ASL_PATH                = DEF(UNIX_IASL_BIN)
+
+*_CLANG38_*_APP_FLAGS               =
+*_CLANG38_*_ASL_FLAGS               = DEF(IASL_FLAGS)
+*_CLANG38_*_ASL_OUTFLAGS            = DEF(IASL_OUTFLAGS)
+
+DEFINE CLANG38_IA32_PREFIX          = ENV(CLANG38_BIN)
+DEFINE CLANG38_X64_PREFIX           = ENV(CLANG38_BIN)
+
+DEFINE CLANG38_IA32_TARGET          = -target i686-pc-linux-gnu
+DEFINE CLANG38_X64_TARGET           = -target x86_64-pc-linux-gnu
+
+DEFINE CLANG38_ALL_CC_FLAGS         = DEF(GCC44_ALL_CC_FLAGS) -Wno-empty-body -fno-stack-protector -fno-builtin -mms-bitfields -Wno-address -Wno-shift-negative-value -Wno-parentheses-equality -Wno-unknown-pragmas -Wno-tautological-constant-out-of-range-compare -Wno-incompatible-library-redeclaration -fno-asynchronous-unwind-tables -mno-sse -mno-mmx -msoft-float -mno-implicit-float  -ftrap-function=undefined_behavior_has_been_optimized_away_by_clang -funsigned-char -fno-ms-extensions -Wno-null-dereference -Wno-tautological-compare
+
+###########################
+# CLANG38 IA32 definitions
+###########################
+*_CLANG38_IA32_OBJCOPY_PATH         = DEF(GCC5_IA32_PREFIX)objcopy
+*_CLANG38_IA32_CC_PATH              = DEF(CLANG38_IA32_PREFIX)clang
+*_CLANG38_IA32_SLINK_PATH           = DEF(CLANG38_IA32_PREFIX)llvm-ar
+*_CLANG38_IA32_DLINK_PATH           = DEF(CLANG38_IA32_PREFIX)clang
+*_CLANG38_IA32_ASLDLINK_PATH        = DEF(CLANG38_IA32_PREFIX)clang
+*_CLANG38_IA32_ASM_PATH             = DEF(CLANG38_IA32_PREFIX)clang
+*_CLANG38_IA32_PP_PATH              = DEF(CLANG38_IA32_PREFIX)clang
+*_CLANG38_IA32_VFRPP_PATH           = DEF(CLANG38_IA32_PREFIX)clang
+*_CLANG38_IA32_ASLCC_PATH           = DEF(CLANG38_IA32_PREFIX)clang
+*_CLANG38_IA32_ASLPP_PATH           = DEF(CLANG38_IA32_PREFIX)clang
+*_CLANG38_IA32_RC_PATH              = DEF(GCC5_IA32_PREFIX)objcopy
+
+*_CLANG38_IA32_ASLCC_FLAGS          = -x c -Os -m32 DEF(CLANG38_IA32_TARGET) -flto
+*_CLANG38_IA32_ASLDLINK_FLAGS       = DEF(GCC5_IA32_X64_DLINK_FLAGS) -Wl,-Oz -Wl,-melf_i386 -Wl,--entry,ReferenceAcpiTable -Wl,-u,ReferenceAcpiTable
+*_CLANG38_IA32_ASM_FLAGS            = DEF(GCC5_ASM_FLAGS) -m32 -march=i386 DEF(CLANG38_IA32_TARGET)
+DEBUG_CLANG38_IA32_CC_FLAGS         = DEF(CLANG38_ALL_CC_FLAGS) -m32 -Oz -flto -march=i586 DEF(CLANG38_IA32_TARGET) -g
+RELEASE_CLANG38_IA32_CC_FLAGS       = DEF(CLANG38_ALL_CC_FLAGS) -m32 -Oz -flto -march=i586 DEF(CLANG38_IA32_TARGET)
+*_CLANG38_IA32_DLINK_FLAGS          = DEF(GCC5_IA32_X64_DLINK_FLAGS) -Wl,-Oz -Wl,-melf_i386 -Wl,--oformat=elf32-i386
+*_CLANG38_IA32_DLINK2_FLAGS         = DEF(GCC5_IA32_DLINK2_FLAGS)
+*_CLANG38_IA32_RC_FLAGS             = DEF(GCC_IA32_RC_FLAGS)
+*_CLANG38_IA32_OBJCOPY_FLAGS        =
+*_CLANG38_IA32_NASM_FLAGS           = -f elf32
+*_CLANG38_IA32_PP_FLAGS             = DEF(GCC_PP_FLAGS) DEF(CLANG38_IA32_TARGET)
+*_CLANG38_IA32_ASLPP_FLAGS          = DEF(GCC_ASLPP_FLAGS) DEF(CLANG38_IA32_TARGET)
+*_CLANG38_IA32_VFRPP_FLAGS          = DEF(GCC_VFRPP_FLAGS) DEF(CLANG38_IA32_TARGET)
+
+##########################
+# CLANG38 X64 definitions
+##########################
+*_CLANG38_X64_OBJCOPY_PATH         = DEF(GCC5_X64_PREFIX)objcopy
+*_CLANG38_X64_CC_PATH              = DEF(CLANG38_X64_PREFIX)clang
+*_CLANG38_X64_SLINK_PATH           = DEF(CLANG38_X64_PREFIX)llvm-ar
+*_CLANG38_X64_DLINK_PATH           = DEF(CLANG38_X64_PREFIX)clang
+*_CLANG38_X64_ASLDLINK_PATH        = DEF(CLANG38_X64_PREFIX)clang
+*_CLANG38_X64_ASM_PATH             = DEF(CLANG38_X64_PREFIX)clang
+*_CLANG38_X64_PP_PATH              = DEF(CLANG38_X64_PREFIX)clang
+*_CLANG38_X64_VFRPP_PATH           = DEF(CLANG38_X64_PREFIX)clang
+*_CLANG38_X64_ASLCC_PATH           = DEF(CLANG38_X64_PREFIX)clang
+*_CLANG38_X64_ASLPP_PATH           = DEF(CLANG38_X64_PREFIX)clang
+*_CLANG38_X64_RC_PATH              = DEF(GCC5_X64_PREFIX)objcopy
+
+*_CLANG38_X64_ASLCC_FLAGS          = -x c -Os -m64 -flto DEF(CLANG38_X64_TARGET)
+*_CLANG38_X64_ASLDLINK_FLAGS       = DEF(GCC5_IA32_X64_DLINK_FLAGS) -Wl,-Oz -Wl,-melf_x86_64 -Wl,--entry,ReferenceAcpiTable -Wl,-u,ReferenceAcpiTable -Wl,-pie -mcmodel=small
+*_CLANG38_X64_ASM_FLAGS            = DEF(GCC5_ASM_FLAGS) -m64 DEF(CLANG38_X64_TARGET)
+DEBUG_CLANG38_X64_CC_FLAGS         = DEF(CLANG38_ALL_CC_FLAGS) -m64 "-DEFIAPI=__attribute__((ms_abi))" -mno-red-zone -mcmodel=small -fpie -Oz -flto DEF(CLANG38_X64_TARGET) -g
+RELEASE_CLANG38_X64_CC_FLAGS       = DEF(CLANG38_ALL_CC_FLAGS) -m64 "-DEFIAPI=__attribute__((ms_abi))" -mno-red-zone -mcmodel=small -fpie -Oz -flto DEF(CLANG38_X64_TARGET)
+*_CLANG38_X64_DLINK_FLAGS          = DEF(GCC5_IA32_X64_DLINK_FLAGS) -Wl,-Oz -Wl,-melf_x86_64 -Wl,--oformat=elf64-x86-64 -Wl,-pie -mcmodel=small
+*_CLANG38_X64_DLINK2_FLAGS         = DEF(GCC5_X64_DLINK2_FLAGS)
+*_CLANG38_X64_RC_FLAGS             = DEF(GCC_X64_RC_FLAGS)
+*_CLANG38_X64_OBJCOPY_FLAGS        =
+*_CLANG38_X64_NASM_FLAGS           = -f elf64
+*_CLANG38_X64_PP_FLAGS             = DEF(GCC_PP_FLAGS) DEF(CLANG38_X64_TARGET)
+*_CLANG38_X64_ASLPP_FLAGS          = DEF(GCC_ASLPP_FLAGS) DEF(CLANG38_X64_TARGET)
+*_CLANG38_X64_VFRPP_FLAGS          = DEF(GCC_VFRPP_FLAGS) DEF(CLANG38_X64_TARGET)
+
+####################################################################################
+#
 # Cygwin GCC And Intel ACPI Compiler
 #
 ####################################################################################
-- 
2.7.4

[PATCH 3/4] ShellPkg-UefiShellTftpCommandLib: Replace compiler builtin

[Shi, Steven]

Use explicit CopyMem to replace compiler builtin to do the structure
values assignment. This change is needed to pass CLANG38 build.

Contributed-under: TianoCore Contribution Agreement 1.0
Signed-off-by: Steven Shi <steven.shi@intel.com>
Reviewed-by: Jaben Carsey <jaben.carsey@intel.com>
---
 ShellPkg/Library/UefiShellTftpCommandLib/Tftp.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
 mode change 100644 => 100755 ShellPkg/Library/UefiShellTftpCommandLib/Tftp.c

diff --git a/ShellPkg/Library/UefiShellTftpCommandLib/Tftp.c b/ShellPkg/Library/UefiShellTftpCommandLib/Tftp.c
old mode 100644
new mode 100755
index 666ee9d..5c50797
--- a/ShellPkg/Library/UefiShellTftpCommandLib/Tftp.c
+++ b/ShellPkg/Library/UefiShellTftpCommandLib/Tftp.c
@@ -342,7 +342,7 @@ ShellCommandRunTftp (
     goto Error;
   }
 
-  Mtftp4ConfigData = DefaultMtftp4ConfigData;
+  CopyMem (&Mtftp4ConfigData, &DefaultMtftp4ConfigData, sizeof (EFI_MTFTP4_CONFIG_DATA));
 
   //
   // Check the host IPv4 address
-- 
2.7.4

[PATCH 4/4] ShellPkg-UefiShellCommandLib: Add EFIAPI in VA_List library function

[Shi, Steven]

Add EFIAPI in CatPrint library function. Every function which uses
variable list need explicit use EFIAPI to force use MS ABI. This change
is needed to pass CLANG38 build.

Contributed-under: TianoCore Contribution Agreement 1.0
Signed-off-by: Steven Shi <steven.shi@intel.com>
---
 ShellPkg/Library/UefiShellCommandLib/ConsistMapping.c | 1 +
 1 file changed, 1 insertion(+)
 mode change 100644 => 100755 ShellPkg/Library/UefiShellCommandLib/ConsistMapping.c

diff --git a/ShellPkg/Library/UefiShellCommandLib/ConsistMapping.c b/ShellPkg/Library/UefiShellCommandLib/ConsistMapping.c
old mode 100644
new mode 100755
index d157ebb..979693a
--- a/ShellPkg/Library/UefiShellCommandLib/ConsistMapping.c
+++ b/ShellPkg/Library/UefiShellCommandLib/ConsistMapping.c
@@ -85,6 +85,7 @@ typedef struct {
 
 **/
 EFI_STATUS
+EFIAPI
 CatPrint (
   IN OUT POOL_PRINT   *Str,
   IN CHAR16           *Fmt,
-- 
2.7.4

Re: [PATCH 1/4] BaseTools-Conf:Remove short dash in ar flag for LLVM

[Ard Biesheuvel]

On 3 August 2016 at 08:48, Shi, Steven <steven.shi@intel.com> wrote:
> Both binutils ar and LLVM ar support "cr", but LLVM ar doens't
> support add "-" in the flags, and llvm-ar cannot accept "-cr".
> So remove the short dash "-" to make llvm archives work.
>
> Contributed-under: TianoCore Contribution Agreement 1.0
> Signed-off-by: Steven Shi <steven.shi@intel.com>

Reviewed-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>

> ---
>  BaseTools/Conf/build_rule.template | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
>  mode change 100644 => 100755 BaseTools/Conf/build_rule.template
>
> diff --git a/BaseTools/Conf/build_rule.template b/BaseTools/Conf/build_rule.template
> old mode 100644
> new mode 100755
> index 9adf391..6e7b7d4
> --- a/BaseTools/Conf/build_rule.template
> +++ b/BaseTools/Conf/build_rule.template
> @@ -266,7 +266,7 @@
>          "$(SLINK)" $(SLINK_FLAGS) /OUT:${dst} @$(OBJECT_FILES_LIST)
>
>      <Command.GCC, Command.GCCLD>
> -        "$(SLINK)" -cr ${dst} $(SLINK_FLAGS) @$(OBJECT_FILES_LIST)
> +        "$(SLINK)" cr ${dst} $(SLINK_FLAGS) @$(OBJECT_FILES_LIST)
>
>      <Command.RVCT>
>          "$(SLINK)" $(SLINK_FLAGS) ${dst} --via $(OBJECT_FILES_LIST)
> --
> 2.7.4
>
> _______________________________________________
> edk2-devel mailing list
> edk2-devel@lists.01.org
> https://lists.01.org/mailman/listinfo/edk2-devel

Re: [PATCH 2/4] BaseTools-Conf:Introduce CLANG38 new toolchain for x86

[Ard Biesheuvel]

Hello Steven,

On 3 August 2016 at 08:48, Shi, Steven <steven.shi@intel.com> wrote:
> This adds support for LLVM 3.8.x in LTO mode for IA32 and X64.
> CLANG38 enable LLVM Link Time Optimization (LTO) and code size
> optimization flag (-Oz) by default for aggressive code size
> improvement. CLANG38 X64 code is small code model + PIE.
>
> CLANG LTO needs PIE in link flags to generate PIE code correctly,
> otherwise the PIE is not really enabled. (e.g. OvmfPkgX64 will
> hang in 64bits SEC at high address because of small model code
> displacement overflow).
>

This is probably caused by the same issue I am addressing with the
series I sent out yesterday, to pass the CC flags to the DLINK
command.

The reason is that code is generated by the link pass, so it needs to
see the same -fpie -mcmodel=small options that we passed to the
compiler as wel.

Could you check whether replacing '-Wl,-pie' with -fpie does the trick
as well? As I mentioned before, creating a PIE executable at link time
is not the same as generatic position independent code at compile time
(whether it is via $(CC) or via $(DLINK)). The PIE executable will
contain a .rela section that partially overlaps with other absolute
relocations, so it is best to avoid it.

Some more comments below

> Test pass platforms: OVMF (OvmfPkgIa32.dsc, OvmfPkgX64.dsc and
> OvmfPkgIa32X64.dsc).
> Test compiler and linker version: LLVM 3.8, GNU ld 2.26.
>
> Example steps to use the CLANG38 tool chain to build OVMF platform:
> 1. Download and extract the llvm 3.8.0 Pre-Built Binaries from
> http://www.llvm.org/releases/ (e.g. http://www.llvm.org/releases/
> 3.8.0/clang+llvm-3.8.0-x86_64-linux-gnu-ubuntu-16.04.tar.xz and
> extract it as ~/clang38).
> 2. Copy LLVMgold.so from https://github.com/shijunjing/edk2/blob/
> llvm/BaseTools/Bin/LLVMgold.so to above clang lib folder (e.g.
> ~/clang38/lib/LLVMgold.so)
> 3. Install new version linker with plugin support (e.g. ld 2.26 in
> GNU Binutils 2.26 or Ubuntu16.04)
> $ cd edk2
> $ git checkout llvm
> $ export CLANG38_BIN=path/to/your/clang38/
> (e.g. export CLANG38_BIN=~/clang38/bin/)
> $ source edksetup.sh
> $ make -C BaseTools/Source/C
> $ build -t CLANG38 -a X64 -p OvmfPkg/OvmfPkgX64.dsc -n 5 -b DEBUG
> -DDEBUG_ON_SERIAL_PORT
> $ cd edk2/Build/OvmfX64/DEBUG_CLANG38/FV
> $ qemu-system-x86_64.exe -bios OVMF.fd -serial file:serial.log -m 4096
>  -hda fat:.
>
> If you want, you can build and install GNU Binutils 2.26 as below steps
> in Ubuntu:
> Download binutils-2.26 source code from http://ftp.gnu.org/gnu/binutils/
>  and extract it to ~/binutils-2.26
> $sudo apt-get install bison
> $sudo apt-get install flex
> Install other necessary binutils build tools if missing
> $ mkdir build
> $ cd build
> $ ../binutils-2.26/configure --enable-gold --enable-plugins
> --disable-werror --prefix=/usr
> $ make -j 5
> $ sudo make install
>
> If you want, you can build LLVMgold.so as below steps
> Download llvm-3.8.0 source code from http://www.llvm.org/releases/
> 3.8.0/llvm-3.8.0.src.tar.xz and extract it to ~/llvm-3.8.0.src
> Download clang3.8.0 source code from http://www.llvm.org/releases/
> 3.8.0/cfe-3.8.0.src.tar.xz and extract it to ~/llvm-3.8.0.src/tools/clang
> Refer http://clang.llvm.org/get_started.html to Install other necessary
> clang build tools if missing
> $ mkdir llvm38build
> $ cd llvm38build
> If your GNU Binutils 2.26 is in /home/jshi19/binutils-2.26,
> $ cmake ../llvm-3.8.0.src -G "Unix Makefiles" -DCMAKE_BUILD_TYPE="Release"
> -DLLVM_TARGETS_TO_BUILD="X86" -DCMAKE_VERBOSE_MAKEFILE=ON
> -DCMAKE_CXX_COMPILER="/usr/bin/g++" -DCMAKE_C_COMPILER="/usr/bin/gcc"
> -DLLVM_BINUTILS_INCDIR=/home/jshi19/binutils-2.26/include
> $ make -j 5 LLVMgold The LLVMgold.so is in ~/llvm38build/lib/LLVMgold.so
>
> Contributed-under: TianoCore Contribution Agreement 1.0
> Signed-off-by: Steven Shi <steven.shi@intel.com>
> ---
>  BaseTools/Conf/tools_def.template | 81 +++++++++++++++++++++++++++++++++++++++
>  1 file changed, 81 insertions(+)
>  mode change 100644 => 100755 BaseTools/Conf/tools_def.template
>
> diff --git a/BaseTools/Conf/tools_def.template b/BaseTools/Conf/tools_def.template
> old mode 100644
> new mode 100755
> index fd9eccb..6d964f1
> --- a/BaseTools/Conf/tools_def.template
> +++ b/BaseTools/Conf/tools_def.template
> @@ -5428,6 +5428,87 @@ RELEASE_CLANG35_AARCH64_CC_FLAGS = DEF(CLANG35_AARCH64_CC_FLAGS) $(ARCHCC_FLAGS)
>
>  ####################################################################################
>  #
> +# Clang 3.8 - This configuration is used to compile under Linux to produce
> +#  PE/COFF binaries using LLVM/Clang 3.8 with Link Time Optimization enabled
> +#
> +####################################################################################
> +*_CLANG38_*_*_FAMILY                = GCC
> +*_CLANG38_*_MAKE_PATH               = DEF(GCC5_IA32_PREFIX)make
> +*_CLANG38_*_*_DLL                   = ENV(CLANG38_DLL)
> +*_CLANG38_*_ASL_PATH                = DEF(UNIX_IASL_BIN)
> +
> +*_CLANG38_*_APP_FLAGS               =
> +*_CLANG38_*_ASL_FLAGS               = DEF(IASL_FLAGS)
> +*_CLANG38_*_ASL_OUTFLAGS            = DEF(IASL_OUTFLAGS)
> +
> +DEFINE CLANG38_IA32_PREFIX          = ENV(CLANG38_BIN)
> +DEFINE CLANG38_X64_PREFIX           = ENV(CLANG38_BIN)
> +
> +DEFINE CLANG38_IA32_TARGET          = -target i686-pc-linux-gnu
> +DEFINE CLANG38_X64_TARGET           = -target x86_64-pc-linux-gnu
> +
> +DEFINE CLANG38_ALL_CC_FLAGS         = DEF(GCC44_ALL_CC_FLAGS) -Wno-empty-body -fno-stack-protector -fno-builtin -mms-bitfields -Wno-address -Wno-shift-negative-value -Wno-parentheses-equality -Wno-unknown-pragmas -Wno-tautological-constant-out-of-range-compare -Wno-incompatible-library-redeclaration -fno-asynchronous-unwind-tables -mno-sse -mno-mmx -msoft-float -mno-implicit-float  -ftrap-function=undefined_behavior_has_been_optimized_away_by_clang -funsigned-char -fno-ms-extensions -Wno-null-dereference -Wno-tautological-compare
> +
> +###########################
> +# CLANG38 IA32 definitions
> +###########################
> +*_CLANG38_IA32_OBJCOPY_PATH         = DEF(GCC5_IA32_PREFIX)objcopy

Why are you using the GCC5 prefix here? A clang user may not set GCC5_BIN

> +*_CLANG38_IA32_CC_PATH              = DEF(CLANG38_IA32_PREFIX)clang
> +*_CLANG38_IA32_SLINK_PATH           = DEF(CLANG38_IA32_PREFIX)llvm-ar
> +*_CLANG38_IA32_DLINK_PATH           = DEF(CLANG38_IA32_PREFIX)clang
> +*_CLANG38_IA32_ASLDLINK_PATH        = DEF(CLANG38_IA32_PREFIX)clang
> +*_CLANG38_IA32_ASM_PATH             = DEF(CLANG38_IA32_PREFIX)clang
> +*_CLANG38_IA32_PP_PATH              = DEF(CLANG38_IA32_PREFIX)clang
> +*_CLANG38_IA32_VFRPP_PATH           = DEF(CLANG38_IA32_PREFIX)clang
> +*_CLANG38_IA32_ASLCC_PATH           = DEF(CLANG38_IA32_PREFIX)clang
> +*_CLANG38_IA32_ASLPP_PATH           = DEF(CLANG38_IA32_PREFIX)clang
> +*_CLANG38_IA32_RC_PATH              = DEF(GCC5_IA32_PREFIX)objcopy
> +
> +*_CLANG38_IA32_ASLCC_FLAGS          = -x c -Os -m32 DEF(CLANG38_IA32_TARGET) -flto

Does LTO make any sense for ACPI tables?

> +*_CLANG38_IA32_ASLDLINK_FLAGS       = DEF(GCC5_IA32_X64_DLINK_FLAGS) -Wl,-Oz -Wl,-melf_i386 -Wl,--entry,ReferenceAcpiTable -Wl,-u,ReferenceAcpiTable
> +*_CLANG38_IA32_ASM_FLAGS            = DEF(GCC5_ASM_FLAGS) -m32 -march=i386 DEF(CLANG38_IA32_TARGET)
> +DEBUG_CLANG38_IA32_CC_FLAGS         = DEF(CLANG38_ALL_CC_FLAGS) -m32 -Oz -flto -march=i586 DEF(CLANG38_IA32_TARGET) -g
> +RELEASE_CLANG38_IA32_CC_FLAGS       = DEF(CLANG38_ALL_CC_FLAGS) -m32 -Oz -flto -march=i586 DEF(CLANG38_IA32_TARGET)
> +*_CLANG38_IA32_DLINK_FLAGS          = DEF(GCC5_IA32_X64_DLINK_FLAGS) -Wl,-Oz -Wl,-melf_i386 -Wl,--oformat=elf32-i386
> +*_CLANG38_IA32_DLINK2_FLAGS         = DEF(GCC5_IA32_DLINK2_FLAGS)
> +*_CLANG38_IA32_RC_FLAGS             = DEF(GCC_IA32_RC_FLAGS)
> +*_CLANG38_IA32_OBJCOPY_FLAGS        =
> +*_CLANG38_IA32_NASM_FLAGS           = -f elf32
> +*_CLANG38_IA32_PP_FLAGS             = DEF(GCC_PP_FLAGS) DEF(CLANG38_IA32_TARGET)
> +*_CLANG38_IA32_ASLPP_FLAGS          = DEF(GCC_ASLPP_FLAGS) DEF(CLANG38_IA32_TARGET)
> +*_CLANG38_IA32_VFRPP_FLAGS          = DEF(GCC_VFRPP_FLAGS) DEF(CLANG38_IA32_TARGET)
> +
> +##########################
> +# CLANG38 X64 definitions
> +##########################
> +*_CLANG38_X64_OBJCOPY_PATH         = DEF(GCC5_X64_PREFIX)objcopy
> +*_CLANG38_X64_CC_PATH              = DEF(CLANG38_X64_PREFIX)clang
> +*_CLANG38_X64_SLINK_PATH           = DEF(CLANG38_X64_PREFIX)llvm-ar
> +*_CLANG38_X64_DLINK_PATH           = DEF(CLANG38_X64_PREFIX)clang
> +*_CLANG38_X64_ASLDLINK_PATH        = DEF(CLANG38_X64_PREFIX)clang
> +*_CLANG38_X64_ASM_PATH             = DEF(CLANG38_X64_PREFIX)clang
> +*_CLANG38_X64_PP_PATH              = DEF(CLANG38_X64_PREFIX)clang
> +*_CLANG38_X64_VFRPP_PATH           = DEF(CLANG38_X64_PREFIX)clang
> +*_CLANG38_X64_ASLCC_PATH           = DEF(CLANG38_X64_PREFIX)clang
> +*_CLANG38_X64_ASLPP_PATH           = DEF(CLANG38_X64_PREFIX)clang
> +*_CLANG38_X64_RC_PATH              = DEF(GCC5_X64_PREFIX)objcopy
> +
> +*_CLANG38_X64_ASLCC_FLAGS          = -x c -Os -m64 -flto DEF(CLANG38_X64_TARGET)
> +*_CLANG38_X64_ASLDLINK_FLAGS       = DEF(GCC5_IA32_X64_DLINK_FLAGS) -Wl,-Oz -Wl,-melf_x86_64 -Wl,--entry,ReferenceAcpiTable -Wl,-u,ReferenceAcpiTable -Wl,-pie -mcmodel=small
> +*_CLANG38_X64_ASM_FLAGS            = DEF(GCC5_ASM_FLAGS) -m64 DEF(CLANG38_X64_TARGET)
> +DEBUG_CLANG38_X64_CC_FLAGS         = DEF(CLANG38_ALL_CC_FLAGS) -m64 "-DEFIAPI=__attribute__((ms_abi))" -mno-red-zone -mcmodel=small -fpie -Oz -flto DEF(CLANG38_X64_TARGET) -g
> +RELEASE_CLANG38_X64_CC_FLAGS       = DEF(CLANG38_ALL_CC_FLAGS) -m64 "-DEFIAPI=__attribute__((ms_abi))" -mno-red-zone -mcmodel=small -fpie -Oz -flto DEF(CLANG38_X64_TARGET)
> +*_CLANG38_X64_DLINK_FLAGS          = DEF(GCC5_IA32_X64_DLINK_FLAGS) -Wl,-Oz -Wl,-melf_x86_64 -Wl,--oformat=elf64-x86-64 -Wl,-pie -mcmodel=small
> +*_CLANG38_X64_DLINK2_FLAGS         = DEF(GCC5_X64_DLINK2_FLAGS)
> +*_CLANG38_X64_RC_FLAGS             = DEF(GCC_X64_RC_FLAGS)
> +*_CLANG38_X64_OBJCOPY_FLAGS        =
> +*_CLANG38_X64_NASM_FLAGS           = -f elf64
> +*_CLANG38_X64_PP_FLAGS             = DEF(GCC_PP_FLAGS) DEF(CLANG38_X64_TARGET)
> +*_CLANG38_X64_ASLPP_FLAGS          = DEF(GCC_ASLPP_FLAGS) DEF(CLANG38_X64_TARGET)
> +*_CLANG38_X64_VFRPP_FLAGS          = DEF(GCC_VFRPP_FLAGS) DEF(CLANG38_X64_TARGET)
> +
> +####################################################################################
> +#
>  # Cygwin GCC And Intel ACPI Compiler
>  #
>  ####################################################################################
> --
> 2.7.4
>
> _______________________________________________
> edk2-devel mailing list
> edk2-devel@lists.01.org
> https://lists.01.org/mailman/listinfo/edk2-devel

Re: [PATCH 2/4] BaseTools-Conf:Introduce CLANG38 new toolchain for x86

[Shi, Steven]

Hello Ard,

> 
> Hello Steven,
> 
> On 3 August 2016 at 08:48, Shi, Steven <steven.shi@intel.com> wrote:
> > This adds support for LLVM 3.8.x in LTO mode for IA32 and X64.
> > CLANG38 enable LLVM Link Time Optimization (LTO) and code size
> > optimization flag (-Oz) by default for aggressive code size
> > improvement. CLANG38 X64 code is small code model + PIE.
> >
> > CLANG LTO needs PIE in link flags to generate PIE code correctly,
> > otherwise the PIE is not really enabled. (e.g. OvmfPkgX64 will
> > hang in 64bits SEC at high address because of small model code
> > displacement overflow).
> >
> 
> This is probably caused by the same issue I am addressing with the
> series I sent out yesterday, to pass the CC flags to the DLINK
> command.
> 
> The reason is that code is generated by the link pass, so it needs to
> see the same -fpie -mcmodel=small options that we passed to the
> compiler as wel.
> 
> Could you check whether replacing '-Wl,-pie' with -fpie does the trick
> as well? As I mentioned before, creating a PIE executable at link time
> is not the same as generatic position independent code at compile time
> (whether it is via $(CC) or via $(DLINK)). The PIE executable will
> contain a .rela section that partially overlaps with other absolute
> relocations, so it is best to avoid it.

[Steven]: I just tried it. No, replacing '-Wl,-pie' with -fpie cannot works for clang38. With -fpie in link flags, the OvmfPkgX64 still hang in 64bits SEC at high address.


> 
> Some more comments below
> 
> > +*_CLANG38_IA32_OBJCOPY_PATH         = DEF(GCC5_IA32_PREFIX)objcopy
> 
> Why are you using the GCC5 prefix here? A clang user may not set GCC5_BIN
> 
[Steven]: OK, I will remove the GCC5 prefix.


> > +*_CLANG38_IA32_ASLCC_FLAGS          = -x c -Os -m32
> DEF(CLANG38_IA32_TARGET) -flto
> 
> Does LTO make any sense for ACPI tables?
> 
[Steven]: OK, I will follow GCC5 to disable the LTO for ASLCC flags.


Thanks
Steven