From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from ma1-aaemail-dr-lapp01.apple.com (ma1-aaemail-dr-lapp01.apple.com [17.171.2.60]) by mx.groups.io with SMTP id smtpd.web10.7874.1573149247932911369 for ; Thu, 07 Nov 2019 09:54:08 -0800 Authentication-Results: mx.groups.io; dkim=pass header.i=@apple.com header.s=20180706 header.b=sFsJUWqr; spf=pass (domain: apple.com, ip: 17.171.2.60, mailfrom: afish@apple.com) Received: from pps.filterd (ma1-aaemail-dr-lapp01.apple.com [127.0.0.1]) by ma1-aaemail-dr-lapp01.apple.com (8.16.0.27/8.16.0.27) with SMTP id xA7HaaXc037916; Thu, 7 Nov 2019 09:54:06 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=apple.com; h=sender : content-type : mime-version : subject : from : in-reply-to : date : cc : content-transfer-encoding : message-id : references : to; s=20180706; bh=JDN9h5nAPJfHBTlOAlkQA1m8lokNWKeE7i8oRRAfNeE=; b=sFsJUWqrl4r47ZluUuqp9Y21HhN5TbHUDmHe5eNag0egjzzoA7EQSvb5eT1g/XQR1yhA u3daGVYfnZ63rAUKtDncNE9fKaofNCg91b2CwjmSbsYW/BlOB36E+Fgo6zMTDS5QXFvU ZadHrkgEG8KzTGsRNOfCBZWHH/sxG2tpkJh3KrkURExwSMva5SWJO+vfUW62gAeJcMz1 JNsWnLEUkuMw3yIGd3LsuAi5xJGfNgbVITY4wsHHXDOr3Ko/s/usgmmRkd0g/5e0J+tC eSUy2yKxbuT+bqzpvdQKLZN228siM1oojub4Wo1uYuUFnLcbalCsxUO7vUS2MBxp+VJq SA== Received: from ma-mailsvcp-mta-lapp03.corp.apple.com (ma-mailsvcp-mta-lapp03.corp.apple.com [10.226.18.135]) by ma1-aaemail-dr-lapp01.apple.com with ESMTP id 2w41v5ghh9-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NO); Thu, 07 Nov 2019 09:54:06 -0800 Received: from ma-mailsvcp-mmp-lapp04.apple.com (ma-mailsvcp-mmp-lapp04.apple.com [17.32.222.17]) by ma-mailsvcp-mta-lapp03.corp.apple.com (Oracle Communications Messaging Server 8.1.0.1.20190704 64bit (built Jul 4 2019)) with ESMTPS id <0Q0M003VN1Q6A620@ma-mailsvcp-mta-lapp03.corp.apple.com>; Thu, 07 Nov 2019 09:54:06 -0800 (PST) Received: from process_milters-daemon.ma-mailsvcp-mmp-lapp04.apple.com by ma-mailsvcp-mmp-lapp04.apple.com (Oracle Communications Messaging Server 8.1.0.1.20190704 64bit (built Jul 4 2019)) id <0Q0M00N000X25I00@ma-mailsvcp-mmp-lapp04.apple.com>; Thu, 07 Nov 2019 09:54:06 -0800 (PST) X-Va-A: X-Va-T-CD: c03d43da4ec6d5e28175d4b24b8703e8 X-Va-E-CD: 0b3ad656057f4d390b5c3920634d9433 X-Va-R-CD: ebad80bb98cf8d02fa370c056d39fa63 X-Va-CD: 0 X-Va-ID: 9097cf9e-1197-42e1-96e8-1c685fa69103 X-V-A: X-V-T-CD: c03d43da4ec6d5e28175d4b24b8703e8 X-V-E-CD: 0b3ad656057f4d390b5c3920634d9433 X-V-R-CD: ebad80bb98cf8d02fa370c056d39fa63 X-V-CD: 0 X-V-ID: e9bc5be7-5e99-497b-895a-0ef70dfc86b9 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:,, definitions=2019-11-07_05:,, signatures=0 Received: from [17.103.45.53] (unknown [17.103.45.53]) by ma-mailsvcp-mmp-lapp04.apple.com (Oracle Communications Messaging Server 8.1.0.1.20190704 64bit (built Jul 4 2019)) with ESMTPSA id <0Q0M00R5E1Q4QU30@ma-mailsvcp-mmp-lapp04.apple.com>; Thu, 07 Nov 2019 09:54:06 -0800 (PST) Sender: afish@apple.com MIME-version: 1.0 (Mac OS X Mail 13.0 \(3594.4.17\)) Subject: Re: [edk2-devel] [Patch v3 10/11] EmulatorPkg: Enable CLANG9 tool chain From: "Andrew Fish" In-reply-to: <20191107171947.GU16820@bivouac.eciton.net> Date: Thu, 07 Nov 2019 11:54:04 -0600 Cc: liming.gao@intel.com, "Ni, Ray" , Jordan Justen Message-id: <36A93358-FADB-4201-9C3F-AFFA2D7D4F0C@apple.com> References: <1571295356-11608-1-git-send-email-liming.gao@intel.com> <1571295356-11608-11-git-send-email-liming.gao@intel.com> <734D49CCEBEEF84792F5B80ED585239D5C325BB0@SHSMSX104.ccr.corp.intel.com> <34BE6669-004C-48AD-A09D-FE1A8475ABE0@apple.com> <4A89E2EF3DFEDB4C8BFDE51014F606A14E51EC84@SHSMSX104.ccr.corp.intel.com> <4A89E2EF3DFEDB4C8BFDE51014F606A14E534DF8@SHSMSX104.ccr.corp.intel.com> <20191107171947.GU16820@bivouac.eciton.net> To: devel@edk2.groups.io, leif.lindholm@linaro.org X-Mailer: Apple Mail (2.3594.4.17) X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:,, definitions=2019-11-07_05:,, signatures=0 Content-type: text/plain; charset=utf-8 Content-transfer-encoding: quoted-printable Leif, I think I proposed CLANG_PDB or CLANG_PECOFF. I seem to like CLANG_PDB as = assuming the PDE debugging experience is awesome/exists on Linux and macOS = is not a given.=20 Thanks, Andrew Fish > On Nov 7, 2019, at 11:19 AM, Leif Lindholm wr= ote: >=20 > Oops, sorry, missed this in search. >=20 > On Wed, Oct 30, 2019 at 03:43:44PM +0000, Liming Gao wrote: >> Andrew: >>=20 >> I prefer to keep short CLANG9 as the tool chain name. I add wiki >> page >> https://github.com/tianocore/tianocore.github.io/wiki/CLANG9-Tools-Cha= in >> to introduce it. >=20 > Why should users be expected to go and read documentation in order to > learn that fundamental and incompatible changes to output and debug > format has been made as part of what looks like a simple version bump? >=20 >> And, we have CLANG38 tool chain. It generates ELF >> image and DWARF debug symbol format. It can work with LLVM 9.0 >> release. Current tool chain tag name includes compiler name and >> version. There is no specific info in the tool chain name. The >> developer can get the more tool chain information from wiki page. >=20 > We aleady have the problem that people think they need to use GCC5 to > build EDK2 since that is the highest named GCC toolchain profile. >=20 > Let's not make this situation *worse* by setting up a multidimensional > feature matrix, based off random numerical values. >=20 > So say that we next have a pressing need to create a new toolchain > profile for clang 10, using the old ELF mechanism. Are you then > suggesting we set up a table on said wiki page where people have to go > and look up what the object and debug formats are for each CLANG## > toolchain profile? >=20 > And what if you then end up needing to do the same for the PDB > flavour? >=20 >> CLANG9 is designed to support Emulator for Windows host only. >=20 > Which is why it makes no sense to name it as a successor of > CLANG38. >=20 >> CLANG38 may be used for Emulator in Linux or Mac. I don=E2=80=99t >> try it before. >>=20 >> CLANG9 goal is to align the same compiler in the different host >> development environment. It can replace VS or GCC compiler. On >> Windows Host, I verify VS debugger for the source level debug. On >> Linux host, I have not verified llvm debugger. I will investigate >> the debugger solution for OVMF in Linux host. >=20 > We are not asking you to throw out this toolchain profile. >=20 > We are saying that since the functionality it provides is completely > unrelated to that of CLANG38, it should not be named in a way that > suggests it is merely a revision update. >=20 > / > Leif >=20 >> Thanks >> Liming >> From: afish@apple.com >> Sent: Saturday, October 26, 2019 2:45 AM >> To: devel@edk2.groups.io; Gao, Liming >> Cc: Ni, Ray ; Justen, Jordan L >> Subject: Re: [edk2-devel] [Patch v3 10/11] EmulatorPkg: Enable CLANG9 t= ool chain >>=20 >> Liming, >>=20 >> Sorry I missed this mail. Thanks for the info! I was doing some researc= h into this too and now I think I finally understand. I think the name for = the tool chain really confused me and we should think about changing the na= me. >>=20 >> From what I understand CLANG9 means produce PE/COFF directly and used t= he PDB debugging format. I see from the llvm site that the linker can produ= ce PDB directly as you mention. This all makes sense to me now as LLVM trie= s to make it easy to be a drop in replacement for VC++ or GCC. So this tool= chain is designed to be able to cross build a "Windows App" on a Linux or = macOS. It also looks like the llvm debugger, lldb, is lagging in its suppor= t for PDB based debugging. >>=20 >>=20 >> Anyway I think Leif and I agree the toolchain name is very confusing. I= 'd rather see it called CLANG9_WIN or CLANG_PDB. >>=20 >>=20 >> On Oct 18, 2019, at 7:27 AM, Liming Gao > wrote: >>=20 >> Andrew: >> Here is the cover letter on CLANG9 introduction. https://edk2.groups.io= /g/devel/message/49157 >>=20 >> 1) Yes. CLANG9 tool chain is added to directly generate PE/COFF im= age (EFI image). >> This tool chain uses LLVM clang C compiler and lld linker, generates PE= /COFF >> image and PDB compatible debug symbol format. Now, it supports IA32/X64= Archs. >> LLVM clang C compiler and lld linker are the standalone tool set. They = don=E2=80=99t depend other lib to generate PE/COFF image. >>=20 >> 2) Yes. CLANG9 is the cross OS tool chain. It can work on Windows/= Linux/Mac host OS. >> LLVM LLD linker uses Windows style arguments. I verify CLANG9 for Ovmf3= 264 in Windows/Linux host OS. >>=20 >>=20 >> On Linux can you source level debug Ovmf? >>=20 >>=20 >> 3) This patch enables WinHost in Windows. It doesn=E2=80=99t enable= UnixHost. Now, EmulatorPkg with CLANG9 only works on Windows Host. >> This patch can make other modules pass build in Windows/Linux/Mac only = if LLVM9 tool set is installed. >> But, the generated image may not work on Linux/Mac. I agree below linke= r flags are specific to windows host. >> So, I suggest to add the conditional statement of $(WIN_HOST_BUILD) =3D= = =3D TRUE for them. >>=20 >>=20 >> For the EmulatorPkg the Host is a native App for that OS you build on, = but it seems like CLANG9 is targeted to build Windows Apps. I'm not sure b= ut you might be able to override all the linker commands to build a native = app, or just use the system linker for the Host? >>=20 >> I'm not sure how well debugging will work mixing PDB and DWARF symbol f= ormats? >>=20 >> Thanks, >>=20 >> Andrew Fish >>=20 >>=20 >>=20 >> !if $(WIN_HOST_BUILD) =3D=3D TRUE >> GCC:*_CLANG9_*_DLINK_FLAGS =3D /ALIGN:4096 /FILEALIGN:4096 /SUBSYS= TEM:CONSOLE >> GCC:DEBUG_CLANG9_*_DLINK_FLAGS =3D /EXPORT:InitializeDriver=3D$(IMAGE_= ENTRY_POINT) /BASE:0x10000 >> GCC:NOOPT_CLANG9_*_DLINK_FLAGS =3D /EXPORT:InitializeDriver=3D$(IMAGE_= ENTRY_POINT) /BASE:0x10000 >> !endif >>=20 >> Thanks >> Liming >> From: afish@apple.com > >> Sent: Friday, October 18, 2019 1:15 AM >> To: Ni, Ray > >> Cc: Gao, Liming >; de= vel@edk2.groups.io; Justen, Jordan L > >> Subject: Re: [Patch v3 10/11] EmulatorPkg: Enable CLANG9 tool chain >>=20 >> Ray, >>=20 >> Sorry I'm coming a little late to this and I'm confused. I have some qu= estions? >> 1) Does CLANG9 imply CLANGPE? >> 2) Does CLANGPE work on Linux and macOS? Can you pass the Windows style= arguments to CLANGPE linker on Linux and macOS? >> 3) For the EmulatorPkg don't you have the extra requirement that compil= er needs a standard C lib (or platform specific libs) to function? >> a) GCC:*_CLANG9_*_DLINK_FLAGS in EmulatorPkg.dsc seems to imply t= he Linux and macOS systems will have a Windows SKD dir and a lot of Windows= DLLs? Does all that come when you install CLANG9 when you install it on Li= nux or macOS? >>=20 >>=20 >>=20 >> So I guess I'm asking is the linker really the same for CLANG9 on all s= ystems? I guess the answer could be yes, but it seems the C lib for the Hos= t App is still an App for that OS and is OS dependent? >>=20 >>=20 >>=20 >> Sorry if I'm missing something fundamental and this is a dumb question. >>=20 >>=20 >>=20 >> Thanks, >>=20 >>=20 >>=20 >> Andrew Fish >>=20 >>=20 >>=20 >>=20 >>=20 >>=20 >>=20 >> On Oct 17, 2019, at 12:27 AM, Ni, Ray > wrote: >>=20 >> Liming, >> Emulator is using a generic SEC module. The host specific module is cal= led "Host". >> So I prefer to change the macro to "WIN_HOST_BUILD", with this change, = Reviewed-by: Ray Ni > >>=20 >>=20 >>=20 >> -----Original Message----- >> From: Gao, Liming > >> Sent: Thursday, October 17, 2019 2:56 PM >> To: devel@edk2.groups.io >> Cc: Justen, Jordan L >; Andrew Fish >> >; Ni, Ray > >> Subject: [Patch v3 10/11] EmulatorPkg: Enable CLANG9 tool chain >>=20 >> BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=3D1603 >> 1. Add WIN_SEC_BUILD macro check for CLANG9 tool chain build -p >> EmulatorPkg\EmulatorPkg.dsc -a IA32 -DWIN_SEC_BUILD=3DTRUE -t CLANG9 >> build -p EmulatorPkg\EmulatorPkg.dsc -a X64 -DWIN_SEC_BUILD=3DTRUE -t >> CLANG9 2. Append CLANG CC and LINK flags to generate windows HOST. >> 3. Fix WinHost issue to call GetProcessAffinityMask() API. >> The input parameter should be UINTN pointer instead of UINT32 pointer. >>=20 >> Cc: Jordan Justen > >> Cc: Andrew Fish > >> Cc: Ray Ni > >> Signed-off-by: Liming Gao > >> --- >> EmulatorPkg/Win/Host/WinHost.c | 6 +++--- >> EmulatorPkg/EmulatorPkg.dsc | 7 ++++++- >> EmulatorPkg/Win/Host/WinHost.inf | 6 ++++++ >> 3 files changed, 15 insertions(+), 4 deletions(-) >>=20 >> diff --git a/EmulatorPkg/Win/Host/WinHost.c >> b/EmulatorPkg/Win/Host/WinHost.c index 9aba3c8959..e40ce32548 100644 >> --- a/EmulatorPkg/Win/Host/WinHost.c >> +++ b/EmulatorPkg/Win/Host/WinHost.c >> @@ -356,7 +356,7 @@ Returns: >> INTN >> EFIAPI >> main ( >> - IN INTN Argc, >> + IN INT Argc, >> IN CHAR8 **Argv, >> IN CHAR8 **Envp >> ) >> @@ -391,8 +391,8 @@ Returns: >> VOID *SecFile; >> CHAR16 *MemorySizeStr; >> CHAR16 *FirmwareVolumesStr; >> - UINT32 ProcessAffinityMask; >> - UINT32 SystemAffinityMask; >> + UINTN ProcessAffinityMask; >> + UINTN SystemAffinityMask; >> INT32 LowBit; >>=20 >> // >> diff --git a/EmulatorPkg/EmulatorPkg.dsc b/EmulatorPkg/EmulatorPkg.dsc >> index 20f1187713..72532f5daf 100644 >> --- a/EmulatorPkg/EmulatorPkg.dsc >> +++ b/EmulatorPkg/EmulatorPkg.dsc >> @@ -237,9 +237,10 @@ >>=20 >> [Components] >> !if "IA32" in $(ARCH) || "X64" in $(ARCH) >> - !if "MSFT" in $(FAMILY) >> + !if "MSFT" in $(FAMILY) || $(WIN_SEC_BUILD) =3D=3D TRUE >> ## >> # Emulator, OS WIN application >> + # CLANG9 is cross OS tool chain. It depends on WIN_SEC_BUILD macr= o. >> ## >> EmulatorPkg/Win/Host/WinHost.inf >> !else >> @@ -419,7 +420,11 @@ >>=20 >> MSFT:DEBUG_*_*_CC_FLAGS =3D /Od /Oy- >> MSFT:NOOPT_*_*_CC_FLAGS =3D /Od /Oy- >> + GCC:DEBUG_CLANG9_*_CC_FLAGS =3D-O0 -Wno-unused-command-line- >> argument >> + -Wno-incompatible-pointer-types -Wno-enum-conversion >> + -Wno-incompatible-pointer-types -Wno-sometimes-uninitialized >> + -Wno-constant-conversion -Wno-main-return-type >>=20 >> MSFT:*_*_*_DLINK_FLAGS =3D /ALIGN:4096 /FILEALIGN:4096 >> /SUBSYSTEM:CONSOLE >> MSFT:DEBUG_*_*_DLINK_FLAGS =3D >> /EXPORT:InitializeDriver=3D$(IMAGE_ENTRY_POINT) /BASE:0x10000 >> MSFT:NOOPT_*_*_DLINK_FLAGS =3D >> /EXPORT:InitializeDriver=3D$(IMAGE_ENTRY_POINT) /BASE:0x10000 >> + GCC:*_CLANG9_*_DLINK_FLAGS =3D /ALIGN:4096 /FILEALIGN:4096 >> /SUBSYSTEM:CONSOLE >> + GCC:DEBUG_CLANG9_*_DLINK_FLAGS =3D >> + /EXPORT:InitializeDriver=3D$(IMAGE_ENTRY_POINT) /BASE:0x10000 >> + GCC:NOOPT_CLANG9_*_DLINK_FLAGS =3D >> + /EXPORT:InitializeDriver=3D$(IMAGE_ENTRY_POINT) /BASE:0x10000 >> diff --git a/EmulatorPkg/Win/Host/WinHost.inf >> b/EmulatorPkg/Win/Host/WinHost.inf >> index 631d5a6470..1adca10d79 100644 >> --- a/EmulatorPkg/Win/Host/WinHost.inf >> +++ b/EmulatorPkg/Win/Host/WinHost.inf >> @@ -95,3 +95,9 @@ >> MSFT:*_VS2017_X64_DLINK_FLAGS =3D >> /LIBPATH:"%VCToolsInstallDir%lib\x64" >> /LIBPATH:"%UniversalCRTSdkDir%lib\%UCRTVersion%\ucrt\x64" >> /LIBPATH:"%WindowsSdkDir%lib\%WindowsSDKLibVersion%\um\x64" >> /NOLOGO /SUBSYSTEM:CONSOLE /NODEFAULTLIB /IGNORE:4086 /MAP >> /OPT:REF /DEBUG /MACHINE:AMD64 /LTCG Kernel32.lib MSVCRTD.lib >> vcruntimed.lib ucrtd.lib Gdi32.lib User32.lib Winmm.lib Advapi32.lib >> MSFT:*_*_X64_ASM_FLAGS =3D=3D /nologo /W3 /WX /c /Cx /Zd /W= 0 /Zi >> MSFT:*_*_X64_ASMLINK_FLAGS =3D=3D /link /nologo >> + >> + GCC:*_CLANG9_X64_DLINK_FLAGS =3D=3D >> /out:"$(BIN_DIR)\$(BASE_NAME).exe" /base:0x10000000 >> /pdb:"$(BIN_DIR)\$(BASE_NAME).pdb" >> /LIBPATH:"%UniversalCRTSdkDir%lib\%UCRTVersion%\ucrt\x64" >> /LIBPATH:"%WindowsSdkDir%lib\%WindowsSDKLibVersion%\um\x64" >> /LIBPATH:"%VCToolsInstallDir%lib\x64" /NOLOGO /SUBSYSTEM:CONSOLE >> /NODEFAULTLIB /IGNORE:4086 /OPT:REF /DEBUG /MACHINE:AMD64 >> Kernel32.lib MSVCRTD.lib vcruntimed.lib ucrtd.lib Gdi32.lib User32.lib >> Winmm.lib Advapi32.lib /lldmap >> /EXPORT:InitializeDriver=3D_ModuleEntryPoint >> + GCC:*_CLANG9_X64_CC_FLAGS =3D=3D -m64 -g -fshort-wchar >> + -fno-strict-aliasing -Wall -c -include AutoGen.h -D >> + _CRT_SECURE_NO_WARNINGS -Wnonportable-include-path -D UNICODE >> -D >> + _CRT_SECURE_NO_DEPRECATE >> + >> + GCC:*_CLANG9_IA32_DLINK_FLAGS =3D=3D >> /out:"$(BIN_DIR)\$(BASE_NAME).exe" /base:0x10000000 >> /pdb:"$(BIN_DIR)\$(BASE_NAME).pdb" >> /LIBPATH:"%UniversalCRTSdkDir%lib\%UCRTVersion%\ucrt\x86" >> /LIBPATH:"%WindowsSdkDir%lib\%WindowsSDKLibVersion%\um\x86" >> /LIBPATH:"%VCToolsInstallDir%ib\x86" /NOLOGO /SUBSYSTEM:CONSOLE >> /NODEFAULTLIB /IGNORE:4086 /OPT:REF /DEBUG /MACHINE:I386 >> Kernel32.lib MSVCRTD.lib vcruntimed.lib ucrtd.lib Gdi32.lib User32.lib >> Winmm.lib Advapi32.lib /lldmap >> /EXPORT:InitializeDriver=3D_ModuleEntryPoint >> + GCC:*_CLANG9_IA32_CC_FLAGS =3D=3D -m32 -g -fshort-wchar >> + -fno-strict-aliasing -Wall -c -include AutoGen.h -D >> + _CRT_SECURE_NO_WARNINGS -Wnonportable-include-path -D UNICODE >> -D >> + _CRT_SECURE_NO_DEPRECATE >> -- >> 2.13.0.windows.1 >>=20 >>=20 >>=20 >>=20 >>=20 >>=20 >>=20 >=20 >=20 >=20