Re: [edk2-devel] [Patch v3 10/11] EmulatorPkg: Enable CLANG9 tool chain

["Liming Gao" <liming.gao@intel.com>]

Ray:
 Thanks for your detail investigation. It is hard to decide the name, because this is not purely technical problem.
 I summary current feedback on this tool chain name as below. 
 The result shows that CLANGPDB is the acceptable tool chain name, although not every one prefers this name. 

Ray: CLANGCODEVIEW, CLANGCV or CLANGPDB.
Mike: CLANGCODEVIEW, CLANGCV or CLANGPDB.
Andrew: CLANGPE, CLANGPDB (Prefer)
Leif: CLANGPE (Prefer), CLANGPDB, PECLANG, PDBCLANG
Liming: CLANGPE (Prefer), CLANGPDB

Thanks
Liming
> -----Original Message-----
> From: Ni, Ray <ray.ni@intel.com>
> Sent: Tuesday, November 12, 2019 2:49 PM
> To: Kinney, Michael D <michael.d.kinney@intel.com>; devel@edk2.groups.io; Gao, Liming <liming.gao@intel.com>; afish@apple.com;
> Leif Lindholm <leif.lindholm@linaro.org>
> Cc: Justen, Jordan L <jordan.l.justen@intel.com>; Shi, Steven <steven.shi@intel.com>
> Subject: RE: [edk2-devel] [Patch v3 10/11] EmulatorPkg: Enable CLANG9 tool chain
> 
> Mike,
> I did a simple research in the web and found the slides:
> https://llvm.org/devmtg/2016-11/Slides/Kleckner-CodeViewInLLVM.pdf
> 
> In page #3, it says:
> "CodeView is stored inside object files and PDBs"
> "PDB is a container format written by linker and read by debugger"
> 
> The relationships between terms are:
> "CodeView :: DWARF"
> "PDB :: dSYM"
> "PDB :: DWP"
> 
> 
> I also found information from "https://llvm.org/docs/PDB/index.html":
> "CodeView is another format which comes into the picture. While MSF
> defines the structure of the overall file, and PDB defines the set of streams
> that appear within the MSF file and the format of those streams, CodeView
> defines the format of symbol and type records that appear within specific
> streams."
> 
> With all these information, I am ok with any of CLANGCODEVIEW, CLANGCV or CLANGPDB.
> 
> Developers who care about symbolic debugging should be aware of what kind of
> debug symbol is generated by build.
> 
> Thanks,
> Ray
> 
> > -----Original Message-----
> > From: Kinney, Michael D <michael.d.kinney@intel.com>
> > Sent: Tuesday, November 12, 2019 11:48 AM
> > To: devel@edk2.groups.io; Ni, Ray <ray.ni@intel.com>; Gao, Liming
> > <liming.gao@intel.com>; afish@apple.com; Leif Lindholm
> > <leif.lindholm@linaro.org>; Kinney, Michael D <michael.d.kinney@intel.com>
> > Cc: Justen, Jordan L <jordan.l.justen@intel.com>; Shi, Steven
> > <steven.shi@intel.com>
> > Subject: RE: [edk2-devel] [Patch v3 10/11] EmulatorPkg: Enable CLANG9 tool
> > chain
> >
> > Ray,
> >
> > I agree that a name that represents the symbolic debug format makes more
> > sense.
> >
> > The term 'PE' is short for PE/COFF, and that is the required final output
> > format.  A toolchain profile that happens to go through other intermediate
> > formats (e.g.
> > elf or macho) before finally generating a PE/COFF image is usually not that
> > interesting to most developers.  So adding 'PE' to the name is redundant
> > information.
> >
> > The attribute of the images that does impact developers is the format of the
> > symbol information.  Even for Visual Studio, PDB is the file extension used
> > when the symbol information is placed in a separate file.  The Visual Studio
> > compilers do support putting the symbol information in a section of the
> > PE/COFF image.  EDK II builds happen to always use the PDB option to
> > minimize the size of firmware images. So PDB may not be the right name
> > either.  I think the symbol format in the PDB that is used is codeview.
> >
> > If we want to plan ahead for toolchain names perhaps:
> >
> > Codeview: CLANGCODEVIEW or CLANGCV or CLANGPDB
> > Dwarf:    CLANGDWARF
> >
> > Best regards,
> >
> > Mike
> >
> > > -----Original Message-----
> > > From: devel@edk2.groups.io <devel@edk2.groups.io> On Behalf Of Ni,
> > Ray
> > > Sent: Monday, November 11, 2019 7:23 PM
> > > To: devel@edk2.groups.io; Ni, Ray <ray.ni@intel.com>; Gao, Liming
> > > <liming.gao@intel.com>; afish@apple.com; Leif Lindholm
> > > <leif.lindholm@linaro.org>
> > > Cc: Justen, Jordan L <jordan.l.justen@intel.com>; Shi, Steven
> > > <steven.shi@intel.com>
> > > Subject: Re: [edk2-devel] [Patch v3 10/11] EmulatorPkg:
> > > Enable CLANG9 tool chain
> > >
> > > And LLVM I believe is capable to support generating PE + DWARF.
> > > That's my second concern: Using CLANGPE may not imply PDB is used from
> > > a LLVM expert's understanding.
> > >
> > >
> > > > -----Original Message-----
> > > > From: devel@edk2.groups.io <devel@edk2.groups.io> On
> > > Behalf Of Ni, Ray
> > > > Sent: Tuesday, November 12, 2019 11:10 AM
> > > > To: devel@edk2.groups.io; Ni, Ray <ray.ni@intel.com>;
> > > Gao, Liming
> > > > <liming.gao@intel.com>; afish@apple.com; Leif Lindholm
> > > > <leif.lindholm@linaro.org>
> > > > Cc: Justen, Jordan L <jordan.l.justen@intel.com>; Shi,
> > > Steven
> > > > <steven.shi@intel.com>
> > > > Subject: Re: [edk2-devel] [Patch v3 10/11]
> > > EmulatorPkg: Enable CLANG9
> > > > tool chain
> > > >
> > > > My point is if we choose CLANGPE, for LLVM toolchain
> > > supporting DWARF
> > > > debug symbol, there is no proper name left: we cannot
> > > use CLANGELF.
> > > >
> > > > Andrew, you said CLANGPDE. A typo?
> > > >
> > > > > -----Original Message-----
> > > > > From: devel@edk2.groups.io <devel@edk2.groups.io> On
> > > Behalf Of Ni,
> > > > Ray
> > > > > Sent: Tuesday, November 12, 2019 9:46 AM
> > > > > To: Gao, Liming <liming.gao@intel.com>;
> > > devel@edk2.groups.io;
> > > > > afish@apple.com; Leif Lindholm
> > > <leif.lindholm@linaro.org>
> > > > > Cc: Justen, Jordan L <jordan.l.justen@intel.com>;
> > > Shi, Steven
> > > > > <steven.shi@intel.com>
> > > > > Subject: Re: [edk2-devel] [Patch v3 10/11]
> > > EmulatorPkg: Enable
> > > > > CLANG9 tool chain
> > > > >
> > > > > Liming,
> > > > > What name will be chosen when in future someone
> > > wants a LLVM
> > > > toolchain
> > > > > which supports DWARF debug symbol?
> > > > >
> > > > >
> > > > > > -----Original Message-----
> > > > > > From: Gao, Liming <liming.gao@intel.com>
> > > > > > Sent: Tuesday, November 12, 2019 8:41 AM
> > > > > > To: devel@edk2.groups.io; afish@apple.com; Leif
> > > Lindholm
> > > > > > <leif.lindholm@linaro.org>
> > > > > > Cc: Ni, Ray <ray.ni@intel.com>; Justen, Jordan L
> > > > > > <jordan.l.justen@intel.com>; Shi, Steven
> > > <steven.shi@intel.com>;
> > > > > > Gao, Liming <liming.gao@intel.com>
> > > > > > Subject: RE: [edk2-devel] [Patch v3 10/11]
> > > EmulatorPkg: Enable
> > > > > > CLANG9 tool chain
> > > > > >
> > > > > > Thanks for your feedback. I will choose CLANGPE as
> > > the tool chain name.
> > > > > >
> > > > > > Thanks
> > > > > > Liming
> > > > > > >-----Original Message-----
> > > > > > >From: devel@edk2.groups.io
> > > [mailto:devel@edk2.groups.io] On
> > > > > > >Behalf Of Andrew Fish via Groups.Io
> > > > > > >Sent: Tuesday, November 12, 2019 12:52 AM
> > > > > > >To: Leif Lindholm <leif.lindholm@linaro.org>
> > > > > > >Cc: Ni, Ray <ray.ni@intel.com>; Gao, Liming
> > > > > > ><liming.gao@intel.com>; devel@edk2.groups.io;
> > > Justen, Jordan L
> > > > > > ><jordan.l.justen@intel.com>; Shi, Steven
> > > <steven.shi@intel.com>
> > > > > > >Subject: Re: [edk2-devel] [Patch v3 10/11]
> > > EmulatorPkg: Enable
> > > > > > >CLANG9 tool chain
> > > > > > >
> > > > > > >Either works for me too. I like the PDE ending a
> > > little more, but
> > > > > > >agree it is a little more obscure.
> > > > > > >> On Nov 11, 2019, at 8:39 AM, Leif Lindholm
> > > > > > >> <leif.lindholm@linaro.org>
> > > > > > wrote:
> > > > > > >>
> > > > > > >> On Fri, Nov 08, 2019 at 03:34:59AM +0000, Ni,
> > > Ray wrote:
> > > > > > >>> Liming,
> > > > > > >>> PE is UEFI spec required executable binary
> > > format. I like
> > > > > > >>> CLANGPDB more than CLANGPE given LLVM may
> > > generate DRAWF
> > > > debug
> > > > > > >>> symbol
> > > > > > >even
> > > > > > >>> when generating PE.
> > > > > > >>
> > > > > > >> Just a comment here to point out that while (as
> > > I stated in
> > > > > > >> reply to
> > > > > > >> Liming) my preference would be CLANGPE, I see
> > > the logic in the
> > > > > > >> above, and would be happy with either.
> > > > > > >>
> > > > > > >> (My reason for preferring CLANGPE is not
> > > exactly techincal - I
> > > > > > >> just expect more people to know what PE is than
> > > who know what
> > > > > > >> PDB
> > > > > > >> is.)
> > > > > > >>
> > > > > > >> Regards,
> > > > > > >>
> > > > > > >> Leif
> > > > > > >>
> > > > > > >>>> -----Original Message-----
> > > > > > >>>> From: Gao, Liming <liming.gao@intel.com>
> > > > > > >>>> Sent: Friday, November 8, 2019 9:50 AM
> > > > > > >>>> To: devel@edk2.groups.io;
> > > leif.lindholm@linaro.org; Andrew
> > > > > > >>>> Fish
> > > > > > ><afish@apple.com>
> > > > > > >>>> Cc: Ni, Ray <ray.ni@intel.com>; Justen,
> > > Jordan L
> > > > > > ><jordan.l.justen@intel.com>; Gao, Liming
> > > <liming.gao@intel.com>;
> > > > > > >Shi,
> > > > > > >>>> Steven <steven.shi@intel.com>
> > > > > > >>>> Subject: RE: [edk2-devel] [Patch v3 10/11]
> > > EmulatorPkg:
> > > > > > >>>> Enable
> > > > > > >>>> CLANG9
> > > > > > >tool chain
> > > > > > >>>>
> > > > > > >>>> Andrew and Leif:
> > > > > > >>>>  Thanks for your comment. CLANG9 is different
> > > from CLANG38.
> > > > > > >>>> And,
> > > > > > >CLANG9 will support LLVM9 or the above.
> > > > > > >>>>  So, this tool chain should have the specific
> > > word for its
> > > > > > >>>> purpose, and
> > > > > > >remove the version 9 to avoid the confuse.
> > > > > > >>>>
> > > > > > >>>>  I prefer the tool chain name as CLANGPE or
> > > CLANGPDB. CLANGPE
> > > > > may
> > > > > > >be better. Now, CLANG9 tool chain
> > > > > > >>>>  build rule family just uses CLANGPE. CLANGPE
> > > lets user know
> > > > > > >>>> this tool
> > > > > > >chain generates PE/COFF image.
> > > > > > >>>>  And, PE/COFF image debug symbol is PDB. So,
> > > PE also means PDB.
> > > > > > >>>>
> > > > > > >>>>  Last, I just review the changes. Tool chain
> > > name update is
> > > > > > >>>> not big. If you
> > > > > > >accept CLANGPE as tool chain name,
> > > > > > >>>>  I will send the patch soon to catch 201911
> > > stable tag.
> > > > > > >>>>
> > > > > > >>>> Thanks
> > > > > > >>>> Liming
> > > > > > >>>>> -----Original Message-----
> > > > > > >>>>> From: devel@edk2.groups.io
> > > [mailto:devel@edk2.groups.io] On
> > > > > > Behalf
> > > > > > >Of Leif
> > > > > > >>>>> Lindholm
> > > > > > >>>>> Sent: Friday, November 08, 2019 2:37 AM
> > > > > > >>>>> To: Andrew Fish <afish@apple.com>
> > > > > > >>>>> Cc: devel@edk2.groups.io; Gao, Liming
> > > > > > >>>>> <liming.gao@intel.com>; Ni, Ray
> > > <ray.ni@intel.com>; Justen,
> > > > > > >>>>> Jordan L <jordan.l.justen@intel.com>
> > > > > > >>>>> Subject: Re: [edk2-devel] [Patch v3 10/11]
> > > EmulatorPkg:
> > > > > > >>>>> Enable
> > > > > > >>>>> CLANG9
> > > > > > >tool
> > > > > > >>>>> chain
> > > > > > >>>>>
> > > > > > >>>>> Hi Andrew,
> > > > > > >>>>>
> > > > > > >>>>> Yeah, I'm pretty easy with regards to what
> > > we change it to.
> > > > > > >>>>> Although if we're bikeshedding, I would
> > > prefer not using _
> > > > > > >>>>> in the toolchain profile name.
> > > > > > >>>>>
> > > > > > >>>>> I'm ... optimistic ... it won't break any of
> > > my existing
> > > > > > >>>>> scripts, but since the build system uses it
> > > as a <toolchain
> > > > > > >>>>> profile>_<architecture>_<build target>
> > > separator, it would
> > > > > > >>>>> profile>be nice
> > > > > > >>>>> if we didn't risk confusion there.
> > > > > > >>>>>
> > > > > > >>>>> (I would be totally happy with PECLANG,
> > > CLANGPE, PDBCLANG,
> > > > > > >CLANGPDB or
> > > > > > >>>>> something along those lines.)
> > > > > > >>>>>
> > > > > > >>>>> Regards,
> > > > > > >>>>>
> > > > > > >>>>> Leif
> > > > > > >>>>>
> > > > > > >>>>> On Thu, Nov 07, 2019 at 11:54:04AM -0600,
> > > Andrew Fish wrote:
> > > > > > >>>>>> Leif,
> > > > > > >>>>>>
> > > > > > >>>>>> I think I proposed CLANG_PDB or
> > > CLANG_PECOFF. I seem to
> > > > > > >>>>>> like
> > > > > > >>>>> CLANG_PDB as assuming the PDE debugging
> > > experience is
> > > > > > >awesome/exists
> > > > > > >>>>> on Linux and macOS is not a given.
> > > > > > >>>>>>
> > > > > > >>>>>> Thanks,
> > > > > > >>>>>>
> > > > > > >>>>>> Andrew Fish
> > > > > > >>>>>>
> > > > > > >>>>>>> On Nov 7, 2019, at 11:19 AM, Leif Lindholm
> > > > > > >>>>>>> <leif.lindholm@linaro.org>
> > > > > > >>>>> wrote:
> > > > > > >>>>>>>
> > > > > > >>>>>>> Oops, sorry, missed this in search.
> > > > > > >>>>>>>
> > > > > > >>>>>>> On Wed, Oct 30, 2019 at 03:43:44PM +0000,
> > > Liming Gao wrote:
> > > > > > >>>>>>>> Andrew:
> > > > > > >>>>>>>>
> > > > > > >>>>>>>> I prefer to keep short CLANG9 as the tool
> > > chain name. I
> > > > > > >>>>>>>> add wiki page
> > > > > > >>>>>>>>
> > > > >
> > > https://github.com/tianocore/tianocore.github.io/wiki/CL
> > > ANG9-
> > > > > > >Tools-
> > > > > > >>>>> Chain
> > > > > > >>>>>>>> to introduce it.
> > > > > > >>>>>>>
> > > > > > >>>>>>> Why should users be expected to go and
> > > read documentation
> > > > in
> > > > > > >order to
> > > > > > >>>>>>> learn that fundamental and incompatible
> > > changes to output
> > > > > > >>>>>>> and
> > > > > > >debug
> > > > > > >>>>>>> format has been made as part of what looks
> > > like a simple
> > > > > > >>>>>>> version
> > > > > > >bump?
> > > > > > >>>>>>>
> > > > > > >>>>>>>> And, we have CLANG38 tool chain. It
> > > generates ELF image
> > > > > > >>>>>>>> and DWARF debug symbol format. It can
> > > work with LLVM 9.0
> > > > release.
> > > > > > >>>>>>>> Current tool chain tag name includes
> > > compiler name and
> > > > version.
> > > > > > >>>>>>>> There is no specific info in the tool
> > > chain name. The
> > > > > > >>>>>>>> developer can get the more tool chain
> > > information from
> > > > > > >>>>>>>> wiki
> > > > > page.
> > > > > > >>>>>>>
> > > > > > >>>>>>> We aleady have the problem that people
> > > think they need to
> > > > > > >>>>>>> use
> > > > > > >GCC5 to
> > > > > > >>>>>>> build EDK2 since that is the highest named
> > > GCC toolchain profile.
> > > > > > >>>>>>>
> > > > > > >>>>>>> Let's not make this situation *worse* by
> > > setting up a
> > > > > > >multidimensional
> > > > > > >>>>>>> feature matrix, based off random numerical
> > > values.
> > > > > > >>>>>>>
> > > > > > >>>>>>> So say that we next have a pressing need
> > > to create a new
> > > > > > >>>>>>> toolchain profile for clang 10, using the
> > > old ELF mechanism.
> > > > > > >>>>>>> Are you then suggesting we set up a table
> > > on said wiki
> > > > > > >>>>>>> page where people have to
> > > > > > >go
> > > > > > >>>>>>> and look up what the object and debug
> > > formats are for each
> > > > > > >CLANG##
> > > > > > >>>>>>> toolchain profile?
> > > > > > >>>>>>>
> > > > > > >>>>>>> And what if you then end up needing to do
> > > the same for the
> > > > > > >>>>>>> PDB flavour?
> > > > > > >>>>>>>
> > > > > > >>>>>>>> CLANG9 is designed to support Emulator
> > > for Windows host
> > > > only.
> > > > > > >>>>>>>
> > > > > > >>>>>>> Which is why it makes no sense to name it
> > > as a successor
> > > > > > >>>>>>> of CLANG38.
> > > > > > >>>>>>>
> > > > > > >>>>>>>> CLANG38 may be used for Emulator in Linux
> > > or Mac. I don’t
> > > > > > >>>>>>>> try it before.
> > > > > > >>>>>>>>
> > > > > > >>>>>>>> CLANG9 goal is to align the same compiler
> > > in the
> > > > > > >>>>>>>> different host development environment.
> > > It can replace VS
> > > > > > >>>>>>>> or GCC compiler. On Windows Host, I
> > > verify VS debugger
> > > > > > >>>>>>>> for the source
> > > > > level debug.
> > > > > > >>>>>>>> On Linux host, I have not verified llvm
> > > debugger. I will
> > > > > > >>>>>>>> investigate the debugger solution for
> > > OVMF in Linux host.
> > > > > > >>>>>>>
> > > > > > >>>>>>> We are not asking you to throw out this
> > > toolchain profile.
> > > > > > >>>>>>>
> > > > > > >>>>>>> We are saying that since the functionality
> > > it provides is
> > > > > > >>>>>>> completely unrelated to that of CLANG38,
> > > it should not be
> > > > > > >>>>>>> named in a way that suggests it is merely
> > > a revision update.
> > > > > > >>>>>>>
> > > > > > >>>>>>> /
> > > > > > >>>>>>>   Leif
> > > > > > >>>>>>>
> > > > > > >>>>>>>> Thanks
> > > > > > >>>>>>>> Liming
> > > > > > >>>>>>>> From: afish@apple.com <afish@apple.com>
> > > > > > >>>>>>>> Sent: Saturday, October 26, 2019 2:45 AM
> > > > > > >>>>>>>> To: devel@edk2.groups.io; Gao, Liming
> > > > > > >>>>>>>> <liming.gao@intel.com>
> > > > > > >>>>>>>> Cc: Ni, Ray <ray.ni@intel.com>; Justen,
> > > Jordan L
> > > > > > >>>>> <jordan.l.justen@intel.com>
> > > > > > >>>>>>>> Subject: Re: [edk2-devel] [Patch v3
> > > 10/11] EmulatorPkg:
> > > > > > >>>>>>>> Enable
> > > > > > >CLANG9
> > > > > > >>>>> tool chain
> > > > > > >>>>>>>>
> > > > > > >>>>>>>> Liming,
> > > > > > >>>>>>>>
> > > > > > >>>>>>>> Sorry I missed this mail. Thanks for the
> > > info! I was
> > > > > > >>>>>>>> doing some
> > > > > > >research
> > > > > > >>>>> into this too and now I think I finally
> > > understand. I think
> > > > > > >>>>> the name for
> > > > > > >the tool
> > > > > > >>>>> chain really confused me and we should think
> > > about changing
> > > > > > >>>>> the
> > > > > > name.
> > > > > > >>>>>>>>
> > > > > > >>>>>>>> From what I understand CLANG9 means
> > > produce PE/COFF
> > > > > directly
> > > > > > >and
> > > > > > >>>>> used the PDB debugging format. I see from
> > > the llvm site that
> > > > > > >>>>> the linker
> > > > > > >can
> > > > > > >>>>> produce PDB directly as you mention. This
> > > all makes sense to
> > > > > > >>>>> me now as
> > > > > > >LLVM
> > > > > > >>>>> tries to make it easy to be a drop in
> > > replacement for VC++ or GCC.
> > > > > > >>>>> So
> > > > > > >this tool
> > > > > > >>>>> chain is designed to be able to cross build
> > > a "Windows App"
> > > > > > >>>>> on a Linux or macOS. It also looks like the
> > > llvm debugger,
> > > > > > >>>>> lldb, is lagging in its support
> > > > > > >for
> > > > > > >>>>> PDB based debugging.
> > > > > > >>>>>>>>
> > > > > > >>>>>>>>
> > > > > > >>>>>>>> Anyway I think Leif and I agree the
> > > toolchain name is
> > > > > > >>>>>>>> very
> > > > > > confusing.
> > > > > > >I'd
> > > > > > >>>>> rather see it called CLANG9_WIN or
> > > CLANG_PDB.
> > > > > > >>>>>>>>
> > > > > > >>>>>>>>
> > > > > > >>>>>>>> On Oct 18, 2019, at 7:27 AM, Liming Gao
> > > > > > >>>>>
> > > <liming.gao@intel.com<mailto:liming.gao@intel.com>>
> > > wrote:
> > > > > > >>>>>>>>
> > > > > > >>>>>>>> Andrew:
> > > > > > >>>>>>>> Here is the cover letter on CLANG9
> > > introduction.
> > > > > > >>>>> https://edk2.groups.io/g/devel/message/49157
> > > > > > >>>>>>>>
> > > > > > >>>>>>>> 1)      Yes. CLANG9 tool chain is added
> > > to directly generate
> > > > > PE/COFF
> > > > > > >image
> > > > > > >>>>> (EFI image).
> > > > > > >>>>>>>> This tool chain uses LLVM clang C
> > > compiler and lld
> > > > > > >>>>>>>> linker, generates
> > > > > > >>>>> PE/COFF
> > > > > > >>>>>>>> image and PDB compatible debug symbol
> > > format. Now, it
> > > > > > supports
> > > > > > >>>>> IA32/X64 Archs.
> > > > > > >>>>>>>> LLVM clang C compiler and lld linker are
> > > the standalone
> > > > > > >>>>>>>> tool
> > > > set.
> > > > > > >They
> > > > > > >>>>> don’t depend other lib to generate PE/COFF
> > > image.
> > > > > > >>>>>>>>
> > > > > > >>>>>>>> 2)      Yes. CLANG9 is the cross OS tool
> > > chain. It can work on
> > > > > > >>>>> Windows/Linux/Mac host OS.
> > > > > > >>>>>>>> LLVM LLD linker uses Windows style
> > > arguments. I verify
> > > > > > >>>>>>>> CLANG9 for
> > > > > > >>>>> Ovmf3264 in Windows/Linux host OS.
> > > > > > >>>>>>>>
> > > > > > >>>>>>>>
> > > > > > >>>>>>>> On Linux can you source level debug Ovmf?
> > > > > > >>>>>>>>
> > > > > > >>>>>>>>
> > > > > > >>>>>>>> 3)     This patch enables WinHost in
> > > Windows. It doesn’t enable
> > > > > > >UnixHost.
> > > > > > >>>>> Now, EmulatorPkg with CLANG9 only works on
> > > Windows Host.
> > > > > > >>>>>>>> This patch can make other modules pass
> > > build in
> > > > > > >Windows/Linux/Mac
> > > > > > >>>>> only if LLVM9 tool set is installed.
> > > > > > >>>>>>>> But, the generated image may not work on
> > > Linux/Mac. I
> > > > > > >>>>>>>> agree
> > > > > > >below
> > > > > > >>>>> linker flags are specific to windows host.
> > > > > > >>>>>>>> So, I suggest to add the conditional
> > > statement of
> > > > > > >$(WIN_HOST_BUILD)
> > > > > > >>>>> == TRUE for them.
> > > > > > >>>>>>>>
> > > > > > >>>>>>>>
> > > > > > >>>>>>>> For the EmulatorPkg the Host is a native
> > > App for that OS
> > > > > > >>>>>>>> you build on,
> > > > > > >but
> > > > > > >>>>> it seems like CLANG9 is targeted to build
> > > Windows Apps. I'm
> > > > > > >>>>> not sure
> > > > > > >but you
> > > > > > >>>>> might be able to override all the linker
> > > commands to build a
> > > > > > >>>>> native app,
> > > > > > >or just
> > > > > > >>>>> use the system linker for the Host?
> > > > > > >>>>>>>>
> > > > > > >>>>>>>> I'm not sure how well debugging will work
> > > mixing PDB and
> > > > > > >>>>>>>> DWARF
> > > > > > >>>>> symbol formats?
> > > > > > >>>>>>>>
> > > > > > >>>>>>>> Thanks,
> > > > > > >>>>>>>>
> > > > > > >>>>>>>> Andrew Fish
> > > > > > >>>>>>>>
> > > > > > >>>>>>>>
> > > > > > >>>>>>>>
> > > > > > >>>>>>>> !if $(WIN_HOST_BUILD) == TRUE
> > > > > > >>>>>>>> GCC:*_CLANG9_*_DLINK_FLAGS     =
> > > /ALIGN:4096
> > > > > > /FILEALIGN:4096
> > > > > > >>>>> /SUBSYSTEM:CONSOLE
> > > > > > >>>>>>>> GCC:DEBUG_CLANG9_*_DLINK_FLAGS =
> > > > > > >>>>>
> > > /EXPORT:InitializeDriver=$(IMAGE_ENTRY_POINT)
> > > /BASE:0x10000
> > > > > > >>>>>>>> GCC:NOOPT_CLANG9_*_DLINK_FLAGS =
> > > > > > >>>>>
> > > /EXPORT:InitializeDriver=$(IMAGE_ENTRY_POINT)
> > > /BASE:0x10000
> > > > > > >>>>>>>> !endif
> > > > > > >>>>>>>>
> > > > > > >>>>>>>> Thanks
> > > > > > >>>>>>>> Liming
> > > > > > >>>>>>>> From:
> > > afish@apple.com<mailto:afish@apple.com>
> > > > > > >>>>> <afish@apple.com<mailto:afish@apple.com>>
> > > > > > >>>>>>>> Sent: Friday, October 18, 2019 1:15 AM
> > > > > > >>>>>>>> To: Ni, Ray
> > > <ray.ni@intel.com<mailto:ray.ni@intel.com>>
> > > > > > >>>>>>>> Cc: Gao, Liming
> > > > > >
> > > ><liming.gao@intel.com<mailto:liming.gao@intel.com>>;
> > > > > > >>>>>
> > > devel@edk2.groups.io<mailto:devel@edk2.groups.io>;
> > > Justen,
> > > > > > Jordan
> > > > > > >>>>> L
> > > > > > >>>>>
> > > <jordan.l.justen@intel.com<mailto:jordan.l.justen@intel.
> > > com>
> > > > > > >>>>> >
> > > > > > >>>>>>>> Subject: Re: [Patch v3 10/11]
> > > EmulatorPkg: Enable CLANG9
> > > > > > >>>>>>>> tool chain
> > > > > > >>>>>>>>
> > > > > > >>>>>>>> Ray,
> > > > > > >>>>>>>>
> > > > > > >>>>>>>> Sorry I'm coming a little late to this
> > > and I'm confused.
> > > > > > >>>>>>>> I have some
> > > > > > >>>>> questions?
> > > > > > >>>>>>>> 1) Does CLANG9 imply CLANGPE?
> > > > > > >>>>>>>> 2) Does CLANGPE work on Linux and macOS?
> > > Can you pass the
> > > > > > >Windows
> > > > > > >>>>> style arguments to CLANGPE linker on Linux
> > > and macOS?
> > > > > > >>>>>>>> 3) For the EmulatorPkg don't you have the
> > > extra
> > > > > > >>>>>>>> requirement that
> > > > > > >>>>> compiler needs a standard C lib (or platform
> > > specific libs)
> > > > > > >>>>> to
> > > > function?
> > > > > > >>>>>>>>      a) GCC:*_CLANG9_*_DLINK_FLAGS in
> > > EmulatorPkg.dsc
> > > > > seems
> > > > > > to
> > > > > > >>>>> imply the Linux and macOS systems will have
> > > a Windows SKD
> > > > > > >>>>> dir and a
> > > > > > >lot of
> > > > > > >>>>> Windows DLLs? Does all that come when you
> > > install CLANG9
> > > > > > >>>>> when
> > > > > > you
> > > > > > >install
> > > > > > >>>>> it on Linux or macOS?
> > > > > > >>>>>>>>
> > > > > > >>>>>>>>
> > > > > > >>>>>>>>
> > > > > > >>>>>>>> So I guess I'm asking is the linker
> > > really the same for
> > > > > > >>>>>>>> CLANG9 on all
> > > > > > >>>>> systems? I guess the answer could be yes,
> > > but it seems the C
> > > > > > >>>>> lib for the
> > > > > > >Host
> > > > > > >>>>> App is still an App for that OS and is OS
> > > dependent?
> > > > > > >>>>>>>>
> > > > > > >>>>>>>>
> > > > > > >>>>>>>>
> > > > > > >>>>>>>> Sorry if I'm missing something
> > > fundamental and this is a
> > > > > > >>>>>>>> dumb
> > > > > > >question.
> > > > > > >>>>>>>>
> > > > > > >>>>>>>>
> > > > > > >>>>>>>>
> > > > > > >>>>>>>> Thanks,
> > > > > > >>>>>>>>
> > > > > > >>>>>>>>
> > > > > > >>>>>>>>
> > > > > > >>>>>>>> Andrew Fish
> > > > > > >>>>>>>>
> > > > > > >>>>>>>>
> > > > > > >>>>>>>>
> > > > > > >>>>>>>>
> > > > > > >>>>>>>>
> > > > > > >>>>>>>>
> > > > > > >>>>>>>>
> > > > > > >>>>>>>> On Oct 17, 2019, at 12:27 AM, Ni, Ray
> > > > > > >>>>> <ray.ni@intel.com<mailto:ray.ni@intel.com>>
> > > wrote:
> > > > > > >>>>>>>>
> > > > > > >>>>>>>> Liming,
> > > > > > >>>>>>>> Emulator is using a generic SEC module.
> > > The host specific
> > > > > > >>>>>>>> module is
> > > > > > >called
> > > > > > >>>>> "Host".
> > > > > > >>>>>>>> So I prefer to change the macro to
> > > "WIN_HOST_BUILD", with
> > > > > > >>>>>>>> this
> > > > > > >change,
> > > > > > >>>>> Reviewed-by: Ray Ni
> > > > > > >>>>> <ray.ni@intel.com<mailto:ray.ni@intel.com>>
> > > > > > >>>>>>>>
> > > > > > >>>>>>>>
> > > > > > >>>>>>>>
> > > > > > >>>>>>>> -----Original Message-----
> > > > > > >>>>>>>> From: Gao, Liming
> > > > > >
> > > ><liming.gao@intel.com<mailto:liming.gao@intel.com>>
> > > > > > >>>>>>>> Sent: Thursday, October 17, 2019 2:56 PM
> > > > > > >>>>>>>> To:
> > > devel@edk2.groups.io<mailto:devel@edk2.groups.io>
> > > > > > >>>>>>>> Cc: Justen, Jordan L
> > > > > > >>>>>
> > > <jordan.l.justen@intel.com<mailto:jordan.l.justen@intel.
> > > com>
> > > > > > >>>>> >;
> > > > > > >Andrew
> > > > > > >>>>> Fish
> > > > > > >>>>>>>>
> > > <afish@apple.com<mailto:afish@apple.com>>; Ni, Ray
> > > > > > >>>>> <ray.ni@intel.com<mailto:ray.ni@intel.com>>
> > > > > > >>>>>>>> Subject: [Patch v3 10/11] EmulatorPkg:
> > > Enable CLANG9 tool
> > > > > > >>>>>>>> chain
> > > > > > >>>>>>>>
> > > > > > >>>>>>>> BZ:
> > > https://bugzilla.tianocore.org/show_bug.cgi?id=1603
> > > > > > >>>>>>>> 1. Add WIN_SEC_BUILD macro check for
> > > CLANG9 tool chain
> > > > > > >>>>>>>> build
> > > > > > >>>>>>>> -
> > > > > > p
> > > > > > >>>>>>>> EmulatorPkg\EmulatorPkg.dsc -a IA32 -
> > > > DWIN_SEC_BUILD=TRUE
> > > > > -t
> > > > > > >>>>> CLANG9
> > > > > > >>>>>>>> build -p EmulatorPkg\EmulatorPkg.dsc -a
> > > X64 -
> > > > > > >DWIN_SEC_BUILD=TRUE -t
> > > > > > >>>>>>>> CLANG9 2. Append CLANG CC and LINK flags
> > > to generate
> > > > > windows
> > > > > > >HOST.
> > > > > > >>>>>>>> 3. Fix WinHost issue to call
> > > GetProcessAffinityMask() API.
> > > > > > >>>>>>>> The input parameter should be UINTN
> > > pointer instead of
> > > > > UINT32
> > > > > > >pointer.
> > > > > > >>>>>>>>
> > > > > > >>>>>>>> Cc: Jordan Justen
> > > > > > >>>>>
> > > <jordan.l.justen@intel.com<mailto:jordan.l.justen@intel.
> > > com>
> > > > > > >>>>> >
> > > > > > >>>>>>>> Cc: Andrew Fish
> > > <afish@apple.com<mailto:afish@apple.com>>
> > > > > > >>>>>>>> Cc: Ray Ni
> > > <ray.ni@intel.com<mailto:ray.ni@intel.com>>
> > > > > > >>>>>>>> Signed-off-by: Liming Gao
> > > > > > >>>>>
> > > <liming.gao@intel.com<mailto:liming.gao@intel.com>>
> > > > > > >>>>>>>> ---
> > > > > > >>>>>>>> EmulatorPkg/Win/Host/WinHost.c   | 6 +++-
> > > --
> > > > > > >>>>>>>> EmulatorPkg/EmulatorPkg.dsc      | 7
> > > ++++++-
> > > > > > >>>>>>>> EmulatorPkg/Win/Host/WinHost.inf | 6
> > > ++++++
> > > > > > >>>>>>>> 3 files changed, 15 insertions(+), 4
> > > deletions(-)
> > > > > > >>>>>>>>
> > > > > > >>>>>>>> diff --git
> > > a/EmulatorPkg/Win/Host/WinHost.c
> > > > > > >>>>>>>> b/EmulatorPkg/Win/Host/WinHost.c index
> > > > > > 9aba3c8959..e40ce32548
> > > > > > >>>>> 100644
> > > > > > >>>>>>>> --- a/EmulatorPkg/Win/Host/WinHost.c
> > > > > > >>>>>>>> +++ b/EmulatorPkg/Win/Host/WinHost.c
> > > > > > >>>>>>>> @@ -356,7 +356,7 @@ Returns:
> > > > > > >>>>>>>> INTN
> > > > > > >>>>>>>> EFIAPI
> > > > > > >>>>>>>> main (
> > > > > > >>>>>>>> -  IN  INTN  Argc,
> > > > > > >>>>>>>> +  IN  INT  Argc,
> > > > > > >>>>>>>> IN  CHAR8 **Argv,
> > > > > > >>>>>>>> IN  CHAR8 **Envp
> > > > > > >>>>>>>> )
> > > > > > >>>>>>>> @@ -391,8 +391,8 @@ Returns:
> > > > > > >>>>>>>> VOID                  *SecFile;
> > > > > > >>>>>>>> CHAR16                *MemorySizeStr;
> > > > > > >>>>>>>> CHAR16
> > > *FirmwareVolumesStr;
> > > > > > >>>>>>>> -  UINT32
> > > ProcessAffinityMask;
> > > > > > >>>>>>>> -  UINT32
> > > SystemAffinityMask;
> > > > > > >>>>>>>> +  UINTN
> > > ProcessAffinityMask;
> > > > > > >>>>>>>> +  UINTN
> > > SystemAffinityMask;
> > > > > > >>>>>>>> INT32                 LowBit;
> > > > > > >>>>>>>>
> > > > > > >>>>>>>> //
> > > > > > >>>>>>>> diff --git a/EmulatorPkg/EmulatorPkg.dsc
> > > > > > >>>>> b/EmulatorPkg/EmulatorPkg.dsc
> > > > > > >>>>>>>> index 20f1187713..72532f5daf 100644
> > > > > > >>>>>>>> --- a/EmulatorPkg/EmulatorPkg.dsc
> > > > > > >>>>>>>> +++ b/EmulatorPkg/EmulatorPkg.dsc
> > > > > > >>>>>>>> @@ -237,9 +237,10 @@
> > > > > > >>>>>>>>
> > > > > > >>>>>>>> [Components]
> > > > > > >>>>>>>> !if "IA32" in $(ARCH) || "X64" in $(ARCH)
> > > > > > >>>>>>>> -  !if "MSFT" in $(FAMILY)
> > > > > > >>>>>>>> +  !if "MSFT" in $(FAMILY) ||
> > > $(WIN_SEC_BUILD) == TRUE
> > > > > > >>>>>>>>   ##
> > > > > > >>>>>>>>   #  Emulator, OS WIN application
> > > > > > >>>>>>>> +    #  CLANG9 is cross OS tool chain. It
> > > depends on
> > > > > > >>>>>>>> + WIN_SEC_BUILD
> > > > > > >>>>> macro.
> > > > > > >>>>>>>>   ##
> > > > > > >>>>>>>>   EmulatorPkg/Win/Host/WinHost.inf !else
> > > @@ -419,7
> > > > > > >>>>>>>> +420,11
> > > > > > @@
> > > > > > >>>>>>>>
> > > > > > >>>>>>>> MSFT:DEBUG_*_*_CC_FLAGS = /Od /Oy-
> > > > > > MSFT:NOOPT_*_*_CC_FLAGS =
> > > > > > >>>>>>>> /Od /Oy-
> > > > > > >>>>>>>> +  GCC:DEBUG_CLANG9_*_CC_FLAGS =-O0 -Wno-
> > > unused-
> > > > > > >command-
> > > > > > >>>>> line-
> > > > > > >>>>>>>> argument
> > > > > > >>>>>>>> + -Wno-incompatible-pointer-types -Wno-
> > > enum-conversion
> > > > > > >>>>>>>> + -Wno-incompatible-pointer-types -Wno-
> > > sometimes-
> > > > > > uninitialized
> > > > > > >>>>>>>> + -Wno-constant-conversion -Wno-main-
> > > return-type
> > > > > > >>>>>>>>
> > > > > > >>>>>>>> MSFT:*_*_*_DLINK_FLAGS     = /ALIGN:4096
> > > /FILEALIGN:4096
> > > > > > >>>>>>>> /SUBSYSTEM:CONSOLE
> > > > > > >>>>>>>> MSFT:DEBUG_*_*_DLINK_FLAGS =
> > > > > > >>>>>>>>
> > > /EXPORT:InitializeDriver=$(IMAGE_ENTRY_POINT)
> > > > > /BASE:0x10000
> > > > > > >>>>>>>> MSFT:NOOPT_*_*_DLINK_FLAGS =
> > > > > > >>>>>>>>
> > > /EXPORT:InitializeDriver=$(IMAGE_ENTRY_POINT)
> > > > > /BASE:0x10000
> > > > > > >>>>>>>> +  GCC:*_CLANG9_*_DLINK_FLAGS     =
> > > /ALIGN:4096
> > > > > > >/FILEALIGN:4096
> > > > > > >>>>>>>> /SUBSYSTEM:CONSOLE
> > > > > > >>>>>>>> +  GCC:DEBUG_CLANG9_*_DLINK_FLAGS =
> > > > > > >>>>>>>> +
> > > /EXPORT:InitializeDriver=$(IMAGE_ENTRY_POINT)
> > > > > > /BASE:0x10000
> > > > > > >>>>>>>> + GCC:NOOPT_CLANG9_*_DLINK_FLAGS =
> > > > > > >>>>>>>> +
> > > /EXPORT:InitializeDriver=$(IMAGE_ENTRY_POINT)
> > > > > > /BASE:0x10000
> > > > > > >>>>>>>> diff --git
> > > a/EmulatorPkg/Win/Host/WinHost.inf
> > > > > > >>>>>>>> b/EmulatorPkg/Win/Host/WinHost.inf
> > > > > > >>>>>>>> index 631d5a6470..1adca10d79 100644
> > > > > > >>>>>>>> --- a/EmulatorPkg/Win/Host/WinHost.inf
> > > > > > >>>>>>>> +++ b/EmulatorPkg/Win/Host/WinHost.inf
> > > > > > >>>>>>>> @@ -95,3 +95,9 @@
> > > > > > >>>>>>>> MSFT:*_VS2017_X64_DLINK_FLAGS      =
> > > > > > >>>>>>>> /LIBPATH:"%VCToolsInstallDir%lib\x64"
> > > > > > >>>>>>>>
> > > > >
> > > /LIBPATH:"%UniversalCRTSdkDir%lib\%UCRTVersion%\ucrt\x64
> > > "
> > > > > > >>>>>>>>
> > > > > >
> > > >/LIBPATH:"%WindowsSdkDir%lib\%WindowsSDKLibVersion%\um\
> > > x64"
> > > > > > >>>>>>>> /NOLOGO /SUBSYSTEM:CONSOLE /NODEFAULTLIB
> > > > > /IGNORE:4086
> > > > > > >/MAP
> > > > > > >>>>>>>> /OPT:REF /DEBUG /MACHINE:AMD64 /LTCG
> > > Kernel32.lib
> > > > > > >MSVCRTD.lib
> > > > > > >>>>>>>> vcruntimed.lib ucrtd.lib Gdi32.lib
> > > User32.lib Winmm.lib
> > > > > > Advapi32.lib
> > > > > > >>>>>>>> MSFT:*_*_X64_ASM_FLAGS            ==
> > > /nologo /W3 /WX /c /Cx
> > > > > /Zd
> > > > > > >/W0
> > > > > > >>>>> /Zi
> > > > > > >>>>>>>> MSFT:*_*_X64_ASMLINK_FLAGS        ==
> > > /link /nologo
> > > > > > >>>>>>>> +
> > > > > > >>>>>>>> +  GCC:*_CLANG9_X64_DLINK_FLAGS ==
> > > > > > >>>>>>>> /out:"$(BIN_DIR)\$(BASE_NAME).exe"
> > > /base:0x10000000
> > > > > > >>>>>>>> /pdb:"$(BIN_DIR)\$(BASE_NAME).pdb"
> > > > > > >>>>>>>>
> > > > >
> > > /LIBPATH:"%UniversalCRTSdkDir%lib\%UCRTVersion%\ucrt\x64
> > > "
> > > > > > >>>>>>>>
> > > > > >
> > > >/LIBPATH:"%WindowsSdkDir%lib\%WindowsSDKLibVersion%\um\
> > > x64"
> > > > > > >>>>>>>> /LIBPATH:"%VCToolsInstallDir%lib\x64"
> > > /NOLOGO
> > > > > > >>>>> /SUBSYSTEM:CONSOLE
> > > > > > >>>>>>>> /NODEFAULTLIB /IGNORE:4086  /OPT:REF
> > > /DEBUG
> > > > > > >/MACHINE:AMD64
> > > > > > >>>>>>>> Kernel32.lib MSVCRTD.lib vcruntimed.lib
> > > ucrtd.lib
> > > > > > >>>>>>>> Gdi32.lib
> > > > > > >User32.lib
> > > > > > >>>>>>>> Winmm.lib Advapi32.lib /lldmap
> > > > > > >>>>>>>>
> > > /EXPORT:InitializeDriver=_ModuleEntryPoint
> > > > > > >>>>>>>> +  GCC:*_CLANG9_X64_CC_FLAGS == -m64 -g -
> > > fshort-wchar
> > > > > > >>>>>>>> + -fno-strict-aliasing -Wall -c -include
> > > AutoGen.h -D
> > > > > > >>>>>>>> + _CRT_SECURE_NO_WARNINGS -Wnonportable-
> > > include-
> > > > path
> > > > > -D
> > > > > > >>>>> UNICODE
> > > > > > >>>>>>>> -D
> > > > > > >>>>>>>> + _CRT_SECURE_NO_DEPRECATE
> > > > > > >>>>>>>> +
> > > > > > >>>>>>>> +  GCC:*_CLANG9_IA32_DLINK_FLAGS ==
> > > > > > >>>>>>>> /out:"$(BIN_DIR)\$(BASE_NAME).exe"
> > > /base:0x10000000
> > > > > > >>>>>>>> /pdb:"$(BIN_DIR)\$(BASE_NAME).pdb"
> > > > > > >>>>>>>>
> > > > >
> > > /LIBPATH:"%UniversalCRTSdkDir%lib\%UCRTVersion%\ucrt\x86
> > > "
> > > > > > >>>>>>>>
> > > > > >
> > > >/LIBPATH:"%WindowsSdkDir%lib\%WindowsSDKLibVersion%\um\
> > > x86"
> > > > > > >>>>>>>> /LIBPATH:"%VCToolsInstallDir%ib\x86"
> > > /NOLOGO
> > > > > > >>>>> /SUBSYSTEM:CONSOLE
> > > > > > >>>>>>>> /NODEFAULTLIB /IGNORE:4086  /OPT:REF
> > > /DEBUG
> > > > > /MACHINE:I386
> > > > > > >>>>>>>> Kernel32.lib MSVCRTD.lib vcruntimed.lib
> > > ucrtd.lib
> > > > > > >>>>>>>> Gdi32.lib
> > > > > > >User32.lib
> > > > > > >>>>>>>> Winmm.lib Advapi32.lib /lldmap
> > > > > > >>>>>>>>
> > > /EXPORT:InitializeDriver=_ModuleEntryPoint
> > > > > > >>>>>>>> +  GCC:*_CLANG9_IA32_CC_FLAGS == -m32 -g
> > > -fshort-wchar
> > > > > > >>>>>>>> + -fno-strict-aliasing -Wall -c -include
> > > AutoGen.h -D
> > > > > > >>>>>>>> + _CRT_SECURE_NO_WARNINGS -Wnonportable-
> > > include-
> > > > path
> > > > > -D
> > > > > > >>>>> UNICODE
> > > > > > >>>>>>>> -D
> > > > > > >>>>>>>> + _CRT_SECURE_NO_DEPRECATE
> > > > > > >>>>>>>> --
> > > > > > >>>>>>>> 2.13.0.windows.1
> > > > > > >>>>>>>>
> > > > > > >>>>>>>>
> > > > > > >>>>>>>>
> > > > > > >>>>>>>>
> > > > > > >>>>>>>>
> > > > > > >>>>>>>>
> > > > > > >>>>>>>>
> > > > > > >>>>>>>
> > > > > > >>>>>>>
> > > > > > >>>>>>>
> > > > > > >>>>>>
> > > > > > >>>>>
> > > > > > >>>>>
> > > > > > >>>
> > > > > > >
> > > > > > >
> > > > >
> > > > >
> > > > >
> > > >
> > > >
> > > >
> > >
> > >
> > >