From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from ma1-aaemail-dr-lapp01.apple.com (ma1-aaemail-dr-lapp01.apple.com [17.171.2.60]) by mx.groups.io with SMTP id smtpd.web12.113.1572283132826930068 for ; Mon, 28 Oct 2019 10:18:53 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=@apple.com header.s=20180706 header.b=n/v4e50S; spf=pass (domain: apple.com, ip: 17.171.2.60, mailfrom: afish@apple.com) Received: from pps.filterd (ma1-aaemail-dr-lapp01.apple.com [127.0.0.1]) by ma1-aaemail-dr-lapp01.apple.com (8.16.0.27/8.16.0.27) with SMTP id x9SHHK2o038102; Mon, 28 Oct 2019 10:18:51 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=apple.com; h=sender : from : message-id : content-type : mime-version : subject : date : in-reply-to : cc : to : references; s=20180706; bh=rdP7O4YHq8qQLzCcSW/pGGiNAr6YVrTLIeNWzRGXK3U=; b=n/v4e50SRqYUFsI6DQaaamsFuQ0KmMwXYUi/4kbMszKlQ8THckMl+U2q4DYbRPV0Mfey gqlOx1W/Jiap54sWBpq7OePQamk+oqs3fGX2TumP1nI97m+XnJC4V0J5ro8k5fzzrGHe XRYODEHICtAmKyGnDNl52Ip1UIzGoJrHuz5bj+emZoh6vYG1uUNn05a6jvrhUFwXyY3X qFYyGtebOQguWgJAqL44j78aGR+1nEHYp+G+xrZ3d81+QvfC0AGwqukRw+ptbmvN3wrA r/7IR5P4wjitOUuLk2LCibCeTuBcvdwluQz9F6n1Il+wDFr2OintYZ1ogFMSLldF9jFn BA== Received: from mr2-mtap-s01.rno.apple.com (mr2-mtap-s01.rno.apple.com [17.179.226.133]) by ma1-aaemail-dr-lapp01.apple.com with ESMTP id 2vvmt44s3p-3 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NO); Mon, 28 Oct 2019 10:18:51 -0700 Received: from nwk-mmpp-sz09.apple.com (nwk-mmpp-sz09.apple.com [17.128.115.80]) by mr2-mtap-s01.rno.apple.com (Oracle Communications Messaging Server 8.0.2.4.20190507 64bit (built May 7 2019)) with ESMTPS id <0Q0300KHLHFEW9D0@mr2-mtap-s01.rno.apple.com>; Mon, 28 Oct 2019 10:18:50 -0700 (PDT) Received: from process_milters-daemon.nwk-mmpp-sz09.apple.com by nwk-mmpp-sz09.apple.com (Oracle Communications Messaging Server 8.0.2.4.20190507 64bit (built May 7 2019)) id <0Q0300I00HDF2300@nwk-mmpp-sz09.apple.com>; Mon, 28 Oct 2019 10:18:50 -0700 (PDT) X-Va-A: X-Va-T-CD: 668e594459b786b05feabaa5e9ab8d36 X-Va-E-CD: 7c049b726eb4bd94d2bc056bc12a4189 X-Va-R-CD: bdc755f033a2e724f8c3b8bf8392ed19 X-Va-CD: 0 X-Va-ID: bbdb3b7c-f631-4fe2-b003-e37b679a9d88 X-V-A: X-V-T-CD: 668e594459b786b05feabaa5e9ab8d36 X-V-E-CD: 7c049b726eb4bd94d2bc056bc12a4189 X-V-R-CD: bdc755f033a2e724f8c3b8bf8392ed19 X-V-CD: 0 X-V-ID: b0711f8b-d409-471e-ae37-a27653135839 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:,, definitions=2019-10-28_07:,, signatures=0 Received: from [17.235.38.139] (unknown [17.235.38.139]) by nwk-mmpp-sz09.apple.com (Oracle Communications Messaging Server 8.0.2.4.20190507 64bit (built May 7 2019)) with ESMTPSA id <0Q0300CVXHFA4020@nwk-mmpp-sz09.apple.com>; Mon, 28 Oct 2019 10:18:47 -0700 (PDT) Sender: afish@apple.com From: "Andrew Fish" Message-id: MIME-version: 1.0 (Mac OS X Mail 13.0 \(3594.4.17\)) Subject: Re: [edk2-devel] [Patch 0/1] BaseTools: Using compiler to generate source code dependency files. Date: Mon, 28 Oct 2019 10:18:46 -0700 In-reply-to: <74D8A39837DF1E4DA445A8C0B3885C503F828017@shsmsx102.ccr.corp.intel.com> Cc: "Feng, Bob C" , Ryszard Knop To: devel@edk2.groups.io, "Yao, Jiewen" References: <20191028104702.30620-1-bob.c.feng@intel.com> <2d66136b-e92e-e7b2-ecd8-3d1cc59bbdec@linux.intel.com> <08650203BA1BD64D8AD9B6D5D74A85D161558C7E@SHSMSX104.ccr.corp.intel.com> <74D8A39837DF1E4DA445A8C0B3885C503F828017@shsmsx102.ccr.corp.intel.com> X-Mailer: Apple Mail (2.3594.4.17) X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:,, definitions=2019-10-28_07:,, signatures=0 Content-type: multipart/alternative; boundary="Apple-Mail=_B2FF70FD-FF62-4CA2-8AF6-82E469FA9D85" --Apple-Mail=_B2FF70FD-FF62-4CA2-8AF6-82E469FA9D85 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=utf-8 The .d is the default file name extension used by GCC for dependency files.= Given the dependency files are in the build output and the makefiles refer= ence them explicitly I'm not sure there is going to be lots of confusion.= =20 But I think it is likely the default dependency file name is mostly just a= convenience in constructing the makefiles, since if you use a custom name = each invocation has to do string operations to create a file with a custom = extension. Given our makefiles are generated by a tool this is probably not= really an issue for us.=20 Does Visual Studio have the concept of a dependency file? If yes what suff= ix does that use, as I guess we could use that? I guess the dependency hand= ling could just be some XML data in some larger Visual Studio meta data.... GCC/clang flags around dependencies.=20 -MD <>-MD is equivalent to -M -MF file, except that -E is not implied. The dr= iver determines file based on whether an -o option is given. If it is, the = driver uses its argument but with a suffix of .d, otherwise it takes the na= me of the input file, removes any directory components and suffix, and appl= ies a .d suffix. If -MD is used in conjunction with -E, any -o switch is understood to spec= ify the dependency output file (see -MF ), but if used without -E, each -o is und= erstood to specify a target object file. Since -E is not implied, -MD can be used to generate a dependency output f= ile as a side effect of the compilation process -MF file <>When used with -M or -MM, specifies a file to write the dependencies to= . If no -MF switch is given the preprocessor sends the rules to the same pl= ace it would send preprocessed output. When used with the driver options -MD or -MMD, -MF overrides the default d= ependency output file. If file is -, then the dependencies are written to stdout. Thanks, Andrew Fish > On Oct 28, 2019, at 8:03 AM, Yao, Jiewen wrote: >=20 > I think we need avoid confusing for future. > I don=E2=80=99t believe .d is good choice, since it is a known conflict. >=20 > Thank you > Yao Jiewen >=20 >=20 >> -----Original Message----- >> From: devel@edk2.groups.io > On Behalf Of Bob Feng >> Sent: Monday, October 28, 2019 10:57 PM >> To: Ryszard Knop >; devel@edk2.groups.io >> Subject: Re: [edk2-devel] [Patch 0/1] BaseTools: Using compiler to gene= rate >> source code dependency files. >>=20 >> Thanks for your comment. I think .d file should be fine since edk2 does= not >> support D language. >>=20 >> Thanks, >> Bob >>=20 >> -----Original Message----- >> From: Ryszard Knop >> Sent: Monday, October 28, 2019 8:24 PM >> To: devel@edk2.groups.io; Feng, Bob C >> Subject: Re: [edk2-devel] [Patch 0/1] BaseTools: Using compiler to gene= rate >> source code dependency files. >>=20 >> Just a quick note: .d files are used by the D language. You might want = to use an >> extension like .deps instead. >>=20 >> On 2019-10-28 11:47, Bob Feng wrote: >>> BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=3D2311 >>>=20 >>> To support incremental build, build tool generates the dependent >>> header file for each of source file. This procedure is done in AutoGen= phase. >>> The build tool goes through all the source file and header file and >>> use regular expression to find out all the dependent files for a >>> source file. This procedure is much time-consuming. And this method >>> can't handle the MACRO in #include, for example #include PATH(xxx.h). >>>=20 >>> This patch is going to use compiler to generate dependent files. This >>> method will be faster and more accurate. >>>=20 >>> The basic idea is: >>> 1. In AutoGen phase, build tool add "!Include deps.txt" into Makefile >>> instead of defining COMMON_DEPS list. >>> 2. During the Make phase, the compilers, Trim and C preprocessor >>> generate dependent files, .d file, for each source file. >>> 3. After Make, The build tool combines the .d files and generate a >>> file deps.txt which list all the included files for a module. >>> 4. Each source file will depends on the Module's includes files. The >>> difference with orignial behavior is that if the user change the >>> source file, build tool will only build that source file in >>> incremental build; while if the user change a module's header file, >>> build tool will build the whole module in incremental build. >>>=20 >>> In this way, the time of AutoGen phase will be reduced much. And since >>> we will use c preprocessor to handle #include, the MACRO will be >>> handled well and the final dependent files will be more accurate. >>>=20 >>> Feng, Bob C (1): >>> BaseTools: Using compiler to generate source code dependency files. >>>=20 >>> BaseTools/Conf/build_rule.template | 89 ++++++----- >>> BaseTools/Conf/tools_def.template | 138 +++++++++--------= - >>> BaseTools/Source/Python/AutoGen/GenMake.py | 73 +++------ >>> .../Source/Python/AutoGen/IncludesAutoGen.py | 99 +++++++++++++ >>> BaseTools/Source/Python/Trim/Trim.py | 113 +++++++++++--- >>> BaseTools/Source/Python/build/build.py | 58 ++++++-- >>> 6 files changed, 378 insertions(+), 192 deletions(-) >>> create mode 100644 >>> BaseTools/Source/Python/AutoGen/IncludesAutoGen.py >>>=20 >>=20 >>=20 >=20 >=20 >=20 --Apple-Mail=_B2FF70FD-FF62-4CA2-8AF6-82E469FA9D85 Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=utf-8
The .d is= the default file name extension used by GCC for dependency files. Given th= e dependency files are in the build output and the makefiles reference them= explicitly I'm not sure there is going to be lots of confusion. 

But I think it is lik= ely the default dependency file name is mostly just a convenience in constr= ucting the makefiles, since if you use a custom name each invocation has to= do string operations to create a file with a custom extension. Given our m= akefiles are generated by a tool this is probably not really an issue for u= s. 

Does Vis= ual Studio have the concept of a dependency file? If yes what suffix does t= hat use, as I guess we could use that? I guess the dependency handling coul= d just be some XML data in some larger Visual Studio meta data....

GCC/clang flags around de= pendencies. 

-MD

-MD is equivalent to -M -MF file, except that -E is no= t implied. The driver determines file based= on whether an -o option is given. If it = is, the driver uses its argument but with a suffix of .d, otherwise it takes the name of the input file, removes any dire= ctory components and suffix, and applies a .d&= nbsp;suffix.

If -MD is u= sed in conjunction with -E, any -o switch is understood to specify the dependency outpu= t file (see -MF), but if used without -E, each -o is understood to specify a tar= get object file.

Since -E&nbs= p;is not implied, -MD can be used to gene= rate a dependency output file as a side effect of the compilation process



-MF file

When used with -M or -MM, specifies a file to write th= e dependencies to. If no -MF switch is gi= ven the preprocessor sends the rules to the same place it would send prepro= cessed output.

When used with the driver options -MD or -MMD, -MF overrides the default dependency output file.=

If file is -, then the dependencies are written to stdout.


Th= anks,

Andrew Fish

On Oct 28, 2019= , at 8:03 AM, Yao, Jiewen <jiewen.yao@intel.com> wrote:

I think we need avoid confusing for fut= ure.
I don=E2=80=99t be= lieve .d is good choice, since it is a known conflict.

Tha= nk you
Yao Jiewen


-----Orig= inal Message-----
From:=  devel@edk2.= groups.io <devel@edk2.groups.io> O= n Behalf Of Bob Feng
Sent: Monday, October 28, 2019 10:57 PM<= br class=3D"">To: Ryszard Knop <ryszard.knop@linux.intel.com>; devel@edk2.groups.io
Subject: Re: [edk2-devel] [P= atch 0/1] BaseTools: Using compiler to generate
source code d= ependency files.

Thanks for your comment. I th= ink .d file should be fine since edk2 does not
support D lang= uage.

Thanks,
Bob
=
-----Original Message-----
From: Ryszard Knop = <ryszard.knop= @linux.intel.com>
Sent: Monday, October 28, 2019 8:24 = PM
To: dev= el@edk2.groups.io; Feng, Bob C <bob.c.feng@intel.com>
Subject: Re: [edk= 2-devel] [Patch 0/1] BaseTools: Using compiler to generate
so= urce code dependency files.

Just a quick note:= .d files are used by the D language. You might want to use an
extension like .deps instead.

On 2019-10-28 = 11:47, Bob Feng wrote:
B= Z: https://bugzilla.tianocore.org/show_bug.cgi?id=3D2311

To support incremental build, build tool generates th= e dependent
header file for each of source file. This procedu= re is done in AutoGen phase.
The build tool goes through all = the source file and header file and
use regular expression to= find out all the dependent files for a
source file. This pro= cedure is much time-consuming. And this method
can't handle t= he MACRO in #include, for example #include PATH(xxx.h).

This patch is going to use compiler to generate dependent files. = This
method will be faster and more accurate.
<= br class=3D"">The basic idea is:
1. In AutoGen phase, build t= ool add "!Include deps.txt" into Makefile
instead of defining= COMMON_DEPS list.
2. During the Make phase, the compilers, T= rim and C preprocessor
generate dependent files, .d file, for= each source file.
3. After Make, The build tool combines the= .d files and generate a
file deps.txt which list all the inc= luded files for a module.
4. Each source file will depends on= the Module's includes files. The
difference with orignial be= havior is that if the user change the
source file, build tool= will only build that source file in
incremental build; while= if the user change a module's header file,
build tool will b= uild the whole module in incremental build.

In= this way, the time of AutoGen phase will be reduced much. And since
we will use c preprocessor to handle #include, the MACRO will behandled well and the final dependent files will be more accurat= e.

Feng, Bob C (1):
  = BaseTools: Using compiler to generate source code dependency files.

 BaseTools/Conf/build_rule.template   =          |  89 ++++++----= -
 BaseTools/Conf/tools_def.template    &= nbsp;        | 138 +++++++++-------= --
 BaseTools/Source/Python/AutoGen/GenMake.py  &nb= sp; |  73 +++------
 .../Source/Python/AutoGen= /IncludesAutoGen.py  |  99 +++++++++++++
 Base= Tools/Source/Python/Trim/Trim.py        =   | 113 +++++++++++---
 BaseTools/Source/Pytho= n/build/build.py        |  58 +++++= +--
 6 files changed, 378 insertions(+), 192 deletions(-= )
 create mode 100644
BaseTools/Source/Pyt= hon/AutoGen/IncludesAutoGen.py





--Apple-Mail=_B2FF70FD-FF62-4CA2-8AF6-82E469FA9D85--