From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mga18.intel.com (mga18.intel.com [134.134.136.126]) by mx.groups.io with SMTP id smtpd.web11.1967.1575425366169425081 for ; Tue, 03 Dec 2019 18:09:26 -0800 Authentication-Results: mx.groups.io; dkim=missing; spf=pass (domain: intel.com, ip: 134.134.136.126, mailfrom: bob.c.feng@intel.com) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga006.jf.intel.com ([10.7.209.51]) by orsmga106.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 03 Dec 2019 18:09:25 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.69,275,1571727600"; d="scan'208";a="213624212" Received: from fmsmsx108.amr.corp.intel.com ([10.18.124.206]) by orsmga006.jf.intel.com with ESMTP; 03 Dec 2019 18:09:25 -0800 Received: from fmsmsx161.amr.corp.intel.com (10.18.125.9) by FMSMSX108.amr.corp.intel.com (10.18.124.206) with Microsoft SMTP Server (TLS) id 14.3.439.0; Tue, 3 Dec 2019 18:09:25 -0800 Received: from shsmsx101.ccr.corp.intel.com (10.239.4.153) by FMSMSX161.amr.corp.intel.com (10.18.125.9) with Microsoft SMTP Server (TLS) id 14.3.439.0; Tue, 3 Dec 2019 18:09:25 -0800 Received: from shsmsx104.ccr.corp.intel.com ([169.254.5.90]) by SHSMSX101.ccr.corp.intel.com ([169.254.1.19]) with mapi id 14.03.0439.000; Wed, 4 Dec 2019 10:09:24 +0800 From: "Bob Feng" To: "Gao, Liming" , "devel@edk2.groups.io" Subject: Re: [edk2-devel] [Patch 0/4 V4] Enhance Incremental Build Thread-Topic: [edk2-devel] [Patch 0/4 V4] Enhance Incremental Build Thread-Index: AQHVqbcVKeqNiRAhTkOqj0EHNfvJb6eoeULggAC7stA= Date: Wed, 4 Dec 2019 02:09:23 +0000 Message-ID: <08650203BA1BD64D8AD9B6D5D74A85D16157AF4F@SHSMSX104.ccr.corp.intel.com> References: <20191203085247.17784-1-bob.c.feng@intel.com> <4A89E2EF3DFEDB4C8BFDE51014F606A14E55B351@SHSMSX104.ccr.corp.intel.com> In-Reply-To: <4A89E2EF3DFEDB4C8BFDE51014F606A14E55B351@SHSMSX104.ccr.corp.intel.com> Accept-Language: zh-CN, en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [10.239.127.40] MIME-Version: 1.0 Return-Path: bob.c.feng@intel.com Content-Language: en-US Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable 1. No. If user does not update build_rule and tools_def, build will not fai= led. No impact for clean build. Since there will no COMMON_DEPS variable ge= nerated in Makefile, if user change a header file which is in original COMM= ON_DEPS, build tool will not rebuild the corresponding module for increment= al build.=20 2. Yes. for GCC and Clang, the deps file are under the same folder as .obj= file localed. But for MSVC, they are in the module_build_folder, I can upd= ate this in new patch set. 3. If the compile failed, there will no deps file generated. Thanks, Bob -----Original Message----- From: Gao, Liming=20 Sent: Tuesday, December 3, 2019 10:33 PM To: devel@edk2.groups.io; Feng, Bob C Subject: RE: [edk2-devel] [Patch 0/4 V4] Enhance Incremental Build Bob: Thanks for your update. I verify this enhancement. It is great. Now, I h= ave some minor comments. Can you help check them? 1. This change requires tools_def and build_rule be updated together. If t= hey are not updated, what will happen? Can build tool report the error and = let user update them? 2. C compiler will generate the include file list for all other files exce= pt for ASL and Nasm,, and output the include file into the screen. Build to= ol catches them, and creates the file xxx.deps. Can Build tool generate xxx= .deps in the same output directory with the output file? For example, if C = source output file is generated into OUTPUT\X64 directory, its deps file is= also generated into OUTPUT\X64 directory.=20 3. Do you verify the error case? For example, if C source file doesn't pas= s compile, its deps will still be generated or not? Thanks Liming > -----Original Message----- > From: devel@edk2.groups.io On Behalf Of Bob=20 > Feng > Sent: Tuesday, December 3, 2019 4:53 PM > To: devel@edk2.groups.io > Subject: [edk2-devel] [Patch 0/4 V4] Enhance Incremental Build >=20 > V4: Add $(DEPS_FLAGS) for rule [C-Code-File.BASE.AARCH64 ...] > V3: Change CLANG9 to CLANGPDB according to commit 14672c34bd > V2: Fixed a bug in patch 4/4. >=20 > Incremental build reduces the build time by only building the module=20 > that need to update. Edk2 Build system is a Makefile based build=20 > system. The incrememtal build ability is provided by the Make program.= =20 > But Edk2 build tool need to generate correct makefile to have Make=20 > program do incremental build correctly. >=20 > The current solution in build tool to support incremental build is=20 > that build tool find out the include file list for each source file of= =20 > a module, and in module's makefile, build tool add the include file list= as the source file's dependency. > In this way Make program can decide if it need to rebuild a source=20 > code by checking its dependency. This solution has 2 shortcommings,=20 > one is the process of finding include list is slow, the other is this=20 > method can't handle case that a MACRO in #include statement so the relat= ed source file is always built. >=20 > This patch provides another method to support incremental build. That=20 > is to use c preprocessor and trim tool to generate dependency files for = the source file. > This method will save much time in AutoGen phase and handle MACRO in #in= clude correctly. >=20 > For C files: > 1. MSVS. > cl.exe has a build option /showIncludes to display include files= on stdout. Build tool captures > that messages and generate dependency files, .deps files. > 2. CLANG and GCC > -MMD -MF build option are used to generate dependency files by p= reprocessor. Build tool updates the > .deps files. >=20 > For ASL files: > 1. Trim find out all the included files, which are asl specific incl= ude format, and generate .trim.deps file. > 2. ASL PP use c preprocessor to find out all included files with #in= clude format and generate a .deps file > 3. build tool updates the .deps file >=20 > For ASM files (.asm, .s or .nasm): > 1. Trim find out all the included files, which are asm specific incl= ude format, and generate .trim.deps file. > 2. ASM PP use c preprocessor to find out all included files with #in= clude format and generate a deps file > 3. build tool updates the .deps file >=20 > Build tool add "include" instruction for those deps files in the Makefil= e. >=20 > This patch does not support RVCT tool chain for the BZ=20 > https://bugzilla.tianocore.org/show_bug.cgi?id=3D1750 >=20 > Feng, Bob C (4): > BaseTools: Add build option for dependency file generation > BaseTools: Generate dependent files for ASL and ASM files > BaseTools: Update build_rule.txt to generate dependent files. > BaseTools: Enhance Basetool for incremental build >=20 > BaseTools/Conf/build_rule.template | 94 ++++--- > BaseTools/Conf/tools_def.template | 173 ++++++------ > BaseTools/Source/Python/AutoGen/GenMake.py | 83 ++---- > .../Source/Python/AutoGen/IncludesAutoGen.py | 255 ++++++++++++++++++ > .../Source/Python/AutoGen/ModuleAutoGen.py | 23 ++ > BaseTools/Source/Python/Trim/Trim.py | 115 ++++++-- > BaseTools/Source/Python/build/build.py | 63 ++++- > 7 files changed, 588 insertions(+), 218 deletions(-) create mode=20 > 100644 BaseTools/Source/Python/AutoGen/IncludesAutoGen.py >=20 > -- > 2.20.1.windows.1 >=20 >=20 >=20