From mboxrd@z Thu Jan 1 00:00:00 1970 Authentication-Results: mx.groups.io; dkim=missing; spf=pass (domain: intel.com, ip: 134.134.136.100, mailfrom: liming.gao@intel.com) Received: from mga07.intel.com (mga07.intel.com [134.134.136.100]) by groups.io with SMTP; Wed, 24 Jul 2019 08:17:59 -0700 X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga001.fm.intel.com ([10.253.24.23]) by orsmga105.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 24 Jul 2019 08:17:58 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.64,303,1559545200"; d="scan'208";a="189118234" Received: from fmsmsx104.amr.corp.intel.com ([10.18.124.202]) by fmsmga001.fm.intel.com with ESMTP; 24 Jul 2019 08:17:58 -0700 Received: from shsmsx151.ccr.corp.intel.com (10.239.6.50) by fmsmsx104.amr.corp.intel.com (10.18.124.202) with Microsoft SMTP Server (TLS) id 14.3.439.0; Wed, 24 Jul 2019 08:17:58 -0700 Received: from shsmsx104.ccr.corp.intel.com ([169.254.5.110]) by SHSMSX151.ccr.corp.intel.com ([169.254.3.55]) with mapi id 14.03.0439.000; Wed, 24 Jul 2019 23:17:56 +0800 From: "Liming Gao" To: "devel@edk2.groups.io" , "'leif.lindholm@linaro.org'" CC: Laszlo Ersek , "Kinney, Michael D" , Ard Biesheuvel , "Wang, Jian J" , "Ye, Ting" Subject: Re: [edk2-devel] [PATCH 0/4] Arm, ArmPlatform, Crypto, Embedded: list internal headers in [Sources] Thread-Topic: [edk2-devel] [PATCH 0/4] Arm, ArmPlatform, Crypto, Embedded: list internal headers in [Sources] Thread-Index: AQHVPlEXtkh4Y6imCEizBR/sM57SaqbV8EyAgABz7oCAABT+gIAARWeAgACqiACAAC78AIAABusAgACL/QD//4Z4gIABuiHA Date: Wed, 24 Jul 2019 15:17:56 +0000 Message-ID: <4A89E2EF3DFEDB4C8BFDE51014F606A14E4B2125@SHSMSX104.ccr.corp.intel.com> References: <20190719164319.9070-1-lersek@redhat.com> <20190722103755.GA11541@bivouac.eciton.net> <591319a9-eceb-ab39-0ec0-ccd2530b0e58@redhat.com> <20190723090644.GD11541@bivouac.eciton.net> <20190723121940.GH11541@bivouac.eciton.net> <4A89E2EF3DFEDB4C8BFDE51014F606A14E4B11C4@SHSMSX104.ccr.corp.intel.com> <20190723132543.GI11541@bivouac.eciton.net> In-Reply-To: <20190723132543.GI11541@bivouac.eciton.net> Accept-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ctpclassification: CTP_NT x-titus-metadata-40: eyJDYXRlZ29yeUxhYmVscyI6IiIsIk1ldGFkYXRhIjp7Im5zIjoiaHR0cDpcL1wvd3d3LnRpdHVzLmNvbVwvbnNcL0ludGVsMyIsImlkIjoiZDZiZWNiNzQtM2VmYi00ODRkLWI2MDktMGZjYjAzMTZmOTE5IiwicHJvcHMiOlt7Im4iOiJDVFBDbGFzc2lmaWNhdGlvbiIsInZhbHMiOlt7InZhbHVlIjoiQ1RQX05UIn1dfV19LCJTdWJqZWN0TGFiZWxzIjpbXSwiVE1DVmVyc2lvbiI6IjE3LjEwLjE4MDQuNDkiLCJUcnVzdGVkTGFiZWxIYXNoIjoieUptNlpLdDBsaytsTzRrbUZuSXdzR1F2dk9OVUFuUVJFXC9qWG9NUjZhTjNvNUZic3Vsa2lNVkIzciswNFFyYlMifQ== dlp-product: dlpe-windows dlp-version: 11.0.600.7 dlp-reaction: no-action x-originating-ip: [10.239.127.40] MIME-Version: 1.0 Return-Path: liming.gao@intel.com Content-Language: en-US Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Leif: > -----Original Message----- > From: devel@edk2.groups.io [mailto:devel@edk2.groups.io] On Behalf Of Le= if Lindholm > Sent: Tuesday, July 23, 2019 9:26 PM > To: Gao, Liming > Cc: devel@edk2.groups.io; Laszlo Ersek ; Kinney, Mich= ael D ; Ard Biesheuvel > ; Wang, Jian J ; Ye, T= ing > Subject: Re: [edk2-devel] [PATCH 0/4] Arm, ArmPlatform, Crypto, Embedded= : list internal headers in [Sources] >=20 > On Tue, Jul 23, 2019 at 01:02:42PM +0000, Gao, Liming wrote: > > > > > I am just not pleased with the issue > > > > > bringing this to the fore is caused by the new caching feature u= sing a > > > > > different mechanism for tracking header file dependencies than t= he > > > > > primary build process. > > > > > > > > Ugh... that's a lot of statements compressed into a single sentenc= e. Can > > > > you please break it down for me? (Yes, I remember the mailing list > > > > reference you posted earlier, that discussion was too divergent fo= r me.) > > > > > > The inclusion of .h files in .inf is not necessary for determining > > > build-time dependencies on the Makefile level. > > > > Right. In fact, build tool will parse source file to find their depend= ent header files, > > and list those header files as the dependency in Makefile. >=20 > If Visual Studio does not have functionality similar to the GCC > family's -M flag, then yeah, I can see why the tool needs to take care > of this itself. Although it would be good if we could find a way to > not fully maintain our own code for this. >=20 > > > Thus, the warnings come out of a different and unrelated level of th= e > > > build system, related to the recent build cache features. Which mean= s > > > we're checking header file build dependencies through two different > > > mechanisms at two different points of the build. > > > > This is related to build feature --hash. When --hash option is enabled= , build will calculate > > the hash value of source files specified in INF file. If hash value is= not changed, build tool > > will not parse source files, and not regenerate Makefile again. So, --= hash option > > is the incremental way in the build parse phase. If some header files = are not > > specified in INF file, it will cause hash value inaccurate. >=20 > Right - so we actually have two levels of dependency scanning in the > build tool itself? One for the .inf file contents, and one for the > source file contents. >=20 > > Then, Makefile may not > > be generated to match the real source code, and cause the incremental = build failure. > > So, the missing header files in INF file may bring the incremental bui= ld issue when --hash option. > > If you don't enable --hash option, there is no problem even if the mis= sing header files exist. >=20 > Right, but we still see the warning messages without using --hash. >=20 > Thank you very much for the summary - it clarifies the situation greatly= . >=20 > Would it be feasible to update the --hash functionality to make use of > the include dependencies extracted from the source files? (Clearly, we > know when the source files change, so we would also know when we would > need to re-run the dependency search.) The design is to save the step to extract the dependencies from the source= files.=20 I can further collect the build performance to be taken on the dependencie= s extraction from the source files, and decide whether take this way. Another simple wa= y is=20 to calculate all source files in module directory and make sure there is n= o file missing for --hash option. >=20 > If not, I think we should make the explicit listing of .h files > in .inf mandatory, triggering a build failure when not the case. >=20 > If it is, then I think we should make it explicitly banned to list .h > files in .inf. (If there is no other dependency, such as doxygen, also > making use of .inf listings of .h files.) I know edk2 also has PI Packaging UPT. PI packaging requires all source=20 files are listed in module INF file. Otherwise, some source files will be = missed in the packaging, and can't be rollback. >=20 > And this may well be a topic requiring much longer discussion. While > undecided, I would definitely prefer if we could disable the warning > when not actually building with --hash. There are more than one usage models for this case. I suggest to provide= =20 one option to disable warning.=20 Thanks Liming >=20 > Best Regards, >=20 > Leif >=20 >=20