From mboxrd@z Thu Jan 1 00:00:00 1970 Authentication-Results: mx.groups.io; dkim=missing; spf=pass (domain: redhat.com, ip: 209.132.183.28, mailfrom: lersek@redhat.com) Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by groups.io with SMTP; Tue, 23 Jul 2019 10:23:20 -0700 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 72F5B300194A; Tue, 23 Jul 2019 17:23:19 +0000 (UTC) Received: from lacos-laptop-7.usersys.redhat.com (ovpn-117-52.ams2.redhat.com [10.36.117.52]) by smtp.corp.redhat.com (Postfix) with ESMTP id AE8155D9D3; Tue, 23 Jul 2019 17:23:17 +0000 (UTC) Subject: Re: [edk2-devel] [PATCH 0/4] Arm, ArmPlatform, Crypto, Embedded: list internal headers in [Sources] To: Leif Lindholm , "Gao, Liming" Cc: "devel@edk2.groups.io" , "Kinney, Michael D" , Ard Biesheuvel , "Wang, Jian J" , "Ye, Ting" References: <20190719164319.9070-1-lersek@redhat.com> <20190722103755.GA11541@bivouac.eciton.net> <591319a9-eceb-ab39-0ec0-ccd2530b0e58@redhat.com> <20190723090644.GD11541@bivouac.eciton.net> <20190723121940.GH11541@bivouac.eciton.net> <4A89E2EF3DFEDB4C8BFDE51014F606A14E4B11C4@SHSMSX104.ccr.corp.intel.com> <20190723132543.GI11541@bivouac.eciton.net> From: "Laszlo Ersek" Message-ID: Date: Tue, 23 Jul 2019 19:23:16 +0200 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Thunderbird/52.9.1 MIME-Version: 1.0 In-Reply-To: <20190723132543.GI11541@bivouac.eciton.net> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.40]); Tue, 23 Jul 2019 17:23:19 +0000 (UTC) Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 7bit On 07/23/19 15:25, Leif Lindholm wrote: > On Tue, Jul 23, 2019 at 01:02:42PM +0000, Gao, Liming wrote: >>>>> I am just not pleased with the issue >>>>> bringing this to the fore is caused by the new caching feature using a >>>>> different mechanism for tracking header file dependencies than the >>>>> primary build process. >>>> >>>> Ugh... that's a lot of statements compressed into a single sentence. Can >>>> you please break it down for me? (Yes, I remember the mailing list >>>> reference you posted earlier, that discussion was too divergent for me.) >>> >>> The inclusion of .h files in .inf is not necessary for determining >>> build-time dependencies on the Makefile level. >> >> Right. In fact, build tool will parse source file to find their dependent header files, >> and list those header files as the dependency in Makefile. > > If Visual Studio does not have functionality similar to the GCC > family's -M flag, then yeah, I can see why the tool needs to take care > of this itself. Although it would be good if we could find a way to > not fully maintain our own code for this. > >>> Thus, the warnings come out of a different and unrelated level of the >>> build system, related to the recent build cache features. Which means >>> we're checking header file build dependencies through two different >>> mechanisms at two different points of the build. >> >> This is related to build feature --hash. When --hash option is enabled, build will calculate >> the hash value of source files specified in INF file. If hash value is not changed, build tool >> will not parse source files, and not regenerate Makefile again. So, --hash option >> is the incremental way in the build parse phase. If some header files are not >> specified in INF file, it will cause hash value inaccurate. > > Right - so we actually have two levels of dependency scanning in the > build tool itself? One for the .inf file contents, and one for the > source file contents. > >> Then, Makefile may not >> be generated to match the real source code, and cause the incremental build failure. >> So, the missing header files in INF file may bring the incremental build issue when --hash option. >> If you don't enable --hash option, there is no problem even if the missing header files exist. > > Right, but we still see the warning messages without using --hash. > > Thank you very much for the summary - it clarifies the situation greatly. +1 > Would it be feasible to update the --hash functionality to make use of > the include dependencies extracted from the source files? (Clearly, we > know when the source files change, so we would also know when we would > need to re-run the dependency search.) > > If not, I think we should make the explicit listing of .h files > in .inf mandatory, triggering a build failure when not the case. I believe I made the exact same request / suggestion, when Christian initially posted the patches. The counter-argument was (back then anyway) that this would break a plethora of platforms. So the idea was to annoy people with warnings just enough to clean up those INF files, and then turn the warnings into errors. If I remember correctly, at least. Given that the (partly generated) OpenSSL INF files still produce warnings, I assume that this cleanup is likely incomplete still, in other (out of tree) platforms as well. > If it is, then I think we should make it explicitly banned to list .h > files in .inf. (If there is no other dependency, such as doxygen, also > making use of .inf listings of .h files.) This looks a bit too recursive for me: - rely on hashes saved from a previous run to avoid parsing the #include directives, - use extracted #include dependencies to determine what files to hash. The possibility of a stale cache concerns me. (*Incremental* dependency extraction with gcc's -MMD concerns me the same, BTW.) > And this may well be a topic requiring much longer discussion. While > undecided, I would definitely prefer if we could disable the warning > when not actually building with --hash. That makes sense. Thanks Laszlo