From mboxrd@z Thu Jan 1 00:00:00 1970 Authentication-Results: mx.groups.io; dkim=missing; spf=pass (domain: intel.com, ip: 134.134.136.65, mailfrom: michael.d.kinney@intel.com) Received: from mga03.intel.com (mga03.intel.com [134.134.136.65]) by groups.io with SMTP; Wed, 28 Aug 2019 11:40:05 -0700 X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga005.jf.intel.com ([10.7.209.41]) by orsmga103.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 28 Aug 2019 11:40:04 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.64,442,1559545200"; d="scan'208";a="356208070" Received: from orsmsx101.amr.corp.intel.com ([10.22.225.128]) by orsmga005.jf.intel.com with ESMTP; 28 Aug 2019 11:40:04 -0700 Received: from orsmsx121.amr.corp.intel.com (10.22.225.226) by ORSMSX101.amr.corp.intel.com (10.22.225.128) with Microsoft SMTP Server (TLS) id 14.3.439.0; Wed, 28 Aug 2019 11:40:04 -0700 Received: from orsmsx113.amr.corp.intel.com ([169.254.9.198]) by ORSMSX121.amr.corp.intel.com ([169.254.10.57]) with mapi id 14.03.0439.000; Wed, 28 Aug 2019 11:40:04 -0700 From: "Michael D Kinney" To: "Gao, Liming" , "Feng, Bob C" , "devel@edk2.groups.io" , "Kinney, Michael D" CC: "Shi, Steven" , "afish@apple.com" , "Laszlo Ersek (lersek@redhat.com)" , "leif.lindholm@linaro.org" Subject: Re: [Patch][edk2-stable201908] BaseTools: fix an incremental build issue. Thread-Topic: [Patch][edk2-stable201908] BaseTools: fix an incremental build issue. Thread-Index: AQHVXbTpX8DWtnZZ5Uy2SsPr9qetiKcQtoDQgAB5y4D//529kIAAFdTA Date: Wed, 28 Aug 2019 18:40:03 +0000 Message-ID: References: <20190828085209.25316-1-bob.c.feng@intel.com> <4A89E2EF3DFEDB4C8BFDE51014F606A14E4DDB59@SHSMSX104.ccr.corp.intel.com> <4A89E2EF3DFEDB4C8BFDE51014F606A14E4DDBDD@SHSMSX104.ccr.corp.intel.com> In-Reply-To: Accept-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: dlp-product: dlpe-windows dlp-version: 11.2.0.6 dlp-reaction: no-action x-originating-ip: [10.22.254.138] MIME-Version: 1.0 Return-Path: michael.d.kinney@intel.com Content-Language: en-US Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Liming, I can no longer reproduce the issues I noted below. I agree this is a critical regression issue that must be fixed for the edk2-stable201908 release. Tested-by: Michael D Kinney Acked-by: Michael D Kinney Thanks, Mike > -----Original Message----- > From: Kinney, Michael D > Sent: Wednesday, August 28, 2019 10:32 AM > To: Gao, Liming ; Feng, Bob C > ; devel@edk2.groups.io; Kinney, > Michael D > Cc: Shi, Steven ; > afish@apple.com; Laszlo Ersek (lersek@redhat.com) > ; leif.lindholm@linaro.org > Subject: RE: [Patch][edk2-stable201908] BaseTools: fix > an incremental build issue. >=20 > Liming, >=20 > This fix does not look complete to me. >=20 > I applied this patch and used FatPkg as a test. >=20 > I modified FatPkg/EnhancedFatDxe/Fat.inf removing > Defines, Source files, Library Classes, GUID, > Protocols, and PCDs and it never rebuilds the module. >=20 > If I remove the BASE_NAME define, I get a stack > trace...which does not appear to be a regression, but a > new issue. >=20 > build.py... > : error C0DE: Unknown fatal error when processing > [c:\work\github\tianocore\edk2\FatPkg\EnhancedFatDxe\Fa > t.inf] >=20 > (Please send email to devel@edk2.groups.io for help, > attaching following call stack trace!) >=20 > (Python 3.7.3 on win32) Traceback (most recent call > last): > File > "c:\work\GitHub\tianocore\edk2\BaseTools\Source\Python\ > build\build.py", line 2493, in Main > MyBuild.Launch() > File > "c:\work\GitHub\tianocore\edk2\BaseTools\Source\Python\ > build\build.py", line 2289, in Launch > self._MultiThreadBuildPlatform() > File > "c:\work\GitHub\tianocore\edk2\BaseTools\Source\Python\ > build\build.py", line 2021, in > _MultiThreadBuildPlatform > self.Progress > File > "c:\work\GitHub\tianocore\edk2\BaseTools\Source\Python\ > AutoGen\WorkspaceAutoGen.py", line 42, in __init__ > self._InitWorker(Workspace, MetaFile, Target, > Toolchain, Arch, *args, **kwargs) > File > "c:\work\GitHub\tianocore\edk2\BaseTools\Source\Python\ > AutoGen\WorkspaceAutoGen.py", line 112, in _InitWorker > self.ProcessPcdType() > File > "c:\work\GitHub\tianocore\edk2\BaseTools\Source\Python\ > AutoGen\WorkspaceAutoGen.py", line 240, in > ProcessPcdType > Platform.Pcds > File > "c:\work\GitHub\tianocore\edk2\BaseTools\Source\Python\ > Workspace\DscBuildData.py", line 1194, in Pcds > self._Pcds =3D > self.UpdateStructuredPcds(MODEL_PCD_TYPE_LIST, > self._Pcds) > File > "c:\work\GitHub\tianocore\edk2\BaseTools\Source\Python\ > Workspace\DscBuildData.py", line 1506, in > UpdateStructuredPcds > for Pcd in self.DecPcds: > File > "c:\work\GitHub\tianocore\edk2\BaseTools\Source\Python\ > Workspace\DscBuildData.py", line 3323, in DecPcds > self._DecPcds, self._GuidDict =3D > GetDeclaredPcd(self, self._Bdb, self._Arch, > self._Target, self._Toolchain, PkgSet) > File > "c:\work\GitHub\tianocore\edk2\BaseTools\Source\Python\ > Workspace\WorkspaceCommon.py", line 58, in > GetDeclaredPcd > PkgList =3D GetPackageList(Platform, BuildDatabase, > Arch, Target, Toolchain) > File > "c:\work\GitHub\tianocore\edk2\BaseTools\Source\Python\ > Workspace\WorkspaceCommon.py", line 42, in > GetPackageList > PkgSet.update(Data.Packages) > File > "c:\work\GitHub\tianocore\edk2\BaseTools\Source\Python\ > Common\caching.py", line 28, in __get__ > Value =3D obj.__dict__[self._function.__name__] =3D > self._function(obj) > File > "c:\work\GitHub\tianocore\edk2\BaseTools\Source\Python\ > Workspace\InfBuildData.py", line 696, in Packages > RecordList =3D self._RawData[MODEL_META_DATA_PACKAGE, > self._Arch, self._Platform] > File > "c:\work\GitHub\tianocore\edk2\BaseTools\Source\Python\ > Workspace\MetaFileParser.py", line 241, in __getitem__ > self.StartParse() > File > "c:\work\GitHub\tianocore\edk2\BaseTools\Source\Python\ > Workspace\MetaFileParser.py", line 260, in StartParse > self.Start() > File > "c:\work\GitHub\tianocore\edk2\BaseTools\Source\Python\ > Workspace\MetaFileParser.py", line 665, in Start > self._SectionParser[self._SectionType](self) > File > "c:\work\GitHub\tianocore\edk2\BaseTools\Source\Python\ > Workspace\MetaFileParser.py", line 46, in MacroParser > Parser(self) > File > "c:\work\GitHub\tianocore\edk2\BaseTools\Source\Python\ > Workspace\MetaFileParser.py", line 736, in > _SourceFileParser > if self._Defines['BASE_NAME'] =3D=3D 'Microcode': > KeyError: 'BASE_NAME' >=20 > If I go back to commit e8449e1d8e, the changes I tried > above to remove lines from the INF file always trigger > a rebuild of the module and fail as expected for the > content I removed. >=20 > I appears that the change to reduce autogen time has > many issues that are just now being discovered. >=20 > Do we need to consider a revert of the entire feature? >=20 > Mike >=20 > > -----Original Message----- > > From: Gao, Liming > > Sent: Wednesday, August 28, 2019 9:11 AM > > To: Kinney, Michael D ; > Feng, Bob C > > ; devel@edk2.groups.io > > Cc: Shi, Steven ; > afish@apple.com; Laszlo Ersek > > (lersek@redhat.com) ; > leif.lindholm@linaro.org > > Subject: RE: [Patch][edk2-stable201908] BaseTools: > fix an incremental > > build issue. > > > > Mike: > > I add notes in BZ. Current problem is that the > change in INF file > > will not trig rebuild. For example: > > > > 1st build: > > build -p MdeModulePkg\MdeModulePkg.dsc -a IA32 -m > > MdeModulePkg\Application\HelloWorld\HelloWorld.inf > > > > 2st build with below change > > update > MdeModulePkg\Application\HelloWorld\HelloWorld.inf > > and remove MdeModulePkg/MdeModulePkg.dec from > [Packages] section > > > > 2st build should update Makefile and trig build. But > now, there is no > > build trig. > > > > So, this is the critical incremental build issue. > > > > > > Thanks > > Liming > > > -----Original Message----- > > > From: Kinney, Michael D > > > Sent: Wednesday, August 28, 2019 11:56 PM > > > To: Gao, Liming ; Feng, Bob C > > > ; devel@edk2.groups.io; > Kinney, > > Michael D > > > > > > Cc: Shi, Steven ; > > afish@apple.com; Laszlo Ersek > > > (lersek@redhat.com) ; > > leif.lindholm@linaro.org > > > Subject: RE: [Patch][edk2-stable201908] BaseTools: > fix > > an incremental build issue. > > > > > > Bob and Liming, > > > > > > What is an example test case that fails? Please > add to > > BZ. > > > > > > We need this information to test that this fix > resolves > > the issue. > > > > > > Mike > > > > > > > -----Original Message----- > > > > From: Gao, Liming > > > > Sent: Wednesday, August 28, 2019 8:26 AM > > > > To: Feng, Bob C ; > > devel@edk2.groups.io > > > > Cc: Shi, Steven ; Kinney, > > Michael D > > > > ; afish@apple.com; > Laszlo > > Ersek > > > > (lersek@redhat.com) ; > > leif.lindholm@linaro.org > > > > Subject: RE: [Patch][edk2-stable201908] > BaseTools: > > fix an > > > > incremental build issue. > > > > > > > > Bob: > > > > This is the critical regression issue for the > > incremental build. I > > > > request to catch it in this stable tag 201908. > > > > > > > > This patch corrects the behavior. Reviewed-by: > > Liming Gao > > > > > > > > > > > > Thanks > > > > Liming > > > > > -----Original Message----- > > > > > From: Feng, Bob C > > > > > Sent: Wednesday, August 28, 2019 4:52 PM > > > > > To: devel@edk2.groups.io > > > > > Cc: Feng, Bob C ; Gao, > Liming > > > > > ; Shi, Steven > > > > > > > > > Subject: [Patch][edk2-stable201908] BaseTools: > fix > > an > > > > incremental build issue. > > > > > > > > > > BZ: > > https://bugzilla.tianocore.org/show_bug.cgi?id=3D2137 > > > > > > > > > > This patch is going to fix a regression issue > of > > > > incremental build > > > > > which was introduced by the commit 94459080c. > > > > > The changing on INF file will not trigger > module > > > > rebuild. > > > > > > > > > > Signed-off-by: Bob Feng > > > > > Cc: Liming Gao > > > > > Cc: Steven Shi > > > > > --- > > > > > > BaseTools/Source/Python/AutoGen/ModuleAutoGen.py | > > 8 > > > > ++------ > > > > > 1 file changed, 2 insertions(+), 6 deletions(- > ) > > > > > > > > > > diff --git > > > > > a/BaseTools/Source/Python/AutoGen/ModuleAutoGen.py > > > > > > b/BaseTools/Source/Python/AutoGen/ModuleAutoGen.py > > > > > index 2cd0d3859e..f610365f6d 100755 > > > > > --- > > a/BaseTools/Source/Python/AutoGen/ModuleAutoGen.py > > > > > +++ > > b/BaseTools/Source/Python/AutoGen/ModuleAutoGen.py > > > > > @@ -1764,16 +1764,12 @@ class > > ModuleAutoGen(AutoGen): > > > > > for f in self.AutoGenDepSet: > > > > > FileSet.add (f.Path) > > > > > > > > > > if os.path.exists > > (self.TimeStampPath): > > > > > os.remove (self.TimeStampPath) > > > > > - with > tempfile.NamedTemporaryFile('w+', > > > > dir=3Dos.path.dirname(self.TimeStampPath), > > delete=3DFalse) as > > > > tf: > > > > > - for f in FileSet: > > > > > - tf.write(f) > > > > > - tf.write("\n") > > > > > - tempname =3D tf.name > > > > > - > SaveFileOnChange(self.TimeStampPath, > > > > tempname, False) > > > > > + > > > > > + > SaveFileOnChange(self.TimeStampPath, > > > > "\n".join(FileSet), > > > > > + False) > > > > > > > > > > # Ignore generating makefile when it > is a > > > > binary module > > > > > if self.IsBinaryModule: > > > > > return > > > > > > > > > > -- > > > > > 2.20.1.windows.1