From mboxrd@z Thu Jan 1 00:00:00 1970 Authentication-Results: mx.groups.io; dkim=missing; spf=pass (domain: intel.com, ip: 134.134.136.20, mailfrom: michael.d.kinney@intel.com) Received: from mga02.intel.com (mga02.intel.com [134.134.136.20]) by groups.io with SMTP; Wed, 28 Aug 2019 10:31:56 -0700 X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga002.jf.intel.com ([10.7.209.21]) by orsmga101.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 28 Aug 2019 10:31:55 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.64,441,1559545200"; d="scan'208";a="192668727" Received: from orsmsx102.amr.corp.intel.com ([10.22.225.129]) by orsmga002.jf.intel.com with ESMTP; 28 Aug 2019 10:31:55 -0700 Received: from orsmsx123.amr.corp.intel.com (10.22.240.116) by ORSMSX102.amr.corp.intel.com (10.22.225.129) with Microsoft SMTP Server (TLS) id 14.3.439.0; Wed, 28 Aug 2019 10:31:55 -0700 Received: from orsmsx113.amr.corp.intel.com ([169.254.9.198]) by ORSMSX123.amr.corp.intel.com ([169.254.1.98]) with mapi id 14.03.0439.000; Wed, 28 Aug 2019 10:31:55 -0700 From: "Michael D Kinney" To: "Gao, Liming" , "Feng, Bob C" , "devel@edk2.groups.io" , "Kinney, Michael D" CC: "Shi, Steven" , "afish@apple.com" , "Laszlo Ersek (lersek@redhat.com)" , "leif.lindholm@linaro.org" Subject: Re: [Patch][edk2-stable201908] BaseTools: fix an incremental build issue. Thread-Topic: [Patch][edk2-stable201908] BaseTools: fix an incremental build issue. Thread-Index: AQHVXbTpX8DWtnZZ5Uy2SsPr9qetiKcQtoDQgAB5y4D//529kA== Date: Wed, 28 Aug 2019 17:31:54 +0000 Message-ID: References: <20190828085209.25316-1-bob.c.feng@intel.com> <4A89E2EF3DFEDB4C8BFDE51014F606A14E4DDB59@SHSMSX104.ccr.corp.intel.com> <4A89E2EF3DFEDB4C8BFDE51014F606A14E4DDBDD@SHSMSX104.ccr.corp.intel.com> In-Reply-To: <4A89E2EF3DFEDB4C8BFDE51014F606A14E4DDBDD@SHSMSX104.ccr.corp.intel.com> Accept-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: dlp-product: dlpe-windows dlp-version: 11.2.0.6 dlp-reaction: no-action x-originating-ip: [10.22.254.138] MIME-Version: 1.0 Return-Path: michael.d.kinney@intel.com Content-Language: en-US Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Liming, This fix does not look complete to me. I applied this patch and used FatPkg as a test. I modified FatPkg/EnhancedFatDxe/Fat.inf removing Defines, Source files, Library Classes, GUID, Protocols,=20 and PCDs and it never rebuilds the module. If I remove the BASE_NAME define, I get a stack trace...which does not appear to be a regression, but a new issue. build.py... : error C0DE: Unknown fatal error when processing [c:\work\github\tianocor= e\edk2\FatPkg\EnhancedFatDxe\Fat.inf] (Please send email to devel@edk2.groups.io for help, attaching following ca= ll stack trace!) (Python 3.7.3 on win32) Traceback (most recent call last): File "c:\work\GitHub\tianocore\edk2\BaseTools\Source\Python\build\build.p= y", line 2493, in Main MyBuild.Launch() File "c:\work\GitHub\tianocore\edk2\BaseTools\Source\Python\build\build.p= y", line 2289, in Launch self._MultiThreadBuildPlatform() File "c:\work\GitHub\tianocore\edk2\BaseTools\Source\Python\build\build.p= y", line 2021, in _MultiThreadBuildPlatform self.Progress File "c:\work\GitHub\tianocore\edk2\BaseTools\Source\Python\AutoGen\Works= paceAutoGen.py", line 42, in __init__ self._InitWorker(Workspace, MetaFile, Target, Toolchain, Arch, *args, *= *kwargs) File "c:\work\GitHub\tianocore\edk2\BaseTools\Source\Python\AutoGen\Works= paceAutoGen.py", line 112, in _InitWorker self.ProcessPcdType() File "c:\work\GitHub\tianocore\edk2\BaseTools\Source\Python\AutoGen\Works= paceAutoGen.py", line 240, in ProcessPcdType Platform.Pcds File "c:\work\GitHub\tianocore\edk2\BaseTools\Source\Python\Workspace\Dsc= BuildData.py", line 1194, in Pcds self._Pcds =3D self.UpdateStructuredPcds(MODEL_PCD_TYPE_LIST, self._Pcd= s) File "c:\work\GitHub\tianocore\edk2\BaseTools\Source\Python\Workspace\Dsc= BuildData.py", line 1506, in UpdateStructuredPcds for Pcd in self.DecPcds: File "c:\work\GitHub\tianocore\edk2\BaseTools\Source\Python\Workspace\Dsc= BuildData.py", line 3323, in DecPcds self._DecPcds, self._GuidDict =3D GetDeclaredPcd(self, self._Bdb, self.= _Arch, self._Target, self._Toolchain, PkgSet) File "c:\work\GitHub\tianocore\edk2\BaseTools\Source\Python\Workspace\Wor= kspaceCommon.py", line 58, in GetDeclaredPcd PkgList =3D GetPackageList(Platform, BuildDatabase, Arch, Target, Toolc= hain) File "c:\work\GitHub\tianocore\edk2\BaseTools\Source\Python\Workspace\Wor= kspaceCommon.py", line 42, in GetPackageList PkgSet.update(Data.Packages) File "c:\work\GitHub\tianocore\edk2\BaseTools\Source\Python\Common\cachin= g.py", line 28, in __get__ Value =3D obj.__dict__[self._function.__name__] =3D self._function(obj) File "c:\work\GitHub\tianocore\edk2\BaseTools\Source\Python\Workspace\Inf= BuildData.py", line 696, in Packages RecordList =3D self._RawData[MODEL_META_DATA_PACKAGE, self._Arch, self.= _Platform] File "c:\work\GitHub\tianocore\edk2\BaseTools\Source\Python\Workspace\Met= aFileParser.py", line 241, in __getitem__ self.StartParse() File "c:\work\GitHub\tianocore\edk2\BaseTools\Source\Python\Workspace\Met= aFileParser.py", line 260, in StartParse self.Start() File "c:\work\GitHub\tianocore\edk2\BaseTools\Source\Python\Workspace\Met= aFileParser.py", line 665, in Start self._SectionParser[self._SectionType](self) File "c:\work\GitHub\tianocore\edk2\BaseTools\Source\Python\Workspace\Met= aFileParser.py", line 46, in MacroParser Parser(self) File "c:\work\GitHub\tianocore\edk2\BaseTools\Source\Python\Workspace\Met= aFileParser.py", line 736, in _SourceFileParser if self._Defines['BASE_NAME'] =3D=3D 'Microcode': KeyError: 'BASE_NAME' If I go back to commit e8449e1d8e, the changes I tried above to remove line= s from the INF file always trigger a rebuild of the module and fail as expect= ed for the content I removed. I appears that the change to reduce autogen time has many issues that are j= ust now being discovered. Do we need to consider a revert of the entire feature? Mike > -----Original Message----- > From: Gao, Liming > Sent: Wednesday, August 28, 2019 9:11 AM > To: Kinney, Michael D ; Feng, > Bob C ; devel@edk2.groups.io > Cc: Shi, Steven ; afish@apple.com; > Laszlo Ersek (lersek@redhat.com) ; > leif.lindholm@linaro.org > Subject: RE: [Patch][edk2-stable201908] BaseTools: fix an > incremental build issue. >=20 > Mike: > I add notes in BZ. Current problem is that the change > in INF file will not trig rebuild. For example: >=20 > 1st build: > build -p MdeModulePkg\MdeModulePkg.dsc -a IA32 -m > MdeModulePkg\Application\HelloWorld\HelloWorld.inf >=20 > 2st build with below change > update MdeModulePkg\Application\HelloWorld\HelloWorld.inf > and remove MdeModulePkg/MdeModulePkg.dec from [Packages] > section >=20 > 2st build should update Makefile and trig build. But now, > there is no build trig. >=20 > So, this is the critical incremental build issue. >=20 >=20 > Thanks > Liming > > -----Original Message----- > > From: Kinney, Michael D > > Sent: Wednesday, August 28, 2019 11:56 PM > > To: Gao, Liming ; Feng, Bob C > > ; devel@edk2.groups.io; Kinney, > Michael D > > > > Cc: Shi, Steven ; > afish@apple.com; Laszlo Ersek > > (lersek@redhat.com) ; > leif.lindholm@linaro.org > > Subject: RE: [Patch][edk2-stable201908] BaseTools: fix > an incremental build issue. > > > > Bob and Liming, > > > > What is an example test case that fails? Please add to > BZ. > > > > We need this information to test that this fix resolves > the issue. > > > > Mike > > > > > -----Original Message----- > > > From: Gao, Liming > > > Sent: Wednesday, August 28, 2019 8:26 AM > > > To: Feng, Bob C ; > devel@edk2.groups.io > > > Cc: Shi, Steven ; Kinney, > Michael D > > > ; afish@apple.com; Laszlo > Ersek > > > (lersek@redhat.com) ; > leif.lindholm@linaro.org > > > Subject: RE: [Patch][edk2-stable201908] BaseTools: > fix an > > > incremental build issue. > > > > > > Bob: > > > This is the critical regression issue for the > incremental build. I > > > request to catch it in this stable tag 201908. > > > > > > This patch corrects the behavior. Reviewed-by: > Liming Gao > > > > > > > > > Thanks > > > Liming > > > > -----Original Message----- > > > > From: Feng, Bob C > > > > Sent: Wednesday, August 28, 2019 4:52 PM > > > > To: devel@edk2.groups.io > > > > Cc: Feng, Bob C ; Gao, Liming > > > > ; Shi, Steven > > > > > > > Subject: [Patch][edk2-stable201908] BaseTools: fix > an > > > incremental build issue. > > > > > > > > BZ: > https://bugzilla.tianocore.org/show_bug.cgi?id=3D2137 > > > > > > > > This patch is going to fix a regression issue of > > > incremental build > > > > which was introduced by the commit 94459080c. > > > > The changing on INF file will not trigger module > > > rebuild. > > > > > > > > Signed-off-by: Bob Feng > > > > Cc: Liming Gao > > > > Cc: Steven Shi > > > > --- > > > > BaseTools/Source/Python/AutoGen/ModuleAutoGen.py | > 8 > > > ++------ > > > > 1 file changed, 2 insertions(+), 6 deletions(-) > > > > > > > > diff --git > > > a/BaseTools/Source/Python/AutoGen/ModuleAutoGen.py > > > > b/BaseTools/Source/Python/AutoGen/ModuleAutoGen.py > > > > index 2cd0d3859e..f610365f6d 100755 > > > > --- > a/BaseTools/Source/Python/AutoGen/ModuleAutoGen.py > > > > +++ > b/BaseTools/Source/Python/AutoGen/ModuleAutoGen.py > > > > @@ -1764,16 +1764,12 @@ class > ModuleAutoGen(AutoGen): > > > > for f in self.AutoGenDepSet: > > > > FileSet.add (f.Path) > > > > > > > > if os.path.exists > (self.TimeStampPath): > > > > os.remove (self.TimeStampPath) > > > > - with tempfile.NamedTemporaryFile('w+', > > > dir=3Dos.path.dirname(self.TimeStampPath), > delete=3DFalse) as > > > tf: > > > > - for f in FileSet: > > > > - tf.write(f) > > > > - tf.write("\n") > > > > - tempname =3D tf.name > > > > - SaveFileOnChange(self.TimeStampPath, > > > tempname, False) > > > > + > > > > + SaveFileOnChange(self.TimeStampPath, > > > "\n".join(FileSet), > > > > + False) > > > > > > > > # Ignore generating makefile when it is a > > > binary module > > > > if self.IsBinaryModule: > > > > return > > > > > > > > -- > > > > 2.20.1.windows.1