From mboxrd@z Thu Jan 1 00:00:00 1970 Authentication-Results: mx.groups.io; dkim=missing; spf=pass (domain: intel.com, ip: 134.134.136.100, mailfrom: jiewen.yao@intel.com) Received: from mga07.intel.com (mga07.intel.com [134.134.136.100]) by groups.io with SMTP; Thu, 09 May 2019 05:35:07 -0700 X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga003.fm.intel.com ([10.253.24.29]) by orsmga105.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 09 May 2019 05:35:07 -0700 X-ExtLoop1: 1 Received: from fmsmsx104.amr.corp.intel.com ([10.18.124.202]) by FMSMGA003.fm.intel.com with ESMTP; 09 May 2019 05:35:06 -0700 Received: from fmsmsx113.amr.corp.intel.com (10.18.116.7) by fmsmsx104.amr.corp.intel.com (10.18.124.202) with Microsoft SMTP Server (TLS) id 14.3.408.0; Thu, 9 May 2019 05:35:06 -0700 Received: from shsmsx105.ccr.corp.intel.com (10.239.4.158) by FMSMSX113.amr.corp.intel.com (10.18.116.7) with Microsoft SMTP Server (TLS) id 14.3.408.0; Thu, 9 May 2019 05:35:05 -0700 Received: from shsmsx102.ccr.corp.intel.com ([169.254.2.249]) by SHSMSX105.ccr.corp.intel.com ([169.254.11.10]) with mapi id 14.03.0415.000; Thu, 9 May 2019 20:34:55 +0800 From: "Yao, Jiewen" To: "Shi, Steven" , "Gao, Liming" , "devel@edk2.groups.io" CC: "Feng, Bob C" , "Rodriguez, Christian" , "Johnson, Michael" Subject: Re: [edk2-devel] [PATCH] BaseTools: Skip .mcb file module in Binary Cache Thread-Topic: [edk2-devel] [PATCH] BaseTools: Skip .mcb file module in Binary Cache Thread-Index: AQHVBiQbDvKwhXdSS0qcIE8w/gBgD6ZiPyKA//+DuwCAAAHOgIAA9CGQ Date: Thu, 9 May 2019 12:34:54 +0000 Message-ID: <74D8A39837DF1E4DA445A8C0B3885C503F637351@shsmsx102.ccr.corp.intel.com> References: <20190509045957.592-1-steven.shi@intel.com> <74D8A39837DF1E4DA445A8C0B3885C503F635E39@shsmsx102.ccr.corp.intel.com> <4A89E2EF3DFEDB4C8BFDE51014F606A14E4457A4@SHSMSX104.ccr.corp.intel.com> <06C8AB66E78EE34A949939824ABE2B313FFB781A@shsmsx102.ccr.corp.intel.com> In-Reply-To: <06C8AB66E78EE34A949939824ABE2B313FFB781A@shsmsx102.ccr.corp.intel.com> Accept-Language: zh-CN, en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-titus-metadata-40: eyJDYXRlZ29yeUxhYmVscyI6IiIsIk1ldGFkYXRhIjp7Im5zIjoiaHR0cDpcL1wvd3d3LnRpdHVzLmNvbVwvbnNcL0ludGVsMyIsImlkIjoiODc3Yzc5MjctNDY3Zi00MjM1LTkyM2QtNTI0ZGIyNjQ5NzVkIiwicHJvcHMiOlt7Im4iOiJDVFBDbGFzc2lmaWNhdGlvbiIsInZhbHMiOlt7InZhbHVlIjoiQ1RQX05UIn1dfV19LCJTdWJqZWN0TGFiZWxzIjpbXSwiVE1DVmVyc2lvbiI6IjE3LjEwLjE4MDQuNDkiLCJUcnVzdGVkTGFiZWxIYXNoIjoiQ0FRclBoS0FhdGtYdENnc05CTEs0V2xUMzczTWhxTDk1UXlIXC80d1NhM1N1RFpMckZ0SzRVMEk1XC9yZjRJdUlWIn0= x-ctpclassification: CTP_NT dlp-product: dlpe-windows dlp-version: 11.0.600.7 dlp-reaction: no-action x-originating-ip: [10.239.127.40] MIME-Version: 1.0 Return-Path: jiewen.yao@intel.com Content-Language: en-US Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Thanks to explain that. I think my general concern is: why we hardcode file suffix "INC" or "MCB" = in the tool source code. > > > > - if '.inc' in str(f_ext): > > > > + if '.inc' in str(f_ext) or '.mcb' in str(f_ext): > > > > return False What happen if we need skip another type of file later? update tool source= code again ? My thought is that: the tool should provide capability to skip *some* type= of file. What "some" type of file to be skipped should be configured outside of too= l source code. As such, when we need skip more type of file, we just update the configura= tion, instead of tool source code. Just like we have tools_def and build_rule today, when we need update thos= e configuration, we don't need update tool source code. Thank you Yao Jiewen > -----Original Message----- > From: Shi, Steven > Sent: Wednesday, May 8, 2019 10:54 PM > To: Gao, Liming ; Yao, Jiewen > ; devel@edk2.groups.io > Cc: Feng, Bob C ; Rodriguez, Christian > ; Johnson, Michael > > Subject: RE: [edk2-devel] [PATCH] BaseTools: Skip .mcb file module in Bi= nary > Cache >=20 > > > Can we have better way to describe the binary cache? > > Yes. Steven provides some wiki pages for it. > https://raw.githubusercontent.com/shijunjing/WikiDoc/master/BuildCache/ > MultipleBuildCache.jpg >=20 > I agree it sounds a bit confusing. Binary Cache is the legacy name for = the > current model level build cache implementation. The current implementati= on > try only save and recovery the module binary files (e.g. only cache .efi= but > no .obj, no .lib) which are necessary for the FV generation. But if we a= dd .lib > cache and .fv cache support in the future, we could rename it as Build C= ache > directly. >=20 >=20 > Thanks > Steven Shi >=20 >=20 > > -----Original Message----- > > From: Gao, Liming > > Sent: Thursday, May 9, 2019 1:47 PM > > To: Yao, Jiewen ; devel@edk2.groups.io; Shi, > Steven > > > > Cc: Feng, Bob C ; Rodriguez, Christian > > ; Johnson, Michael > > > > Subject: RE: [edk2-devel] [PATCH] BaseTools: Skip .mcb file module in > Binary > > Cache > > > > Jiewen: > > > > > -----Original Message----- > > > From: Yao, Jiewen > > > Sent: Thursday, May 9, 2019 1:13 PM > > > To: devel@edk2.groups.io; Shi, Steven > > > Cc: Feng, Bob C ; Gao, Liming > > ; Rodriguez, Christian > > ; > > > Johnson, Michael > > > Subject: RE: [edk2-devel] [PATCH] BaseTools: Skip .mcb file module i= n > Binary > > Cache > > > > > > Hi > > > Can we have better way to describe the binary cache? > > Yes. Steven provides some wiki pages for it. > > > > > > I have 2 concern: > > > 1) we have other type of binary besides MCB. How can we indicate tha= t > > without update base tool source code ? > > MCB will be combined to single bin file. The problem is that the platf= orm > uses > > the bin file from output IA32 or X64 > > directory instead of the module output directory. Binary cache is for = the > > module output directory. > > I suggest to update the platform to consume the module output file. Bu= t, I > > meet with one tool issue to support > > RAW FILE type, because Microcode uses RAW file type. BZ > > https://bugzilla.tianocore.org/show_bug.cgi?id=3D1765 is submitted. > > I suggest to fix this tool issue first. > > > > > 2) .inc might be text file, instead of binary. How can we support th= at? > > .inc is similar to .c source file. Its output is .mcb, then be combine= d to .bin > > file. .bin file will be cached. > > .inc is not required to be cached. On next build, build tool will comp= are the > > hash value of source file, if > > hash value is not changed. Previous cached .bin file will be used. > > > > > > > > Thank you > > > Yao Jiewen > > > > > > > > > > -----Original Message----- > > > > From: devel@edk2.groups.io [mailto:devel@edk2.groups.io] On Behalf > Of > > > > Steven Shi > > > > Sent: Wednesday, May 8, 2019 10:00 PM > > > > To: devel@edk2.groups.io > > > > Cc: Feng, Bob C ; Gao, Liming > > > > ; Rodriguez, Christian > > > > ; Johnson, Michael > > > > > > > > Subject: [edk2-devel] [PATCH] BaseTools: Skip .mcb file module in > Binary > > > > Cache > > > > > > > > REF: https://bugzilla.tianocore.org/show_bug.cgi?id=3D1723 > > > > > > > > Current Kabylake open platform will build fail if enabled > > > > to consume the binary cache, because the binary cache doesn't > > > > support to recovery the .mcb microcode file, > > > > e.g. m80406E8_00000026.mcb, in a platform level folder which > > > > is outside of the module output folder. In normal build > > > > without cache, the .mcb file is copied through OS copy/move > > > > commands defined in build rules which are not supported by > > > > Binary Cache. > > > > Change the Binary Cache to skip the .mcb file type module and > > > > always rebuild the module to apply the full build rules if > > > > it contains .mcb file. > > > > --- > > > > BaseTools/Source/Python/AutoGen/AutoGen.py | 8 ++++---- > > > > 1 file changed, 4 insertions(+), 4 deletions(-) > > > > > > > > diff --git a/BaseTools/Source/Python/AutoGen/AutoGen.py > > > > b/BaseTools/Source/Python/AutoGen/AutoGen.py > > > > index 31721a6f9f..6b596c8a65 100644 > > > > --- a/BaseTools/Source/Python/AutoGen/AutoGen.py > > > > +++ b/BaseTools/Source/Python/AutoGen/AutoGen.py > > > > @@ -3925,9 +3925,9 @@ class ModuleAutoGen(AutoGen): > > > > # If library or Module is binary do not skip by hash > > > > if self.IsBinaryModule: > > > > return False > > > > - # .inc is contains binary information so do not skip by h= ash as > well > > > > + # .inc and .mcb is contains binary information so do not = skip > by > > > > hash as well > > > > for f_ext in self.SourceFileList: > > > > - if '.inc' in str(f_ext): > > > > + if '.inc' in str(f_ext) or '.mcb' in str(f_ext): > > > > return False > > > > FileDir =3D path.join(GlobalData.gBinCacheSource, > > > > self.PlatformInfo.Name, self.BuildTarget + "_" + self.ToolChain, > self.Arch, > > > > self.SourceDir, self.MetaFile.BaseName) > > > > HashFile =3D path.join(FileDir, self.Name + '.hash') > > > > @@ -4138,9 +4138,9 @@ class ModuleAutoGen(AutoGen): > > > > # If library or Module is binary do not skip by hash > > > > if self.IsBinaryModule: > > > > return False > > > > - # .inc is contains binary information so do not skip by h= ash as > well > > > > + # .inc or '.mcb' is contains binary information so do not= skip > by > > > > hash as well > > > > for f_ext in self.SourceFileList: > > > > - if '.inc' in str(f_ext): > > > > + if '.inc' in str(f_ext) or '.mcb' in str(f_ext): > > > > return False > > > > if GlobalData.gUseHashCache: > > > > # If there is a valid hash or function generated a va= lid > hash; > > > > function will return False > > > > -- > > > > 2.17.1.windows.2 > > > > > > > > > > > >=20