From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received-SPF: Pass (sender SPF authorized) identity=mailfrom; client-ip=134.134.136.31; helo=mga06.intel.com; envelope-from=liming.gao@intel.com; receiver=edk2-devel@lists.01.org Received: from mga06.intel.com (mga06.intel.com [134.134.136.31]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ml01.01.org (Postfix) with ESMTPS id 7A67F211EC0FF for ; Fri, 29 Mar 2019 04:41:35 -0700 (PDT) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga006.fm.intel.com ([10.253.24.20]) by orsmga104.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 29 Mar 2019 04:41:35 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.60,284,1549958400"; d="scan'208";a="331852409" Received: from fmsmsx105.amr.corp.intel.com ([10.18.124.203]) by fmsmga006.fm.intel.com with ESMTP; 29 Mar 2019 04:41:34 -0700 Received: from fmsmsx116.amr.corp.intel.com (10.18.116.20) by FMSMSX105.amr.corp.intel.com (10.18.124.203) with Microsoft SMTP Server (TLS) id 14.3.408.0; Fri, 29 Mar 2019 04:41:34 -0700 Received: from shsmsx103.ccr.corp.intel.com (10.239.4.69) by fmsmsx116.amr.corp.intel.com (10.18.116.20) with Microsoft SMTP Server (TLS) id 14.3.408.0; Fri, 29 Mar 2019 04:41:34 -0700 Received: from shsmsx104.ccr.corp.intel.com ([169.254.5.74]) by SHSMSX103.ccr.corp.intel.com ([169.254.4.134]) with mapi id 14.03.0415.000; Fri, 29 Mar 2019 19:41:32 +0800 From: "Gao, Liming" To: "Feng, Bob C" , "edk2-devel@lists.01.org" Thread-Topic: [Patch] BaseTools: Fixed issue in MultiThread Genfds function Thread-Index: AQHU5h99+njQK3KUTUyQPIANObxYSKYifFOw Date: Fri, 29 Mar 2019 11:41:32 +0000 Message-ID: <4A89E2EF3DFEDB4C8BFDE51014F606A14E40EBB2@SHSMSX104.ccr.corp.intel.com> References: <20190329110631.79000-1-bob.c.feng@intel.com> In-Reply-To: <20190329110631.79000-1-bob.c.feng@intel.com> Accept-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ctpclassification: CTP_NT x-titus-metadata-40: eyJDYXRlZ29yeUxhYmVscyI6IiIsIk1ldGFkYXRhIjp7Im5zIjoiaHR0cDpcL1wvd3d3LnRpdHVzLmNvbVwvbnNcL0ludGVsMyIsImlkIjoiYzIxZjc3ZDYtNzk5Yy00M2Y5LWIyZWUtMjQwNmRiMzlhMTY1IiwicHJvcHMiOlt7Im4iOiJDVFBDbGFzc2lmaWNhdGlvbiIsInZhbHMiOlt7InZhbHVlIjoiQ1RQX05UIn1dfV19LCJTdWJqZWN0TGFiZWxzIjpbXSwiVE1DVmVyc2lvbiI6IjE3LjEwLjE4MDQuNDkiLCJUcnVzdGVkTGFiZWxIYXNoIjoiUlBMczFid28wZHRrbTFFWFRZbHBUQU9yUzRDWU01K0g1SFc5QTJyNzdMb3lMRlk0a2lLTTFiSjIwVllyMHp5WSJ9 dlp-product: dlpe-windows dlp-version: 11.0.400.15 dlp-reaction: no-action x-originating-ip: [10.239.127.40] MIME-Version: 1.0 Subject: Re: [Patch] BaseTools: Fixed issue in MultiThread Genfds function X-BeenThere: edk2-devel@lists.01.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: EDK II Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 29 Mar 2019 11:41:36 -0000 Content-Language: en-US Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Bob: Could you list the more information on why update build_rule.txt? > -----Original Message----- > From: Feng, Bob C > Sent: Friday, March 29, 2019 7:07 PM > To: edk2-devel@lists.01.org > Cc: Feng, Bob C ; Gao, Liming > Subject: [Patch] BaseTools: Fixed issue in MultiThread Genfds function >=20 > https://bugzilla.tianocore.org/show_bug.cgi?id=3D1450 > In the Multiple thread Genfds feature, build tool generates > GenSec, GenFFS command in Makefile. >=20 > The Non-Hii Driver does not generate .offset file for uni string offset, > but the build tool has not knowledge about this in autogen phase. So > in this patch, I add a check in Makefile for GenSec command. If the GenSe= c > input file does not exist, the GenSec will not be called. And if GenSec > command is not called, its output file, which is also the input file of > GenFfs command, will also not exist.So for GenFfs command, > I add a new command parameter -oi which means > the input file is an optional input file which would not exist. so > that I can generate GenFfs command with "-oi" parameter in Makefile. >=20 > Contributed-under: TianoCore Contribution Agreement 1.1 > Signed-off-by: Bob Feng > Cc: Liming Gao > --- > BaseTools/Conf/build_rule.template | 14 +++++++------- > BaseTools/Source/C/GenFfs/GenFfs.c | 17 +++++++++++++++-- > BaseTools/Source/Python/AutoGen/GenMake.py | 2 +- > .../Python/GenFds/GenFdsGlobalVariable.py | 10 +++++++++- > .../Source/Python/Workspace/DscBuildData.py | 3 +++ > BaseTools/Source/Python/build/build.py | 10 +++++----- > 6 files changed, 40 insertions(+), 16 deletions(-) >=20 > diff --git a/BaseTools/Conf/build_rule.template b/BaseTools/Conf/build_ru= le.template > index e56b1d9c59..8cd7d0e445 100755 > --- a/BaseTools/Conf/build_rule.template > +++ b/BaseTools/Conf/build_rule.template > @@ -357,16 +357,16 @@ > [Dynamic-Library-File] > > ?.dll >=20 > > - $(DEBUG_DIR)(+)$(MODULE_NAME).efi > + $(DEBUG_DIR)(+)$(MODULE_NAME).efi $(OUTPUT_DIR)(+)$(MODULE_NAME)= .efi >=20 > > "$(GENFW)" -e $(MODULE_TYPE) -o ${dst} ${src} $(GENFW_FLAGS) > - $(CP) ${dst} $(OUTPUT_DIR) > - $(CP) ${dst} $(BIN_DIR)(+)$(MODULE_NAME_GUID).efi > + $(CP) $(DEBUG_DIR)(+)$(MODULE_NAME).efi $(OUTPUT_DIR) > + $(CP) $(DEBUG_DIR)(+)$(MODULE_NAME).efi $(BIN_DIR)(+)$(MODULE_NA= ME_GUID).efi > -$(CP) $(DEBUG_DIR)(+)*.map $(OUTPUT_DIR) > -$(CP) $(DEBUG_DIR)(+)*.pdb $(OUTPUT_DIR) > > $(CP) ${src} $(DEBUG_DIR)(+)$(MODULE_NAME).debug > $(OBJCOPY) --strip-unneeded -R .eh_frame ${src} > @@ -376,22 +376,22 @@ > # > -$(OBJCOPY) $(OBJCOPY_ADDDEBUGFLAG) ${src} > -$(CP) $(DEBUG_DIR)(+)$(MODULE_NAME).debug $(BIN_DIR)(+)$(MODULE= _NAME_GUID).debug >=20 > "$(GENFW)" -e $(MODULE_TYPE) -o ${dst} ${src} $(GENFW_FLAGS) > - $(CP) ${dst} $(OUTPUT_DIR) > - $(CP) ${dst} $(BIN_DIR)(+)$(MODULE_NAME_GUID).efi > + $(CP) $(DEBUG_DIR)(+)$(MODULE_NAME).efi $(OUTPUT_DIR) > + $(CP) $(DEBUG_DIR)(+)$(MODULE_NAME).efi $(BIN_DIR)(+)$(MODULE_NA= ME_GUID).efi > -$(CP) $(DEBUG_DIR)(+)*.map $(OUTPUT_DIR) >=20 > > # tool to convert Mach-O to PE/COFF > "$(MTOC)" -subsystem $(MODULE_TYPE) $(MTOC_FLAGS) ${src} $(DE= BUG_DIR)(+)$(MODULE_NAME).pecoff > # create symbol file for GDB debug > -$(DSYMUTIL) ${src} > "$(GENFW)" -e $(MODULE_TYPE) -o ${dst} $(DEBUG_DIR)(+)$(MODULE_N= AME).pecoff $(GENFW_FLAGS) > - $(CP) ${dst} $(OUTPUT_DIR) > - $(CP) ${dst} $(BIN_DIR)(+)$(MODULE_NAME_GUID).efi > + $(CP) $(DEBUG_DIR)(+)$(MODULE_NAME).efi $(OUTPUT_DIR) > + $(CP) $(DEBUG_DIR)(+)$(MODULE_NAME).efi $(BIN_DIR)(+)$(MODULE_NA= ME_GUID).efi > -$(CP) $(DEBUG_DIR)(+)*.map $(OUTPUT_DIR) >=20 > [Dependency-Expression-File] > > ?.dxs, ?.Dxs, ?.DXS > diff --git a/BaseTools/Source/C/GenFfs/GenFfs.c b/BaseTools/Source/C/GenF= fs/GenFfs.c > index 02c7ac30f0..ea36b22bef 100644 > --- a/BaseTools/Source/C/GenFfs/GenFfs.c > +++ b/BaseTools/Source/C/GenFfs/GenFfs.c > @@ -17,10 +17,14 @@ WITHOUT WARRANTIES OR REPRESENTATIONS OF ANY KIND, EI= THER EXPRESS OR IMPLIED. > #include > #include > #include > #endif >=20 > +#ifdef __GNUC__ > +#include > +#endif > + > #include > #include > #include >=20 > #include > @@ -158,10 +162,12 @@ Returns: > FileAlign points to file alignment, which only s= upport\n\ > the following align: 1,2,4,8,16,128,512,1K,4K,32= K,64K\n\ > 128K,256K,512K,1M,2M,4M,8M,16M\n"); > fprintf (stdout, " -i SectionFile, --sectionfile SectionFile\n\ > Section file will be contained in this FFS file.= \n"); > + fprintf (stdout, " -oi SectionFile, --optionalsectionfile SectionFile= \n\ > + If the Section file exists, it will be contained= in this FFS file, otherwise, it will be ignored.\n"); > fprintf (stdout, " -n SectionAlign, --sectionalign SectionAlign\n\ > SectionAlign points to section alignment, which = support\n\ > the alignment scope 0~16M. If SectionAlign is sp= ecified\n\ > as 0, tool get alignment value from SectionFile.= It is\n\ > specified together with sectionfile to point its= \n\ > @@ -734,19 +740,26 @@ Returns: > argc -=3D 2; > argv +=3D 2; > continue; > } >=20 > - if ((stricmp (argv[0], "-i") =3D=3D 0) || (stricmp (argv[0], "--sect= ionfile") =3D=3D 0)) { > + if ((stricmp (argv[0], "-oi") =3D=3D 0) || (stricmp (argv[0], "--opt= ionalsectionfile") =3D=3D 0) || (stricmp (argv[0], "-i") =3D=3D 0) || (stri= cmp (argv[0], > "--sectionfile") =3D=3D 0)) { > // > // Get Input file name and its alignment > // > if (argv[1] =3D=3D NULL || argv[1][0] =3D=3D '-') { > Error (NULL, 0, 1003, "Invalid option value", "input section fil= e is missing for -i option"); > goto Finish; > } > - > + if ((stricmp (argv[0], "-oi") =3D=3D 0) || (stricmp (argv[0], "--o= ptionalsectionfile") =3D=3D 0) ){ > + if (-1 =3D=3D access(argv[1] , 0)){ > + Warning(NULL, 0, 0001, "File is not found.", argv[1]); > + argc -=3D 2; > + argv +=3D 2; > + continue; > + } > + } > // > // Allocate Input file name buffer and its alignment buffer. > // > if ((InputFileNum =3D=3D 0) && (InputFileName =3D=3D NULL)) { > InputFileName =3D (CHAR8 **) malloc (MAXIMUM_INPUT_FILE_NUM * si= zeof (CHAR8 *)); > diff --git a/BaseTools/Source/Python/AutoGen/GenMake.py b/BaseTools/Sourc= e/Python/AutoGen/GenMake.py > index b441817b52..2374bcf550 100644 > --- a/BaseTools/Source/Python/AutoGen/GenMake.py > +++ b/BaseTools/Source/Python/AutoGen/GenMake.py > @@ -714,11 +714,11 @@ cleanlib: >=20 > FfsCmdList =3D Cmd[0] > for index, Str in enumerate(FfsCmdList): > if '-o' =3D=3D Str: > OutputFile =3D FfsCmdList[index + 1] > - if '-i' =3D=3D Str: > + if '-i' =3D=3D Str or "-oi" =3D=3D Str: > if DepsFileList =3D=3D []: > DepsFileList =3D [FfsCmdList[index + 1]] > else: > DepsFileList.append(FfsCmdList[index + 1]) > DepsFileString =3D ' '.join(DepsFileList).strip() > diff --git a/BaseTools/Source/Python/GenFds/GenFdsGlobalVariable.py b/Bas= eTools/Source/Python/GenFds/GenFdsGlobalVariable.py > index 028bcc480c..b42829b698 100644 > --- a/BaseTools/Source/Python/GenFds/GenFdsGlobalVariable.py > +++ b/BaseTools/Source/Python/GenFds/GenFdsGlobalVariable.py > @@ -33,10 +33,11 @@ from Common.ToolDefClassObject import ToolDefClassObj= ect, ToolDefDict > from AutoGen.BuildEngine import BuildRule > import Common.DataType as DataType > from Common.Misc import PathClass > from Common.LongFilePathSupport import OpenLongFilePath as open > from Common.MultipleWorkspace import MultipleWorkspace as mws > +import Common.GlobalData as GlobalData >=20 > ## Global variables > # > # > class GenFdsGlobalVariable: > @@ -499,10 +500,14 @@ class GenFdsGlobalVariable: > Cmd +=3D ("-o", Output) > Cmd +=3D Input >=20 > SaveFileOnChange(CommandFile, ' '.join(Cmd), False) > if IsMakefile: > + if GlobalData.gGlobalDefines.get("FAMILY") =3D=3D "MSFT"= : > + Cmd =3D ['if', 'exist', Input[0]] + Cmd > + else: > + Cmd =3D ['test', '-e', Input[0], "&&"] + Cmd > if ' '.join(Cmd).strip() not in GenFdsGlobalVariable.Sec= CmdList: > GenFdsGlobalVariable.SecCmdList.append(' '.join(Cmd)= .strip()) > elif GenFdsGlobalVariable.NeedsUpdate(Output, list(Input) + = [CommandFile]): > GenFdsGlobalVariable.DebugLogger(EdkLogger.DEBUG_5, "%s = needs update because of newer %s" % (Output, Input)) > GenFdsGlobalVariable.CallExternalTool(Cmd, "Failed to ge= nerate section") > @@ -540,11 +545,14 @@ class GenFdsGlobalVariable: > Align =3D mFfsValidAlign[index + 1] > Cmd +=3D ("-a", Align) >=20 > Cmd +=3D ("-o", Output) > for I in range(0, len(Input)): > - Cmd +=3D ("-i", Input[I]) > + if MakefilePath: > + Cmd +=3D ("-oi", Input[I]) > + else: > + Cmd +=3D ("-i", Input[I]) > if SectionAlign and SectionAlign[I]: > Cmd +=3D ("-n", SectionAlign[I]) >=20 > CommandFile =3D Output + '.txt' > SaveFileOnChange(CommandFile, ' '.join(Cmd), False) > diff --git a/BaseTools/Source/Python/Workspace/DscBuildData.py b/BaseTool= s/Source/Python/Workspace/DscBuildData.py > index 129c0c950b..ac95c7df17 100644 > --- a/BaseTools/Source/Python/Workspace/DscBuildData.py > +++ b/BaseTools/Source/Python/Workspace/DscBuildData.py > @@ -318,10 +318,13 @@ class DscBuildData(PlatformBuildClassObject): >=20 > ## Get architecture > @property > def Arch(self): > return self._Arch > + @property > + def Dir(self): > + return self.MetaFile.Dir >=20 > ## Retrieve all information in [Defines] section > # > # (Retrieving all [Defines] information in one-shot is just to sav= e time.) > # > diff --git a/BaseTools/Source/Python/build/build.py b/BaseTools/Source/Py= thon/build/build.py > index de641fb452..41d405e6ff 100644 > --- a/BaseTools/Source/Python/build/build.py > +++ b/BaseTools/Source/Python/build/build.py > @@ -1667,11 +1667,11 @@ class Build(): > self.Progress.Stop("done!") >=20 > # Add ffs build to makefile > CmdListDict =3D {} > if GlobalData.gEnableGenfdsMultiThread and self.Fdf: > - CmdListDict =3D self._GenFfsCmd() > + CmdListDict =3D self._GenFfsCmd(Wa.ArchList) >=20 > for Arch in Wa.ArchList: > GlobalData.gGlobalDefines['ARCH'] =3D Arch > Pa =3D PlatformAutoGen(Wa, self.PlatformFile, BuildT= arget, ToolChain, Arch) > for Module in Pa.Platform.Modules: > @@ -1760,11 +1760,11 @@ class Build(): > self.LoadFixAddress =3D Wa.Platform.LoadFixAddress > Wa.CreateMakeFile(False) > # Add ffs build to makefile > CmdListDict =3D None > if GlobalData.gEnableGenfdsMultiThread and self.Fdf: > - CmdListDict =3D self._GenFfsCmd() > + CmdListDict =3D self._GenFfsCmd(Wa.ArchList) > self.Progress.Stop("done!") > MaList =3D [] > ExitFlag =3D threading.Event() > ExitFlag.clear() > self.AutoGenTime +=3D int(round((time.time() - Workspace= AutoGenTime))) > @@ -1879,15 +1879,15 @@ class Build(): > # > # Save MAP buffer into MAP file. > # > self._SaveMapFile (MapBuffer, Wa) >=20 > - def _GenFfsCmd(self): > + def _GenFfsCmd(self,ArchList): > # convert dictionary of Cmd:(Inf,Arch) > # to a new dictionary of (Inf,Arch):Cmd,Cmd,Cmd... > CmdSetDict =3D defaultdict(set) > - GenFfsDict =3D GenFds.GenFfsMakefile('', GlobalData.gFdfParser, = self, self.ArchList, GlobalData) > + GenFfsDict =3D GenFds.GenFfsMakefile('', GlobalData.gFdfParser, = self, ArchList, GlobalData) > for Cmd in GenFfsDict: > tmpInf, tmpArch =3D GenFfsDict[Cmd] > CmdSetDict[tmpInf, tmpArch].add(Cmd) > return CmdSetDict >=20 > @@ -1927,11 +1927,11 @@ class Build(): > Wa.CreateMakeFile(False) >=20 > # Add ffs build to makefile > CmdListDict =3D None > if GlobalData.gEnableGenfdsMultiThread and self.Fdf: > - CmdListDict =3D self._GenFfsCmd() > + CmdListDict =3D self._GenFfsCmd(Wa.ArchList) >=20 > # multi-thread exit flag > ExitFlag =3D threading.Event() > ExitFlag.clear() > self.AutoGenTime +=3D int(round((time.time() - Workspace= AutoGenTime))) > -- > 2.20.1.windows.1