From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received-SPF: Pass (sender SPF authorized) identity=mailfrom; client-ip=192.55.52.120; helo=mga04.intel.com; envelope-from=bob.c.feng@intel.com; receiver=edk2-devel@lists.01.org Received: from mga04.intel.com (mga04.intel.com [192.55.52.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ml01.01.org (Postfix) with ESMTPS id 8337E211EB83A for ; Fri, 29 Mar 2019 04:06:36 -0700 (PDT) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga004.jf.intel.com ([10.7.209.38]) by fmsmga104.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 29 Mar 2019 04:06:36 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.60,284,1549958400"; d="scan'208";a="286981049" Received: from shwdepsi1121.ccr.corp.intel.com ([10.239.158.47]) by orsmga004.jf.intel.com with ESMTP; 29 Mar 2019 04:06:34 -0700 From: "Feng, Bob C" To: edk2-devel@lists.01.org Cc: Bob Feng , Liming Gao Date: Fri, 29 Mar 2019 19:06:31 +0800 Message-Id: <20190329110631.79000-1-bob.c.feng@intel.com> X-Mailer: git-send-email 2.20.1.windows.1 MIME-Version: 1.0 Subject: [Patch] BaseTools: Fixed issue in MultiThread Genfds function X-BeenThere: edk2-devel@lists.01.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: EDK II Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 29 Mar 2019 11:06:36 -0000 Content-Transfer-Encoding: 8bit https://bugzilla.tianocore.org/show_bug.cgi?id=1450 In the Multiple thread Genfds feature, build tool generates GenSec, GenFFS command in Makefile. The Non-Hii Driver does not generate .offset file for uni string offset, but the build tool has not knowledge about this in autogen phase. So in this patch, I add a check in Makefile for GenSec command. If the GenSec input file does not exist, the GenSec will not be called. And if GenSec command is not called, its output file, which is also the input file of GenFfs command, will also not exist.So for GenFfs command, I add a new command parameter -oi which means the input file is an optional input file which would not exist. so that I can generate GenFfs command with "-oi" parameter in Makefile. Contributed-under: TianoCore Contribution Agreement 1.1 Signed-off-by: Bob Feng Cc: Liming Gao --- BaseTools/Conf/build_rule.template | 14 +++++++------- BaseTools/Source/C/GenFfs/GenFfs.c | 17 +++++++++++++++-- BaseTools/Source/Python/AutoGen/GenMake.py | 2 +- .../Python/GenFds/GenFdsGlobalVariable.py | 10 +++++++++- .../Source/Python/Workspace/DscBuildData.py | 3 +++ BaseTools/Source/Python/build/build.py | 10 +++++----- 6 files changed, 40 insertions(+), 16 deletions(-) diff --git a/BaseTools/Conf/build_rule.template b/BaseTools/Conf/build_rule.template index e56b1d9c59..8cd7d0e445 100755 --- a/BaseTools/Conf/build_rule.template +++ b/BaseTools/Conf/build_rule.template @@ -357,16 +357,16 @@ [Dynamic-Library-File] ?.dll - $(DEBUG_DIR)(+)$(MODULE_NAME).efi + $(DEBUG_DIR)(+)$(MODULE_NAME).efi $(OUTPUT_DIR)(+)$(MODULE_NAME).efi "$(GENFW)" -e $(MODULE_TYPE) -o ${dst} ${src} $(GENFW_FLAGS) - $(CP) ${dst} $(OUTPUT_DIR) - $(CP) ${dst} $(BIN_DIR)(+)$(MODULE_NAME_GUID).efi + $(CP) $(DEBUG_DIR)(+)$(MODULE_NAME).efi $(OUTPUT_DIR) + $(CP) $(DEBUG_DIR)(+)$(MODULE_NAME).efi $(BIN_DIR)(+)$(MODULE_NAME_GUID).efi -$(CP) $(DEBUG_DIR)(+)*.map $(OUTPUT_DIR) -$(CP) $(DEBUG_DIR)(+)*.pdb $(OUTPUT_DIR) $(CP) ${src} $(DEBUG_DIR)(+)$(MODULE_NAME).debug $(OBJCOPY) --strip-unneeded -R .eh_frame ${src} @@ -376,22 +376,22 @@ # -$(OBJCOPY) $(OBJCOPY_ADDDEBUGFLAG) ${src} -$(CP) $(DEBUG_DIR)(+)$(MODULE_NAME).debug $(BIN_DIR)(+)$(MODULE_NAME_GUID).debug "$(GENFW)" -e $(MODULE_TYPE) -o ${dst} ${src} $(GENFW_FLAGS) - $(CP) ${dst} $(OUTPUT_DIR) - $(CP) ${dst} $(BIN_DIR)(+)$(MODULE_NAME_GUID).efi + $(CP) $(DEBUG_DIR)(+)$(MODULE_NAME).efi $(OUTPUT_DIR) + $(CP) $(DEBUG_DIR)(+)$(MODULE_NAME).efi $(BIN_DIR)(+)$(MODULE_NAME_GUID).efi -$(CP) $(DEBUG_DIR)(+)*.map $(OUTPUT_DIR) # tool to convert Mach-O to PE/COFF "$(MTOC)" -subsystem $(MODULE_TYPE) $(MTOC_FLAGS) ${src} $(DEBUG_DIR)(+)$(MODULE_NAME).pecoff # create symbol file for GDB debug -$(DSYMUTIL) ${src} "$(GENFW)" -e $(MODULE_TYPE) -o ${dst} $(DEBUG_DIR)(+)$(MODULE_NAME).pecoff $(GENFW_FLAGS) - $(CP) ${dst} $(OUTPUT_DIR) - $(CP) ${dst} $(BIN_DIR)(+)$(MODULE_NAME_GUID).efi + $(CP) $(DEBUG_DIR)(+)$(MODULE_NAME).efi $(OUTPUT_DIR) + $(CP) $(DEBUG_DIR)(+)$(MODULE_NAME).efi $(BIN_DIR)(+)$(MODULE_NAME_GUID).efi -$(CP) $(DEBUG_DIR)(+)*.map $(OUTPUT_DIR) [Dependency-Expression-File] ?.dxs, ?.Dxs, ?.DXS diff --git a/BaseTools/Source/C/GenFfs/GenFfs.c b/BaseTools/Source/C/GenFfs/GenFfs.c index 02c7ac30f0..ea36b22bef 100644 --- a/BaseTools/Source/C/GenFfs/GenFfs.c +++ b/BaseTools/Source/C/GenFfs/GenFfs.c @@ -17,10 +17,14 @@ WITHOUT WARRANTIES OR REPRESENTATIONS OF ANY KIND, EITHER EXPRESS OR IMPLIED. #include #include #include #endif +#ifdef __GNUC__ +#include +#endif + #include #include #include #include @@ -158,10 +162,12 @@ Returns: FileAlign points to file alignment, which only support\n\ the following align: 1,2,4,8,16,128,512,1K,4K,32K,64K\n\ 128K,256K,512K,1M,2M,4M,8M,16M\n"); fprintf (stdout, " -i SectionFile, --sectionfile SectionFile\n\ Section file will be contained in this FFS file.\n"); + fprintf (stdout, " -oi SectionFile, --optionalsectionfile SectionFile\n\ + If the Section file exists, it will be contained in this FFS file, otherwise, it will be ignored.\n"); fprintf (stdout, " -n SectionAlign, --sectionalign SectionAlign\n\ SectionAlign points to section alignment, which support\n\ the alignment scope 0~16M. If SectionAlign is specified\n\ as 0, tool get alignment value from SectionFile. It is\n\ specified together with sectionfile to point its\n\ @@ -734,19 +740,26 @@ Returns: argc -= 2; argv += 2; continue; } - if ((stricmp (argv[0], "-i") == 0) || (stricmp (argv[0], "--sectionfile") == 0)) { + if ((stricmp (argv[0], "-oi") == 0) || (stricmp (argv[0], "--optionalsectionfile") == 0) || (stricmp (argv[0], "-i") == 0) || (stricmp (argv[0], "--sectionfile") == 0)) { // // Get Input file name and its alignment // if (argv[1] == NULL || argv[1][0] == '-') { Error (NULL, 0, 1003, "Invalid option value", "input section file is missing for -i option"); goto Finish; } - + if ((stricmp (argv[0], "-oi") == 0) || (stricmp (argv[0], "--optionalsectionfile") == 0) ){ + if (-1 == access(argv[1] , 0)){ + Warning(NULL, 0, 0001, "File is not found.", argv[1]); + argc -= 2; + argv += 2; + continue; + } + } // // Allocate Input file name buffer and its alignment buffer. // if ((InputFileNum == 0) && (InputFileName == NULL)) { InputFileName = (CHAR8 **) malloc (MAXIMUM_INPUT_FILE_NUM * sizeof (CHAR8 *)); diff --git a/BaseTools/Source/Python/AutoGen/GenMake.py b/BaseTools/Source/Python/AutoGen/GenMake.py index b441817b52..2374bcf550 100644 --- a/BaseTools/Source/Python/AutoGen/GenMake.py +++ b/BaseTools/Source/Python/AutoGen/GenMake.py @@ -714,11 +714,11 @@ cleanlib: FfsCmdList = Cmd[0] for index, Str in enumerate(FfsCmdList): if '-o' == Str: OutputFile = FfsCmdList[index + 1] - if '-i' == Str: + if '-i' == Str or "-oi" == Str: if DepsFileList == []: DepsFileList = [FfsCmdList[index + 1]] else: DepsFileList.append(FfsCmdList[index + 1]) DepsFileString = ' '.join(DepsFileList).strip() diff --git a/BaseTools/Source/Python/GenFds/GenFdsGlobalVariable.py b/BaseTools/Source/Python/GenFds/GenFdsGlobalVariable.py index 028bcc480c..b42829b698 100644 --- a/BaseTools/Source/Python/GenFds/GenFdsGlobalVariable.py +++ b/BaseTools/Source/Python/GenFds/GenFdsGlobalVariable.py @@ -33,10 +33,11 @@ from Common.ToolDefClassObject import ToolDefClassObject, ToolDefDict from AutoGen.BuildEngine import BuildRule import Common.DataType as DataType from Common.Misc import PathClass from Common.LongFilePathSupport import OpenLongFilePath as open from Common.MultipleWorkspace import MultipleWorkspace as mws +import Common.GlobalData as GlobalData ## Global variables # # class GenFdsGlobalVariable: @@ -499,10 +500,14 @@ class GenFdsGlobalVariable: Cmd += ("-o", Output) Cmd += Input SaveFileOnChange(CommandFile, ' '.join(Cmd), False) if IsMakefile: + if GlobalData.gGlobalDefines.get("FAMILY") == "MSFT": + Cmd = ['if', 'exist', Input[0]] + Cmd + else: + Cmd = ['test', '-e', Input[0], "&&"] + Cmd if ' '.join(Cmd).strip() not in GenFdsGlobalVariable.SecCmdList: GenFdsGlobalVariable.SecCmdList.append(' '.join(Cmd).strip()) elif GenFdsGlobalVariable.NeedsUpdate(Output, list(Input) + [CommandFile]): GenFdsGlobalVariable.DebugLogger(EdkLogger.DEBUG_5, "%s needs update because of newer %s" % (Output, Input)) GenFdsGlobalVariable.CallExternalTool(Cmd, "Failed to generate section") @@ -540,11 +545,14 @@ class GenFdsGlobalVariable: Align = mFfsValidAlign[index + 1] Cmd += ("-a", Align) Cmd += ("-o", Output) for I in range(0, len(Input)): - Cmd += ("-i", Input[I]) + if MakefilePath: + Cmd += ("-oi", Input[I]) + else: + Cmd += ("-i", Input[I]) if SectionAlign and SectionAlign[I]: Cmd += ("-n", SectionAlign[I]) CommandFile = Output + '.txt' SaveFileOnChange(CommandFile, ' '.join(Cmd), False) diff --git a/BaseTools/Source/Python/Workspace/DscBuildData.py b/BaseTools/Source/Python/Workspace/DscBuildData.py index 129c0c950b..ac95c7df17 100644 --- a/BaseTools/Source/Python/Workspace/DscBuildData.py +++ b/BaseTools/Source/Python/Workspace/DscBuildData.py @@ -318,10 +318,13 @@ class DscBuildData(PlatformBuildClassObject): ## Get architecture @property def Arch(self): return self._Arch + @property + def Dir(self): + return self.MetaFile.Dir ## Retrieve all information in [Defines] section # # (Retrieving all [Defines] information in one-shot is just to save time.) # diff --git a/BaseTools/Source/Python/build/build.py b/BaseTools/Source/Python/build/build.py index de641fb452..41d405e6ff 100644 --- a/BaseTools/Source/Python/build/build.py +++ b/BaseTools/Source/Python/build/build.py @@ -1667,11 +1667,11 @@ class Build(): self.Progress.Stop("done!") # Add ffs build to makefile CmdListDict = {} if GlobalData.gEnableGenfdsMultiThread and self.Fdf: - CmdListDict = self._GenFfsCmd() + CmdListDict = self._GenFfsCmd(Wa.ArchList) for Arch in Wa.ArchList: GlobalData.gGlobalDefines['ARCH'] = Arch Pa = PlatformAutoGen(Wa, self.PlatformFile, BuildTarget, ToolChain, Arch) for Module in Pa.Platform.Modules: @@ -1760,11 +1760,11 @@ class Build(): self.LoadFixAddress = Wa.Platform.LoadFixAddress Wa.CreateMakeFile(False) # Add ffs build to makefile CmdListDict = None if GlobalData.gEnableGenfdsMultiThread and self.Fdf: - CmdListDict = self._GenFfsCmd() + CmdListDict = self._GenFfsCmd(Wa.ArchList) self.Progress.Stop("done!") MaList = [] ExitFlag = threading.Event() ExitFlag.clear() self.AutoGenTime += int(round((time.time() - WorkspaceAutoGenTime))) @@ -1879,15 +1879,15 @@ class Build(): # # Save MAP buffer into MAP file. # self._SaveMapFile (MapBuffer, Wa) - def _GenFfsCmd(self): + def _GenFfsCmd(self,ArchList): # convert dictionary of Cmd:(Inf,Arch) # to a new dictionary of (Inf,Arch):Cmd,Cmd,Cmd... CmdSetDict = defaultdict(set) - GenFfsDict = GenFds.GenFfsMakefile('', GlobalData.gFdfParser, self, self.ArchList, GlobalData) + GenFfsDict = GenFds.GenFfsMakefile('', GlobalData.gFdfParser, self, ArchList, GlobalData) for Cmd in GenFfsDict: tmpInf, tmpArch = GenFfsDict[Cmd] CmdSetDict[tmpInf, tmpArch].add(Cmd) return CmdSetDict @@ -1927,11 +1927,11 @@ class Build(): Wa.CreateMakeFile(False) # Add ffs build to makefile CmdListDict = None if GlobalData.gEnableGenfdsMultiThread and self.Fdf: - CmdListDict = self._GenFfsCmd() + CmdListDict = self._GenFfsCmd(Wa.ArchList) # multi-thread exit flag ExitFlag = threading.Event() ExitFlag.clear() self.AutoGenTime += int(round((time.time() - WorkspaceAutoGenTime))) -- 2.20.1.windows.1