From: "Liming Gao" <liming.gao@intel.com>
To: "Feng, Bob C" <bob.c.feng@intel.com>,
"devel@edk2.groups.io" <devel@edk2.groups.io>
Subject: Re: [Patch V2] BaseTools: Fixed issue in MultiThread Genfds function
Date: Tue, 9 Apr 2019 00:18:43 +0000 [thread overview]
Message-ID: <4A89E2EF3DFEDB4C8BFDE51014F606A14E420B33@SHSMSX104.ccr.corp.intel.com> (raw)
In-Reply-To: <08650203BA1BD64D8AD9B6D5D74A85D1600D1AE0@SHSMSX101.ccr.corp.intel.com>
If so, the change is good to me. Reviewed-by: Liming Gao <liming.gao@intel.com>
> -----Original Message-----
> From: Feng, Bob C
> Sent: Wednesday, April 3, 2019 10:56 AM
> To: Gao, Liming <liming.gao@intel.com>
> Cc: edk2-devel@lists.01.org
> Subject: RE: [Patch V2] BaseTools: Fixed issue in MultiThread Genfds function
>
> If user does not use multiple thread genfds, there will be no problem.
>
> -----Original Message-----
> From: Gao, Liming
> Sent: Wednesday, April 3, 2019 10:49 AM
> To: Feng, Bob C <bob.c.feng@intel.com>
> Cc: edk2-devel@lists.01.org
> Subject: RE: [Patch V2] BaseTools: Fixed issue in MultiThread Genfds function
>
> Bob:
> Thanks for your updating. If user only update BaseTools, and don't update Conf\build_rule.txt. What problem will happen?
>
> Thanks
> Liming
> > -----Original Message-----
> > From: Feng, Bob C
> > Sent: Wednesday, April 3, 2019 10:17 AM
> > To: edk2-devel@lists.01.org
> > Cc: Feng, Bob C <bob.c.feng@intel.com>; Gao, Liming
> > <liming.gao@intel.com>
> > Subject: [Patch V2] BaseTools: Fixed issue in MultiThread Genfds
> > function
> >
> > https://bugzilla.tianocore.org/show_bug.cgi?id=1450
> > In the Multiple thread Genfds feature, build tool generates GenSec,
> > GenFFS command in Makefile.
> >
> > The Non-Hii Driver does not generate .offset file for uni string
> > offset, but the build tool has not knowledge about this in autogen
> > phase. So in this patch, I add a check in Makefile for GenSec command.
> > If the GenSec input file does not exist, the GenSec will not be
> > called. And if GenSec command is not called, its output file, which is
> > also the input file of GenFfs command, will also not exist.So for
> > GenFfs command, I add a new command parameter -oi which means the
> > input file is an optional input file which would not exist. so that I
> > can generate GenFfs command with "-oi" parameter in Makefile.
> >
> > Contributed-under: TianoCore Contribution Agreement 1.1
> > Signed-off-by: Bob Feng <bob.c.feng@intel.com>
> > Cc: Liming Gao <liming.gao@intel.com>
> > ---
> > BaseTools/Conf/build_rule.template | 8 ++++----
> > BaseTools/Source/C/GenFfs/GenFfs.c | 17 +++++++++++++++--
> > BaseTools/Source/Python/AutoGen/GenMake.py | 2 +-
> > .../Python/GenFds/GenFdsGlobalVariable.py | 10 +++++++++-
> > .../Source/Python/Workspace/DscBuildData.py | 3 +++
> > BaseTools/Source/Python/build/build.py | 10 +++++-----
> > 6 files changed, 37 insertions(+), 13 deletions(-)
> >
> > diff --git a/BaseTools/Conf/build_rule.template
> > b/BaseTools/Conf/build_rule.template
> > index e56b1d9c59..17e7353063 100755
> > --- a/BaseTools/Conf/build_rule.template
> > +++ b/BaseTools/Conf/build_rule.template
> > @@ -357,15 +357,15 @@
> > [Dynamic-Library-File]
> > <InputFile>
> > ?.dll
> >
> > <OutputFile>
> > - $(DEBUG_DIR)(+)$(MODULE_NAME).efi
> > + $(OUTPUT_DIR)(+)$(MODULE_NAME).efi
> >
> > <Command.MSFT, Command.INTEL, Command.RVCT>
> > "$(GENFW)" -e $(MODULE_TYPE) -o ${dst} ${src} $(GENFW_FLAGS)
> > - $(CP) ${dst} $(OUTPUT_DIR)
> > + $(CP) ${dst} $(DEBUG_DIR)
> > $(CP) ${dst} $(BIN_DIR)(+)$(MODULE_NAME_GUID).efi
> > -$(CP) $(DEBUG_DIR)(+)*.map $(OUTPUT_DIR)
> > -$(CP) $(DEBUG_DIR)(+)*.pdb $(OUTPUT_DIR)
> > <Command.GCC>
> > $(CP) ${src} $(DEBUG_DIR)(+)$(MODULE_NAME).debug
> > @@ -376,21 +376,21 @@
> > #
> > -$(OBJCOPY) $(OBJCOPY_ADDDEBUGFLAG) ${src}
> > -$(CP) $(DEBUG_DIR)(+)$(MODULE_NAME).debug
> > $(BIN_DIR)(+)$(MODULE_NAME_GUID).debug
> >
> > "$(GENFW)" -e $(MODULE_TYPE) -o ${dst} ${src} $(GENFW_FLAGS)
> > - $(CP) ${dst} $(OUTPUT_DIR)
> > + $(CP) ${dst} $(DEBUG_DIR)
> > $(CP) ${dst} $(BIN_DIR)(+)$(MODULE_NAME_GUID).efi
> > -$(CP) $(DEBUG_DIR)(+)*.map $(OUTPUT_DIR)
> >
> > <Command.XCODE>
> > # tool to convert Mach-O to PE/COFF
> > "$(MTOC)" -subsystem $(MODULE_TYPE) $(MTOC_FLAGS) ${src} $(DEBUG_DIR)(+)$(MODULE_NAME).pecoff
> > # create symbol file for GDB debug
> > -$(DSYMUTIL) ${src}
> > "$(GENFW)" -e $(MODULE_TYPE) -o ${dst} $(DEBUG_DIR)(+)$(MODULE_NAME).pecoff $(GENFW_FLAGS)
> > - $(CP) ${dst} $(OUTPUT_DIR)
> > + $(CP) ${dst} $(DEBUG_DIR)
> > $(CP) ${dst} $(BIN_DIR)(+)$(MODULE_NAME_GUID).efi
> > -$(CP) $(DEBUG_DIR)(+)*.map $(OUTPUT_DIR)
> >
> > [Dependency-Expression-File]
> > <InputFile>
> > diff --git a/BaseTools/Source/C/GenFfs/GenFfs.c
> > b/BaseTools/Source/C/GenFfs/GenFfs.c
> > index 02c7ac30f0..ea36b22bef 100644
> > --- a/BaseTools/Source/C/GenFfs/GenFfs.c
> > +++ b/BaseTools/Source/C/GenFfs/GenFfs.c
> > @@ -17,10 +17,14 @@ WITHOUT WARRANTIES OR REPRESENTATIONS OF ANY KIND, EITHER EXPRESS OR IMPLIED.
> > #include <io.h>
> > #include <sys/types.h>
> > #include <sys/stat.h>
> > #endif
> >
> > +#ifdef __GNUC__
> > +#include <unistd.h>
> > +#endif
> > +
> > #include <stdio.h>
> > #include <stdlib.h>
> > #include <string.h>
> >
> > #include <Common/UefiBaseTypes.h>
> > @@ -158,10 +162,12 @@ Returns:
> > FileAlign points to file alignment, which only support\n\
> > the following align: 1,2,4,8,16,128,512,1K,4K,32K,64K\n\
> > 128K,256K,512K,1M,2M,4M,8M,16M\n");
> > fprintf (stdout, " -i SectionFile, --sectionfile SectionFile\n\
> > Section file will be contained in this FFS
> > file.\n");
> > + fprintf (stdout, " -oi SectionFile, --optionalsectionfile SectionFile\n\
> > + If the Section file exists, it will be
> > + contained in this FFS file, otherwise, it will be ignored.\n");
> > fprintf (stdout, " -n SectionAlign, --sectionalign SectionAlign\n\
> > SectionAlign points to section alignment, which support\n\
> > the alignment scope 0~16M. If SectionAlign is specified\n\
> > as 0, tool get alignment value from SectionFile. It is\n\
> > specified together with sectionfile to point
> > its\n\ @@ -734,19 +740,26 @@ Returns:
> > argc -= 2;
> > argv += 2;
> > continue;
> > }
> >
> > - if ((stricmp (argv[0], "-i") == 0) || (stricmp (argv[0], "--sectionfile") == 0)) {
> > + if ((stricmp (argv[0], "-oi") == 0) || (stricmp (argv[0],
> > + "--optionalsectionfile") == 0) || (stricmp (argv[0], "-i") == 0) ||
> > + (stricmp (argv[0],
> > "--sectionfile") == 0)) {
> > //
> > // Get Input file name and its alignment
> > //
> > if (argv[1] == NULL || argv[1][0] == '-') {
> > Error (NULL, 0, 1003, "Invalid option value", "input section file is missing for -i option");
> > goto Finish;
> > }
> > -
> > + if ((stricmp (argv[0], "-oi") == 0) || (stricmp (argv[0], "--optionalsectionfile") == 0) ){
> > + if (-1 == access(argv[1] , 0)){
> > + Warning(NULL, 0, 0001, "File is not found.", argv[1]);
> > + argc -= 2;
> > + argv += 2;
> > + continue;
> > + }
> > + }
> > //
> > // Allocate Input file name buffer and its alignment buffer.
> > //
> > if ((InputFileNum == 0) && (InputFileName == NULL)) {
> > InputFileName = (CHAR8 **) malloc (MAXIMUM_INPUT_FILE_NUM *
> > sizeof (CHAR8 *)); diff --git
> > a/BaseTools/Source/Python/AutoGen/GenMake.py
> > b/BaseTools/Source/Python/AutoGen/GenMake.py
> > index b441817b52..2374bcf550 100644
> > --- a/BaseTools/Source/Python/AutoGen/GenMake.py
> > +++ b/BaseTools/Source/Python/AutoGen/GenMake.py
> > @@ -714,11 +714,11 @@ cleanlib:
> >
> > FfsCmdList = Cmd[0]
> > for index, Str in enumerate(FfsCmdList):
> > if '-o' == Str:
> > OutputFile = FfsCmdList[index + 1]
> > - if '-i' == Str:
> > + if '-i' == Str or "-oi" == Str:
> > if DepsFileList == []:
> > DepsFileList = [FfsCmdList[index + 1]]
> > else:
> > DepsFileList.append(FfsCmdList[index + 1])
> > DepsFileString = ' '.join(DepsFileList).strip() diff
> > --git a/BaseTools/Source/Python/GenFds/GenFdsGlobalVariable.py
> > b/BaseTools/Source/Python/GenFds/GenFdsGlobalVariable.py
> > index 028bcc480c..b42829b698 100644
> > --- a/BaseTools/Source/Python/GenFds/GenFdsGlobalVariable.py
> > +++ b/BaseTools/Source/Python/GenFds/GenFdsGlobalVariable.py
> > @@ -33,10 +33,11 @@ from Common.ToolDefClassObject import
> > ToolDefClassObject, ToolDefDict from AutoGen.BuildEngine import
> > BuildRule import Common.DataType as DataType from Common.Misc import
> > PathClass from Common.LongFilePathSupport import OpenLongFilePath as
> > open from Common.MultipleWorkspace import MultipleWorkspace as mws
> > +import Common.GlobalData as GlobalData
> >
> > ## Global variables
> > #
> > #
> > class GenFdsGlobalVariable:
> > @@ -499,10 +500,14 @@ class GenFdsGlobalVariable:
> > Cmd += ("-o", Output)
> > Cmd += Input
> >
> > SaveFileOnChange(CommandFile, ' '.join(Cmd), False)
> > if IsMakefile:
> > + if GlobalData.gGlobalDefines.get("FAMILY") == "MSFT":
> > + Cmd = ['if', 'exist', Input[0]] + Cmd
> > + else:
> > + Cmd = ['test', '-e', Input[0], "&&"] + Cmd
> > if ' '.join(Cmd).strip() not in GenFdsGlobalVariable.SecCmdList:
> > GenFdsGlobalVariable.SecCmdList.append(' '.join(Cmd).strip())
> > elif GenFdsGlobalVariable.NeedsUpdate(Output, list(Input) + [CommandFile]):
> > GenFdsGlobalVariable.DebugLogger(EdkLogger.DEBUG_5, "%s needs update because of newer %s" % (Output,
> Input))
> > GenFdsGlobalVariable.CallExternalTool(Cmd, "Failed to
> > generate section") @@ -540,11 +545,14 @@ class GenFdsGlobalVariable:
> > Align = mFfsValidAlign[index + 1]
> > Cmd += ("-a", Align)
> >
> > Cmd += ("-o", Output)
> > for I in range(0, len(Input)):
> > - Cmd += ("-i", Input[I])
> > + if MakefilePath:
> > + Cmd += ("-oi", Input[I])
> > + else:
> > + Cmd += ("-i", Input[I])
> > if SectionAlign and SectionAlign[I]:
> > Cmd += ("-n", SectionAlign[I])
> >
> > CommandFile = Output + '.txt'
> > SaveFileOnChange(CommandFile, ' '.join(Cmd), False) diff
> > --git a/BaseTools/Source/Python/Workspace/DscBuildData.py
> > b/BaseTools/Source/Python/Workspace/DscBuildData.py
> > index 129c0c950b..ac95c7df17 100644
> > --- a/BaseTools/Source/Python/Workspace/DscBuildData.py
> > +++ b/BaseTools/Source/Python/Workspace/DscBuildData.py
> > @@ -318,10 +318,13 @@ class DscBuildData(PlatformBuildClassObject):
> >
> > ## Get architecture
> > @property
> > def Arch(self):
> > return self._Arch
> > + @property
> > + def Dir(self):
> > + return self.MetaFile.Dir
> >
> > ## Retrieve all information in [Defines] section
> > #
> > # (Retrieving all [Defines] information in one-shot is just to save time.)
> > #
> > diff --git a/BaseTools/Source/Python/build/build.py
> > b/BaseTools/Source/Python/build/build.py
> > index de641fb452..41d405e6ff 100644
> > --- a/BaseTools/Source/Python/build/build.py
> > +++ b/BaseTools/Source/Python/build/build.py
> > @@ -1667,11 +1667,11 @@ class Build():
> > self.Progress.Stop("done!")
> >
> > # Add ffs build to makefile
> > CmdListDict = {}
> > if GlobalData.gEnableGenfdsMultiThread and self.Fdf:
> > - CmdListDict = self._GenFfsCmd()
> > + CmdListDict = self._GenFfsCmd(Wa.ArchList)
> >
> > for Arch in Wa.ArchList:
> > GlobalData.gGlobalDefines['ARCH'] = Arch
> > Pa = PlatformAutoGen(Wa, self.PlatformFile, BuildTarget, ToolChain, Arch)
> > for Module in Pa.Platform.Modules:
> > @@ -1760,11 +1760,11 @@ class Build():
> > self.LoadFixAddress = Wa.Platform.LoadFixAddress
> > Wa.CreateMakeFile(False)
> > # Add ffs build to makefile
> > CmdListDict = None
> > if GlobalData.gEnableGenfdsMultiThread and self.Fdf:
> > - CmdListDict = self._GenFfsCmd()
> > + CmdListDict = self._GenFfsCmd(Wa.ArchList)
> > self.Progress.Stop("done!")
> > MaList = []
> > ExitFlag = threading.Event()
> > ExitFlag.clear()
> > self.AutoGenTime += int(round((time.time() -
> > WorkspaceAutoGenTime))) @@ -1879,15 +1879,15 @@ class Build():
> > #
> > # Save MAP buffer into MAP file.
> > #
> > self._SaveMapFile (MapBuffer, Wa)
> >
> > - def _GenFfsCmd(self):
> > + def _GenFfsCmd(self,ArchList):
> > # convert dictionary of Cmd:(Inf,Arch)
> > # to a new dictionary of (Inf,Arch):Cmd,Cmd,Cmd...
> > CmdSetDict = defaultdict(set)
> > - GenFfsDict = GenFds.GenFfsMakefile('', GlobalData.gFdfParser, self, self.ArchList, GlobalData)
> > + GenFfsDict = GenFds.GenFfsMakefile('', GlobalData.gFdfParser,
> > + self, ArchList, GlobalData)
> > for Cmd in GenFfsDict:
> > tmpInf, tmpArch = GenFfsDict[Cmd]
> > CmdSetDict[tmpInf, tmpArch].add(Cmd)
> > return CmdSetDict
> >
> > @@ -1927,11 +1927,11 @@ class Build():
> > Wa.CreateMakeFile(False)
> >
> > # Add ffs build to makefile
> > CmdListDict = None
> > if GlobalData.gEnableGenfdsMultiThread and self.Fdf:
> > - CmdListDict = self._GenFfsCmd()
> > + CmdListDict = self._GenFfsCmd(Wa.ArchList)
> >
> > # multi-thread exit flag
> > ExitFlag = threading.Event()
> > ExitFlag.clear()
> > self.AutoGenTime += int(round((time.time() -
> > WorkspaceAutoGenTime)))
> > --
> > 2.20.1.windows.1
prev parent reply other threads:[~2019-04-09 0:18 UTC|newest]
Thread overview: 4+ messages / expand[flat|nested] mbox.gz Atom feed top
2019-04-03 2:17 [Patch V2] BaseTools: Fixed issue in MultiThread Genfds function Feng, Bob C
2019-04-03 2:48 ` Gao, Liming
2019-04-03 2:55 ` Feng, Bob C
2019-04-09 0:18 ` Liming Gao [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-list from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=4A89E2EF3DFEDB4C8BFDE51014F606A14E420B33@SHSMSX104.ccr.corp.intel.com \
--to=devel@edk2.groups.io \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox