* [Patch] BaseTools: Fixed issue in MultiThread Genfds function
@ 2019-03-29 11:06 Feng, Bob C
2019-03-29 11:41 ` Gao, Liming
0 siblings, 1 reply; 7+ messages in thread
From: Feng, Bob C @ 2019-03-29 11:06 UTC (permalink / raw)
To: edk2-devel; +Cc: Bob Feng, Liming Gao
https://bugzilla.tianocore.org/show_bug.cgi?id=1450
In the Multiple thread Genfds feature, build tool generates
GenSec, GenFFS command in Makefile.
The Non-Hii Driver does not generate .offset file for uni string offset,
but the build tool has not knowledge about this in autogen phase. So
in this patch, I add a check in Makefile for GenSec command. If the GenSec
input file does not exist, the GenSec will not be called. And if GenSec
command is not called, its output file, which is also the input file of
GenFfs command, will also not exist.So for GenFfs command,
I add a new command parameter -oi which means
the input file is an optional input file which would not exist. so
that I can generate GenFfs command with "-oi" parameter in Makefile.
Contributed-under: TianoCore Contribution Agreement 1.1
Signed-off-by: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
---
BaseTools/Conf/build_rule.template | 14 +++++++-------
BaseTools/Source/C/GenFfs/GenFfs.c | 17 +++++++++++++++--
BaseTools/Source/Python/AutoGen/GenMake.py | 2 +-
.../Python/GenFds/GenFdsGlobalVariable.py | 10 +++++++++-
.../Source/Python/Workspace/DscBuildData.py | 3 +++
BaseTools/Source/Python/build/build.py | 10 +++++-----
6 files changed, 40 insertions(+), 16 deletions(-)
diff --git a/BaseTools/Conf/build_rule.template b/BaseTools/Conf/build_rule.template
index e56b1d9c59..8cd7d0e445 100755
--- a/BaseTools/Conf/build_rule.template
+++ b/BaseTools/Conf/build_rule.template
@@ -357,16 +357,16 @@
[Dynamic-Library-File]
<InputFile>
?.dll
<OutputFile>
- $(DEBUG_DIR)(+)$(MODULE_NAME).efi
+ $(DEBUG_DIR)(+)$(MODULE_NAME).efi $(OUTPUT_DIR)(+)$(MODULE_NAME).efi
<Command.MSFT, Command.INTEL, Command.RVCT>
"$(GENFW)" -e $(MODULE_TYPE) -o ${dst} ${src} $(GENFW_FLAGS)
- $(CP) ${dst} $(OUTPUT_DIR)
- $(CP) ${dst} $(BIN_DIR)(+)$(MODULE_NAME_GUID).efi
+ $(CP) $(DEBUG_DIR)(+)$(MODULE_NAME).efi $(OUTPUT_DIR)
+ $(CP) $(DEBUG_DIR)(+)$(MODULE_NAME).efi $(BIN_DIR)(+)$(MODULE_NAME_GUID).efi
-$(CP) $(DEBUG_DIR)(+)*.map $(OUTPUT_DIR)
-$(CP) $(DEBUG_DIR)(+)*.pdb $(OUTPUT_DIR)
<Command.GCC>
$(CP) ${src} $(DEBUG_DIR)(+)$(MODULE_NAME).debug
$(OBJCOPY) --strip-unneeded -R .eh_frame ${src}
@@ -376,22 +376,22 @@
#
-$(OBJCOPY) $(OBJCOPY_ADDDEBUGFLAG) ${src}
-$(CP) $(DEBUG_DIR)(+)$(MODULE_NAME).debug $(BIN_DIR)(+)$(MODULE_NAME_GUID).debug
"$(GENFW)" -e $(MODULE_TYPE) -o ${dst} ${src} $(GENFW_FLAGS)
- $(CP) ${dst} $(OUTPUT_DIR)
- $(CP) ${dst} $(BIN_DIR)(+)$(MODULE_NAME_GUID).efi
+ $(CP) $(DEBUG_DIR)(+)$(MODULE_NAME).efi $(OUTPUT_DIR)
+ $(CP) $(DEBUG_DIR)(+)$(MODULE_NAME).efi $(BIN_DIR)(+)$(MODULE_NAME_GUID).efi
-$(CP) $(DEBUG_DIR)(+)*.map $(OUTPUT_DIR)
<Command.XCODE>
# tool to convert Mach-O to PE/COFF
"$(MTOC)" -subsystem $(MODULE_TYPE) $(MTOC_FLAGS) ${src} $(DEBUG_DIR)(+)$(MODULE_NAME).pecoff
# create symbol file for GDB debug
-$(DSYMUTIL) ${src}
"$(GENFW)" -e $(MODULE_TYPE) -o ${dst} $(DEBUG_DIR)(+)$(MODULE_NAME).pecoff $(GENFW_FLAGS)
- $(CP) ${dst} $(OUTPUT_DIR)
- $(CP) ${dst} $(BIN_DIR)(+)$(MODULE_NAME_GUID).efi
+ $(CP) $(DEBUG_DIR)(+)$(MODULE_NAME).efi $(OUTPUT_DIR)
+ $(CP) $(DEBUG_DIR)(+)$(MODULE_NAME).efi $(BIN_DIR)(+)$(MODULE_NAME_GUID).efi
-$(CP) $(DEBUG_DIR)(+)*.map $(OUTPUT_DIR)
[Dependency-Expression-File]
<InputFile>
?.dxs, ?.Dxs, ?.DXS
diff --git a/BaseTools/Source/C/GenFfs/GenFfs.c b/BaseTools/Source/C/GenFfs/GenFfs.c
index 02c7ac30f0..ea36b22bef 100644
--- a/BaseTools/Source/C/GenFfs/GenFfs.c
+++ b/BaseTools/Source/C/GenFfs/GenFfs.c
@@ -17,10 +17,14 @@ WITHOUT WARRANTIES OR REPRESENTATIONS OF ANY KIND, EITHER EXPRESS OR IMPLIED.
#include <io.h>
#include <sys/types.h>
#include <sys/stat.h>
#endif
+#ifdef __GNUC__
+#include <unistd.h>
+#endif
+
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <Common/UefiBaseTypes.h>
@@ -158,10 +162,12 @@ Returns:
FileAlign points to file alignment, which only support\n\
the following align: 1,2,4,8,16,128,512,1K,4K,32K,64K\n\
128K,256K,512K,1M,2M,4M,8M,16M\n");
fprintf (stdout, " -i SectionFile, --sectionfile SectionFile\n\
Section file will be contained in this FFS file.\n");
+ fprintf (stdout, " -oi SectionFile, --optionalsectionfile SectionFile\n\
+ If the Section file exists, it will be contained in this FFS file, otherwise, it will be ignored.\n");
fprintf (stdout, " -n SectionAlign, --sectionalign SectionAlign\n\
SectionAlign points to section alignment, which support\n\
the alignment scope 0~16M. If SectionAlign is specified\n\
as 0, tool get alignment value from SectionFile. It is\n\
specified together with sectionfile to point its\n\
@@ -734,19 +740,26 @@ Returns:
argc -= 2;
argv += 2;
continue;
}
- if ((stricmp (argv[0], "-i") == 0) || (stricmp (argv[0], "--sectionfile") == 0)) {
+ if ((stricmp (argv[0], "-oi") == 0) || (stricmp (argv[0], "--optionalsectionfile") == 0) || (stricmp (argv[0], "-i") == 0) || (stricmp (argv[0], "--sectionfile") == 0)) {
//
// Get Input file name and its alignment
//
if (argv[1] == NULL || argv[1][0] == '-') {
Error (NULL, 0, 1003, "Invalid option value", "input section file is missing for -i option");
goto Finish;
}
-
+ if ((stricmp (argv[0], "-oi") == 0) || (stricmp (argv[0], "--optionalsectionfile") == 0) ){
+ if (-1 == access(argv[1] , 0)){
+ Warning(NULL, 0, 0001, "File is not found.", argv[1]);
+ argc -= 2;
+ argv += 2;
+ continue;
+ }
+ }
//
// Allocate Input file name buffer and its alignment buffer.
//
if ((InputFileNum == 0) && (InputFileName == NULL)) {
InputFileName = (CHAR8 **) malloc (MAXIMUM_INPUT_FILE_NUM * sizeof (CHAR8 *));
diff --git a/BaseTools/Source/Python/AutoGen/GenMake.py b/BaseTools/Source/Python/AutoGen/GenMake.py
index b441817b52..2374bcf550 100644
--- a/BaseTools/Source/Python/AutoGen/GenMake.py
+++ b/BaseTools/Source/Python/AutoGen/GenMake.py
@@ -714,11 +714,11 @@ cleanlib:
FfsCmdList = Cmd[0]
for index, Str in enumerate(FfsCmdList):
if '-o' == Str:
OutputFile = FfsCmdList[index + 1]
- if '-i' == Str:
+ if '-i' == Str or "-oi" == Str:
if DepsFileList == []:
DepsFileList = [FfsCmdList[index + 1]]
else:
DepsFileList.append(FfsCmdList[index + 1])
DepsFileString = ' '.join(DepsFileList).strip()
diff --git a/BaseTools/Source/Python/GenFds/GenFdsGlobalVariable.py b/BaseTools/Source/Python/GenFds/GenFdsGlobalVariable.py
index 028bcc480c..b42829b698 100644
--- a/BaseTools/Source/Python/GenFds/GenFdsGlobalVariable.py
+++ b/BaseTools/Source/Python/GenFds/GenFdsGlobalVariable.py
@@ -33,10 +33,11 @@ from Common.ToolDefClassObject import ToolDefClassObject, ToolDefDict
from AutoGen.BuildEngine import BuildRule
import Common.DataType as DataType
from Common.Misc import PathClass
from Common.LongFilePathSupport import OpenLongFilePath as open
from Common.MultipleWorkspace import MultipleWorkspace as mws
+import Common.GlobalData as GlobalData
## Global variables
#
#
class GenFdsGlobalVariable:
@@ -499,10 +500,14 @@ class GenFdsGlobalVariable:
Cmd += ("-o", Output)
Cmd += Input
SaveFileOnChange(CommandFile, ' '.join(Cmd), False)
if IsMakefile:
+ if GlobalData.gGlobalDefines.get("FAMILY") == "MSFT":
+ Cmd = ['if', 'exist', Input[0]] + Cmd
+ else:
+ Cmd = ['test', '-e', Input[0], "&&"] + Cmd
if ' '.join(Cmd).strip() not in GenFdsGlobalVariable.SecCmdList:
GenFdsGlobalVariable.SecCmdList.append(' '.join(Cmd).strip())
elif GenFdsGlobalVariable.NeedsUpdate(Output, list(Input) + [CommandFile]):
GenFdsGlobalVariable.DebugLogger(EdkLogger.DEBUG_5, "%s needs update because of newer %s" % (Output, Input))
GenFdsGlobalVariable.CallExternalTool(Cmd, "Failed to generate section")
@@ -540,11 +545,14 @@ class GenFdsGlobalVariable:
Align = mFfsValidAlign[index + 1]
Cmd += ("-a", Align)
Cmd += ("-o", Output)
for I in range(0, len(Input)):
- Cmd += ("-i", Input[I])
+ if MakefilePath:
+ Cmd += ("-oi", Input[I])
+ else:
+ Cmd += ("-i", Input[I])
if SectionAlign and SectionAlign[I]:
Cmd += ("-n", SectionAlign[I])
CommandFile = Output + '.txt'
SaveFileOnChange(CommandFile, ' '.join(Cmd), False)
diff --git a/BaseTools/Source/Python/Workspace/DscBuildData.py b/BaseTools/Source/Python/Workspace/DscBuildData.py
index 129c0c950b..ac95c7df17 100644
--- a/BaseTools/Source/Python/Workspace/DscBuildData.py
+++ b/BaseTools/Source/Python/Workspace/DscBuildData.py
@@ -318,10 +318,13 @@ class DscBuildData(PlatformBuildClassObject):
## Get architecture
@property
def Arch(self):
return self._Arch
+ @property
+ def Dir(self):
+ return self.MetaFile.Dir
## Retrieve all information in [Defines] section
#
# (Retrieving all [Defines] information in one-shot is just to save time.)
#
diff --git a/BaseTools/Source/Python/build/build.py b/BaseTools/Source/Python/build/build.py
index de641fb452..41d405e6ff 100644
--- a/BaseTools/Source/Python/build/build.py
+++ b/BaseTools/Source/Python/build/build.py
@@ -1667,11 +1667,11 @@ class Build():
self.Progress.Stop("done!")
# Add ffs build to makefile
CmdListDict = {}
if GlobalData.gEnableGenfdsMultiThread and self.Fdf:
- CmdListDict = self._GenFfsCmd()
+ CmdListDict = self._GenFfsCmd(Wa.ArchList)
for Arch in Wa.ArchList:
GlobalData.gGlobalDefines['ARCH'] = Arch
Pa = PlatformAutoGen(Wa, self.PlatformFile, BuildTarget, ToolChain, Arch)
for Module in Pa.Platform.Modules:
@@ -1760,11 +1760,11 @@ class Build():
self.LoadFixAddress = Wa.Platform.LoadFixAddress
Wa.CreateMakeFile(False)
# Add ffs build to makefile
CmdListDict = None
if GlobalData.gEnableGenfdsMultiThread and self.Fdf:
- CmdListDict = self._GenFfsCmd()
+ CmdListDict = self._GenFfsCmd(Wa.ArchList)
self.Progress.Stop("done!")
MaList = []
ExitFlag = threading.Event()
ExitFlag.clear()
self.AutoGenTime += int(round((time.time() - WorkspaceAutoGenTime)))
@@ -1879,15 +1879,15 @@ class Build():
#
# Save MAP buffer into MAP file.
#
self._SaveMapFile (MapBuffer, Wa)
- def _GenFfsCmd(self):
+ def _GenFfsCmd(self,ArchList):
# convert dictionary of Cmd:(Inf,Arch)
# to a new dictionary of (Inf,Arch):Cmd,Cmd,Cmd...
CmdSetDict = defaultdict(set)
- GenFfsDict = GenFds.GenFfsMakefile('', GlobalData.gFdfParser, self, self.ArchList, GlobalData)
+ GenFfsDict = GenFds.GenFfsMakefile('', GlobalData.gFdfParser, self, ArchList, GlobalData)
for Cmd in GenFfsDict:
tmpInf, tmpArch = GenFfsDict[Cmd]
CmdSetDict[tmpInf, tmpArch].add(Cmd)
return CmdSetDict
@@ -1927,11 +1927,11 @@ class Build():
Wa.CreateMakeFile(False)
# Add ffs build to makefile
CmdListDict = None
if GlobalData.gEnableGenfdsMultiThread and self.Fdf:
- CmdListDict = self._GenFfsCmd()
+ CmdListDict = self._GenFfsCmd(Wa.ArchList)
# multi-thread exit flag
ExitFlag = threading.Event()
ExitFlag.clear()
self.AutoGenTime += int(round((time.time() - WorkspaceAutoGenTime)))
--
2.20.1.windows.1
^ permalink raw reply related [flat|nested] 7+ messages in thread
* Re: [Patch] BaseTools: Fixed issue in MultiThread Genfds function
2019-03-29 11:06 [Patch] BaseTools: Fixed issue in MultiThread Genfds function Feng, Bob C
@ 2019-03-29 11:41 ` Gao, Liming
2019-03-29 12:00 ` Feng, Bob C
0 siblings, 1 reply; 7+ messages in thread
From: Gao, Liming @ 2019-03-29 11:41 UTC (permalink / raw)
To: Feng, Bob C, edk2-devel@lists.01.org
Bob:
Could you list the more information on why update build_rule.txt?
> -----Original Message-----
> From: Feng, Bob C
> Sent: Friday, March 29, 2019 7:07 PM
> To: edk2-devel@lists.01.org
> Cc: Feng, Bob C <bob.c.feng@intel.com>; Gao, Liming <liming.gao@intel.com>
> Subject: [Patch] BaseTools: Fixed issue in MultiThread Genfds function
>
> https://bugzilla.tianocore.org/show_bug.cgi?id=1450
> In the Multiple thread Genfds feature, build tool generates
> GenSec, GenFFS command in Makefile.
>
> The Non-Hii Driver does not generate .offset file for uni string offset,
> but the build tool has not knowledge about this in autogen phase. So
> in this patch, I add a check in Makefile for GenSec command. If the GenSec
> input file does not exist, the GenSec will not be called. And if GenSec
> command is not called, its output file, which is also the input file of
> GenFfs command, will also not exist.So for GenFfs command,
> I add a new command parameter -oi which means
> the input file is an optional input file which would not exist. so
> that I can generate GenFfs command with "-oi" parameter in Makefile.
>
> Contributed-under: TianoCore Contribution Agreement 1.1
> Signed-off-by: Bob Feng <bob.c.feng@intel.com>
> Cc: Liming Gao <liming.gao@intel.com>
> ---
> BaseTools/Conf/build_rule.template | 14 +++++++-------
> BaseTools/Source/C/GenFfs/GenFfs.c | 17 +++++++++++++++--
> BaseTools/Source/Python/AutoGen/GenMake.py | 2 +-
> .../Python/GenFds/GenFdsGlobalVariable.py | 10 +++++++++-
> .../Source/Python/Workspace/DscBuildData.py | 3 +++
> BaseTools/Source/Python/build/build.py | 10 +++++-----
> 6 files changed, 40 insertions(+), 16 deletions(-)
>
> diff --git a/BaseTools/Conf/build_rule.template b/BaseTools/Conf/build_rule.template
> index e56b1d9c59..8cd7d0e445 100755
> --- a/BaseTools/Conf/build_rule.template
> +++ b/BaseTools/Conf/build_rule.template
> @@ -357,16 +357,16 @@
> [Dynamic-Library-File]
> <InputFile>
> ?.dll
>
> <OutputFile>
> - $(DEBUG_DIR)(+)$(MODULE_NAME).efi
> + $(DEBUG_DIR)(+)$(MODULE_NAME).efi $(OUTPUT_DIR)(+)$(MODULE_NAME).efi
>
> <Command.MSFT, Command.INTEL, Command.RVCT>
> "$(GENFW)" -e $(MODULE_TYPE) -o ${dst} ${src} $(GENFW_FLAGS)
> - $(CP) ${dst} $(OUTPUT_DIR)
> - $(CP) ${dst} $(BIN_DIR)(+)$(MODULE_NAME_GUID).efi
> + $(CP) $(DEBUG_DIR)(+)$(MODULE_NAME).efi $(OUTPUT_DIR)
> + $(CP) $(DEBUG_DIR)(+)$(MODULE_NAME).efi $(BIN_DIR)(+)$(MODULE_NAME_GUID).efi
> -$(CP) $(DEBUG_DIR)(+)*.map $(OUTPUT_DIR)
> -$(CP) $(DEBUG_DIR)(+)*.pdb $(OUTPUT_DIR)
> <Command.GCC>
> $(CP) ${src} $(DEBUG_DIR)(+)$(MODULE_NAME).debug
> $(OBJCOPY) --strip-unneeded -R .eh_frame ${src}
> @@ -376,22 +376,22 @@
> #
> -$(OBJCOPY) $(OBJCOPY_ADDDEBUGFLAG) ${src}
> -$(CP) $(DEBUG_DIR)(+)$(MODULE_NAME).debug $(BIN_DIR)(+)$(MODULE_NAME_GUID).debug
>
> "$(GENFW)" -e $(MODULE_TYPE) -o ${dst} ${src} $(GENFW_FLAGS)
> - $(CP) ${dst} $(OUTPUT_DIR)
> - $(CP) ${dst} $(BIN_DIR)(+)$(MODULE_NAME_GUID).efi
> + $(CP) $(DEBUG_DIR)(+)$(MODULE_NAME).efi $(OUTPUT_DIR)
> + $(CP) $(DEBUG_DIR)(+)$(MODULE_NAME).efi $(BIN_DIR)(+)$(MODULE_NAME_GUID).efi
> -$(CP) $(DEBUG_DIR)(+)*.map $(OUTPUT_DIR)
>
> <Command.XCODE>
> # tool to convert Mach-O to PE/COFF
> "$(MTOC)" -subsystem $(MODULE_TYPE) $(MTOC_FLAGS) ${src} $(DEBUG_DIR)(+)$(MODULE_NAME).pecoff
> # create symbol file for GDB debug
> -$(DSYMUTIL) ${src}
> "$(GENFW)" -e $(MODULE_TYPE) -o ${dst} $(DEBUG_DIR)(+)$(MODULE_NAME).pecoff $(GENFW_FLAGS)
> - $(CP) ${dst} $(OUTPUT_DIR)
> - $(CP) ${dst} $(BIN_DIR)(+)$(MODULE_NAME_GUID).efi
> + $(CP) $(DEBUG_DIR)(+)$(MODULE_NAME).efi $(OUTPUT_DIR)
> + $(CP) $(DEBUG_DIR)(+)$(MODULE_NAME).efi $(BIN_DIR)(+)$(MODULE_NAME_GUID).efi
> -$(CP) $(DEBUG_DIR)(+)*.map $(OUTPUT_DIR)
>
> [Dependency-Expression-File]
> <InputFile>
> ?.dxs, ?.Dxs, ?.DXS
> diff --git a/BaseTools/Source/C/GenFfs/GenFfs.c b/BaseTools/Source/C/GenFfs/GenFfs.c
> index 02c7ac30f0..ea36b22bef 100644
> --- a/BaseTools/Source/C/GenFfs/GenFfs.c
> +++ b/BaseTools/Source/C/GenFfs/GenFfs.c
> @@ -17,10 +17,14 @@ WITHOUT WARRANTIES OR REPRESENTATIONS OF ANY KIND, EITHER EXPRESS OR IMPLIED.
> #include <io.h>
> #include <sys/types.h>
> #include <sys/stat.h>
> #endif
>
> +#ifdef __GNUC__
> +#include <unistd.h>
> +#endif
> +
> #include <stdio.h>
> #include <stdlib.h>
> #include <string.h>
>
> #include <Common/UefiBaseTypes.h>
> @@ -158,10 +162,12 @@ Returns:
> FileAlign points to file alignment, which only support\n\
> the following align: 1,2,4,8,16,128,512,1K,4K,32K,64K\n\
> 128K,256K,512K,1M,2M,4M,8M,16M\n");
> fprintf (stdout, " -i SectionFile, --sectionfile SectionFile\n\
> Section file will be contained in this FFS file.\n");
> + fprintf (stdout, " -oi SectionFile, --optionalsectionfile SectionFile\n\
> + If the Section file exists, it will be contained in this FFS file, otherwise, it will be ignored.\n");
> fprintf (stdout, " -n SectionAlign, --sectionalign SectionAlign\n\
> SectionAlign points to section alignment, which support\n\
> the alignment scope 0~16M. If SectionAlign is specified\n\
> as 0, tool get alignment value from SectionFile. It is\n\
> specified together with sectionfile to point its\n\
> @@ -734,19 +740,26 @@ Returns:
> argc -= 2;
> argv += 2;
> continue;
> }
>
> - if ((stricmp (argv[0], "-i") == 0) || (stricmp (argv[0], "--sectionfile") == 0)) {
> + if ((stricmp (argv[0], "-oi") == 0) || (stricmp (argv[0], "--optionalsectionfile") == 0) || (stricmp (argv[0], "-i") == 0) || (stricmp (argv[0],
> "--sectionfile") == 0)) {
> //
> // Get Input file name and its alignment
> //
> if (argv[1] == NULL || argv[1][0] == '-') {
> Error (NULL, 0, 1003, "Invalid option value", "input section file is missing for -i option");
> goto Finish;
> }
> -
> + if ((stricmp (argv[0], "-oi") == 0) || (stricmp (argv[0], "--optionalsectionfile") == 0) ){
> + if (-1 == access(argv[1] , 0)){
> + Warning(NULL, 0, 0001, "File is not found.", argv[1]);
> + argc -= 2;
> + argv += 2;
> + continue;
> + }
> + }
> //
> // Allocate Input file name buffer and its alignment buffer.
> //
> if ((InputFileNum == 0) && (InputFileName == NULL)) {
> InputFileName = (CHAR8 **) malloc (MAXIMUM_INPUT_FILE_NUM * sizeof (CHAR8 *));
> diff --git a/BaseTools/Source/Python/AutoGen/GenMake.py b/BaseTools/Source/Python/AutoGen/GenMake.py
> index b441817b52..2374bcf550 100644
> --- a/BaseTools/Source/Python/AutoGen/GenMake.py
> +++ b/BaseTools/Source/Python/AutoGen/GenMake.py
> @@ -714,11 +714,11 @@ cleanlib:
>
> FfsCmdList = Cmd[0]
> for index, Str in enumerate(FfsCmdList):
> if '-o' == Str:
> OutputFile = FfsCmdList[index + 1]
> - if '-i' == Str:
> + if '-i' == Str or "-oi" == Str:
> if DepsFileList == []:
> DepsFileList = [FfsCmdList[index + 1]]
> else:
> DepsFileList.append(FfsCmdList[index + 1])
> DepsFileString = ' '.join(DepsFileList).strip()
> diff --git a/BaseTools/Source/Python/GenFds/GenFdsGlobalVariable.py b/BaseTools/Source/Python/GenFds/GenFdsGlobalVariable.py
> index 028bcc480c..b42829b698 100644
> --- a/BaseTools/Source/Python/GenFds/GenFdsGlobalVariable.py
> +++ b/BaseTools/Source/Python/GenFds/GenFdsGlobalVariable.py
> @@ -33,10 +33,11 @@ from Common.ToolDefClassObject import ToolDefClassObject, ToolDefDict
> from AutoGen.BuildEngine import BuildRule
> import Common.DataType as DataType
> from Common.Misc import PathClass
> from Common.LongFilePathSupport import OpenLongFilePath as open
> from Common.MultipleWorkspace import MultipleWorkspace as mws
> +import Common.GlobalData as GlobalData
>
> ## Global variables
> #
> #
> class GenFdsGlobalVariable:
> @@ -499,10 +500,14 @@ class GenFdsGlobalVariable:
> Cmd += ("-o", Output)
> Cmd += Input
>
> SaveFileOnChange(CommandFile, ' '.join(Cmd), False)
> if IsMakefile:
> + if GlobalData.gGlobalDefines.get("FAMILY") == "MSFT":
> + Cmd = ['if', 'exist', Input[0]] + Cmd
> + else:
> + Cmd = ['test', '-e', Input[0], "&&"] + Cmd
> if ' '.join(Cmd).strip() not in GenFdsGlobalVariable.SecCmdList:
> GenFdsGlobalVariable.SecCmdList.append(' '.join(Cmd).strip())
> elif GenFdsGlobalVariable.NeedsUpdate(Output, list(Input) + [CommandFile]):
> GenFdsGlobalVariable.DebugLogger(EdkLogger.DEBUG_5, "%s needs update because of newer %s" % (Output, Input))
> GenFdsGlobalVariable.CallExternalTool(Cmd, "Failed to generate section")
> @@ -540,11 +545,14 @@ class GenFdsGlobalVariable:
> Align = mFfsValidAlign[index + 1]
> Cmd += ("-a", Align)
>
> Cmd += ("-o", Output)
> for I in range(0, len(Input)):
> - Cmd += ("-i", Input[I])
> + if MakefilePath:
> + Cmd += ("-oi", Input[I])
> + else:
> + Cmd += ("-i", Input[I])
> if SectionAlign and SectionAlign[I]:
> Cmd += ("-n", SectionAlign[I])
>
> CommandFile = Output + '.txt'
> SaveFileOnChange(CommandFile, ' '.join(Cmd), False)
> diff --git a/BaseTools/Source/Python/Workspace/DscBuildData.py b/BaseTools/Source/Python/Workspace/DscBuildData.py
> index 129c0c950b..ac95c7df17 100644
> --- a/BaseTools/Source/Python/Workspace/DscBuildData.py
> +++ b/BaseTools/Source/Python/Workspace/DscBuildData.py
> @@ -318,10 +318,13 @@ class DscBuildData(PlatformBuildClassObject):
>
> ## Get architecture
> @property
> def Arch(self):
> return self._Arch
> + @property
> + def Dir(self):
> + return self.MetaFile.Dir
>
> ## Retrieve all information in [Defines] section
> #
> # (Retrieving all [Defines] information in one-shot is just to save time.)
> #
> diff --git a/BaseTools/Source/Python/build/build.py b/BaseTools/Source/Python/build/build.py
> index de641fb452..41d405e6ff 100644
> --- a/BaseTools/Source/Python/build/build.py
> +++ b/BaseTools/Source/Python/build/build.py
> @@ -1667,11 +1667,11 @@ class Build():
> self.Progress.Stop("done!")
>
> # Add ffs build to makefile
> CmdListDict = {}
> if GlobalData.gEnableGenfdsMultiThread and self.Fdf:
> - CmdListDict = self._GenFfsCmd()
> + CmdListDict = self._GenFfsCmd(Wa.ArchList)
>
> for Arch in Wa.ArchList:
> GlobalData.gGlobalDefines['ARCH'] = Arch
> Pa = PlatformAutoGen(Wa, self.PlatformFile, BuildTarget, ToolChain, Arch)
> for Module in Pa.Platform.Modules:
> @@ -1760,11 +1760,11 @@ class Build():
> self.LoadFixAddress = Wa.Platform.LoadFixAddress
> Wa.CreateMakeFile(False)
> # Add ffs build to makefile
> CmdListDict = None
> if GlobalData.gEnableGenfdsMultiThread and self.Fdf:
> - CmdListDict = self._GenFfsCmd()
> + CmdListDict = self._GenFfsCmd(Wa.ArchList)
> self.Progress.Stop("done!")
> MaList = []
> ExitFlag = threading.Event()
> ExitFlag.clear()
> self.AutoGenTime += int(round((time.time() - WorkspaceAutoGenTime)))
> @@ -1879,15 +1879,15 @@ class Build():
> #
> # Save MAP buffer into MAP file.
> #
> self._SaveMapFile (MapBuffer, Wa)
>
> - def _GenFfsCmd(self):
> + def _GenFfsCmd(self,ArchList):
> # convert dictionary of Cmd:(Inf,Arch)
> # to a new dictionary of (Inf,Arch):Cmd,Cmd,Cmd...
> CmdSetDict = defaultdict(set)
> - GenFfsDict = GenFds.GenFfsMakefile('', GlobalData.gFdfParser, self, self.ArchList, GlobalData)
> + GenFfsDict = GenFds.GenFfsMakefile('', GlobalData.gFdfParser, self, ArchList, GlobalData)
> for Cmd in GenFfsDict:
> tmpInf, tmpArch = GenFfsDict[Cmd]
> CmdSetDict[tmpInf, tmpArch].add(Cmd)
> return CmdSetDict
>
> @@ -1927,11 +1927,11 @@ class Build():
> Wa.CreateMakeFile(False)
>
> # Add ffs build to makefile
> CmdListDict = None
> if GlobalData.gEnableGenfdsMultiThread and self.Fdf:
> - CmdListDict = self._GenFfsCmd()
> + CmdListDict = self._GenFfsCmd(Wa.ArchList)
>
> # multi-thread exit flag
> ExitFlag = threading.Event()
> ExitFlag.clear()
> self.AutoGenTime += int(round((time.time() - WorkspaceAutoGenTime)))
> --
> 2.20.1.windows.1
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [Patch] BaseTools: Fixed issue in MultiThread Genfds function
2019-03-29 11:41 ` Gao, Liming
@ 2019-03-29 12:00 ` Feng, Bob C
2019-03-29 12:33 ` Gao, Liming
0 siblings, 1 reply; 7+ messages in thread
From: Feng, Bob C @ 2019-03-29 12:00 UTC (permalink / raw)
To: Gao, Liming, edk2-devel@lists.01.org
That's to fix the issue that make will fail if enable multiple thread genfds.
For example, If enable Multiple thread Genfds, there will be GenSec command in the Makefile like below.
$(FFS_OUTPUT_DIR)\$(MODULE_GUID)SEC2.1.1.1.pe32 : $(OUTPUT_DIR)\LogoDxe.efi
GenSec -s EFI_SECTION_PE32 -o $(FFS_OUTPUT_DIR)\$(MODULE_GUID)SEC2.1.1.1.pe32 $(OUTPUT_DIR)\LogoDxe.efi
The $(OUTPUT_DIR)\LogoDxe.efi is the dependency, but there is no make rule for $(OUTPUT_DIR)\LogoDxe.efi generated in the Makefile. Make program does not know how to make $(OUTPUT_DIR)\LogoDxe.efi. Make fails for this case.
In the build_rule.txt, the $(OUTPUT_DIR)\ )(+)$(MODULE_NAME).efi is generated by the command $(CP) $(DEBUG_DIR)(+)$(MODULE_NAME).efi $(OUTPUT_DIR) under Dynamic-Library-File section, so $(OUTPUT_DIR)\ )(+)$(MODULE_NAME).efi should also be a output for Dynamic-Library-File
Thanks,
Bob
-----Original Message-----
From: Gao, Liming
Sent: Friday, March 29, 2019 7:42 PM
To: Feng, Bob C <bob.c.feng@intel.com>; edk2-devel@lists.01.org
Subject: RE: [Patch] BaseTools: Fixed issue in MultiThread Genfds function
Bob:
Could you list the more information on why update build_rule.txt?
> -----Original Message-----
> From: Feng, Bob C
> Sent: Friday, March 29, 2019 7:07 PM
> To: edk2-devel@lists.01.org
> Cc: Feng, Bob C <bob.c.feng@intel.com>; Gao, Liming
> <liming.gao@intel.com>
> Subject: [Patch] BaseTools: Fixed issue in MultiThread Genfds function
>
> https://bugzilla.tianocore.org/show_bug.cgi?id=1450
> In the Multiple thread Genfds feature, build tool generates GenSec,
> GenFFS command in Makefile.
>
> The Non-Hii Driver does not generate .offset file for uni string
> offset, but the build tool has not knowledge about this in autogen
> phase. So in this patch, I add a check in Makefile for GenSec command.
> If the GenSec input file does not exist, the GenSec will not be
> called. And if GenSec command is not called, its output file, which is
> also the input file of GenFfs command, will also not exist.So for
> GenFfs command, I add a new command parameter -oi which means the
> input file is an optional input file which would not exist. so that I
> can generate GenFfs command with "-oi" parameter in Makefile.
>
> Contributed-under: TianoCore Contribution Agreement 1.1
> Signed-off-by: Bob Feng <bob.c.feng@intel.com>
> Cc: Liming Gao <liming.gao@intel.com>
> ---
> BaseTools/Conf/build_rule.template | 14 +++++++-------
> BaseTools/Source/C/GenFfs/GenFfs.c | 17 +++++++++++++++--
> BaseTools/Source/Python/AutoGen/GenMake.py | 2 +-
> .../Python/GenFds/GenFdsGlobalVariable.py | 10 +++++++++-
> .../Source/Python/Workspace/DscBuildData.py | 3 +++
> BaseTools/Source/Python/build/build.py | 10 +++++-----
> 6 files changed, 40 insertions(+), 16 deletions(-)
>
> diff --git a/BaseTools/Conf/build_rule.template
> b/BaseTools/Conf/build_rule.template
> index e56b1d9c59..8cd7d0e445 100755
> --- a/BaseTools/Conf/build_rule.template
> +++ b/BaseTools/Conf/build_rule.template
> @@ -357,16 +357,16 @@
> [Dynamic-Library-File]
> <InputFile>
> ?.dll
>
> <OutputFile>
> - $(DEBUG_DIR)(+)$(MODULE_NAME).efi
> + $(DEBUG_DIR)(+)$(MODULE_NAME).efi
> + $(OUTPUT_DIR)(+)$(MODULE_NAME).efi
>
> <Command.MSFT, Command.INTEL, Command.RVCT>
> "$(GENFW)" -e $(MODULE_TYPE) -o ${dst} ${src} $(GENFW_FLAGS)
> - $(CP) ${dst} $(OUTPUT_DIR)
> - $(CP) ${dst} $(BIN_DIR)(+)$(MODULE_NAME_GUID).efi
> + $(CP) $(DEBUG_DIR)(+)$(MODULE_NAME).efi $(OUTPUT_DIR)
> + $(CP) $(DEBUG_DIR)(+)$(MODULE_NAME).efi
> + $(BIN_DIR)(+)$(MODULE_NAME_GUID).efi
> -$(CP) $(DEBUG_DIR)(+)*.map $(OUTPUT_DIR)
> -$(CP) $(DEBUG_DIR)(+)*.pdb $(OUTPUT_DIR)
> <Command.GCC>
> $(CP) ${src} $(DEBUG_DIR)(+)$(MODULE_NAME).debug
> $(OBJCOPY) --strip-unneeded -R .eh_frame ${src} @@ -376,22
> +376,22 @@
> #
> -$(OBJCOPY) $(OBJCOPY_ADDDEBUGFLAG) ${src}
> -$(CP) $(DEBUG_DIR)(+)$(MODULE_NAME).debug
> $(BIN_DIR)(+)$(MODULE_NAME_GUID).debug
>
> "$(GENFW)" -e $(MODULE_TYPE) -o ${dst} ${src} $(GENFW_FLAGS)
> - $(CP) ${dst} $(OUTPUT_DIR)
> - $(CP) ${dst} $(BIN_DIR)(+)$(MODULE_NAME_GUID).efi
> + $(CP) $(DEBUG_DIR)(+)$(MODULE_NAME).efi $(OUTPUT_DIR)
> + $(CP) $(DEBUG_DIR)(+)$(MODULE_NAME).efi
> + $(BIN_DIR)(+)$(MODULE_NAME_GUID).efi
> -$(CP) $(DEBUG_DIR)(+)*.map $(OUTPUT_DIR)
>
> <Command.XCODE>
> # tool to convert Mach-O to PE/COFF
> "$(MTOC)" -subsystem $(MODULE_TYPE) $(MTOC_FLAGS) ${src} $(DEBUG_DIR)(+)$(MODULE_NAME).pecoff
> # create symbol file for GDB debug
> -$(DSYMUTIL) ${src}
> "$(GENFW)" -e $(MODULE_TYPE) -o ${dst} $(DEBUG_DIR)(+)$(MODULE_NAME).pecoff $(GENFW_FLAGS)
> - $(CP) ${dst} $(OUTPUT_DIR)
> - $(CP) ${dst} $(BIN_DIR)(+)$(MODULE_NAME_GUID).efi
> + $(CP) $(DEBUG_DIR)(+)$(MODULE_NAME).efi $(OUTPUT_DIR)
> + $(CP) $(DEBUG_DIR)(+)$(MODULE_NAME).efi
> + $(BIN_DIR)(+)$(MODULE_NAME_GUID).efi
> -$(CP) $(DEBUG_DIR)(+)*.map $(OUTPUT_DIR)
>
> [Dependency-Expression-File]
> <InputFile>
> ?.dxs, ?.Dxs, ?.DXS
> diff --git a/BaseTools/Source/C/GenFfs/GenFfs.c
> b/BaseTools/Source/C/GenFfs/GenFfs.c
> index 02c7ac30f0..ea36b22bef 100644
> --- a/BaseTools/Source/C/GenFfs/GenFfs.c
> +++ b/BaseTools/Source/C/GenFfs/GenFfs.c
> @@ -17,10 +17,14 @@ WITHOUT WARRANTIES OR REPRESENTATIONS OF ANY KIND, EITHER EXPRESS OR IMPLIED.
> #include <io.h>
> #include <sys/types.h>
> #include <sys/stat.h>
> #endif
>
> +#ifdef __GNUC__
> +#include <unistd.h>
> +#endif
> +
> #include <stdio.h>
> #include <stdlib.h>
> #include <string.h>
>
> #include <Common/UefiBaseTypes.h>
> @@ -158,10 +162,12 @@ Returns:
> FileAlign points to file alignment, which only support\n\
> the following align: 1,2,4,8,16,128,512,1K,4K,32K,64K\n\
> 128K,256K,512K,1M,2M,4M,8M,16M\n");
> fprintf (stdout, " -i SectionFile, --sectionfile SectionFile\n\
> Section file will be contained in this FFS
> file.\n");
> + fprintf (stdout, " -oi SectionFile, --optionalsectionfile SectionFile\n\
> + If the Section file exists, it will be
> + contained in this FFS file, otherwise, it will be ignored.\n");
> fprintf (stdout, " -n SectionAlign, --sectionalign SectionAlign\n\
> SectionAlign points to section alignment, which support\n\
> the alignment scope 0~16M. If SectionAlign is specified\n\
> as 0, tool get alignment value from SectionFile. It is\n\
> specified together with sectionfile to point
> its\n\ @@ -734,19 +740,26 @@ Returns:
> argc -= 2;
> argv += 2;
> continue;
> }
>
> - if ((stricmp (argv[0], "-i") == 0) || (stricmp (argv[0], "--sectionfile") == 0)) {
> + if ((stricmp (argv[0], "-oi") == 0) || (stricmp (argv[0],
> + "--optionalsectionfile") == 0) || (stricmp (argv[0], "-i") == 0) ||
> + (stricmp (argv[0],
> "--sectionfile") == 0)) {
> //
> // Get Input file name and its alignment
> //
> if (argv[1] == NULL || argv[1][0] == '-') {
> Error (NULL, 0, 1003, "Invalid option value", "input section file is missing for -i option");
> goto Finish;
> }
> -
> + if ((stricmp (argv[0], "-oi") == 0) || (stricmp (argv[0], "--optionalsectionfile") == 0) ){
> + if (-1 == access(argv[1] , 0)){
> + Warning(NULL, 0, 0001, "File is not found.", argv[1]);
> + argc -= 2;
> + argv += 2;
> + continue;
> + }
> + }
> //
> // Allocate Input file name buffer and its alignment buffer.
> //
> if ((InputFileNum == 0) && (InputFileName == NULL)) {
> InputFileName = (CHAR8 **) malloc (MAXIMUM_INPUT_FILE_NUM *
> sizeof (CHAR8 *)); diff --git
> a/BaseTools/Source/Python/AutoGen/GenMake.py
> b/BaseTools/Source/Python/AutoGen/GenMake.py
> index b441817b52..2374bcf550 100644
> --- a/BaseTools/Source/Python/AutoGen/GenMake.py
> +++ b/BaseTools/Source/Python/AutoGen/GenMake.py
> @@ -714,11 +714,11 @@ cleanlib:
>
> FfsCmdList = Cmd[0]
> for index, Str in enumerate(FfsCmdList):
> if '-o' == Str:
> OutputFile = FfsCmdList[index + 1]
> - if '-i' == Str:
> + if '-i' == Str or "-oi" == Str:
> if DepsFileList == []:
> DepsFileList = [FfsCmdList[index + 1]]
> else:
> DepsFileList.append(FfsCmdList[index + 1])
> DepsFileString = ' '.join(DepsFileList).strip() diff
> --git a/BaseTools/Source/Python/GenFds/GenFdsGlobalVariable.py
> b/BaseTools/Source/Python/GenFds/GenFdsGlobalVariable.py
> index 028bcc480c..b42829b698 100644
> --- a/BaseTools/Source/Python/GenFds/GenFdsGlobalVariable.py
> +++ b/BaseTools/Source/Python/GenFds/GenFdsGlobalVariable.py
> @@ -33,10 +33,11 @@ from Common.ToolDefClassObject import
> ToolDefClassObject, ToolDefDict from AutoGen.BuildEngine import
> BuildRule import Common.DataType as DataType from Common.Misc import
> PathClass from Common.LongFilePathSupport import OpenLongFilePath as
> open from Common.MultipleWorkspace import MultipleWorkspace as mws
> +import Common.GlobalData as GlobalData
>
> ## Global variables
> #
> #
> class GenFdsGlobalVariable:
> @@ -499,10 +500,14 @@ class GenFdsGlobalVariable:
> Cmd += ("-o", Output)
> Cmd += Input
>
> SaveFileOnChange(CommandFile, ' '.join(Cmd), False)
> if IsMakefile:
> + if GlobalData.gGlobalDefines.get("FAMILY") == "MSFT":
> + Cmd = ['if', 'exist', Input[0]] + Cmd
> + else:
> + Cmd = ['test', '-e', Input[0], "&&"] + Cmd
> if ' '.join(Cmd).strip() not in GenFdsGlobalVariable.SecCmdList:
> GenFdsGlobalVariable.SecCmdList.append(' '.join(Cmd).strip())
> elif GenFdsGlobalVariable.NeedsUpdate(Output, list(Input) + [CommandFile]):
> GenFdsGlobalVariable.DebugLogger(EdkLogger.DEBUG_5, "%s needs update because of newer %s" % (Output, Input))
> GenFdsGlobalVariable.CallExternalTool(Cmd, "Failed to
> generate section") @@ -540,11 +545,14 @@ class GenFdsGlobalVariable:
> Align = mFfsValidAlign[index + 1]
> Cmd += ("-a", Align)
>
> Cmd += ("-o", Output)
> for I in range(0, len(Input)):
> - Cmd += ("-i", Input[I])
> + if MakefilePath:
> + Cmd += ("-oi", Input[I])
> + else:
> + Cmd += ("-i", Input[I])
> if SectionAlign and SectionAlign[I]:
> Cmd += ("-n", SectionAlign[I])
>
> CommandFile = Output + '.txt'
> SaveFileOnChange(CommandFile, ' '.join(Cmd), False) diff
> --git a/BaseTools/Source/Python/Workspace/DscBuildData.py
> b/BaseTools/Source/Python/Workspace/DscBuildData.py
> index 129c0c950b..ac95c7df17 100644
> --- a/BaseTools/Source/Python/Workspace/DscBuildData.py
> +++ b/BaseTools/Source/Python/Workspace/DscBuildData.py
> @@ -318,10 +318,13 @@ class DscBuildData(PlatformBuildClassObject):
>
> ## Get architecture
> @property
> def Arch(self):
> return self._Arch
> + @property
> + def Dir(self):
> + return self.MetaFile.Dir
>
> ## Retrieve all information in [Defines] section
> #
> # (Retrieving all [Defines] information in one-shot is just to save time.)
> #
> diff --git a/BaseTools/Source/Python/build/build.py
> b/BaseTools/Source/Python/build/build.py
> index de641fb452..41d405e6ff 100644
> --- a/BaseTools/Source/Python/build/build.py
> +++ b/BaseTools/Source/Python/build/build.py
> @@ -1667,11 +1667,11 @@ class Build():
> self.Progress.Stop("done!")
>
> # Add ffs build to makefile
> CmdListDict = {}
> if GlobalData.gEnableGenfdsMultiThread and self.Fdf:
> - CmdListDict = self._GenFfsCmd()
> + CmdListDict = self._GenFfsCmd(Wa.ArchList)
>
> for Arch in Wa.ArchList:
> GlobalData.gGlobalDefines['ARCH'] = Arch
> Pa = PlatformAutoGen(Wa, self.PlatformFile, BuildTarget, ToolChain, Arch)
> for Module in Pa.Platform.Modules:
> @@ -1760,11 +1760,11 @@ class Build():
> self.LoadFixAddress = Wa.Platform.LoadFixAddress
> Wa.CreateMakeFile(False)
> # Add ffs build to makefile
> CmdListDict = None
> if GlobalData.gEnableGenfdsMultiThread and self.Fdf:
> - CmdListDict = self._GenFfsCmd()
> + CmdListDict = self._GenFfsCmd(Wa.ArchList)
> self.Progress.Stop("done!")
> MaList = []
> ExitFlag = threading.Event()
> ExitFlag.clear()
> self.AutoGenTime += int(round((time.time() -
> WorkspaceAutoGenTime))) @@ -1879,15 +1879,15 @@ class Build():
> #
> # Save MAP buffer into MAP file.
> #
> self._SaveMapFile (MapBuffer, Wa)
>
> - def _GenFfsCmd(self):
> + def _GenFfsCmd(self,ArchList):
> # convert dictionary of Cmd:(Inf,Arch)
> # to a new dictionary of (Inf,Arch):Cmd,Cmd,Cmd...
> CmdSetDict = defaultdict(set)
> - GenFfsDict = GenFds.GenFfsMakefile('', GlobalData.gFdfParser, self, self.ArchList, GlobalData)
> + GenFfsDict = GenFds.GenFfsMakefile('', GlobalData.gFdfParser,
> + self, ArchList, GlobalData)
> for Cmd in GenFfsDict:
> tmpInf, tmpArch = GenFfsDict[Cmd]
> CmdSetDict[tmpInf, tmpArch].add(Cmd)
> return CmdSetDict
>
> @@ -1927,11 +1927,11 @@ class Build():
> Wa.CreateMakeFile(False)
>
> # Add ffs build to makefile
> CmdListDict = None
> if GlobalData.gEnableGenfdsMultiThread and self.Fdf:
> - CmdListDict = self._GenFfsCmd()
> + CmdListDict = self._GenFfsCmd(Wa.ArchList)
>
> # multi-thread exit flag
> ExitFlag = threading.Event()
> ExitFlag.clear()
> self.AutoGenTime += int(round((time.time() -
> WorkspaceAutoGenTime)))
> --
> 2.20.1.windows.1
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [Patch] BaseTools: Fixed issue in MultiThread Genfds function
2019-03-29 12:00 ` Feng, Bob C
@ 2019-03-29 12:33 ` Gao, Liming
2019-03-29 12:39 ` Feng, Bob C
0 siblings, 1 reply; 7+ messages in thread
From: Gao, Liming @ 2019-03-29 12:33 UTC (permalink / raw)
To: Feng, Bob C, edk2-devel@lists.01.org
Bob:
Could you update the rule to depend on EFI image from DEBUG_DIR? I would like to remove the additional copy from DEBUG dir to OUTPUT dir and save the disk and performance.
$(FFS_OUTPUT_DIR)\$(MODULE_GUID)SEC2.1.1.1.pe32 : $(DEBUG_DIR)\LogoDxe.efi
GenSec -s EFI_SECTION_PE32 -o $(FFS_OUTPUT_DIR)\$(MODULE_GUID)SEC2.1.1.1.pe32 $(DEBUG_DIR)\LogoDxe.efi
Thanks
Liming
> -----Original Message-----
> From: Feng, Bob C
> Sent: Friday, March 29, 2019 8:00 PM
> To: Gao, Liming <liming.gao@intel.com>; edk2-devel@lists.01.org
> Subject: RE: [Patch] BaseTools: Fixed issue in MultiThread Genfds function
>
> That's to fix the issue that make will fail if enable multiple thread genfds.
>
> For example, If enable Multiple thread Genfds, there will be GenSec command in the Makefile like below.
>
> $(FFS_OUTPUT_DIR)\$(MODULE_GUID)SEC2.1.1.1.pe32 : $(OUTPUT_DIR)\LogoDxe.efi
> GenSec -s EFI_SECTION_PE32 -o $(FFS_OUTPUT_DIR)\$(MODULE_GUID)SEC2.1.1.1.pe32 $(OUTPUT_DIR)\LogoDxe.efi
>
>
> The $(OUTPUT_DIR)\LogoDxe.efi is the dependency, but there is no make rule for $(OUTPUT_DIR)\LogoDxe.efi generated in the Makefile.
> Make program does not know how to make $(OUTPUT_DIR)\LogoDxe.efi. Make fails for this case.
>
> In the build_rule.txt, the $(OUTPUT_DIR)\ )(+)$(MODULE_NAME).efi is generated by the command $(CP)
> $(DEBUG_DIR)(+)$(MODULE_NAME).efi $(OUTPUT_DIR) under Dynamic-Library-File section, so $(OUTPUT_DIR)\ )(+)$(MODULE_NAME).efi
> should also be a output for Dynamic-Library-File
>
> Thanks,
> Bob
>
> -----Original Message-----
> From: Gao, Liming
> Sent: Friday, March 29, 2019 7:42 PM
> To: Feng, Bob C <bob.c.feng@intel.com>; edk2-devel@lists.01.org
> Subject: RE: [Patch] BaseTools: Fixed issue in MultiThread Genfds function
>
> Bob:
> Could you list the more information on why update build_rule.txt?
>
> > -----Original Message-----
> > From: Feng, Bob C
> > Sent: Friday, March 29, 2019 7:07 PM
> > To: edk2-devel@lists.01.org
> > Cc: Feng, Bob C <bob.c.feng@intel.com>; Gao, Liming
> > <liming.gao@intel.com>
> > Subject: [Patch] BaseTools: Fixed issue in MultiThread Genfds function
> >
> > https://bugzilla.tianocore.org/show_bug.cgi?id=1450
> > In the Multiple thread Genfds feature, build tool generates GenSec,
> > GenFFS command in Makefile.
> >
> > The Non-Hii Driver does not generate .offset file for uni string
> > offset, but the build tool has not knowledge about this in autogen
> > phase. So in this patch, I add a check in Makefile for GenSec command.
> > If the GenSec input file does not exist, the GenSec will not be
> > called. And if GenSec command is not called, its output file, which is
> > also the input file of GenFfs command, will also not exist.So for
> > GenFfs command, I add a new command parameter -oi which means the
> > input file is an optional input file which would not exist. so that I
> > can generate GenFfs command with "-oi" parameter in Makefile.
> >
> > Contributed-under: TianoCore Contribution Agreement 1.1
> > Signed-off-by: Bob Feng <bob.c.feng@intel.com>
> > Cc: Liming Gao <liming.gao@intel.com>
> > ---
> > BaseTools/Conf/build_rule.template | 14 +++++++-------
> > BaseTools/Source/C/GenFfs/GenFfs.c | 17 +++++++++++++++--
> > BaseTools/Source/Python/AutoGen/GenMake.py | 2 +-
> > .../Python/GenFds/GenFdsGlobalVariable.py | 10 +++++++++-
> > .../Source/Python/Workspace/DscBuildData.py | 3 +++
> > BaseTools/Source/Python/build/build.py | 10 +++++-----
> > 6 files changed, 40 insertions(+), 16 deletions(-)
> >
> > diff --git a/BaseTools/Conf/build_rule.template
> > b/BaseTools/Conf/build_rule.template
> > index e56b1d9c59..8cd7d0e445 100755
> > --- a/BaseTools/Conf/build_rule.template
> > +++ b/BaseTools/Conf/build_rule.template
> > @@ -357,16 +357,16 @@
> > [Dynamic-Library-File]
> > <InputFile>
> > ?.dll
> >
> > <OutputFile>
> > - $(DEBUG_DIR)(+)$(MODULE_NAME).efi
> > + $(DEBUG_DIR)(+)$(MODULE_NAME).efi
> > + $(OUTPUT_DIR)(+)$(MODULE_NAME).efi
> >
> > <Command.MSFT, Command.INTEL, Command.RVCT>
> > "$(GENFW)" -e $(MODULE_TYPE) -o ${dst} ${src} $(GENFW_FLAGS)
> > - $(CP) ${dst} $(OUTPUT_DIR)
> > - $(CP) ${dst} $(BIN_DIR)(+)$(MODULE_NAME_GUID).efi
> > + $(CP) $(DEBUG_DIR)(+)$(MODULE_NAME).efi $(OUTPUT_DIR)
> > + $(CP) $(DEBUG_DIR)(+)$(MODULE_NAME).efi
> > + $(BIN_DIR)(+)$(MODULE_NAME_GUID).efi
> > -$(CP) $(DEBUG_DIR)(+)*.map $(OUTPUT_DIR)
> > -$(CP) $(DEBUG_DIR)(+)*.pdb $(OUTPUT_DIR)
> > <Command.GCC>
> > $(CP) ${src} $(DEBUG_DIR)(+)$(MODULE_NAME).debug
> > $(OBJCOPY) --strip-unneeded -R .eh_frame ${src} @@ -376,22
> > +376,22 @@
> > #
> > -$(OBJCOPY) $(OBJCOPY_ADDDEBUGFLAG) ${src}
> > -$(CP) $(DEBUG_DIR)(+)$(MODULE_NAME).debug
> > $(BIN_DIR)(+)$(MODULE_NAME_GUID).debug
> >
> > "$(GENFW)" -e $(MODULE_TYPE) -o ${dst} ${src} $(GENFW_FLAGS)
> > - $(CP) ${dst} $(OUTPUT_DIR)
> > - $(CP) ${dst} $(BIN_DIR)(+)$(MODULE_NAME_GUID).efi
> > + $(CP) $(DEBUG_DIR)(+)$(MODULE_NAME).efi $(OUTPUT_DIR)
> > + $(CP) $(DEBUG_DIR)(+)$(MODULE_NAME).efi
> > + $(BIN_DIR)(+)$(MODULE_NAME_GUID).efi
> > -$(CP) $(DEBUG_DIR)(+)*.map $(OUTPUT_DIR)
> >
> > <Command.XCODE>
> > # tool to convert Mach-O to PE/COFF
> > "$(MTOC)" -subsystem $(MODULE_TYPE) $(MTOC_FLAGS) ${src} $(DEBUG_DIR)(+)$(MODULE_NAME).pecoff
> > # create symbol file for GDB debug
> > -$(DSYMUTIL) ${src}
> > "$(GENFW)" -e $(MODULE_TYPE) -o ${dst} $(DEBUG_DIR)(+)$(MODULE_NAME).pecoff $(GENFW_FLAGS)
> > - $(CP) ${dst} $(OUTPUT_DIR)
> > - $(CP) ${dst} $(BIN_DIR)(+)$(MODULE_NAME_GUID).efi
> > + $(CP) $(DEBUG_DIR)(+)$(MODULE_NAME).efi $(OUTPUT_DIR)
> > + $(CP) $(DEBUG_DIR)(+)$(MODULE_NAME).efi
> > + $(BIN_DIR)(+)$(MODULE_NAME_GUID).efi
> > -$(CP) $(DEBUG_DIR)(+)*.map $(OUTPUT_DIR)
> >
> > [Dependency-Expression-File]
> > <InputFile>
> > ?.dxs, ?.Dxs, ?.DXS
> > diff --git a/BaseTools/Source/C/GenFfs/GenFfs.c
> > b/BaseTools/Source/C/GenFfs/GenFfs.c
> > index 02c7ac30f0..ea36b22bef 100644
> > --- a/BaseTools/Source/C/GenFfs/GenFfs.c
> > +++ b/BaseTools/Source/C/GenFfs/GenFfs.c
> > @@ -17,10 +17,14 @@ WITHOUT WARRANTIES OR REPRESENTATIONS OF ANY KIND, EITHER EXPRESS OR IMPLIED.
> > #include <io.h>
> > #include <sys/types.h>
> > #include <sys/stat.h>
> > #endif
> >
> > +#ifdef __GNUC__
> > +#include <unistd.h>
> > +#endif
> > +
> > #include <stdio.h>
> > #include <stdlib.h>
> > #include <string.h>
> >
> > #include <Common/UefiBaseTypes.h>
> > @@ -158,10 +162,12 @@ Returns:
> > FileAlign points to file alignment, which only support\n\
> > the following align: 1,2,4,8,16,128,512,1K,4K,32K,64K\n\
> > 128K,256K,512K,1M,2M,4M,8M,16M\n");
> > fprintf (stdout, " -i SectionFile, --sectionfile SectionFile\n\
> > Section file will be contained in this FFS
> > file.\n");
> > + fprintf (stdout, " -oi SectionFile, --optionalsectionfile SectionFile\n\
> > + If the Section file exists, it will be
> > + contained in this FFS file, otherwise, it will be ignored.\n");
> > fprintf (stdout, " -n SectionAlign, --sectionalign SectionAlign\n\
> > SectionAlign points to section alignment, which support\n\
> > the alignment scope 0~16M. If SectionAlign is specified\n\
> > as 0, tool get alignment value from SectionFile. It is\n\
> > specified together with sectionfile to point
> > its\n\ @@ -734,19 +740,26 @@ Returns:
> > argc -= 2;
> > argv += 2;
> > continue;
> > }
> >
> > - if ((stricmp (argv[0], "-i") == 0) || (stricmp (argv[0], "--sectionfile") == 0)) {
> > + if ((stricmp (argv[0], "-oi") == 0) || (stricmp (argv[0],
> > + "--optionalsectionfile") == 0) || (stricmp (argv[0], "-i") == 0) ||
> > + (stricmp (argv[0],
> > "--sectionfile") == 0)) {
> > //
> > // Get Input file name and its alignment
> > //
> > if (argv[1] == NULL || argv[1][0] == '-') {
> > Error (NULL, 0, 1003, "Invalid option value", "input section file is missing for -i option");
> > goto Finish;
> > }
> > -
> > + if ((stricmp (argv[0], "-oi") == 0) || (stricmp (argv[0], "--optionalsectionfile") == 0) ){
> > + if (-1 == access(argv[1] , 0)){
> > + Warning(NULL, 0, 0001, "File is not found.", argv[1]);
> > + argc -= 2;
> > + argv += 2;
> > + continue;
> > + }
> > + }
> > //
> > // Allocate Input file name buffer and its alignment buffer.
> > //
> > if ((InputFileNum == 0) && (InputFileName == NULL)) {
> > InputFileName = (CHAR8 **) malloc (MAXIMUM_INPUT_FILE_NUM *
> > sizeof (CHAR8 *)); diff --git
> > a/BaseTools/Source/Python/AutoGen/GenMake.py
> > b/BaseTools/Source/Python/AutoGen/GenMake.py
> > index b441817b52..2374bcf550 100644
> > --- a/BaseTools/Source/Python/AutoGen/GenMake.py
> > +++ b/BaseTools/Source/Python/AutoGen/GenMake.py
> > @@ -714,11 +714,11 @@ cleanlib:
> >
> > FfsCmdList = Cmd[0]
> > for index, Str in enumerate(FfsCmdList):
> > if '-o' == Str:
> > OutputFile = FfsCmdList[index + 1]
> > - if '-i' == Str:
> > + if '-i' == Str or "-oi" == Str:
> > if DepsFileList == []:
> > DepsFileList = [FfsCmdList[index + 1]]
> > else:
> > DepsFileList.append(FfsCmdList[index + 1])
> > DepsFileString = ' '.join(DepsFileList).strip() diff
> > --git a/BaseTools/Source/Python/GenFds/GenFdsGlobalVariable.py
> > b/BaseTools/Source/Python/GenFds/GenFdsGlobalVariable.py
> > index 028bcc480c..b42829b698 100644
> > --- a/BaseTools/Source/Python/GenFds/GenFdsGlobalVariable.py
> > +++ b/BaseTools/Source/Python/GenFds/GenFdsGlobalVariable.py
> > @@ -33,10 +33,11 @@ from Common.ToolDefClassObject import
> > ToolDefClassObject, ToolDefDict from AutoGen.BuildEngine import
> > BuildRule import Common.DataType as DataType from Common.Misc import
> > PathClass from Common.LongFilePathSupport import OpenLongFilePath as
> > open from Common.MultipleWorkspace import MultipleWorkspace as mws
> > +import Common.GlobalData as GlobalData
> >
> > ## Global variables
> > #
> > #
> > class GenFdsGlobalVariable:
> > @@ -499,10 +500,14 @@ class GenFdsGlobalVariable:
> > Cmd += ("-o", Output)
> > Cmd += Input
> >
> > SaveFileOnChange(CommandFile, ' '.join(Cmd), False)
> > if IsMakefile:
> > + if GlobalData.gGlobalDefines.get("FAMILY") == "MSFT":
> > + Cmd = ['if', 'exist', Input[0]] + Cmd
> > + else:
> > + Cmd = ['test', '-e', Input[0], "&&"] + Cmd
> > if ' '.join(Cmd).strip() not in GenFdsGlobalVariable.SecCmdList:
> > GenFdsGlobalVariable.SecCmdList.append(' '.join(Cmd).strip())
> > elif GenFdsGlobalVariable.NeedsUpdate(Output, list(Input) + [CommandFile]):
> > GenFdsGlobalVariable.DebugLogger(EdkLogger.DEBUG_5, "%s needs update because of newer %s" % (Output,
> Input))
> > GenFdsGlobalVariable.CallExternalTool(Cmd, "Failed to
> > generate section") @@ -540,11 +545,14 @@ class GenFdsGlobalVariable:
> > Align = mFfsValidAlign[index + 1]
> > Cmd += ("-a", Align)
> >
> > Cmd += ("-o", Output)
> > for I in range(0, len(Input)):
> > - Cmd += ("-i", Input[I])
> > + if MakefilePath:
> > + Cmd += ("-oi", Input[I])
> > + else:
> > + Cmd += ("-i", Input[I])
> > if SectionAlign and SectionAlign[I]:
> > Cmd += ("-n", SectionAlign[I])
> >
> > CommandFile = Output + '.txt'
> > SaveFileOnChange(CommandFile, ' '.join(Cmd), False) diff
> > --git a/BaseTools/Source/Python/Workspace/DscBuildData.py
> > b/BaseTools/Source/Python/Workspace/DscBuildData.py
> > index 129c0c950b..ac95c7df17 100644
> > --- a/BaseTools/Source/Python/Workspace/DscBuildData.py
> > +++ b/BaseTools/Source/Python/Workspace/DscBuildData.py
> > @@ -318,10 +318,13 @@ class DscBuildData(PlatformBuildClassObject):
> >
> > ## Get architecture
> > @property
> > def Arch(self):
> > return self._Arch
> > + @property
> > + def Dir(self):
> > + return self.MetaFile.Dir
> >
> > ## Retrieve all information in [Defines] section
> > #
> > # (Retrieving all [Defines] information in one-shot is just to save time.)
> > #
> > diff --git a/BaseTools/Source/Python/build/build.py
> > b/BaseTools/Source/Python/build/build.py
> > index de641fb452..41d405e6ff 100644
> > --- a/BaseTools/Source/Python/build/build.py
> > +++ b/BaseTools/Source/Python/build/build.py
> > @@ -1667,11 +1667,11 @@ class Build():
> > self.Progress.Stop("done!")
> >
> > # Add ffs build to makefile
> > CmdListDict = {}
> > if GlobalData.gEnableGenfdsMultiThread and self.Fdf:
> > - CmdListDict = self._GenFfsCmd()
> > + CmdListDict = self._GenFfsCmd(Wa.ArchList)
> >
> > for Arch in Wa.ArchList:
> > GlobalData.gGlobalDefines['ARCH'] = Arch
> > Pa = PlatformAutoGen(Wa, self.PlatformFile, BuildTarget, ToolChain, Arch)
> > for Module in Pa.Platform.Modules:
> > @@ -1760,11 +1760,11 @@ class Build():
> > self.LoadFixAddress = Wa.Platform.LoadFixAddress
> > Wa.CreateMakeFile(False)
> > # Add ffs build to makefile
> > CmdListDict = None
> > if GlobalData.gEnableGenfdsMultiThread and self.Fdf:
> > - CmdListDict = self._GenFfsCmd()
> > + CmdListDict = self._GenFfsCmd(Wa.ArchList)
> > self.Progress.Stop("done!")
> > MaList = []
> > ExitFlag = threading.Event()
> > ExitFlag.clear()
> > self.AutoGenTime += int(round((time.time() -
> > WorkspaceAutoGenTime))) @@ -1879,15 +1879,15 @@ class Build():
> > #
> > # Save MAP buffer into MAP file.
> > #
> > self._SaveMapFile (MapBuffer, Wa)
> >
> > - def _GenFfsCmd(self):
> > + def _GenFfsCmd(self,ArchList):
> > # convert dictionary of Cmd:(Inf,Arch)
> > # to a new dictionary of (Inf,Arch):Cmd,Cmd,Cmd...
> > CmdSetDict = defaultdict(set)
> > - GenFfsDict = GenFds.GenFfsMakefile('', GlobalData.gFdfParser, self, self.ArchList, GlobalData)
> > + GenFfsDict = GenFds.GenFfsMakefile('', GlobalData.gFdfParser,
> > + self, ArchList, GlobalData)
> > for Cmd in GenFfsDict:
> > tmpInf, tmpArch = GenFfsDict[Cmd]
> > CmdSetDict[tmpInf, tmpArch].add(Cmd)
> > return CmdSetDict
> >
> > @@ -1927,11 +1927,11 @@ class Build():
> > Wa.CreateMakeFile(False)
> >
> > # Add ffs build to makefile
> > CmdListDict = None
> > if GlobalData.gEnableGenfdsMultiThread and self.Fdf:
> > - CmdListDict = self._GenFfsCmd()
> > + CmdListDict = self._GenFfsCmd(Wa.ArchList)
> >
> > # multi-thread exit flag
> > ExitFlag = threading.Event()
> > ExitFlag.clear()
> > self.AutoGenTime += int(round((time.time() -
> > WorkspaceAutoGenTime)))
> > --
> > 2.20.1.windows.1
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [Patch] BaseTools: Fixed issue in MultiThread Genfds function
2019-03-29 12:33 ` Gao, Liming
@ 2019-03-29 12:39 ` Feng, Bob C
2019-03-29 14:09 ` Gao, Liming
0 siblings, 1 reply; 7+ messages in thread
From: Feng, Bob C @ 2019-03-29 12:39 UTC (permalink / raw)
To: Gao, Liming, edk2-devel@lists.01.org
That will impact the original GenFds since the command is generate by reusing the GenFds functions. What about enter a new BZ to do that change?
Thanks,
Bob
-----Original Message-----
From: Gao, Liming
Sent: Friday, March 29, 2019 8:33 PM
To: Feng, Bob C <bob.c.feng@intel.com>; edk2-devel@lists.01.org
Subject: RE: [Patch] BaseTools: Fixed issue in MultiThread Genfds function
Bob:
Could you update the rule to depend on EFI image from DEBUG_DIR? I would like to remove the additional copy from DEBUG dir to OUTPUT dir and save the disk and performance.
$(FFS_OUTPUT_DIR)\$(MODULE_GUID)SEC2.1.1.1.pe32 : $(DEBUG_DIR)\LogoDxe.efi
GenSec -s EFI_SECTION_PE32 -o $(FFS_OUTPUT_DIR)\$(MODULE_GUID)SEC2.1.1.1.pe32 $(DEBUG_DIR)\LogoDxe.efi
Thanks
Liming
> -----Original Message-----
> From: Feng, Bob C
> Sent: Friday, March 29, 2019 8:00 PM
> To: Gao, Liming <liming.gao@intel.com>; edk2-devel@lists.01.org
> Subject: RE: [Patch] BaseTools: Fixed issue in MultiThread Genfds
> function
>
> That's to fix the issue that make will fail if enable multiple thread genfds.
>
> For example, If enable Multiple thread Genfds, there will be GenSec command in the Makefile like below.
>
> $(FFS_OUTPUT_DIR)\$(MODULE_GUID)SEC2.1.1.1.pe32 : $(OUTPUT_DIR)\LogoDxe.efi
> GenSec -s EFI_SECTION_PE32 -o
> $(FFS_OUTPUT_DIR)\$(MODULE_GUID)SEC2.1.1.1.pe32
> $(OUTPUT_DIR)\LogoDxe.efi
>
>
> The $(OUTPUT_DIR)\LogoDxe.efi is the dependency, but there is no make rule for $(OUTPUT_DIR)\LogoDxe.efi generated in the Makefile.
> Make program does not know how to make $(OUTPUT_DIR)\LogoDxe.efi. Make fails for this case.
>
> In the build_rule.txt, the $(OUTPUT_DIR)\ )(+)$(MODULE_NAME).efi is generated by the command $(CP)
> $(DEBUG_DIR)(+)$(MODULE_NAME).efi $(OUTPUT_DIR) under
> Dynamic-Library-File section, so $(OUTPUT_DIR)\ )(+)$(MODULE_NAME).efi
> should also be a output for Dynamic-Library-File
>
> Thanks,
> Bob
>
> -----Original Message-----
> From: Gao, Liming
> Sent: Friday, March 29, 2019 7:42 PM
> To: Feng, Bob C <bob.c.feng@intel.com>; edk2-devel@lists.01.org
> Subject: RE: [Patch] BaseTools: Fixed issue in MultiThread Genfds
> function
>
> Bob:
> Could you list the more information on why update build_rule.txt?
>
> > -----Original Message-----
> > From: Feng, Bob C
> > Sent: Friday, March 29, 2019 7:07 PM
> > To: edk2-devel@lists.01.org
> > Cc: Feng, Bob C <bob.c.feng@intel.com>; Gao, Liming
> > <liming.gao@intel.com>
> > Subject: [Patch] BaseTools: Fixed issue in MultiThread Genfds
> > function
> >
> > https://bugzilla.tianocore.org/show_bug.cgi?id=1450
> > In the Multiple thread Genfds feature, build tool generates GenSec,
> > GenFFS command in Makefile.
> >
> > The Non-Hii Driver does not generate .offset file for uni string
> > offset, but the build tool has not knowledge about this in autogen
> > phase. So in this patch, I add a check in Makefile for GenSec command.
> > If the GenSec input file does not exist, the GenSec will not be
> > called. And if GenSec command is not called, its output file, which
> > is also the input file of GenFfs command, will also not exist.So for
> > GenFfs command, I add a new command parameter -oi which means the
> > input file is an optional input file which would not exist. so that
> > I can generate GenFfs command with "-oi" parameter in Makefile.
> >
> > Contributed-under: TianoCore Contribution Agreement 1.1
> > Signed-off-by: Bob Feng <bob.c.feng@intel.com>
> > Cc: Liming Gao <liming.gao@intel.com>
> > ---
> > BaseTools/Conf/build_rule.template | 14 +++++++-------
> > BaseTools/Source/C/GenFfs/GenFfs.c | 17 +++++++++++++++--
> > BaseTools/Source/Python/AutoGen/GenMake.py | 2 +-
> > .../Python/GenFds/GenFdsGlobalVariable.py | 10 +++++++++-
> > .../Source/Python/Workspace/DscBuildData.py | 3 +++
> > BaseTools/Source/Python/build/build.py | 10 +++++-----
> > 6 files changed, 40 insertions(+), 16 deletions(-)
> >
> > diff --git a/BaseTools/Conf/build_rule.template
> > b/BaseTools/Conf/build_rule.template
> > index e56b1d9c59..8cd7d0e445 100755
> > --- a/BaseTools/Conf/build_rule.template
> > +++ b/BaseTools/Conf/build_rule.template
> > @@ -357,16 +357,16 @@
> > [Dynamic-Library-File]
> > <InputFile>
> > ?.dll
> >
> > <OutputFile>
> > - $(DEBUG_DIR)(+)$(MODULE_NAME).efi
> > + $(DEBUG_DIR)(+)$(MODULE_NAME).efi
> > + $(OUTPUT_DIR)(+)$(MODULE_NAME).efi
> >
> > <Command.MSFT, Command.INTEL, Command.RVCT>
> > "$(GENFW)" -e $(MODULE_TYPE) -o ${dst} ${src} $(GENFW_FLAGS)
> > - $(CP) ${dst} $(OUTPUT_DIR)
> > - $(CP) ${dst} $(BIN_DIR)(+)$(MODULE_NAME_GUID).efi
> > + $(CP) $(DEBUG_DIR)(+)$(MODULE_NAME).efi $(OUTPUT_DIR)
> > + $(CP) $(DEBUG_DIR)(+)$(MODULE_NAME).efi
> > + $(BIN_DIR)(+)$(MODULE_NAME_GUID).efi
> > -$(CP) $(DEBUG_DIR)(+)*.map $(OUTPUT_DIR)
> > -$(CP) $(DEBUG_DIR)(+)*.pdb $(OUTPUT_DIR)
> > <Command.GCC>
> > $(CP) ${src} $(DEBUG_DIR)(+)$(MODULE_NAME).debug
> > $(OBJCOPY) --strip-unneeded -R .eh_frame ${src} @@ -376,22
> > +376,22 @@
> > #
> > -$(OBJCOPY) $(OBJCOPY_ADDDEBUGFLAG) ${src}
> > -$(CP) $(DEBUG_DIR)(+)$(MODULE_NAME).debug
> > $(BIN_DIR)(+)$(MODULE_NAME_GUID).debug
> >
> > "$(GENFW)" -e $(MODULE_TYPE) -o ${dst} ${src} $(GENFW_FLAGS)
> > - $(CP) ${dst} $(OUTPUT_DIR)
> > - $(CP) ${dst} $(BIN_DIR)(+)$(MODULE_NAME_GUID).efi
> > + $(CP) $(DEBUG_DIR)(+)$(MODULE_NAME).efi $(OUTPUT_DIR)
> > + $(CP) $(DEBUG_DIR)(+)$(MODULE_NAME).efi
> > + $(BIN_DIR)(+)$(MODULE_NAME_GUID).efi
> > -$(CP) $(DEBUG_DIR)(+)*.map $(OUTPUT_DIR)
> >
> > <Command.XCODE>
> > # tool to convert Mach-O to PE/COFF
> > "$(MTOC)" -subsystem $(MODULE_TYPE) $(MTOC_FLAGS) ${src} $(DEBUG_DIR)(+)$(MODULE_NAME).pecoff
> > # create symbol file for GDB debug
> > -$(DSYMUTIL) ${src}
> > "$(GENFW)" -e $(MODULE_TYPE) -o ${dst} $(DEBUG_DIR)(+)$(MODULE_NAME).pecoff $(GENFW_FLAGS)
> > - $(CP) ${dst} $(OUTPUT_DIR)
> > - $(CP) ${dst} $(BIN_DIR)(+)$(MODULE_NAME_GUID).efi
> > + $(CP) $(DEBUG_DIR)(+)$(MODULE_NAME).efi $(OUTPUT_DIR)
> > + $(CP) $(DEBUG_DIR)(+)$(MODULE_NAME).efi
> > + $(BIN_DIR)(+)$(MODULE_NAME_GUID).efi
> > -$(CP) $(DEBUG_DIR)(+)*.map $(OUTPUT_DIR)
> >
> > [Dependency-Expression-File]
> > <InputFile>
> > ?.dxs, ?.Dxs, ?.DXS
> > diff --git a/BaseTools/Source/C/GenFfs/GenFfs.c
> > b/BaseTools/Source/C/GenFfs/GenFfs.c
> > index 02c7ac30f0..ea36b22bef 100644
> > --- a/BaseTools/Source/C/GenFfs/GenFfs.c
> > +++ b/BaseTools/Source/C/GenFfs/GenFfs.c
> > @@ -17,10 +17,14 @@ WITHOUT WARRANTIES OR REPRESENTATIONS OF ANY KIND, EITHER EXPRESS OR IMPLIED.
> > #include <io.h>
> > #include <sys/types.h>
> > #include <sys/stat.h>
> > #endif
> >
> > +#ifdef __GNUC__
> > +#include <unistd.h>
> > +#endif
> > +
> > #include <stdio.h>
> > #include <stdlib.h>
> > #include <string.h>
> >
> > #include <Common/UefiBaseTypes.h>
> > @@ -158,10 +162,12 @@ Returns:
> > FileAlign points to file alignment, which only support\n\
> > the following align: 1,2,4,8,16,128,512,1K,4K,32K,64K\n\
> > 128K,256K,512K,1M,2M,4M,8M,16M\n");
> > fprintf (stdout, " -i SectionFile, --sectionfile SectionFile\n\
> > Section file will be contained in this FFS
> > file.\n");
> > + fprintf (stdout, " -oi SectionFile, --optionalsectionfile SectionFile\n\
> > + If the Section file exists, it will be
> > + contained in this FFS file, otherwise, it will be ignored.\n");
> > fprintf (stdout, " -n SectionAlign, --sectionalign SectionAlign\n\
> > SectionAlign points to section alignment, which support\n\
> > the alignment scope 0~16M. If SectionAlign is specified\n\
> > as 0, tool get alignment value from SectionFile. It is\n\
> > specified together with sectionfile to
> > point its\n\ @@ -734,19 +740,26 @@ Returns:
> > argc -= 2;
> > argv += 2;
> > continue;
> > }
> >
> > - if ((stricmp (argv[0], "-i") == 0) || (stricmp (argv[0], "--sectionfile") == 0)) {
> > + if ((stricmp (argv[0], "-oi") == 0) || (stricmp (argv[0],
> > + "--optionalsectionfile") == 0) || (stricmp (argv[0], "-i") == 0)
> > + || (stricmp (argv[0],
> > "--sectionfile") == 0)) {
> > //
> > // Get Input file name and its alignment
> > //
> > if (argv[1] == NULL || argv[1][0] == '-') {
> > Error (NULL, 0, 1003, "Invalid option value", "input section file is missing for -i option");
> > goto Finish;
> > }
> > -
> > + if ((stricmp (argv[0], "-oi") == 0) || (stricmp (argv[0], "--optionalsectionfile") == 0) ){
> > + if (-1 == access(argv[1] , 0)){
> > + Warning(NULL, 0, 0001, "File is not found.", argv[1]);
> > + argc -= 2;
> > + argv += 2;
> > + continue;
> > + }
> > + }
> > //
> > // Allocate Input file name buffer and its alignment buffer.
> > //
> > if ((InputFileNum == 0) && (InputFileName == NULL)) {
> > InputFileName = (CHAR8 **) malloc (MAXIMUM_INPUT_FILE_NUM *
> > sizeof (CHAR8 *)); diff --git
> > a/BaseTools/Source/Python/AutoGen/GenMake.py
> > b/BaseTools/Source/Python/AutoGen/GenMake.py
> > index b441817b52..2374bcf550 100644
> > --- a/BaseTools/Source/Python/AutoGen/GenMake.py
> > +++ b/BaseTools/Source/Python/AutoGen/GenMake.py
> > @@ -714,11 +714,11 @@ cleanlib:
> >
> > FfsCmdList = Cmd[0]
> > for index, Str in enumerate(FfsCmdList):
> > if '-o' == Str:
> > OutputFile = FfsCmdList[index + 1]
> > - if '-i' == Str:
> > + if '-i' == Str or "-oi" == Str:
> > if DepsFileList == []:
> > DepsFileList = [FfsCmdList[index + 1]]
> > else:
> > DepsFileList.append(FfsCmdList[index + 1])
> > DepsFileString = ' '.join(DepsFileList).strip() diff
> > --git a/BaseTools/Source/Python/GenFds/GenFdsGlobalVariable.py
> > b/BaseTools/Source/Python/GenFds/GenFdsGlobalVariable.py
> > index 028bcc480c..b42829b698 100644
> > --- a/BaseTools/Source/Python/GenFds/GenFdsGlobalVariable.py
> > +++ b/BaseTools/Source/Python/GenFds/GenFdsGlobalVariable.py
> > @@ -33,10 +33,11 @@ from Common.ToolDefClassObject import
> > ToolDefClassObject, ToolDefDict from AutoGen.BuildEngine import
> > BuildRule import Common.DataType as DataType from Common.Misc
> > import PathClass from Common.LongFilePathSupport import
> > OpenLongFilePath as open from Common.MultipleWorkspace import
> > MultipleWorkspace as mws
> > +import Common.GlobalData as GlobalData
> >
> > ## Global variables
> > #
> > #
> > class GenFdsGlobalVariable:
> > @@ -499,10 +500,14 @@ class GenFdsGlobalVariable:
> > Cmd += ("-o", Output)
> > Cmd += Input
> >
> > SaveFileOnChange(CommandFile, ' '.join(Cmd), False)
> > if IsMakefile:
> > + if GlobalData.gGlobalDefines.get("FAMILY") == "MSFT":
> > + Cmd = ['if', 'exist', Input[0]] + Cmd
> > + else:
> > + Cmd = ['test', '-e', Input[0], "&&"] + Cmd
> > if ' '.join(Cmd).strip() not in GenFdsGlobalVariable.SecCmdList:
> > GenFdsGlobalVariable.SecCmdList.append(' '.join(Cmd).strip())
> > elif GenFdsGlobalVariable.NeedsUpdate(Output, list(Input) + [CommandFile]):
> > GenFdsGlobalVariable.DebugLogger(EdkLogger.DEBUG_5,
> > "%s needs update because of newer %s" % (Output,
> Input))
> > GenFdsGlobalVariable.CallExternalTool(Cmd, "Failed
> > to generate section") @@ -540,11 +545,14 @@ class GenFdsGlobalVariable:
> > Align = mFfsValidAlign[index + 1]
> > Cmd += ("-a", Align)
> >
> > Cmd += ("-o", Output)
> > for I in range(0, len(Input)):
> > - Cmd += ("-i", Input[I])
> > + if MakefilePath:
> > + Cmd += ("-oi", Input[I])
> > + else:
> > + Cmd += ("-i", Input[I])
> > if SectionAlign and SectionAlign[I]:
> > Cmd += ("-n", SectionAlign[I])
> >
> > CommandFile = Output + '.txt'
> > SaveFileOnChange(CommandFile, ' '.join(Cmd), False) diff
> > --git a/BaseTools/Source/Python/Workspace/DscBuildData.py
> > b/BaseTools/Source/Python/Workspace/DscBuildData.py
> > index 129c0c950b..ac95c7df17 100644
> > --- a/BaseTools/Source/Python/Workspace/DscBuildData.py
> > +++ b/BaseTools/Source/Python/Workspace/DscBuildData.py
> > @@ -318,10 +318,13 @@ class DscBuildData(PlatformBuildClassObject):
> >
> > ## Get architecture
> > @property
> > def Arch(self):
> > return self._Arch
> > + @property
> > + def Dir(self):
> > + return self.MetaFile.Dir
> >
> > ## Retrieve all information in [Defines] section
> > #
> > # (Retrieving all [Defines] information in one-shot is just to save time.)
> > #
> > diff --git a/BaseTools/Source/Python/build/build.py
> > b/BaseTools/Source/Python/build/build.py
> > index de641fb452..41d405e6ff 100644
> > --- a/BaseTools/Source/Python/build/build.py
> > +++ b/BaseTools/Source/Python/build/build.py
> > @@ -1667,11 +1667,11 @@ class Build():
> > self.Progress.Stop("done!")
> >
> > # Add ffs build to makefile
> > CmdListDict = {}
> > if GlobalData.gEnableGenfdsMultiThread and self.Fdf:
> > - CmdListDict = self._GenFfsCmd()
> > + CmdListDict = self._GenFfsCmd(Wa.ArchList)
> >
> > for Arch in Wa.ArchList:
> > GlobalData.gGlobalDefines['ARCH'] = Arch
> > Pa = PlatformAutoGen(Wa, self.PlatformFile, BuildTarget, ToolChain, Arch)
> > for Module in Pa.Platform.Modules:
> > @@ -1760,11 +1760,11 @@ class Build():
> > self.LoadFixAddress = Wa.Platform.LoadFixAddress
> > Wa.CreateMakeFile(False)
> > # Add ffs build to makefile
> > CmdListDict = None
> > if GlobalData.gEnableGenfdsMultiThread and self.Fdf:
> > - CmdListDict = self._GenFfsCmd()
> > + CmdListDict = self._GenFfsCmd(Wa.ArchList)
> > self.Progress.Stop("done!")
> > MaList = []
> > ExitFlag = threading.Event()
> > ExitFlag.clear()
> > self.AutoGenTime += int(round((time.time() -
> > WorkspaceAutoGenTime))) @@ -1879,15 +1879,15 @@ class Build():
> > #
> > # Save MAP buffer into MAP file.
> > #
> > self._SaveMapFile (MapBuffer, Wa)
> >
> > - def _GenFfsCmd(self):
> > + def _GenFfsCmd(self,ArchList):
> > # convert dictionary of Cmd:(Inf,Arch)
> > # to a new dictionary of (Inf,Arch):Cmd,Cmd,Cmd...
> > CmdSetDict = defaultdict(set)
> > - GenFfsDict = GenFds.GenFfsMakefile('', GlobalData.gFdfParser, self, self.ArchList, GlobalData)
> > + GenFfsDict = GenFds.GenFfsMakefile('',
> > + GlobalData.gFdfParser, self, ArchList, GlobalData)
> > for Cmd in GenFfsDict:
> > tmpInf, tmpArch = GenFfsDict[Cmd]
> > CmdSetDict[tmpInf, tmpArch].add(Cmd)
> > return CmdSetDict
> >
> > @@ -1927,11 +1927,11 @@ class Build():
> > Wa.CreateMakeFile(False)
> >
> > # Add ffs build to makefile
> > CmdListDict = None
> > if GlobalData.gEnableGenfdsMultiThread and self.Fdf:
> > - CmdListDict = self._GenFfsCmd()
> > + CmdListDict = self._GenFfsCmd(Wa.ArchList)
> >
> > # multi-thread exit flag
> > ExitFlag = threading.Event()
> > ExitFlag.clear()
> > self.AutoGenTime += int(round((time.time() -
> > WorkspaceAutoGenTime)))
> > --
> > 2.20.1.windows.1
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [Patch] BaseTools: Fixed issue in MultiThread Genfds function
2019-03-29 12:39 ` Feng, Bob C
@ 2019-03-29 14:09 ` Gao, Liming
2019-04-03 2:15 ` Feng, Bob C
0 siblings, 1 reply; 7+ messages in thread
From: Gao, Liming @ 2019-03-29 14:09 UTC (permalink / raw)
To: Feng, Bob C, edk2-devel@lists.01.org
Yes. Please submit one BZ for it.
> -----Original Message-----
> From: Feng, Bob C
> Sent: Friday, March 29, 2019 8:40 PM
> To: Gao, Liming <liming.gao@intel.com>; edk2-devel@lists.01.org
> Subject: RE: [Patch] BaseTools: Fixed issue in MultiThread Genfds function
>
> That will impact the original GenFds since the command is generate by reusing the GenFds functions. What about enter a new BZ to do
> that change?
>
> Thanks,
> Bob
>
> -----Original Message-----
> From: Gao, Liming
> Sent: Friday, March 29, 2019 8:33 PM
> To: Feng, Bob C <bob.c.feng@intel.com>; edk2-devel@lists.01.org
> Subject: RE: [Patch] BaseTools: Fixed issue in MultiThread Genfds function
>
> Bob:
> Could you update the rule to depend on EFI image from DEBUG_DIR? I would like to remove the additional copy from DEBUG dir to
> OUTPUT dir and save the disk and performance.
>
> $(FFS_OUTPUT_DIR)\$(MODULE_GUID)SEC2.1.1.1.pe32 : $(DEBUG_DIR)\LogoDxe.efi
> GenSec -s EFI_SECTION_PE32 -o $(FFS_OUTPUT_DIR)\$(MODULE_GUID)SEC2.1.1.1.pe32 $(DEBUG_DIR)\LogoDxe.efi
>
> Thanks
> Liming
> > -----Original Message-----
> > From: Feng, Bob C
> > Sent: Friday, March 29, 2019 8:00 PM
> > To: Gao, Liming <liming.gao@intel.com>; edk2-devel@lists.01.org
> > Subject: RE: [Patch] BaseTools: Fixed issue in MultiThread Genfds
> > function
> >
> > That's to fix the issue that make will fail if enable multiple thread genfds.
> >
> > For example, If enable Multiple thread Genfds, there will be GenSec command in the Makefile like below.
> >
> > $(FFS_OUTPUT_DIR)\$(MODULE_GUID)SEC2.1.1.1.pe32 : $(OUTPUT_DIR)\LogoDxe.efi
> > GenSec -s EFI_SECTION_PE32 -o
> > $(FFS_OUTPUT_DIR)\$(MODULE_GUID)SEC2.1.1.1.pe32
> > $(OUTPUT_DIR)\LogoDxe.efi
> >
> >
> > The $(OUTPUT_DIR)\LogoDxe.efi is the dependency, but there is no make rule for $(OUTPUT_DIR)\LogoDxe.efi generated in the Makefile.
> > Make program does not know how to make $(OUTPUT_DIR)\LogoDxe.efi. Make fails for this case.
> >
> > In the build_rule.txt, the $(OUTPUT_DIR)\ )(+)$(MODULE_NAME).efi is generated by the command $(CP)
> > $(DEBUG_DIR)(+)$(MODULE_NAME).efi $(OUTPUT_DIR) under
> > Dynamic-Library-File section, so $(OUTPUT_DIR)\ )(+)$(MODULE_NAME).efi
> > should also be a output for Dynamic-Library-File
> >
> > Thanks,
> > Bob
> >
> > -----Original Message-----
> > From: Gao, Liming
> > Sent: Friday, March 29, 2019 7:42 PM
> > To: Feng, Bob C <bob.c.feng@intel.com>; edk2-devel@lists.01.org
> > Subject: RE: [Patch] BaseTools: Fixed issue in MultiThread Genfds
> > function
> >
> > Bob:
> > Could you list the more information on why update build_rule.txt?
> >
> > > -----Original Message-----
> > > From: Feng, Bob C
> > > Sent: Friday, March 29, 2019 7:07 PM
> > > To: edk2-devel@lists.01.org
> > > Cc: Feng, Bob C <bob.c.feng@intel.com>; Gao, Liming
> > > <liming.gao@intel.com>
> > > Subject: [Patch] BaseTools: Fixed issue in MultiThread Genfds
> > > function
> > >
> > > https://bugzilla.tianocore.org/show_bug.cgi?id=1450
> > > In the Multiple thread Genfds feature, build tool generates GenSec,
> > > GenFFS command in Makefile.
> > >
> > > The Non-Hii Driver does not generate .offset file for uni string
> > > offset, but the build tool has not knowledge about this in autogen
> > > phase. So in this patch, I add a check in Makefile for GenSec command.
> > > If the GenSec input file does not exist, the GenSec will not be
> > > called. And if GenSec command is not called, its output file, which
> > > is also the input file of GenFfs command, will also not exist.So for
> > > GenFfs command, I add a new command parameter -oi which means the
> > > input file is an optional input file which would not exist. so that
> > > I can generate GenFfs command with "-oi" parameter in Makefile.
> > >
> > > Contributed-under: TianoCore Contribution Agreement 1.1
> > > Signed-off-by: Bob Feng <bob.c.feng@intel.com>
> > > Cc: Liming Gao <liming.gao@intel.com>
> > > ---
> > > BaseTools/Conf/build_rule.template | 14 +++++++-------
> > > BaseTools/Source/C/GenFfs/GenFfs.c | 17 +++++++++++++++--
> > > BaseTools/Source/Python/AutoGen/GenMake.py | 2 +-
> > > .../Python/GenFds/GenFdsGlobalVariable.py | 10 +++++++++-
> > > .../Source/Python/Workspace/DscBuildData.py | 3 +++
> > > BaseTools/Source/Python/build/build.py | 10 +++++-----
> > > 6 files changed, 40 insertions(+), 16 deletions(-)
> > >
> > > diff --git a/BaseTools/Conf/build_rule.template
> > > b/BaseTools/Conf/build_rule.template
> > > index e56b1d9c59..8cd7d0e445 100755
> > > --- a/BaseTools/Conf/build_rule.template
> > > +++ b/BaseTools/Conf/build_rule.template
> > > @@ -357,16 +357,16 @@
> > > [Dynamic-Library-File]
> > > <InputFile>
> > > ?.dll
> > >
> > > <OutputFile>
> > > - $(DEBUG_DIR)(+)$(MODULE_NAME).efi
> > > + $(DEBUG_DIR)(+)$(MODULE_NAME).efi
> > > + $(OUTPUT_DIR)(+)$(MODULE_NAME).efi
> > >
> > > <Command.MSFT, Command.INTEL, Command.RVCT>
> > > "$(GENFW)" -e $(MODULE_TYPE) -o ${dst} ${src} $(GENFW_FLAGS)
> > > - $(CP) ${dst} $(OUTPUT_DIR)
> > > - $(CP) ${dst} $(BIN_DIR)(+)$(MODULE_NAME_GUID).efi
> > > + $(CP) $(DEBUG_DIR)(+)$(MODULE_NAME).efi $(OUTPUT_DIR)
> > > + $(CP) $(DEBUG_DIR)(+)$(MODULE_NAME).efi
> > > + $(BIN_DIR)(+)$(MODULE_NAME_GUID).efi
> > > -$(CP) $(DEBUG_DIR)(+)*.map $(OUTPUT_DIR)
> > > -$(CP) $(DEBUG_DIR)(+)*.pdb $(OUTPUT_DIR)
> > > <Command.GCC>
> > > $(CP) ${src} $(DEBUG_DIR)(+)$(MODULE_NAME).debug
> > > $(OBJCOPY) --strip-unneeded -R .eh_frame ${src} @@ -376,22
> > > +376,22 @@
> > > #
> > > -$(OBJCOPY) $(OBJCOPY_ADDDEBUGFLAG) ${src}
> > > -$(CP) $(DEBUG_DIR)(+)$(MODULE_NAME).debug
> > > $(BIN_DIR)(+)$(MODULE_NAME_GUID).debug
> > >
> > > "$(GENFW)" -e $(MODULE_TYPE) -o ${dst} ${src} $(GENFW_FLAGS)
> > > - $(CP) ${dst} $(OUTPUT_DIR)
> > > - $(CP) ${dst} $(BIN_DIR)(+)$(MODULE_NAME_GUID).efi
> > > + $(CP) $(DEBUG_DIR)(+)$(MODULE_NAME).efi $(OUTPUT_DIR)
> > > + $(CP) $(DEBUG_DIR)(+)$(MODULE_NAME).efi
> > > + $(BIN_DIR)(+)$(MODULE_NAME_GUID).efi
> > > -$(CP) $(DEBUG_DIR)(+)*.map $(OUTPUT_DIR)
> > >
> > > <Command.XCODE>
> > > # tool to convert Mach-O to PE/COFF
> > > "$(MTOC)" -subsystem $(MODULE_TYPE) $(MTOC_FLAGS) ${src} $(DEBUG_DIR)(+)$(MODULE_NAME).pecoff
> > > # create symbol file for GDB debug
> > > -$(DSYMUTIL) ${src}
> > > "$(GENFW)" -e $(MODULE_TYPE) -o ${dst} $(DEBUG_DIR)(+)$(MODULE_NAME).pecoff $(GENFW_FLAGS)
> > > - $(CP) ${dst} $(OUTPUT_DIR)
> > > - $(CP) ${dst} $(BIN_DIR)(+)$(MODULE_NAME_GUID).efi
> > > + $(CP) $(DEBUG_DIR)(+)$(MODULE_NAME).efi $(OUTPUT_DIR)
> > > + $(CP) $(DEBUG_DIR)(+)$(MODULE_NAME).efi
> > > + $(BIN_DIR)(+)$(MODULE_NAME_GUID).efi
> > > -$(CP) $(DEBUG_DIR)(+)*.map $(OUTPUT_DIR)
> > >
> > > [Dependency-Expression-File]
> > > <InputFile>
> > > ?.dxs, ?.Dxs, ?.DXS
> > > diff --git a/BaseTools/Source/C/GenFfs/GenFfs.c
> > > b/BaseTools/Source/C/GenFfs/GenFfs.c
> > > index 02c7ac30f0..ea36b22bef 100644
> > > --- a/BaseTools/Source/C/GenFfs/GenFfs.c
> > > +++ b/BaseTools/Source/C/GenFfs/GenFfs.c
> > > @@ -17,10 +17,14 @@ WITHOUT WARRANTIES OR REPRESENTATIONS OF ANY KIND, EITHER EXPRESS OR IMPLIED.
> > > #include <io.h>
> > > #include <sys/types.h>
> > > #include <sys/stat.h>
> > > #endif
> > >
> > > +#ifdef __GNUC__
> > > +#include <unistd.h>
> > > +#endif
> > > +
> > > #include <stdio.h>
> > > #include <stdlib.h>
> > > #include <string.h>
> > >
> > > #include <Common/UefiBaseTypes.h>
> > > @@ -158,10 +162,12 @@ Returns:
> > > FileAlign points to file alignment, which only support\n\
> > > the following align: 1,2,4,8,16,128,512,1K,4K,32K,64K\n\
> > > 128K,256K,512K,1M,2M,4M,8M,16M\n");
> > > fprintf (stdout, " -i SectionFile, --sectionfile SectionFile\n\
> > > Section file will be contained in this FFS
> > > file.\n");
> > > + fprintf (stdout, " -oi SectionFile, --optionalsectionfile SectionFile\n\
> > > + If the Section file exists, it will be
> > > + contained in this FFS file, otherwise, it will be ignored.\n");
> > > fprintf (stdout, " -n SectionAlign, --sectionalign SectionAlign\n\
> > > SectionAlign points to section alignment, which support\n\
> > > the alignment scope 0~16M. If SectionAlign is specified\n\
> > > as 0, tool get alignment value from SectionFile. It is\n\
> > > specified together with sectionfile to
> > > point its\n\ @@ -734,19 +740,26 @@ Returns:
> > > argc -= 2;
> > > argv += 2;
> > > continue;
> > > }
> > >
> > > - if ((stricmp (argv[0], "-i") == 0) || (stricmp (argv[0], "--sectionfile") == 0)) {
> > > + if ((stricmp (argv[0], "-oi") == 0) || (stricmp (argv[0],
> > > + "--optionalsectionfile") == 0) || (stricmp (argv[0], "-i") == 0)
> > > + || (stricmp (argv[0],
> > > "--sectionfile") == 0)) {
> > > //
> > > // Get Input file name and its alignment
> > > //
> > > if (argv[1] == NULL || argv[1][0] == '-') {
> > > Error (NULL, 0, 1003, "Invalid option value", "input section file is missing for -i option");
> > > goto Finish;
> > > }
> > > -
> > > + if ((stricmp (argv[0], "-oi") == 0) || (stricmp (argv[0], "--optionalsectionfile") == 0) ){
> > > + if (-1 == access(argv[1] , 0)){
> > > + Warning(NULL, 0, 0001, "File is not found.", argv[1]);
> > > + argc -= 2;
> > > + argv += 2;
> > > + continue;
> > > + }
> > > + }
> > > //
> > > // Allocate Input file name buffer and its alignment buffer.
> > > //
> > > if ((InputFileNum == 0) && (InputFileName == NULL)) {
> > > InputFileName = (CHAR8 **) malloc (MAXIMUM_INPUT_FILE_NUM *
> > > sizeof (CHAR8 *)); diff --git
> > > a/BaseTools/Source/Python/AutoGen/GenMake.py
> > > b/BaseTools/Source/Python/AutoGen/GenMake.py
> > > index b441817b52..2374bcf550 100644
> > > --- a/BaseTools/Source/Python/AutoGen/GenMake.py
> > > +++ b/BaseTools/Source/Python/AutoGen/GenMake.py
> > > @@ -714,11 +714,11 @@ cleanlib:
> > >
> > > FfsCmdList = Cmd[0]
> > > for index, Str in enumerate(FfsCmdList):
> > > if '-o' == Str:
> > > OutputFile = FfsCmdList[index + 1]
> > > - if '-i' == Str:
> > > + if '-i' == Str or "-oi" == Str:
> > > if DepsFileList == []:
> > > DepsFileList = [FfsCmdList[index + 1]]
> > > else:
> > > DepsFileList.append(FfsCmdList[index + 1])
> > > DepsFileString = ' '.join(DepsFileList).strip() diff
> > > --git a/BaseTools/Source/Python/GenFds/GenFdsGlobalVariable.py
> > > b/BaseTools/Source/Python/GenFds/GenFdsGlobalVariable.py
> > > index 028bcc480c..b42829b698 100644
> > > --- a/BaseTools/Source/Python/GenFds/GenFdsGlobalVariable.py
> > > +++ b/BaseTools/Source/Python/GenFds/GenFdsGlobalVariable.py
> > > @@ -33,10 +33,11 @@ from Common.ToolDefClassObject import
> > > ToolDefClassObject, ToolDefDict from AutoGen.BuildEngine import
> > > BuildRule import Common.DataType as DataType from Common.Misc
> > > import PathClass from Common.LongFilePathSupport import
> > > OpenLongFilePath as open from Common.MultipleWorkspace import
> > > MultipleWorkspace as mws
> > > +import Common.GlobalData as GlobalData
> > >
> > > ## Global variables
> > > #
> > > #
> > > class GenFdsGlobalVariable:
> > > @@ -499,10 +500,14 @@ class GenFdsGlobalVariable:
> > > Cmd += ("-o", Output)
> > > Cmd += Input
> > >
> > > SaveFileOnChange(CommandFile, ' '.join(Cmd), False)
> > > if IsMakefile:
> > > + if GlobalData.gGlobalDefines.get("FAMILY") == "MSFT":
> > > + Cmd = ['if', 'exist', Input[0]] + Cmd
> > > + else:
> > > + Cmd = ['test', '-e', Input[0], "&&"] + Cmd
> > > if ' '.join(Cmd).strip() not in GenFdsGlobalVariable.SecCmdList:
> > > GenFdsGlobalVariable.SecCmdList.append(' '.join(Cmd).strip())
> > > elif GenFdsGlobalVariable.NeedsUpdate(Output, list(Input) + [CommandFile]):
> > > GenFdsGlobalVariable.DebugLogger(EdkLogger.DEBUG_5,
> > > "%s needs update because of newer %s" % (Output,
> > Input))
> > > GenFdsGlobalVariable.CallExternalTool(Cmd, "Failed
> > > to generate section") @@ -540,11 +545,14 @@ class GenFdsGlobalVariable:
> > > Align = mFfsValidAlign[index + 1]
> > > Cmd += ("-a", Align)
> > >
> > > Cmd += ("-o", Output)
> > > for I in range(0, len(Input)):
> > > - Cmd += ("-i", Input[I])
> > > + if MakefilePath:
> > > + Cmd += ("-oi", Input[I])
> > > + else:
> > > + Cmd += ("-i", Input[I])
> > > if SectionAlign and SectionAlign[I]:
> > > Cmd += ("-n", SectionAlign[I])
> > >
> > > CommandFile = Output + '.txt'
> > > SaveFileOnChange(CommandFile, ' '.join(Cmd), False) diff
> > > --git a/BaseTools/Source/Python/Workspace/DscBuildData.py
> > > b/BaseTools/Source/Python/Workspace/DscBuildData.py
> > > index 129c0c950b..ac95c7df17 100644
> > > --- a/BaseTools/Source/Python/Workspace/DscBuildData.py
> > > +++ b/BaseTools/Source/Python/Workspace/DscBuildData.py
> > > @@ -318,10 +318,13 @@ class DscBuildData(PlatformBuildClassObject):
> > >
> > > ## Get architecture
> > > @property
> > > def Arch(self):
> > > return self._Arch
> > > + @property
> > > + def Dir(self):
> > > + return self.MetaFile.Dir
> > >
> > > ## Retrieve all information in [Defines] section
> > > #
> > > # (Retrieving all [Defines] information in one-shot is just to save time.)
> > > #
> > > diff --git a/BaseTools/Source/Python/build/build.py
> > > b/BaseTools/Source/Python/build/build.py
> > > index de641fb452..41d405e6ff 100644
> > > --- a/BaseTools/Source/Python/build/build.py
> > > +++ b/BaseTools/Source/Python/build/build.py
> > > @@ -1667,11 +1667,11 @@ class Build():
> > > self.Progress.Stop("done!")
> > >
> > > # Add ffs build to makefile
> > > CmdListDict = {}
> > > if GlobalData.gEnableGenfdsMultiThread and self.Fdf:
> > > - CmdListDict = self._GenFfsCmd()
> > > + CmdListDict = self._GenFfsCmd(Wa.ArchList)
> > >
> > > for Arch in Wa.ArchList:
> > > GlobalData.gGlobalDefines['ARCH'] = Arch
> > > Pa = PlatformAutoGen(Wa, self.PlatformFile, BuildTarget, ToolChain, Arch)
> > > for Module in Pa.Platform.Modules:
> > > @@ -1760,11 +1760,11 @@ class Build():
> > > self.LoadFixAddress = Wa.Platform.LoadFixAddress
> > > Wa.CreateMakeFile(False)
> > > # Add ffs build to makefile
> > > CmdListDict = None
> > > if GlobalData.gEnableGenfdsMultiThread and self.Fdf:
> > > - CmdListDict = self._GenFfsCmd()
> > > + CmdListDict = self._GenFfsCmd(Wa.ArchList)
> > > self.Progress.Stop("done!")
> > > MaList = []
> > > ExitFlag = threading.Event()
> > > ExitFlag.clear()
> > > self.AutoGenTime += int(round((time.time() -
> > > WorkspaceAutoGenTime))) @@ -1879,15 +1879,15 @@ class Build():
> > > #
> > > # Save MAP buffer into MAP file.
> > > #
> > > self._SaveMapFile (MapBuffer, Wa)
> > >
> > > - def _GenFfsCmd(self):
> > > + def _GenFfsCmd(self,ArchList):
> > > # convert dictionary of Cmd:(Inf,Arch)
> > > # to a new dictionary of (Inf,Arch):Cmd,Cmd,Cmd...
> > > CmdSetDict = defaultdict(set)
> > > - GenFfsDict = GenFds.GenFfsMakefile('', GlobalData.gFdfParser, self, self.ArchList, GlobalData)
> > > + GenFfsDict = GenFds.GenFfsMakefile('',
> > > + GlobalData.gFdfParser, self, ArchList, GlobalData)
> > > for Cmd in GenFfsDict:
> > > tmpInf, tmpArch = GenFfsDict[Cmd]
> > > CmdSetDict[tmpInf, tmpArch].add(Cmd)
> > > return CmdSetDict
> > >
> > > @@ -1927,11 +1927,11 @@ class Build():
> > > Wa.CreateMakeFile(False)
> > >
> > > # Add ffs build to makefile
> > > CmdListDict = None
> > > if GlobalData.gEnableGenfdsMultiThread and self.Fdf:
> > > - CmdListDict = self._GenFfsCmd()
> > > + CmdListDict = self._GenFfsCmd(Wa.ArchList)
> > >
> > > # multi-thread exit flag
> > > ExitFlag = threading.Event()
> > > ExitFlag.clear()
> > > self.AutoGenTime += int(round((time.time() -
> > > WorkspaceAutoGenTime)))
> > > --
> > > 2.20.1.windows.1
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [Patch] BaseTools: Fixed issue in MultiThread Genfds function
2019-03-29 14:09 ` Gao, Liming
@ 2019-04-03 2:15 ` Feng, Bob C
0 siblings, 0 replies; 7+ messages in thread
From: Feng, Bob C @ 2019-04-03 2:15 UTC (permalink / raw)
To: Gao, Liming, edk2-devel@lists.01.org
BZ for remove additional copy action.
https://bugzilla.tianocore.org/show_bug.cgi?id=1673
-----Original Message-----
From: Gao, Liming
Sent: Friday, March 29, 2019 10:09 PM
To: Feng, Bob C <bob.c.feng@intel.com>; edk2-devel@lists.01.org
Subject: RE: [Patch] BaseTools: Fixed issue in MultiThread Genfds function
Yes. Please submit one BZ for it.
> -----Original Message-----
> From: Feng, Bob C
> Sent: Friday, March 29, 2019 8:40 PM
> To: Gao, Liming <liming.gao@intel.com>; edk2-devel@lists.01.org
> Subject: RE: [Patch] BaseTools: Fixed issue in MultiThread Genfds
> function
>
> That will impact the original GenFds since the command is generate by
> reusing the GenFds functions. What about enter a new BZ to do that change?
>
> Thanks,
> Bob
>
> -----Original Message-----
> From: Gao, Liming
> Sent: Friday, March 29, 2019 8:33 PM
> To: Feng, Bob C <bob.c.feng@intel.com>; edk2-devel@lists.01.org
> Subject: RE: [Patch] BaseTools: Fixed issue in MultiThread Genfds
> function
>
> Bob:
> Could you update the rule to depend on EFI image from DEBUG_DIR? I
> would like to remove the additional copy from DEBUG dir to OUTPUT dir and save the disk and performance.
>
> $(FFS_OUTPUT_DIR)\$(MODULE_GUID)SEC2.1.1.1.pe32 : $(DEBUG_DIR)\LogoDxe.efi
> GenSec -s EFI_SECTION_PE32 -o
> $(FFS_OUTPUT_DIR)\$(MODULE_GUID)SEC2.1.1.1.pe32
> $(DEBUG_DIR)\LogoDxe.efi
>
> Thanks
> Liming
> > -----Original Message-----
> > From: Feng, Bob C
> > Sent: Friday, March 29, 2019 8:00 PM
> > To: Gao, Liming <liming.gao@intel.com>; edk2-devel@lists.01.org
> > Subject: RE: [Patch] BaseTools: Fixed issue in MultiThread Genfds
> > function
> >
> > That's to fix the issue that make will fail if enable multiple thread genfds.
> >
> > For example, If enable Multiple thread Genfds, there will be GenSec command in the Makefile like below.
> >
> > $(FFS_OUTPUT_DIR)\$(MODULE_GUID)SEC2.1.1.1.pe32 : $(OUTPUT_DIR)\LogoDxe.efi
> > GenSec -s EFI_SECTION_PE32 -o
> > $(FFS_OUTPUT_DIR)\$(MODULE_GUID)SEC2.1.1.1.pe32
> > $(OUTPUT_DIR)\LogoDxe.efi
> >
> >
> > The $(OUTPUT_DIR)\LogoDxe.efi is the dependency, but there is no make rule for $(OUTPUT_DIR)\LogoDxe.efi generated in the Makefile.
> > Make program does not know how to make $(OUTPUT_DIR)\LogoDxe.efi. Make fails for this case.
> >
> > In the build_rule.txt, the $(OUTPUT_DIR)\ )(+)$(MODULE_NAME).efi is generated by the command $(CP)
> > $(DEBUG_DIR)(+)$(MODULE_NAME).efi $(OUTPUT_DIR) under
> > Dynamic-Library-File section, so $(OUTPUT_DIR)\
> > )(+)$(MODULE_NAME).efi should also be a output for
> > Dynamic-Library-File
> >
> > Thanks,
> > Bob
> >
> > -----Original Message-----
> > From: Gao, Liming
> > Sent: Friday, March 29, 2019 7:42 PM
> > To: Feng, Bob C <bob.c.feng@intel.com>; edk2-devel@lists.01.org
> > Subject: RE: [Patch] BaseTools: Fixed issue in MultiThread Genfds
> > function
> >
> > Bob:
> > Could you list the more information on why update build_rule.txt?
> >
> > > -----Original Message-----
> > > From: Feng, Bob C
> > > Sent: Friday, March 29, 2019 7:07 PM
> > > To: edk2-devel@lists.01.org
> > > Cc: Feng, Bob C <bob.c.feng@intel.com>; Gao, Liming
> > > <liming.gao@intel.com>
> > > Subject: [Patch] BaseTools: Fixed issue in MultiThread Genfds
> > > function
> > >
> > > https://bugzilla.tianocore.org/show_bug.cgi?id=1450
> > > In the Multiple thread Genfds feature, build tool generates
> > > GenSec, GenFFS command in Makefile.
> > >
> > > The Non-Hii Driver does not generate .offset file for uni string
> > > offset, but the build tool has not knowledge about this in autogen
> > > phase. So in this patch, I add a check in Makefile for GenSec command.
> > > If the GenSec input file does not exist, the GenSec will not be
> > > called. And if GenSec command is not called, its output file,
> > > which is also the input file of GenFfs command, will also not
> > > exist.So for GenFfs command, I add a new command parameter -oi
> > > which means the input file is an optional input file which would
> > > not exist. so that I can generate GenFfs command with "-oi" parameter in Makefile.
> > >
> > > Contributed-under: TianoCore Contribution Agreement 1.1
> > > Signed-off-by: Bob Feng <bob.c.feng@intel.com>
> > > Cc: Liming Gao <liming.gao@intel.com>
> > > ---
> > > BaseTools/Conf/build_rule.template | 14 +++++++-------
> > > BaseTools/Source/C/GenFfs/GenFfs.c | 17 +++++++++++++++--
> > > BaseTools/Source/Python/AutoGen/GenMake.py | 2 +-
> > > .../Python/GenFds/GenFdsGlobalVariable.py | 10 +++++++++-
> > > .../Source/Python/Workspace/DscBuildData.py | 3 +++
> > > BaseTools/Source/Python/build/build.py | 10 +++++-----
> > > 6 files changed, 40 insertions(+), 16 deletions(-)
> > >
> > > diff --git a/BaseTools/Conf/build_rule.template
> > > b/BaseTools/Conf/build_rule.template
> > > index e56b1d9c59..8cd7d0e445 100755
> > > --- a/BaseTools/Conf/build_rule.template
> > > +++ b/BaseTools/Conf/build_rule.template
> > > @@ -357,16 +357,16 @@
> > > [Dynamic-Library-File]
> > > <InputFile>
> > > ?.dll
> > >
> > > <OutputFile>
> > > - $(DEBUG_DIR)(+)$(MODULE_NAME).efi
> > > + $(DEBUG_DIR)(+)$(MODULE_NAME).efi
> > > + $(OUTPUT_DIR)(+)$(MODULE_NAME).efi
> > >
> > > <Command.MSFT, Command.INTEL, Command.RVCT>
> > > "$(GENFW)" -e $(MODULE_TYPE) -o ${dst} ${src} $(GENFW_FLAGS)
> > > - $(CP) ${dst} $(OUTPUT_DIR)
> > > - $(CP) ${dst} $(BIN_DIR)(+)$(MODULE_NAME_GUID).efi
> > > + $(CP) $(DEBUG_DIR)(+)$(MODULE_NAME).efi $(OUTPUT_DIR)
> > > + $(CP) $(DEBUG_DIR)(+)$(MODULE_NAME).efi
> > > + $(BIN_DIR)(+)$(MODULE_NAME_GUID).efi
> > > -$(CP) $(DEBUG_DIR)(+)*.map $(OUTPUT_DIR)
> > > -$(CP) $(DEBUG_DIR)(+)*.pdb $(OUTPUT_DIR)
> > > <Command.GCC>
> > > $(CP) ${src} $(DEBUG_DIR)(+)$(MODULE_NAME).debug
> > > $(OBJCOPY) --strip-unneeded -R .eh_frame ${src} @@
> > > -376,22
> > > +376,22 @@
> > > #
> > > -$(OBJCOPY) $(OBJCOPY_ADDDEBUGFLAG) ${src}
> > > -$(CP) $(DEBUG_DIR)(+)$(MODULE_NAME).debug
> > > $(BIN_DIR)(+)$(MODULE_NAME_GUID).debug
> > >
> > > "$(GENFW)" -e $(MODULE_TYPE) -o ${dst} ${src} $(GENFW_FLAGS)
> > > - $(CP) ${dst} $(OUTPUT_DIR)
> > > - $(CP) ${dst} $(BIN_DIR)(+)$(MODULE_NAME_GUID).efi
> > > + $(CP) $(DEBUG_DIR)(+)$(MODULE_NAME).efi $(OUTPUT_DIR)
> > > + $(CP) $(DEBUG_DIR)(+)$(MODULE_NAME).efi
> > > + $(BIN_DIR)(+)$(MODULE_NAME_GUID).efi
> > > -$(CP) $(DEBUG_DIR)(+)*.map $(OUTPUT_DIR)
> > >
> > > <Command.XCODE>
> > > # tool to convert Mach-O to PE/COFF
> > > "$(MTOC)" -subsystem $(MODULE_TYPE) $(MTOC_FLAGS) ${src} $(DEBUG_DIR)(+)$(MODULE_NAME).pecoff
> > > # create symbol file for GDB debug
> > > -$(DSYMUTIL) ${src}
> > > "$(GENFW)" -e $(MODULE_TYPE) -o ${dst} $(DEBUG_DIR)(+)$(MODULE_NAME).pecoff $(GENFW_FLAGS)
> > > - $(CP) ${dst} $(OUTPUT_DIR)
> > > - $(CP) ${dst} $(BIN_DIR)(+)$(MODULE_NAME_GUID).efi
> > > + $(CP) $(DEBUG_DIR)(+)$(MODULE_NAME).efi $(OUTPUT_DIR)
> > > + $(CP) $(DEBUG_DIR)(+)$(MODULE_NAME).efi
> > > + $(BIN_DIR)(+)$(MODULE_NAME_GUID).efi
> > > -$(CP) $(DEBUG_DIR)(+)*.map $(OUTPUT_DIR)
> > >
> > > [Dependency-Expression-File]
> > > <InputFile>
> > > ?.dxs, ?.Dxs, ?.DXS
> > > diff --git a/BaseTools/Source/C/GenFfs/GenFfs.c
> > > b/BaseTools/Source/C/GenFfs/GenFfs.c
> > > index 02c7ac30f0..ea36b22bef 100644
> > > --- a/BaseTools/Source/C/GenFfs/GenFfs.c
> > > +++ b/BaseTools/Source/C/GenFfs/GenFfs.c
> > > @@ -17,10 +17,14 @@ WITHOUT WARRANTIES OR REPRESENTATIONS OF ANY KIND, EITHER EXPRESS OR IMPLIED.
> > > #include <io.h>
> > > #include <sys/types.h>
> > > #include <sys/stat.h>
> > > #endif
> > >
> > > +#ifdef __GNUC__
> > > +#include <unistd.h>
> > > +#endif
> > > +
> > > #include <stdio.h>
> > > #include <stdlib.h>
> > > #include <string.h>
> > >
> > > #include <Common/UefiBaseTypes.h> @@ -158,10 +162,12 @@ Returns:
> > > FileAlign points to file alignment, which only support\n\
> > > the following align: 1,2,4,8,16,128,512,1K,4K,32K,64K\n\
> > > 128K,256K,512K,1M,2M,4M,8M,16M\n");
> > > fprintf (stdout, " -i SectionFile, --sectionfile SectionFile\n\
> > > Section file will be contained in this
> > > FFS file.\n");
> > > + fprintf (stdout, " -oi SectionFile, --optionalsectionfile SectionFile\n\
> > > + If the Section file exists, it will be
> > > + contained in this FFS file, otherwise, it will be ignored.\n");
> > > fprintf (stdout, " -n SectionAlign, --sectionalign SectionAlign\n\
> > > SectionAlign points to section alignment, which support\n\
> > > the alignment scope 0~16M. If SectionAlign is specified\n\
> > > as 0, tool get alignment value from SectionFile. It is\n\
> > > specified together with sectionfile to
> > > point its\n\ @@ -734,19 +740,26 @@ Returns:
> > > argc -= 2;
> > > argv += 2;
> > > continue;
> > > }
> > >
> > > - if ((stricmp (argv[0], "-i") == 0) || (stricmp (argv[0], "--sectionfile") == 0)) {
> > > + if ((stricmp (argv[0], "-oi") == 0) || (stricmp (argv[0],
> > > + "--optionalsectionfile") == 0) || (stricmp (argv[0], "-i") == 0)
> > > + || (stricmp (argv[0],
> > > "--sectionfile") == 0)) {
> > > //
> > > // Get Input file name and its alignment
> > > //
> > > if (argv[1] == NULL || argv[1][0] == '-') {
> > > Error (NULL, 0, 1003, "Invalid option value", "input section file is missing for -i option");
> > > goto Finish;
> > > }
> > > -
> > > + if ((stricmp (argv[0], "-oi") == 0) || (stricmp (argv[0], "--optionalsectionfile") == 0) ){
> > > + if (-1 == access(argv[1] , 0)){
> > > + Warning(NULL, 0, 0001, "File is not found.", argv[1]);
> > > + argc -= 2;
> > > + argv += 2;
> > > + continue;
> > > + }
> > > + }
> > > //
> > > // Allocate Input file name buffer and its alignment buffer.
> > > //
> > > if ((InputFileNum == 0) && (InputFileName == NULL)) {
> > > InputFileName = (CHAR8 **) malloc (MAXIMUM_INPUT_FILE_NUM
> > > * sizeof (CHAR8 *)); diff --git
> > > a/BaseTools/Source/Python/AutoGen/GenMake.py
> > > b/BaseTools/Source/Python/AutoGen/GenMake.py
> > > index b441817b52..2374bcf550 100644
> > > --- a/BaseTools/Source/Python/AutoGen/GenMake.py
> > > +++ b/BaseTools/Source/Python/AutoGen/GenMake.py
> > > @@ -714,11 +714,11 @@ cleanlib:
> > >
> > > FfsCmdList = Cmd[0]
> > > for index, Str in enumerate(FfsCmdList):
> > > if '-o' == Str:
> > > OutputFile = FfsCmdList[index + 1]
> > > - if '-i' == Str:
> > > + if '-i' == Str or "-oi" == Str:
> > > if DepsFileList == []:
> > > DepsFileList = [FfsCmdList[index + 1]]
> > > else:
> > > DepsFileList.append(FfsCmdList[index + 1])
> > > DepsFileString = ' '.join(DepsFileList).strip() diff
> > > --git a/BaseTools/Source/Python/GenFds/GenFdsGlobalVariable.py
> > > b/BaseTools/Source/Python/GenFds/GenFdsGlobalVariable.py
> > > index 028bcc480c..b42829b698 100644
> > > --- a/BaseTools/Source/Python/GenFds/GenFdsGlobalVariable.py
> > > +++ b/BaseTools/Source/Python/GenFds/GenFdsGlobalVariable.py
> > > @@ -33,10 +33,11 @@ from Common.ToolDefClassObject import
> > > ToolDefClassObject, ToolDefDict from AutoGen.BuildEngine import
> > > BuildRule import Common.DataType as DataType from Common.Misc
> > > import PathClass from Common.LongFilePathSupport import
> > > OpenLongFilePath as open from Common.MultipleWorkspace import
> > > MultipleWorkspace as mws
> > > +import Common.GlobalData as GlobalData
> > >
> > > ## Global variables
> > > #
> > > #
> > > class GenFdsGlobalVariable:
> > > @@ -499,10 +500,14 @@ class GenFdsGlobalVariable:
> > > Cmd += ("-o", Output)
> > > Cmd += Input
> > >
> > > SaveFileOnChange(CommandFile, ' '.join(Cmd), False)
> > > if IsMakefile:
> > > + if GlobalData.gGlobalDefines.get("FAMILY") == "MSFT":
> > > + Cmd = ['if', 'exist', Input[0]] + Cmd
> > > + else:
> > > + Cmd = ['test', '-e', Input[0], "&&"] + Cmd
> > > if ' '.join(Cmd).strip() not in GenFdsGlobalVariable.SecCmdList:
> > > GenFdsGlobalVariable.SecCmdList.append(' '.join(Cmd).strip())
> > > elif GenFdsGlobalVariable.NeedsUpdate(Output, list(Input) + [CommandFile]):
> > >
> > > GenFdsGlobalVariable.DebugLogger(EdkLogger.DEBUG_5,
> > > "%s needs update because of newer %s" % (Output,
> > Input))
> > > GenFdsGlobalVariable.CallExternalTool(Cmd,
> > > "Failed to generate section") @@ -540,11 +545,14 @@ class GenFdsGlobalVariable:
> > > Align = mFfsValidAlign[index + 1]
> > > Cmd += ("-a", Align)
> > >
> > > Cmd += ("-o", Output)
> > > for I in range(0, len(Input)):
> > > - Cmd += ("-i", Input[I])
> > > + if MakefilePath:
> > > + Cmd += ("-oi", Input[I])
> > > + else:
> > > + Cmd += ("-i", Input[I])
> > > if SectionAlign and SectionAlign[I]:
> > > Cmd += ("-n", SectionAlign[I])
> > >
> > > CommandFile = Output + '.txt'
> > > SaveFileOnChange(CommandFile, ' '.join(Cmd), False) diff
> > > --git a/BaseTools/Source/Python/Workspace/DscBuildData.py
> > > b/BaseTools/Source/Python/Workspace/DscBuildData.py
> > > index 129c0c950b..ac95c7df17 100644
> > > --- a/BaseTools/Source/Python/Workspace/DscBuildData.py
> > > +++ b/BaseTools/Source/Python/Workspace/DscBuildData.py
> > > @@ -318,10 +318,13 @@ class DscBuildData(PlatformBuildClassObject):
> > >
> > > ## Get architecture
> > > @property
> > > def Arch(self):
> > > return self._Arch
> > > + @property
> > > + def Dir(self):
> > > + return self.MetaFile.Dir
> > >
> > > ## Retrieve all information in [Defines] section
> > > #
> > > # (Retrieving all [Defines] information in one-shot is just to save time.)
> > > #
> > > diff --git a/BaseTools/Source/Python/build/build.py
> > > b/BaseTools/Source/Python/build/build.py
> > > index de641fb452..41d405e6ff 100644
> > > --- a/BaseTools/Source/Python/build/build.py
> > > +++ b/BaseTools/Source/Python/build/build.py
> > > @@ -1667,11 +1667,11 @@ class Build():
> > > self.Progress.Stop("done!")
> > >
> > > # Add ffs build to makefile
> > > CmdListDict = {}
> > > if GlobalData.gEnableGenfdsMultiThread and self.Fdf:
> > > - CmdListDict = self._GenFfsCmd()
> > > + CmdListDict = self._GenFfsCmd(Wa.ArchList)
> > >
> > > for Arch in Wa.ArchList:
> > > GlobalData.gGlobalDefines['ARCH'] = Arch
> > > Pa = PlatformAutoGen(Wa, self.PlatformFile, BuildTarget, ToolChain, Arch)
> > > for Module in Pa.Platform.Modules:
> > > @@ -1760,11 +1760,11 @@ class Build():
> > > self.LoadFixAddress = Wa.Platform.LoadFixAddress
> > > Wa.CreateMakeFile(False)
> > > # Add ffs build to makefile
> > > CmdListDict = None
> > > if GlobalData.gEnableGenfdsMultiThread and self.Fdf:
> > > - CmdListDict = self._GenFfsCmd()
> > > + CmdListDict = self._GenFfsCmd(Wa.ArchList)
> > > self.Progress.Stop("done!")
> > > MaList = []
> > > ExitFlag = threading.Event()
> > > ExitFlag.clear()
> > > self.AutoGenTime += int(round((time.time() -
> > > WorkspaceAutoGenTime))) @@ -1879,15 +1879,15 @@ class Build():
> > > #
> > > # Save MAP buffer into MAP file.
> > > #
> > > self._SaveMapFile (MapBuffer, Wa)
> > >
> > > - def _GenFfsCmd(self):
> > > + def _GenFfsCmd(self,ArchList):
> > > # convert dictionary of Cmd:(Inf,Arch)
> > > # to a new dictionary of (Inf,Arch):Cmd,Cmd,Cmd...
> > > CmdSetDict = defaultdict(set)
> > > - GenFfsDict = GenFds.GenFfsMakefile('', GlobalData.gFdfParser, self, self.ArchList, GlobalData)
> > > + GenFfsDict = GenFds.GenFfsMakefile('',
> > > + GlobalData.gFdfParser, self, ArchList, GlobalData)
> > > for Cmd in GenFfsDict:
> > > tmpInf, tmpArch = GenFfsDict[Cmd]
> > > CmdSetDict[tmpInf, tmpArch].add(Cmd)
> > > return CmdSetDict
> > >
> > > @@ -1927,11 +1927,11 @@ class Build():
> > > Wa.CreateMakeFile(False)
> > >
> > > # Add ffs build to makefile
> > > CmdListDict = None
> > > if GlobalData.gEnableGenfdsMultiThread and self.Fdf:
> > > - CmdListDict = self._GenFfsCmd()
> > > + CmdListDict = self._GenFfsCmd(Wa.ArchList)
> > >
> > > # multi-thread exit flag
> > > ExitFlag = threading.Event()
> > > ExitFlag.clear()
> > > self.AutoGenTime += int(round((time.time() -
> > > WorkspaceAutoGenTime)))
> > > --
> > > 2.20.1.windows.1
^ permalink raw reply [flat|nested] 7+ messages in thread
end of thread, other threads:[~2019-04-03 2:15 UTC | newest]
Thread overview: 7+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2019-03-29 11:06 [Patch] BaseTools: Fixed issue in MultiThread Genfds function Feng, Bob C
2019-03-29 11:41 ` Gao, Liming
2019-03-29 12:00 ` Feng, Bob C
2019-03-29 12:33 ` Gao, Liming
2019-03-29 12:39 ` Feng, Bob C
2019-03-29 14:09 ` Gao, Liming
2019-04-03 2:15 ` Feng, Bob C
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox