public inbox for devel@edk2.groups.io
 help / color / mirror / Atom feed
* [PATCH v4 0/2] BaseTools: Rationalise makefile generation
@ 2020-02-10 10:49 PierreGondois
  2020-02-10 10:49 ` [PATCH v4 1/2] " PierreGondois
                   ` (3 more replies)
  0 siblings, 4 replies; 5+ messages in thread
From: PierreGondois @ 2020-02-10 10:49 UTC (permalink / raw)
  To: devel
  Cc: Pierre Gondois, bob.c.feng, liming.gao, Sami.Mujawar,
	pierre.gondois, nd

v4:
  Remove caret in NASM_INC macro:
  I could not reproduce the build fail Bob Feng had when issuing
  the following command:
    build -p UefiCpuPkg\UefiCpuPkg.dsc -a IA32 -t VS2015x86
  Nonetheless, this v4 should only have an impact on the builds
  made on Windows platforms using nmake.

The changes can be seen at https://github.com/PierreARM/edk2/tree/720_BaseTools_Rationalise_makefile_generation_v4

Pierre Gondois (2):
  BaseTools: Rationalise makefile generation
  BaseTools: Remove caret in NASM_INC macro

 BaseTools/Source/Python/AutoGen/GenMake.py         | 145 +++++++++++---------
 BaseTools/Source/Python/AutoGen/IncludesAutoGen.py |  34 +++--
 BaseTools/Source/Python/AutoGen/PlatformAutoGen.py |  13 +-
 BaseTools/Source/Python/build/build.py             |  25 ++--
 4 files changed, 120 insertions(+), 97 deletions(-)

-- 
'Guid(CE165669-3EF3-493F-B85D-6190EE5B9759)'


^ permalink raw reply	[flat|nested] 5+ messages in thread

* [PATCH v4 1/2] BaseTools: Rationalise makefile generation
  2020-02-10 10:49 [PATCH v4 0/2] BaseTools: Rationalise makefile generation PierreGondois
@ 2020-02-10 10:49 ` PierreGondois
  2020-02-10 10:49 ` [PATCH v4 2/2] BaseTools: Remove caret in NASM_INC macro PierreGondois
                   ` (2 subsequent siblings)
  3 siblings, 0 replies; 5+ messages in thread
From: PierreGondois @ 2020-02-10 10:49 UTC (permalink / raw)
  To: devel
  Cc: Pierre Gondois, bob.c.feng, liming.gao, Sami.Mujawar,
	pierre.gondois, nd

From: Pierre Gondois <pierre.gondois@arm.com>

The GenMake.py script tests the platform environment
to determine the type of makefile that needs to be
generated. If a Windows build host is detected, the
makefile generated is of Nmake type. Otherwise a
GNUmake type is generated.

Furthermore, the <TARGET>_<TAGNAME>_<ARCH>_MAKE_PATH
option in tools_def.template defines the make tool
to use.
E.g.: for VS2017 this is configured to use Nmake, cf.
*_VS2017_*_MAKE_PATH = DEF(VS2017_BIN_HOST)\nmake.exe
while for GCC5 it is setup to use GNU make.
*_GCC5_*_MAKE_PATH = DEF(GCC_HOST_PREFIX)make

This prevents using the GCC compiler toolchain on a
Windows build host.

To address this issue this patch introduces 2 factors
to determine the generated makefile output.
  1. Platform -> to determine shell commands used
                 in makefile.
  2. MakeTool -> to determine the type of makefile
                 that needs to be generated.

Signed-off-by: Pierre Gondois <pierre.gondois@arm.com>
Signed-off-by: Sami Mujawar <sami.mujawar@arm.com>
---

The changes can be seen at https://github.com/PierreARM/edk2/tree/720_BaseTools_Rationalise_makefile_generation_v4

 BaseTools/Source/Python/AutoGen/GenMake.py         | 138 +++++++++++---------
 BaseTools/Source/Python/AutoGen/IncludesAutoGen.py |  34 +++--
 BaseTools/Source/Python/AutoGen/PlatformAutoGen.py |  13 +-
 BaseTools/Source/Python/build/build.py             |  25 ++--
 4 files changed, 116 insertions(+), 94 deletions(-)

diff --git a/BaseTools/Source/Python/AutoGen/GenMake.py b/BaseTools/Source/Python/AutoGen/GenMake.py
index 9ae09c47caf8a674168de1ef4667f17af4f06a73..f5d15c40ad3196e5ea56f375bec12c38d313c9ee 100755
--- a/BaseTools/Source/Python/AutoGen/GenMake.py
+++ b/BaseTools/Source/Python/AutoGen/GenMake.py
@@ -2,6 +2,7 @@
 # Create makefile for MS nmake and GNU make
 #
 # Copyright (c) 2007 - 2020, Intel Corporation. All rights reserved.<BR>
+# Copyright (c) 2020, ARM Limited. All rights reserved.<BR>
 # SPDX-License-Identifier: BSD-2-Clause-Patent
 #
 
@@ -52,13 +53,10 @@ gIncludeMacroConversion = {
   "EFI_PPI_DEPENDENCY"              :   gPpiDefinition,
 }
 
-## default makefile type
-gMakeType = ""
-if sys.platform == "win32":
-    gMakeType = "nmake"
-else:
-    gMakeType = "gmake"
-
+NMAKE_FILETYPE = "nmake"
+GMAKE_FILETYPE = "gmake"
+WIN32_PLATFORM = "win32"
+POSIX_PLATFORM = "posix"
 
 ## BuildFile class
 #
@@ -73,10 +71,17 @@ class BuildFile(object):
 
     ## default file name for each type of build file
     _FILE_NAME_ = {
-        "nmake" :   "Makefile",
-        "gmake" :   "GNUmakefile"
+        NMAKE_FILETYPE :   "Makefile",
+        GMAKE_FILETYPE :   "GNUmakefile"
     }
 
+    # Get Makefile name.
+    def getMakefileName(self):
+        if not self._FileType:
+            return self._DEFAULT_FILE_NAME_
+        else:
+            return self._FILE_NAME_[self._FileType]
+
     ## Fixed header string for makefile
     _MAKEFILE_HEADER = '''#
 # DO NOT EDIT
@@ -94,8 +99,8 @@ class BuildFile(object):
 
     ## Header string for each type of build file
     _FILE_HEADER_ = {
-        "nmake" :   _MAKEFILE_HEADER % _FILE_NAME_["nmake"],
-        "gmake" :   _MAKEFILE_HEADER % _FILE_NAME_["gmake"]
+        NMAKE_FILETYPE :   _MAKEFILE_HEADER % _FILE_NAME_[NMAKE_FILETYPE],
+        GMAKE_FILETYPE :   _MAKEFILE_HEADER % _FILE_NAME_[GMAKE_FILETYPE]
     }
 
     ## shell commands which can be used in build file in the form of macro
@@ -106,7 +111,7 @@ class BuildFile(object):
     #   $(RD)     remove dir command
     #
     _SHELL_CMD_ = {
-        "nmake" : {
+        WIN32_PLATFORM : {
             "CP"    :   "copy /y",
             "MV"    :   "move /y",
             "RM"    :   "del /f /q",
@@ -114,7 +119,7 @@ class BuildFile(object):
             "RD"    :   "rmdir /s /q",
         },
 
-        "gmake" : {
+        POSIX_PLATFORM : {
             "CP"    :   "cp -f",
             "MV"    :   "mv -f",
             "RM"    :   "rm -f",
@@ -125,40 +130,40 @@ class BuildFile(object):
 
     ## directory separator
     _SEP_ = {
-        "nmake" :   "\\",
-        "gmake" :   "/"
+        WIN32_PLATFORM :   "\\",
+        POSIX_PLATFORM :   "/"
     }
 
     ## directory creation template
     _MD_TEMPLATE_ = {
-        "nmake" :   'if not exist %(dir)s $(MD) %(dir)s',
-        "gmake" :   "$(MD) %(dir)s"
+        WIN32_PLATFORM :   'if not exist %(dir)s $(MD) %(dir)s',
+        POSIX_PLATFORM :   "$(MD) %(dir)s"
     }
 
     ## directory removal template
     _RD_TEMPLATE_ = {
-        "nmake" :   'if exist %(dir)s $(RD) %(dir)s',
-        "gmake" :   "$(RD) %(dir)s"
+        WIN32_PLATFORM :   'if exist %(dir)s $(RD) %(dir)s',
+        POSIX_PLATFORM :   "$(RD) %(dir)s"
     }
     ## cp if exist
     _CP_TEMPLATE_ = {
-        "nmake" :   'if exist %(Src)s $(CP) %(Src)s %(Dst)s',
-        "gmake" :   "test -f %(Src)s && $(CP) %(Src)s %(Dst)s"
+        WIN32_PLATFORM :   'if exist %(Src)s $(CP) %(Src)s %(Dst)s',
+        POSIX_PLATFORM :   "test -f %(Src)s && $(CP) %(Src)s %(Dst)s"
     }
 
     _CD_TEMPLATE_ = {
-        "nmake" :   'if exist %(dir)s cd %(dir)s',
-        "gmake" :   "test -e %(dir)s && cd %(dir)s"
+        WIN32_PLATFORM :   'if exist %(dir)s cd %(dir)s',
+        POSIX_PLATFORM :   "test -e %(dir)s && cd %(dir)s"
     }
 
     _MAKE_TEMPLATE_ = {
-        "nmake" :   'if exist %(file)s "$(MAKE)" $(MAKE_FLAGS) -f %(file)s',
-        "gmake" :   'test -e %(file)s && "$(MAKE)" $(MAKE_FLAGS) -f %(file)s'
+        WIN32_PLATFORM :   'if exist %(file)s "$(MAKE)" $(MAKE_FLAGS) -f %(file)s',
+        POSIX_PLATFORM :   'test -e %(file)s && "$(MAKE)" $(MAKE_FLAGS) -f %(file)s'
     }
 
     _INCLUDE_CMD_ = {
-        "nmake" :   '!INCLUDE',
-        "gmake" :   "include"
+        NMAKE_FILETYPE :   '!INCLUDE',
+        GMAKE_FILETYPE :   "include"
     }
 
     _INC_FLAG_ = {TAB_COMPILER_MSFT : "/I", "GCC" : "-I", "INTEL" : "-I", "RVCT" : "-I", "NASM" : "-I"}
@@ -169,22 +174,30 @@ class BuildFile(object):
     #
     def __init__(self, AutoGenObject):
         self._AutoGenObject = AutoGenObject
-        self._FileType = gMakeType
 
-    ## Create build file
+        MakePath = AutoGenObject.BuildOption.get('MAKE', {}).get('PATH')
+        if not MakePath:
+            self._FileType = ""
+        elif "nmake" in MakePath:
+            self._FileType = NMAKE_FILETYPE
+        else:
+            self._FileType = "gmake"
+
+        if sys.platform == "win32":
+            self._Platform = WIN32_PLATFORM
+        else:
+            self._Platform = POSIX_PLATFORM
+
+    ## Create build file.
     #
-    #   @param  FileType    Type of build file. Only nmake and gmake are supported now.
+    #  Only nmake and gmake are supported.
     #
-    #   @retval TRUE        The build file is created or re-created successfully
-    #   @retval FALSE       The build file exists and is the same as the one to be generated
+    #  @retval TRUE     The build file is created or re-created successfully.
+    #  @retval FALSE    The build file exists and is the same as the one to be generated.
     #
-    def Generate(self, FileType=gMakeType):
-        if FileType not in self._FILE_NAME_:
-            EdkLogger.error("build", PARAMETER_INVALID, "Invalid build type [%s]" % FileType,
-                            ExtraData="[%s]" % str(self._AutoGenObject))
-        self._FileType = FileType
+    def Generate(self):
         FileContent = self._TEMPLATE_.Replace(self._TemplateDict)
-        FileName = self._FILE_NAME_[FileType]
+        FileName = self.getMakefileName()
         if not os.path.exists(os.path.join(self._AutoGenObject.MakeFileDir, "deps.txt")):
             with open(os.path.join(self._AutoGenObject.MakeFileDir, "deps.txt"),"w+") as fd:
                 fd.write("")
@@ -203,7 +216,7 @@ class BuildFile(object):
     #   @retval     list        The directory creation command list
     #
     def GetCreateDirectoryCommand(self, DirList):
-        return [self._MD_TEMPLATE_[self._FileType] % {'dir':Dir} for Dir in DirList]
+        return [self._MD_TEMPLATE_[self._Platform] % {'dir':Dir} for Dir in DirList]
 
     ## Return a list of directory removal command string
     #
@@ -212,7 +225,7 @@ class BuildFile(object):
     #   @retval     list        The directory removal command list
     #
     def GetRemoveDirectoryCommand(self, DirList):
-        return [self._RD_TEMPLATE_[self._FileType] % {'dir':Dir} for Dir in DirList]
+        return [self._RD_TEMPLATE_[self._Platform] % {'dir':Dir} for Dir in DirList]
 
     def PlaceMacro(self, Path, MacroDefinitions=None):
         if Path.startswith("$("):
@@ -462,11 +475,8 @@ cleanlib:
     # Compose a dict object containing information used to do replacement in template
     @property
     def _TemplateDict(self):
-        if self._FileType not in self._SEP_:
-            EdkLogger.error("build", PARAMETER_INVALID, "Invalid Makefile type [%s]" % self._FileType,
-                            ExtraData="[%s]" % str(self._AutoGenObject))
         MyAgo = self._AutoGenObject
-        Separator = self._SEP_[self._FileType]
+        Separator = self._SEP_[self._Platform]
 
         # break build if no source files and binary files are found
         if len(MyAgo.SourceFileList) == 0 and len(MyAgo.BinaryFileList) == 0:
@@ -628,10 +638,10 @@ cleanlib:
 
         BcTargetList = []
 
-        MakefileName = self._FILE_NAME_[self._FileType]
+        MakefileName = self.getMakefileName()
         LibraryMakeCommandList = []
         for D in self.LibraryBuildDirectoryList:
-            Command = self._MAKE_TEMPLATE_[self._FileType] % {"file":os.path.join(D, MakefileName)}
+            Command = self._MAKE_TEMPLATE_[self._Platform] % {"file":os.path.join(D, MakefileName)}
             LibraryMakeCommandList.append(Command)
 
         package_rel_dir = MyAgo.SourceDir
@@ -683,8 +693,8 @@ cleanlib:
             "separator"                 : Separator,
             "module_tool_definitions"   : ToolsDef,
 
-            "shell_command_code"        : list(self._SHELL_CMD_[self._FileType].keys()),
-            "shell_command"             : list(self._SHELL_CMD_[self._FileType].values()),
+            "shell_command_code"        : list(self._SHELL_CMD_[self._Platform].keys()),
+            "shell_command"             : list(self._SHELL_CMD_[self._Platform].values()),
 
             "module_entry_point"        : ModuleEntryPoint,
             "image_entry_point"         : ImageEntryPoint,
@@ -721,7 +731,7 @@ cleanlib:
                         self.ResultFileList.append(Dst)
                     if '%s :' %(Dst) not in self.BuildTargetList:
                         self.BuildTargetList.append("%s : %s" %(Dst,Src))
-                        self.BuildTargetList.append('\t' + self._CP_TEMPLATE_[self._FileType] %{'Src': Src, 'Dst': Dst})
+                        self.BuildTargetList.append('\t' + self._CP_TEMPLATE_[self._Platform] %{'Src': Src, 'Dst': Dst})
 
             FfsCmdList = Cmd[0]
             for index, Str in enumerate(FfsCmdList):
@@ -1222,7 +1232,7 @@ ${BEGIN}\t-@${create_directory_command}\n${END}\
     # Compose a dict object containing information used to do replacement in template
     @property
     def _TemplateDict(self):
-        Separator = self._SEP_[self._FileType]
+        Separator = self._SEP_[self._Platform]
         MyAgo = self._AutoGenObject
         if self._FileType not in MyAgo.CustomMakefile:
             EdkLogger.error('build', OPTION_NOT_SUPPORTED, "No custom makefile for %s" % self._FileType,
@@ -1252,7 +1262,7 @@ ${BEGIN}\t-@${create_directory_command}\n${END}\
                     ToolsDef.append("%s_%s = %s" % (Tool, Attr, MyAgo.BuildOption[Tool][Attr]))
             ToolsDef.append("")
 
-        MakefileName = self._FILE_NAME_[self._FileType]
+        MakefileName = self.getMakefileName()
         MakefileTemplateDict = {
             "makefile_header"           : self._FILE_HEADER_[self._FileType],
             "makefile_path"             : os.path.join("$(MODULE_BUILD_DIR)", MakefileName),
@@ -1285,8 +1295,8 @@ ${BEGIN}\t-@${create_directory_command}\n${END}\
             "separator"                 : Separator,
             "module_tool_definitions"   : ToolsDef,
 
-            "shell_command_code"        : list(self._SHELL_CMD_[self._FileType].keys()),
-            "shell_command"             : list(self._SHELL_CMD_[self._FileType].values()),
+            "shell_command_code"        : list(self._SHELL_CMD_[self._Platform].keys()),
+            "shell_command"             : list(self._SHELL_CMD_[self._Platform].values()),
 
             "create_directory_command"  : self.GetCreateDirectoryCommand(self.IntermediateDirectoryList),
             "custom_makefile_content"   : CustomMakefile
@@ -1413,7 +1423,7 @@ cleanlib:
     # Compose a dict object containing information used to do replacement in template
     @property
     def _TemplateDict(self):
-        Separator = self._SEP_[self._FileType]
+        Separator = self._SEP_[self._Platform]
 
         MyAgo = self._AutoGenObject
         if "MAKE" not in MyAgo.ToolDefinition or "PATH" not in MyAgo.ToolDefinition["MAKE"]:
@@ -1424,13 +1434,13 @@ cleanlib:
         self.ModuleBuildDirectoryList = self.GetModuleBuildDirectoryList()
         self.LibraryBuildDirectoryList = self.GetLibraryBuildDirectoryList()
 
-        MakefileName = self._FILE_NAME_[self._FileType]
+        MakefileName = self.getMakefileName()
         LibraryMakefileList = []
         LibraryMakeCommandList = []
         for D in self.LibraryBuildDirectoryList:
             D = self.PlaceMacro(D, {"BUILD_DIR":MyAgo.BuildDir})
             Makefile = os.path.join(D, MakefileName)
-            Command = self._MAKE_TEMPLATE_[self._FileType] % {"file":Makefile}
+            Command = self._MAKE_TEMPLATE_[self._Platform] % {"file":Makefile}
             LibraryMakefileList.append(Makefile)
             LibraryMakeCommandList.append(Command)
         self.LibraryMakeCommandList = LibraryMakeCommandList
@@ -1440,7 +1450,7 @@ cleanlib:
         for D in self.ModuleBuildDirectoryList:
             D = self.PlaceMacro(D, {"BUILD_DIR":MyAgo.BuildDir})
             Makefile = os.path.join(D, MakefileName)
-            Command = self._MAKE_TEMPLATE_[self._FileType] % {"file":Makefile}
+            Command = self._MAKE_TEMPLATE_[self._Platform] % {"file":Makefile}
             ModuleMakefileList.append(Makefile)
             ModuleMakeCommandList.append(Command)
 
@@ -1460,8 +1470,8 @@ cleanlib:
 
             "toolchain_tag"             : MyAgo.ToolChain,
             "build_target"              : MyAgo.BuildTarget,
-            "shell_command_code"        : list(self._SHELL_CMD_[self._FileType].keys()),
-            "shell_command"             : list(self._SHELL_CMD_[self._FileType].values()),
+            "shell_command_code"        : list(self._SHELL_CMD_[self._Platform].keys()),
+            "shell_command"             : list(self._SHELL_CMD_[self._Platform].values()),
             "build_architecture_list"   : MyAgo.Arch,
             "architecture"              : MyAgo.Arch,
             "separator"                 : Separator,
@@ -1519,7 +1529,7 @@ class TopLevelMakefile(BuildFile):
     # Compose a dict object containing information used to do replacement in template
     @property
     def _TemplateDict(self):
-        Separator = self._SEP_[self._FileType]
+        Separator = self._SEP_[self._Platform]
 
         # any platform autogen object is ok because we just need common information
         MyAgo = self._AutoGenObject
@@ -1575,10 +1585,10 @@ class TopLevelMakefile(BuildFile):
             else:
                 ExtraOption += " --pcd " + pcdname + '=' + pcd[3]
 
-        MakefileName = self._FILE_NAME_[self._FileType]
+        MakefileName = self.getMakefileName()
         SubBuildCommandList = []
         for A in MyAgo.ArchList:
-            Command = self._MAKE_TEMPLATE_[self._FileType] % {"file":os.path.join("$(BUILD_DIR)", A, MakefileName)}
+            Command = self._MAKE_TEMPLATE_[self._Platform] % {"file":os.path.join("$(BUILD_DIR)", A, MakefileName)}
             SubBuildCommandList.append(Command)
 
         MakefileTemplateDict = {
@@ -1593,8 +1603,8 @@ class TopLevelMakefile(BuildFile):
 
             "toolchain_tag"             : MyAgo.ToolChain,
             "build_target"              : MyAgo.BuildTarget,
-            "shell_command_code"        : list(self._SHELL_CMD_[self._FileType].keys()),
-            "shell_command"             : list(self._SHELL_CMD_[self._FileType].values()),
+            "shell_command_code"        : list(self._SHELL_CMD_[self._Platform].keys()),
+            "shell_command"             : list(self._SHELL_CMD_[self._Platform].values()),
             'arch'                      : list(MyAgo.ArchList),
             "build_architecture_list"   : ','.join(MyAgo.ArchList),
             "separator"                 : Separator,
diff --git a/BaseTools/Source/Python/AutoGen/IncludesAutoGen.py b/BaseTools/Source/Python/AutoGen/IncludesAutoGen.py
index 1ca1798907ef9b46a17d8938bb2f673d5ade662f..ca9e02d19b4afad82241abb8e15845731cabd22f 100644
--- a/BaseTools/Source/Python/AutoGen/IncludesAutoGen.py
+++ b/BaseTools/Source/Python/AutoGen/IncludesAutoGen.py
@@ -2,6 +2,7 @@
 # Build cache intermediate result and state
 #
 # Copyright (c) 2019 - 2020, Intel Corporation. All rights reserved.<BR>
+# Copyright (c) 2020, ARM Limited. All rights reserved.<BR>
 # SPDX-License-Identifier: BSD-2-Clause-Patent
 #
 from Common.caching import cached_property
@@ -12,20 +13,6 @@ from Common.Misc import SaveFileOnChange, PathClass
 from Common.Misc import TemplateString
 import sys
 gIsFileMap = {}
-if sys.platform == "win32":
-    _INCLUDE_DEPS_TEMPLATE = TemplateString('''
-${BEGIN}
-!IF EXIST(${deps_file})
-!INCLUDE ${deps_file}
-!ENDIF
-${END}
-        ''')
-else:
-    _INCLUDE_DEPS_TEMPLATE = TemplateString('''
-${BEGIN}
--include ${deps_file}
-${END}
-        ''')
 
 DEP_FILE_TAIL = "# Updated \n"
 
@@ -59,6 +46,25 @@ class IncludesAutoGen():
 
     def CreateDepsInclude(self):
         deps_file = {'deps_file':self.deps_files}
+
+        MakePath = self.module_autogen.BuildOption.get('MAKE', {}).get('PATH')
+        if not MakePath:
+            EdkLogger.error("build", PARAMETER_MISSING, Message="No Make path available.")
+        elif "nmake" in MakePath:
+            _INCLUDE_DEPS_TEMPLATE = TemplateString('''
+        ${BEGIN}
+        !IF EXIST(${deps_file})
+        !INCLUDE ${deps_file}
+        !ENDIF
+        ${END}
+               ''')
+        else:
+            _INCLUDE_DEPS_TEMPLATE = TemplateString('''
+        ${BEGIN}
+        -include ${deps_file}
+        ${END}
+               ''')
+
         try:
             deps_include_str = _INCLUDE_DEPS_TEMPLATE.Replace(deps_file)
         except Exception as e:
diff --git a/BaseTools/Source/Python/AutoGen/PlatformAutoGen.py b/BaseTools/Source/Python/AutoGen/PlatformAutoGen.py
index 7bd24dad42f8142694659861e03033fa082f2e32..d32178b00c936b9682b2cba5f1209e8efcb5c3e5 100644
--- a/BaseTools/Source/Python/AutoGen/PlatformAutoGen.py
+++ b/BaseTools/Source/Python/AutoGen/PlatformAutoGen.py
@@ -2,6 +2,7 @@
 # Create makefile for MS nmake and GNU make
 #
 # Copyright (c) 2019, Intel Corporation. All rights reserved.<BR>
+#  Copyright (c) 2020, ARM Limited. All rights reserved.<BR>
 # SPDX-License-Identifier: BSD-2-Clause-Patent
 #
 
@@ -106,8 +107,9 @@ class PlatformAutoGen(AutoGen):
         self.BuildDatabase = Workspace.BuildDatabase
         self.DscBuildDataObj = Workspace.Platform
 
-        # flag indicating if the makefile/C-code file has been created or not
-        self.IsMakeFileCreated  = False
+        # MakeFileName is used to get the Makefile name and as a flag
+        # indicating whether the file has been created.
+        self.MakeFileName = ""
 
         self._DynamicPcdList = None    # [(TokenCName1, TokenSpaceGuidCName1), (TokenCName2, TokenSpaceGuidCName2), ...]
         self._NonDynamicPcdList = None # [(TokenCName1, TokenSpaceGuidCName1), (TokenCName2, TokenSpaceGuidCName2), ...]
@@ -191,16 +193,15 @@ class PlatformAutoGen(AutoGen):
         self.CreateLibModuelDirs()
 
     def CreateLibModuelDirs(self):
-        # no need to create makefile for the platform more than once
-        if self.IsMakeFileCreated:
+        # No need to create makefile for the platform more than once.
+        if self.MakeFileName:
             return
 
         # create library/module build dirs for platform
         Makefile = GenMake.PlatformMakefile(self)
         self.LibraryBuildDirectoryList = Makefile.GetLibraryBuildDirectoryList()
         self.ModuleBuildDirectoryList = Makefile.GetModuleBuildDirectoryList()
-
-        self.IsMakeFileCreated = True
+        self.MakeFileName = Makefile.getMakefileName()
 
     @property
     def AllPcdList(self):
diff --git a/BaseTools/Source/Python/build/build.py b/BaseTools/Source/Python/build/build.py
index 1e47e382cba94dc6fba7d8ecad2bc924b02cea1d..d841fefdc502b1ff9e494d6e64aa8b2ae7aa6751 100755
--- a/BaseTools/Source/Python/build/build.py
+++ b/BaseTools/Source/Python/build/build.py
@@ -4,6 +4,7 @@
 #  Copyright (c) 2014, Hewlett-Packard Development Company, L.P.<BR>
 #  Copyright (c) 2007 - 2020, Intel Corporation. All rights reserved.<BR>
 #  Copyright (c) 2018, Hewlett Packard Enterprise Development, L.P.<BR>
+#  Copyright (c) 2020, ARM Limited. All rights reserved.<BR>
 #
 #  SPDX-License-Identifier: BSD-2-Clause-Patent
 #
@@ -736,6 +737,7 @@ class Build():
         self.AutoGenTime    = 0
         self.MakeTime       = 0
         self.GenFdsTime     = 0
+        self.MakeFileName   = ""
         TargetObj = TargetTxtDict()
         ToolDefObj = ToolDefDict((os.path.join(os.getenv("WORKSPACE"),"Conf")))
         self.TargetTxt = TargetObj.Target
@@ -1251,8 +1253,6 @@ class Build():
                                 (AutoGenObject.BuildTarget, AutoGenObject.ToolChain, AutoGenObject.Arch),
                             ExtraData=str(AutoGenObject))
 
-        makefile = GenMake.BuildFile(AutoGenObject)._FILE_NAME_[GenMake.gMakeType]
-
         # run
         if Target == 'run':
             return True
@@ -1278,7 +1278,7 @@ class Build():
                 if not Lib.IsBinaryModule:
                     DirList.append((os.path.join(AutoGenObject.BuildDir, Lib.BuildDir),Lib))
             for Lib, LibAutoGen in DirList:
-                NewBuildCommand = BuildCommand + ['-f', os.path.normpath(os.path.join(Lib, makefile)), 'pbuild']
+                NewBuildCommand = BuildCommand + ['-f', os.path.normpath(os.path.join(Lib, self.MakeFileName)), 'pbuild']
                 LaunchCommand(NewBuildCommand, AutoGenObject.MakeFileDir,LibAutoGen)
             return True
 
@@ -1289,7 +1289,7 @@ class Build():
                 if not Lib.IsBinaryModule:
                     DirList.append((os.path.join(AutoGenObject.BuildDir, Lib.BuildDir),Lib))
             for Lib, LibAutoGen in DirList:
-                NewBuildCommand = BuildCommand + ['-f', os.path.normpath(os.path.join(Lib, makefile)), 'pbuild']
+                NewBuildCommand = BuildCommand + ['-f', os.path.normpath(os.path.join(Lib, self.MakeFileName)), 'pbuild']
                 LaunchCommand(NewBuildCommand, AutoGenObject.MakeFileDir,LibAutoGen)
 
             DirList = []
@@ -1297,7 +1297,7 @@ class Build():
                 if not ModuleAutoGen.IsBinaryModule:
                     DirList.append((os.path.join(AutoGenObject.BuildDir, ModuleAutoGen.BuildDir),ModuleAutoGen))
             for Mod,ModAutoGen in DirList:
-                NewBuildCommand = BuildCommand + ['-f', os.path.normpath(os.path.join(Mod, makefile)), 'pbuild']
+                NewBuildCommand = BuildCommand + ['-f', os.path.normpath(os.path.join(Mod, self.MakeFileName)), 'pbuild']
                 LaunchCommand(NewBuildCommand, AutoGenObject.MakeFileDir,ModAutoGen)
             self.CreateAsBuiltInf()
             if GlobalData.gBinCacheDest:
@@ -1312,7 +1312,7 @@ class Build():
         # cleanlib
         if Target == 'cleanlib':
             for Lib in AutoGenObject.LibraryBuildDirectoryList:
-                LibMakefile = os.path.normpath(os.path.join(Lib, makefile))
+                LibMakefile = os.path.normpath(os.path.join(Lib, self.MakeFileName))
                 if os.path.exists(LibMakefile):
                     NewBuildCommand = BuildCommand + ['-f', LibMakefile, 'cleanall']
                     LaunchCommand(NewBuildCommand, AutoGenObject.MakeFileDir)
@@ -1321,12 +1321,12 @@ class Build():
         # clean
         if Target == 'clean':
             for Mod in AutoGenObject.ModuleBuildDirectoryList:
-                ModMakefile = os.path.normpath(os.path.join(Mod, makefile))
+                ModMakefile = os.path.normpath(os.path.join(Mod, self.MakeFileName))
                 if os.path.exists(ModMakefile):
                     NewBuildCommand = BuildCommand + ['-f', ModMakefile, 'cleanall']
                     LaunchCommand(NewBuildCommand, AutoGenObject.MakeFileDir)
             for Lib in AutoGenObject.LibraryBuildDirectoryList:
-                LibMakefile = os.path.normpath(os.path.join(Lib, makefile))
+                LibMakefile = os.path.normpath(os.path.join(Lib, self.MakeFileName))
                 if os.path.exists(LibMakefile):
                     NewBuildCommand = BuildCommand + ['-f', LibMakefile, 'cleanall']
                     LaunchCommand(NewBuildCommand, AutoGenObject.MakeFileDir)
@@ -2040,10 +2040,10 @@ class Build():
             ModuleBuildDirectoryList = data_pipe.Get("ModuleBuildDirectoryList")
 
             for m_build_dir in LibraryBuildDirectoryList:
-                if not os.path.exists(os.path.join(m_build_dir,GenMake.BuildFile._FILE_NAME_[GenMake.gMakeType])):
+                if not os.path.exists(os.path.join(m_build_dir,self.MakeFileName)):
                     return None
             for m_build_dir in ModuleBuildDirectoryList:
-                if not os.path.exists(os.path.join(m_build_dir,GenMake.BuildFile._FILE_NAME_[GenMake.gMakeType])):
+                if not os.path.exists(os.path.join(m_build_dir,self.MakeFileName)):
                     return None
             Wa = WorkSpaceInfo(
                 workspacedir,active_p,target,toolchain,archlist
@@ -2128,6 +2128,11 @@ class Build():
             Pa.DataPipe.DataContainer = {"Workspace_timestamp": Wa._SrcTimeStamp}
             Pa.DataPipe.DataContainer = {"CommandTarget": self.Target}
             Pa.CreateLibModuelDirs()
+            # Fetch the MakeFileName.
+            self.MakeFileName = Pa.MakeFileName
+            if not self.MakeFileName:
+                self.MakeFileName = Pa.MakeFile
+
             Pa.DataPipe.DataContainer = {"LibraryBuildDirectoryList":Pa.LibraryBuildDirectoryList}
             Pa.DataPipe.DataContainer = {"ModuleBuildDirectoryList":Pa.ModuleBuildDirectoryList}
             Pa.DataPipe.DataContainer = {"FdsCommandDict": Wa.GenFdsCommandDict}
-- 
'Guid(CE165669-3EF3-493F-B85D-6190EE5B9759)'


^ permalink raw reply related	[flat|nested] 5+ messages in thread

* [PATCH v4 2/2] BaseTools: Remove caret in NASM_INC macro
  2020-02-10 10:49 [PATCH v4 0/2] BaseTools: Rationalise makefile generation PierreGondois
  2020-02-10 10:49 ` [PATCH v4 1/2] " PierreGondois
@ 2020-02-10 10:49 ` PierreGondois
  2020-02-12  2:07 ` [edk2-devel] [PATCH v4 0/2] BaseTools: Rationalise makefile generation Bob Feng
       [not found] ` <15F285876DA55E24.22872@groups.io>
  3 siblings, 0 replies; 5+ messages in thread
From: PierreGondois @ 2020-02-10 10:49 UTC (permalink / raw)
  To: devel
  Cc: Pierre Gondois, bob.c.feng, liming.gao, Sami.Mujawar,
	pierre.gondois, nd

From: Pierre Gondois <pierre.gondois@arm.com>

NASM_INC contains the list of directory to include when using
the nasm assembler.

In nmake makefiles, a trailing backslash escapes the newline char
and replaces it with a space ' '. To have a literal trailing
backslash, it must be escaped with a caret '^'. This is not
necessary for GNU makefiles.

On windows platforms, for the NASM_INC macro, a caret escaping
a trailing a backslash was appended to the last included
folder regardless of the makefile type.
For instance, "/Include/" was replaced by "/Include/^\".

This is causing a build failure on windows platforms using
GNU makefiles since the caret '^' doesn't escape any chars in
GNU makefiles and is thus conserved.
"/Include^\" was replaced by "/Include\/" in nmake makefiles,
but remained "/Include/^\" in GNU makefiles.

This patch removes the caret '^' on the build using GNU makefiles.

Signed-off-by: Pierre Gondois <pierre.gondois@arm.com>
---

The changes can be seen at https://github.com/PierreARM/edk2/tree/720_BaseTools_Rationalise_makefile_generation_v4

 BaseTools/Source/Python/AutoGen/GenMake.py | 7 ++++---
 1 file changed, 4 insertions(+), 3 deletions(-)

diff --git a/BaseTools/Source/Python/AutoGen/GenMake.py b/BaseTools/Source/Python/AutoGen/GenMake.py
index f5d15c40ad3196e5ea56f375bec12c38d313c9ee..bbb3c29446f53fa7f2cb61a216a5b119f72c3fbc 100755
--- a/BaseTools/Source/Python/AutoGen/GenMake.py
+++ b/BaseTools/Source/Python/AutoGen/GenMake.py
@@ -612,9 +612,10 @@ cleanlib:
                 IncludePath = self._INC_FLAG_['NASM'] + self.PlaceMacro(P, self.Macros)
                 if IncludePath.endswith(os.sep):
                     IncludePath = IncludePath.rstrip(os.sep)
-                # When compiling .nasm files, need to add a literal backslash at each path
-                # To specify a literal backslash at the end of the line, precede it with a caret (^)
-                if P == MyAgo.IncludePathList[-1] and os.sep == '\\':
+                # When compiling .nasm files, need to add a literal backslash at each path.
+                # In nmake makfiles, a trailing literal backslash must be escaped with a caret ('^').
+                # It is otherwise replaced with a space (' '). This is not necessary for GNU makfefiles.
+                if P == MyAgo.IncludePathList[-1] and self._Platform == WIN32_PLATFORM and self._FileType == NMAKE_FILETYPE:
                     IncludePath = ''.join([IncludePath, '^', os.sep])
                 else:
                     IncludePath = os.path.join(IncludePath, '')
-- 
'Guid(CE165669-3EF3-493F-B85D-6190EE5B9759)'


^ permalink raw reply related	[flat|nested] 5+ messages in thread

* Re: [edk2-devel] [PATCH v4 0/2] BaseTools: Rationalise makefile generation
  2020-02-10 10:49 [PATCH v4 0/2] BaseTools: Rationalise makefile generation PierreGondois
  2020-02-10 10:49 ` [PATCH v4 1/2] " PierreGondois
  2020-02-10 10:49 ` [PATCH v4 2/2] BaseTools: Remove caret in NASM_INC macro PierreGondois
@ 2020-02-12  2:07 ` Bob Feng
       [not found] ` <15F285876DA55E24.22872@groups.io>
  3 siblings, 0 replies; 5+ messages in thread
From: Bob Feng @ 2020-02-12  2:07 UTC (permalink / raw)
  To: devel@edk2.groups.io, pierre.gondois@arm.com
  Cc: Gao, Liming, Sami.Mujawar@arm.com, nd@arm.com

This patch set is good to me.

Reviewed-by: Bob Feng <bob.c.feng@intel.com>


-----Original Message-----
From: devel@edk2.groups.io <devel@edk2.groups.io> On Behalf Of PierreGondois
Sent: Monday, February 10, 2020 6:49 PM
To: devel@edk2.groups.io
Cc: Pierre Gondois <Pierre.Gondois@arm.com>; Feng, Bob C <bob.c.feng@intel.com>; Gao, Liming <liming.gao@intel.com>; Sami.Mujawar@arm.com; pierre.gondois@arm.com; nd@arm.com
Subject: [edk2-devel] [PATCH v4 0/2] BaseTools: Rationalise makefile generation

v4:
  Remove caret in NASM_INC macro:
  I could not reproduce the build fail Bob Feng had when issuing
  the following command:
    build -p UefiCpuPkg\UefiCpuPkg.dsc -a IA32 -t VS2015x86
  Nonetheless, this v4 should only have an impact on the builds
  made on Windows platforms using nmake.

The changes can be seen at https://github.com/PierreARM/edk2/tree/720_BaseTools_Rationalise_makefile_generation_v4

Pierre Gondois (2):
  BaseTools: Rationalise makefile generation
  BaseTools: Remove caret in NASM_INC macro

 BaseTools/Source/Python/AutoGen/GenMake.py         | 145 +++++++++++---------
 BaseTools/Source/Python/AutoGen/IncludesAutoGen.py |  34 +++--  BaseTools/Source/Python/AutoGen/PlatformAutoGen.py |  13 +-
 BaseTools/Source/Python/build/build.py             |  25 ++--
 4 files changed, 120 insertions(+), 97 deletions(-)

--
'Guid(CE165669-3EF3-493F-B85D-6190EE5B9759)'





^ permalink raw reply	[flat|nested] 5+ messages in thread

* Re: [edk2-devel] [PATCH v4 0/2] BaseTools: Rationalise makefile generation
       [not found] ` <15F285876DA55E24.22872@groups.io>
@ 2020-02-12  2:37   ` Bob Feng
  0 siblings, 0 replies; 5+ messages in thread
From: Bob Feng @ 2020-02-12  2:37 UTC (permalink / raw)
  To: devel@edk2.groups.io, Feng, Bob C, pierre.gondois@arm.com
  Cc: Gao, Liming, Sami.Mujawar@arm.com, nd@arm.com

Pushed. 

-----Original Message-----
From: devel@edk2.groups.io <devel@edk2.groups.io> On Behalf Of Bob Feng
Sent: Wednesday, February 12, 2020 10:07 AM
To: devel@edk2.groups.io; pierre.gondois@arm.com
Cc: Gao, Liming <liming.gao@intel.com>; Sami.Mujawar@arm.com; nd@arm.com
Subject: Re: [edk2-devel] [PATCH v4 0/2] BaseTools: Rationalise makefile generation

This patch set is good to me.

Reviewed-by: Bob Feng <bob.c.feng@intel.com>


-----Original Message-----
From: devel@edk2.groups.io <devel@edk2.groups.io> On Behalf Of PierreGondois
Sent: Monday, February 10, 2020 6:49 PM
To: devel@edk2.groups.io
Cc: Pierre Gondois <Pierre.Gondois@arm.com>; Feng, Bob C <bob.c.feng@intel.com>; Gao, Liming <liming.gao@intel.com>; Sami.Mujawar@arm.com; pierre.gondois@arm.com; nd@arm.com
Subject: [edk2-devel] [PATCH v4 0/2] BaseTools: Rationalise makefile generation

v4:
  Remove caret in NASM_INC macro:
  I could not reproduce the build fail Bob Feng had when issuing
  the following command:
    build -p UefiCpuPkg\UefiCpuPkg.dsc -a IA32 -t VS2015x86
  Nonetheless, this v4 should only have an impact on the builds
  made on Windows platforms using nmake.

The changes can be seen at https://github.com/PierreARM/edk2/tree/720_BaseTools_Rationalise_makefile_generation_v4

Pierre Gondois (2):
  BaseTools: Rationalise makefile generation
  BaseTools: Remove caret in NASM_INC macro

 BaseTools/Source/Python/AutoGen/GenMake.py         | 145 +++++++++++---------
 BaseTools/Source/Python/AutoGen/IncludesAutoGen.py |  34 +++--  BaseTools/Source/Python/AutoGen/PlatformAutoGen.py |  13 +-
 BaseTools/Source/Python/build/build.py             |  25 ++--
 4 files changed, 120 insertions(+), 97 deletions(-)

--
'Guid(CE165669-3EF3-493F-B85D-6190EE5B9759)'








^ permalink raw reply	[flat|nested] 5+ messages in thread

end of thread, other threads:[~2020-02-12  2:37 UTC | newest]

Thread overview: 5+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2020-02-10 10:49 [PATCH v4 0/2] BaseTools: Rationalise makefile generation PierreGondois
2020-02-10 10:49 ` [PATCH v4 1/2] " PierreGondois
2020-02-10 10:49 ` [PATCH v4 2/2] BaseTools: Remove caret in NASM_INC macro PierreGondois
2020-02-12  2:07 ` [edk2-devel] [PATCH v4 0/2] BaseTools: Rationalise makefile generation Bob Feng
     [not found] ` <15F285876DA55E24.22872@groups.io>
2020-02-12  2:37   ` Bob Feng

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox