Re: [edk2-devel] [PATCH v1 1/3] BaseTools: Generate multiple rules when multiple output files

["Bob Feng" <bob.c.feng@intel.com>]

My comments are inline marked as [Bob].

Thanks,
Bob

-----Original Message-----
From: devel@edk2.groups.io <devel@edk2.groups.io> On Behalf Of PierreGondois
Sent: Monday, May 18, 2020 10:11 PM
To: devel@edk2.groups.io
Cc: Pierre Gondois <Pierre.Gondois@arm.com>; Feng, Bob C <bob.c.feng@intel.com>; Gao, Liming <liming.gao@intel.com>; Sami.Mujawar@arm.com; Tomas.Pilar@arm.com; nd@arm.com
Subject: [edk2-devel] [PATCH v1 1/3] BaseTools: Generate multiple rules when multiple output files

From: Pierre Gondois <pierre.gondois@arm.com>

BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=2425

This patch modifies the Makefile generation not to stop adding Makfile rules when the first final target is found.
E.g.:
If the following rules are described in build_rule.txt:
 -[Rule1]: .X files generate .Y and .Z files;
 -[Rule2]: .Z files generate .Z1 files.
Currently, if a File1.X file was part of the sources of a module, only [Rule1] would be generated in the Makefile.
Indeed, there are no rules to apply to .Y files: .Y files are a final target. However, there is still [Rule2] to apply to .Z files.

[Bob] I think currently a rule's output file will be added back to source file list, and in the later loop, that output file will be handled by another rule.  Doesn't that algorithm handle your case above?

This patch also adds a dependency between the first ouput file of a rule and the other output files.
For instance, with the same example as above, File1.Y and File1.Z are generated by the following rule:
File1.Y: File1.X
    <Generate File1.Y>
    <Generate File1.Z>

and the new dependency is:
File1.Z: File1.Y

This is necessary to keep a dependency order during the execution of the Makefile. Indeed, .Y and .Z files are generated by the execution of a common set of commands, and without this rule, there is no explicit dependency relation between them.

[Bob] If there are 3 output files, for example: 
<OutputFile>
        A
        B
        C
Will the dependency relationship be 
B: A
C: B

Signed-off-by: Pierre Gondois <Pierre.Gondois@arm.com>
---

The changes can be seen at: https://github.com/PierreARM/edk2/tree/803_Compile_AML_bytecode_array_into_OBJ_file_v1

Notes:
    v1:
     - Generate multiple rules when multiple output files
       are specified in the build_rule.txt file. [Pierre]

 BaseTools/Source/Python/AutoGen/GenMake.py       |  6 +++
 BaseTools/Source/Python/AutoGen/ModuleAutoGen.py | 40 ++++++++++----------
 2 files changed, 25 insertions(+), 21 deletions(-)

diff --git a/BaseTools/Source/Python/AutoGen/GenMake.py b/BaseTools/Source/Python/AutoGen/GenMake.py
index bbb3c29446f53fa7f2cb61a216a5b119f72c3fbc..0314d0ea34d99a014379e8d30c46ac0f0a7068ce 100755
--- a/BaseTools/Source/Python/AutoGen/GenMake.py
+++ b/BaseTools/Source/Python/AutoGen/GenMake.py
@@ -1054,6 +1054,12 @@ cleanlib:
                     TargetDict = {"target": self.PlaceMacro(T.Target.Path, self.Macros), "cmd": "\n\t".join(T.Commands),"deps": Deps}
                     self.BuildTargetList.append(self._BUILD_TARGET_TEMPLATE.Replace(TargetDict))
 
+                    # Add a Makefile rule for targets generating multiple files.
+                    # The main output is a prerequisite for the other output files.
+                    for i in T.Outputs[1:]:
+                        AnnexeTargetDict = {"target": self.PlaceMacro(i.Path, self.Macros), "cmd": "", "deps": self.PlaceMacro(T.Target.Path, self.Macros)}
+
+ self.BuildTargetList.append(self._BUILD_TARGET_TEMPLATE.Replace(Annexe
+ TargetDict))
+
     def ParserCCodeFile(self, T, Type, CmdSumDict, CmdTargetDict, CmdCppDict, DependencyDict):
         if not CmdSumDict:
             for item in self._AutoGenObject.Targets[Type]:
diff --git a/BaseTools/Source/Python/AutoGen/ModuleAutoGen.py b/BaseTools/Source/Python/AutoGen/ModuleAutoGen.py
index aad591de65f086043d55aeea5661f59c53792e7c..82facdb2774c56833718317f4414e0c1861de47f 100755
--- a/BaseTools/Source/Python/AutoGen/ModuleAutoGen.py
+++ b/BaseTools/Source/Python/AutoGen/ModuleAutoGen.py
@@ -860,7 +860,8 @@ class ModuleAutoGen(AutoGen):
         SubDirectory = os.path.join(self.OutputDir, File.SubDir)
         if not os.path.exists(SubDirectory):
             CreateDirectory(SubDirectory)
-        LastTarget = None
+        TargetList = set()
+        FinalTargetName = set()
         RuleChain = set()
         SourceList = [File]
         Index = 0
@@ -872,7 +873,7 @@ class ModuleAutoGen(AutoGen):
         while Index < len(SourceList):
             Source = SourceList[Index]
             Index = Index + 1
-
+            FileType = TAB_UNKNOWN_FILE
             if Source != File:
                 CreateDirectory(Source.Dir)
 
@@ -881,34 +882,27 @@ class ModuleAutoGen(AutoGen):
                 if not self.IsLibrary:
                     continue
                 RuleObject = self.BuildRules[TAB_DEFAULT_BINARY_FILE]
-            elif FileType in self.BuildRules:
-                RuleObject = self.BuildRules[FileType]
             elif Source.Ext in self.BuildRules:
                 RuleObject = self.BuildRules[Source.Ext]
             else:
-                # stop at no more rules
-                if LastTarget:
-                    self._FinalBuildTargetList.add(LastTarget)
-                break
-
+                # No more rule to apply: Source is a final target.
+                FinalTargetName.add(Source)
+                continue
             FileType = RuleObject.SourceFileType
             self._FileTypes[FileType].add(Source)
 
             # stop at STATIC_LIBRARY for library
             if self.IsLibrary and FileType == TAB_STATIC_LIBRARY:
-                if LastTarget:
-                    self._FinalBuildTargetList.add(LastTarget)
-                break
+                FinalTargetName.add(Source)
+                continue
 
             Target = RuleObject.Apply(Source, self.BuildRuleOrder)
             if not Target:
-                if LastTarget:
-                    self._FinalBuildTargetList.add(LastTarget)
-                break
-            elif not Target.Outputs:
-                # Only do build for target with outputs
-                self._FinalBuildTargetList.add(Target)
+                # No Target: Source is a final target.
+                FinalTargetName.add(Source)
+                continue
 
+            TargetList.add(Target)
             self._BuildTargets[FileType].add(Target)
 
             if not Source.IsBinary and Source == File:
@@ -916,12 +910,16 @@ class ModuleAutoGen(AutoGen):
 
             # to avoid cyclic rule
             if FileType in RuleChain:
-                break
+                EdkLogger.error("build", ERROR_STATEMENT, "Cyclic 
+ dependency detected while generating rule for %s" % str(Source))
 
             RuleChain.add(FileType)
             SourceList.extend(Target.Outputs)
-            LastTarget = Target
-            FileType = TAB_UNKNOWN_FILE
+
+        # For each final target, retrieve the TargetDescBlock instance.
+        for FTargetName in FinalTargetName:
+            for Target in TargetList:
+                if FTargetName == Target.Target:
+                    self._FinalBuildTargetList.add(Target)
 
     @cached_property
     def Targets(self):
--
'Guid(CE165669-3EF3-493F-B85D-6190EE5B9759)'