From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mga07.intel.com (mga07.intel.com [134.134.136.100]) by mx.groups.io with SMTP id smtpd.web12.8212.1591096546765802495 for ; Tue, 02 Jun 2020 04:15:47 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=@intel.onmicrosoft.com header.s=selector2-intel-onmicrosoft-com header.b=jenoItTq; spf=pass (domain: intel.com, ip: 134.134.136.100, mailfrom: bob.c.feng@intel.com) IronPort-SDR: OiBUNNlIP4OFxxFdM8ijNx4u/atjTzgWHszVATfkENjcGGg6oHX/qBzAZhiNTtK/KIYAbl+QRQ e+uKpZWNMQMQ== X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga008.jf.intel.com ([10.7.209.65]) by orsmga105.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 02 Jun 2020 04:15:45 -0700 IronPort-SDR: YWkQDSIwA1FA6jul4M+Mkr/P4fzM+zcScEeYTX5SejdYkEXWhXWx2JKN8h96gTNPDeWWiN5B+4 8Mb7pyK861ng== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.73,464,1583222400"; d="scan'208";a="304200500" Received: from orsmsx105.amr.corp.intel.com ([10.22.225.132]) by orsmga008.jf.intel.com with ESMTP; 02 Jun 2020 04:15:45 -0700 Received: from orsmsx114.amr.corp.intel.com (10.22.240.10) by ORSMSX105.amr.corp.intel.com (10.22.225.132) with Microsoft SMTP Server (TLS) id 14.3.439.0; Tue, 2 Jun 2020 04:15:45 -0700 Received: from ORSEDG002.ED.cps.intel.com (10.7.248.5) by ORSMSX114.amr.corp.intel.com (10.22.240.10) with Microsoft SMTP Server (TLS) id 14.3.439.0; Tue, 2 Jun 2020 04:15:45 -0700 Received: from NAM12-DM6-obe.outbound.protection.outlook.com (104.47.59.176) by edgegateway.intel.com (134.134.137.101) with Microsoft SMTP Server (TLS) id 14.3.439.0; Tue, 2 Jun 2020 04:15:45 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=LQ1W8bdENDcqcCRidUCeZogWCpUIBBdW3IrUOwappDd2+11ozIWl2NOE6VPO4BzeEntNQAsPhmEx/7J9J6nZL5Ib69ms4AI0uS/HDwOIPgjL3ngJiIgAsYOFDUJh7gGHWxdIxIwipmQczhN2PJwechQHyhC2ff2YS24Ke9Z8uvT10QwUcSJ2ws1mtSK++SsFz9d50Flj05MLrnj4HeobV7610lw4+GmSUfmtvbWAJA8OPEPojj1BU573BIe6CSvfqgAMaixti+3mHSzvlKnq4wfDoHpSCGw3+8wFxrod716ur2Qwi6qqJ48DaewzUyi0CMkdkXrAJF15F6/myXp51w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=2hxHSQjcxfJXbrdrXDzKpOXex7mwp0YLi8TUyMa6xxQ=; b=LrD1wMxGo8bulvaKtyk7RcvjyPBeAVg8g1NzACBLqhFtRaURjPiobbnGD5xocj1kk/7j6vdpuSoPs67K1AyqJvSueGlslEF231IOkV3l0dANd+iW7hUiOttVNAKATHU1hmqfZ50o7FH4XYG5ofKxZPMUQBXFdrRt6F0JNS/1tzQUZrt69QsIcmRCWhd+eq7/ijPEgy6RHjetgW8s9D/Xdcs72461uHYOEi43sR6zadJ3+O6G2Y9Vqr1+mb7jLwvGMQ1j4+8+jPVmch3OoJysjCssyqCfSqabwl8v3Iw/v0h0oslV8wyrd8QAjD0wZvY5c+LLJi6XpIAme0EBD/iykQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com; dkim=pass header.d=intel.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=intel.onmicrosoft.com; s=selector2-intel-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=2hxHSQjcxfJXbrdrXDzKpOXex7mwp0YLi8TUyMa6xxQ=; b=jenoItTqDCy+ooy6CqpIEXFcDCRTRvr+RQbt951fFsINezr/zoa/FR3R60c1uxLmLoteeERMarElzo2Lc1M8KJwu2FTXPBRIzYoFuBhEsVHSKBOTarxp8WeGYAsuPXtShCt+Mbo+nvra3V9CxSWb/tY9u2D74WKf/pgeSP0PgHA= Received: from BN6PR11MB0068.namprd11.prod.outlook.com (2603:10b6:405:69::17) by BN6PR11MB2001.namprd11.prod.outlook.com (2603:10b6:404:49::12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3045.25; Tue, 2 Jun 2020 11:15:42 +0000 Received: from BN6PR11MB0068.namprd11.prod.outlook.com ([fe80::2d53:d47c:1269:d601]) by BN6PR11MB0068.namprd11.prod.outlook.com ([fe80::2d53:d47c:1269:d601%6]) with mapi id 15.20.3045.022; Tue, 2 Jun 2020 11:15:42 +0000 From: "Bob Feng" To: "devel@edk2.groups.io" , "pierre.gondois@arm.com" CC: "Gao, Liming" , "Sami.Mujawar@arm.com" , "Tomas.Pilar@arm.com" , "nd@arm.com" Subject: Re: [edk2-devel] [PATCH v1 1/3] BaseTools: Generate multiple rules when multiple output files Thread-Topic: [edk2-devel] [PATCH v1 1/3] BaseTools: Generate multiple rules when multiple output files Thread-Index: AQHWLR5Z/G9BS65hW0CiAqbPj8x3x6jFQTvA Date: Tue, 2 Jun 2020 11:15:42 +0000 Message-ID: References: <20200518141120.66132-1-pierre.gondois@arm.com> <20200518141120.66132-2-pierre.gondois@arm.com> In-Reply-To: <20200518141120.66132-2-pierre.gondois@arm.com> Accept-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: edk2.groups.io; dkim=none (message not signed) header.d=none;edk2.groups.io; dmarc=none action=none header.from=intel.com; x-originating-ip: [192.55.46.36] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 588eb4f7-64dd-4dce-341c-08d806e649f5 x-ms-traffictypediagnostic: BN6PR11MB2001: x-ms-exchange-transport-forked: True x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:8882; x-forefront-prvs: 0422860ED4 x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: 4JIqFLdJWYcgY8MmYVqqnJKfc96TpcdnexTB26xX/+1TDav14HHhubg7Ub3Jduqalb73w9f/ycXf2Dh3prZuovZAF+YSGuJgVL9NgSJMVsXokHFB4z7WI5vJ30jQBZLC8u+X3Bnvg8jwqhEDeqs4IrLVYWmQVqJEeNLAXs8WjRmN+QqaJF9/xw6s1VzY78FvWaYTGLsjlCm5KiX0Bzh0Bt55mSqtGwurz1wVOV8sNY+a5pfaSDgPk8SWJSCDxAsHGcOgYhXA1UJHeCZxG4wWDkMBy667jWr6vqFwGHvwMVyWOoDzRWuJp0rftrR1B475Nu+zQGQUxqA48eClYSYkXEk1UIw8vW/vYwuqCASRN1BF0N1KP5OFV9kNjwDf7nDO2LlxLQKflSE6aKLlvi7Z/w== x-forefront-antispam-report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BN6PR11MB0068.namprd11.prod.outlook.com;PTR:;CAT:NONE;SFTY:;SFS:(396003)(39860400002)(136003)(346002)(376002)(366004)(71200400001)(6506007)(53546011)(186003)(64756008)(66946007)(26005)(66476007)(86362001)(66446008)(66556008)(316002)(54906003)(7696005)(110136005)(76116006)(2906002)(83380400001)(8676002)(8936002)(55016002)(9686003)(52536014)(5660300002)(966005)(478600001)(33656002)(4326008);DIR:OUT;SFP:1102; x-ms-exchange-antispam-messagedata: i1s9ZVRYtUFiY5zFix4LVGW+88/UEtbcwKF9nQTVtE4D6a3rdB5zu/vmpOFVjLt0ggln7vxcfFCKw9sV3MeR7ZYaXEJ0SkHmy29Y//+vIik6KabghrpLMZW6JrlycJeIM1geO2dWlMpel4mSXOC2aGs7t+BZJEGg3c6gRzKJoX4kI+JIOTm5MKCqofs6ueQiE99ySJ1zm60M9iwRA49ELLd6f9r0S9jeG0wVGYj4IHWCYP80eX/OdC6uk8UyjpbXo4zuJg7mRycU5FriOgSFarbsBV0Osc/rz5bmatZaW1TGcX5K97/GtJjlTqSFLwXXP06oKoSRtmmD02lJVJcRF/JRopKJJ48mookfmQ2ABb09+t6SUYHEYcOwTA4cwhquYtIjlNMfQpAus7ToNsGVxOPApvcK2GaGhbB1L0AVHk1zCNx6j2F6XewUoyTpJGvmp0rfLwxWm4EhTVeMuhtdBhFmO1eoE2qiHFaSyWWmm2A= MIME-Version: 1.0 X-MS-Exchange-CrossTenant-Network-Message-Id: 588eb4f7-64dd-4dce-341c-08d806e649f5 X-MS-Exchange-CrossTenant-originalarrivaltime: 02 Jun 2020 11:15:42.4597 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 46c98d88-e344-4ed4-8496-4ed7712e255d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: NqtGF+NXjPHSHlrMGGpB2I3+gq6maAgdbC9tifdbBwnOspYUcEZK6Zcx1CbQBuZ9x7GuuueOAl3ZmpTDAscp/A== X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN6PR11MB2001 Return-Path: bob.c.feng@intel.com X-OriginatorOrg: intel.com Content-Language: en-US Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable My comments are inline marked as [Bob]. Thanks, Bob -----Original Message----- From: devel@edk2.groups.io On Behalf Of PierreGondo= is Sent: Monday, May 18, 2020 10:11 PM To: devel@edk2.groups.io Cc: Pierre Gondois ; Feng, Bob C ; Gao, Liming ; Sami.Mujawar@arm.com; Tomas.Pila= r@arm.com; nd@arm.com Subject: [edk2-devel] [PATCH v1 1/3] BaseTools: Generate multiple rules wh= en multiple output files From: Pierre Gondois BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=3D2425 This patch modifies the Makefile generation not to stop adding Makfile rul= es when the first final target is found. E.g.: If the following rules are described in build_rule.txt: -[Rule1]: .X files generate .Y and .Z files; -[Rule2]: .Z files generate .Z1 files. Currently, if a File1.X file was part of the sources of a module, only [Ru= le1] would be generated in the Makefile. Indeed, there are no rules to apply to .Y files: .Y files are a final targ= et. However, there is still [Rule2] to apply to .Z files. [Bob] I think currently a rule's output file will be added back to source = file list, and in the later loop, that output file will be handled by anoth= er rule. Doesn't that algorithm handle your case above? This patch also adds a dependency between the first ouput file of a rule a= nd the other output files. For instance, with the same example as above, File1.Y and File1.Z are gene= rated by the following rule: File1.Y: File1.X and the new dependency is: File1.Z: File1.Y This is necessary to keep a dependency order during the execution of the M= akefile. Indeed, .Y and .Z files are generated by the execution of a common= set of commands, and without this rule, there is no explicit dependency re= lation between them. [Bob] If there are 3 output files, for example:=20 A B C Will the dependency relationship be=20 B: A C: B Signed-off-by: Pierre Gondois --- The changes can be seen at: https://github.com/PierreARM/edk2/tree/803_Com= pile_AML_bytecode_array_into_OBJ_file_v1 Notes: v1: - Generate multiple rules when multiple output files are specified in the build_rule.txt file. [Pierre] BaseTools/Source/Python/AutoGen/GenMake.py | 6 +++ BaseTools/Source/Python/AutoGen/ModuleAutoGen.py | 40 ++++++++++---------= - 2 files changed, 25 insertions(+), 21 deletions(-) diff --git a/BaseTools/Source/Python/AutoGen/GenMake.py b/BaseTools/Source= /Python/AutoGen/GenMake.py index bbb3c29446f53fa7f2cb61a216a5b119f72c3fbc..0314d0ea34d99a014379e8d30c= 46ac0f0a7068ce 100755 --- a/BaseTools/Source/Python/AutoGen/GenMake.py +++ b/BaseTools/Source/Python/AutoGen/GenMake.py @@ -1054,6 +1054,12 @@ cleanlib: TargetDict =3D {"target": self.PlaceMacro(T.Target.Pa= th, self.Macros), "cmd": "\n\t".join(T.Commands),"deps": Deps} self.BuildTargetList.append(self._BUILD_TARGET_TEMPLA= TE.Replace(TargetDict)) =20 + # Add a Makefile rule for targets generating multiple= files. + # The main output is a prerequisite for the other out= put files. + for i in T.Outputs[1:]: + AnnexeTargetDict =3D {"target": self.PlaceMacro(i= .Path, self.Macros), "cmd": "", "deps": self.PlaceMacro(T.Target.Path, self= .Macros)} + + self.BuildTargetList.append(self._BUILD_TARGET_TEMPLATE.Replace(Annexe + TargetDict)) + def ParserCCodeFile(self, T, Type, CmdSumDict, CmdTargetDict, CmdCppD= ict, DependencyDict): if not CmdSumDict: for item in self._AutoGenObject.Targets[Type]: diff --git a/BaseTools/Source/Python/AutoGen/ModuleAutoGen.py b/BaseTools/= Source/Python/AutoGen/ModuleAutoGen.py index aad591de65f086043d55aeea5661f59c53792e7c..82facdb2774c56833718317f44= 14e0c1861de47f 100755 --- a/BaseTools/Source/Python/AutoGen/ModuleAutoGen.py +++ b/BaseTools/Source/Python/AutoGen/ModuleAutoGen.py @@ -860,7 +860,8 @@ class ModuleAutoGen(AutoGen): SubDirectory =3D os.path.join(self.OutputDir, File.SubDir) if not os.path.exists(SubDirectory): CreateDirectory(SubDirectory) - LastTarget =3D None + TargetList =3D set() + FinalTargetName =3D set() RuleChain =3D set() SourceList =3D [File] Index =3D 0 @@ -872,7 +873,7 @@ class ModuleAutoGen(AutoGen): while Index < len(SourceList): Source =3D SourceList[Index] Index =3D Index + 1 - + FileType =3D TAB_UNKNOWN_FILE if Source !=3D File: CreateDirectory(Source.Dir) =20 @@ -881,34 +882,27 @@ class ModuleAutoGen(AutoGen): if not self.IsLibrary: continue RuleObject =3D self.BuildRules[TAB_DEFAULT_BINARY_FILE] - elif FileType in self.BuildRules: - RuleObject =3D self.BuildRules[FileType] elif Source.Ext in self.BuildRules: RuleObject =3D self.BuildRules[Source.Ext] else: - # stop at no more rules - if LastTarget: - self._FinalBuildTargetList.add(LastTarget) - break - + # No more rule to apply: Source is a final target. + FinalTargetName.add(Source) + continue FileType =3D RuleObject.SourceFileType self._FileTypes[FileType].add(Source) =20 # stop at STATIC_LIBRARY for library if self.IsLibrary and FileType =3D=3D TAB_STATIC_LIBRARY: - if LastTarget: - self._FinalBuildTargetList.add(LastTarget) - break + FinalTargetName.add(Source) + continue =20 Target =3D RuleObject.Apply(Source, self.BuildRuleOrder) if not Target: - if LastTarget: - self._FinalBuildTargetList.add(LastTarget) - break - elif not Target.Outputs: - # Only do build for target with outputs - self._FinalBuildTargetList.add(Target) + # No Target: Source is a final target. + FinalTargetName.add(Source) + continue =20 + TargetList.add(Target) self._BuildTargets[FileType].add(Target) =20 if not Source.IsBinary and Source =3D=3D File: @@ -916,12 +910,16 @@ class ModuleAutoGen(AutoGen): =20 # to avoid cyclic rule if FileType in RuleChain: - break + EdkLogger.error("build", ERROR_STATEMENT, "Cyclic=20 + dependency detected while generating rule for %s" % str(Source)) =20 RuleChain.add(FileType) SourceList.extend(Target.Outputs) - LastTarget =3D Target - FileType =3D TAB_UNKNOWN_FILE + + # For each final target, retrieve the TargetDescBlock instance. + for FTargetName in FinalTargetName: + for Target in TargetList: + if FTargetName =3D=3D Target.Target: + self._FinalBuildTargetList.add(Target) =20 @cached_property def Targets(self): -- 'Guid(CE165669-3EF3-493F-B85D-6190EE5B9759)'