From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from EUR05-AM6-obe.outbound.protection.outlook.com (EUR05-AM6-obe.outbound.protection.outlook.com [40.107.22.46]) by mx.groups.io with SMTP id smtpd.web11.10058.1591103046345086955 for ; Tue, 02 Jun 2020 06:04:07 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=@armh.onmicrosoft.com header.s=selector2-armh-onmicrosoft-com header.b=mKH3Ncy5; spf=pass (domain: arm.com, ip: 40.107.22.46, mailfrom: pierre.gondois@arm.com) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector2-armh-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=exV3EGlnNrnD3e4Rbn1ItsWTnLy0NEP6TyTTEQZs/3E=; b=mKH3Ncy57ucy7GlV8axRvTMHarnvQnilqGU59UYB6bgbb18pAEJNAjgx8UsjRTM/C9Pmygcq1lfO7Nxo8hvb6f6kfF7uY8tmCZ0OQvVIW93VnHJC2i2cWF/ppB0Voughe/l/36Wy0BdK47mH/c79I5YIiVYRxc7UuTUvVH+iK30= Received: from AM6PR05CA0015.eurprd05.prod.outlook.com (2603:10a6:20b:2e::28) by VI1PR0802MB2463.eurprd08.prod.outlook.com (2603:10a6:800:bb::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3045.17; Tue, 2 Jun 2020 13:04:03 +0000 Received: from AM5EUR03FT018.eop-EUR03.prod.protection.outlook.com (2603:10a6:20b:2e:cafe::c1) by AM6PR05CA0015.outlook.office365.com (2603:10a6:20b:2e::28) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3045.19 via Frontend Transport; Tue, 2 Jun 2020 13:04:01 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123) smtp.mailfrom=arm.com; edk2.groups.io; dkim=pass (signature was verified) header.d=armh.onmicrosoft.com;edk2.groups.io; dmarc=bestguesspass action=none header.from=arm.com; Received-SPF: Pass (protection.outlook.com: domain of arm.com designates 63.35.35.123 as permitted sender) receiver=protection.outlook.com; client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com; Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by AM5EUR03FT018.mail.protection.outlook.com (10.152.16.114) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3021.23 via Frontend Transport; Tue, 2 Jun 2020 13:04:00 +0000 Received: ("Tessian outbound cff7dd4de28a:v57"); Tue, 02 Jun 2020 13:04:00 +0000 X-CR-MTA-TID: 64aa7808 Received: from 616d6a067058.1 by 64aa7808-outbound-1.mta.getcheckrecipient.com id 33030BCA-46CE-4A6D-9ABC-88E9F28595A9.1; Tue, 02 Jun 2020 13:03:55 +0000 Received: from EUR05-DB8-obe.outbound.protection.outlook.com by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 616d6a067058.1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384); Tue, 02 Jun 2020 13:03:55 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=b6FkWojtY/asbRazys/Y8QF50VmEjj5B1MM5R2JCY+qQ9CoeREGCQT/XqdOvGi9OclnAu4xmOHrp6ET0I0l6pAY+v9XdQIncS16dt9Aa0ITNsU9aMEoxStDakjVTbmvlsLn9PscTgX7E+O3Lu/KrXsAa8ZlpuuBUdzecOATP5SL/bCLHN5tx7Jv//Ihqya95fOpjrrspm1RRh141SIwN5Cqv0RJRqwxTIf7uWCW+lfb3fLMEePj029lt9k2wKlkEkuIG6KaT5FFkayUZA2A6bwrr7qv629uYOeoYjvPPdzHzhTd+tDVgNjOCp1ewHxcgyTq6J3XeOVhe9k7K7tOCdg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=exV3EGlnNrnD3e4Rbn1ItsWTnLy0NEP6TyTTEQZs/3E=; b=ajARAgU+3nCo4oHPv2iqawsr1T342TlAbqO6EnNC6D40eMc/hqcSYTJCp9+XPPFl0SBAFBb71T0IF3cxALbiZydntgSWfHcAYep5DIHlA0YXVrx2oCbPQUBhdW3nrAKscxRkYXveLnYD5xor1UUhbPdUUksmbnmj25wslsY4tIKusRI1WiGaipiNDFHtPNhFKvDfBmzptPuhMfzxMuFxkHz4E0tga88Ni4lRHAe3KIzZ2Cqxm7wkfZ3UMC8Ve0XlCFWYmGHZHz1a82+vcgjobmj2LB1p4AGfjCiK060z+gzuneYWYYnW3OcAVy8hDfCfiTpslNiRNDfBezwfM5n0Kg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass header.d=arm.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector2-armh-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=exV3EGlnNrnD3e4Rbn1ItsWTnLy0NEP6TyTTEQZs/3E=; b=mKH3Ncy57ucy7GlV8axRvTMHarnvQnilqGU59UYB6bgbb18pAEJNAjgx8UsjRTM/C9Pmygcq1lfO7Nxo8hvb6f6kfF7uY8tmCZ0OQvVIW93VnHJC2i2cWF/ppB0Voughe/l/36Wy0BdK47mH/c79I5YIiVYRxc7UuTUvVH+iK30= Received: from DB7PR08MB3113.eurprd08.prod.outlook.com (2603:10a6:5:20::21) by DB7PR08MB3530.eurprd08.prod.outlook.com (2603:10a6:10:4a::30) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3045.21; Tue, 2 Jun 2020 13:03:54 +0000 Received: from DB7PR08MB3113.eurprd08.prod.outlook.com ([fe80::4178:71bd:a2a4:78bf]) by DB7PR08MB3113.eurprd08.prod.outlook.com ([fe80::4178:71bd:a2a4:78bf%7]) with mapi id 15.20.3066.018; Tue, 2 Jun 2020 13:03:53 +0000 From: "PierreGondois" To: "Feng, Bob C" , "devel@edk2.groups.io" CC: "Gao, Liming" , Sami Mujawar , Tomas Pilar , nd Subject: Re: [edk2-devel] [PATCH v1 1/3] BaseTools: Generate multiple rules when multiple output files Thread-Topic: [edk2-devel] [PATCH v1 1/3] BaseTools: Generate multiple rules when multiple output files Thread-Index: AQHWLR5DvrgOttPYDUayOoMOpRHh2qjFRBEAgAAUpFA= Date: Tue, 2 Jun 2020 13:03:53 +0000 Message-ID: References: <20200518141120.66132-1-pierre.gondois@arm.com> <20200518141120.66132-2-pierre.gondois@arm.com> In-Reply-To: Accept-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ts-tracking-id: 80842822-1ea6-4f53-941f-c7eb4942192f.0 x-checkrecipientchecked: true Authentication-Results-Original: intel.com; dkim=none (message not signed) header.d=none;intel.com; dmarc=none action=none header.from=arm.com; x-originating-ip: [217.140.106.52] x-ms-publictraffictype: Email X-MS-Office365-Filtering-HT: Tenant X-MS-Office365-Filtering-Correlation-Id: 66293703-7516-453f-c196-08d806f56b6f x-ms-traffictypediagnostic: DB7PR08MB3530:|VI1PR0802MB2463: x-ms-exchange-transport-forked: True X-Microsoft-Antispam-PRVS: x-checkrecipientrouted: true nodisclaimer: true x-ms-oob-tlc-oobclassifiers: OLM:10000;OLM:10000; x-forefront-prvs: 0422860ED4 X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam-Untrusted: BCL:0; X-Microsoft-Antispam-Message-Info-Original: P7sf5fsElPqswCBQsWEkm2+rJA6/gv9lLxNh6PBAaOs7nib/s0omBwzzUk64Y2dKu4V+Oaq9thaILVh/5xfLvPrhXncvSzn9gJl6bsQWkWPvmgjbqLEGJotbjmif2HQzHQpKin7LG7v6DgqjTTzMg5UJHDQHu2yirJnuQdu+3EIGR98UtCojdCamMDTkECJPknapa5qr2moHd5xCdQId7AfkN3x5vvyy3rw/UWt1U5z0gUfZf7CLTaApo6sNOf0NOeqCGBe0/PHJqx3pplwjj2fxHk7cmd6RCiJe23VnOBTDclSCmn6qL7PWY2R6bVbrP1ciLDMPbKTgfE2S0gNEudbYVAk8UHyL1NwaSr/BWM4Q6JzmBS/s4rPrxfjJcLigbHdqlWr3+ORzixE4g66sfQ== X-Forefront-Antispam-Report-Untrusted: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DB7PR08MB3113.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFTY:;SFS:(4636009)(366004)(346002)(136003)(396003)(39860400002)(376002)(4326008)(9686003)(64756008)(66946007)(66476007)(66556008)(55016002)(5660300002)(66446008)(53546011)(76116006)(6506007)(110136005)(26005)(54906003)(316002)(2906002)(33656002)(7696005)(186003)(52536014)(71200400001)(86362001)(8936002)(966005)(478600001)(83380400001)(8676002);DIR:OUT;SFP:1101; x-ms-exchange-antispam-messagedata: vQkKFSRNO1gIDFblXGMLyBMcoN/PdE0pGvT/l2hdbAsNHrD/6AyBur1AMzoZU9fwT1roeHv8klGOUPkzHoXbOU2F6ZS3TapWFcHhMj5stIH2RwITCxgxLTUTomA2s6D19kjOlOVqSGbKCnnPPX8eSTWRdgeF/nPd+BT8I6ZcrIR1LsGIU4g915Uf+P0zV2jEtV+1j73xrgaDvJ8MSQPWlRLOytKOIzK5jZxhsgeelIJpnwP+rffJusHlkVxV+PnEjgrOJfpN9KK29jjXAzIjKaFkB0T85T9KJokHYSw1m7x6F/tpQpr+Of1/m08XSKczn+kfxjL+k0RGp7rBMOuwTB8up9NCDDlgmP3OLIj9puBO09wg9Zi6IwWBiVuerws1cQ6h8ERPZnZwmZSaw6v4njqQY2U73KuQYm3v9g2VEYIjZTGRz7p9oZVMniXPH8iEzuSN8/44zcUNkUcEvlORsTiKxUx+TtHCJ5sVxm1AEpQM3wDVvPUGq0k8kRvQUSwL MIME-Version: 1.0 X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB7PR08MB3530 Original-Authentication-Results: intel.com; dkim=none (message not signed) header.d=none;intel.com; dmarc=none action=none header.from=arm.com; Return-Path: Pierre.Gondois@arm.com X-EOPAttributedMessage: 0 X-MS-Exchange-Transport-CrossTenantHeadersStripped: AM5EUR03FT018.eop-EUR03.prod.protection.outlook.com X-Forefront-Antispam-Report: CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFTY:;SFS:(4636009)(136003)(376002)(396003)(346002)(39860400002)(46966005)(36906005)(5660300002)(316002)(33656002)(52536014)(2906002)(336012)(9686003)(8676002)(86362001)(55016002)(478600001)(966005)(53546011)(6506007)(8936002)(110136005)(54906003)(7696005)(26005)(186003)(81166007)(4326008)(356005)(82310400002)(70206006)(70586007)(47076004)(83380400001)(82740400003);DIR:OUT;SFP:1101; X-MS-Office365-Filtering-Correlation-Id-Prvs: 223cbc13-f731-4102-66db-08d806f56720 X-Forefront-PRVS: 0422860ED4 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: o9ePUSab1Tb9kzrDyXDwzstWhi3XhRVhrPl679GAeU3qT+6wwRhkXE8aBMmMCToYIVPBsUPODLLZ39aE7kahRpz0DO9kPmIrhdJq19VDs+f6BrietugBpcu5g/p0UHzF2UtU74T+Q2+vHCSXrw5N3Ni8uhYEl/5HTM+4r4hkWyPqqVzYFJgaq8xHo3pfc/D1EzKfdL/2f3kxNbwSvn9bTIAFCLX4ilavyF1162wuetXDgHXZckWzHEKUu1tnCUVhShMy1Eubgu8cd1THB9ZAsigS4CqWON44+rqL3Lbzlx8dvCA/ngExEnlOfEdGNhFwDPukhLzRqZI20Ycg8BCldfS2GCLNOg80yn7ZzCNbuWpDZVrX+RoFLKMzIem5Eob7ouYEI6YLaix4r63ztSQJWBGo8Cy/80T9Aw7pWfryukRSPuclizeqDLFAj9Sr9GHNxit3LpXlzKX4CamKrFJdgGtomejK1NTQELrrxTAYVVI= X-OriginatorOrg: arm.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 02 Jun 2020 13:04:00.9946 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 66293703-7516-453f-c196-08d806f56b6f X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com] X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR0802MB2463 Content-Language: en-US Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Hello Bob, Thank you for your answer, I put my comments as [Pierre], Regards, Pierre -----Original Message----- From: Feng, Bob C =20 Sent: 02 June 2020 12:16 To: devel@edk2.groups.io; Pierre Gondois Cc: Gao, Liming ; Sami Mujawar ; Tomas Pilar ; nd Subject: RE: [edk2-devel] [PATCH v1 1/3] BaseTools: Generate multiple rule= s when multiple output files My comments are inline marked as [Bob]. Thanks, Bob -----Original Message----- From: devel@edk2.groups.io On Behalf Of PierreGondo= is Sent: Monday, May 18, 2020 10:11 PM To: devel@edk2.groups.io Cc: Pierre Gondois ; Feng, Bob C ; Gao, Liming ; Sami.Mujawar@arm.com; Tomas.Pila= r@arm.com; nd@arm.com Subject: [edk2-devel] [PATCH v1 1/3] BaseTools: Generate multiple rules wh= en multiple output files From: Pierre Gondois BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=3D2425 This patch modifies the Makefile generation not to stop adding Makfile rul= es when the first final target is found. E.g.: If the following rules are described in build_rule.txt: -[Rule1]: .X files generate .Y and .Z files; -[Rule2]: .Z files generate .Z1 files. Currently, if a File1.X file was part of the sources of a module, only [Ru= le1] would be generated in the Makefile. Indeed, there are no rules to apply to .Y files: .Y files are a final targ= et. However, there is still [Rule2] to apply to .Z files. [Bob] I think currently a rule's output file will be added back to source = file list, and in the later loop, that output file will be handled by anoth= er rule. Doesn't that algorithm handle your case above? [Pierre] The rule's output file was effectively added to the list of source files, = and a rule was searched for this output file. However, the loop stopped wh= en the first final target was found. By final target I mean "a file that is= n't the input of a rule". For the asl/aml/amli case, this meant that: -(first loop iteration: treating the ASL file) The rule for ASL files was= found and applied to the input ASL file. The AML and ".amli" files were ad= ded to the list of source files to look for a rule to apply to them. The lo= op continues. -(second loop iteration: treating the AML file) There is no rule for AML = files. This means that the AML file is a final target. The loop ends withou= t having applied the rule on the ".amli" file. This is why all the "break" instructions of the loop have been replaced wi= th "continue". This prevents the loop from exiting without having treated a= ll the files in the "SourceList". This patch also adds a dependency between the first ouput file of a rule a= nd the other output files. For instance, with the same example as above, File1.Y and File1.Z are gene= rated by the following rule: File1.Y: File1.X and the new dependency is: File1.Z: File1.Y This is necessary to keep a dependency order during the execution of the M= akefile. Indeed, .Y and .Z files are generated by the execution of a common= set of commands, and without this rule, there is no explicit dependency re= lation between them. [Bob] If there are 3 output files, for example:=20 A B C Will the dependency relationship be B: A C: B [Pierre] Currently, the dependency relation that would be created is: B: A C: A This can be changed to the dependency you described. Signed-off-by: Pierre Gondois --- The changes can be seen at: https://github.com/PierreARM/edk2/tree/803_Com= pile_AML_bytecode_array_into_OBJ_file_v1 Notes: v1: - Generate multiple rules when multiple output files are specified in the build_rule.txt file. [Pierre] BaseTools/Source/Python/AutoGen/GenMake.py | 6 +++ BaseTools/Source/Python/AutoGen/ModuleAutoGen.py | 40 ++++++++++---------= - 2 files changed, 25 insertions(+), 21 deletions(-) diff --git a/BaseTools/Source/Python/AutoGen/GenMake.py b/BaseTools/Source= /Python/AutoGen/GenMake.py index bbb3c29446f53fa7f2cb61a216a5b119f72c3fbc..0314d0ea34d99a014379e8d30c= 46ac0f0a7068ce 100755 --- a/BaseTools/Source/Python/AutoGen/GenMake.py +++ b/BaseTools/Source/Python/AutoGen/GenMake.py @@ -1054,6 +1054,12 @@ cleanlib: TargetDict =3D {"target": self.PlaceMacro(T.Target.Pa= th, self.Macros), "cmd": "\n\t".join(T.Commands),"deps": Deps} self.BuildTargetList.append(self._BUILD_TARGET_TEMPLA= TE.Replace(TargetDict)) =20 + # Add a Makefile rule for targets generating multiple= files. + # The main output is a prerequisite for the other out= put files. + for i in T.Outputs[1:]: + AnnexeTargetDict =3D {"target":=20 + self.PlaceMacro(i.Path, self.Macros), "cmd": "", "deps":=20 + self.PlaceMacro(T.Target.Path, self.Macros)} + + self.BuildTargetList.append(self._BUILD_TARGET_TEMPLATE.Replace(Annexe + TargetDict)) + def ParserCCodeFile(self, T, Type, CmdSumDict, CmdTargetDict, CmdCppD= ict, DependencyDict): if not CmdSumDict: for item in self._AutoGenObject.Targets[Type]: diff --git a/BaseTools/Source/Python/AutoGen/ModuleAutoGen.py b/BaseTools/= Source/Python/AutoGen/ModuleAutoGen.py index aad591de65f086043d55aeea5661f59c53792e7c..82facdb2774c56833718317f44= 14e0c1861de47f 100755 --- a/BaseTools/Source/Python/AutoGen/ModuleAutoGen.py +++ b/BaseTools/Source/Python/AutoGen/ModuleAutoGen.py @@ -860,7 +860,8 @@ class ModuleAutoGen(AutoGen): SubDirectory =3D os.path.join(self.OutputDir, File.SubDir) if not os.path.exists(SubDirectory): CreateDirectory(SubDirectory) - LastTarget =3D None + TargetList =3D set() + FinalTargetName =3D set() RuleChain =3D set() SourceList =3D [File] Index =3D 0 @@ -872,7 +873,7 @@ class ModuleAutoGen(AutoGen): while Index < len(SourceList): Source =3D SourceList[Index] Index =3D Index + 1 - + FileType =3D TAB_UNKNOWN_FILE if Source !=3D File: CreateDirectory(Source.Dir) =20 @@ -881,34 +882,27 @@ class ModuleAutoGen(AutoGen): if not self.IsLibrary: continue RuleObject =3D self.BuildRules[TAB_DEFAULT_BINARY_FILE] - elif FileType in self.BuildRules: - RuleObject =3D self.BuildRules[FileType] elif Source.Ext in self.BuildRules: RuleObject =3D self.BuildRules[Source.Ext] else: - # stop at no more rules - if LastTarget: - self._FinalBuildTargetList.add(LastTarget) - break - + # No more rule to apply: Source is a final target. + FinalTargetName.add(Source) + continue FileType =3D RuleObject.SourceFileType self._FileTypes[FileType].add(Source) =20 # stop at STATIC_LIBRARY for library if self.IsLibrary and FileType =3D=3D TAB_STATIC_LIBRARY: - if LastTarget: - self._FinalBuildTargetList.add(LastTarget) - break + FinalTargetName.add(Source) + continue =20 Target =3D RuleObject.Apply(Source, self.BuildRuleOrder) if not Target: - if LastTarget: - self._FinalBuildTargetList.add(LastTarget) - break - elif not Target.Outputs: - # Only do build for target with outputs - self._FinalBuildTargetList.add(Target) + # No Target: Source is a final target. + FinalTargetName.add(Source) + continue =20 + TargetList.add(Target) self._BuildTargets[FileType].add(Target) =20 if not Source.IsBinary and Source =3D=3D File: @@ -916,12 +910,16 @@ class ModuleAutoGen(AutoGen): =20 # to avoid cyclic rule if FileType in RuleChain: - break + EdkLogger.error("build", ERROR_STATEMENT, "Cyclic=20 + dependency detected while generating rule for %s" % str(Source)) =20 RuleChain.add(FileType) SourceList.extend(Target.Outputs) - LastTarget =3D Target - FileType =3D TAB_UNKNOWN_FILE + + # For each final target, retrieve the TargetDescBlock instance. + for FTargetName in FinalTargetName: + for Target in TargetList: + if FTargetName =3D=3D Target.Target: + self._FinalBuildTargetList.add(Target) =20 @cached_property def Targets(self): -- 'Guid(CE165669-3EF3-493F-B85D-6190EE5B9759)'