From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from EUR05-VI1-obe.outbound.protection.outlook.com (EUR05-VI1-obe.outbound.protection.outlook.com [40.107.21.87]) by mx.groups.io with SMTP id smtpd.web10.11099.1592930026837195638 for ; Tue, 23 Jun 2020 09:33:48 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=@armh.onmicrosoft.com header.s=selector2-armh-onmicrosoft-com header.b=xoQfKbmZ; spf=pass (domain: arm.com, ip: 40.107.21.87, mailfrom: pierre.gondois@arm.com) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector2-armh-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=BgZzoPRVE3mGERhJ/eVI8rYc+IDLKoslYcVizsAufHw=; b=xoQfKbmZ0nMqAaLU83KgTISFbXD3NOuJ8/27HArmTqsO3Yemob/ZTqW0fujg4IwO5Jq1ZXGoICD0tiSMUg4tIAUW+CfI2IZUc2IV265dIUBxcYwEuBj+miiL73uF6ve7JXZM6g+V3X44NxVuRGzOzLc1JCLz6kIj2UWlbwWI4gM= Received: from AM5PR0601CA0053.eurprd06.prod.outlook.com (2603:10a6:206::18) by VE1PR08MB5213.eurprd08.prod.outlook.com (2603:10a6:803:10d::17) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3109.22; Tue, 23 Jun 2020 16:33:44 +0000 Received: from AM5EUR03FT037.eop-EUR03.prod.protection.outlook.com (2603:10a6:206:0:cafe::5a) by AM5PR0601CA0053.outlook.office365.com (2603:10a6:206::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3109.22 via Frontend Transport; Tue, 23 Jun 2020 16:33:43 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123) smtp.mailfrom=arm.com; edk2.groups.io; dkim=pass (signature was verified) header.d=armh.onmicrosoft.com;edk2.groups.io; dmarc=bestguesspass action=none header.from=arm.com; Received-SPF: Pass (protection.outlook.com: domain of arm.com designates 63.35.35.123 as permitted sender) receiver=protection.outlook.com; client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com; Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by AM5EUR03FT037.mail.protection.outlook.com (10.152.17.241) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3109.22 via Frontend Transport; Tue, 23 Jun 2020 16:33:43 +0000 Received: ("Tessian outbound 022d32fb9a40:v59"); Tue, 23 Jun 2020 16:33:43 +0000 X-CheckRecipientChecked: true X-CR-MTA-CID: 8fdf8dd733f1bf5b X-CR-MTA-TID: 64aa7808 Received: from 76f4392eee6e.2 by 64aa7808-outbound-1.mta.getcheckrecipient.com id E9782F1C-47B7-4867-9FDA-ACF1C215DA1E.1; Tue, 23 Jun 2020 16:33:38 +0000 Received: from EUR02-HE1-obe.outbound.protection.outlook.com by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 76f4392eee6e.2 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384); Tue, 23 Jun 2020 16:33:38 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=jkt2W+q2OF4K+JX0J4pkjECDenbXYlL66X1+vvR3fo5G24w5+eygTJRTG7e8vz8akE/CXGeO/PUYqsr1V7JN7tuSqnk63/uOIbo6rvMqXz4o0Iw26RyPx1G3fBLPT1PwVkoyrPokz//xxy271sbdXA0+MAMYorrRW0XVZeD+IC+PpfLsBhmBTEuYW7cm8glKNlCoB7J8AL/XpdauHOHCQNCURkayqK381IV9PAmq94yaxVwarnNitQDIXJXwSInIQmVIXsbfGUUK+UetObTeILRqN566rCO47LPrNfaGYYHwsSXFUY/vU4N3n8uidqyrRi2JZypbDpKwRGSCr3gAog== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=BgZzoPRVE3mGERhJ/eVI8rYc+IDLKoslYcVizsAufHw=; b=gOSkem0CoXYIYTG8vFDAObfdWjw7UjR4HnYtxaKvBzfld3puk9bXJfFrSOVDRs4kagkWAPTfFIdGXw2jNI3UunNq8aDxM6ra71/BRcx7dDgzsn2kdDiy70G8t8iPYHrJX2JhYSJ2UM6prJTEWjGl0zlKkH1HwupOBn8rHXr314SDcu4BBtspIfnIwS+4XZALaj3LgvPXpaB8RVwkx4wxHf6G6yI7GjVu7zxqwsxU6Td0DDPDsLBUQ0HUPH/OCCQX1En0Wxc6Hu1l5zwO8OJlQLlc7eE+KKX4fxAdwmSHE8DI37V6x8CKRXJcVvIyzvR3nhCxNKnNHzomwUnUiASesA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass header.d=arm.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector2-armh-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=BgZzoPRVE3mGERhJ/eVI8rYc+IDLKoslYcVizsAufHw=; b=xoQfKbmZ0nMqAaLU83KgTISFbXD3NOuJ8/27HArmTqsO3Yemob/ZTqW0fujg4IwO5Jq1ZXGoICD0tiSMUg4tIAUW+CfI2IZUc2IV265dIUBxcYwEuBj+miiL73uF6ve7JXZM6g+V3X44NxVuRGzOzLc1JCLz6kIj2UWlbwWI4gM= Received: from DB7PR08MB3113.eurprd08.prod.outlook.com (2603:10a6:5:20::21) by DB7PR08MB3595.eurprd08.prod.outlook.com (2603:10a6:10:40::14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3109.22; Tue, 23 Jun 2020 16:33:36 +0000 Received: from DB7PR08MB3113.eurprd08.prod.outlook.com ([fe80::4178:71bd:a2a4:78bf]) by DB7PR08MB3113.eurprd08.prod.outlook.com ([fe80::4178:71bd:a2a4:78bf%7]) with mapi id 15.20.3109.027; Tue, 23 Jun 2020 16:33:36 +0000 From: "PierreGondois" To: "Chen, Yuwei" , "devel@edk2.groups.io" CC: Sami Mujawar , Tomas Pilar , "Feng, Bob C" , "Gao, Liming" , nd Subject: Re: [edk2-devel] [PATCH v2 1/4] BaseTools: Generate multiple rules when multiple output files Thread-Topic: [edk2-devel] [PATCH v2 1/4] BaseTools: Generate multiple rules when multiple output files Thread-Index: AQHWRXydXxIxXnqgGU2+L6GY+yzhr6jffHYAgAbwNCA= Date: Tue, 23 Jun 2020 16:33:35 +0000 Message-ID: References: <20200618142713.146412-1-pierre.gondois@arm.com> <20200618142713.146412-2-pierre.gondois@arm.com> In-Reply-To: Accept-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: Authentication-Results-Original: intel.com; dkim=none (message not signed) header.d=none;intel.com; dmarc=none action=none header.from=arm.com; x-originating-ip: [217.140.99.251] x-ms-publictraffictype: Email X-MS-Office365-Filtering-HT: Tenant X-MS-Office365-Filtering-Correlation-Id: 05e6248d-928a-49bd-08fd-08d8179331e6 x-ms-traffictypediagnostic: DB7PR08MB3595:|VE1PR08MB5213: x-ms-exchange-transport-forked: True X-Microsoft-Antispam-PRVS: x-checkrecipientrouted: true nodisclaimer: true x-ms-oob-tlc-oobclassifiers: OLM:8273;OLM:8273; x-forefront-prvs: 04433051BF X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam-Untrusted: BCL:0; X-Microsoft-Antispam-Message-Info-Original: Eeg8GYL6S/feZwjX/bx7/tHtM3eA9IqpfbU/2XjSCUxn1PltUaY6MCYPJTj/y3B9lV4i7vErgwaSO7BVTB1FAt0f+9XAEDiZbf+tcrGjNC6MfB4qpqNrfjuV9rTtcdKJeXMt6C2mN1lkGUdik5Bi7iWEB5vnEU3O0OtjEqE776UmBg+3mW+5QTEs5k9FzJkUnZqQkvuiicgVeLR0yKmR16rG3NUQgrVvi/sOq0kxB9t6NavqQ6df6dRdLVIYQ+93yxnXihuRIQGy1XUhtyVFLCKSt0j2Z6YW2K4WfFv2NWa4FGDy8cnJlzkx+D735IYKK6NhMEsWSMJpKB1kD7Up+KlJDrvrjzkUsaF+XJHUQXoFJ5EDu/9VcRKP5PgSWTgNRckWk1tensFJoHo38EI/Jg== X-Forefront-Antispam-Report-Untrusted: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DB7PR08MB3113.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFTY:;SFS:(4636009)(396003)(136003)(376002)(39860400002)(346002)(366004)(52536014)(6506007)(53546011)(83380400001)(86362001)(186003)(5660300002)(26005)(966005)(76116006)(66946007)(66476007)(66446008)(66556008)(110136005)(64756008)(478600001)(2906002)(33656002)(7696005)(71200400001)(4326008)(8936002)(316002)(9686003)(54906003)(8676002)(55016002);DIR:OUT;SFP:1101; x-ms-exchange-antispam-messagedata: 0sNXfUjk44MIJ3izb5Pa7BUJZQ9rK0+6fVBaubLVhkVVvWEtTBQA8pojQan8SscWmMo99XVCu5hFKhhOnQVdUSgRKLwk/51+yCrRqP2mXRodi0aMQUExFB+IotjXQ5TIZOO5mvhRnLEhHe986oc2rXFqeGlo4FFpR8C/IGAkSGXNmxCyqLEj2OLjZTDc7IE79JdbzIHm2/4Du4WZupcwnBx7ghUbYSBAPfehpFSA1FlYx4lpK/o/ELzGGsjiLPOb4Z1qfkNkQ2nGoiTcqwPbolkVaDMvYecatWrzbzetZ9UkgOdoB0YyKJiug77fSyANEFWY3mCIBw2nPkyGt9bG817Plr460WFojmRcJPlsckJwiIgH1bM6IMkis5axsMK5p9JTCLT5nrOIoMOFd4aDQL2gyLUrO3LKhap/kLn8TTlYaeEfBWeZ2uak9CEKtQN3nkctA9KABl1vA+y5Eim4813/yQqJYRkdceFWpIy5FI3EAVQayUUhMskEs6VRppGn MIME-Version: 1.0 X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB7PR08MB3595 Original-Authentication-Results: intel.com; dkim=none (message not signed) header.d=none;intel.com; dmarc=none action=none header.from=arm.com; Return-Path: Pierre.Gondois@arm.com X-EOPAttributedMessage: 0 X-MS-Exchange-Transport-CrossTenantHeadersStripped: AM5EUR03FT037.eop-EUR03.prod.protection.outlook.com X-Forefront-Antispam-Report: CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFTY:;SFS:(4636009)(396003)(136003)(346002)(39860400002)(376002)(46966005)(55016002)(7696005)(54906003)(316002)(110136005)(36906005)(52536014)(82740400003)(26005)(966005)(186003)(82310400002)(81166007)(47076004)(478600001)(356005)(8676002)(70586007)(2906002)(336012)(5660300002)(9686003)(33656002)(70206006)(86362001)(6506007)(53546011)(4326008)(8936002)(83380400001);DIR:OUT;SFP:1101; X-MS-Office365-Filtering-Correlation-Id-Prvs: 73cc6246-b247-4bfd-b0b0-08d817932d50 X-Forefront-PRVS: 04433051BF X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: ilBWc6i0shF9hf+Q2126TjqMYJ2WuGochXFxAldGDv6b8Em8mYeP7FmtecL32QBIBI1LtwVAwbJFR7xgRERFAk8vvrbfqxi5vOE43lvYdUgS06pIiL3dCIC2E/3Drytq9AVVLaTM8SqLCsP6SN46kg0HJGRucXzWHWMPY4wbdQmPxNotSG7F28urR5O1rT9N+rntt52sc2BDmickuJPAIyCW6xv91ESq2yR14yBbukF6NsVwuUeOPl/e9o76yglULqKwSuRPi9uxqVZhZtyJCd1z9tope1lindCf/HnwwrNnQzNXdnyq4BLk5rjv3yd7PhRJaxsVZ7nUN7qgmgdaJqAugCzL2WF2W0frXLTIKR4KsDxMX7QPzbXM2cZteo980FczH0z5k2D9Y1mwXMBoH5De991sKcf2tGU10Ut4VWy7T7jPleTu+CtkvTouIkeKWYpwN1NZA74zD46uOY4wVe61yOzUQwnwlIbbo+nOlUc= X-OriginatorOrg: arm.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 23 Jun 2020 16:33:43.6896 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 05e6248d-928a-49bd-08fd-08d8179331e6 X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com] X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: VE1PR08MB5213 Content-Language: en-US Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Hello Yuwei and Bob, Thank you for the review Yuwei. Bob, are the patches acceptable? Regards, Pierre -----Original Message----- From: Chen, Yuwei =20 Sent: Friday, June 19, 2020 7:35 AM To: devel@edk2.groups.io; Pierre Gondois Cc: Sami Mujawar ; Tomas Pilar = ; Feng, Bob C ; Gao, Liming ; n= d Subject: RE: [edk2-devel] [PATCH v2 1/4] BaseTools: Generate multiple rule= s when multiple output files Hi, Pierre The patch looks good. Thanks, Yuwei > -----Original Message----- > From: devel@edk2.groups.io On Behalf Of=20 > PierreGondois > Sent: Thursday, June 18, 2020 10:27 PM > To: devel@edk2.groups.io > Cc: Pierre Gondois ; sami.mujawar@arm.com;=20 > tomas.pilar@arm.com; Feng, Bob C ; Gao, Liming=20 > ; nd@arm.com > Subject: [edk2-devel] [PATCH v2 1/4] BaseTools: Generate multiple=20 > rules when multiple output files >=20 > From: Pierre Gondois >=20 > This patch modifies the Makefile generation not to stop adding Makfile= =20 > rules when the first final target is found. > E.g.: > If the following rules are described in build_rule.txt: > -[Rule1]: .X files generate .Y and .Z files; > -[Rule2]: .Z files generate .Z1 files. > Currently, if a File1.X file was part of the sources of a module, only= =20 > [Rule1] would be generated in the Makefile. > Indeed, there are no rules to apply to .Y files: .Y files are a final ta= rget. > However, there is still [Rule2] to apply to .Z files. >=20 > This patch also adds a dependency between the first ouput file of a=20 > rule and the other output files. > For instance, with the same example as above, File1.Y and File1.Z are=20 > generated by the following rule: > File1.Y: File1.X > > >=20 > and the new dependency is: > File1.Z: File1.Y >=20 > This is necessary to keep a dependency order during the execution of=20 > the Makefile. Indeed, .Y and .Z files are generated by the execution=20 > of a common set of commands, and without this rule, there is no=20 > explicit dependency relation between them. >=20 > Signed-off-by: Pierre Gondois > --- >=20 > The changes can be seen at > https://github.com/PierreARM/edk2/commits/pg/803_Compile_AML_bytec > ode_array_into_OBJ_file_v2 >=20 > Notes: > Notes: > v1: > - Generate multiple rules when multiple output files > are specified in the build_rule.txt file. [Pierre] > v2: > - Use the "FileType" variable in the _ApplyBuildRule > function as it is in the current state. [Pierre] >=20 > BaseTools/Source/Python/AutoGen/GenMake.py | 6 ++++ > BaseTools/Source/Python/AutoGen/ModuleAutoGen.py | 38 > +++++++++++--------- > 2 files changed, 27 insertions(+), 17 deletions(-) >=20 > diff --git a/BaseTools/Source/Python/AutoGen/GenMake.py > b/BaseTools/Source/Python/AutoGen/GenMake.py > index > bbb3c29446f53fa7f2cb61a216a5b119f72c3fbc..0314d0ea34d99a014379e8d30c > 46ac0f0a7068ce 100755 > --- a/BaseTools/Source/Python/AutoGen/GenMake.py > +++ b/BaseTools/Source/Python/AutoGen/GenMake.py > @@ -1054,6 +1054,12 @@ cleanlib: > TargetDict =3D {"target":=20 > self.PlaceMacro(T.Target.Path, self.Macros), > "cmd": "\n\t".join(T.Commands),"deps": Deps} >=20 > self.BuildTargetList.append(self._BUILD_TARGET_TEMPLATE.Replace(Target > Dict)) >=20 > + # Add a Makefile rule for targets generating multip= le files. > + # The main output is a prerequisite for the other o= utput files. > + for i in T.Outputs[1:]: > + AnnexeTargetDict =3D {"target":=20 > + self.PlaceMacro(i.Path, > self.Macros), "cmd": "", "deps": self.PlaceMacro(T.Target.Path,=20 > self.Macros)} > + > + > self.BuildTargetList.append(self._BUILD_TARGET_TEMPLATE.Replace(Annex > e > + TargetDict)) > + > def ParserCCodeFile(self, T, Type, CmdSumDict, CmdTargetDict,=20 > CmdCppDict, DependencyDict): > if not CmdSumDict: > for item in self._AutoGenObject.Targets[Type]: > diff --git a/BaseTools/Source/Python/AutoGen/ModuleAutoGen.py > b/BaseTools/Source/Python/AutoGen/ModuleAutoGen.py > index > aad591de65f086043d55aeea5661f59c53792e7c..dc8b1fe3d160cac2da22227fc2 > 33e3aa0d92cb1e 100755 > --- a/BaseTools/Source/Python/AutoGen/ModuleAutoGen.py > +++ b/BaseTools/Source/Python/AutoGen/ModuleAutoGen.py > @@ -860,7 +860,8 @@ class ModuleAutoGen(AutoGen): > SubDirectory =3D os.path.join(self.OutputDir, File.SubDir) > if not os.path.exists(SubDirectory): > CreateDirectory(SubDirectory) > - LastTarget =3D None > + TargetList =3D set() > + FinalTargetName =3D set() > RuleChain =3D set() > SourceList =3D [File] > Index =3D 0 > @@ -870,6 +871,9 @@ class ModuleAutoGen(AutoGen): > self.BuildOption >=20 > while Index < len(SourceList): > + # Reset the FileType if not the first iteration. > + if Index > 0: > + FileType =3D TAB_UNKNOWN_FILE > Source =3D SourceList[Index] > Index =3D Index + 1 >=20 > @@ -886,29 +890,25 @@ class ModuleAutoGen(AutoGen): > elif Source.Ext in self.BuildRules: > RuleObject =3D self.BuildRules[Source.Ext] > else: > - # stop at no more rules > - if LastTarget: > - self._FinalBuildTargetList.add(LastTarget) > - break > + # No more rule to apply: Source is a final target. > + FinalTargetName.add(Source) > + continue >=20 > FileType =3D RuleObject.SourceFileType > self._FileTypes[FileType].add(Source) >=20 > # stop at STATIC_LIBRARY for library > if self.IsLibrary and FileType =3D=3D TAB_STATIC_LIBRARY: > - if LastTarget: > - self._FinalBuildTargetList.add(LastTarget) > - break > + FinalTargetName.add(Source) > + continue >=20 > Target =3D RuleObject.Apply(Source, self.BuildRuleOrder) > if not Target: > - if LastTarget: > - self._FinalBuildTargetList.add(LastTarget) > - break > - elif not Target.Outputs: > - # Only do build for target with outputs > - self._FinalBuildTargetList.add(Target) > + # No Target: Source is a final target. > + FinalTargetName.add(Source) > + continue >=20 > + TargetList.add(Target) > self._BuildTargets[FileType].add(Target) >=20 > if not Source.IsBinary and Source =3D=3D File: > @@ -916,12 +916,16 @@ class ModuleAutoGen(AutoGen): >=20 > # to avoid cyclic rule > if FileType in RuleChain: > - break > + EdkLogger.error("build", ERROR_STATEMENT, "Cyclic=20 > + dependency detected while generating rule for %s" % str(Source)) >=20 > RuleChain.add(FileType) > SourceList.extend(Target.Outputs) > - LastTarget =3D Target > - FileType =3D TAB_UNKNOWN_FILE > + > + # For each final target name, retrieve the corresponding > TargetDescBlock instance. > + for FTargetName in FinalTargetName: > + for Target in TargetList: > + if FTargetName =3D=3D Target.Target: > + self._FinalBuildTargetList.add(Target) >=20 > @cached_property > def Targets(self): > -- > 'Guid(CE165669-3EF3-493F-B85D-6190EE5B9759)' >=20 >=20 >=20