From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mga02.intel.com (mga02.intel.com [134.134.136.20]) by mx.groups.io with SMTP id smtpd.web10.3550.1591761486741983501 for ; Tue, 09 Jun 2020 20:58:07 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=@intel.onmicrosoft.com header.s=selector2-intel-onmicrosoft-com header.b=zXA1GPSs; spf=pass (domain: intel.com, ip: 134.134.136.20, mailfrom: bob.c.feng@intel.com) IronPort-SDR: AeEo65/Trb9Yqyz+9G57Nh0Ry0qjcy2YjOOUaViDuE+kK3CUGFw6O6xclWTAowY25t41OBJxCr H8zwnwZadNdQ== X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga002.fm.intel.com ([10.253.24.26]) by orsmga101.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 09 Jun 2020 20:58:05 -0700 IronPort-SDR: tw3aLBKp+sQYRd9JVy+I5Vpvlcl94Ep4kUhTjSr6/WutUZsWUE0KITY+hOT5KPOneWh/xb/2Fw sLw/wq8i7sDg== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.73,494,1583222400"; d="scan'208";a="306412474" Received: from orsmsx108.amr.corp.intel.com ([10.22.240.6]) by fmsmga002.fm.intel.com with ESMTP; 09 Jun 2020 20:58:05 -0700 Received: from orsmsx124.amr.corp.intel.com (10.22.240.120) by ORSMSX108.amr.corp.intel.com (10.22.240.6) with Microsoft SMTP Server (TLS) id 14.3.439.0; Tue, 9 Jun 2020 20:58:04 -0700 Received: from ORSEDG002.ED.cps.intel.com (10.7.248.5) by ORSMSX124.amr.corp.intel.com (10.22.240.120) with Microsoft SMTP Server (TLS) id 14.3.439.0; Tue, 9 Jun 2020 20:58:04 -0700 Received: from NAM02-CY1-obe.outbound.protection.outlook.com (104.47.37.56) by edgegateway.intel.com (134.134.137.101) with Microsoft SMTP Server (TLS) id 14.3.439.0; Tue, 9 Jun 2020 20:58:04 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=FfJD9OdUYPeZ+GL2pwAjEKTF0I2Q2cdqCr3dG2azVN9K4oiMYu2mQiEtA6RhfLj64uGva4m7RRh8w1WFd0hEZEsFOLoaxkmIQ2iFmyHyVaNkZI7fObMyPmHT4D0SOq/Lc4VitESImziyG/o5pmN7OY8vVGPyf1gO2ZWr/dC3xxUFBPWj/B1NwTW1t3EepQET3lh4foOrlHsNWifnmRO5B8CTWEPJTYZHI54hoxDvjgxsegZ9VYzWbkmhmqx1110n4f/ek0zz83ScYU/mryD6g2O10ypg2VDS75Lr26TAY/nZvdn4E9SkPVcLAZeI2EJQKFYOPM2BHLOYyBqhbgYaNQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=CT/MhelkQdvAzHn5ZNQYDXJijovFdHEc2GjkD8aPtwA=; b=Sph3mp65pAYwQr/1qIlwbOKjEaoGtR4ZEUaig8nR7uDDLF1uO9Nucojgn9hJ0L4nD4uFJDt4qhAiig/OIMtxgwr31Xs+VpNqEPQmJ3hXnTo3PduvB+lv0emHb3xaKbiSzvCVrdCGwkx5sdHjiVQoQSqSdrG+I0npTxkqWoUDRiYmV75TPMxh1iSBk1Vj08V8JEDS3+cOJcEqbEgyKTDPFBATHJvcWXfXr1pf3vdbZdnkIivQw/h4wO7ULdJv41d6qih0K7qWLsnzkzHj1mfgvOHb7aV/JCH9v0/TZ+jMJVafVd+Eyff4wTD/pj5bA0uv95lXwT8BJB+xjeDvglzzBA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com; dkim=pass header.d=intel.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=intel.onmicrosoft.com; s=selector2-intel-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=CT/MhelkQdvAzHn5ZNQYDXJijovFdHEc2GjkD8aPtwA=; b=zXA1GPSsRnRsvAoNctoaO6C/y0mkd6VSWOCAJAou6YL0oS/rg9bJOBjIu3VDsxvLoHskBalrB0YyJF2FqKX6USQOWsZC7d9ga68Kht2rlN3Yj5XCccsf8vAB3Wzhpj2ITKpAtk4o9dGXf83A3pgL7URUn0O8IX0VFZEme17xkVE= Received: from BN6PR11MB0068.namprd11.prod.outlook.com (2603:10b6:405:69::17) by BN6PR11MB1762.namprd11.prod.outlook.com (2603:10b6:404:100::14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3088.18; Wed, 10 Jun 2020 03:58:03 +0000 Received: from BN6PR11MB0068.namprd11.prod.outlook.com ([fe80::6870:82d3:5bde:4153]) by BN6PR11MB0068.namprd11.prod.outlook.com ([fe80::6870:82d3:5bde:4153%6]) with mapi id 15.20.3066.023; Wed, 10 Jun 2020 03:58:03 +0000 From: "Bob Feng" To: Pierre Gondois , "devel@edk2.groups.io" CC: "Gao, Liming" , Sami Mujawar , Tomas Pilar , nd Subject: Re: [edk2-devel] [PATCH v1 1/3] BaseTools: Generate multiple rules when multiple output files Thread-Topic: [edk2-devel] [PATCH v1 1/3] BaseTools: Generate multiple rules when multiple output files Thread-Index: AQHWLR5Z/G9BS65hW0CiAqbPj8x3x6jFQTvAgAAhEICACX3vgIACcE3A Date: Wed, 10 Jun 2020 03:58:02 +0000 Message-ID: References: <20200518141120.66132-1-pierre.gondois@arm.com> <20200518141120.66132-2-pierre.gondois@arm.com> In-Reply-To: Accept-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: arm.com; dkim=none (message not signed) header.d=none;arm.com; dmarc=none action=none header.from=intel.com; x-originating-ip: [192.102.204.36] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: dece55c2-89b6-4824-b5aa-08d80cf27965 x-ms-traffictypediagnostic: BN6PR11MB1762: x-ms-exchange-transport-forked: True x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:10000; x-forefront-prvs: 0430FA5CB7 x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: J+ENjC48QU1xpdkX2jDTpDk8VJVI1cFOBBvM0ushyxKHuxL01BVAduBTfjgYV7bLvP+TFnOBRraOIooOguFUWYEPQTSBi4Hv3T1XWq+Y/2TdhjtWRs8gG3gRHadyHwhrCkBxtnHqnjsAmfaRh9SfABKdSQ7bluV6LBg6laV7CkQz4nyaGMHSx/dtib93V5MbMKg1lXYeR670MX4tX99GBHSOS651x8VwfGV0UGn6XyZRujAhwK4TsmuTJFj38/4e6Bna4i5Bi3x4pUJb3Z91u1vpnVZHDoCOTvaqxhvLHYDuWDiQiJkAMgEa3p4NEDDqiZKOZWzIfclVqryAdurKUIYbcen0deg2IvChGZr4GBeP9iuRG9s/3sp1oQdLDcz9xLfclM8UqmPsb1rW2g2m4Q== x-forefront-antispam-report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BN6PR11MB0068.namprd11.prod.outlook.com;PTR:;CAT:NONE;SFTY:;SFS:(346002)(376002)(39860400002)(136003)(366004)(396003)(8676002)(52536014)(71200400001)(83380400001)(66946007)(6506007)(66556008)(76116006)(66476007)(66446008)(64756008)(55016002)(966005)(316002)(4326008)(33656002)(26005)(2906002)(478600001)(86362001)(5660300002)(8936002)(53546011)(186003)(54906003)(9686003)(7696005)(110136005);DIR:OUT;SFP:1102; x-ms-exchange-antispam-messagedata: jurm3sVovWpAAI6ZA+81fAnG1DPCXlQxVaBuqduFYmPJiuhm6YV7na6sKUyP0f8QpQ3KNJXciOYagW8rV2bcloQogiPRwqFZnEcAAf6x5NYjJzUYksYwdIO5/fTWz2/unI2KIZ1FVk/sB5NedR2q5oTmMIiX9oEPx7nCUmECTyjy6kfapW2Lo0HNwCEZK8hvi6TtRp7JdUwCvjilePb+dnRJz5E63XqnwiEBqKiFve23lk8cAtLYCvQV0mOHBPR/X7nXFb0ZZLdJ0uRKK06stlfUPMNE4iVbgvy2dhejNrX2xwaileqPZuLcJaORS/miGeIwYM0pCN1GOeahEG0c1PevsAnaI/nSbBrUgKpv3FmDiZwfpeTfM5WBWd7M2ghETQyOagD/T4P654IHBM4tnJ21Rk1rt741bBh4ZMI07pgFMKQUO3mx7dZXunD/tWqIBF2661r2XYKILB6EwiR2YeEc3UhDdd2O4vyW6+kJv1Cz1AJ09nhvwE0HDWDekrN0 MIME-Version: 1.0 X-MS-Exchange-CrossTenant-Network-Message-Id: dece55c2-89b6-4824-b5aa-08d80cf27965 X-MS-Exchange-CrossTenant-originalarrivaltime: 10 Jun 2020 03:58:02.9094 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 46c98d88-e344-4ed4-8496-4ed7712e255d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: xrIm1aNDjW5LQkxvn2G3N7VXdCSbpPYkJSMRf855Aa83qyburWfW49h87uDlE+ccBqfGU9ekGVKSvSfCtpltGA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN6PR11MB1762 Return-Path: bob.c.feng@intel.com X-OriginatorOrg: intel.com Content-Language: en-US Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Hi Pierre, I found there is an incorrect target generated in the OvmfPkg/AcpiTables M= akefile when I tried to build Ovmf. That incorrect target causes ovmf to bu= ild failed. $(DEBUG_DIR)\PlatformAcpiTables : $(MAKE_FILE) $(DEBUG_DIR)\PlatformAcpiTables : $(STATIC_LIBRARY_FILES) $(DEBUG_DIR)\PlatformAcpiTables : $(STATIC_LIBRARY_FILES_LIST) "$(DLINK)" $(DLINK_FLAGS) $(DLINK_SPATH) @$(STATIC_LIBRARY_FILES_LIST) Regarding this patch, I think it changes the logic of the _ApplyBuildRule,= replacing "break" with "continue" and removing some if and elseif blocks, = so it would be hard for me to make sure your current logic can cover all th= e original cases. Would you show me how many testing you have done? Thanks, Bob=20 -----Original Message----- From: Pierre Gondois =20 Sent: Monday, June 8, 2020 10:01 PM To: Feng, Bob C ; devel@edk2.groups.io Cc: Gao, Liming ; Sami Mujawar ; Tomas Pilar ; nd Subject: RE: [edk2-devel] [PATCH v1 1/3] BaseTools: Generate multiple rule= s when multiple output files Hello Bob, Should I modify the patch ? Regards, Pierre -----Original Message----- From: Pierre Gondois Sent: Tuesday, June 2, 2020 2:04 PM To: Feng, Bob C ; devel@edk2.groups.io Cc: Gao, Liming ; Sami Mujawar ; Tomas Pilar ; nd Subject: RE: [edk2-devel] [PATCH v1 1/3] BaseTools: Generate multiple rule= s when multiple output files Hello Bob, Thank you for your answer, I put my comments as [Pierre], Regards, Pierre -----Original Message----- From: Feng, Bob C Sent: 02 June 2020 12:16 To: devel@edk2.groups.io; Pierre Gondois Cc: Gao, Liming ; Sami Mujawar ; Tomas Pilar ; nd Subject: RE: [edk2-devel] [PATCH v1 1/3] BaseTools: Generate multiple rule= s when multiple output files My comments are inline marked as [Bob]. Thanks, Bob -----Original Message----- From: devel@edk2.groups.io On Behalf Of PierreGondo= is Sent: Monday, May 18, 2020 10:11 PM To: devel@edk2.groups.io Cc: Pierre Gondois ; Feng, Bob C ; Gao, Liming ; Sami.Mujawar@arm.com; Tomas.Pila= r@arm.com; nd@arm.com Subject: [edk2-devel] [PATCH v1 1/3] BaseTools: Generate multiple rules wh= en multiple output files From: Pierre Gondois BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=3D2425 This patch modifies the Makefile generation not to stop adding Makfile rul= es when the first final target is found. E.g.: If the following rules are described in build_rule.txt: -[Rule1]: .X files generate .Y and .Z files; -[Rule2]: .Z files generate .Z1 files. Currently, if a File1.X file was part of the sources of a module, only [Ru= le1] would be generated in the Makefile. Indeed, there are no rules to apply to .Y files: .Y files are a final targ= et. However, there is still [Rule2] to apply to .Z files. [Bob] I think currently a rule's output file will be added back to source = file list, and in the later loop, that output file will be handled by anoth= er rule. Doesn't that algorithm handle your case above? [Pierre] The rule's output file was effectively added to the list of source files, = and a rule was searched for this output file. However, the loop stopped wh= en the first final target was found. By final target I mean "a file that is= n't the input of a rule". For the asl/aml/amli case, this meant that: -(first loop iteration: treating the ASL file) The rule for ASL files was= found and applied to the input ASL file. The AML and ".amli" files were ad= ded to the list of source files to look for a rule to apply to them. The lo= op continues. -(second loop iteration: treating the AML file) There is no rule for AML = files. This means that the AML file is a final target. The loop ends withou= t having applied the rule on the ".amli" file. This is why all the "break" instructions of the loop have been replaced wi= th "continue". This prevents the loop from exiting without having treated a= ll the files in the "SourceList". This patch also adds a dependency between the first ouput file of a rule a= nd the other output files. For instance, with the same example as above, File1.Y and File1.Z are gene= rated by the following rule: File1.Y: File1.X and the new dependency is: File1.Z: File1.Y This is necessary to keep a dependency order during the execution of the M= akefile. Indeed, .Y and .Z files are generated by the execution of a common= set of commands, and without this rule, there is no explicit dependency re= lation between them. [Bob] If there are 3 output files, for example:=20 A B C Will the dependency relationship be B: A C: B [Pierre] Currently, the dependency relation that would be created is: B: A C: A This can be changed to the dependency you described. Signed-off-by: Pierre Gondois --- The changes can be seen at: https://github.com/PierreARM/edk2/tree/803_Com= pile_AML_bytecode_array_into_OBJ_file_v1 Notes: v1: - Generate multiple rules when multiple output files are specified in the build_rule.txt file. [Pierre] BaseTools/Source/Python/AutoGen/GenMake.py | 6 +++ BaseTools/Source/Python/AutoGen/ModuleAutoGen.py | 40 ++++++++++---------= - 2 files changed, 25 insertions(+), 21 deletions(-) diff --git a/BaseTools/Source/Python/AutoGen/GenMake.py b/BaseTools/Source= /Python/AutoGen/GenMake.py index bbb3c29446f53fa7f2cb61a216a5b119f72c3fbc..0314d0ea34d99a014379e8d30c= 46ac0f0a7068ce 100755 --- a/BaseTools/Source/Python/AutoGen/GenMake.py +++ b/BaseTools/Source/Python/AutoGen/GenMake.py @@ -1054,6 +1054,12 @@ cleanlib: TargetDict =3D {"target": self.PlaceMacro(T.Target.Pa= th, self.Macros), "cmd": "\n\t".join(T.Commands),"deps": Deps} self.BuildTargetList.append(self._BUILD_TARGET_TEMPLA= TE.Replace(TargetDict)) =20 + # Add a Makefile rule for targets generating multiple= files. + # The main output is a prerequisite for the other out= put files. + for i in T.Outputs[1:]: + AnnexeTargetDict =3D {"target":=20 + self.PlaceMacro(i.Path, self.Macros), "cmd": "", "deps":=20 + self.PlaceMacro(T.Target.Path, self.Macros)} + + self.BuildTargetList.append(self._BUILD_TARGET_TEMPLATE.Replace(Annexe + TargetDict)) + def ParserCCodeFile(self, T, Type, CmdSumDict, CmdTargetDict, CmdCppD= ict, DependencyDict): if not CmdSumDict: for item in self._AutoGenObject.Targets[Type]: diff --git a/BaseTools/Source/Python/AutoGen/ModuleAutoGen.py b/BaseTools/= Source/Python/AutoGen/ModuleAutoGen.py index aad591de65f086043d55aeea5661f59c53792e7c..82facdb2774c56833718317f44= 14e0c1861de47f 100755 --- a/BaseTools/Source/Python/AutoGen/ModuleAutoGen.py +++ b/BaseTools/Source/Python/AutoGen/ModuleAutoGen.py @@ -860,7 +860,8 @@ class ModuleAutoGen(AutoGen): SubDirectory =3D os.path.join(self.OutputDir, File.SubDir) if not os.path.exists(SubDirectory): CreateDirectory(SubDirectory) - LastTarget =3D None + TargetList =3D set() + FinalTargetName =3D set() RuleChain =3D set() SourceList =3D [File] Index =3D 0 @@ -872,7 +873,7 @@ class ModuleAutoGen(AutoGen): while Index < len(SourceList): Source =3D SourceList[Index] Index =3D Index + 1 - + FileType =3D TAB_UNKNOWN_FILE if Source !=3D File: CreateDirectory(Source.Dir) =20 @@ -881,34 +882,27 @@ class ModuleAutoGen(AutoGen): if not self.IsLibrary: continue RuleObject =3D self.BuildRules[TAB_DEFAULT_BINARY_FILE] - elif FileType in self.BuildRules: - RuleObject =3D self.BuildRules[FileType] elif Source.Ext in self.BuildRules: RuleObject =3D self.BuildRules[Source.Ext] else: - # stop at no more rules - if LastTarget: - self._FinalBuildTargetList.add(LastTarget) - break - + # No more rule to apply: Source is a final target. + FinalTargetName.add(Source) + continue FileType =3D RuleObject.SourceFileType self._FileTypes[FileType].add(Source) =20 # stop at STATIC_LIBRARY for library if self.IsLibrary and FileType =3D=3D TAB_STATIC_LIBRARY: - if LastTarget: - self._FinalBuildTargetList.add(LastTarget) - break + FinalTargetName.add(Source) + continue =20 Target =3D RuleObject.Apply(Source, self.BuildRuleOrder) if not Target: - if LastTarget: - self._FinalBuildTargetList.add(LastTarget) - break - elif not Target.Outputs: - # Only do build for target with outputs - self._FinalBuildTargetList.add(Target) + # No Target: Source is a final target. + FinalTargetName.add(Source) + continue =20 + TargetList.add(Target) self._BuildTargets[FileType].add(Target) =20 if not Source.IsBinary and Source =3D=3D File: @@ -916,12 +910,16 @@ class ModuleAutoGen(AutoGen): =20 # to avoid cyclic rule if FileType in RuleChain: - break + EdkLogger.error("build", ERROR_STATEMENT, "Cyclic=20 + dependency detected while generating rule for %s" % str(Source)) =20 RuleChain.add(FileType) SourceList.extend(Target.Outputs) - LastTarget =3D Target - FileType =3D TAB_UNKNOWN_FILE + + # For each final target, retrieve the TargetDescBlock instance. + for FTargetName in FinalTargetName: + for Target in TargetList: + if FTargetName =3D=3D Target.Target: + self._FinalBuildTargetList.add(Target) =20 @cached_property def Targets(self): -- 'Guid(CE165669-3EF3-493F-B85D-6190EE5B9759)'