From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mga01.intel.com (mga01.intel.com [192.55.52.88]) by mx.groups.io with SMTP id smtpd.web10.1544.1589318336845675191 for ; Tue, 12 May 2020 14:18:59 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=@intel.onmicrosoft.com header.s=selector2-intel-onmicrosoft-com header.b=CfkGA8Nj; spf=pass (domain: intel.com, ip: 192.55.52.88, mailfrom: ashley.e.desimone@intel.com) IronPort-SDR: /Go43l5hRqi7rCHEn+f/mJ5DezdHiJJP9ZaxBsDMO4UUsfS7sENVi2vrpHskJXF/T2we7ljaIw v21TrskuYztg== X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga002.jf.intel.com ([10.7.209.21]) by fmsmga101.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 12 May 2020 14:18:55 -0700 IronPort-SDR: xQKFY9CxoKoRONFYk4JmEYqWbiaiZKZIZWXwJlfo2wpfk8Glf5hCstoZOLQCeiBNoCQMmiZ8PI EWrI/LOksiMg== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.73,385,1583222400"; d="scan'208";a="280272783" Received: from orsmsx106.amr.corp.intel.com ([10.22.225.133]) by orsmga002.jf.intel.com with ESMTP; 12 May 2020 14:18:54 -0700 Received: from orsmsx160.amr.corp.intel.com (10.22.226.43) by ORSMSX106.amr.corp.intel.com (10.22.225.133) with Microsoft SMTP Server (TLS) id 14.3.439.0; Tue, 12 May 2020 14:18:54 -0700 Received: from ORSEDG001.ED.cps.intel.com (10.7.248.4) by ORSMSX160.amr.corp.intel.com (10.22.226.43) with Microsoft SMTP Server (TLS) id 14.3.439.0; Tue, 12 May 2020 14:18:54 -0700 Received: from NAM11-DM6-obe.outbound.protection.outlook.com (104.47.57.176) by edgegateway.intel.com (134.134.137.100) with Microsoft SMTP Server (TLS) id 14.3.439.0; Tue, 12 May 2020 14:18:54 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=LmROxbwxgAzqvLG9lxU9w75023gK9mCxHXNvoQT9bF3wId7yju6IBVNPkjdHEjvZlxnCIG1sNt5IIU9VEz4uY2LEMJil1otDWolj+nzAC5XDSkOnmcD5841eVVqiWNs0zAhOI56Vq7qdx1sxk9ylnSHL+65hhkH/P8DIrzNikmuxir1bZDMJFTc4ipWuZxQtnt9xObZ1fn3Ndwqnpuzhhp65TaHY+7iK30o6/FMpiyG46X4UI02GnqLhQdLRZhJpcoNfuUB7UJUhMaD9hDuhHGvypbrtFSW+gu8N6tfZyBvxq4r/dmcrzawo20LMVnD48USsvzmjhgz4xxT/ysqy9A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=kDwr/ryJ4ANgXyKS3jdKsEp0C7Y8dvd/iuPR2vuy/5o=; b=DSiRfz9GHO1K2Bq4tzIpaRR34D9ZAvhkK6HTOUShmnpjap0HY5t4uGYJXCr9TupiFOrbg11maxGdc+qYohNySdGz+WTJqHSTEYjJFyAhiUsL6XdgIN+r/73i5MABYKaeF11sbojETpTk5jyj8FZVfzi3fDDZFgObT7mdU7dGCKTLbiopHX55cpz0GhjtUv/nrhPQ5WSUlpryWffYyymUgDJBQtkWfvZbQIBUstPAe9ZoYr0G/O9+/KFVm+NLshwkz14/cPzDJPWPrBnYQtM6g9T/xcTt8uicSpYRQjAz2bY0HF79CL+TGc3jQYzsXBwGarUyuOKVTdAEFHSKlNGAag== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com; dkim=pass header.d=intel.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=intel.onmicrosoft.com; s=selector2-intel-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=kDwr/ryJ4ANgXyKS3jdKsEp0C7Y8dvd/iuPR2vuy/5o=; b=CfkGA8Nj7gdSRW7swKRupvQkDXWaZo+OEG5Y8LVXwwNzxXEtUVOuuDcldl4ALgbID27KR5MtzIGy8d1DkLtSvpQXsb/7I4n3utwZT/aSu44RgPv9PMBBwT/ikxNJWpko5vYtxAeig8Y9ayLzxlsEHU/DJKZhZGGR4xqyxwCM0T0= Received: from BY5PR11MB3973.namprd11.prod.outlook.com (2603:10b6:a03:185::29) by BY5PR11MB4008.namprd11.prod.outlook.com (2603:10b6:a03:186::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2979.33; Tue, 12 May 2020 21:18:52 +0000 Received: from BY5PR11MB3973.namprd11.prod.outlook.com ([fe80::296a:eb1c:2b52:2904]) by BY5PR11MB3973.namprd11.prod.outlook.com ([fe80::296a:eb1c:2b52:2904%7]) with mapi id 15.20.2979.033; Tue, 12 May 2020 21:18:52 +0000 From: "Ashley E Desimone" To: "Bjorge, Erik C" , "devel@edk2.groups.io" CC: "Desimone, Nathaniel L" , "Pandya, Puja" , Bret Barkelew , "Agyeman, Prince" Subject: Re: [edk2-staging/EdkRepo] [PATCH v1] EdkRepo: Adding manifest support for submodule init Thread-Topic: [edk2-staging/EdkRepo] [PATCH v1] EdkRepo: Adding manifest support for submodule init Thread-Index: AQHWKIfNSD8MgIWBV0CFcmPFcQF5daik9MMA Date: Tue, 12 May 2020 21:18:52 +0000 Message-ID: References: <643f03028f1dd1cfc8b51dcbb024b851b9414900.1589306304.git.erik.c.bjorge@intel.com> In-Reply-To: <643f03028f1dd1cfc8b51dcbb024b851b9414900.1589306304.git.erik.c.bjorge@intel.com> Accept-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: dlp-version: 11.2.0.6 dlp-reaction: no-action dlp-product: dlpe-windows authentication-results: intel.com; dkim=none (message not signed) header.d=none;intel.com; dmarc=none action=none header.from=intel.com; x-originating-ip: [192.55.52.218] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 638dc6bb-44f4-44ed-3951-08d7f6ba122a x-ms-traffictypediagnostic: BY5PR11MB4008: x-ms-exchange-transport-forked: True x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:529; x-forefront-prvs: 0401647B7F x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: cHEeSUS+zZeaEk3sIaIYNd0SwA/2ThQpDZDY0g6PVBut9AarrwljVvqIytPRT1gnPMwYOcHAjalajvATGs+MLLXCswGkxNJvP6bVZkErWcHyRLrFLRe77hnPLfNh1VGnEoUQ9VyxxS57pIdU5vLRqlX9zIvUpeY30b1b0NjQANeuCHIPJ+NsSiYtl/8qDAEJlnGzqZhlK5i6WXW22cCxB7kQVeCIMMxbPdniGH4jPjMi6Ijm/4oqRC8Bzr9IpRjPU9fxyB6wJv/DcFHPcX48GMvzL4mkTCih2zKZEJdgn51xjq4bRvTHkW82z1QgsqZMY3w2UkdbPL8rx6Dv16musX0CdOG8yUBumbQpexWHB799sOwQ3X4B5haYSYgVDNhGz9VMF+8gWwD14hjcaCPRaaoTFDw+ya87aNLPsHhzMZiqefqXjdeIuZTa8B8/MmJvoi8x3Mg8X6R6Tuo94DmGY2MJQlaHYdYei6fWx5FPDj/ZGU+NazljN3fnnYUcBr7TDfyKhIo20ZnaY3tTFehgLA== x-forefront-antispam-report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BY5PR11MB3973.namprd11.prod.outlook.com;PTR:;CAT:NONE;SFTY:;SFS:(346002)(366004)(396003)(136003)(39840400004)(376002)(33430700001)(2906002)(110136005)(71200400001)(52536014)(66946007)(316002)(54906003)(76116006)(478600001)(4326008)(66476007)(6506007)(26005)(8936002)(55016002)(53546011)(33656002)(66446008)(66556008)(86362001)(33440700001)(107886003)(9686003)(8676002)(186003)(5660300002)(7696005)(64756008);DIR:OUT;SFP:1102; x-ms-exchange-antispam-messagedata: pENPcDvHvn7Vr6Q+BCSSfORxBczo/Yc2yOnQM2BF6t/iDDldtOoNNXvy8jiJ7TGCkjGRWMbFLiP73Oc/5TggyVV2YkccP7GVOwf4Xv5kRfJfQvGgYVmswO6XG3N6rOlVF37BVk+UrfXwR3LOZ+2VVv5vtEbkppNl13eKeCKJ4XvG8GMAn3U+uuo55bi5BqFHS4i0CYyCqGPNRAy2wJL7Io5sdspbykzHEQF8pNDbq0WNu3uALDlquYBiwv9ovzUcikQkTy0R+fAPIMYQUeWWCgKoOyUEo3JxgzZSV8rkLWIs1cq2+Tgx8RfciLgBsuEaiPBOTBU9Os3hGh6zZXp/gJRellOG/S+/Opsu9jMr3Aw8WpB9auaeOgLpcgQN0nUCpy+SwXqX5Oo7C3E6JGG0ybvHWVy7/DFRQPsDydTOv94VyEQC6Y0i/mX5jpL+yS9Cdo7yAzTtFW/dntHV+uiDgBb5vPrXTHlEQpib8UfOF5+PxdawOg4BRklEf/OQVD1q MIME-Version: 1.0 X-MS-Exchange-CrossTenant-Network-Message-Id: 638dc6bb-44f4-44ed-3951-08d7f6ba122a X-MS-Exchange-CrossTenant-originalarrivaltime: 12 May 2020 21:18:52.3282 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 46c98d88-e344-4ed4-8496-4ed7712e255d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: uVCBN06GNjiUjWv4u1zQpoooFzseEKDAQcKW1g+FBUSGMUXbfajl8rfEHjLvjfzR7/0n85Vvm5FYy1GDNK4GS+gpOxgwfBh75e7PJRYGPAQ= X-MS-Exchange-Transport-CrossTenantHeadersStamped: BY5PR11MB4008 Return-Path: ashley.e.desimone@intel.com X-OriginatorOrg: intel.com Content-Language: en-US Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Reviewed-by: Ashley DeSimone -----Original Message----- From: Bjorge, Erik C =20 Sent: Tuesday, May 12, 2020 11:04 AM To: devel@edk2.groups.io Cc: Desimone, Ashley E ; Desimone, Nathaniel L= ; Pandya, Puja ; Br= et Barkelew ; Agyeman, Prince Subject: [edk2-staging/EdkRepo] [PATCH v1] EdkRepo: Adding manifest support= for submodule init Enabling support for selective submodule initialization in the manifest fil= e. This will allow for more control over the submodules that are initializ= ed. If this section is not present the existing functionality will be used= . Signed-off-by: Erik Bjorge Cc: Ashley E Desimone Cc: Nate DeSimone Cc: Puja Pandya Cc: Bret Barkelew Cc: Prince Agyeman --- edkrepo_manifest_parser/edk_manifest.py | 78 +++++++++++++++++++++++++ 1 file changed, 78 insertions(+) diff --git a/edkrepo_manifest_parser/edk_manifest.py b/edkrepo_manifest_par= ser/edk_manifest.py index 1e2b111..45be2fe 100644 --- a/edkrepo_manifest_parser/edk_manifest.py +++ b/edkrepo_manifest_parser/edk_manifest.py @@ -37,6 +37,7 @@ FolderToFolderMappingFolder =3D namedtuple('FolderToFolde= rMappingFolder', ['projec FolderToFolderMappingFolderExclude =3D namedtupl= e('FolderToFolderMappingFolderExclude', ['path']) =20 SubmoduleAlternateRemote =3D namedtuple('SubmoduleAlternateRemote', ['remo= te_name', 'original_url', 'alternate_url']) +SubmoduleInitPath =3D namedtuple('SubmoduleInitPath', ['remote_name',=20 +'combo', 'recursive', 'path']) =20 REQUIRED_ATTRIB_ERROR_MSG =3D "Required attribute malformed in <{}>: {}" NO_ASSOCIATED_REMOTE =3D 'There are no remotes associated with the ClientG= itHook entry:\nsource:{} destination:{}' \ @@ -138,6 +139,7 @@ class Manife= stXml(BaseXmlHelper): self._commit_templates =3D {} # dict of commit message t= emplates with the remote name as the key self._folder_to_folder_mappings =3D [] # List of FolderToFolderMa= pping objects self._submodule_alternate_remotes =3D [] + self._submodule_init_list =3D [] =20 # # Append include XML's to the Manifest etree before parsing @@ -19= 1,6 +193,13 @@ class ManifestXml(BaseXmlHelper): for element in subroot.iter(tag=3D'SubmoduleAlternateRemote'): self._submodule_alternate_remotes.append(_SubmoduleAlterna= teRemote(element, self._remotes)) =20 + # + # Determine submodule initialization paths + # + for subroot in self._tree.iter(tag=3D'SelectiveSubmoduleInitList')= : + for element in subroot.iter(tag=3D'Submodule'): + =20 + self._submodule_init_list.append(_SubmoduleInitEntry(element)) + # # parse tags # requires RemoteList to be parsed first @@ -386,6 +395,22 @@ clas= s ManifestXml(BaseXmlHelper): alternates.append(alternate.tuple) return alternates =20 + def get_submodule_init_paths(self, remote_name=3DNone, combo=3DNone): + submodule_list =3D [] + if remote_name is None and combo is None: + submodule_list =3D self._tuple_list(self._submodule_init_list) + elif remote_name is not None and combo is None: + submodule_list =3D self._tuple_list( + [x for x in self._submodule_init_list if x.remote_name =3D= =3D remote_name]) + elif remote_name is None and combo is not None: + submodule_list =3D self._tuple_list( + [x for x in self._submodule_init_list if x.combo =3D=3D co= mbo or x.combo is None]) + else: + submodule_list =3D self._tuple_list( + [x for x in self._submodule_init_list + if x.remote_name =3D=3D remote_name and (x.combo =3D=3D c= ombo or x.combo is None)]) + return submodule_list + def write_current_combo(self, combo_name, filename=3DNone): # # Updates the CurrentClonedCombo tag of _tree attribute and writes= the entire tree out to the @@ -460,6 +485,10 @@ class ManifestXml(BaseXmlH= elper): if self._tree.find('SubmoduleAlternateRemotes'): submodule_alt_url_root =3D ET.SubElement(pin_root, 'SubmoduleA= lternateRemotes') =20 + selective_submodules_root =3D None + if self._tree.find('SelectiveSubmoduleInitList'): + selective_submodules_root =3D ET.SubElement(pin_root,=20 + 'SelectiveSubmoduleInitList') + remote_root =3D ET.SubElement(pin_root, 'RemoteList') source_root =3D ET.SubElement(pin_root, 'Combination') source_root.attrib['name'] =3D self._combinations[combo_name].name @@ -487,6 +516,13 @@ class ManifestXml(BaseXmlHelper): if alt_url_element.attrib['remote'] =3D=3D src_tuple.r= emote_name: submodule_alt_url_root.append(alt_url_element) =20 + for subroot_selective_subs in self._tree.iter('SelectiveSubmod= uleInitList'): + for selective_sub in subroot_selective_subs.iter('Submodul= e'): + if selective_sub.attrib['remote'] =3D=3D src_tuple.rem= ote_name: + if 'combo' in selective_sub.attrib and selective_s= ub.attrib['combo'] !=3D combo_name: + continue + selective_submodules_root.append(selective_sub) + sparse =3D 'true' if src_tuple.sparse else 'false' sub =3D 'true' if src_tuple.enable_submodule else 'false' # Write the source element based on what value branch or commi= t is available. @@ -535,6 +571,10 @@ class ManifestXml(BaseXmlHelper): submodule_alt_url_root.text =3D '\n ' submodule_alt_url_root.tail =3D '\n\n ' list(submodule_alt_url_root)[-1].tail =3D '\n ' + if selective_submodules_root: + selective_submodules_root.text =3D '\n ' + selective_submodules_root.tail =3D '\n\n ' + list(selective_submodules_root)[-1].tail =3D '\n ' remote_root.text =3D '\n ' remote_root.tail =3D '\n\n ' list(remote_root)[-1].tail =3D '\n ' @@ -882,6 +922,27 @@ class _SubmoduleAlternateRemote(): return SubmoduleAlternateRemote(self.remote_name, self.originalUrl= , self.altUrl) =20 =20 +class _SubmoduleInitEntry(): + def __init__(self, element): + try: + self.remote_name =3D element.attrib['remote'] + self.path =3D element.text + except KeyError as k: + raise KeyError(REQUIRED_ATTRIB_ERROR_MSG.format(k, element.tag= )) + try: + self.combo =3D element.attrib['combo'] + except Exception: + self.combo =3D None + try: + self.recursive =3D element.attrib['recursive'].lower() =3D=3D = 'true' + except Exception: + self.recursive =3D False + + @property + def tuple(self): + return SubmoduleInitPath(self.remote_name, self.combo,=20 + self.recursive, self.path) + + # # Optional entry point for debug and validation of the CiIndexXml & Manife= stXml classes # @@ -956,6 +1017,23 @@ def main(): for alt in alts: print(alt) =20 + print('\nGet Submodule Init Objects') + print('\nAll:') + for entry in test_manifest.get_submodule_init_paths(): + print('+ {}'.format(entry)) + print('\nPer Remote:') + for remote in test_manifest.remotes: + for entry in test_manifest.get_submodule_init_paths(remote.nam= e): + print('+ {}'.format(entry)) + print('\nCurrent Combo:') + current_combo =3D test_manifest.general_config.current_combo + for entry in test_manifest.get_submodule_init_paths(combo=3Dcurren= t_combo): + print('+ {}'.format(entry)) + print('\nCurrent Combo Per Remote:') + for remote in test_manifest.remotes: + for entry in test_manifest.get_submodule_init_paths(remote.nam= e, current_combo): + print('+ {}'.format(entry)) + if not test_manifest.is_pin_file(): print('\nSparse settings:') print(test_manifest.sparse_settings) -- 2.21.0.windows.1