From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mga01.intel.com (mga01.intel.com [192.55.52.88]) by mx.groups.io with SMTP id smtpd.web12.1840.1589319374875386902 for ; Tue, 12 May 2020 14:36:15 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=@intel.onmicrosoft.com header.s=selector2-intel-onmicrosoft-com header.b=uQwYUyKk; spf=pass (domain: intel.com, ip: 192.55.52.88, mailfrom: ashley.e.desimone@intel.com) IronPort-SDR: 61SsT+gwNcp23Ts0SC4So3rvek2iRw1jHb8SEmgMTV4C9jGU24NC+XVjiRITnIKLpzGu5r/xlT qohlVA46B4Mg== X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga005.fm.intel.com ([10.253.24.32]) by fmsmga101.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 12 May 2020 14:36:13 -0700 IronPort-SDR: GGDrgV/zI6KV9j2huvk0i3FklS/4iqGZlP4mCeEYeBEmFRKnpScNeFVIZFqm4eGDP2xsTukBEz QEAvRyqXSMRQ== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.73,385,1583222400"; d="scan'208";a="463696935" Received: from fmsmsx108.amr.corp.intel.com ([10.18.124.206]) by fmsmga005.fm.intel.com with ESMTP; 12 May 2020 14:36:13 -0700 Received: from fmsmsx122.amr.corp.intel.com (10.18.125.37) by FMSMSX108.amr.corp.intel.com (10.18.124.206) with Microsoft SMTP Server (TLS) id 14.3.439.0; Tue, 12 May 2020 14:36:13 -0700 Received: from FMSEDG001.ED.cps.intel.com (10.1.192.133) by fmsmsx122.amr.corp.intel.com (10.18.125.37) with Microsoft SMTP Server (TLS) id 14.3.439.0; Tue, 12 May 2020 14:36:12 -0700 Received: from NAM04-BN3-obe.outbound.protection.outlook.com (104.47.46.50) by edgegateway.intel.com (192.55.55.68) with Microsoft SMTP Server (TLS) id 14.3.439.0; Tue, 12 May 2020 14:36:12 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Mzlw5rb4fR3BJhZN+b1GLhu32Bs3AxymSdLb2taB9UTfryf287r/AE+xY+PDx68PYRgrfUvwK0cQGT9ftNEhbuIJwynWTNijJ7YXG4sHbTK/VqiKwopa0CQmUdyXrJEY1lGK4Lo/tOlMCptUSME52qwnyul5Tmnl3g5mXkxZE8fZGolyf1usN1MGv6iNwdFXKHlouy6mBAODda2kd5v0Bb0pi7qdyAQ2gf+GlQ1kQIadP3D7w4n3vdUA1dRSlhpJ2LGQi4VEq5Zon235st9XQx7FoPA4a7uD+wmuHF3mcw6QMHyhoy83gmooQvbABPMPoWikngWxiW/yPQkFQt527w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=NbLqnlcR56gxo2MhVH8vvJj3lvcM4CCcpv7rAjrOHDI=; b=i8NEkalb43bNhP3XYVntMTq4CLHcHpjbLdQje5WJv+4tNXLANoRFHFySwY8fd8GB5uinnpl/MD1NPvorwAlLGD4B2I3hFJlUlmj7YSYSzj2CQHST2nCh4Jhxbp20sIZaMu4Bwy7Dl67IcsmrTJrOSxOEvcgdMTT2ZBsAgmLnsp3XqJb7m1UdY3O8ytaBnfcmzRhUGAJHnu4hAaOykcjVGLYl+sG+jEmQh/QspmNrQ11dWDRnRKZ78zBQa0mvplf3mgQt93mDih1fIIKy8/0OnZD3/a4i7FImh6Vhb7ZCumlFDgEqGVii/E/STIrxsLH00sCTFYOXlbLL6pmb6gpQDA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com; dkim=pass header.d=intel.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=intel.onmicrosoft.com; s=selector2-intel-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=NbLqnlcR56gxo2MhVH8vvJj3lvcM4CCcpv7rAjrOHDI=; b=uQwYUyKkKvrtxE45UvPt2T72KX+kBrDAUEjgVLOzqUVSFjJQ+9xwRsEutQVqFnT4ecr38/74pBFGI4sbirHtX+njH30uOgzL9CdD07zq5ZN/Mg8DsRhaUDHm8ubWlz+frda71rWFGxlSDvbLrSIhG6zaHTvZJRCxFEXq3PotzxY= Received: from BY5PR11MB3973.namprd11.prod.outlook.com (2603:10b6:a03:185::29) by BY5PR11MB4337.namprd11.prod.outlook.com (2603:10b6:a03:1c1::14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2958.27; Tue, 12 May 2020 21:36:10 +0000 Received: from BY5PR11MB3973.namprd11.prod.outlook.com ([fe80::296a:eb1c:2b52:2904]) by BY5PR11MB3973.namprd11.prod.outlook.com ([fe80::296a:eb1c:2b52:2904%7]) with mapi id 15.20.2979.033; Tue, 12 May 2020 21:36:10 +0000 From: "Ashley E Desimone" To: "Bjorge, Erik C" , "devel@edk2.groups.io" CC: "Desimone, Nathaniel L" , "Pandya, Puja" , Bret Barkelew , "Agyeman, Prince" Subject: Re: [edk2-staging/EdkRepo] [PATCH v1] EdkRepo: Adding manifest support for submodule init Thread-Topic: [edk2-staging/EdkRepo] [PATCH v1] EdkRepo: Adding manifest support for submodule init Thread-Index: AQHWKIfNSD8MgIWBV0CFcmPFcQF5daik+XJQ Date: Tue, 12 May 2020 21:36:10 +0000 Message-ID: References: <643f03028f1dd1cfc8b51dcbb024b851b9414900.1589306304.git.erik.c.bjorge@intel.com> In-Reply-To: <643f03028f1dd1cfc8b51dcbb024b851b9414900.1589306304.git.erik.c.bjorge@intel.com> Accept-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: dlp-version: 11.2.0.6 dlp-reaction: no-action dlp-product: dlpe-windows authentication-results: intel.com; dkim=none (message not signed) header.d=none;intel.com; dmarc=none action=none header.from=intel.com; x-originating-ip: [192.55.52.218] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 8c5358c7-3990-4d32-15e8-08d7f6bc7cde x-ms-traffictypediagnostic: BY5PR11MB4337: x-ms-exchange-transport-forked: True x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:529; x-forefront-prvs: 0401647B7F x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: Au1ZERRmiA/N9SngO4wVexaT7+2AligJpOKnp0EHSOv3RjMS0g6gFbb26WLb5vtm0psJOUjWXCRtWrfywfHKCeE3Oq11timHcf4B/2akvpY/RNPgHRZhOSbFHofPcls98kM4+fmhKq/PjULGfGgJuNfQVK9xcika2sy65CzJGyytHlS/gIZVAyl2l7N65NCuW+nc4ZncCVZyM75ZNHKqTpChdh1dWolLF43t1l5ggk5DOWefH4TcjK3mqrj93OP3ycHlzVX5V0BhN4Lon+tM/y+bZmSj9H4HDG0LLVosyidEn76hUnlfFxGCgTu9yO7dJLjyYoQwzZx14lGu2tjUdNJwTV2C3nRo7qzZcOQF125TlbKddc8LOk0iP0+O5A3VnYTUaugbhCpOF32s7FeuqIz6FOLxYs5Z71X38ufwiNNq4Pj1INpBAvcmhiwa9af41ZAwOFon5i4A3z1XPtdFJjsNlc/QrtgW0M0ZOAJBevsmseG5M1YNuLvHuNCr7KX/ydRE/lnjlC4ko6Tty2/zvA== x-forefront-antispam-report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BY5PR11MB3973.namprd11.prod.outlook.com;PTR:;CAT:NONE;SFTY:;SFS:(346002)(39860400002)(136003)(396003)(376002)(366004)(33430700001)(53546011)(66946007)(26005)(54906003)(478600001)(186003)(110136005)(4326008)(7696005)(316002)(6506007)(55016002)(66476007)(9686003)(33656002)(71200400001)(8936002)(66556008)(64756008)(52536014)(86362001)(33440700001)(2906002)(5660300002)(76116006)(66446008)(107886003)(8676002);DIR:OUT;SFP:1102; x-ms-exchange-antispam-messagedata: AkUIHug1utONIWCvc57PORN2oZSWpJFpMk8gz8U9uKYvpNrHkrVCxBp1td/1JXrQdcfrevYnmDl00o0vO4gx+iZo2Dd7PyjabKSrUk5eAdblMpqH7Vy1p1obqEmjgpmdLYajeiuudGr2Ic40uTtqh3MJjz6Y4SzP8N6DzprubADZSmmQce2zz612w5uSXbtwJ9poVh746XooveGvcDPGJ6b9zvgum+9wACB7hFUXDz3XKaAdT3lhHzvc35CAsdv/UAgW6XBAOxvhxLQrNeztu8zBm01pMVRVuYxUHlK4cl74nZ4yow6fH+ZIFWSt792AMX+m4dpiIcqiJBVPkmencnpzeClFQpPnUdOKoL8A/pwFTFAD31+y9+Yme3/FYOvLr0FpV5zXL+HrSqoxqDlDYqA3RgP28Ssu4sD0fu99AKIGdnqA3o1IHhGc6eiTb+Ql5GYFQnUmiKlFqb6IaEyj5M1NkxxUadBPRO+L7lCNrQNEPQ5MyBketR0hq2FuFAGl MIME-Version: 1.0 X-MS-Exchange-CrossTenant-Network-Message-Id: 8c5358c7-3990-4d32-15e8-08d7f6bc7cde X-MS-Exchange-CrossTenant-originalarrivaltime: 12 May 2020 21:36:10.2949 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 46c98d88-e344-4ed4-8496-4ed7712e255d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: t+6k41A8nzRknuayqTvUVlvIYwEgK2Q7rVEk9hC60mPuq7+t+EaVGdz76VYXQWoSBTirGzGGQTpaje193lGvqLn4yJUxsOrx+Aax79EY7r8= X-MS-Exchange-Transport-CrossTenantHeadersStamped: BY5PR11MB4337 Return-Path: ashley.e.desimone@intel.com X-OriginatorOrg: intel.com Content-Language: en-US Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Pushed: 9106d381b5f22df819ed66e97d6a61f37747262e -----Original Message----- From: Bjorge, Erik C =20 Sent: Tuesday, May 12, 2020 11:04 AM To: devel@edk2.groups.io Cc: Desimone, Ashley E ; Desimone, Nathaniel L= ; Pandya, Puja ; Br= et Barkelew ; Agyeman, Prince Subject: [edk2-staging/EdkRepo] [PATCH v1] EdkRepo: Adding manifest support= for submodule init Enabling support for selective submodule initialization in the manifest fil= e. This will allow for more control over the submodules that are initializ= ed. If this section is not present the existing functionality will be used= . Signed-off-by: Erik Bjorge Cc: Ashley E Desimone Cc: Nate DeSimone Cc: Puja Pandya Cc: Bret Barkelew Cc: Prince Agyeman --- edkrepo_manifest_parser/edk_manifest.py | 78 +++++++++++++++++++++++++ 1 file changed, 78 insertions(+) diff --git a/edkrepo_manifest_parser/edk_manifest.py b/edkrepo_manifest_par= ser/edk_manifest.py index 1e2b111..45be2fe 100644 --- a/edkrepo_manifest_parser/edk_manifest.py +++ b/edkrepo_manifest_parser/edk_manifest.py @@ -37,6 +37,7 @@ FolderToFolderMappingFolder =3D namedtuple('FolderToFolde= rMappingFolder', ['projec FolderToFolderMappingFolderExclude =3D namedtupl= e('FolderToFolderMappingFolderExclude', ['path']) =20 SubmoduleAlternateRemote =3D namedtuple('SubmoduleAlternateRemote', ['remo= te_name', 'original_url', 'alternate_url']) +SubmoduleInitPath =3D namedtuple('SubmoduleInitPath', ['remote_name',=20 +'combo', 'recursive', 'path']) =20 REQUIRED_ATTRIB_ERROR_MSG =3D "Required attribute malformed in <{}>: {}" NO_ASSOCIATED_REMOTE =3D 'There are no remotes associated with the ClientG= itHook entry:\nsource:{} destination:{}' \ @@ -138,6 +139,7 @@ class Manife= stXml(BaseXmlHelper): self._commit_templates =3D {} # dict of commit message t= emplates with the remote name as the key self._folder_to_folder_mappings =3D [] # List of FolderToFolderMa= pping objects self._submodule_alternate_remotes =3D [] + self._submodule_init_list =3D [] =20 # # Append include XML's to the Manifest etree before parsing @@ -19= 1,6 +193,13 @@ class ManifestXml(BaseXmlHelper): for element in subroot.iter(tag=3D'SubmoduleAlternateRemote'): self._submodule_alternate_remotes.append(_SubmoduleAlterna= teRemote(element, self._remotes)) =20 + # + # Determine submodule initialization paths + # + for subroot in self._tree.iter(tag=3D'SelectiveSubmoduleInitList')= : + for element in subroot.iter(tag=3D'Submodule'): + =20 + self._submodule_init_list.append(_SubmoduleInitEntry(element)) + # # parse tags # requires RemoteList to be parsed first @@ -386,6 +395,22 @@ clas= s ManifestXml(BaseXmlHelper): alternates.append(alternate.tuple) return alternates =20 + def get_submodule_init_paths(self, remote_name=3DNone, combo=3DNone): + submodule_list =3D [] + if remote_name is None and combo is None: + submodule_list =3D self._tuple_list(self._submodule_init_list) + elif remote_name is not None and combo is None: + submodule_list =3D self._tuple_list( + [x for x in self._submodule_init_list if x.remote_name =3D= =3D remote_name]) + elif remote_name is None and combo is not None: + submodule_list =3D self._tuple_list( + [x for x in self._submodule_init_list if x.combo =3D=3D co= mbo or x.combo is None]) + else: + submodule_list =3D self._tuple_list( + [x for x in self._submodule_init_list + if x.remote_name =3D=3D remote_name and (x.combo =3D=3D c= ombo or x.combo is None)]) + return submodule_list + def write_current_combo(self, combo_name, filename=3DNone): # # Updates the CurrentClonedCombo tag of _tree attribute and writes= the entire tree out to the @@ -460,6 +485,10 @@ class ManifestXml(BaseXmlH= elper): if self._tree.find('SubmoduleAlternateRemotes'): submodule_alt_url_root =3D ET.SubElement(pin_root, 'SubmoduleA= lternateRemotes') =20 + selective_submodules_root =3D None + if self._tree.find('SelectiveSubmoduleInitList'): + selective_submodules_root =3D ET.SubElement(pin_root,=20 + 'SelectiveSubmoduleInitList') + remote_root =3D ET.SubElement(pin_root, 'RemoteList') source_root =3D ET.SubElement(pin_root, 'Combination') source_root.attrib['name'] =3D self._combinations[combo_name].name @@ -487,6 +516,13 @@ class ManifestXml(BaseXmlHelper): if alt_url_element.attrib['remote'] =3D=3D src_tuple.r= emote_name: submodule_alt_url_root.append(alt_url_element) =20 + for subroot_selective_subs in self._tree.iter('SelectiveSubmod= uleInitList'): + for selective_sub in subroot_selective_subs.iter('Submodul= e'): + if selective_sub.attrib['remote'] =3D=3D src_tuple.rem= ote_name: + if 'combo' in selective_sub.attrib and selective_s= ub.attrib['combo'] !=3D combo_name: + continue + selective_submodules_root.append(selective_sub) + sparse =3D 'true' if src_tuple.sparse else 'false' sub =3D 'true' if src_tuple.enable_submodule else 'false' # Write the source element based on what value branch or commi= t is available. @@ -535,6 +571,10 @@ class ManifestXml(BaseXmlHelper): submodule_alt_url_root.text =3D '\n ' submodule_alt_url_root.tail =3D '\n\n ' list(submodule_alt_url_root)[-1].tail =3D '\n ' + if selective_submodules_root: + selective_submodules_root.text =3D '\n ' + selective_submodules_root.tail =3D '\n\n ' + list(selective_submodules_root)[-1].tail =3D '\n ' remote_root.text =3D '\n ' remote_root.tail =3D '\n\n ' list(remote_root)[-1].tail =3D '\n ' @@ -882,6 +922,27 @@ class _SubmoduleAlternateRemote(): return SubmoduleAlternateRemote(self.remote_name, self.originalUrl= , self.altUrl) =20 =20 +class _SubmoduleInitEntry(): + def __init__(self, element): + try: + self.remote_name =3D element.attrib['remote'] + self.path =3D element.text + except KeyError as k: + raise KeyError(REQUIRED_ATTRIB_ERROR_MSG.format(k, element.tag= )) + try: + self.combo =3D element.attrib['combo'] + except Exception: + self.combo =3D None + try: + self.recursive =3D element.attrib['recursive'].lower() =3D=3D = 'true' + except Exception: + self.recursive =3D False + + @property + def tuple(self): + return SubmoduleInitPath(self.remote_name, self.combo,=20 + self.recursive, self.path) + + # # Optional entry point for debug and validation of the CiIndexXml & Manife= stXml classes # @@ -956,6 +1017,23 @@ def main(): for alt in alts: print(alt) =20 + print('\nGet Submodule Init Objects') + print('\nAll:') + for entry in test_manifest.get_submodule_init_paths(): + print('+ {}'.format(entry)) + print('\nPer Remote:') + for remote in test_manifest.remotes: + for entry in test_manifest.get_submodule_init_paths(remote.nam= e): + print('+ {}'.format(entry)) + print('\nCurrent Combo:') + current_combo =3D test_manifest.general_config.current_combo + for entry in test_manifest.get_submodule_init_paths(combo=3Dcurren= t_combo): + print('+ {}'.format(entry)) + print('\nCurrent Combo Per Remote:') + for remote in test_manifest.remotes: + for entry in test_manifest.get_submodule_init_paths(remote.nam= e, current_combo): + print('+ {}'.format(entry)) + if not test_manifest.is_pin_file(): print('\nSparse settings:') print(test_manifest.sparse_settings) -- 2.21.0.windows.1