From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mga18.intel.com (mga18.intel.com [134.134.136.126]) by mx.groups.io with SMTP id smtpd.web10.994.1588282103199315923 for ; Thu, 30 Apr 2020 14:28:23 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=@intel.onmicrosoft.com header.s=selector2-intel-onmicrosoft-com header.b=ULjpZ5RW; spf=pass (domain: intel.com, ip: 134.134.136.126, mailfrom: nathaniel.l.desimone@intel.com) IronPort-SDR: 49Oa7/rt7971+LGW2aewENzQIaiZXJHjAMHB+NX8Gzuz/AtEEKoGE54y74nSWgkIp6iy6JyrQf jGfcTWhallxQ== X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga006.jf.intel.com ([10.7.209.51]) by orsmga106.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 30 Apr 2020 14:28:22 -0700 IronPort-SDR: xloFhbmVTOtc/QAWg7irdpe+CcbnPPN6TEFUOQYmxDrBqqRkvwyPB2TJfkUOBw13EDP71riwEL WHVTnRMSimqA== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.73,337,1583222400"; d="scan'208";a="261917221" Received: from orsmsx108.amr.corp.intel.com ([10.22.240.6]) by orsmga006.jf.intel.com with ESMTP; 30 Apr 2020 14:28:21 -0700 Received: from ORSEDG001.ED.cps.intel.com (10.7.248.4) by ORSMSX108.amr.corp.intel.com (10.22.240.6) with Microsoft SMTP Server (TLS) id 14.3.439.0; Thu, 30 Apr 2020 14:28:21 -0700 Received: from NAM04-BN3-obe.outbound.protection.outlook.com (104.47.46.50) by edgegateway.intel.com (134.134.137.100) with Microsoft SMTP Server (TLS) id 14.3.439.0; Thu, 30 Apr 2020 14:28:21 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=UoBurh8k4zMuQ1VMteqvIbNqOaGrlPY+y1C/R2ZRypsDsuuMmXarWaLweamOy99Sx11B9LOL+vu60+b7NeP1r0i2LN0Dl8z+S7xIXEMu4YDu2Xt8m9Cj90DR3dbda8iWfemRyB7xejIGCIbW7opaRRA8XcECJFUNbf/KbDmKbn5VdKKyg/YEqwqw2MPhBjIzfS2pgRaRgT7LD9bhzH+hSX7KSK/uB95EJ5kCqTYPTimZyCT45Proc0EzJNFyIyYxtwLvEeDaPdpufc0vneGLzIA07qiTK5wQNKt1lfTBoP0HUz46EqSGBhjDZ3J9y5nW55URrShV0NgbfwnHDOTPVw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=cMwfl1AAJsGHMXriEUoDN2Y7fUiXq+0xt3VXEzRdC6Y=; b=Mdkq1oUyLPQs3mQjiK8VLSR9BtGIxiVox6q5mdQgOY/LxC8ude3P3Gqlj1wz0T3kSiHl1ZGrtjVDyK5mvCu1Tqiun2mw1FoAE27pL/+DhlPhvYio1oEyXb9UVG4bOwDfEBryozsJafya8+oZNw2ZX9yO5IspZL0/4CHd+I7rs2gfLQAFf8W0jWOmHaJGffPdWPwfFjrK/TSBlmGcRrE8B66B6WA+JgjvyYiK9pr4a5uJBR48XymPtxXTHzxIKI/8ttpz2jyu2ASLs1DfZi9sNaqnWLzByXVwgBm7tCGy71umh1cXUYhiN86og17QECjBZx1YTGO8ywSQP4szEAcwkw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com; dkim=pass header.d=intel.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=intel.onmicrosoft.com; s=selector2-intel-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=cMwfl1AAJsGHMXriEUoDN2Y7fUiXq+0xt3VXEzRdC6Y=; b=ULjpZ5RWYfzUmHTPSiibSVtU4ho9VvMYvaNOfQnnNHO7B/MrB9cCjv5RXV/lBRZjRIT1JSyHTxS4QpefH6hUshP4uRU13DAyrKO9pb17XYA4FRMOP06JYAuqhrd8N2c0LzkDobthZmC9eCno0AfmryWvohDuPdZQtrY0IdXiaBE= Received: from BL0PR11MB3489.namprd11.prod.outlook.com (2603:10b6:208:75::29) by BL0PR11MB3075.namprd11.prod.outlook.com (2603:10b6:208:7a::14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2958.20; Thu, 30 Apr 2020 21:28:20 +0000 Received: from BL0PR11MB3489.namprd11.prod.outlook.com ([fe80::d0f8:bfd7:bae2:3e42]) by BL0PR11MB3489.namprd11.prod.outlook.com ([fe80::d0f8:bfd7:bae2:3e42%7]) with mapi id 15.20.2937.026; Thu, 30 Apr 2020 21:28:20 +0000 From: "Nate DeSimone" To: "devel@edk2.groups.io" , "Desimone, Ashley E" CC: "Pandya, Puja" , "Bjorge, Erik C" , Bret Barkelew , "Agyeman, Prince" Subject: Re: [edk2-devel] [edk2-staging/EdkRepo] [PATCH 2/7] EdkRepo: Add downloading all available manifest repositories Thread-Topic: [edk2-devel] [edk2-staging/EdkRepo] [PATCH 2/7] EdkRepo: Add downloading all available manifest repositories Thread-Index: AQHWHagOliCjFjeKbEOP/9Tfu2B2zaiSMPaQ Date: Thu, 30 Apr 2020 21:28:20 +0000 Message-ID: References: <20200428215710.45504-1-ashley.e.desimone@intel.com> <20200428215710.45504-3-ashley.e.desimone@intel.com> In-Reply-To: <20200428215710.45504-3-ashley.e.desimone@intel.com> Accept-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-titus-metadata-40: eyJDYXRlZ29yeUxhYmVscyI6IiIsIk1ldGFkYXRhIjp7Im5zIjoiaHR0cDpcL1wvd3d3LnRpdHVzLmNvbVwvbnNcL0ludGVsMyIsImlkIjoiYzdlNWZkNjktODk2NC00YWQ1LTlmNDAtNjY1MWVkMTdiNTg2IiwicHJvcHMiOlt7Im4iOiJDVFBDbGFzc2lmaWNhdGlvbiIsInZhbHMiOlt7InZhbHVlIjoiQ1RQX05UIn1dfV19LCJTdWJqZWN0TGFiZWxzIjpbXSwiVE1DVmVyc2lvbiI6IjE3LjEwLjE4MDQuNDkiLCJUcnVzdGVkTGFiZWxIYXNoIjoiYUhaQmNBVExpV0JhKzUwZlhsc2pHOUtXYnJjZ01sUmxwTEF1MnN2bnpcL0FocE94VlBQUkRLcjFIQkptNW45ZXoifQ== dlp-reaction: no-action dlp-version: 11.2.0.6 dlp-product: dlpe-windows x-ctpclassification: CTP_NT authentication-results: edk2.groups.io; dkim=none (message not signed) header.d=none;edk2.groups.io; dmarc=none action=none header.from=intel.com; x-originating-ip: [134.134.136.198] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: e4c75edb-4297-4685-d626-08d7ed4d6793 x-ms-traffictypediagnostic: BL0PR11MB3075: x-ms-exchange-transport-forked: True x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:8273; x-forefront-prvs: 0389EDA07F x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: 62hyuvfn2i1BJuji5X2oVSAwSC/IVh5TYjgfxBmkOwwDI8SAGVbOuKBlV4aBVKjPBSPGCtRzd0XEVyaf52E9sSxo08lPfnEt6cTL9ijgaCltTWCL/5Y+B5Xzs474hcR1MbJz4gLSaEM4DAleaMzx4/+ABdXZNkdxsQL68Pk8rfg7WxvOBHPMTTFCCbIz1aq2p7w+Bi/NFR9wIjUgDICrsIYTG9jHzXd2JtbjPjQDNvvp2OQLn2Iv47Q4Y9cOAvYcKWcIhdbabs7WXGEv507TZHZacpJy6PfiOQ6p4jDrXT5E8ZM7Y6GKnNqcalmWA4/VoXWkwVNwDZMj5tdtGsoodDz/MKY5/IG8L4IyHmx1aGCVtFdDCCw/i3g2rNfC/fjhwlhA9kYQnEinRCn5BEBkzVDqQRCSktpqcQ42kTviLoVm4/uRJUIIkPGxZX/uR28sEyudiiqWjW7wLp+mfIzz4ST9FCqBdEkHx5ABbZncPaYfPZeHF12kfoMomzZmfrTmeSsQEjN8RO/Xpg/2mlKsIQ== x-forefront-antispam-report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BL0PR11MB3489.namprd11.prod.outlook.com;PTR:;CAT:NONE;SFTY:;SFS:(136003)(376002)(366004)(346002)(39860400002)(396003)(966005)(478600001)(86362001)(6506007)(54906003)(71200400001)(26005)(53546011)(316002)(186003)(6636002)(107886003)(5660300002)(110136005)(7696005)(4326008)(8936002)(66946007)(2906002)(66476007)(55016002)(64756008)(66446008)(66556008)(52536014)(33656002)(9686003)(76116006)(8676002);DIR:OUT;SFP:1102; x-ms-exchange-antispam-messagedata: B7xcQpO/0tsJI07iMnHfJdlqRPVV1nu9s0ZiukEClw50OMJqiox4Fsz/9Tt+JOsX9N9ww/uJEVGop7mGMRSED9FJpMV56+29g57TzwMInmiRBQu0UrUQvjcyobVBh+hI1vUk4aAqEn1gdV0YY43qCoZZTWdT5g86AI7dWWSZmebAdBhHcXfnqPvuu2a8GWxEipHkjyw0L39OJAVtEqx/Ko/xnP7eOYt4s+cEp2TyIxWDS503WDkXiq6/sA3bZlq3yQyAivp2vgki9epKpFKbW6NgE1I+MbKBZ4EreArzxsUWSc7OIhS9JYOZginH8XGXVRHTcwWL0u3L9nLkibUrLoCd4mKzPNMA6Dz63Q3LzwsMcZ82f3XAMiLgcXchOAbEe0yJZaXEPdUlyxlGeI7lXN6/LrxsP9bDqe/n/+9t/bEFRIIFp2RNJqOLjnuHGoF/YNhH9z203XaRjD7HxKMUmkbjwetFkfFF3RfAZJZ7bnX0mVcwD4Aowa3/ABSmirA1TuBB67Vd6hz4reLCIZzZLMq7jgQPqYrwzkWmsXnFq+sHSEe2NDuLU9knfUkwyOevarbATn4ZWaJ+4+6wvMqcW1FkyG+mIWV0tblPpRfrvBQNWRNKE92lkK5i9YRKHnJljXO9g+Rx2DWPBgdnDTOHlrIUZ5fDNLEhpkK9k4q04RJMvCXBTjGoqljt8kQbw1oUWc0/TCF90GzEhUuln6zVrImA99yh6s0i96kqvqz7ZSskGmzIiegQ3wz4lvCMNC+3ebrYu8dy8cv9gTvGZJigNgySo88EdR2STEWPaM/tB4B3LqFGzWWkCe5+qMFNu1pq MIME-Version: 1.0 X-MS-Exchange-CrossTenant-Network-Message-Id: e4c75edb-4297-4685-d626-08d7ed4d6793 X-MS-Exchange-CrossTenant-originalarrivaltime: 30 Apr 2020 21:28:20.1331 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 46c98d88-e344-4ed4-8496-4ed7712e255d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: RP8BQCvO+5lgXFpSozlpMXBAu0pIXRMthg2rJUPHYgdVjYYkohWQILnvDPpzTP4coUKMqe5Ivxc60W+m+blU2epCwttoVcXlkt6g1JickX8= X-MS-Exchange-Transport-CrossTenantHeadersStamped: BL0PR11MB3075 Return-Path: nathaniel.l.desimone@intel.com X-OriginatorOrg: intel.com Content-Language: en-US Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Reviewed-by: Nate DeSimone > -----Original Message----- > From: devel@edk2.groups.io On Behalf Of Ashley E > Desimone > Sent: Tuesday, April 28, 2020 2:57 PM > To: devel@edk2.groups.io > Cc: Desimone, Nathaniel L ; Pandya, Puja > ; Bjorge, Erik C ; Bret > Barkelew ; Agyeman, Prince > > Subject: [edk2-devel] [edk2-staging/EdkRepo] [PATCH 2/7] EdkRepo: Add > downloading all available manifest repositories >=20 > Add a function that will download all available manifest repositories de= fined > in either the edkrepo.cfg or the edkrepo_user.cfg >=20 > Signed-off-by: Ashley E Desimone > Cc: Nate DeSimone > Cc: Puja Pandya > Cc: Erik Bjorge > Cc: Bret Barkelew > Cc: Prince Agyeman > --- > .../humble/manifest_repos_maintenance_humble.py | 4 +++ > .../manifest_repos_maintenance.py | 38 > ++++++++++++++++++++++ > 2 files changed, 42 insertions(+) >=20 > diff --git > a/edkrepo/common/workspace_maintenance/humble/manifest_repos_ma > intenance_humble.py > b/edkrepo/common/workspace_maintenance/humble/manifest_repos_ma > intenance_humble.py > index 440fd8a..e592f19 100644 > --- > a/edkrepo/common/workspace_maintenance/humble/manifest_repos_ma > intenance_humble.py > +++ > b/edkrepo/common/workspace_maintenance/humble/manifest_repos_ma > inten > +++ ance_humble.py > @@ -21,3 +21,7 @@ SINGLE_MAN_REPO_NOT_CFG_BRANCH =3D ('The > current active branch, {}, is not the ' > 'specified branch for global manifst = repository: {}') > SINGLE_MAN_REPO_CHECKOUT_CFG_BRANCH =3D 'Checking out the > specified branch: {} prior to syncing' > SINGLE_MAN_REPO_MOVED =3D '{}{}WARNING:{}{} The global manifest > repository has moved. Backing up previous global manifest repository to: > {{}}{}\n'.format(Style.BRIGHT, Fore.RED, Style.RESET_ALL, Fore.RED, > Style.RESET_ALL) > +CONFLICT_NO_CLONE =3D ('The definition of global manifest repository, {= }, ' > + 'in the edkrepo_user.cfg does not match the defini= tion in the > edkrepo.cfg. ' > + 'This global manifest repository will not be downl= oaded or > updated. ' > + 'Resolve the conflict and then re-run the failed > +operation') > diff --git > a/edkrepo/common/workspace_maintenance/manifest_repos_maintenanc > e.py > b/edkrepo/common/workspace_maintenance/manifest_repos_maintenanc > e.py > index ad6ddbc..24ad76a 100644 > --- > a/edkrepo/common/workspace_maintenance/manifest_repos_maintenanc > e.py > +++ > b/edkrepo/common/workspace_maintenance/manifest_repos_maintenanc > e.py > @@ -57,6 +57,44 @@ def pull_single_manifest_repo(url, branch, local_path= , > reset_hard=3DFalse): > print (humble.CLONE_SINGLE_MAN_REPO.format(local_path, url)= ) > repo =3D Repo.clone_from(url, local_path, > progress=3DGitProgressHandler(), branch=3Dbranch) >=20 > +def pull_all_manifest_repos(edkrepo_cfg, edkrepo_user_cfg, > reset_hard=3DFalse): > + ''' > + Clones or syncs all global manifest repositories defined in both th= e > + edkrepo_cfg and the edkrepo_user.cfg) > + ''' > + cfg_man_repos =3D [] > + user_cfg_man_repos =3D [] > + conflicts, duplicates =3D > detect_man_repo_conflicts_duplicates(edkrepo_cfg, edkrepo_user_cfg) > + if not conflicts and not duplicates: > + cfg_man_repos.extend(edkrepo_cfg.manifest_repo_list) > + user_cfg_man_repos.extend(edkrepo_user_cfg.manifest_repo_list) > + elif conflicts: > + for conflict in conflicts: > + # In the case of a conflict do not pull conflicting repo > + print(humble.CONFLICT_NO_CLONE.format(conflict)) > + cfg_man_repos.extend(edkrepo_cfg.manifest_repo_list) > + cfg_man_repos.remove(conflict) > + user_cfg_man_repos.extend(edkrepo_user_cfg.manifest_repo_li= st) > + user_cfg_man_repos.remove(conflict) > + elif duplicates: > + for duplicate in duplicates: > + # the duplicate needs to be ignored in on of the repo lists= so it is > + # not cloned/pulled twice > + cfg_man_repos.extend(edkrepo_cfg.manifest_repo_list) > + user_cfg_man_repos.extend(edkrepo_user_cfg.manifest_repo_li= st) > + user_cfg_man_repos.remove(conflict) > + for repo in cfg_man_repos: > + pull_single_manifest_repo(edkrepo_cfg.get_manifest_repo_url(rep= o), > + edkrepo_cfg.get_manifest_repo_branch(= repo), > + edkrepo_cfg.get_manifest_repo_local_p= ath(repo), > + reset_hard) > + for repo in user_cfg_man_repos: > + > pull_single_manifest_repo(edkrepo_user_cfg.get_manifest_repo_url(repo) > , > + edkrepo_user_cfg.get_manifest_repo_br= anch(repo), > + edkrepo_user_cfg.get_manifest_repo_lo= cal_path(repo), > + reset_hard) > + > + > def detect_man_repo_conflicts_duplicates(edkrepo_cfg, > edkrepo_user_cfg): > ''' > Determines whether there is are conflicting or duplicated manifest > -- > 2.16.2.windows.1 >=20 >=20 >=20