From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mga02.intel.com (mga02.intel.com [134.134.136.20]) by mx.groups.io with SMTP id smtpd.web12.10237.1589086044111742912 for ; Sat, 09 May 2020 21:47:24 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=@intel.onmicrosoft.com header.s=selector2-intel-onmicrosoft-com header.b=T77UYNQF; spf=pass (domain: intel.com, ip: 134.134.136.20, mailfrom: nathaniel.l.desimone@intel.com) IronPort-SDR: jloSiQoKsVFTTxoAocwnzC25ccdTUBhQ/1Mf28HmF6dbQGBGygjkmlR+AqDM8zHEYD6/79tudl ByAudgkJIESQ== X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga006.fm.intel.com ([10.253.24.20]) by orsmga101.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 09 May 2020 21:47:23 -0700 IronPort-SDR: L+C9B2TGRppedd02wTFjKC8/g3uVHXJewavApDJr5ftn8AckcQZJiFDtraGdDsDm77Uq2DUSLA 9T5A+gGHFv2g== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.73,374,1583222400"; d="scan'208";a="463014569" Received: from fmsmsx108.amr.corp.intel.com ([10.18.124.206]) by fmsmga006.fm.intel.com with ESMTP; 09 May 2020 21:47:23 -0700 Received: from FMSMSX110.amr.corp.intel.com (10.18.116.10) by FMSMSX108.amr.corp.intel.com (10.18.124.206) with Microsoft SMTP Server (TLS) id 14.3.439.0; Sat, 9 May 2020 21:47:23 -0700 Received: from FMSEDG001.ED.cps.intel.com (10.1.192.133) by fmsmsx110.amr.corp.intel.com (10.18.116.10) with Microsoft SMTP Server (TLS) id 14.3.439.0; Sat, 9 May 2020 21:47:22 -0700 Received: from NAM12-MW2-obe.outbound.protection.outlook.com (104.47.66.43) by edgegateway.intel.com (192.55.55.68) with Microsoft SMTP Server (TLS) id 14.3.439.0; Sat, 9 May 2020 21:47:22 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=HI61Dcekwwv3UN7/R71hZchgzFV3o6cIawITVZS1xo1M4sgWqeBm7qxPZRZcHk1Gar50eY25hucOSCBOkMq1JSb1ZRvrnzZYwxBty3unUIEK+t4vlzcuBBPKM8GUvPNHBk7YquueVT8o6/pEGdm8PFZnmlAKePTMvJBMnTaN+5AVq+GLNUCrfHZr75+PO+MvdN2aPQ3jeOkfs2mLKbjM9OskXe8eDM1agrX0NZEDZ8ECkB5RjURO0mzgXWpogpb7P9fjkS1Au2VFVkha11g+y4+Jxu0CWG01MH/9i5fy7AqFFpVlJNUiOSdmxUxwUuTE1F8lw7ebmIhGWny/j1K9Jg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=B3EOyfdn8+IcKMnuBiRO2Z/AC1dWgBB3vlTJ8n03scE=; b=cYuaRSVFYklB7voaHxm/ePESTJJPtZw4Pawq4kZLZHHgf+pkZbB5CLaJ95rRNUdRj2XAZ4ax9qIH9ie4sVQAcUccm0DzFTgTrHveOrsYpQAbdB0jlEEjl59xhqaJtY5NU/ewAAiuYp6ijOYBe+67kxOYfl9TpmZM1hFfRkZRyNLa3SozCZEoEswPAjxbEmplgOs+wJu0VoSLfTMgNC9fTOtoU/EPeLbHMVy3pq8Em8gi4vv9NBNjz0BHxWS0B/7HdoWXrLitPNWHaUfH9ryPq1lzPcxyNxtMpCosMW0hUfBtYwwKgX79YkpdRUcDXtKU/QAVgXJSghk7xiA3x3XOCQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com; dkim=pass header.d=intel.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=intel.onmicrosoft.com; s=selector2-intel-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=B3EOyfdn8+IcKMnuBiRO2Z/AC1dWgBB3vlTJ8n03scE=; b=T77UYNQFbmbTmwtH+dI5MteW/d8ZCt88b5glrXqcndOHRWJyoPdlfVFSi3qmF4RNIdNbeWOXTyKoHld/e38aDjanvxL9qHPugCpbMgE3i9CPNX/1p5Hv729YhAs2DBMgGcOqhp9E0JCsiwu95TtiWFL94Byn7E2xDO3gfgCgfpA= Received: from BL0PR11MB3489.namprd11.prod.outlook.com (2603:10b6:208:75::29) by BL0PR11MB3460.namprd11.prod.outlook.com (2603:10b6:208:6e::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2979.26; Sun, 10 May 2020 04:47:21 +0000 Received: from BL0PR11MB3489.namprd11.prod.outlook.com ([fe80::d0f8:bfd7:bae2:3e42]) by BL0PR11MB3489.namprd11.prod.outlook.com ([fe80::d0f8:bfd7:bae2:3e42%7]) with mapi id 15.20.2979.033; Sun, 10 May 2020 04:47:21 +0000 From: "Nate DeSimone" To: "Desimone, Ashley E" , "devel@edk2.groups.io" CC: "Pandya, Puja" , "Bjorge, Erik C" , Bret Barkelew , "Agyeman, Prince" Subject: Re: [edk2-staging/EdkRepo] [PATCH V2] EdkRepo: Integrate multiple manifest repository support in the manifest command Thread-Topic: [edk2-staging/EdkRepo] [PATCH V2] EdkRepo: Integrate multiple manifest repository support in the manifest command Thread-Index: AQHWJoJ8p90RDORw6U6NRId0CHpyY6igvyLw Date: Sun, 10 May 2020 04:47:21 +0000 Message-ID: References: <20200510042108.20544-1-ashley.e.desimone@intel.com> In-Reply-To: <20200510042108.20544-1-ashley.e.desimone@intel.com> Accept-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-titus-metadata-40: eyJDYXRlZ29yeUxhYmVscyI6IiIsIk1ldGFkYXRhIjp7Im5zIjoiaHR0cDpcL1wvd3d3LnRpdHVzLmNvbVwvbnNcL0ludGVsMyIsImlkIjoiZGE4MTBhNzEtM2U1YS00YmY5LTk2M2ItYzg0NmIyY2RmMGRjIiwicHJvcHMiOlt7Im4iOiJDVFBDbGFzc2lmaWNhdGlvbiIsInZhbHMiOlt7InZhbHVlIjoiQ1RQX05UIn1dfV19LCJTdWJqZWN0TGFiZWxzIjpbXSwiVE1DVmVyc2lvbiI6IjE3LjEwLjE4MDQuNDkiLCJUcnVzdGVkTGFiZWxIYXNoIjoiNnV6WTFvMlIrK3hCVDU3VngxQ0RYZVRKQVhCWTFmblBFd24rYVI3S0xEckNZWElOcWhVVDFvcDhPTWdieFY1ViJ9 dlp-reaction: no-action dlp-version: 11.2.0.6 dlp-product: dlpe-windows x-ctpclassification: CTP_NT authentication-results: intel.com; dkim=none (message not signed) header.d=none;intel.com; dmarc=none action=none header.from=intel.com; x-originating-ip: [50.53.190.176] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 2f71fe6d-a866-490d-b517-08d7f49d39d7 x-ms-traffictypediagnostic: BL0PR11MB3460: x-ms-exchange-transport-forked: True x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:3276; x-forefront-prvs: 039975700A x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: qe6oUVlR0AGsTpaoWKl9s2niS5TWjFphzMpbSMHTwx2rdoP0CxguQIeMPi2319Wl/ITfZ9fnDesYaCvuhCHCC7VX8EzFpDNChGYpc0/M13pqp/uhU6RkGLKM2oqaRL5N2Ad5Ch3Xz2hIq/ICXSk+/f1g+C9waaYYMqtfbG0VrhEFGFX8x916v7HfD6eCq8PYieJFA08CDbl2Pc61jOZL5K1qKfTqXvjDr+tdrzrcO/SkRoVdk6pOpMQxhMh3UcTFqskRwDZfdYNLM7INMooQ3+1Tjx13VV9XWYxDvwAnHEwsmaRQRk53Q4WyTGMutvAZswgiDf/oA4x0YucAzh5rBbqNikZR108zbWkJIXM8AWuk7OHPZKcTEY+jPe/5See0UWHOFzlgZeWLRybfOXtForJvA8wKhBg/67TW3dn0v99qMXD4QsKKUSMfz/ydfWL9OgHVTDWKfTVcDFhxgkH9E7i8UnQXRz3uNiHApVMSm+hZRFt39ZSKRIuFipdWXrou/GEW5hKUvx4FOCxfLTDWww== x-forefront-antispam-report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BL0PR11MB3489.namprd11.prod.outlook.com;PTR:;CAT:NONE;SFTY:;SFS:(39860400002)(376002)(136003)(366004)(396003)(346002)(33430700001)(110136005)(33656002)(33440700001)(86362001)(54906003)(8676002)(8936002)(53546011)(478600001)(7696005)(6506007)(52536014)(107886003)(316002)(66946007)(5660300002)(4326008)(71200400001)(55016002)(66556008)(66446008)(9686003)(186003)(66476007)(64756008)(26005)(76116006)(2906002);DIR:OUT;SFP:1102; x-ms-exchange-antispam-messagedata: Szi3M/Ljpf4RRaU8WHzzrVh5/JV/YbW73UUWOeccLMmPOElvL5A7NHXwn87lXzkUgC7mVqct6zEtUIvWPIdS9bQIQHU8CyEMZjIKd00LS253VrFB5uBG7IDSWj6J0yD/W5QcNmsq0LfFUJE/CRrtWSbmP8pmZoVknNeLHKEer9Z08O+vXJd+aKtAIKuHVaqR8a38joPyCLZ6mnTdMS7i7mvwypvmjClbkZQfseW/sWjpJvmJ6QFQjlWADrmsw+pQI1l0yHbMvUMrNr4hPbipvZx+ZKoGi1/WzMGGwBv70MwNBYmMRsfWrclSoNtWy85FanrUxZVj9/gTCTilVGdCK/p2g9UbbYQiylvo57588tGkN9/8LrdQfa5ZB2OhVTBwb757UpYWZwjZF7rdaZRBym0CE3VwGVp+zXR5Eie6jTuiF+aSZx+EO77vq3Nhv9PzhogjDuvPMDqM3JKVYrSVpTNG/T0W/ZGFjYHwAj7lYFniELaJ9PHrQxeTL951xVZR MIME-Version: 1.0 X-MS-Exchange-CrossTenant-Network-Message-Id: 2f71fe6d-a866-490d-b517-08d7f49d39d7 X-MS-Exchange-CrossTenant-originalarrivaltime: 10 May 2020 04:47:21.2925 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 46c98d88-e344-4ed4-8496-4ed7712e255d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: 6iYwdrk99UnkwD694WCnaQ+w+KDstrBS93J3ags6G3A+UNePGSx0aIlFgYQbkCTFWcovdD3/nqFwjRlgtgcVkti9bqFA/XzuGcs2O47JQfQ= X-MS-Exchange-Transport-CrossTenantHeadersStamped: BL0PR11MB3460 Return-Path: nathaniel.l.desimone@intel.com X-OriginatorOrg: intel.com Content-Language: en-US Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Pushed: d574765910ea770e57809be232f1dcad8c6ef5ba > -----Original Message----- > From: Desimone, Ashley E > Sent: Saturday, May 9, 2020 9:21 PM > To: devel@edk2.groups.io > Cc: Desimone, Nathaniel L ; Pandya, Puja > ; Bjorge, Erik C ; Bret > Barkelew ; Agyeman, Prince > > Subject: [edk2-staging/EdkRepo] [PATCH V2] EdkRepo: Integrate multiple > manifest repository support in the manifest command >=20 > Integrate multiple manifest repository support in the manifest command >=20 > Add the validate_manifest_repo() function to common_repo_functions >=20 > Signed-off-by: Ashley E Desimone > Cc: Nate DeSimone > Cc: Puja Pandya > Cc: Erik Bjorge > Cc: Bret Barkelew > Cc: Prince Agyeman > --- > edkrepo/commands/manifest_command.py | 86 > +++++++++++++++++++++------------ > edkrepo/common/common_repo_functions.py | 14 +++++- > 2 files changed, 68 insertions(+), 32 deletions(-) >=20 > diff --git a/edkrepo/commands/manifest_command.py > b/edkrepo/commands/manifest_command.py > index 44218c9..bb6252d 100644 > --- a/edkrepo/commands/manifest_command.py > +++ b/edkrepo/commands/manifest_command.py > @@ -3,7 +3,7 @@ > ## @file > # manifest_command.py > # > -# Copyright (c) 2017- 2019, Intel Corporation. All rights reserved.
> +# Copyright (c) 2017- 2020, Intel Corporation. All rights reserved.
> # SPDX-License-Identifier: BSD-2-Clause-Patent # >=20 > @@ -15,9 +15,12 @@ from colorama import Fore from > edkrepo.commands.edkrepo_command import EdkrepoCommand from > edkrepo.commands.edkrepo_command import ColorArgument import > edkrepo.commands.arguments.manifest_args as arguments -from > edkrepo.common.edkrepo_exception import > EdkrepoWorkspaceInvalidException -from > edkrepo.common.common_repo_functions import > pull_latest_manifest_repo, verify_manifest_data > +from edkrepo.common.edkrepo_exception import > +EdkrepoWorkspaceInvalidException, EdkrepoManifestNotFoundException > from > +edkrepo.common.common_repo_functions import > pull_latest_manifest_repo, > +validate_manifest_repo > from edkrepo.common.ui_functions import init_color_console > +from > edkrepo.common.workspace_maintenance.manifest_repos_maintenance > +import list_available_manifest_repos from > +edkrepo.common.workspace_maintenance.manifest_repos_maintenance > import > +pull_all_manifest_repos from > +edkrepo.common.workspace_maintenance.manifest_repos_maintenance > import > +find_source_manifest_repo > from edkrepo.config.config_factory import get_workspace_manifest from > edkrepo_manifest_parser.edk_manifest import CiIndexXml >=20 > @@ -44,46 +47,67 @@ class ManifestCommand(EdkrepoCommand): > print() > init_color_console(args.color) >=20 > - # Get path to global manifest file > - global_manifest_directory =3D > config['cfg_file'].manifest_repo_abs_local_path > - if args.verbose: > - print("Manifest directory:") > - print(global_manifest_directory) > - print() > - index_path =3D os.path.join(global_manifest_directory, 'CiIndex.= xml') > + cfg_file =3D config['cfg_file'] > + user_cfg =3D config['user_cfg_file'] > + cfg_man_repos, user_cfg_man_repos, conflicts =3D > list_available_manifest_repos(cfg_file, user_cfg) > + man_repos =3D {} >=20 > - pull_latest_manifest_repo(args, config) > - print() > + pull_all_manifest_repos(cfg_file, user_cfg, False) >=20 > - ci_index_xml =3D CiIndexXml(index_path) > + # Get paths to the global manifest dirs and their index files > + for repo in cfg_man_repos: > + global_manifest_directory =3D cfg_file.manifest_repo_abs_pat= h(repo) > + index_path =3D os.path.join(global_manifest_directory, 'CiIn= dex.xml') > + man_repos[repo] =3D (global_manifest_directory, index_path) > + for repo in user_cfg_man_repos: > + global_manifest_directory =3D > user_cfg.manifest_repo_abs_path(repo) > + index_path =3D os.path.join(global_manifest_directory, 'CiIn= dex.xml') > + man_repos[repo] =3D (global_manifest_directory, index_path) >=20 > try: > - current_project =3D get_workspace_manifest().project_info.co= dename > + wkspc_manifest =3D get_workspace_manifest() > + current_project =3D wkspc_manifest.project_info.codename > + src_man_repo =3D find_source_manifest_repo(wkspc_manifest, > + cfg_file, user_cfg, None) > except EdkrepoWorkspaceInvalidException: > current_project =3D None > + src_man_repo =3D None > + except EdkrepoManifestNotFoundException: > + src_man_repo =3D None >=20 > - # Attempt to make sure the manifest data is good > - try: > - verify_manifest_data(global_manifest_directory, config, > verbose=3Dargs.verbose, verify_all=3DTrue, verify_archived=3Dargs.archive= d) > - except: > - print() >=20 > - print("Projects:") > - for project in ci_index_xml.project_list: > - if project =3D=3D current_project: > - print("* {}{}{}".format(Fore.GREEN, project, Fore.RESET)= ) > - else: > - print(" {}".format(project)) > + for repo in man_repos.keys(): > + print() > + print("Manifest directory:") > + print(repo) > if args.verbose: > - print(" -> {}".format(ci_index_xml.get_project_xml(pro= ject))) > - > - if args.archived: > + print('Manifest directory path:') > + print(man_repos[repo][0]) > print() > - print("Archived Projects:") > - for project in ci_index_xml.archived_project_list: > - if project =3D=3D current_project: > + > + ci_index_xml =3D CiIndexXml(man_repos[repo][1]) > + > + # Attempt to make sure the manifest data is good > + try: > + validate_manifest_repo(man_repos[repo][0], args.verbose, > args.archived) > + except: > + print() > + > + print("Projects:") > + for project in sorted(ci_index_xml.project_list): > + if (project =3D=3D current_project and src_man_repo =3D= =3D repo) or (not > src_man_repo and project =3D=3D current_project): > print("* {}{}{}".format(Fore.GREEN, project, Fore.RE= SET)) > else: > print(" {}".format(project)) > if args.verbose: > print(" -> {}".format(ci_index_xml.get_project_xml= (project))) > + > + if args.archived: > + print() > + print("Archived Projects:") > + for project in sorted(ci_index_xml.archived_project_list= ): > + if project =3D=3D current_project: > + print("* {}{}{}".format(Fore.GREEN, project, For= e.RESET)) > + else: > + print(" {}".format(project)) > + if args.verbose: > + print(" -> {}".format(ci_index_xml.get_project= _xml(project))) > diff --git a/edkrepo/common/common_repo_functions.py > b/edkrepo/common/common_repo_functions.py > index 20dcb8a..eb6c4c0 100644 > --- a/edkrepo/common/common_repo_functions.py > +++ b/edkrepo/common/common_repo_functions.py > @@ -54,6 +54,7 @@ from edkrepo.common.pathfix import get_actual_path > from project_utils.sparse import BuildInfo, process_sparse_checkout from > edkrepo.config.config_factory import get_workspace_path from > edkrepo.config.config_factory import get_workspace_manifest > +from edkrepo.config.tool_config import CI_INDEX_FILE_NAME > from edkrepo.common.edkrepo_exception import > EdkrepoInvalidParametersException from > edkrepo_manifest_parser.edk_manifest import CiIndexXml, ManifestXml > from edkrepo.common.edkrepo_exception import > EdkrepoNotFoundException, EdkrepoGitException, > EdkrepoWarningException @@ -426,11 +427,22 @@ def > verify_manifest_data(global_manifest_directory, config, verbose=3DFalse, > verif > if verbose: > print_manifest_errors(manifestfile_validation_data) >=20 > +def validate_manifest_repo(manifest_repo, verbose=3DFalse, > archived=3DFalse): > + print(VERIFY_GLOBAL) > + if archived: > + print(VERIFY_ARCHIVED) > + manifest_validation_data =3D validate_manifestrepo(manifest_repo, > archived) > + manifest_repo_error =3D > get_manifest_validation_status(manifest_validation_data) > + if manifest_repo_error: > + print(VERIFY_GLOBAL_FAIL) > + if verbose: > + print_manifest_errors(manifest_validation_data) > + > def verify_single_manifest(cfg_file, manifest_repo, manifest_path, > verbose=3DFalse): > manifest =3D ManifestXml(manifest_path) > print(VERIFY_PROJ.format(manifest.project_info.codename)) > index_path =3D > os.path.join(cfg_file.manifest_repo_abs_path(manifest_repo), > CI_INDEX_FILE_NAME) > - proj_val_data =3D validate_manifestfiles(index_path, [manifest_path]= ) > + proj_val_data =3D validate_manifestfiles([manifest_path]) > proj_val_error =3D get_manifest_validation_status(proj_val_data) > if proj_val_error: > if verbose: > -- > 2.16.2.windows.1