From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mga03.intel.com (mga03.intel.com [134.134.136.65]) by mx.groups.io with SMTP id smtpd.web12.10226.1589085933711042977 for ; Sat, 09 May 2020 21:45:33 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=@intel.onmicrosoft.com header.s=selector2-intel-onmicrosoft-com header.b=m+WICZfN; spf=pass (domain: intel.com, ip: 134.134.136.65, mailfrom: nathaniel.l.desimone@intel.com) IronPort-SDR: zbSuyRlZiIDse1TgZtv8tXnxprqgPhgc/tKVbgDZX/DvZ2p1kD5+Sch6zQzqRlxuu9Qw8WUiNt pNGSihshVCnw== X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga008.jf.intel.com ([10.7.209.65]) by orsmga103.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 09 May 2020 21:45:33 -0700 IronPort-SDR: Cr+i/MuQtFUsSJjbpq5Gwx1ZXZRk73E3wV9XryhsSoggpM+OwDMY9AKMma+cwGgSrpswAD6rGf Wfjz0LtoUyXA== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.73,374,1583222400"; d="scan'208";a="297344186" Received: from orsmsx106.amr.corp.intel.com ([10.22.225.133]) by orsmga008.jf.intel.com with ESMTP; 09 May 2020 21:45:33 -0700 Received: from ORSEDG001.ED.cps.intel.com (10.7.248.4) by ORSMSX106.amr.corp.intel.com (10.22.225.133) with Microsoft SMTP Server (TLS) id 14.3.439.0; Sat, 9 May 2020 21:45:32 -0700 Received: from NAM12-MW2-obe.outbound.protection.outlook.com (104.47.66.49) by edgegateway.intel.com (134.134.137.100) with Microsoft SMTP Server (TLS) id 14.3.439.0; Sat, 9 May 2020 21:45:32 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=HTciGN6jJdOvF4Rqk2NLhTlSPpUDHtD/VMWfFF6OYivHy9CYtBo7wxX8rl0q72LdzPytuWHd7wwJLYzBLysEMRgb/wJr1nw/PJQkjxRWKOuCt0U7K0NJkIGCfn3ROMwmMcZPcMctLoLs3+qUvRx6PpMarf651/72TJHUQIJIuzZoPJaggpZHlBhL2u5feZgoOrGEx7ufJx7pSe8ExNoRJqhK/Llhc0ScMGez80icSMH76ow+ziZJ54F0GwrvmOowzyHYL0XcevH9m3GORlu8GylIIxdGcpQo0l64ogPIytSCzDt1fZ6DDRwJUx/U3g1zItRDIqcluHDBRHfh64wHzQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=AqsGMyn5TPs5Q4beKEoVIbOcCuF2X6yj2QdvXMtGCQ0=; b=NeRi5VDvEeK+ON94E+ODZrOt/7Y2+5QNMZbrGeg15NeMztyiAwdIcJqrPvj11NPrhMv5yVsg340S869VRPdLclPsQlsUReEHXXZRaJKWoofzAJf6AMAa+oe5M1cUJNvtBdGK1CUOuiGh2kSbk2xtCSwJ8QQ/jLgeXPHgemmr8WijbuONI9RgJuebYQl5hVmkkyOo9CMUydlKBcX8euYB64dar9bGDQ5OlinAU7KIuUb1kj//bHgMGVRiE9TxniygGydgdgwnGukf8pc3aH5fKsBzdtB6z97B8X/ELB2Yj7r5pdVUpXzv/cmhvc4/3R5mGQA12Z8Mw6jH+7Eq6thL0w== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com; dkim=pass header.d=intel.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=intel.onmicrosoft.com; s=selector2-intel-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=AqsGMyn5TPs5Q4beKEoVIbOcCuF2X6yj2QdvXMtGCQ0=; b=m+WICZfNYTVYBnAAYxTjwCuVQx5gb3U6RUxJLhHMwaPbiuSzcmYfQsSlh/RKi01+CNxuZqcxNrhG3j7UPeSu+JCgrRskoJP3wbG5d5gak8Oylx18VUkkl9L5DZVsvKBUqJDfjENfY02/oQ8lYPfmx+q/HrCyFyz06hrzsIwM7/0= Received: from BL0PR11MB3489.namprd11.prod.outlook.com (2603:10b6:208:75::29) by BL0PR11MB3460.namprd11.prod.outlook.com (2603:10b6:208:6e::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2979.26; Sun, 10 May 2020 04:45:31 +0000 Received: from BL0PR11MB3489.namprd11.prod.outlook.com ([fe80::d0f8:bfd7:bae2:3e42]) by BL0PR11MB3489.namprd11.prod.outlook.com ([fe80::d0f8:bfd7:bae2:3e42%7]) with mapi id 15.20.2979.033; Sun, 10 May 2020 04:45:31 +0000 From: "Nate DeSimone" To: "Desimone, Ashley E" , "devel@edk2.groups.io" CC: "Pandya, Puja" , "Bjorge, Erik C" , Bret Barkelew , "Agyeman, Prince" Subject: Re: [edk2-staging/EdkRepo] [PATCH V2] EdkRepo: Integrate multiple manifest repository support in the manifest command Thread-Topic: [edk2-staging/EdkRepo] [PATCH V2] EdkRepo: Integrate multiple manifest repository support in the manifest command Thread-Index: AQHWJoJ8p90RDORw6U6NRId0CHpyY6igvpHg Date: Sun, 10 May 2020 04:45:31 +0000 Message-ID: References: <20200510042108.20544-1-ashley.e.desimone@intel.com> In-Reply-To: <20200510042108.20544-1-ashley.e.desimone@intel.com> Accept-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-titus-metadata-40: eyJDYXRlZ29yeUxhYmVscyI6IiIsIk1ldGFkYXRhIjp7Im5zIjoiaHR0cDpcL1wvd3d3LnRpdHVzLmNvbVwvbnNcL0ludGVsMyIsImlkIjoiNmMxNjAxZjEtOWNiYy00NDlmLWE2ZDUtZGVhYWUxYmJmNGFjIiwicHJvcHMiOlt7Im4iOiJDVFBDbGFzc2lmaWNhdGlvbiIsInZhbHMiOlt7InZhbHVlIjoiQ1RQX05UIn1dfV19LCJTdWJqZWN0TGFiZWxzIjpbXSwiVE1DVmVyc2lvbiI6IjE3LjEwLjE4MDQuNDkiLCJUcnVzdGVkTGFiZWxIYXNoIjoiUmJqaklzWjczM0RvVTlscldEak1JRnRMUzVTWUQ0VTFXQkNud3BwTXp4N0JpNzB4Kzc0QWtreHlaWDF5c2IrNyJ9 dlp-reaction: no-action dlp-version: 11.2.0.6 dlp-product: dlpe-windows x-ctpclassification: CTP_NT authentication-results: intel.com; dkim=none (message not signed) header.d=none;intel.com; dmarc=none action=none header.from=intel.com; x-originating-ip: [50.53.190.176] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: d3171571-793a-4e45-b8ae-08d7f49cf845 x-ms-traffictypediagnostic: BL0PR11MB3460: x-ms-exchange-transport-forked: True x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:3276; x-forefront-prvs: 039975700A x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: 3YaOZMJvw4ahWHbQMOAa/cjO0CAKhuyfVyZqPxyNIvnEF82ZpvmCgJ+3rs2cl//TxmaYUmg3R3MS3EDmBxphAI1mEINEtSSrw9ZS/CKVORn6kUA0NB3KoE9ODP9A7bksQiXQRmRrf8BV4I+w/FLjum/gOVoARx3tA4aB/cN44We4fGqMeLDftrKV5TXJL2CsQAFx6QmhLGD3DMNyudL7fo2FoHrfGBVLrWR7WzNWQWPmbgFrYkAkpGJL0i3TYpY4WPstw8IFeDY3rTvXOH1/pgckX4khLH7kOLlJsmV40fNk9VtZpnGk6Y+gcewvrsDy1UPkjhT6i5wVkO4Df3SAzX2OclZo4BuiJeXYUYVxO43SOwEZFCRj8C2mrpBIOTWRv49VS8sR57TlOYqAPhcvG6DJQOYoPFihh+7+K6msQe1CKLM8GFprsJWL0Rz0KXRT0w32foWbbXIqgBFmIM0oTRYlz8Q9l/gDYHx2sTnG4exP9yGqM0J/iGwdSt5P1Bo0n+u94IfTXCvTs2p9EEhS6g== x-forefront-antispam-report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BL0PR11MB3489.namprd11.prod.outlook.com;PTR:;CAT:NONE;SFTY:;SFS:(39860400002)(376002)(136003)(366004)(396003)(346002)(33430700001)(110136005)(33656002)(33440700001)(86362001)(54906003)(8676002)(8936002)(53546011)(478600001)(7696005)(6506007)(52536014)(107886003)(316002)(66946007)(5660300002)(4326008)(71200400001)(55016002)(66556008)(66446008)(9686003)(186003)(66476007)(64756008)(26005)(76116006)(2906002);DIR:OUT;SFP:1102; x-ms-exchange-antispam-messagedata: IrMDI5MkacnS+rwcKpv4BuL9/b0ejdM5l9jl1ClxUOlURw0DSdqcvs0dzy9/R5bQPPJEpMTR5v7IJPu/hbiW4EqTARZlXxXfWWg4qWDFj3OaY49shQxWjxofkEuFJtuw7s+8jOncEjwOTfL0FMG2LvL1K2OvC5H+PZF8FYYnyXDVk1bRFFkZY7rb6U2EHMjkPZNU/ZI0aaeJnAF211U7axG2evCi/tKUGZNyBH8jUhkIt0mXIIPqtVa0Ke9qUYhsqkPY5Q3svElv6MmkSidtLfOnvBBAAkWsECLvFSRbYY88JZTc6x6ZAzQP2Fk6z1K+sHu+dDil5la0datRIdc0XO7KYfR0Co6Z5Gb+837KDy4YWrclJ49F7idbf1pcTYSrB6j0qNj6cvbqNUXiCa0gGwMyPVLHmyx6nCmKP+VIpbCnpE/+EVfBNe/qK3LWzyXAZckemqPTLipZXcodEBCNPUPGGmklI3aZ0yve/PEHNhOdmbZPTI8eJZspxjnBwFlx MIME-Version: 1.0 X-MS-Exchange-CrossTenant-Network-Message-Id: d3171571-793a-4e45-b8ae-08d7f49cf845 X-MS-Exchange-CrossTenant-originalarrivaltime: 10 May 2020 04:45:31.3099 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 46c98d88-e344-4ed4-8496-4ed7712e255d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: dSUGzr0Csep0oOz+KDsAW8fiaW/MDahZ1a4oweWkr8irQi+um4zy+M1bq/XfWN1sjTkHaSHmEow8lcqx+4225A8PFLq8zRS2juaJvy2/S+A= X-MS-Exchange-Transport-CrossTenantHeadersStamped: BL0PR11MB3460 Return-Path: nathaniel.l.desimone@intel.com X-OriginatorOrg: intel.com Content-Language: en-US Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Reviewed-by: Nate DeSimone > -----Original Message----- > From: Desimone, Ashley E > Sent: Saturday, May 9, 2020 9:21 PM > To: devel@edk2.groups.io > Cc: Desimone, Nathaniel L ; Pandya, Puja > ; Bjorge, Erik C ; Bret > Barkelew ; Agyeman, Prince > > Subject: [edk2-staging/EdkRepo] [PATCH V2] EdkRepo: Integrate multiple > manifest repository support in the manifest command >=20 > Integrate multiple manifest repository support in the manifest command >=20 > Add the validate_manifest_repo() function to common_repo_functions >=20 > Signed-off-by: Ashley E Desimone > Cc: Nate DeSimone > Cc: Puja Pandya > Cc: Erik Bjorge > Cc: Bret Barkelew > Cc: Prince Agyeman > --- > edkrepo/commands/manifest_command.py | 86 > +++++++++++++++++++++------------ > edkrepo/common/common_repo_functions.py | 14 +++++- > 2 files changed, 68 insertions(+), 32 deletions(-) >=20 > diff --git a/edkrepo/commands/manifest_command.py > b/edkrepo/commands/manifest_command.py > index 44218c9..bb6252d 100644 > --- a/edkrepo/commands/manifest_command.py > +++ b/edkrepo/commands/manifest_command.py > @@ -3,7 +3,7 @@ > ## @file > # manifest_command.py > # > -# Copyright (c) 2017- 2019, Intel Corporation. All rights reserved.
> +# Copyright (c) 2017- 2020, Intel Corporation. All rights reserved.
> # SPDX-License-Identifier: BSD-2-Clause-Patent # >=20 > @@ -15,9 +15,12 @@ from colorama import Fore from > edkrepo.commands.edkrepo_command import EdkrepoCommand from > edkrepo.commands.edkrepo_command import ColorArgument import > edkrepo.commands.arguments.manifest_args as arguments -from > edkrepo.common.edkrepo_exception import > EdkrepoWorkspaceInvalidException -from > edkrepo.common.common_repo_functions import > pull_latest_manifest_repo, verify_manifest_data > +from edkrepo.common.edkrepo_exception import > +EdkrepoWorkspaceInvalidException, EdkrepoManifestNotFoundException > from > +edkrepo.common.common_repo_functions import > pull_latest_manifest_repo, > +validate_manifest_repo > from edkrepo.common.ui_functions import init_color_console > +from > edkrepo.common.workspace_maintenance.manifest_repos_maintenance > +import list_available_manifest_repos from > +edkrepo.common.workspace_maintenance.manifest_repos_maintenance > import > +pull_all_manifest_repos from > +edkrepo.common.workspace_maintenance.manifest_repos_maintenance > import > +find_source_manifest_repo > from edkrepo.config.config_factory import get_workspace_manifest from > edkrepo_manifest_parser.edk_manifest import CiIndexXml >=20 > @@ -44,46 +47,67 @@ class ManifestCommand(EdkrepoCommand): > print() > init_color_console(args.color) >=20 > - # Get path to global manifest file > - global_manifest_directory =3D > config['cfg_file'].manifest_repo_abs_local_path > - if args.verbose: > - print("Manifest directory:") > - print(global_manifest_directory) > - print() > - index_path =3D os.path.join(global_manifest_directory, 'CiIndex.= xml') > + cfg_file =3D config['cfg_file'] > + user_cfg =3D config['user_cfg_file'] > + cfg_man_repos, user_cfg_man_repos, conflicts =3D > list_available_manifest_repos(cfg_file, user_cfg) > + man_repos =3D {} >=20 > - pull_latest_manifest_repo(args, config) > - print() > + pull_all_manifest_repos(cfg_file, user_cfg, False) >=20 > - ci_index_xml =3D CiIndexXml(index_path) > + # Get paths to the global manifest dirs and their index files > + for repo in cfg_man_repos: > + global_manifest_directory =3D cfg_file.manifest_repo_abs_pat= h(repo) > + index_path =3D os.path.join(global_manifest_directory, 'CiIn= dex.xml') > + man_repos[repo] =3D (global_manifest_directory, index_path) > + for repo in user_cfg_man_repos: > + global_manifest_directory =3D > user_cfg.manifest_repo_abs_path(repo) > + index_path =3D os.path.join(global_manifest_directory, 'CiIn= dex.xml') > + man_repos[repo] =3D (global_manifest_directory, index_path) >=20 > try: > - current_project =3D get_workspace_manifest().project_info.co= dename > + wkspc_manifest =3D get_workspace_manifest() > + current_project =3D wkspc_manifest.project_info.codename > + src_man_repo =3D find_source_manifest_repo(wkspc_manifest, > + cfg_file, user_cfg, None) > except EdkrepoWorkspaceInvalidException: > current_project =3D None > + src_man_repo =3D None > + except EdkrepoManifestNotFoundException: > + src_man_repo =3D None >=20 > - # Attempt to make sure the manifest data is good > - try: > - verify_manifest_data(global_manifest_directory, config, > verbose=3Dargs.verbose, verify_all=3DTrue, verify_archived=3Dargs.archive= d) > - except: > - print() >=20 > - print("Projects:") > - for project in ci_index_xml.project_list: > - if project =3D=3D current_project: > - print("* {}{}{}".format(Fore.GREEN, project, Fore.RESET)= ) > - else: > - print(" {}".format(project)) > + for repo in man_repos.keys(): > + print() > + print("Manifest directory:") > + print(repo) > if args.verbose: > - print(" -> {}".format(ci_index_xml.get_project_xml(pro= ject))) > - > - if args.archived: > + print('Manifest directory path:') > + print(man_repos[repo][0]) > print() > - print("Archived Projects:") > - for project in ci_index_xml.archived_project_list: > - if project =3D=3D current_project: > + > + ci_index_xml =3D CiIndexXml(man_repos[repo][1]) > + > + # Attempt to make sure the manifest data is good > + try: > + validate_manifest_repo(man_repos[repo][0], args.verbose, > args.archived) > + except: > + print() > + > + print("Projects:") > + for project in sorted(ci_index_xml.project_list): > + if (project =3D=3D current_project and src_man_repo =3D= =3D repo) or (not > src_man_repo and project =3D=3D current_project): > print("* {}{}{}".format(Fore.GREEN, project, Fore.RE= SET)) > else: > print(" {}".format(project)) > if args.verbose: > print(" -> {}".format(ci_index_xml.get_project_xml= (project))) > + > + if args.archived: > + print() > + print("Archived Projects:") > + for project in sorted(ci_index_xml.archived_project_list= ): > + if project =3D=3D current_project: > + print("* {}{}{}".format(Fore.GREEN, project, For= e.RESET)) > + else: > + print(" {}".format(project)) > + if args.verbose: > + print(" -> {}".format(ci_index_xml.get_project= _xml(project))) > diff --git a/edkrepo/common/common_repo_functions.py > b/edkrepo/common/common_repo_functions.py > index 20dcb8a..eb6c4c0 100644 > --- a/edkrepo/common/common_repo_functions.py > +++ b/edkrepo/common/common_repo_functions.py > @@ -54,6 +54,7 @@ from edkrepo.common.pathfix import get_actual_path > from project_utils.sparse import BuildInfo, process_sparse_checkout from > edkrepo.config.config_factory import get_workspace_path from > edkrepo.config.config_factory import get_workspace_manifest > +from edkrepo.config.tool_config import CI_INDEX_FILE_NAME > from edkrepo.common.edkrepo_exception import > EdkrepoInvalidParametersException from > edkrepo_manifest_parser.edk_manifest import CiIndexXml, ManifestXml > from edkrepo.common.edkrepo_exception import > EdkrepoNotFoundException, EdkrepoGitException, > EdkrepoWarningException @@ -426,11 +427,22 @@ def > verify_manifest_data(global_manifest_directory, config, verbose=3DFalse, > verif > if verbose: > print_manifest_errors(manifestfile_validation_data) >=20 > +def validate_manifest_repo(manifest_repo, verbose=3DFalse, > archived=3DFalse): > + print(VERIFY_GLOBAL) > + if archived: > + print(VERIFY_ARCHIVED) > + manifest_validation_data =3D validate_manifestrepo(manifest_repo, > archived) > + manifest_repo_error =3D > get_manifest_validation_status(manifest_validation_data) > + if manifest_repo_error: > + print(VERIFY_GLOBAL_FAIL) > + if verbose: > + print_manifest_errors(manifest_validation_data) > + > def verify_single_manifest(cfg_file, manifest_repo, manifest_path, > verbose=3DFalse): > manifest =3D ManifestXml(manifest_path) > print(VERIFY_PROJ.format(manifest.project_info.codename)) > index_path =3D > os.path.join(cfg_file.manifest_repo_abs_path(manifest_repo), > CI_INDEX_FILE_NAME) > - proj_val_data =3D validate_manifestfiles(index_path, [manifest_path]= ) > + proj_val_data =3D validate_manifestfiles([manifest_path]) > proj_val_error =3D get_manifest_validation_status(proj_val_data) > if proj_val_error: > if verbose: > -- > 2.16.2.windows.1