From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mga01.intel.com (mga01.intel.com [192.55.52.88]) by mx.groups.io with SMTP id smtpd.web11.73933.1584379292782997428 for ; Mon, 16 Mar 2020 10:21:32 -0700 Authentication-Results: mx.groups.io; dkim=missing; spf=pass (domain: intel.com, ip: 192.55.52.88, mailfrom: nathaniel.l.desimone@intel.com) IronPort-SDR: +kNmsNVPHyUri05dNKj3xyjd43K+XlSvi9Ubyitkla2/5nJ6/k0E+vyem+eUZUpLNk4ZkL/MHQ Kgm7X7/jX7Qw== X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga006.jf.intel.com ([10.7.209.51]) by fmsmga101.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 16 Mar 2020 10:21:32 -0700 IronPort-SDR: kbWUIUNZwjJldLGuHavFdOw625nIR2ya+PRhXm3Bj5UFo0DR9cqQuTmRALQSy+9wVna6hyFhQE e3V0KRyoOObw== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.70,561,1574150400"; d="scan'208";a="247525959" Received: from orsmsx102.amr.corp.intel.com ([10.22.225.129]) by orsmga006.jf.intel.com with ESMTP; 16 Mar 2020 10:21:31 -0700 Received: from orsmsx157.amr.corp.intel.com (10.22.240.23) by ORSMSX102.amr.corp.intel.com (10.22.225.129) with Microsoft SMTP Server (TLS) id 14.3.439.0; Mon, 16 Mar 2020 10:21:31 -0700 Received: from orsmsx114.amr.corp.intel.com ([169.254.8.205]) by ORSMSX157.amr.corp.intel.com ([169.254.9.107]) with mapi id 14.03.0439.000; Mon, 16 Mar 2020 10:21:31 -0700 From: "Nate DeSimone" To: "Desimone, Ashley E" , "devel@edk2.groups.io" CC: "Pandya, Puja" , "Bjorge, Erik C" Subject: Re: [edk2-staging/EdkRepo] [PATCH] EdkRepo: Initial commit of the Create Pin command Thread-Topic: [edk2-staging/EdkRepo] [PATCH] EdkRepo: Initial commit of the Create Pin command Thread-Index: AQHV9/UgHUn5roLFVk+YDRAOoZK5fahLfqhQ Date: Mon, 16 Mar 2020 17:21:31 +0000 Message-ID: <02A34F284D1DA44BB705E61F7180EF0AB5DF22D7@ORSMSX114.amr.corp.intel.com> References: <20200311223329.22952-1-ashley.e.desimone@intel.com> In-Reply-To: <20200311223329.22952-1-ashley.e.desimone@intel.com> Accept-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: dlp-product: dlpe-windows dlp-version: 11.2.0.6 dlp-reaction: no-action x-titus-metadata-40: eyJDYXRlZ29yeUxhYmVscyI6IiIsIk1ldGFkYXRhIjp7Im5zIjoiaHR0cDpcL1wvd3d3LnRpdHVzLmNvbVwvbnNcL0ludGVsMyIsImlkIjoiZjFiMmQzYjctYzNmNi00MTFiLWI1NmMtNmYxNDY0YzdkZmUwIiwicHJvcHMiOlt7Im4iOiJDVFBDbGFzc2lmaWNhdGlvbiIsInZhbHMiOlt7InZhbHVlIjoiQ1RQX05UIn1dfV19LCJTdWJqZWN0TGFiZWxzIjpbXSwiVE1DVmVyc2lvbiI6IjE3LjEwLjE4MDQuNDkiLCJUcnVzdGVkTGFiZWxIYXNoIjoieHVER3c3SFNOemNrWjVuMFI4TEd3ZXZTUERhZGx0R29nVTdBcmVWVVFFRzg5bUxiUHY0eU41cHNrSkhcL2p1RzQifQ== x-ctpclassification: CTP_NT x-originating-ip: [10.22.254.140] MIME-Version: 1.0 Return-Path: nathaniel.l.desimone@intel.com Content-Language: en-US Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Reviewed-by: Nate DeSimone -----Original Message----- From: Desimone, Ashley E =20 Sent: Wednesday, March 11, 2020 3:33 PM To: devel@edk2.groups.io Cc: Desimone, Nathaniel L ; Pandya, Puja ; Bjorge, Erik C Subject: [edk2-staging/EdkRepo] [PATCH] EdkRepo: Initial commit of the Crea= te Pin command Add edkrepo create-pin command allowing users to create a pin file to recor= d their workspace status at a specific point in time. Signed-off-by: Ashley E Desimone Cc: Nate DeSimone Cc: Puja Pandya Cc: Erik Bjorge --- edkrepo/commands/arguments/create_pin_args.py | 21 +++++ edkrepo/commands/create_pin_command.py | 127 ++++++++++++++++++++++= ++++ edkrepo/common/edkrepo_exception.py | 6 +- edkrepo/common/humble.py | 17 +++- 4 files changed, 169 insertions(+), 2 deletions(-) create mode 100644 edk= repo/commands/arguments/create_pin_args.py create mode 100644 edkrepo/commands/create_pin_command.py diff --git a/edkrepo/commands/arguments/create_pin_args.py b/edkrepo/comman= ds/arguments/create_pin_args.py new file mode 100644 index 0000000..92958ef --- /dev/null +++ b/edkrepo/commands/arguments/create_pin_args.py @@ -0,0 +1,21 @@ +#!/usr/bin/env python3 +# +## @file +# create_pin_command.py +# +# Copyright (c) 2020, Intel Corporation. All rights reserved.
#=20 +SPDX-License-Identifier: BSD-2-Clause-Patent # + +''' Contains the help and description strings for arguments in the=20 +create pin command meta data. +''' + +COMMAND_DESCRIPTION =3D 'Creates a PIN file based on the current workspace= state' +NAME_HELP =3D ('The name of the PIN file. Extension must be .xml. File pat= hs are ' + 'supported only if the --push option is not used.')=20 +DESCRIPTION_HELP =3D 'A short summary of the PIN file contents. Must be co= ntained in ""' +PUSH_HELP =3D ('Automatically commit and push the PIN file to the global m= anifest ' + 'repository at the location specified by the Pin-Path field i= n ' + 'the project manifest file.') + diff --git a/edkrepo/commands/create_pin_command.py b/edkrepo/commands/crea= te_pin_command.py new file mode 100644 index 0000000..df3fccc --- /dev/null +++ b/edkrepo/commands/create_pin_command.py @@ -0,0 +1,127 @@ +#!/usr/bin/env python3 +# +## @file +# create_pin_command.py +# +# Copyright (c) 2017 - 2020, Intel Corporation. All rights=20 +reserved.
# SPDX-License-Identifier: BSD-2-Clause-Patent # + +import os +from collections import namedtuple + +from git import Repo + +from edkrepo.commands.edkrepo_command import EdkrepoCommand import=20 +edkrepo.commands.arguments.create_pin_args as arguments from=20 +edkrepo.common.common_repo_functions import pull_latest_manifest_repo=20 +from edkrepo.common.edkrepo_exception import=20 +EdkrepoManifestInvalidException, EdkrepoInvalidParametersException from=20 +edkrepo.common.edkrepo_exception import=20 +EdkrepoWorkspaceCorruptException from edkrepo.common.humble import=20 +WRITING_PIN_FILE, GENERATING_PIN_DATA, GENERATING_REPO_DATA, BRANCH,=20 +COMMIT from edkrepo.common.humble import COMMIT_MESSAGE,=20 +PIN_PATH_NOT_PRESENT, PIN_FILE_ALREADY_EXISTS, PATH_AND_FILEPATH_USED=20 +from edkrepo.common.humble import MISSING_REPO from=20 +edkrepo.config.config_factory import get_workspace_manifest,=20 +get_workspace_path from edkrepo_manifest_parser.edk_manifest import=20 +ManifestXml + + +class CreatePinCommand(EdkrepoCommand): + def __init__(self): + super().__init__() + + def get_metadata(self): + metadata =3D {} + metadata['name'] =3D 'create-pin' + metadata['help-text'] =3D arguments.COMMAND_DESCRIPTION + metadata['alias'] =3D 'crp' + args =3D [] + metadata['arguments'] =3D args + args.append({'name' : 'PinFileName', + 'positional' : True, + 'position' : 0, + 'required' : True, + 'help-text' : arguments.NAME_HELP}) + args.append({'name' : 'Description', + 'positional' : True, + 'position' : 1, + 'required' : True, + 'help-text' : arguments.DESCRIPTION_HELP}) + args.append({'name': 'push', + 'positional': False, + 'required': False, + 'help-text': arguments.PUSH_HELP}) + return metadata + + def run_command(self, args, config): + # Check if --push and file path provided + if args.push and os.path.dirname(args.PinFileName): + raise=20 + EdkrepoInvalidParametersException(PATH_AND_FILEPATH_USED) + + pull_latest_manifest_repo(args, config) + workspace_path =3D get_workspace_path() + manifest =3D get_workspace_manifest() + + # If the push flag is enabled use general_config.pin_path to deter= mine global manifest relative location to save + # pin file to. + if args.push and manifest.general_config.pin_path is not None: + pin_dir =3D os.path.join(config['cfg_file'].manifest_repo_abs_= local_path, os.path.normpath(manifest.general_config.pin_path)) + pin_file_name =3D os.path.join(pin_dir, args.PinFileName) + elif args.push and manifest.general_config.pin_path is None: + raise EdkrepoManifestInvalidException(PIN_PATH_NOT_PRESENT) + # If not using the push flag ignore the local manifest's pin file = field and set the pinname/path =3D=3D to the file + # name provided. If a relative paths is provided save the file rel= ative to the current working directory. + elif not args.push and os.path.isabs(os.path.normpath(args.PinFile= Name)): + pin_file_name =3D os.path.normpath(args.PinFileName) + elif not args.push: + pin_file_name =3D os.path.abspath(os.path.normpath(args.PinFil= eName)) + # If the directory that the pin file is saved in does not exist cr= eate it and ensure pin file name uniqueness. + if os.path.isfile(pin_file_name): + raise EdkrepoInvalidParametersException(PIN_FILE_ALREADY_EXIST= S) + if not os.path.exists(os.path.dirname(pin_file_name)): + os.mkdir(os.path.dirname(pin_file_name)) + + repo_sources =3D=20 + manifest.get_repo_sources(manifest.general_config.current_combo) + + # get the repo sources and commit ids for the pin + print(GENERATING_PIN_DATA.format(manifest.project_info.codename, m= anifest.general_config.current_combo)) + updated_repo_sources =3D [] + for repo_source in repo_sources: + local_repo_path =3D os.path.join(workspace_path, repo_source.r= oot) + if not os.path.exists(local_repo_path): + raise EdkrepoWorkspaceCorruptException(MISSING_REPO.format= (repo_source.root)) + repo =3D Repo(local_repo_path) + commit_id =3D repo.head.commit.hexsha + if args.verbose: + print(GENERATING_REPO_DATA.format(repo_source.root)) + print(BRANCH.format(repo_source.branch)) + print(COMMIT.format(commit_id)) + updated_repo_source =3D repo_source._replace(commit=3Dcommit_i= d) + updated_repo_sources.append(updated_repo_source) + + # create the pin + print(WRITING_PIN_FILE.format(pin_file_name)) + manifest.generate_pin_xml(args.Description, manifest.general_confi= g.current_combo, updated_repo_sources, + filename=3Dpin_file_name) + + # commit and push the pin file + if args.push: + manifest_repo =3D Repo(config['cfg_file'].manifest_repo_abs_lo= cal_path) + # Create a local branch with the same name as the pin file arg= and check it out before attempting the push + # to master + master_branch =3D manifest_repo.active_branch + local_branch =3D manifest_repo.create_head(args.PinFileName) + manifest_repo.heads[local_branch.name].checkout() + manifest_repo.git.add(pin_file_name) + manifest_repo.git.commit(m=3DCOMMIT_MESSAGE.format(manifest.pr= oject_info.codename, args.Description)) + # Try to push if the push fails re update the manifest repo an= d rebase from master + try: + manifest_repo.git.push('origin', 'HEAD:master') + except: + manifest_repo.heads[master_branch.name].checkout() + origin =3D manifest_repo.remotes.origin + origin.pull() + manifest_repo.heads[local_branch.name].checkout() + manifest_repo.git.rebase('master') + manifest_repo.git.push('origin', 'HEAD:master') + finally: + manifest_repo.heads[master_branch.name].checkout() + manifest_repo.delete_head(local_branch, '-D') \ No newline at end of file diff --git a/edkrepo/common/edkrepo_exception.py b/edkrepo/common/edkrepo_e= xception.py index 9d2b102..b6ea3dd 100644 --- a/edkrepo/common/edkrepo_exception.py +++ b/edkrepo/common/edkrepo_exception.py @@ -3,7 +3,7 @@ ## @file # edkrepo_exception.py # -# Copyright (c) 2017- 2019, Intel Corporation. All rights reserved.
+# Copyright (c) 2017- 2020, Intel Corporation. All rights reserved.
# SPDX-License-Identifier: BSD-2-Clause-Patent # =20 @@ -74,6 +74,10 @@ class EdkrepoFoundMultipleException(EdkrepoException): def __init__(self, message): super().__init__(message, 117) =20 +class EdkrepoWorkspaceCorruptException(EdkrepoException): + def __ini__(self, message): + super().__init__(message, 118) + class EdkrepoWarningException(EdkrepoException): def __init__(self, message): super().__init__(message, 123) diff --git a/edkrepo/common/humble.py b/edkrepo/common/humble.py index f0fb= d29..5326e88 100644 --- a/edkrepo/common/humble.py +++ b/edkrepo/common/humble.py @@ -3,7 +3,7 @@ ## @file # humble.py # -# Copyright (c) 2017 - 2019, Intel Corporation. All rights reserved.
+# Copyright (c) 2017 - 2020, Intel Corporation. All rights=20 +reserved.
# SPDX-License-Identifier: BSD-2-Clause-Patent # =20 @@ -147,3 +147,18 @@ VERIFY_PROJ_FAIL =3D 'Unable to verify the global mani= fest repository entry for pr =20 # Git Command Error Messages GIT_CMD_ERROR =3D 'The git command: {} failed to complete successfully wit= h the following errors.\n' + +# Error messages for create_pin_command.py CREATE_PIN_EXIT =3D 'Exiting=20 +without creating pin file ...' +PIN_PATH_NOT_PRESENT =3D 'Pin Path not present in Manifest.xml ' +=20 +CREATE_PIN_EXIT PIN_FILE_ALREADY_EXISTS =3D 'A pin file with that name=20 +already exists for this project. Please rerun the command with a new=20 +filename. ' + CREATE_PIN_EXIT PATH_AND_FILEPATH_USED =3D 'Providing a=20 +file path for the PinFileName argument is not supported when using the=20 +--push flag. ' + CREATE_PIN_EXIT MISSING_REPO =3D 'The {} repository is=20 +missing from your workspace. ' + CREATE_PIN_EXIT + +# Informational messages for create_pin_command.py GENERATING_PIN_DATA=20 +=3D 'Generating pin data for {0} project based on {1} combination ...' +GENERATING_REPO_DATA =3D 'Generating pin data for {0} repo:' +BRANCH =3D ' Branch : {0}' +COMMIT =3D ' Commit Id: {0}' +WRITING_PIN_FILE =3D 'Writing pin file to {0} ...' +COMMIT_MESSAGE =3D 'Pin file for project: {0} \nPin Description: {1}' -- 2.16.2.windows.1