From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mga09.intel.com (mga09.intel.com [134.134.136.24]) by mx.groups.io with SMTP id smtpd.web12.288.1576274976081903250 for ; Fri, 13 Dec 2019 14:09:36 -0800 Authentication-Results: mx.groups.io; dkim=missing; spf=pass (domain: intel.com, ip: 134.134.136.24, mailfrom: ashley.e.desimone@intel.com) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga005.jf.intel.com ([10.7.209.41]) by orsmga102.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 13 Dec 2019 14:09:35 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.69,311,1571727600"; d="scan'208";a="388812256" Received: from orsmsx104.amr.corp.intel.com ([10.22.225.131]) by orsmga005.jf.intel.com with ESMTP; 13 Dec 2019 14:09:35 -0800 Received: from orsmsx114.amr.corp.intel.com (10.22.240.10) by ORSMSX104.amr.corp.intel.com (10.22.225.131) with Microsoft SMTP Server (TLS) id 14.3.439.0; Fri, 13 Dec 2019 14:09:34 -0800 Received: from orsmsx116.amr.corp.intel.com ([169.254.7.30]) by ORSMSX114.amr.corp.intel.com ([169.254.8.106]) with mapi id 14.03.0439.000; Fri, 13 Dec 2019 14:09:34 -0800 From: "Desimone, Ashley E" To: "Desimone, Nathaniel L" , "devel@edk2.groups.io" CC: "Pandya, Puja" , "Bjorge, Erik C" , Bret Barkelew Subject: Re: [edk2-staging/EdkRepo] [PATCH V3 1/1] EdkRepo: Add squash command Thread-Topic: [edk2-staging/EdkRepo] [PATCH V3 1/1] EdkRepo: Add squash command Thread-Index: AQHVsYp0W98OeZSXqkOR2o0XCF6Lq6e4oNWA Date: Fri, 13 Dec 2019 22:09:34 +0000 Message-ID: <4CF3A9EB60ABDA47BE7821A4DA3A0A3353CC6937@ORSMSX116.amr.corp.intel.com> References: <20191213075337.4416-1-nathaniel.l.desimone@intel.com> <20191213075337.4416-2-nathaniel.l.desimone@intel.com> In-Reply-To: <20191213075337.4416-2-nathaniel.l.desimone@intel.com> Accept-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-titus-metadata-40: eyJDYXRlZ29yeUxhYmVscyI6IiIsIk1ldGFkYXRhIjp7Im5zIjoiaHR0cDpcL1wvd3d3LnRpdHVzLmNvbVwvbnNcL0ludGVsMyIsImlkIjoiYTUyYzdhNGQtZDk4Zi00NWRmLWFjODUtMTRhN2EzYmI4NDM5IiwicHJvcHMiOlt7Im4iOiJDVFBDbGFzc2lmaWNhdGlvbiIsInZhbHMiOlt7InZhbHVlIjoiQ1RQX05UIn1dfV19LCJTdWJqZWN0TGFiZWxzIjpbXSwiVE1DVmVyc2lvbiI6IjE3LjEwLjE4MDQuNDkiLCJUcnVzdGVkTGFiZWxIYXNoIjoiWk5qV09MbUJsc1laTHh1UjhubU1RWDhlZUJHSkw1V2JLTmpvZjNoTE81ZUhmUnQ1RUZyNThOcm1QakpOcFwvVVUifQ== x-ctpclassification: CTP_NT dlp-product: dlpe-windows dlp-version: 11.2.0.6 dlp-reaction: no-action x-originating-ip: [10.22.254.140] MIME-Version: 1.0 Return-Path: ashley.e.desimone@intel.com Content-Language: en-US Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Reviewed-by: Ashley DeSimone -----Original Message----- From: Desimone, Nathaniel L=20 Sent: Thursday, December 12, 2019 11:54 PM To: devel@edk2.groups.io Cc: Desimone, Ashley E ; Pandya, Puja ; Bjorge, Erik C ; Bret Barkelew Subject: [edk2-staging/EdkRepo] [PATCH V3 1/1] EdkRepo: Add squash command Adds the squash command, which takes a range of commits and compacts them i= nto a single commit. Cc: Ashley DeSimone Cc: Puja Pandya Cc: Erik Bjorge Cc: Bret Barkelew Signed-off-by: Nate DeSimone --- edkrepo/commands/arguments/squash_args.py | 19 +++++ edkrepo/commands/hum= ble/squash_humble.py | 19 +++++ edkrepo/commands/squash_command.py | 97 +++++++++++++++++++++++ edkrepo/common/humble.py | 7 +- edkrepo/common/squash.py | 93 ++++++++++++++++++++++ edkrepo/git_automation/__init__.py | 8 ++ edkrepo/git_automation/commit_msg.py | 21 +++++ edkrepo/git_automation/rebase_squash.py | 23 ++++++ setup.py | 4 +- 9 files changed, 289 insertions(+), 2 deletions(-) create mode 100644 edk= repo/commands/arguments/squash_args.py create mode 100644 edkrepo/commands/humble/squash_humble.py create mode 100644 edkrepo/commands/squash_command.py create mode 100644 edkrepo/common/squash.py create mode 100644 edkrepo/gi= t_automation/__init__.py create mode 100644 edkrepo/git_automation/commit_msg.py create mode 100644 edkrepo/git_automation/rebase_squash.py diff --git a/edkrepo/commands/arguments/squash_args.py b/edkrepo/commands/a= rguments/squash_args.py new file mode 100644 index 0000000..db859ae --- /dev/null +++ b/edkrepo/commands/arguments/squash_args.py @@ -0,0 +1,19 @@ +#!/usr/bin/env python3 +# +## @file +# squash_args.py +# +# Copyright (c) 2019, Intel Corporation. All rights reserved.
#=20 +SPDX-License-Identifier: BSD-2-Clause-Patent # + +''' Contains the help and description strings for arguments in the=20 +squash command meta data. +''' + +COMMAND_DESCRIPTION =3D 'Convert multiple commits in to a single commit' +COMMIT_ISH_DESCRIPTION =3D 'A range of commits.' +COMMIT_ISH_HELP =3D 'The range of commits to be squashed, specified using = the same syntax as git rev-list.' +NEW_BRANCH_DESCRIPTION =3D 'The name of the branch to be created' +NEW_BRANCH_HELP =3D 'The single commit that is the result of the squash op= eration will be placed in to a new branch with this name.' +ONELINE_HELP =3D 'Compact the commit messsages of the squashed commits dow= n to one line' diff --git a/edkrepo/commands/humble/squash_humble.py b/edkrepo/commands/hu= mble/squash_humble.py new file mode 100644 index 0000000..a7e3348 --- /dev/null +++ b/edkrepo/commands/humble/squash_humble.py @@ -0,0 +1,19 @@ +#!/usr/bin/env python3 +# +## @file +# squash_humble.py +# +# Copyright (c) 2019, Intel Corporation. All rights reserved.
#=20 +SPDX-License-Identifier: BSD-2-Clause-Patent # + +''' +Contains user visible strings printed by the squash command. +''' + +#from colorama import Fore @todo +#from colorama import Style + +BRANCH_EXISTS =3D 'A branch with the name {} already exists, a non-existan= t branch must be provided' +MULTIPLE_COMMITS_REQUIRED =3D 'A range of commits is required, only a sing= le commit was given' +COMMIT_MESSAGE =3D 'Squash of the following commits:\n\n' diff --git a/edkrepo/commands/squash_command.py b/edkrepo/commands/squash_c= ommand.py new file mode 100644 index 0000000..5f74390 --- /dev/null +++ b/edkrepo/commands/squash_command.py @@ -0,0 +1,97 @@ +#!/usr/bin/env python3 +# +## @file +# squash_command.py +# +# Copyright (c) 2018 - 2019, Intel Corporation. All rights=20 +reserved.
# SPDX-License-Identifier: BSD-2-Clause-Patent # + +import os + +from git import Repo + +from edkrepo.commands.edkrepo_command import EdkrepoCommand import=20 +edkrepo.commands.arguments.squash_args as arguments import=20 +edkrepo.commands.humble.squash_humble as humble from=20 +edkrepo.common.edkrepo_exception import=20 +EdkrepoInvalidParametersException, EdkrepoWorkspaceInvalidException=20 +from edkrepo.common.squash import get_git_repo_root,=20 +split_commit_range, get_start_and_end_commit from edkrepo.common.squash=20 +import commit_list_to_message, squash_commits + +class SquashCommand(EdkrepoCommand): + def __init__(self): + super().__init__() + + def get_metadata(self): + metadata =3D {} + metadata['name'] =3D 'squash' + metadata['help-text'] =3D arguments.COMMAND_DESCRIPTION + args =3D [] + metadata['arguments'] =3D args + args.append({'name' : 'commit-ish', + 'positional' : True, + 'position' : 0, + 'required': True, + 'description' : arguments.COMMIT_ISH_DESCRIPTION, + 'help-text' : arguments.COMMIT_ISH_HELP}) + args.append({'name' : 'new-branch', + 'positional' : True, + 'position' : 1, + 'required': True, + 'description' : arguments.NEW_BRANCH_DESCRIPTION, + 'help-text' : arguments.NEW_BRANCH_HELP}) + args.append({'name': 'oneline', + 'positional': False, + 'required': False, + 'help-text': arguments.ONELINE_HELP}) + return metadata + + def run_command(self, args, config): + commit_ish =3D vars(args)['commit-ish'] + new_branch =3D vars(args)['new-branch'] + one_line =3D vars(args)['oneline'] + repo_path =3D get_git_repo_root() + #Initialize GitPython + repo =3D Repo(repo_path) + #Make sure the branch does not exist + if branch_name_exists(new_branch, repo): + raise EdkrepoInvalidParametersException(humble.BRANCH_EXISTS.f= ormat(new_branch)) + single_commit =3D True + try: + repo.rev_parse(commit_ish) + except: + if len(split_commit_range(commit_ish)) <=3D 1: + raise EdkrepoInvalidParametersException(humble.MULTIPLE_CO= MMITS_REQUIRED) + single_commit =3D False + if single_commit: + raise EdkrepoInvalidParametersException(humble.MULTIPLE_COMMIT= S_REQUIRED) + (start_commit, end_commit) =3D get_start_and_end_commit(commit_ish= , repo) + if one_line: + commit_message =3D humble.COMMIT_MESSAGE + else: + commit_message =3D '' + commit_message +=3D get_squash_commit_message_list(repo, start_com= mit, end_commit, one_line) + if one_line: + commit_message +=3D '\n' + original_branch =3D repo.heads[repo.active_branch.name] + try: + squash_commits(start_commit, end_commit, new_branch, commit_me= ssage, repo) + except: + if new_branch in repo.heads: + original_branch.checkout() + repo.git.branch('-D', new_branch) + raise + finally: + original_branch.checkout() + +def branch_name_exists(branch_name, repo): + if branch_name in [x.name for x in repo.heads]: + return True + else: + return False + +def get_commit_list(repo, start_commit, end_commit): + return repo.git.rev_list('{}..{}'.format(start_commit,=20 +end_commit)).split() + +def get_squash_commit_message_list(repo, start_commit, end_commit, one_lin= e): + return commit_list_to_message(get_commit_list(repo, start_commit,=20 +end_commit), one_line, repo) diff --git a/edkrepo/common/humble.py b/edkrepo/common/humble.py index 11e8= 53c..7f28d08 100644 --- a/edkrepo/common/humble.py +++ b/edkrepo/common/humble.py @@ -3,7 +3,7 @@ ## @file # humble.py # -# Copyright (c) 2017- 2019, Intel Corporation. All rights reserved.
+# Copyright (c) 2017 - 2019, Intel Corporation. All rights=20 +reserved.
# SPDX-License-Identifier: BSD-2-Clause-Patent # =20 @@ -23,11 +23,13 @@ RESET_HEAD =3D 'use "git reset HEAD ..."' CHECKOUT_HEAD =3D 'use "git checkout HEAD ..."' CHECKOUT =3D 'use "git checkout -- ..."' ADD =3D 'use "git add ..."' +COMMIT_NOT_FOUND =3D 'The commit {} does not exist.' BRANCH_BEHIND =3D 'Your branch \'{local_branch}\' is {behind_count} commit= (s) behind \'{target_remote}/{target_branch}\' and should be rebased.' COMMAND_NOT_SUPPORTED_MAC_OS =3D ' is not supported on macOS.' COMMAND_NOT_SUPPORT_LINUX =3D ' is not supported on Linux.' KEYBOARD_INTERRUPT =3D '\n\nKeyboard Interrupt' SUBMODULE_FAILURE =3D 'Error while performing submodule initialization and= clone operations for {} repo.\n' +NOT_GIT_REPO =3D 'The current directory does not appear to be a git reposi= tory' MULTIPLE_SOURCE_ATTRIBUTES_SPECIFIED =3D 'BRANCH or TAG name present with = COMMIT ID in combination field for {} repo. Using COMMIT ID.\n' TAG_AND_BRANCH_SPECIFIED =3D 'BRANCH AND TAG name present in combination f= ield for {} repo. Using TAG.\n' CHECKING_CONNECTION =3D 'Checking connection to remote url: {}\n' @@ -134,6 +136,9 @@ INCLUDED_INSTEAD_OF_LINE =3D ' insteadOf =3D {}\n' INCLUDED_FILE_NAME =3D '.gitconfig-{}' ERROR_WRITING_INCLUDE =3D 'An error occured while writting the URL redirec= tion configuration for {} repo.\n' =20 +#Error messages for squash.py +SQUASH_COMMON_ANCESTOR_REQUIRED =3D '{} is not in the same branch history = as {}, unable to operate on this commit range.' + # Messages for common_repo_functions.verify_manifest_data() VERIFY_GLOBAL =3D 'Verifying the active projects in the global manifest re= pository\n' VERIFY_ARCHIVED =3D 'Verifying the archived projects in the global manifes= t repository\n' diff --git a/edkrepo/common/squash.py b/edkrepo/common/squash.py new file m= ode 100644 index 0000000..df37c5e --- /dev/null +++ b/edkrepo/common/squash.py @@ -0,0 +1,93 @@ +#!/usr/bin/env python3 +# +## @file +# squash.py +# +# Copyright (c) 2018 - 2019, Intel Corporation. All rights=20 +reserved.
# SPDX-License-Identifier: BSD-2-Clause-Patent # + +import os +import sys +from subprocess import check_call + +from edkrepo.common.edkrepo_exception import=20 +EdkrepoInvalidParametersException, EdkrepoWorkspaceInvalidException=20 +from edkrepo.common.humble import COMMIT_NOT_FOUND, NOT_GIT_REPO,=20 +SQUASH_COMMON_ANCESTOR_REQUIRED + +def get_git_repo_root(): + path =3D os.path.realpath(os.getcwd()) + while True: + if os.path.isdir(os.path.join(path, '.git')): + return path + if os.path.dirname(path) =3D=3D path: + break + path =3D os.path.dirname(path) + raise EdkrepoWorkspaceInvalidException(NOT_GIT_REPO) + +def split_commit_range(commit_range): + if len(commit_range.split('...')) =3D=3D 2: + return commit_range.split('...') + else: + return commit_range.split('..') + +def get_start_and_end_commit(commit_ish, repo): + #Determine the start and end commit for the upcoming squash + (commit1, commit2) =3D split_commit_range(commit_ish) + try: + repo.rev_parse(commit1) + except: + raise EdkrepoInvalidParametersException(COMMIT_NOT_FOUND.format(co= mmit1)) + try: + repo.rev_parse(commit2) + except: + raise EdkrepoInvalidParametersException(COMMIT_NOT_FOUND.format(co= mmit2)) + if len(repo.git.rev_list(commit_ish).split()) <=3D 0: + raise EdkrepoInvalidParametersException(SQUASH_COMMON_ANCESTOR_REQ= UIRED.format(commit1, commit2)) + if str(repo.commit(commit1)) not in repo.git.rev_list(commit2).split()= : + temp_commit =3D commit1 + commit1 =3D commit2 + commit2 =3D temp_commit + if str(repo.commit(commit1)) not in repo.git.rev_list(commit2).spl= it(): + raise EdkrepoInvalidParametersException(SQUASH_COMMON_ANCESTOR= _REQUIRED.format(commit1, commit2)) + start_commit =3D get_oldest_ancestor(commit1, commit2, repo) + end_commit =3D str(repo.commit(commit2)) + return (start_commit, end_commit) + +def get_oldest_ancestor(commit1, commit2, repo): + a =3D repo.git.rev_list('--first-parent', commit1).split() + b =3D repo.git.rev_list('--first-parent', commit2).split() + for commit in a: + if commit in b: + return commit + return None + +def commit_list_to_message(commit_list, one_line, repo): + message_list =3D [] + for commit in commit_list: + if one_line: + message_list.append('{}: {}'.format(commit, repo.commit(commit= ).message.split('\n')[0])) + else: + for line in repo.commit(commit).message.split('\n'): + message_list.append(line) + return '\n'.join(message_list) + +def squash_commits(start_commit, end_commit, branch_name, commit_message, = repo, reset_author=3DTrue): + #Create a branch at the latest commit + local_branch =3D repo.create_head(branch_name, end_commit) + repo.heads[local_branch.name].checkout() + #Set up environment variables for automating the interactive rebase + git_automation_dir =3D os.path.join(os.path.dirname(os.path.dirname(os= .path.realpath(__file__))), 'git_automation') + if sys.platform =3D=3D "win32": + os.environ['GIT_SEQUENCE_EDITOR'] =3D 'py "{}"'.format(os.path.joi= n(git_automation_dir, "rebase_squash.py")) + os.environ['GIT_EDITOR'] =3D 'py "{}"'.format(os.path.join(git_aut= omation_dir, "commit_msg.py")) + else: + os.environ['GIT_SEQUENCE_EDITOR'] =3D os.path.join(git_automation_= dir, "rebase_squash.py") + os.environ['GIT_EDITOR'] =3D os.path.join(git_automation_dir, "com= mit_msg.py") + os.environ['COMMIT_MESSAGE'] =3D commit_message + #Do an interactive rebase back to the oldest commit, squashing all com= mits between then and now + check_call(['git', 'rebase', '-i', '{}'.format(start_commit)]) + if reset_author: + repo.git.commit('--amend', '--no-edit', '--reset-author', '--signo= ff') + del os.environ['GIT_SEQUENCE_EDITOR'] + del os.environ['GIT_EDITOR'] + del os.environ['COMMIT_MESSAGE'] diff --git a/edkrepo/git_automation/__init__.py b/edkrepo/git_automation/__= init__.py new file mode 100644 index 0000000..c810c1f --- /dev/null +++ b/edkrepo/git_automation/__init__.py @@ -0,0 +1,8 @@ +#!/usr/bin/env python3 +# +## @file +# __init__.py +# +# Copyright (c) 2018 - 2019, Intel Corporation. All rights=20 +reserved.
# SPDX-License-Identifier: BSD-2-Clause-Patent # \ No newline at end of file diff --git a/edkrepo/git_automation/commit_msg.py b/edkrepo/git_automation/= commit_msg.py new file mode 100644 index 0000000..228a8ab --- /dev/null +++ b/edkrepo/git_automation/commit_msg.py @@ -0,0 +1,21 @@ +#!/usr/bin/env python3 +# +## @file +# commit_msg.py +# +# Copyright (c) 2018 - 2019, Intel Corporation. All rights=20 +reserved.
# SPDX-License-Identifier: BSD-2-Clause-Patent # + +import sys +import os +def main(): + if 'COMMIT_MESSAGE_NO_EDIT' in os.environ: + return + with open(sys.argv[1], 'w', encoding=3D"utf-8", errors=3D"backslashrep= lace") as f: + if 'COMMIT_MESSAGE' in os.environ: + f.write(os.environ['COMMIT_MESSAGE']) + else: + f.write("Cherry Pick\n") +if __name__ =3D=3D "__main__": + main() diff --git a/edkrepo/git_automation/rebase_squash.py b/edkrepo/git_automati= on/rebase_squash.py new file mode 100644 index 0000000..26abbbc --- /dev/null +++ b/edkrepo/git_automation/rebase_squash.py @@ -0,0 +1,23 @@ +#!/usr/bin/env python3 +# +## @file +# rebase_squash.py +# +# Copyright (c) 2018 - 2019, Intel Corporation. All rights=20 +reserved.
# SPDX-License-Identifier: BSD-2-Clause-Patent # + +import sys +import re +def main(): + with open(sys.argv[1], 'r', errors=3D"surrogateescape") as f: + lines =3D f.readlines() + r =3D re.compile("^pick(.*)$") + with open(sys.argv[1], 'w', encoding=3D"utf-8", errors=3D"backslashrep= lace") as f: + f.write(lines[0]) + for i in range(1,len(lines)): + m =3D r.match(lines[i]) + if m: f.write("s {}\n".format(m.group(1))) + else: f.write(lines[i]) +if __name__ =3D=3D "__main__": + main() diff --git a/setup.py b/setup.py index 1b9edad..e14aed1 100755 --- a/setup.py +++ b/setup.py @@ -12,7 +12,9 @@ from setuptools import setup setup(name=3D'edkrepo', version=3D'2.0.0', description=3D'The edkrepo tools', - packages=3D['edkrepo', 'edkrepo.commands', 'edkrepo.commands.argumen= ts', 'edkrepo.commands.humble', 'edkrepo.common', 'edkrepo.config', 'edkrep= o_manifest_parser', 'project_utils'], + packages=3D['edkrepo', 'edkrepo.commands', 'edkrepo.commands.argumen= ts', 'edkrepo.commands.humble', + 'edkrepo.git_automation', 'edkrepo.common', 'edkrepo.confi= g', 'edkrepo_manifest_parser', + 'project_utils'], package_data=3D{ }, include_package_data=3DTrue, -- 2.24.0.windows.2