From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mga06.intel.com (mga06.intel.com [134.134.136.31]) by mx.groups.io with SMTP id smtpd.web11.1123.1589906743463718489 for ; Tue, 19 May 2020 09:45:43 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=@intel.onmicrosoft.com header.s=selector2-intel-onmicrosoft-com header.b=IV6mmUZn; spf=pass (domain: intel.com, ip: 134.134.136.31, mailfrom: ashley.e.desimone@intel.com) IronPort-SDR: LbUp4foKUqC+vinYQNydBXPTuxpCdU5JpHGg96sTIosYqndlMwu4WeO9uuwXKIGzRI5WUq/UVk WxPkvw4kjnoA== X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga006.fm.intel.com ([10.253.24.20]) by orsmga104.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 19 May 2020 09:45:42 -0700 IronPort-SDR: Gj7GOaqKjqueeTCDtdluKXC0d3KkxuZWuxflldcLwT15nTbtBOB0ygqUVxqiU1AX4rmnf+OBOE 49B7UG2rlyQg== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.73,410,1583222400"; d="scan'208";a="466193219" Received: from fmsmsx105.amr.corp.intel.com ([10.18.124.203]) by fmsmga006.fm.intel.com with ESMTP; 19 May 2020 09:45:42 -0700 Received: from fmsmsx118.amr.corp.intel.com (10.18.116.18) by FMSMSX105.amr.corp.intel.com (10.18.124.203) with Microsoft SMTP Server (TLS) id 14.3.439.0; Tue, 19 May 2020 09:45:42 -0700 Received: from FMSEDG001.ED.cps.intel.com (10.1.192.133) by fmsmsx118.amr.corp.intel.com (10.18.116.18) with Microsoft SMTP Server (TLS) id 14.3.439.0; Tue, 19 May 2020 09:45:42 -0700 Received: from NAM10-MW2-obe.outbound.protection.outlook.com (104.47.55.108) by edgegateway.intel.com (192.55.55.68) with Microsoft SMTP Server (TLS) id 14.3.439.0; Tue, 19 May 2020 09:45:41 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=mxyKrNfpQttiI7IDoYJDL45VexW0SDLgIAzx1GN+G7eOk7XUV5ol1OWC/DdKgp09nGC2/rr35IbJBBzg6OfPCRKi3O5cg/ERBw2SLONt9mg6ulKVDDqtZTDJv0V5zpPmpkUOjdTT2PWg+16m1uVPhHkbtITm79AeBXyscTFnqzJV61KPZoz8BsSCna9M5klCGo81eVWSJWmPuDm/6299L6dKPAOiBsKjQXolbiSGpaOu5QDdV9fYvwc7x/0Y/E4+On1ipqIDn3uDcN0j1bIti2yNSpYoIjAuQ0z6TUs8F3NY5WKGsm0l23SE+v9Erptlw9XaZbBPNciqhJ7Mq8RYOw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=hRKxvmFtu3u3YLqbOBg3FAr1I547xRQ9LLtjqJ7v3bY=; b=Ks2rVCokFNlGsKyF34NrIUkDtWIVBmFmVgybrGAJGM6dW20zrlmDpLslF3d2lm1mHCWJylxJ0F2OoYeANSglzjlNTyZEtt049Og08BCFkHAfV4jAelxksnZ2OCaLr+s3V7UwysiD4Tk8E63NKNo+/t2XehNcai8/dDGBrie/vQpdjvi6JcyuyqTUfjo82T+0eKqoZkcPPRFdJZa/qPNzTlaOQmf2uqHfjcZ44SpMNHPfpeNJeEkUtHpXbhOqNmLZb85T+QEcdiffnXH8NFHNmTgnUtZeTJ4F4+Tyh0EDcJv+uDSO93xLY1k4XbjGBVUfVU3Dd6FnpxRcq31sUsETZA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com; dkim=pass header.d=intel.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=intel.onmicrosoft.com; s=selector2-intel-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=hRKxvmFtu3u3YLqbOBg3FAr1I547xRQ9LLtjqJ7v3bY=; b=IV6mmUZn9CKC2GGa8F4VMMAPjFMT10Za8mxFkStOXtZ3DP8mHmHj32KFpebx93KJhda4TLgDFBQR44Z94RJyQTXddXMPaR2jRF1LRNkpVJvjY3quA70suv+7IU/YRjtqidWWiMsXFfvih13fsK0HM8P0GUiDckJRbCnm1lkSA8U= Received: from BY5PR11MB3973.namprd11.prod.outlook.com (2603:10b6:a03:185::29) by BY5PR11MB4356.namprd11.prod.outlook.com (2603:10b6:a03:1bf::10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3000.25; Tue, 19 May 2020 16:45:40 +0000 Received: from BY5PR11MB3973.namprd11.prod.outlook.com ([fe80::296a:eb1c:2b52:2904]) by BY5PR11MB3973.namprd11.prod.outlook.com ([fe80::296a:eb1c:2b52:2904%7]) with mapi id 15.20.3021.020; Tue, 19 May 2020 16:45:40 +0000 From: "Ashley E Desimone" To: "Bjorge, Erik C" , "devel@edk2.groups.io" CC: "Desimone, Nathaniel L" , "Pandya, Puja" , Bret Barkelew , "Agyeman, Prince" Subject: Re: [edk2-staging/EdkRepo] [PATCH v1 2/2] EdkRepo: Update commands to use new submodule code Thread-Topic: [edk2-staging/EdkRepo] [PATCH v1 2/2] EdkRepo: Update commands to use new submodule code Thread-Index: AQHWK+q29VPGx7C1oki/wg1/ciyXmKiuurzwgAADC0CAAOMBEA== Date: Tue, 19 May 2020 16:45:40 +0000 Message-ID: References: <66a3235a3f4de04e0b8a549901f8321f8d1cf950.1589678697.git.erik.c.bjorge@intel.com> In-Reply-To: Accept-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: dlp-version: 11.2.0.6 dlp-reaction: no-action dlp-product: dlpe-windows authentication-results: intel.com; dkim=none (message not signed) header.d=none;intel.com; dmarc=none action=none header.from=intel.com; x-originating-ip: [192.55.52.201] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: ff6208ea-034a-470e-c75b-08d7fc141081 x-ms-traffictypediagnostic: BY5PR11MB4356: x-ms-exchange-transport-forked: True x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:8882; x-forefront-prvs: 040866B734 x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: PCTvO2vYmuM5mpiCJSg9untRkDQuqaTEHUw9JVrCYllL2v55DWF5KidHNTIMk+JuaVEO1dw6Ej7sAtwWqRbRqvpFrJ/582aQlKHZnzTz9b9Loy68qdvgjXO1WQOJyStud1F+I3ROTZIPkOoB0fM7vlzou5l1TwUINauDiBFMKlpCj8qi5FozRFEW11w7zDbCViTF0MCIw1FL7BCoQJB4OfU6YBzHIAYzJy/wpMVAHpjpU/dBpSACngw7OtgKdOJMu2gE1LCEjL7K7qX+qvQew7T0448H+59UYs/KUnJW6OesVahQxylpIbV1TZJb2z/0PtSZvkpBplI+9Eh4pya1Y12loBI6Sw/MqLLynIdmQ9J0QU/OqlcnM9Rjpz0RZRRgtJLSL0GMKg0azfvUGaB/fZtPoKAxY2xHWz06RnB5kPeMe7FiHSZJQbIdhYDEEDRv x-forefront-antispam-report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BY5PR11MB3973.namprd11.prod.outlook.com;PTR:;CAT:NONE;SFTY:;SFS:(396003)(39860400002)(366004)(136003)(376002)(346002)(71200400001)(7696005)(8936002)(26005)(316002)(52536014)(478600001)(186003)(8676002)(53546011)(6506007)(5660300002)(66446008)(110136005)(4326008)(55016002)(30864003)(54906003)(33656002)(66476007)(66556008)(9686003)(66946007)(64756008)(2906002)(107886003)(76116006)(86362001)(15650500001);DIR:OUT;SFP:1102; x-ms-exchange-antispam-messagedata: Hg2b/9Dki7Ce7e9OQdgl44zWvOMd+K/v+UqeCZrEnvJOvVIHOOfwbdEYB6YmoawiD0+im/3RhQf3BXnpqWorkZvyIfaPJ2fzfy3tl4/eV7Lh4UqbUSEg5F2+5t8Sb2v9V7OBt/lEpw6AEGHmsEXz0lzHkVsTJkRCahN1N9eKuR6Y9tgnD0uwCmk3nJaEqaSiRPCE3W1xECBhUOT5XcVrjEK4l0q7AIYBYrICFHz1ArM/zPJdUiQ75g+qL50az2Gzjcv605Gg8VMg0QwIDOXvotg8vsPpy6HZtLBTQV0IayyihCez/ZRD/UIg3PoJPso29dvx72g7oLjKN/oIVuxtji5kk3v2kKGEad326c6zjK0vEqfpIM2n3vnn5exJzELprVM4zAKYhgMOxuXVYvhN3JQGhuto7iXundxmuOq5utMXKIRRueS4YfFvkhvW+FwfhZ0pNgRQvfsqPcpf9P/WgNbMXDSBaROJX9RDxy5mYrzjSl2+bZ1vMyvIGc4XVLf3 MIME-Version: 1.0 X-MS-Exchange-CrossTenant-Network-Message-Id: ff6208ea-034a-470e-c75b-08d7fc141081 X-MS-Exchange-CrossTenant-originalarrivaltime: 19 May 2020 16:45:40.1391 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 46c98d88-e344-4ed4-8496-4ed7712e255d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: 8PYy1oJeT0W5XDuXHvIjpPB0XZdn7C5ZCEBN6L4LZM64TinSwL8NLBS6JAY3EEqyzybhROHvCb6+4wFaBrIOedRi7BR2yzmJV0p2qCBcqow= X-MS-Exchange-Transport-CrossTenantHeadersStamped: BY5PR11MB4356 Return-Path: ashley.e.desimone@intel.com X-OriginatorOrg: intel.com Content-Language: en-US Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Hi Erik, Thanks for clarifying. I reapplied the patches this morning to look at and = you are correct. Either I missed the content or made a mistake in applying = the patches. Please ignore these comments. Thanks, Ashley Reviewed-by: Ashley DeSimone -----Original Message----- From: Bjorge, Erik C =20 Sent: Monday, May 18, 2020 8:14 PM To: Desimone, Ashley E ; devel@edk2.groups.io Cc: Desimone, Nathaniel L ; Pandya, Puja ; Bret Barkelew ; Agyeman= , Prince Subject: RE: [edk2-staging/EdkRepo] [PATCH v1 2/2] EdkRepo: Update commands= to use new submodule code Ashley, I am confused. Here are the lines from the patch that remove the skip_subm= odule parameter. -def clone_repos(args, workspace_dir, repos_to_clone, project_client_side_h= ooks, config, skip_submodule, manifest): +def clone_repos(args, workspace_dir, repos_to_clone, project_client_side_h= ooks, config, manifest): Here is the code from below where the call is removed in clone_repos. else: raise EdkrepoManifestInvalidException(MISSING_BRANCH_COMMIT) =20 - if not skip_submodule: - if repo_to_clone.enable_submodule: - maintain_submodules(repo_to_clone, repo, args.verbose) - Can you please double check these comments? Thanks, -Erik -----Original Message----- From: Desimone, Ashley E Sent: Monday, May 18, 2020 8:01 PM To: Bjorge, Erik C ; devel@edk2.groups.io Cc: Desimone, Nathaniel L ; Pandya, Puja ; Bret Barkelew ; Agyeman= , Prince Subject: RE: [edk2-staging/EdkRepo] [PATCH v1 2/2] EdkRepo: Update commands= to use new submodule code Hi Erik, You have not updated the definition of clone_repos() in common_repo_functio= ns.py to reflect that you are no longer passing the skip_submodule paramete= r which is not set to a default value. Additionally you have not removed th= e calls to the previous submodule initialization code from clone_repos().=20 Thanks, Ashley -----Original Message----- From: Bjorge, Erik C Sent: Saturday, May 16, 2020 6:29 PM To: devel@edk2.groups.io Cc: Desimone, Ashley E ; Desimone, Nathaniel L= ; Pandya, Puja ; Br= et Barkelew ; Agyeman, Prince Subject: [edk2-staging/EdkRepo] [PATCH v1 2/2] EdkRepo: Update commands to = use new submodule code Replaced use of maintain_submodules to use new common code. This allows fo= r selective submodule initialization via the manifest file. Cc: Ashley E Desimone Cc: Nate DeSimone Cc: Puja Pandya Cc: Bret Barkelew Cc: Prince Agyeman Cc: Erik Bjorge Signed-off-by: Erik Bjorge --- edkrepo/commands/checkout_pin_command.py | 6 ++++ edkrepo/commands/clone_command.py | 20 +++++++---- edkrepo/commands/sync_command.py | 23 +++++++++---- edkrepo/common/common_repo_functions.py | 43 +++++------------------- 4 files changed, 44 insertions(+), 48 deletions(-) diff --git a/edkrepo/commands/checkout_pin_command.py b/edkrepo/commands/ch= eckout_pin_command.py index 4aaf2b5..c250c49 100644 --- a/edkrepo/commands/checkout_pin_command.py +++ b/edkrepo/commands/checkout_pin_command.py @@ -22,6 +22,8 @@ from edkrepo.common.workspace_maintenance.manifest_repos_= maintenance import list from edkrepo.common.workspace_maintenance.manifest= _repos_maintenance import find_source_manifest_repo from edkrepo.config.co= nfig_factory import get_workspace_path, get_workspace_manifest from edkrep= o_manifest_parser.edk_manifest import ManifestXml +from project_utils.submodule import deinit_submodules, init_submodules + =20 class CheckoutPinCommand(EdkrepoCommand): def __init__(self): @@ -68,11 +70,15 @@ class CheckoutPinCommand(EdkrepoCommand): if sparse_enabled: print(SPARSE_RESET) reset_sparse_checkout(workspace_path, manifest_sources) + submodule_combo =3D pin.general_config.current_combo + deinit_submodules(workspace_path, manifest, manifest.general_confi= g.current_combo, + pin, submodule_combo, args.verbose) pin_repo_sources =3D pin.get_repo_sources(pin.general_config.curre= nt_combo) try: checkout_repos(args.verbose, args.override, pin_repo_sources, = workspace_path, manifest) manifest.write_current_combo(humble.PIN_COMBO.format(args.pinf= ile)) finally: + init_submodules(workspace_path, pin, submodule_combo, + args.verbose) if sparse_enabled: print(SPARSE_CHECKOUT) sparse_checkout(workspace_path, pin_repo_sources, manifest= ) diff --git a/edkrepo/commands/clone_command.py b/edkrepo/commands/clone_c= ommand.py index 179aaf7..653e10b 100644 --- a/edkrepo/commands/clone_command.py +++ b/edkrepo/commands/clone_command.py @@ -24,6 +24,7 @@ from edkrepo.common.workspace_maintenance.manifest_repos_= maintenance import pull from edkrepo.common.workspace_maintenance.manifest= _repos_maintenance import list_available_manifest_repos from edkrepo.commo= n.workspace_maintenance.humble.manifest_repos_maintenance_humble import PRO= J_NOT_IN_REPO, SOURCE_MANIFEST_REPO_NOT_FOUND from edkrepo_manifest_parser= .edk_manifest import CiIndexXml, ManifestXml +from project_utils.submodule import init_submodules =20 =20 class CloneCommand(EdkrepoCommand): @@ -104,7 +105,6 @@ class CloneCommand(EdkrepoCommand): manifest =3D ManifestXml(local_manifest_path) =20 # Process the combination name and make sure it can be found in th= e manifest - combo_name =3D None if args.Combination is not None: try: combo_name =3D case_insensitive_single_match(args.Combinat= ion, combinations_in_manifest(manifest)) @@ -118,13 +118,15 @@ class CloneCommand(EdkrepoCommand): # Since pin files are subset of manifest files they do not hav= e a "default combo" it is set to None. In this # case use the current_combo instead. combo_name =3D manifest.general_config.current_combo + else: + # If a combo was not specified or a pin file used the default = combo should be cloned. Also ensure that the + # current combo is updated to match. + combo_name =3D manifest.general_config.default_combo + manifest.write_current_combo(combo_name) =20 # Get the list of repos to clone and clone them - manifest_config =3D manifest.general_config - if combo_name: - repo_sources_to_clone =3D manifest.get_repo_sources(combo_name= ) - else: - repo_sources_to_clone =3D manifest.get_repo_sources(manifest_c= onfig.default_combo) + repo_sources_to_clone =3D manifest.get_repo_sources(combo_name) + #check that the repo sources do not contain duplicated local roots local_roots =3D [r.root for r in repo_sources_to_clone] for root in local_roots: @@ -137,7 +139,11 @@ class CloneCommand(EdkrepoCommand): # Set up submodule alt url config settings prior to cloning any re= pos submodule_included_configs =3D write_included_config(manifest.remo= tes, manifest.submodule_alternate_remotes, local_manifest_dir) write_conditional_include(workspace_dir, repo_sources_to_clone, su= bmodule_included_configs) - clone_repos(args, workspace_dir, repo_sources_to_clone, project_cl= ient_side_hooks, config, args.skip_submodule, manifest) + clone_repos(args, workspace_dir, repo_sources_to_clone,=20 + project_client_side_hooks, config, manifest) + + # Init submodules + if not args.skip_submodule: + init_submodules(workspace_dir, manifest, combo_name, + args.verbose) =20 # Perform a sparse checkout if requested. use_sparse =3D args.sparse diff --git a/edkrepo/commands/sync_command.py b/edkrepo/commands/sync_comma= nd.py index da99397..d3c724f 100644 --- a/edkrepo/commands/sync_command.py +++ b/edkrepo/commands/sync_command.py @@ -38,7 +38,7 @@ from edkrepo.common.common_repo_functions import checkout= _repos, check_dirty_rep from edkrepo.common.common_repo_functions import u= pdate_editor_config from edkrepo.common.common_repo_functions import updat= e_repo_commit_template, get_latest_sha from edkrepo.common.common_repo_fun= ctions import has_primary_repo_remote, fetch_from_primary_repo, in_sync_wit= h_primary -from edkrepo.common.common_repo_functions import update_hooks, m= aintain_submodules, combinations_in_manifest +from edkrepo.common.common_repo_functions import update_hooks,=20 +combinations_in_manifest from edkrepo.common.common_repo_functions import write_included_config, re= move_included_config from edkrepo.common.workspace_maintenance.workspace_m= aintenance import generate_name_for_obsolete_backup from edkrepo.common.wo= rkspace_maintenance.manifest_repos_maintenance import pull_workspace_manife= st_repo @@ -48,6 +48,7 @@ from edkrepo.common.ui_functions import init_colo= r_console from edkrepo.config.config_factory import get_workspace_path, ge= t_workspace_manifest, get_edkrepo_global_data_directory from edkrepo.confi= g.config_factory import get_workspace_manifest_file from edkrepo_manifest_= parser.edk_manifest import CiIndexXml, ManifestXml +from project_utils.submodule import deinit_submodules, init_submodules =20 =20 class SyncCommand(EdkrepoCommand): @@ -84,6 +85,7 @@ class SyncCommand(EdkrepoCommand): current_combo =3D initial_manifest.general_config.current_combo initial_sources =3D initial_manifest.get_repo_sources(current_comb= o) initial_hooks =3D initial_manifest.repo_hooks + initial_combo =3D current_combo =20 source_global_manifest_repo =3D find_source_manifest_repo(initial_= manifest, config['cfg_file'], config['user_cfg_file'], args.source_manifest= _repo) pull_workspace_manifest_repo(initial_manifest, config['cfg_file'],= config['user_cfg_file'], args.source_manifest_repo, False) @@ -100,6 +102,= 7 @@ class SyncCommand(EdkrepoCommand): if not args.update_local_manifest: self.__check_for_new_manifest(args, config, initial_manifest, = workspace_path, global_manifest_directory) check_dirty_repos(initial_manifest, workspace_path) + # Determine if sparse checkout needs to be disabled for this opera= tion sparse_settings =3D initial_manifest.sparse_settings sparse_enabled =3D sparse_checkout_enabled(workspace_path, initial= _sources) @@ -113,7 +116,7 @@ class SyncCommand(EdkrepoCommand): reset_sparse_checkout(workspace_path, initial_sources) =20 # Get the latest manifest if requested - if args.update_local_manifest: #NOTE: hyphens in arg name replaced= with underscores due to argparse + if args.update_local_manifest: # NOTE: hyphens in arg name=20 + replaced with underscores due to argparse self.__update_local_manifest(args, config, initial_manifest, w= orkspace_path, global_manifest_directory) manifest =3D get_workspace_manifest() if args.update_local_manifest: @@ -127,6 +130,12 @@ class SyncCommand(EdkrepoCommand): repo_sources_to_sync =3D manifest.get_repo_sources(current_com= bo) manifest.write_current_combo(current_combo) =20 + # At this point both new and old manifest files are ready so we ca= n deinit any + # submodules that are removed due to a manifest update. + if not args.skip_submodule: + deinit_submodules(workspace_path, initial_manifest, initial_co= mbo, + manifest, current_combo, args.verbose) + sync_error =3D False # Calculate the hooks which need to be updated, added or removed f= or the sync if args.update_local_manifest: @@ -195,16 +204,16 @@ class SyncCommand(EdkrepoCommand): elif args.verbose: print(NO_SYNC_DETACHED_HEAD.format(repo_to_sync.root)) =20 - if not args.skip_submodule: - if repo_to_sync.enable_submodule: - # Perform submodule updates and url redirection - maintain_submodules(repo_to_sync, repo) # Update commit message templates update_repo_commit_template(workspace_path, repo, repo_to_sync= , config, source_global_manifest_repo) =20 if sync_error: print(SYNC_ERROR) =20 + # Initialize submodules + if not args.skip_submodule: + init_submodules(workspace_path, manifest, current_combo, + args.verbose) + # Restore sparse checkout state if sparse_enabled: print(SPARSE_CHECKOUT) @@ -326,7 +335,7 @@ class SyncCommand(EdkrepoCommand): print(path_to_source) if len(sources_to_remove) > 0: print(SYNC_REMOVE_LIST_END_FORMATTING) - clone_repos(args, workspace_path, sources_to_clone, new_manife= st_to_check.repo_hooks, config, args.skip_submodule, new_manifest_to_check) + clone_repos(args, workspace_path, sources_to_clone,=20 + new_manifest_to_check.repo_hooks, config, new_manifest_to_check) # Make a list of and only checkout repos that were newly clone= d. Sync keeps repos on their initial active branches # cloning the entire combo can prevent existing repos from cor= rectly being returned to their proper branch repos_to_checkout =3D [] diff --git a/edkrepo/common/common_repo_functions.py b/edkrepo/common/commo= n_repo_functions.py index 61133f1..18e545b 100644 --- a/edkrepo/common/common_repo_functions.py +++ b/edkrepo/common/common_repo_functions.py @@ -68,12 +68,13 @@ from edkrepo_manifest_parser.edk_manifest_validation im= port validate_manifestrep from edkrepo_manifest_parser.edk_manifest_valida= tion import get_manifest_validation_status from edkrepo_manifest_parser.ed= k_manifest_validation import print_manifest_errors from edkrepo_manifest_p= arser.edk_manifest_validation import validate_manifestfiles +from project_utils.submodule import deinit_submodules, init_submodules =20 CLEAR_LINE =3D '\x1b[K' DEFAULT_REMOTE_NAME =3D 'origin' PRIMARY_REMOTE_NAME =3D 'primary' =20 -def clone_repos(args, workspace_dir, repos_to_clone, project_client_side_h= ooks, config, skip_submodule, manifest): +def clone_repos(args, workspace_dir, repos_to_clone, project_client_side_h= ooks, config, manifest): for repo_to_clone in repos_to_clone: local_repo_path =3D os.path.join(workspace_dir, repo_to_clone.root= ) local_repo_url =3D repo_to_clone.remote_url @@ -108,10 +109,6 @@ d= ef clone_repos(args, workspace_dir, repos_to_clone, project_client_side_hoo= ks, else: raise EdkrepoManifestInvalidException(MISSING_BRANCH_COMMIT) =20 - if not skip_submodule: - if repo_to_clone.enable_submodule: - maintain_submodules(repo_to_clone, repo, args.verbose) - try: if 'source_manifest_repo' in vars(args).keys(): src_manifest_repo =3D find_source_manifest_repo(manifest, = config['cfg_file'], config['user_cfg_file'], args.source_manifest_repo) @@ = -184,32 +181,6 @@ def write_conditional_include(workspace_path, repo_source= s, included_configs): gitglobalconfig.add_section(section) gitglobalconfig.set(section, 'path', path) =20 -def maintain_submodules(repo_sources, repo, verbose =3D False): - try: - output_data =3D repo.git.execute(['git', 'submodule', 'init'], wit= h_extended_output=3DTrue, with_stdout=3DTrue) - if verbose and output_data[0]: - print(output_data[0]) - if output_data[1]: - print(output_data[1]) - if verbose and output_data[2]: - print(output_data[2]) - output_data =3D repo.git.execute(['git', 'submodule', 'sync', '--r= ecursive'], with_extended_output=3DTrue, with_stdout=3DTrue) - if verbose and output_data[0]: - print(output_data[0]) - if output_data[1]: - print(output_data[1]) - if verbose and output_data[2]: - print(output_data[2]) - output_data =3D repo.git.execute(['git', 'submodule', 'update', '-= -recursive'], with_extended_output=3DTrue, with_stdout=3DTrue) - if verbose and output_data[0]: - print(output_data[0]) - if output_data[1]: - print(output_data[1]) - if verbose and output_data[2]: - print(output_data[2]) - except: - raise EdkrepoGitException(SUBMODULE_FAILURE.format(repo_sources.re= mote_name)) - def install_hooks(hooks, local_repo_path, repo_for_install, config, global= _manifest_directory): # Determine the which hooks are for the repo in question and which are= from a URL based source or are in a global # manifest repo relative path @@ -399,9 +370,6 @@ def checkout_repos(verbose, override, repos_to_checkout= , workspace_path, manifes else: raise EdkrepoManifestInvalidException(MISSING_BRANCH_COMMIT) =20 - if repo_to_checkout.enable_submodule: - maintain_submodules(repo_to_checkout, repo, verbose) - def validate_manifest_repo(manifest_repo, verbose=3DFalse, archived=3DFals= e): print(VERIFY_GLOBAL) if archived: @@ -514,9 +482,11 @@ def checkout(combination_or_sha, verbose=3DFalse, over= ride=3DFalse, log=3DNone): # Create combo_or_sha so we have original input and do not introduce a= ny # unintended behavior by messing with parameters. combo_or_sha =3D combination_or_sha + submodule_combo =3D manifest.general_config.current_combo try: # Try to handle normalize combo name to match the manifest file. combo_or_sha =3D case_insensitive_single_match(combo_or_sha, combi= nations_in_manifest(manifest)) + submodule_combo =3D combo_or_sha except: # No match so leave it alone. It must be a SHA1 or a bad combo na= me. pass @@ -528,6 +498,10 @@ def checkout(combination_or_sha, verbose=3DFalse, over= ride=3DFalse, log=3DNone): log=3Dlog) initial_repo_sources =3D manifest.get_repo_sources(manifest.general_co= nfig.current_combo) =20 + # Deinit any submodules that have been removed. + deinit_submodules(workspace_path, manifest, manifest.general_config.cu= rrent_combo, + manifest, submodule_combo, verbose) + # Disable sparse checkout current_repos =3D initial_repo_sources sparse_enabled =3D sparse_checkout_enabled(workspace_path, initial_rep= o_sources) @@ -566,6 +540,7 @@ def checkout(combination_or_sha, verbose=3DF= alse, override=3DFalse, log=3DNone): # Return to the initial combo, since there was an issue with cheki= ng out the selected combo checkout_repos(verbose, override, initial_repo_sources, workspace_= path, manifest) finally: + init_submodules(workspace_path, manifest, submodule_combo, + verbose) if sparse_enabled or sparse_diff: print(SPARSE_CHECKOUT) sparse_checkout(workspace_path, current_repos, manifest) -- 2.21.0.windows.1