From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mga05.intel.com (mga05.intel.com [192.55.52.43]) by mx.groups.io with SMTP id smtpd.web10.5526.1589858024539496738 for ; Mon, 18 May 2020 20:13:44 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=@intel.onmicrosoft.com header.s=selector2-intel-onmicrosoft-com header.b=H+ISIsuR; spf=pass (domain: intel.com, ip: 192.55.52.43, mailfrom: erik.c.bjorge@intel.com) IronPort-SDR: tajJEHG2BtX7iUXyVeatCQlqO45Ux1pNeemicheFpCZTiJyQFdsW9lkR3DNTD0CY6VjBoHlHXo WeEASxGDVWfQ== X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga001.jf.intel.com ([10.7.209.18]) by fmsmga105.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 18 May 2020 20:13:44 -0700 IronPort-SDR: SSQwQkrVER7KNVDwCKm9KIVC+lmLP3vQhIEG8+Vcmpel0IorGhjJdfNF3GoZYsXBLCgnhV2RnN O30EUX85sazA== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.73,408,1583222400"; d="scan'208";a="343001445" Received: from fmsmsx108.amr.corp.intel.com ([10.18.124.206]) by orsmga001.jf.intel.com with ESMTP; 18 May 2020 20:13:43 -0700 Received: from fmsmsx609.amr.corp.intel.com (10.18.126.89) by FMSMSX108.amr.corp.intel.com (10.18.124.206) with Microsoft SMTP Server (TLS) id 14.3.439.0; Mon, 18 May 2020 20:13:43 -0700 Received: from fmsmsx609.amr.corp.intel.com (10.18.126.89) by fmsmsx609.amr.corp.intel.com (10.18.126.89) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.1713.5; Mon, 18 May 2020 20:13:43 -0700 Received: from FMSEDG001.ED.cps.intel.com (10.1.192.133) by fmsmsx609.amr.corp.intel.com (10.18.126.89) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256) id 15.1.1713.5 via Frontend Transport; Mon, 18 May 2020 20:13:43 -0700 Received: from NAM12-DM6-obe.outbound.protection.outlook.com (104.47.59.176) by edgegateway.intel.com (192.55.55.68) with Microsoft SMTP Server (TLS) id 14.3.439.0; Mon, 18 May 2020 20:13:43 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=N3cr+A2+sQ6MZwa1as6SQHIxKvE3S3YL3k4M5jVO/LCc17pW6LjBNbHBR/S9hV7Fw/94WptY1qwG27HohPSrzdwkaWmUGr2kGnoefGtEBv+ubeEmQE2CT93OdGZze1ksiizLOf+7Oram0YKFsoNOb4lepYgu+miZtxVLXlutIo1LeDFI56vtPBhfgz9tzkN/9CS+ISu7+O//FoUkuRCC9RSE+gniHj6HNDm+fxgQaP9imn9bUXSScx+zBFqssSOMfMwGzT4W6kGNdx9g81QlooCmGXqgF+GxmwU39i4KOS+t22y/OjHJCFU/nAe54Lz/PQYGji+/VfFVVjHLC4UFwg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=BgjcFvQwt8wMkUt27K7lTgwKt/eX+TPRGs/Z+SdsnWw=; b=GbXkYxJjWHzAuvCgIBTsc+ptWNOWqff18RQewqQVSpFtnVuHvNtCZLhjwx8OQC/nbK5bVPf3f2f5LK9jSgUttShO041nz//LiudAS0UHe4MKsmv2ZPXQ7S5sI3tqPbwjFxfegyYSSUY+eBM/DTCStKjSpNfWHzBSBCpxlXWkUS+9JX1EoYH1bj34nZbDT1byPA2RN6LjYFrGQh5Lq/8HcjmwBwCpehkFUVSXT5eXsQItEeGoLJgUbaJhhJ4I6jVZsjyfec/bkToPSi9+rJ0zNkNqq+To47gckE0sQ4i8FadQfN7PdP6unAo0du3qpmqQU2kyTKdaLmz0uOJy9ULtFA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com; dkim=pass header.d=intel.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=intel.onmicrosoft.com; s=selector2-intel-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=BgjcFvQwt8wMkUt27K7lTgwKt/eX+TPRGs/Z+SdsnWw=; b=H+ISIsuRvB5FoTZpfaGD63NxBjHyYoZobkFYnwne0sTqT2wng827khS35aVMK9cCa6wDKvXAm28BwDlmOsgHx5AvJ8VNYMwVkZqwy/zsz+14AiiszG6XvDQCy1BF42bWcQR0oe+T8pla92ESvS11GGyRG6IuFRRxRYrHOybwi/w= Received: from MW3PR11MB4554.namprd11.prod.outlook.com (2603:10b6:303:5d::7) by MW3PR11MB4668.namprd11.prod.outlook.com (2603:10b6:303:54::10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3000.25; Tue, 19 May 2020 03:13:41 +0000 Received: from MW3PR11MB4554.namprd11.prod.outlook.com ([fe80::c569:9903:4c10:9281]) by MW3PR11MB4554.namprd11.prod.outlook.com ([fe80::c569:9903:4c10:9281%6]) with mapi id 15.20.3000.033; Tue, 19 May 2020 03:13:41 +0000 From: "Bjorge, Erik C" To: "Desimone, Ashley E" , "devel@edk2.groups.io" CC: "Desimone, Nathaniel L" , "Pandya, Puja" , Bret Barkelew , "Agyeman, Prince" Subject: Re: [edk2-staging/EdkRepo] [PATCH v1 2/2] EdkRepo: Update commands to use new submodule code Thread-Topic: [edk2-staging/EdkRepo] [PATCH v1 2/2] EdkRepo: Update commands to use new submodule code Thread-Index: AQHWK+q29VPGx7C1oki/wg1/ciyXmKiuurzwgAADC0A= Date: Tue, 19 May 2020 03:13:41 +0000 Message-ID: References: <66a3235a3f4de04e0b8a549901f8321f8d1cf950.1589678697.git.erik.c.bjorge@intel.com> In-Reply-To: Accept-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: dlp-reaction: no-action dlp-version: 11.2.0.6 dlp-product: dlpe-windows authentication-results: intel.com; dkim=none (message not signed) header.d=none;intel.com; dmarc=none action=none header.from=intel.com; x-originating-ip: [50.53.38.57] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 84f9ae2a-2af0-4cc3-41e8-08d7fba2a1db x-ms-traffictypediagnostic: MW3PR11MB4668: x-ms-exchange-transport-forked: True x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:8273; x-forefront-prvs: 040866B734 x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: dhOPKYXfJPNzb13Fku8BpGibWc0hJslEaPxWTKgSZakwcJQNTMJI9uY+JeZc35Rp6vJptxdPRA2pvE3gWzloq8MIBzOyFuScCZSdht/W5dtFSw6yKBiey3JeSoCBJ1Cfrpa87BL+ReobpHOCc6f/HOaE7ECGf28v8WFWyxQ63Nvhya7XdM0CmYLB6BIOTA9+d20Sh0A8PEUeLgJ4R5gZSN9Y5uSOOwYLC6nlhNl7jXYaKwUdLWiUI4eXe9Odvvehl4Bq2ofgpT+ZzCQBNVFwrwmfFCaX+oybRYPtgUr03ip2vWL2R0lYek/KVxek8IFTNsubytixGnc7D9C9Ra4rhfO7OidEt3vpEz9dT9lri2rgDGhtCyGuvyQDFcm7GBRPiMXDagr/QTvptB69YiOqJB6bYjlGjGUTp8X9TIO/7FA0EF7QLQzxyQpjEm2FojHl x-forefront-antispam-report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:MW3PR11MB4554.namprd11.prod.outlook.com;PTR:;CAT:NONE;SFTY:;SFS:(39860400002)(366004)(396003)(346002)(376002)(136003)(478600001)(186003)(6506007)(15650500001)(9686003)(110136005)(53546011)(8936002)(64756008)(66446008)(52536014)(66556008)(26005)(66476007)(33656002)(107886003)(86362001)(66946007)(55016002)(5660300002)(8676002)(54906003)(30864003)(7696005)(2906002)(71200400001)(76116006)(316002)(4326008);DIR:OUT;SFP:1102; x-ms-exchange-antispam-messagedata: 84scBoPRziTaLrjMUW3OHuBD5/8dTsZBv8KPNCpX+jAHb/53Kx3DCCUCitF8NT+FBYQl/cE7zOUFGivmLqYgvbdyaBF31uXaoTLV2QJe5V1Q4UN/c85AC12lqO8m34Ix8Bjhu1h20FnpvWZMx+5PNRRPYxV4yfy+W9bFgNmtQFIBGFBP4+ZrRdoRw9Urlj3P8Ff4+0B/z9G+SkPgzIFLE/6eKdUXS0tp7cAQm38rOb9AOhWeBKyfZF85kyMAZqhTlCnLafW754pNKvUc3EF6H70pLpQwqaBPuAFKw+SWtXo6TEjlzGg6kWIm5UVemUpc2vvLIgzMX/tgWgu2qOBWEJ/lxyJj9zVMnH0dlvPxoIjy7Kovrc1atbHt0IYPcegdgWAmoBOCjecGm55SVQQnlBsT+gnr45bNaGfzw+AHF31w/bkpJOBM85HYxWcr4eto9rGP7JSveZQTLoqZdUgmbWrqYHiWrMitfkXKwkVBbmQ= MIME-Version: 1.0 X-MS-Exchange-CrossTenant-Network-Message-Id: 84f9ae2a-2af0-4cc3-41e8-08d7fba2a1db X-MS-Exchange-CrossTenant-originalarrivaltime: 19 May 2020 03:13:41.3834 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 46c98d88-e344-4ed4-8496-4ed7712e255d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: ArvAANnejVvS75bjcea/O56fjGKiZWClg5MgUtXQSWxZ4Tf+NtLZbtBjLTw+81DX0DrLLLHg6qQPlocVGgCVBg== X-MS-Exchange-Transport-CrossTenantHeadersStamped: MW3PR11MB4668 Return-Path: erik.c.bjorge@intel.com X-OriginatorOrg: intel.com Content-Language: en-US Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Ashley, I am confused. Here are the lines from the patch that remove the skip_subm= odule parameter. -def clone_repos(args, workspace_dir, repos_to_clone, project_client_side_h= ooks, config, skip_submodule, manifest): +def clone_repos(args, workspace_dir, repos_to_clone, project_client_side_h= ooks, config, manifest): Here is the code from below where the call is removed in clone_repos. else: raise EdkrepoManifestInvalidException(MISSING_BRANCH_COMMIT) =20 - if not skip_submodule: - if repo_to_clone.enable_submodule: - maintain_submodules(repo_to_clone, repo, args.verbose) - Can you please double check these comments? Thanks, -Erik -----Original Message----- From: Desimone, Ashley E =20 Sent: Monday, May 18, 2020 8:01 PM To: Bjorge, Erik C ; devel@edk2.groups.io Cc: Desimone, Nathaniel L ; Pandya, Puja ; Bret Barkelew ; Agyeman= , Prince Subject: RE: [edk2-staging/EdkRepo] [PATCH v1 2/2] EdkRepo: Update commands= to use new submodule code Hi Erik, You have not updated the definition of clone_repos() in common_repo_functio= ns.py to reflect that you are no longer passing the skip_submodule paramete= r which is not set to a default value. Additionally you have not removed th= e calls to the previous submodule initialization code from clone_repos().=20 Thanks, Ashley -----Original Message----- From: Bjorge, Erik C Sent: Saturday, May 16, 2020 6:29 PM To: devel@edk2.groups.io Cc: Desimone, Ashley E ; Desimone, Nathaniel L= ; Pandya, Puja ; Br= et Barkelew ; Agyeman, Prince Subject: [edk2-staging/EdkRepo] [PATCH v1 2/2] EdkRepo: Update commands to = use new submodule code Replaced use of maintain_submodules to use new common code. This allows fo= r selective submodule initialization via the manifest file. Cc: Ashley E Desimone Cc: Nate DeSimone Cc: Puja Pandya Cc: Bret Barkelew Cc: Prince Agyeman Cc: Erik Bjorge Signed-off-by: Erik Bjorge --- edkrepo/commands/checkout_pin_command.py | 6 ++++ edkrepo/commands/clone_command.py | 20 +++++++---- edkrepo/commands/sync_command.py | 23 +++++++++---- edkrepo/common/common_repo_functions.py | 43 +++++------------------- 4 files changed, 44 insertions(+), 48 deletions(-) diff --git a/edkrepo/commands/checkout_pin_command.py b/edkrepo/commands/ch= eckout_pin_command.py index 4aaf2b5..c250c49 100644 --- a/edkrepo/commands/checkout_pin_command.py +++ b/edkrepo/commands/checkout_pin_command.py @@ -22,6 +22,8 @@ from edkrepo.common.workspace_maintenance.manifest_repos_= maintenance import list from edkrepo.common.workspace_maintenance.manifest= _repos_maintenance import find_source_manifest_repo from edkrepo.config.co= nfig_factory import get_workspace_path, get_workspace_manifest from edkrep= o_manifest_parser.edk_manifest import ManifestXml +from project_utils.submodule import deinit_submodules, init_submodules + =20 class CheckoutPinCommand(EdkrepoCommand): def __init__(self): @@ -68,11 +70,15 @@ class CheckoutPinCommand(EdkrepoCommand): if sparse_enabled: print(SPARSE_RESET) reset_sparse_checkout(workspace_path, manifest_sources) + submodule_combo =3D pin.general_config.current_combo + deinit_submodules(workspace_path, manifest, manifest.general_confi= g.current_combo, + pin, submodule_combo, args.verbose) pin_repo_sources =3D pin.get_repo_sources(pin.general_config.curre= nt_combo) try: checkout_repos(args.verbose, args.override, pin_repo_sources, = workspace_path, manifest) manifest.write_current_combo(humble.PIN_COMBO.format(args.pinf= ile)) finally: + init_submodules(workspace_path, pin, submodule_combo, + args.verbose) if sparse_enabled: print(SPARSE_CHECKOUT) sparse_checkout(workspace_path, pin_repo_sources, manifest= ) diff --git a/edkrepo/commands/clone_command.py b/edkrepo/commands/clone_c= ommand.py index 179aaf7..653e10b 100644 --- a/edkrepo/commands/clone_command.py +++ b/edkrepo/commands/clone_command.py @@ -24,6 +24,7 @@ from edkrepo.common.workspace_maintenance.manifest_repos_= maintenance import pull from edkrepo.common.workspace_maintenance.manifest= _repos_maintenance import list_available_manifest_repos from edkrepo.commo= n.workspace_maintenance.humble.manifest_repos_maintenance_humble import PRO= J_NOT_IN_REPO, SOURCE_MANIFEST_REPO_NOT_FOUND from edkrepo_manifest_parser= .edk_manifest import CiIndexXml, ManifestXml +from project_utils.submodule import init_submodules =20 =20 class CloneCommand(EdkrepoCommand): @@ -104,7 +105,6 @@ class CloneCommand(EdkrepoCommand): manifest =3D ManifestXml(local_manifest_path) =20 # Process the combination name and make sure it can be found in th= e manifest - combo_name =3D None if args.Combination is not None: try: combo_name =3D case_insensitive_single_match(args.Combinat= ion, combinations_in_manifest(manifest)) @@ -118,13 +118,15 @@ class CloneCommand(EdkrepoCommand): # Since pin files are subset of manifest files they do not hav= e a "default combo" it is set to None. In this # case use the current_combo instead. combo_name =3D manifest.general_config.current_combo + else: + # If a combo was not specified or a pin file used the default = combo should be cloned. Also ensure that the + # current combo is updated to match. + combo_name =3D manifest.general_config.default_combo + manifest.write_current_combo(combo_name) =20 # Get the list of repos to clone and clone them - manifest_config =3D manifest.general_config - if combo_name: - repo_sources_to_clone =3D manifest.get_repo_sources(combo_name= ) - else: - repo_sources_to_clone =3D manifest.get_repo_sources(manifest_c= onfig.default_combo) + repo_sources_to_clone =3D manifest.get_repo_sources(combo_name) + #check that the repo sources do not contain duplicated local roots local_roots =3D [r.root for r in repo_sources_to_clone] for root in local_roots: @@ -137,7 +139,11 @@ class CloneCommand(EdkrepoCommand): # Set up submodule alt url config settings prior to cloning any re= pos submodule_included_configs =3D write_included_config(manifest.remo= tes, manifest.submodule_alternate_remotes, local_manifest_dir) write_conditional_include(workspace_dir, repo_sources_to_clone, su= bmodule_included_configs) - clone_repos(args, workspace_dir, repo_sources_to_clone, project_cl= ient_side_hooks, config, args.skip_submodule, manifest) + clone_repos(args, workspace_dir, repo_sources_to_clone,=20 + project_client_side_hooks, config, manifest) + + # Init submodules + if not args.skip_submodule: + init_submodules(workspace_dir, manifest, combo_name, + args.verbose) =20 # Perform a sparse checkout if requested. use_sparse =3D args.sparse diff --git a/edkrepo/commands/sync_command.py b/edkrepo/commands/sync_comma= nd.py index da99397..d3c724f 100644 --- a/edkrepo/commands/sync_command.py +++ b/edkrepo/commands/sync_command.py @@ -38,7 +38,7 @@ from edkrepo.common.common_repo_functions import checkout= _repos, check_dirty_rep from edkrepo.common.common_repo_functions import u= pdate_editor_config from edkrepo.common.common_repo_functions import updat= e_repo_commit_template, get_latest_sha from edkrepo.common.common_repo_fun= ctions import has_primary_repo_remote, fetch_from_primary_repo, in_sync_wit= h_primary -from edkrepo.common.common_repo_functions import update_hooks, m= aintain_submodules, combinations_in_manifest +from edkrepo.common.common_repo_functions import update_hooks,=20 +combinations_in_manifest from edkrepo.common.common_repo_functions import write_included_config, re= move_included_config from edkrepo.common.workspace_maintenance.workspace_m= aintenance import generate_name_for_obsolete_backup from edkrepo.common.wo= rkspace_maintenance.manifest_repos_maintenance import pull_workspace_manife= st_repo @@ -48,6 +48,7 @@ from edkrepo.common.ui_functions import init_colo= r_console from edkrepo.config.config_factory import get_workspace_path, ge= t_workspace_manifest, get_edkrepo_global_data_directory from edkrepo.confi= g.config_factory import get_workspace_manifest_file from edkrepo_manifest_= parser.edk_manifest import CiIndexXml, ManifestXml +from project_utils.submodule import deinit_submodules, init_submodules =20 =20 class SyncCommand(EdkrepoCommand): @@ -84,6 +85,7 @@ class SyncCommand(EdkrepoCommand): current_combo =3D initial_manifest.general_config.current_combo initial_sources =3D initial_manifest.get_repo_sources(current_comb= o) initial_hooks =3D initial_manifest.repo_hooks + initial_combo =3D current_combo =20 source_global_manifest_repo =3D find_source_manifest_repo(initial_= manifest, config['cfg_file'], config['user_cfg_file'], args.source_manifest= _repo) pull_workspace_manifest_repo(initial_manifest, config['cfg_file'],= config['user_cfg_file'], args.source_manifest_repo, False) @@ -100,6 +102,= 7 @@ class SyncCommand(EdkrepoCommand): if not args.update_local_manifest: self.__check_for_new_manifest(args, config, initial_manifest, = workspace_path, global_manifest_directory) check_dirty_repos(initial_manifest, workspace_path) + # Determine if sparse checkout needs to be disabled for this opera= tion sparse_settings =3D initial_manifest.sparse_settings sparse_enabled =3D sparse_checkout_enabled(workspace_path, initial= _sources) @@ -113,7 +116,7 @@ class SyncCommand(EdkrepoCommand): reset_sparse_checkout(workspace_path, initial_sources) =20 # Get the latest manifest if requested - if args.update_local_manifest: #NOTE: hyphens in arg name replaced= with underscores due to argparse + if args.update_local_manifest: # NOTE: hyphens in arg name=20 + replaced with underscores due to argparse self.__update_local_manifest(args, config, initial_manifest, w= orkspace_path, global_manifest_directory) manifest =3D get_workspace_manifest() if args.update_local_manifest: @@ -127,6 +130,12 @@ class SyncCommand(EdkrepoCommand): repo_sources_to_sync =3D manifest.get_repo_sources(current_com= bo) manifest.write_current_combo(current_combo) =20 + # At this point both new and old manifest files are ready so we ca= n deinit any + # submodules that are removed due to a manifest update. + if not args.skip_submodule: + deinit_submodules(workspace_path, initial_manifest, initial_co= mbo, + manifest, current_combo, args.verbose) + sync_error =3D False # Calculate the hooks which need to be updated, added or removed f= or the sync if args.update_local_manifest: @@ -195,16 +204,16 @@ class SyncCommand(EdkrepoCommand): elif args.verbose: print(NO_SYNC_DETACHED_HEAD.format(repo_to_sync.root)) =20 - if not args.skip_submodule: - if repo_to_sync.enable_submodule: - # Perform submodule updates and url redirection - maintain_submodules(repo_to_sync, repo) # Update commit message templates update_repo_commit_template(workspace_path, repo, repo_to_sync= , config, source_global_manifest_repo) =20 if sync_error: print(SYNC_ERROR) =20 + # Initialize submodules + if not args.skip_submodule: + init_submodules(workspace_path, manifest, current_combo, + args.verbose) + # Restore sparse checkout state if sparse_enabled: print(SPARSE_CHECKOUT) @@ -326,7 +335,7 @@ class SyncCommand(EdkrepoCommand): print(path_to_source) if len(sources_to_remove) > 0: print(SYNC_REMOVE_LIST_END_FORMATTING) - clone_repos(args, workspace_path, sources_to_clone, new_manife= st_to_check.repo_hooks, config, args.skip_submodule, new_manifest_to_check) + clone_repos(args, workspace_path, sources_to_clone,=20 + new_manifest_to_check.repo_hooks, config, new_manifest_to_check) # Make a list of and only checkout repos that were newly clone= d. Sync keeps repos on their initial active branches # cloning the entire combo can prevent existing repos from cor= rectly being returned to their proper branch repos_to_checkout =3D [] diff --git a/edkrepo/common/common_repo_functions.py b/edkrepo/common/commo= n_repo_functions.py index 61133f1..18e545b 100644 --- a/edkrepo/common/common_repo_functions.py +++ b/edkrepo/common/common_repo_functions.py @@ -68,12 +68,13 @@ from edkrepo_manifest_parser.edk_manifest_validation im= port validate_manifestrep from edkrepo_manifest_parser.edk_manifest_valida= tion import get_manifest_validation_status from edkrepo_manifest_parser.ed= k_manifest_validation import print_manifest_errors from edkrepo_manifest_p= arser.edk_manifest_validation import validate_manifestfiles +from project_utils.submodule import deinit_submodules, init_submodules =20 CLEAR_LINE =3D '\x1b[K' DEFAULT_REMOTE_NAME =3D 'origin' PRIMARY_REMOTE_NAME =3D 'primary' =20 -def clone_repos(args, workspace_dir, repos_to_clone, project_client_side_h= ooks, config, skip_submodule, manifest): +def clone_repos(args, workspace_dir, repos_to_clone, project_client_side_h= ooks, config, manifest): for repo_to_clone in repos_to_clone: local_repo_path =3D os.path.join(workspace_dir, repo_to_clone.root= ) local_repo_url =3D repo_to_clone.remote_url @@ -108,10 +109,6 @@ d= ef clone_repos(args, workspace_dir, repos_to_clone, project_client_side_hoo= ks, else: raise EdkrepoManifestInvalidException(MISSING_BRANCH_COMMIT) =20 - if not skip_submodule: - if repo_to_clone.enable_submodule: - maintain_submodules(repo_to_clone, repo, args.verbose) - try: if 'source_manifest_repo' in vars(args).keys(): src_manifest_repo =3D find_source_manifest_repo(manifest, = config['cfg_file'], config['user_cfg_file'], args.source_manifest_repo) @@ = -184,32 +181,6 @@ def write_conditional_include(workspace_path, repo_source= s, included_configs): gitglobalconfig.add_section(section) gitglobalconfig.set(section, 'path', path) =20 -def maintain_submodules(repo_sources, repo, verbose =3D False): - try: - output_data =3D repo.git.execute(['git', 'submodule', 'init'], wit= h_extended_output=3DTrue, with_stdout=3DTrue) - if verbose and output_data[0]: - print(output_data[0]) - if output_data[1]: - print(output_data[1]) - if verbose and output_data[2]: - print(output_data[2]) - output_data =3D repo.git.execute(['git', 'submodule', 'sync', '--r= ecursive'], with_extended_output=3DTrue, with_stdout=3DTrue) - if verbose and output_data[0]: - print(output_data[0]) - if output_data[1]: - print(output_data[1]) - if verbose and output_data[2]: - print(output_data[2]) - output_data =3D repo.git.execute(['git', 'submodule', 'update', '-= -recursive'], with_extended_output=3DTrue, with_stdout=3DTrue) - if verbose and output_data[0]: - print(output_data[0]) - if output_data[1]: - print(output_data[1]) - if verbose and output_data[2]: - print(output_data[2]) - except: - raise EdkrepoGitException(SUBMODULE_FAILURE.format(repo_sources.re= mote_name)) - def install_hooks(hooks, local_repo_path, repo_for_install, config, global= _manifest_directory): # Determine the which hooks are for the repo in question and which are= from a URL based source or are in a global # manifest repo relative path @@ -399,9 +370,6 @@ def checkout_repos(verbose, override, repos_to_checkout= , workspace_path, manifes else: raise EdkrepoManifestInvalidException(MISSING_BRANCH_COMMIT) =20 - if repo_to_checkout.enable_submodule: - maintain_submodules(repo_to_checkout, repo, verbose) - def validate_manifest_repo(manifest_repo, verbose=3DFalse, archived=3DFals= e): print(VERIFY_GLOBAL) if archived: @@ -514,9 +482,11 @@ def checkout(combination_or_sha, verbose=3DFalse, over= ride=3DFalse, log=3DNone): # Create combo_or_sha so we have original input and do not introduce a= ny # unintended behavior by messing with parameters. combo_or_sha =3D combination_or_sha + submodule_combo =3D manifest.general_config.current_combo try: # Try to handle normalize combo name to match the manifest file. combo_or_sha =3D case_insensitive_single_match(combo_or_sha, combi= nations_in_manifest(manifest)) + submodule_combo =3D combo_or_sha except: # No match so leave it alone. It must be a SHA1 or a bad combo na= me. pass @@ -528,6 +498,10 @@ def checkout(combination_or_sha, verbose=3DFalse, over= ride=3DFalse, log=3DNone): log=3Dlog) initial_repo_sources =3D manifest.get_repo_sources(manifest.general_co= nfig.current_combo) =20 + # Deinit any submodules that have been removed. + deinit_submodules(workspace_path, manifest, manifest.general_config.cu= rrent_combo, + manifest, submodule_combo, verbose) + # Disable sparse checkout current_repos =3D initial_repo_sources sparse_enabled =3D sparse_checkout_enabled(workspace_path, initial_rep= o_sources) @@ -566,6 +540,7 @@ def checkout(combination_or_sha, verbose=3DF= alse, override=3DFalse, log=3DNone): # Return to the initial combo, since there was an issue with cheki= ng out the selected combo checkout_repos(verbose, override, initial_repo_sources, workspace_= path, manifest) finally: + init_submodules(workspace_path, manifest, submodule_combo, + verbose) if sparse_enabled or sparse_diff: print(SPARSE_CHECKOUT) sparse_checkout(workspace_path, current_repos, manifest) -- 2.21.0.windows.1