From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mga11.intel.com (mga11.intel.com [192.55.52.93]) by mx.groups.io with SMTP id smtpd.web12.2746.1592956411106840206 for ; Tue, 23 Jun 2020 16:53:31 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=@intel.onmicrosoft.com header.s=selector2-intel-onmicrosoft-com header.b=ZGws+IPD; spf=pass (domain: intel.com, ip: 192.55.52.93, mailfrom: nathaniel.l.desimone@intel.com) IronPort-SDR: z3C1z0gxFIgWNDQnHE9pQCoNFHV9zYrLmIzRl0TEwA3bo1ch3oOi9ULrg+CiXKC4QOdm1DO5CO RLMVj11RPvPQ== X-IronPort-AV: E=McAfee;i="6000,8403,9661"; a="142494255" X-IronPort-AV: E=Sophos;i="5.75,272,1589266800"; d="scan'208";a="142494255" X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga005.fm.intel.com ([10.253.24.32]) by fmsmga102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 23 Jun 2020 16:53:30 -0700 IronPort-SDR: 1hSeLvzL3Q6IpM41oWxipUdIGrcuALKUIUILhcvKXgL9+MpzfFtp2Fy8DSMiHmx9IGjVuE/SKX ii6L9gi6CnDg== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.75,272,1589266800"; d="scan'208";a="479061991" Received: from orsmsx103.amr.corp.intel.com ([10.22.225.130]) by fmsmga005.fm.intel.com with ESMTP; 23 Jun 2020 16:53:30 -0700 Received: from orsmsx161.amr.corp.intel.com (10.22.240.84) by ORSMSX103.amr.corp.intel.com (10.22.225.130) with Microsoft SMTP Server (TLS) id 14.3.439.0; Tue, 23 Jun 2020 16:53:30 -0700 Received: from ORSEDG001.ED.cps.intel.com (10.7.248.4) by ORSMSX161.amr.corp.intel.com (10.22.240.84) with Microsoft SMTP Server (TLS) id 14.3.439.0; Tue, 23 Jun 2020 16:53:30 -0700 Received: from NAM10-BN7-obe.outbound.protection.outlook.com (104.47.70.106) by edgegateway.intel.com (134.134.137.100) with Microsoft SMTP Server (TLS) id 14.3.439.0; Tue, 23 Jun 2020 16:53:29 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=aSzkLgHGXz7gYxsk/2AjM44VJ1/cfLufCClfAvng+JcOGjISv9dESXujZLLuYj8Dheh5cKnN8zbRo30yMW/zA1uWQBY3ySRwfWJaVnBogPj3vSEcwCAZl/7yzBXsC0vjn0+YMuo5dMFyPjMAsmlkPr6xYI1EGvL3iyRG640Eg46+DK7LW7KxZhL7p5wAB4EaZY9Pa7cshgMLU4v9SDzpRw/j2iX9s7yxCOHBhrRyt4VI3zndALCFxFyJKTvz58mSvEs1wbHxcaWCSYMuTTCn5pVlPYBZVk/8japz3zHP8BBv3+YIIz1QmPSU/86lSJl5qSSt2PwQqqTG68Zv4tWs7g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=YmRmBwc8cRMQHELEdNnNcK1FZQ+5opJrhy2nfiiwHC4=; b=Q1k1201PZaa+PBM3hUUtiTlgI23sLJ8gnZPhjwhoGUaMWUAeZlTLYGTfZY8vrId+ZA+gFB+5RdsWlfUO3KubUbTqj5q1x+9urTbbMJ0UFIbBR9tyky4Di5MSrHDxvugs4laQF4yWehpesCtygqWtVnT2NN4L6JvC0UTxmjlAGSlhsxKskd3AJvyWVZE0yOeTTHgAizW3b+esfvj/JOleZ6/IP5SxixHHX6UaL5VB/iN8qivpn3dlS6I1ybjpWlS6U0lIXz/Ex2zZt/Bc/o2HNCRBfCBkTyvR3x2a9FOR2fiPtEgGQOdXiLJyavqI4wqsWpEiqsq5nedFZkoYnmvJGQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com; dkim=pass header.d=intel.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=intel.onmicrosoft.com; s=selector2-intel-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=YmRmBwc8cRMQHELEdNnNcK1FZQ+5opJrhy2nfiiwHC4=; b=ZGws+IPD3RcMrOGNcurRuijVJ8LnvWSHzZrH79sE8V1flZZz382PbNbsDAjzTXVL2RUsTAGyQqpT4oHR1a2adwPF+WlY89xBeNJ0ar3hwfgOnSi5Ci4tnnUZd9qe8mkO95EUQHWE3i8MQum/HNbYfxVhh4URp+ZWisKOmoZtG1Y= Received: from MWHPR1101MB2160.namprd11.prod.outlook.com (2603:10b6:301:5b::12) by MW3PR11MB4746.namprd11.prod.outlook.com (2603:10b6:303:5f::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3109.22; Tue, 23 Jun 2020 23:53:28 +0000 Received: from MWHPR1101MB2160.namprd11.prod.outlook.com ([fe80::889c:c0bd:5262:7c80]) by MWHPR1101MB2160.namprd11.prod.outlook.com ([fe80::889c:c0bd:5262:7c80%8]) with mapi id 15.20.3131.020; Tue, 23 Jun 2020 23:53:28 +0000 From: "Nate DeSimone" To: "Bjorge, Erik C" , "Desimone, Ashley E" , "devel@edk2.groups.io" CC: "Pandya, Puja" , Bret Barkelew , "Agyeman, Prince" Subject: Re: [edk2-staging/EdkRepo] [PATCH] EdkRepo: EdkRepo Sync Fails to Update Local Manifest Thread-Topic: [edk2-staging/EdkRepo] [PATCH] EdkRepo: EdkRepo Sync Fails to Update Local Manifest Thread-Index: AQHWRpH32HhielTAxUu1aSF+sq6akajk3aOAgAIHzNA= Date: Tue, 23 Jun 2020 23:53:27 +0000 Message-ID: References: <20200619233247.2184-1-ashley.e.desimone@intel.com> In-Reply-To: Accept-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-titus-metadata-40: eyJDYXRlZ29yeUxhYmVscyI6IiIsIk1ldGFkYXRhIjp7Im5zIjoiaHR0cDpcL1wvd3d3LnRpdHVzLmNvbVwvbnNcL0ludGVsMyIsImlkIjoiZTliOTI5M2UtN2Y2Yi00YjIwLWI4NTQtOWJlNjhmZDY5MTYxIiwicHJvcHMiOlt7Im4iOiJDVFBDbGFzc2lmaWNhdGlvbiIsInZhbHMiOlt7InZhbHVlIjoiQ1RQX05UIn1dfV19LCJTdWJqZWN0TGFiZWxzIjpbXSwiVE1DVmVyc2lvbiI6IjE3LjEwLjE4MDQuNDkiLCJUcnVzdGVkTGFiZWxIYXNoIjoiYVNPTU83clBWdHhMMlZzVjNHSm4wdUQxZ1NhTjhpWkMrKytpdWtoekZDNzNiTlBGdm56T1dTZzh3VGE3dGhZeiJ9 dlp-reaction: no-action dlp-version: 11.2.0.6 dlp-product: dlpe-windows x-ctpclassification: CTP_NT authentication-results: intel.com; dkim=none (message not signed) header.d=none;intel.com; dmarc=none action=none header.from=intel.com; x-originating-ip: [50.53.190.176] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: da3fbbf5-074c-42d8-1fea-08d817d0a02f x-ms-traffictypediagnostic: MW3PR11MB4746: x-ms-exchange-transport-forked: True x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:2276; x-forefront-prvs: 04433051BF x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: /JsxJKGVgtRoaHV0dTJEWzNLgO8w5VKlJqGX26TRV9BaB+dLzqweyXuzG6LaW8B8ddq9nipC5S0HbB4Aw3nQ1eA7pPuIruak9sP+c/iYf3jGM2+C55j/6TtMD+o+OVoBEAy1OSE0ul5NMtVsnnHjbwAupSJs+xbIgPye022cuKYbcsEEKp2MaSw5D+J5AUB1BCPXQprvu2+CuPT3GLybbpkUxUbI4dnxl+v8sAog+2IaC8COZ/Zb285kcPVih/YAisHPIeIeS8KqYk9GM4XBAHTzQ/femRZX176C9u1NEiEFcBG6e+3d1Oaki45kSFrDczObWS09SCUh3d01ugLPuA== x-forefront-antispam-report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:MWHPR1101MB2160.namprd11.prod.outlook.com;PTR:;CAT:NONE;SFTY:;SFS:(4636009)(39860400002)(366004)(346002)(376002)(396003)(136003)(66446008)(71200400001)(33656002)(4326008)(66476007)(15650500001)(316002)(110136005)(478600001)(54906003)(83380400001)(107886003)(7696005)(52536014)(9686003)(64756008)(53546011)(6506007)(26005)(55016002)(5660300002)(8936002)(66556008)(186003)(66946007)(76116006)(8676002)(2906002)(86362001);DIR:OUT;SFP:1102; x-ms-exchange-antispam-messagedata: rti4+nfJyTm7/IvSSXpLo2vj49xL8xRsEC0kP4eiGTeTx8c2yRLbipe2ToHpydeqEg8RQqXijkRi8axoZYdx93CuD7gPlcUyP8JlUoKZQxGESbQlboDZXIRQr90nU8y8TqKwvmQyhsWGdLQhDt5vCPKokQu5CMSIGS1ng8gsawDboNZrKGLnDG1U7YKN9Lz1ZJh2iDnpVEFv329UGwBSClGrgJqSum6pwIUPAzB6IxaGJ9DmSuxA/kYt8HKk8zFZsMhKomwNoKC4PVTjnuPf4HdF/2bPDktyympOUjwXlgUvuntLTcvyVl57LgaA9C6jA5uTTkpBUgTe9EhmUMyH466UONS4ZbGCagMAncmyt73cCkeKlEPMeKt8WlJd1eSeItXSfpcUTrvJ5NUVAleOID4jE7Qzxm/BxlsCBjG1znLwiExiIW9ZloDVk8IWOUOOo/muPd8AimtkRoIpuNyDhRKJksz7Hzp/tqVLW/XcwSBBsDeck2tQE+uH/AxZh/J4 MIME-Version: 1.0 X-MS-Exchange-CrossTenant-Network-Message-Id: da3fbbf5-074c-42d8-1fea-08d817d0a02f X-MS-Exchange-CrossTenant-originalarrivaltime: 23 Jun 2020 23:53:27.9115 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 46c98d88-e344-4ed4-8496-4ed7712e255d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: OAisRbMLl3LSJTP3l0XflKiGiWxCem6tCIIVwav4/nRKZeRdQ3s+BYeg/blf2GVwSW/Gn/Rghor0mIaXvAUZUOqLXc09W6PZyzad2tehtXE= X-MS-Exchange-Transport-CrossTenantHeadersStamped: MW3PR11MB4746 Return-Path: nathaniel.l.desimone@intel.com X-OriginatorOrg: intel.com Content-Language: en-US Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable For files that we don't execute via shell #!, CRLF should be fine. However,= please take care as some of the .py files are launched that way on *nix sy= stems. Thanks, Nate -----Original Message----- From: Bjorge, Erik C =20 Sent: Monday, June 22, 2020 9:52 AM To: Desimone, Ashley E ; devel@edk2.groups.io Cc: Desimone, Nathaniel L ; Pandya, Puja ; Bret Barkelew ; Agyeman= , Prince Subject: RE: [edk2-staging/EdkRepo] [PATCH] EdkRepo: EdkRepo Sync Fails to = Update Local Manifest It seems when patches are being applied we are getting mixed line endings. = This is making the patches a bit more difficult to review. We may want to= look into fixing this. The quick fix would be to convert all Python files= to CRLF. Reviewed-by: Erik Bjorge -----Original Message----- From: Desimone, Ashley E Sent: Friday, June 19, 2020 4:33 PM To: devel@edk2.groups.io Cc: Desimone, Nathaniel L ; Pandya, Puja ; Bjorge, Erik C ; Bret Barke= lew ; Agyeman, Prince Subject: [edk2-staging/EdkRepo] [PATCH] EdkRepo: EdkRepo Sync Fails to Upda= te Local Manifest Only use the new manifests default combination if the current manifest is n= ot present in the list of new combos Remove the extranous comparison of repo sources as that content is checked = later in the function. Signed-off-by: Ashley E Desimone Cc: Nate DeSimone Cc: Puja Pandya Cc: Erik Bjorge Cc: Bret Barkelew Cc: Prince Agyeman --- edkrepo/commands/sync_command.py | 45 ++++++++++++++------------------ 1 file changed, 20 insertions(+), 25 deletions(-) diff --git a/edkrepo/commands/sync_command.py b/edkrepo/commands/sync_comma= nd.py index dd7202d..c87861b 100644 --- a/edkrepo/commands/sync_command.py +++ b/edkrepo/commands/sync_command.py @@ -38,7 +38,7 @@ from edkrepo.common.common_repo_functions import checkout= _repos, check_dirty_rep from edkrepo.common.common_repo_functions import u= pdate_editor_config from edkrepo.common.common_repo_functions import updat= e_repo_commit_template, get_latest_sha from edkrepo.common.common_repo_fun= ctions import has_primary_repo_remote, fetch_from_primary_repo, in_sync_wit= h_primary -from edkrepo.common.common_repo_functions import update_hooks, c= ombinations_in_manifest+from edkrepo.common.common_repo_functions import up= date_hooks, combinations_in_manifest from edkrepo.common.common_repo_funct= ions import write_included_config, remove_included_config from edkrepo.com= mon.workspace_maintenance.workspace_maintenance import generate_name_for_ob= solete_backup from edkrepo.common.workspace_maintenance.manifest_repos_mai= ntenance import pull_workspace_manifest_repo @@ -48,7 +48,7 @@ from edkrepo= .common.ui_functions import init_color_console from edkrepo.config.config_= factory import get_workspace_path, get_workspace_manifest, get_edkrepo_glob= al_data_directory from edkrepo.config.config_factory import get_workspace_= manifest_file from edkrepo_manifest_parser.edk_manifest import CiIndexXml,= ManifestXml -from project_utils.submodule import deinit_submodules, mainta= in_submodules+from project_utils.submodule import deinit_submodules, mainta= in_submodules =20 =20 class SyncCommand(EdkrepoCommand): @@ -85,7 +85,7 @@ class SyncCommand(EdkrepoCommand): current_combo =3D initial_manifest.general_config.current_combo initial_sources =3D initial_manifest.get_repo_sources(current_comb= o) initial_hooks =3D initial_manifest.repo_hooks - initial_combo =3D current_combo+ initial_combo =3D current_= combo =20 source_global_manifest_repo =3D find_source_manifest_repo(initial_= manifest, config['cfg_file'], config['user_cfg_file'], args.source_manifest= _repo) pull_workspace_manifest_repo(initial_manifest, config['cfg_file'],= config['user_cfg_file'], args.source_manifest_repo, False) @@ -102,7 +102,= 7 @@ class SyncCommand(EdkrepoCommand): if not args.update_local_manifest: self.__check_for_new_manifest(args, config, initial_manifest, = workspace_path, global_manifest_directory) check_dirty_repos(initial_manifest, workspace_path) -+ # Determine if sparse checkout needs to be disabled for this opera= tion sparse_settings =3D initial_manifest.sparse_settings sparse_enabled =3D sparse_checkout_enabled(workspace_path, initial= _sources) @@ -116,7 +116,7 @@ class SyncCommand(EdkrepoCommand): reset_sparse_checkout(workspace_path, initial_sources) =20 # Get the latest manifest if requested - if args.update_local_manifest: # NOTE: hyphens in arg name replac= ed with underscores due to argparse+ if args.update_local_manifest: = # NOTE: hyphens in arg name replaced with underscores due to argparse self.__update_local_manifest(args, config, initial_manifest, w= orkspace_path, global_manifest_directory) manifest =3D get_workspace_manifest() if args.update_local_manifest: @@ -130,12 +130,12 @@ class SyncCommand(EdkrepoCommand): repo_sources_to_sync =3D manifest.get_repo_sources(current_com= bo) manifest.write_current_combo(current_combo) =20 - # At this point both new and old manifest files are ready so we ca= n deinit any- # submodules that are removed due to a manifest update= .- if not args.skip_submodule:- deinit_submodules(workspa= ce_path, initial_manifest, initial_combo,- man= ifest, current_combo, args.verbose)-+ # At this point both new and o= ld manifest files are ready so we can deinit any + # submodules that are removed due to a manifest update. + if not args.skip_submodule: + deinit_submodules(workspace_path, initial_manifest, initial_co= mbo, + manifest, current_combo, args.verbose) + sync_error =3D False # Calculate the hooks which need to be updated, added or removed f= or the sync if args.update_local_manifest: @@ -205,15 +205,15 @@ class SyncCommand(EdkrepoCommand): print(NO_SYNC_DETACHED_HEAD.format(repo_to_sync.root)) =20 # Update commit message templates - update_repo_commit_template(workspace_path, repo, repo_to_sync= , config, global_manifest_directory)+ update_repo_commit_templat= e(workspace_path, repo, repo_to_sync, config, global_manifest_directory) =20 if sync_error: print(SYNC_ERROR) =20 - # Initialize submodules- if not args.skip_submodule:- = maintain_submodules(workspace_path, manifest, current_combo, args.ver= bose)-+ # Initialize submodules + if not args.skip_submodule: + maintain_submodules(workspace_path, manifest,=20 + current_combo, args.verbose) + # Restore sparse checkout state if sparse_enabled: print(SPARSE_CHECKOUT) @@ -244,14 +244,9 @@ class SyncCommand(EdkrepoCommand): # the default combo initial_combos =3D combinations_in_manifest(initial_manifest) new_combos =3D combinations_in_manifest(new_manifest_to_check) - if (current_combo not in new_combos) or (set(new_combos) !=3D set(= initial_combos)): - if initial_manifest.get_repo_sources(current_combo) =3D=3D new= _manifest_to_check.get_repo_sources(new_manifest_to_check.general_config.de= fault_combo): - new_sources_for_current_combo =3D new_manifest_to_check.ge= t_repo_sources(new_manifest_to_check.general_config.default_combo) - new_sources =3D new_sources_for_current_combo - else: - # Since asymetric combinations are not supported error out= with an IMCOMPATIBLE_COMBO warning - print(SYNC_COMBO_CHANGE.format(current_combo, initial_mani= fest.project_info.codename)) - raise EdkrepoManifestChangedException(SYNC_INCOMPATIBLE_CO= MBO) + if current_combo not in new_combos: + new_sources_for_current_combo =3D new_manifest_to_check.get_re= po_sources(new_manifest_to_check.general_config.default_combo) + new_sources =3D new_sources_for_current_combo else: new_sources_for_current_combo =3D new_manifest_to_check.get_re= po_sources(current_combo) new_sources =3D new_manifest_to_check.get_repo_sources(current= _combo) @@ -335,7 +330,7 @@ class SyncCommand(EdkrepoCommand): print(path_to_source) if len(sources_to_remove) > 0: print(SYNC_REMOVE_LIST_END_FORMATTING) - clone_repos(args, workspace_path, sources_to_clone, new_manife= st_to_check.repo_hooks, config, new_manifest_to_check)+ clone_re= pos(args, workspace_path, sources_to_clone, new_manifest_to_check.repo_hook= s, config, new_manifest_to_check) # Make a list of and only checkout repos that were newly clone= d. Sync keeps repos on their initial active branches # cloning the entire combo can prevent existing repos from cor= rectly being returned to their proper branch repos_to_checkout =3D [] -- 2.26.2.windows.1