[edk2-staging/EdkRepo] [PATCH 0/2] EdkRepo: Update Clone Command

[edk2-staging/EdkRepo] [PATCH 0/2] EdkRepo: Update Clone Command

[Ashley E Desimone]

Update the clone command to support multiple manifest repositories
and add verify_single_manifest to perform validation of a
single manifest file in a single manifest repository.

Signed-off-by: Ashley E Desimone <ashley.e.desimone@intel.com>
Cc: Nate DeSimone <nathaniel.l.desimone@intel.com>
Cc: Puja Pandya <puja.pandya@intel.com>
Cc: Erik Bjorge <erik.c.bjorge@intel.com>
Cc: Bret Barkelew <Bret.Barkelew@microsoft.com>
Cc: Prince Agyeman <prince.agyeman@intel.com>

Ashley E Desimone (2):
  EdkRepo: Add verify_single_manifest()
  EdkRepo: Add multiple manifest repository support to clone

 edkrepo/commands/clone_command.py       | 36 +++++++++++++++++++--------------
 edkrepo/common/common_repo_functions.py | 11 ++++++++++
 2 files changed, 32 insertions(+), 15 deletions(-)

-- 
2.16.2.windows.1

[edk2-staging/EdkRepo] [PATCH 1/2] EdkRepo: Add verify_single_manifest()

[Ashley E Desimone]

Signed-off-by: Ashley E Desimone <ashley.e.desimone@intel.com>
Cc: Nate DeSimone <nathaniel.l.desimone@intel.com>
Cc: Puja Pandya <puja.pandya@intel.com>
Cc: Erik Bjorge <erik.c.bjorge@intel.com>
Cc: Bret Barkelew <Bret.Barkelew@microsoft.com>
Cc: Prince Agyeman <prince.agyeman@intel.com>
---
 edkrepo/common/common_repo_functions.py | 11 +++++++++++
 1 file changed, 11 insertions(+)

diff --git a/edkrepo/common/common_repo_functions.py b/edkrepo/common/common_repo_functions.py
index 485aa26..20dcb8a 100644
--- a/edkrepo/common/common_repo_functions.py
+++ b/edkrepo/common/common_repo_functions.py
@@ -426,6 +426,17 @@ def verify_manifest_data(global_manifest_directory, config, verbose=False, verif
             if verbose:
                 print_manifest_errors(manifestfile_validation_data)
 
+def verify_single_manifest(cfg_file, manifest_repo, manifest_path, verbose=False):
+    manifest = ManifestXml(manifest_path)
+    print(VERIFY_PROJ.format(manifest.project_info.codename))
+    index_path = os.path.join(cfg_file.manifest_repo_abs_path(manifest_repo), CI_INDEX_FILE_NAME)
+    proj_val_data = validate_manifestfiles(index_path, [manifest_path])
+    proj_val_error = get_manifest_validation_status(proj_val_data)
+    if proj_val_error:
+        if verbose:
+            print_manifest_errors(proj_val_data)
+        raise EdkrepoManifestInvalidException(VERIFY_PROJ_FAIL.format(manifest.project_info.codename))
+
 def sort_commits(manifest, workspace_path, max_commits=None):
     colorama.init()
     repo_sources_to_log = manifest.get_repo_sources(manifest.general_config.current_combo)
-- 
2.16.2.windows.1

[edk2-staging/EdkRepo] [PATCH 2/2] EdkRepo: Add multiple manifest repository support to clone

[Ashley E Desimone]

Update the clone command to support searching for the project
to be cloned in all manifest repositories defined within both
the edkrepo.cfg and the edkrepo_user.cfg files.

Signed-off-by: Ashley E Desimone <ashley.e.desimone@intel.com>
Cc: Nate DeSimone <nathaniel.l.desimone@intel.com>
Cc: Puja Pandya <puja.pandya@intel.com>
Cc: Erik Bjorge <erik.c.bjorge@intel.com>
Cc: Bret Barkelew <Bret.Barkelew@microsoft.com>
Cc: Prince Agyeman <prince.agyeman@intel.com>
---
 edkrepo/commands/clone_command.py | 36 +++++++++++++++++++++---------------
 1 file changed, 21 insertions(+), 15 deletions(-)

diff --git a/edkrepo/commands/clone_command.py b/edkrepo/commands/clone_command.py
index cd65fe6..b7f3158 100644
--- a/edkrepo/commands/clone_command.py
+++ b/edkrepo/commands/clone_command.py
@@ -11,16 +11,18 @@ import os
 import shutil
 
 from edkrepo.commands.edkrepo_command import EdkrepoCommand
-from edkrepo.commands.edkrepo_command import SubmoduleSkipArgument
+from edkrepo.commands.edkrepo_command import SubmoduleSkipArgument, SourceManifestRepoArgument
 import edkrepo.commands.arguments.clone_args as arguments
-from edkrepo.common.common_repo_functions import pull_latest_manifest_repo, clone_repos, sparse_checkout, verify_manifest_data
-from edkrepo.common.common_repo_functions import update_editor_config
+from edkrepo.common.common_repo_functions import clone_repos, sparse_checkout, verify_single_manifest
+from edkrepo.common.common_repo_functions import update_editor_config, combinations_in_manifest
 from edkrepo.common.common_repo_functions import write_included_config, write_conditional_include
-from edkrepo.common.common_repo_functions import find_project_in_index, combinations_in_manifest
 from edkrepo.common.edkrepo_exception import EdkrepoInvalidParametersException, EdkrepoManifestInvalidException
 from edkrepo.common.humble import CLONE_INVALID_WORKSPACE, CLONE_INVALID_PROJECT_ARG, CLONE_INVALID_COMBO_ARG
 from edkrepo.common.humble import SPARSE_CHECKOUT, CLONE_INVALID_LOCAL_ROOTS
 from edkrepo.common.workspace_maintenance.workspace_maintenance import case_insensitive_single_match
+from edkrepo.common.workspace_maintenance.manifest_repos_maintenance import pull_all_manifest_repos, find_project_in_all_indices
+from edkrepo.common.workspace_maintenance.manifest_repos_maintenance import list_available_manifest_repos
+from edkrepo.common.workspace_maintenance.humble.manifest_repos_maintenance_humble import PROJ_NOT_IN_REPO, SOURCE_MANIFEST_REPO_NOT_FOUND
 from edkrepo_manifest_parser.edk_manifest import CiIndexXml, ManifestXml
 
 
@@ -58,12 +60,14 @@ class CloneCommand(EdkrepoCommand):
                      'required': False,
                      'help-text': arguments.NO_SPARSE_HELP})
         args.append(SubmoduleSkipArgument)
+        args.append(SourceManifestRepoArgument)
         return metadata
 
 
     def run_command(self, args, config):
-        pull_latest_manifest_repo(args, config)
+        pull_all_manifest_repos(config['cfg_file'], config['user_cfg_file'], False)
         update_editor_config(config)
+
         name_or_manifest = args.ProjectNameOrManifestFile
         workspace_dir = args.Workspace
         # Check to see if requested workspace exists. If not create it. If so check for empty
@@ -77,17 +81,19 @@ class CloneCommand(EdkrepoCommand):
         if not os.path.isdir(workspace_dir):
             os.makedirs(workspace_dir)
 
-        # Get path to global manifest file
-        global_manifest_directory = config['cfg_file'].manifest_repo_abs_local_path
-
-        # Verify the manifest directory at this point.
-        verify_manifest_data(global_manifest_directory, config, verbose=args.verbose, verify_proj=name_or_manifest)
-
-        # Now try to find the correct manifest file.
-        index_path = os.path.join(global_manifest_directory, 'CiIndex.xml')
-        ci_index_xml = CiIndexXml(index_path)
+        cfg, user_cfg, conflicts = list_available_manifest_repos(config['cfg_file'], config['user_cfg_file'])
+        manifest_repo, source_cfg, global_manifest_path = find_project_in_all_indices(args.ProjectNameOrManifestFile,
+                                                                  config['cfg_file'],
+                                                                  config['user_cfg_file'],
+                                                                  PROJ_NOT_IN_REPO.format(args.ProjectNameOrManifestFile),
+                                                                  SOURCE_MANIFEST_REPO_NOT_FOUND.format(args.ProjectNameOrManifestFile),
+                                                                  args.source_manifest_repo)
 
-        global_manifest_path = find_project_in_index(name_or_manifest, ci_index_xml, global_manifest_directory, CLONE_INVALID_PROJECT_ARG)
+        # If this manifest is in a defined manifest repository validate the manifest within the manifest repo
+        if manifest_repo in cfg:
+            verify_single_manifest(config['cfg_file'], manifest_repo, global_manifest_path)
+        elif manifest_repo in user_cfg:
+            verify_single_manifest(config['user_cfg_file'], manifest_repo, global_manifest_path)
 
         # Copy project manifest to local manifest dir and rename it Manifest.xml.
         local_manifest_dir = os.path.join(workspace_dir, "repo")
-- 
2.16.2.windows.1

Re: [edk2-devel] [edk2-staging/EdkRepo] [PATCH 0/2] EdkRepo: Update Clone Command

[Nate DeSimone]

For the series...

Reviewed-by: Nate DeSimone <nathaniel.l.desimone@intel.com>

> -----Original Message-----
> From: devel@edk2.groups.io <devel@edk2.groups.io> On Behalf Of Ashley E
> Desimone
> Sent: Friday, May 8, 2020 4:47 PM
> To: devel@edk2.groups.io
> Cc: Desimone, Nathaniel L <nathaniel.l.desimone@intel.com>; Pandya, Puja
> <puja.pandya@intel.com>; Bjorge, Erik C <erik.c.bjorge@intel.com>; Bret
> Barkelew <Bret.Barkelew@microsoft.com>; Agyeman, Prince
> <prince.agyeman@intel.com>
> Subject: [edk2-devel] [edk2-staging/EdkRepo] [PATCH 0/2] EdkRepo: Update
> Clone Command
> 
> Update the clone command to support multiple manifest repositories and
> add verify_single_manifest to perform validation of a single manifest file in a
> single manifest repository.
> 
> Signed-off-by: Ashley E Desimone <ashley.e.desimone@intel.com>
> Cc: Nate DeSimone <nathaniel.l.desimone@intel.com>
> Cc: Puja Pandya <puja.pandya@intel.com>
> Cc: Erik Bjorge <erik.c.bjorge@intel.com>
> Cc: Bret Barkelew <Bret.Barkelew@microsoft.com>
> Cc: Prince Agyeman <prince.agyeman@intel.com>
> 
> Ashley E Desimone (2):
>   EdkRepo: Add verify_single_manifest()
>   EdkRepo: Add multiple manifest repository support to clone
> 
>  edkrepo/commands/clone_command.py       | 36 +++++++++++++++++++---
> -----------
>  edkrepo/common/common_repo_functions.py | 11 ++++++++++
>  2 files changed, 32 insertions(+), 15 deletions(-)
> 
> --
> 2.16.2.windows.1
> 
> 
> 

Re: [edk2-devel] [edk2-staging/EdkRepo] [PATCH 0/2] EdkRepo: Update Clone Command

[Nate DeSimone]

Pushed: e14aaf17~..1134a30b

> -----Original Message-----
> From: devel@edk2.groups.io <devel@edk2.groups.io> On Behalf Of Ashley E
> Desimone
> Sent: Friday, May 8, 2020 4:47 PM
> To: devel@edk2.groups.io
> Cc: Desimone, Nathaniel L <nathaniel.l.desimone@intel.com>; Pandya, Puja
> <puja.pandya@intel.com>; Bjorge, Erik C <erik.c.bjorge@intel.com>; Bret
> Barkelew <Bret.Barkelew@microsoft.com>; Agyeman, Prince
> <prince.agyeman@intel.com>
> Subject: [edk2-devel] [edk2-staging/EdkRepo] [PATCH 0/2] EdkRepo: Update
> Clone Command
> 
> Update the clone command to support multiple manifest repositories and
> add verify_single_manifest to perform validation of a single manifest file in a
> single manifest repository.
> 
> Signed-off-by: Ashley E Desimone <ashley.e.desimone@intel.com>
> Cc: Nate DeSimone <nathaniel.l.desimone@intel.com>
> Cc: Puja Pandya <puja.pandya@intel.com>
> Cc: Erik Bjorge <erik.c.bjorge@intel.com>
> Cc: Bret Barkelew <Bret.Barkelew@microsoft.com>
> Cc: Prince Agyeman <prince.agyeman@intel.com>
> 
> Ashley E Desimone (2):
>   EdkRepo: Add verify_single_manifest()
>   EdkRepo: Add multiple manifest repository support to clone
> 
>  edkrepo/commands/clone_command.py       | 36 +++++++++++++++++++---
> -----------
>  edkrepo/common/common_repo_functions.py | 11 ++++++++++
>  2 files changed, 32 insertions(+), 15 deletions(-)
> 
> --
> 2.16.2.windows.1
> 
> 
>