Re: [PATCH] Platform/Intel/Tools: Add scripts to set PACKAGES_PATH environment

["Ni, Ray" <ray.ni@intel.com>]

Reviewed-by: Ray Ni <ray.ni@intel.com>

> -----Original Message-----
> From: Luo, Heng <heng.luo@intel.com>
> Sent: Friday, April 10, 2020 9:06 AM
> To: devel@edk2.groups.io
> Cc: Ni, Ray <ray.ni@intel.com>; Dong, Eric <eric.dong@intel.com>; Gao, Liming <liming.gao@intel.com>; Feng, Bob C
> <bob.c.feng@intel.com>; Chan, Amy <amy.chan@intel.com>
> Subject: [PATCH] Platform/Intel/Tools: Add scripts to set PACKAGES_PATH environment
> 
> Ref: https://bugzilla.tianocore.org/show_bug.cgi?id=2656
> 
> 1. Add GetPackagesPath.py, it will be used to get package pathes from
>   special directories. A sub directory is a qualified package path
>   when an EDKII Package can be found under it.
> 2. Add AppendPackagesPath.bat and AppendPackagesPath.sh, these scripts
>   call GetPackagesPath.py to collect all package paths under specified
>   directories and append them to PACKAGES_PATH environment variable.
> 
> Cc: Ray Ni <ray.ni@intel.com>
> Cc: Eric Dong <eric.dong@intel.com>
> Cc: Liming Gao <liming.gao@intel.com>
> Cc: Bob Feng <bob.c.feng@intel.com>
> Cc: Amy Chan <amy.chan@intel.com>
> Signed-off-by: Heng Luo <heng.luo@intel.com>
> ---
>  Platform/Intel/Tools/AppendPackagesPath/AppendPackagesPath.bat | 33 +++++++++++++++++++++++++++++++++
>  Platform/Intel/Tools/AppendPackagesPath/AppendPackagesPath.sh  | 42
> ++++++++++++++++++++++++++++++++++++++++++
>  Platform/Intel/Tools/AppendPackagesPath/GetPackagesPath.py     | 98
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>  Platform/Intel/Tools/AppendPackagesPath/Readme.md              | 28 ++++++++++++++++++++++++++++
>  4 files changed, 201 insertions(+)
> 
> diff --git a/Platform/Intel/Tools/AppendPackagesPath/AppendPackagesPath.bat
> b/Platform/Intel/Tools/AppendPackagesPath/AppendPackagesPath.bat
> new file mode 100644
> index 0000000000..da15d9c451
> --- /dev/null
> +++ b/Platform/Intel/Tools/AppendPackagesPath/AppendPackagesPath.bat
> @@ -0,0 +1,33 @@
> +@REM @file
> 
> +@REM Windows batch file to set PACKAGES_PATH environment
> 
> +@REM
> 
> +@REM Copyright (c) 2020, Intel Corporation. All rights reserved.<BR>
> 
> +@REM SPDX-License-Identifier: BSD-2-Clause-Patent
> 
> +@REM
> 
> +@REM This script calls GetPackagesPath.py to collect all package paths under
> 
> +@REM specified directories and appends them to PACKAGES_PATH environment
> 
> +@REM variable. A sub directory is a qualified package path when an EDKII
> 
> +@REM Package can be found under it.
> 
> +
> 
> +@echo off
> 
> +@if /I "%1"=="" @goto Usage
> 
> +@if /I "%1"=="-h" @goto Usage
> 
> +@if /I "%1"=="--help" @goto Usage
> 
> +@if /I "%1"=="/?" @goto Usage
> 
> +
> 
> +for /f %%i in ('python %~dp0\GetPackagesPath.py %*') do (
> 
> +    if defined PACKAGES_PATH (
> 
> +        set "PACKAGES_PATH=%PACKAGES_PATH%;%%i"
> 
> +    ) else (
> 
> +        set "PACKAGES_PATH=%%i"
> 
> +    )
> 
> +)
> 
> +@goto End
> 
> +
> 
> +:Usage
> 
> +@echo Usage: AppendPackagesPath.bat directory [directory ...]
> 
> +@echo Copyright(c) 2020, Intel Corporation. All rights reserved.
> 
> +@echo Options:
> 
> +@echo   --help, -h     Print this help screen and exit
> 
> +
> 
> +:End
> 
> diff --git a/Platform/Intel/Tools/AppendPackagesPath/AppendPackagesPath.sh
> b/Platform/Intel/Tools/AppendPackagesPath/AppendPackagesPath.sh
> new file mode 100644
> index 0000000000..599c8d073b
> --- /dev/null
> +++ b/Platform/Intel/Tools/AppendPackagesPath/AppendPackagesPath.sh
> @@ -0,0 +1,42 @@
> +#!/bin/bash
> +
> +#
> +# Copyright (c) 2020, Intel Corporation. All rights reserved.<BR>
> +# SPDX-License-Identifier: BSD-2-Clause-Patent
> +#
> +# This script calls GetPackagesPath.py to collect all package paths under
> +# specified directories and appends them to PACKAGES_PATH environment
> +# variable. A sub directory is a qualified package path when an EDKII
> +# Package can be found under it.
> +#
> +# Note: This script must be \'sourced\' so the environment can be changed:
> +# source SetPackagesPath.sh
> +# . AppendPackagesPath.sh
> +
> +function Usage()
> +{
> +    echo "Usage: source AppendPackagesPath.sh directory [directory ...]"
> +    echo "Copyright(c) 2020, Intel Corporation. All rights reserved."
> +    echo "Options:"
> +    echo "  --help, -h     Print this help screen and exit"
> +    echo "Please note: This script must be \'sourced\' so the environment can be changed."
> +    echo ". AppendPackagesPath.sh"
> +    echo "source AppendPackagesPath.sh"
> +}
> +
> +function SetEnv()
> +{
> +    local tool_path=$(dirname "$BASH_SOURCE")
> +    local paths=$(python $tool_path/GetPackagesPath.py $@)
> +    if [ "$PACKAGES_PATH" ]; then
> +        PACKAGES_PATH=$PACKAGES_PATH:$paths
> +    else
> +        PACKAGES_PATH=$paths
> +    fi
> +}
> +
> +if [ $# -eq 0 -o "$1" == "-h" -o "$1" == "--help" -o "$1" == "/?" ]; then
> +    Usage
> +else
> +    SetEnv $@
> +fi
> diff --git a/Platform/Intel/Tools/AppendPackagesPath/GetPackagesPath.py
> b/Platform/Intel/Tools/AppendPackagesPath/GetPackagesPath.py
> new file mode 100644
> index 0000000000..31ed44cfa3
> --- /dev/null
> +++ b/Platform/Intel/Tools/AppendPackagesPath/GetPackagesPath.py
> @@ -0,0 +1,98 @@
> +## @file
> 
> +# Get all recursive package paths from special directories.
> 
> +#
> 
> +# Copyright (c) 2020, Intel Corporation. All rights reserved.<BR>
> 
> +# SPDX-License-Identifier: BSD-2-Clause-Patent
> 
> +#
> 
> +
> 
> +import os
> 
> +import glob
> 
> +import argparse
> 
> +
> 
> +#
> 
> +# Globals for help information
> 
> +#
> 
> +__prog__ = 'GetPackagesPath.py'
> 
> +__copyright__ = 'Copyright (c) 2020, Intel Corporation. All rights reserved.'
> 
> +__description__ = 'Gets all recursive package paths in specified directory.\n'
> 
> +
> 
> +def __get_packages_path(root):
> 
> +    """ Gets all recursive package paths in specified directory.
> 
> +        A directory is a package path if it satisfies conditions below:
> 
> +        1. it is a directory
> 
> +        2. it is not an EDK II Package. An EDK II Package (directory) is
> 
> +           a directory that contains an EDK II package declaration (DEC) file.
> 
> +        3. it contains at least one first level EDK II Package.
> 
> +        Note: A directory is not package path but its subdirectory could be.
> 
> +        Example: edk2-platforms/Features is not package path
> 
> +        but edk2-platforms/Features/Intel is.
> 
> +
> 
> +        :param root: The specified directory to find package paths in it,
> 
> +            the caller should ensure it is an valid directory
> 
> +        :type root: String
> 
> +        :returns: Return all recursive package paths
> 
> +        :rtype: String list
> 
> +    """
> 
> +
> 
> +    paths = []
> 
> +    contain_package = False
> 
> +    for filename in os.listdir(root):
> 
> +        # skip files whose name starts with ".", such as ".git"
> 
> +        if filename.startswith('.'):
> 
> +            continue
> 
> +        filepath = os.path.join(root, filename)
> 
> +        if os.path.isdir(filepath):
> 
> +            if glob.glob(os.path.join(filepath, '*.dec')):
> 
> +                # it is an EDK II Package
> 
> +                contain_package = True
> 
> +            else:
> 
> +                # get package paths for subdirectory if it is not package
> 
> +                paths = paths + __get_packages_path(filepath)
> 
> +
> 
> +    if contain_package:
> 
> +        # root is a package path because it contains EDK II Package
> 
> +        # in first level folder, inset it to head of list
> 
> +        paths.insert(0, root)
> 
> +
> 
> +    # return package paths
> 
> +    return paths
> 
> +
> 
> +def get_packages_path(directories):
> 
> +    """ For each direcory in directories, gets all recursive package paths
> 
> +        in this directory and joins them into one string.
> 
> +
> 
> +        :param directories: the list of directory
> 
> +        :type directories: String list
> 
> +        :returns: Return string of package paths
> 
> +        :rtype: String
> 
> +    """
> 
> +
> 
> +    packages_path = ''
> 
> +    for directory in directories:
> 
> +        directory = os.path.abspath(directory)
> 
> +        if (not os.path.exists(directory)) or (not os.path.isdir(directory)):
> 
> +            continue
> 
> +
> 
> +        if glob.glob(os.path.join(directory, '*.dec')):
> 
> +            # it is an EDK II Package
> 
> +            continue
> 
> +
> 
> +        paths = __get_packages_path(directory)
> 
> +        for path in paths:
> 
> +            if packages_path == '':
> 
> +                packages_path = path
> 
> +            else:
> 
> +                packages_path += os.pathsep + path
> 
> +    return packages_path
> 
> +
> 
> +if __name__ == '__main__':
> 
> +    # Create command line argument parser object
> 
> +    parser = argparse.ArgumentParser(
> 
> +            prog=__prog__,
> 
> +            description=__description__ + __copyright__,
> 
> +            conflict_handler='resolve'
> 
> +    )
> 
> +    parser.add_argument('directory', nargs='+',
> 
> +            help='Specified directory where package packages are got from')
> 
> +    args = parser.parse_args()
> 
> +    print(get_packages_path(args.directory))
> 
> diff --git a/Platform/Intel/Tools/AppendPackagesPath/Readme.md
> b/Platform/Intel/Tools/AppendPackagesPath/Readme.md
> new file mode 100644
> index 0000000000..66aebee7cf
> --- /dev/null
> +++ b/Platform/Intel/Tools/AppendPackagesPath/Readme.md
> @@ -0,0 +1,28 @@
> +
> 
> +# How to use AppendPackagesPath
> 
> +
> 
> +## Overview
> 
> +
> 
> +This script calls GetPackagesPath.py to collect all package paths under specified directories and appends them to
> PACKAGES_PATH environment variable. A sub directory is a qualified package path when an EDKII Package can be found
> under it.
> 
> +
> 
> +**Notice**:
> 
> +The old PACKAGES_PATH will be replaced by new one.s
> 
> +
> 
> +## The usage of the tool
> 
> +
> 
> +### Windows
> 
> +
> 
> +Usage: AppendPackagesPath.bat directory [directory ...]
> 
> +Copyright(c) 2020, Intel Corporation. All rights reserved.
> 
> +Options:
> 
> +  --help, -h     Print this help screen and exit
> 
> +
> 
> +### Ubuntu
> 
> +
> 
> +Usage: source AppendPackagesPath.sh directory [directory ...]
> 
> +Copyright(c) 2020, Intel Corporation. All rights reserved.
> 
> +Options:
> 
> +  --help, -h     Print this help screen and exit
> 
> +Please note: This script must be \'sourced\' so the environment can be changed.
> 
> +. AppendPackagesPath.sh
> 
> +source AppendPackagesPath.sh
> 
> --
> 2.24.0.windows.2