* [PATCHv v2 00/15] Add a plugin LicenseCheck in open ci
@ 2020-07-29 12:28 Zhang, Shenglei
2020-07-29 12:28 ` [PATCH v2 01/15] .pytool/Plugin: Add a plugin LicenseCheck Zhang, Shenglei
0 siblings, 1 reply; 6+ messages in thread
From: Zhang, Shenglei @ 2020-07-29 12:28 UTC (permalink / raw)
To: devel
Cc: Michael D Kinney, Sean Brogan, Bret Barkelew, Eric Dong,
Laszlo Ersek, Zhichao Gao, Jiewen Yao, Chao Zhang, Jordan Justen,
Maciej Rabeda, Jiaxin Wu, Siyuan Fu, Liming Gao, Jian J Wang,
Hao A Wu, Andrew Fish, Ray Ni, Xiaoyu Lu, Ard Biesheuvel,
Leif Lindholm
REF: https://bugzilla.tianocore.org/show_bug.cgi?id=2691
REF: https://bugzilla.tianocore.org/show_bug.cgi?id=2833
LicenseCheck is now enabled in PatchCheck.py. But there's
a patch "Revert 'BaseTools/PatchCheck.py: Add LicenseCheck'"
to suggest revert the change.These patch series introduce
a plugin LicenseCheck into open ci so that license issues can
still be checked after the checker is disabled in PatchCheck.py.
1/15 is the plugin implementation.
2/15 ~ 15/15 introduce sections "IgnoreFiles" to allow developers
to skip license check for some files like generated files.
Only BSD-2-Clause-Patent can pass this checker.
v2: BSD-3-Clause-Patent can not pass this checker in v2 patch.
* 2/15~15/15 have been given reviewed-by. To avoid making noise,
I only send out cover letter and 1/15.
Cc: Michael D Kinney <michael.d.kinney@intel.com>
Cc: Sean Brogan <sean.brogan@microsoft.com>
Cc: Bret Barkelew <Bret.Barkelew@microsoft.com>
Cc: Eric Dong <eric.dong@intel.com>
Cc: Laszlo Ersek <lersek@redhat.com>
Cc: Zhichao Gao <zhichao.gao@intel.com>
Cc: Jiewen Yao <jiewen.yao@intel.com>
Cc: Chao Zhang <chao.b.zhang@intel.com>
Cc: Jordan Justen <jordan.l.justen@intel.com>
Cc: Maciej Rabeda <maciej.rabeda@linux.intel.com>
Cc: Jiaxin Wu <jiaxin.wu@intel.com>
Cc: Siyuan Fu <siyuan.fu@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Cc: Jian J Wang <jian.j.wang@intel.com>
Cc: Hao A Wu <hao.a.wu@intel.com>
Cc: Jordan Justen <jordan.l.justen@intel.com>
Cc: Andrew Fish <afish@apple.com>
Cc: Ray Ni <ray.ni@intel.com>
Cc: Jian J Wang <jian.j.wang@intel.com>
Cc: Xiaoyu Lu <xiaoyux.lu@intel.com>
Cc: Ard Biesheuvel <ard.biesheuvel@arm.com>
Cc: Leif Lindholm <leif@nuviainc.com>
Shenglei Zhang (15):
.pytool/Plugin: Add a plugin LicenseCheck
FatPkg/FatPkg.ci.yaml: Add configuration for LicenseCheck
ArmVirtPkg/ArmVirtPkg.ci.yaml: Add configuration for LicenseCheck
CryptoPkg/CryptoPkg.ci.yaml: Add configuration for LicenseCheck
EmulatorPkg/EmulatorPkg.ci.yaml: Add configuration for LicenseCheck
FmpDevicePkg/FmpDevicePkg.ci.yaml: Add configuration for LicenseCheck
MdeModulePkg/MdeModulePkg.ci.yaml: Add configuration for LicenseCheck
MdePkg/MdePkg.ci.yaml: Add configuration for LicenseCheck
NetworkPkg/NetworkPkg.ci.yaml: Add configuration for LicenseCheck
OvmfPkg/OvmfPkg.ci.yaml: Add configuration for LicenseCheck
PcAtChipsetPkg/PcAtChipsetPkg.ci.yaml: Add configuration for
LicenseCheck
SecurityPkg/SecurityPkg.ci.yaml: Add configuration for LicenseCheck
ShellPkg/ShellPkg.ci.yaml: Add configuration for LicenseCheck
UefiCpuPkg/UefiCpuPkg.ci.yaml: Add configuration for LicenseCheck
UnitTestFrameworkPkg: Add configuration for LicenseCheck in yaml file
.pytool/Plugin/LicenseCheck/LicenseCheck.py | 115 ++++++++++++++++++
.../LicenseCheck/LicenseCheck_plug_in.yaml | 11 ++
.pytool/Plugin/LicenseCheck/Readme.md | 17 +++
ArmVirtPkg/ArmVirtPkg.ci.yaml | 4 +
CryptoPkg/CryptoPkg.ci.yaml | 3 +
EmulatorPkg/EmulatorPkg.ci.yaml | 4 +
FatPkg/FatPkg.ci.yaml | 3 +
FmpDevicePkg/FmpDevicePkg.ci.yaml | 3 +
MdeModulePkg/MdeModulePkg.ci.yaml | 4 +
MdePkg/MdePkg.ci.yaml | 4 +
NetworkPkg/NetworkPkg.ci.yaml | 3 +
OvmfPkg/OvmfPkg.ci.yaml | 4 +
PcAtChipsetPkg/PcAtChipsetPkg.ci.yaml | 3 +
SecurityPkg/SecurityPkg.ci.yaml | 3 +
ShellPkg/ShellPkg.ci.yaml | 3 +
UefiCpuPkg/UefiCpuPkg.ci.yaml | 3 +
.../UnitTestFrameworkPkg.ci.yaml | 4 +
17 files changed, 191 insertions(+)
create mode 100644 .pytool/Plugin/LicenseCheck/LicenseCheck.py
create mode 100644 .pytool/Plugin/LicenseCheck/LicenseCheck_plug_in.yaml
create mode 100644 .pytool/Plugin/LicenseCheck/Readme.md
--
2.18.0.windows.1
^ permalink raw reply [flat|nested] 6+ messages in thread
* [PATCH v2 01/15] .pytool/Plugin: Add a plugin LicenseCheck
2020-07-29 12:28 [PATCHv v2 00/15] Add a plugin LicenseCheck in open ci Zhang, Shenglei
@ 2020-07-29 12:28 ` Zhang, Shenglei
2020-07-29 14:01 ` Liming Gao
0 siblings, 1 reply; 6+ messages in thread
From: Zhang, Shenglei @ 2020-07-29 12:28 UTC (permalink / raw)
To: devel; +Cc: Sean Brogan, Bret Barkelew, Michael D Kinney, Liming Gao
REF: https://bugzilla.tianocore.org/show_bug.cgi?id=2691
REF: https://bugzilla.tianocore.org/show_bug.cgi?id=2833
Add a plugin to check license conflict for new added
files in a patch. It will report out errors when meeting
files which are now contributed under BSD-2-Clause-Patent.
Cc: Sean Brogan <sean.brogan@microsoft.com>
Cc: Bret Barkelew <Bret.Barkelew@microsoft.com>
Cc: Michael D Kinney <michael.d.kinney@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Signed-off-by: Shenglei Zhang <shenglei.zhang@intel.com>
---
.pytool/Plugin/LicenseCheck/LicenseCheck.py | 115 ++++++++++++++++++
.../LicenseCheck/LicenseCheck_plug_in.yaml | 11 ++
.pytool/Plugin/LicenseCheck/Readme.md | 17 +++
3 files changed, 143 insertions(+)
create mode 100644 .pytool/Plugin/LicenseCheck/LicenseCheck.py
create mode 100644 .pytool/Plugin/LicenseCheck/LicenseCheck_plug_in.yaml
create mode 100644 .pytool/Plugin/LicenseCheck/Readme.md
diff --git a/.pytool/Plugin/LicenseCheck/LicenseCheck.py b/.pytool/Plugin/LicenseCheck/LicenseCheck.py
new file mode 100644
index 000000000000..5733f7bf4ec0
--- /dev/null
+++ b/.pytool/Plugin/LicenseCheck/LicenseCheck.py
@@ -0,0 +1,115 @@
+# @file LicenseCheck.py
+#
+# Copyright (c) 2020, Intel Corporation. All rights reserved.<BR>
+# SPDX-License-Identifier: BSD-2-Clause-Patent
+##
+
+import os
+import logging
+import re
+from io import StringIO
+from typing import List, Tuple
+from edk2toolext.environment.plugintypes.ci_build_plugin import ICiBuildPlugin
+from edk2toolext.environment.var_dict import VarDict
+from edk2toollib.utility_functions import RunCmd
+
+
+class LicenseCheck(ICiBuildPlugin):
+
+ """
+ A CiBuildPlugin to check the license for new added files.
+
+ Configuration options:
+ "LicenseCheck": {
+ "IgnoreFiles": []
+ },
+ """
+
+ license_format_preflix = 'SPDX-License-Identifier'
+
+ bsd2_patent = 'BSD-2-Clause-Patent'
+
+ Readdedfileformat = re.compile(r'\+\+\+ b\/(.*)')
+
+ file_extension_list = [".c", ".h", ".inf", ".dsc", ".dec", ".py", ".bat", ".sh", ".uni", ".yaml",
+ ".fdf", ".inc", "yml", ".asm", ".asm16", ".asl", ".vfr", ".s", ".S", ".aslc",
+ ".nasm", ".nasmb", ".idf", ".Vfr", ".H"]
+
+ def GetTestName(self, packagename: str, environment: VarDict) -> tuple:
+ """ Provide the testcase name and classname for use in reporting
+ testclassname: a descriptive string for the testcase can include whitespace
+ classname: should be patterned <packagename>.<plugin>.<optionally any unique condition>
+
+ Args:
+ packagename: string containing name of package to build
+ environment: The VarDict for the test to run in
+ Returns:
+ a tuple containing the testcase name and the classname
+ (testcasename, classname)
+ """
+ return ("Check for license for " + packagename, packagename + ".LicenseCheck")
+
+ ##
+ # External function of plugin. This function is used to perform the task of the ci_build_plugin Plugin
+ #
+ # - package is the edk2 path to package. This means workspace/packagepath relative.
+ # - edk2path object configured with workspace and packages path
+ # - PkgConfig Object (dict) for the pkg
+ # - EnvConfig Object
+ # - Plugin Manager Instance
+ # - Plugin Helper Obj Instance
+ # - Junit Logger
+ # - output_stream the StringIO output stream from this plugin via logging
+ def RunBuildPlugin(self, packagename, Edk2pathObj, pkgconfig, environment, PLM, PLMHelper, tc, output_stream=None):
+ return_buffer = StringIO()
+ params = "diff --unified=0 origin/master HEAD"
+ RunCmd("git", params, outstream=return_buffer)
+ p = return_buffer.getvalue().strip()
+ patch = p.split("\n")
+ return_buffer.close()
+
+ ignore_files = []
+ if "IgnoreFiles" in pkgconfig:
+ ignore_files = pkgconfig["IgnoreFiles"]
+
+ self.ok = True
+ self.startcheck = False
+ self.license = True
+ self.all_file_pass = True
+ count = len(patch)
+ line_index = 0
+ for line in patch:
+ if line.startswith('--- /dev/null'):
+ nextline = patch[line_index + 1]
+ added_file = self.Readdedfileformat.search(nextline).group(1)
+ added_file_extension = os.path.splitext(added_file)[1]
+ if added_file_extension in self.file_extension_list and packagename in added_file:
+ if (self.IsIgnoreFile(added_file, ignore_files)):
+ line_index = line_index + 1
+ continue
+ self.startcheck = True
+ self.license = False
+ if self.startcheck and self.license_format_preflix in line:
+ if self.bsd2_patent in line:
+ self.license = True
+ if line_index + 1 == count or patch[line_index + 1].startswith('diff --') and self.startcheck:
+ if not self.license:
+ self.all_file_pass = False
+ error_message = "Invalid license in: " + added_file + " Hint: Only BSD-2-Clause-Patent is accepted."
+ logging.error(error_message)
+ self.startcheck = False
+ self.license = True
+ line_index = line_index + 1
+
+ if self.all_file_pass:
+ tc.SetSuccess()
+ return 0
+ else:
+ tc.SetFailed("License Check {0} Failed. ".format(packagename), "LICENSE_CHECK_FAILED")
+ return 1
+
+ def IsIgnoreFile(self, file: str, ignore_files: List[str]) -> bool:
+ for f in ignore_files:
+ if f in file:
+ return True
+ return False
diff --git a/.pytool/Plugin/LicenseCheck/LicenseCheck_plug_in.yaml b/.pytool/Plugin/LicenseCheck/LicenseCheck_plug_in.yaml
new file mode 100644
index 000000000000..2fdb0aa72359
--- /dev/null
+++ b/.pytool/Plugin/LicenseCheck/LicenseCheck_plug_in.yaml
@@ -0,0 +1,11 @@
+## @file
+# CiBuildPlugin used to check license issues for new added files
+#
+# Copyright (c) 2020, Intel Corporation. All rights reserved.
+# SPDX-License-Identifier: BSD-2-Clause-Patent
+##
+{
+ "scope": "cibuild",
+ "name": "License Check Test",
+ "module": "LicenseCheck"
+}
diff --git a/.pytool/Plugin/LicenseCheck/Readme.md b/.pytool/Plugin/LicenseCheck/Readme.md
new file mode 100644
index 000000000000..b0e1bbea9e21
--- /dev/null
+++ b/.pytool/Plugin/LicenseCheck/Readme.md
@@ -0,0 +1,17 @@
+# License Check Plugin
+
+This CiBuildPlugin scans all new added files in a package to make sure code
+is contributed under BSD-2-Clause-Patent.
+
+## Configuration
+
+The plugin can be configured to ignore certain files.
+
+``` yaml
+"LicenseCheck": {
+ "IgnoreFiles": []
+}
+```
+### IgnoreFiles
+
+OPTIONAL List of file to ignore.
--
2.18.0.windows.1
^ permalink raw reply related [flat|nested] 6+ messages in thread
* Re: [PATCH v2 01/15] .pytool/Plugin: Add a plugin LicenseCheck
2020-07-29 12:28 ` [PATCH v2 01/15] .pytool/Plugin: Add a plugin LicenseCheck Zhang, Shenglei
@ 2020-07-29 14:01 ` Liming Gao
2020-07-30 3:07 ` Zhang, Shenglei
0 siblings, 1 reply; 6+ messages in thread
From: Liming Gao @ 2020-07-29 14:01 UTC (permalink / raw)
To: Zhang, Shenglei, devel@edk2.groups.io
Cc: Sean Brogan, Bret Barkelew, Kinney, Michael D
Shenglei:
> -----Original Message-----
> From: Zhang, Shenglei <shenglei.zhang@intel.com>
> Sent: Wednesday, July 29, 2020 8:28 PM
> To: devel@edk2.groups.io
> Cc: Sean Brogan <sean.brogan@microsoft.com>; Bret Barkelew <Bret.Barkelew@microsoft.com>; Kinney, Michael D
> <michael.d.kinney@intel.com>; Gao, Liming <liming.gao@intel.com>
> Subject: [PATCH v2 01/15] .pytool/Plugin: Add a plugin LicenseCheck
>
> REF: https://bugzilla.tianocore.org/show_bug.cgi?id=2691
> REF: https://bugzilla.tianocore.org/show_bug.cgi?id=2833
> Add a plugin to check license conflict for new added
> files in a patch. It will report out errors when meeting
> files which are now contributed under BSD-2-Clause-Patent.
files which are now contributed under BSD-2-Clause-Patent.
==>
files which are not contributed under BSD-2-Clause-Patent.
Thanks
Liming
>
> Cc: Sean Brogan <sean.brogan@microsoft.com>
> Cc: Bret Barkelew <Bret.Barkelew@microsoft.com>
> Cc: Michael D Kinney <michael.d.kinney@intel.com>
> Cc: Liming Gao <liming.gao@intel.com>
> Signed-off-by: Shenglei Zhang <shenglei.zhang@intel.com>
> ---
> .pytool/Plugin/LicenseCheck/LicenseCheck.py | 115 ++++++++++++++++++
> .../LicenseCheck/LicenseCheck_plug_in.yaml | 11 ++
> .pytool/Plugin/LicenseCheck/Readme.md | 17 +++
> 3 files changed, 143 insertions(+)
> create mode 100644 .pytool/Plugin/LicenseCheck/LicenseCheck.py
> create mode 100644 .pytool/Plugin/LicenseCheck/LicenseCheck_plug_in.yaml
> create mode 100644 .pytool/Plugin/LicenseCheck/Readme.md
>
> diff --git a/.pytool/Plugin/LicenseCheck/LicenseCheck.py b/.pytool/Plugin/LicenseCheck/LicenseCheck.py
> new file mode 100644
> index 000000000000..5733f7bf4ec0
> --- /dev/null
> +++ b/.pytool/Plugin/LicenseCheck/LicenseCheck.py
> @@ -0,0 +1,115 @@
> +# @file LicenseCheck.py
> +#
> +# Copyright (c) 2020, Intel Corporation. All rights reserved.<BR>
> +# SPDX-License-Identifier: BSD-2-Clause-Patent
> +##
> +
> +import os
> +import logging
> +import re
> +from io import StringIO
> +from typing import List, Tuple
> +from edk2toolext.environment.plugintypes.ci_build_plugin import ICiBuildPlugin
> +from edk2toolext.environment.var_dict import VarDict
> +from edk2toollib.utility_functions import RunCmd
> +
> +
> +class LicenseCheck(ICiBuildPlugin):
> +
> + """
> + A CiBuildPlugin to check the license for new added files.
> +
> + Configuration options:
> + "LicenseCheck": {
> + "IgnoreFiles": []
> + },
> + """
> +
> + license_format_preflix = 'SPDX-License-Identifier'
> +
> + bsd2_patent = 'BSD-2-Clause-Patent'
> +
> + Readdedfileformat = re.compile(r'\+\+\+ b\/(.*)')
> +
> + file_extension_list = [".c", ".h", ".inf", ".dsc", ".dec", ".py", ".bat", ".sh", ".uni", ".yaml",
> + ".fdf", ".inc", "yml", ".asm", ".asm16", ".asl", ".vfr", ".s", ".S", ".aslc",
> + ".nasm", ".nasmb", ".idf", ".Vfr", ".H"]
> +
> + def GetTestName(self, packagename: str, environment: VarDict) -> tuple:
> + """ Provide the testcase name and classname for use in reporting
> + testclassname: a descriptive string for the testcase can include whitespace
> + classname: should be patterned <packagename>.<plugin>.<optionally any unique condition>
> +
> + Args:
> + packagename: string containing name of package to build
> + environment: The VarDict for the test to run in
> + Returns:
> + a tuple containing the testcase name and the classname
> + (testcasename, classname)
> + """
> + return ("Check for license for " + packagename, packagename + ".LicenseCheck")
> +
> + ##
> + # External function of plugin. This function is used to perform the task of the ci_build_plugin Plugin
> + #
> + # - package is the edk2 path to package. This means workspace/packagepath relative.
> + # - edk2path object configured with workspace and packages path
> + # - PkgConfig Object (dict) for the pkg
> + # - EnvConfig Object
> + # - Plugin Manager Instance
> + # - Plugin Helper Obj Instance
> + # - Junit Logger
> + # - output_stream the StringIO output stream from this plugin via logging
> + def RunBuildPlugin(self, packagename, Edk2pathObj, pkgconfig, environment, PLM, PLMHelper, tc, output_stream=None):
> + return_buffer = StringIO()
> + params = "diff --unified=0 origin/master HEAD"
> + RunCmd("git", params, outstream=return_buffer)
> + p = return_buffer.getvalue().strip()
> + patch = p.split("\n")
> + return_buffer.close()
> +
> + ignore_files = []
> + if "IgnoreFiles" in pkgconfig:
> + ignore_files = pkgconfig["IgnoreFiles"]
> +
> + self.ok = True
> + self.startcheck = False
> + self.license = True
> + self.all_file_pass = True
> + count = len(patch)
> + line_index = 0
> + for line in patch:
> + if line.startswith('--- /dev/null'):
> + nextline = patch[line_index + 1]
> + added_file = self.Readdedfileformat.search(nextline).group(1)
> + added_file_extension = os.path.splitext(added_file)[1]
> + if added_file_extension in self.file_extension_list and packagename in added_file:
> + if (self.IsIgnoreFile(added_file, ignore_files)):
> + line_index = line_index + 1
> + continue
> + self.startcheck = True
> + self.license = False
> + if self.startcheck and self.license_format_preflix in line:
> + if self.bsd2_patent in line:
> + self.license = True
> + if line_index + 1 == count or patch[line_index + 1].startswith('diff --') and self.startcheck:
> + if not self.license:
> + self.all_file_pass = False
> + error_message = "Invalid license in: " + added_file + " Hint: Only BSD-2-Clause-Patent is accepted."
> + logging.error(error_message)
> + self.startcheck = False
> + self.license = True
> + line_index = line_index + 1
> +
> + if self.all_file_pass:
> + tc.SetSuccess()
> + return 0
> + else:
> + tc.SetFailed("License Check {0} Failed. ".format(packagename), "LICENSE_CHECK_FAILED")
> + return 1
> +
> + def IsIgnoreFile(self, file: str, ignore_files: List[str]) -> bool:
> + for f in ignore_files:
> + if f in file:
> + return True
> + return False
> diff --git a/.pytool/Plugin/LicenseCheck/LicenseCheck_plug_in.yaml b/.pytool/Plugin/LicenseCheck/LicenseCheck_plug_in.yaml
> new file mode 100644
> index 000000000000..2fdb0aa72359
> --- /dev/null
> +++ b/.pytool/Plugin/LicenseCheck/LicenseCheck_plug_in.yaml
> @@ -0,0 +1,11 @@
> +## @file
> +# CiBuildPlugin used to check license issues for new added files
> +#
> +# Copyright (c) 2020, Intel Corporation. All rights reserved.
> +# SPDX-License-Identifier: BSD-2-Clause-Patent
> +##
> +{
> + "scope": "cibuild",
> + "name": "License Check Test",
> + "module": "LicenseCheck"
> +}
> diff --git a/.pytool/Plugin/LicenseCheck/Readme.md b/.pytool/Plugin/LicenseCheck/Readme.md
> new file mode 100644
> index 000000000000..b0e1bbea9e21
> --- /dev/null
> +++ b/.pytool/Plugin/LicenseCheck/Readme.md
> @@ -0,0 +1,17 @@
> +# License Check Plugin
> +
> +This CiBuildPlugin scans all new added files in a package to make sure code
> +is contributed under BSD-2-Clause-Patent.
> +
> +## Configuration
> +
> +The plugin can be configured to ignore certain files.
> +
> +``` yaml
> +"LicenseCheck": {
> + "IgnoreFiles": []
> +}
> +```
> +### IgnoreFiles
> +
> +OPTIONAL List of file to ignore.
> --
> 2.18.0.windows.1
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH v2 01/15] .pytool/Plugin: Add a plugin LicenseCheck
2020-07-29 14:01 ` Liming Gao
@ 2020-07-30 3:07 ` Zhang, Shenglei
2020-07-30 3:15 ` Liming Gao
[not found] ` <162669507765F26E.19559@groups.io>
0 siblings, 2 replies; 6+ messages in thread
From: Zhang, Shenglei @ 2020-07-30 3:07 UTC (permalink / raw)
To: Gao, Liming, devel@edk2.groups.io
Cc: Sean Brogan, Bret Barkelew, Kinney, Michael D
> -----Original Message-----
> From: Gao, Liming <liming.gao@intel.com>
> Sent: Wednesday, July 29, 2020 10:02 PM
> To: Zhang, Shenglei <shenglei.zhang@intel.com>; devel@edk2.groups.io
> Cc: Sean Brogan <sean.brogan@microsoft.com>; Bret Barkelew
> <Bret.Barkelew@microsoft.com>; Kinney, Michael D
> <michael.d.kinney@intel.com>
> Subject: RE: [PATCH v2 01/15] .pytool/Plugin: Add a plugin LicenseCheck
>
> Shenglei:
>
> > -----Original Message-----
> > From: Zhang, Shenglei <shenglei.zhang@intel.com>
> > Sent: Wednesday, July 29, 2020 8:28 PM
> > To: devel@edk2.groups.io
> > Cc: Sean Brogan <sean.brogan@microsoft.com>; Bret Barkelew
> <Bret.Barkelew@microsoft.com>; Kinney, Michael D
> > <michael.d.kinney@intel.com>; Gao, Liming <liming.gao@intel.com>
> > Subject: [PATCH v2 01/15] .pytool/Plugin: Add a plugin LicenseCheck
> >
> > REF: https://bugzilla.tianocore.org/show_bug.cgi?id=2691
> > REF: https://bugzilla.tianocore.org/show_bug.cgi?id=2833
> > Add a plugin to check license conflict for new added
> > files in a patch. It will report out errors when meeting
> > files which are now contributed under BSD-2-Clause-Patent.
>
> files which are now contributed under BSD-2-Clause-Patent.
> ==>
> files which are not contributed under BSD-2-Clause-Patent.
>
Yes. It's typo.
Thanks,
Shenglei
> Thanks
> Liming
> >
> > Cc: Sean Brogan <sean.brogan@microsoft.com>
> > Cc: Bret Barkelew <Bret.Barkelew@microsoft.com>
> > Cc: Michael D Kinney <michael.d.kinney@intel.com>
> > Cc: Liming Gao <liming.gao@intel.com>
> > Signed-off-by: Shenglei Zhang <shenglei.zhang@intel.com>
> > ---
> > .pytool/Plugin/LicenseCheck/LicenseCheck.py | 115
> ++++++++++++++++++
> > .../LicenseCheck/LicenseCheck_plug_in.yaml | 11 ++
> > .pytool/Plugin/LicenseCheck/Readme.md | 17 +++
> > 3 files changed, 143 insertions(+)
> > create mode 100644 .pytool/Plugin/LicenseCheck/LicenseCheck.py
> > create mode
> 100644 .pytool/Plugin/LicenseCheck/LicenseCheck_plug_in.yaml
> > create mode 100644 .pytool/Plugin/LicenseCheck/Readme.md
> >
> > diff --git a/.pytool/Plugin/LicenseCheck/LicenseCheck.py
> b/.pytool/Plugin/LicenseCheck/LicenseCheck.py
> > new file mode 100644
> > index 000000000000..5733f7bf4ec0
> > --- /dev/null
> > +++ b/.pytool/Plugin/LicenseCheck/LicenseCheck.py
> > @@ -0,0 +1,115 @@
> > +# @file LicenseCheck.py
> > +#
> > +# Copyright (c) 2020, Intel Corporation. All rights reserved.<BR>
> > +# SPDX-License-Identifier: BSD-2-Clause-Patent
> > +##
> > +
> > +import os
> > +import logging
> > +import re
> > +from io import StringIO
> > +from typing import List, Tuple
> > +from edk2toolext.environment.plugintypes.ci_build_plugin import
> ICiBuildPlugin
> > +from edk2toolext.environment.var_dict import VarDict
> > +from edk2toollib.utility_functions import RunCmd
> > +
> > +
> > +class LicenseCheck(ICiBuildPlugin):
> > +
> > + """
> > + A CiBuildPlugin to check the license for new added files.
> > +
> > + Configuration options:
> > + "LicenseCheck": {
> > + "IgnoreFiles": []
> > + },
> > + """
> > +
> > + license_format_preflix = 'SPDX-License-Identifier'
> > +
> > + bsd2_patent = 'BSD-2-Clause-Patent'
> > +
> > + Readdedfileformat = re.compile(r'\+\+\+ b\/(.*)')
> > +
> > + file_extension_list = [".c", ".h", ".inf", ".dsc", ".dec", ".py", ".bat", ".sh",
> ".uni", ".yaml",
> > + ".fdf", ".inc", "yml", ".asm", ".asm16", ".asl", ".vfr", ".s", ".S",
> ".aslc",
> > + ".nasm", ".nasmb", ".idf", ".Vfr", ".H"]
> > +
> > + def GetTestName(self, packagename: str, environment: VarDict) ->
> tuple:
> > + """ Provide the testcase name and classname for use in reporting
> > + testclassname: a descriptive string for the testcase can include
> whitespace
> > + classname: should be patterned
> <packagename>.<plugin>.<optionally any unique condition>
> > +
> > + Args:
> > + packagename: string containing name of package to build
> > + environment: The VarDict for the test to run in
> > + Returns:
> > + a tuple containing the testcase name and the classname
> > + (testcasename, classname)
> > + """
> > + return ("Check for license for " + packagename, packagename +
> ".LicenseCheck")
> > +
> > + ##
> > + # External function of plugin. This function is used to perform the task
> of the ci_build_plugin Plugin
> > + #
> > + # - package is the edk2 path to package. This means
> workspace/packagepath relative.
> > + # - edk2path object configured with workspace and packages path
> > + # - PkgConfig Object (dict) for the pkg
> > + # - EnvConfig Object
> > + # - Plugin Manager Instance
> > + # - Plugin Helper Obj Instance
> > + # - Junit Logger
> > + # - output_stream the StringIO output stream from this plugin via
> logging
> > + def RunBuildPlugin(self, packagename, Edk2pathObj, pkgconfig,
> environment, PLM, PLMHelper, tc, output_stream=None):
> > + return_buffer = StringIO()
> > + params = "diff --unified=0 origin/master HEAD"
> > + RunCmd("git", params, outstream=return_buffer)
> > + p = return_buffer.getvalue().strip()
> > + patch = p.split("\n")
> > + return_buffer.close()
> > +
> > + ignore_files = []
> > + if "IgnoreFiles" in pkgconfig:
> > + ignore_files = pkgconfig["IgnoreFiles"]
> > +
> > + self.ok = True
> > + self.startcheck = False
> > + self.license = True
> > + self.all_file_pass = True
> > + count = len(patch)
> > + line_index = 0
> > + for line in patch:
> > + if line.startswith('--- /dev/null'):
> > + nextline = patch[line_index + 1]
> > + added_file = self.Readdedfileformat.search(nextline).group(1)
> > + added_file_extension = os.path.splitext(added_file)[1]
> > + if added_file_extension in self.file_extension_list and
> packagename in added_file:
> > + if (self.IsIgnoreFile(added_file, ignore_files)):
> > + line_index = line_index + 1
> > + continue
> > + self.startcheck = True
> > + self.license = False
> > + if self.startcheck and self.license_format_preflix in line:
> > + if self.bsd2_patent in line:
> > + self.license = True
> > + if line_index + 1 == count or patch[line_index + 1].startswith('diff --')
> and self.startcheck:
> > + if not self.license:
> > + self.all_file_pass = False
> > + error_message = "Invalid license in: " + added_file + " Hint: Only
> BSD-2-Clause-Patent is accepted."
> > + logging.error(error_message)
> > + self.startcheck = False
> > + self.license = True
> > + line_index = line_index + 1
> > +
> > + if self.all_file_pass:
> > + tc.SetSuccess()
> > + return 0
> > + else:
> > + tc.SetFailed("License Check {0} Failed. ".format(packagename),
> "LICENSE_CHECK_FAILED")
> > + return 1
> > +
> > + def IsIgnoreFile(self, file: str, ignore_files: List[str]) -> bool:
> > + for f in ignore_files:
> > + if f in file:
> > + return True
> > + return False
> > diff --git a/.pytool/Plugin/LicenseCheck/LicenseCheck_plug_in.yaml
> b/.pytool/Plugin/LicenseCheck/LicenseCheck_plug_in.yaml
> > new file mode 100644
> > index 000000000000..2fdb0aa72359
> > --- /dev/null
> > +++ b/.pytool/Plugin/LicenseCheck/LicenseCheck_plug_in.yaml
> > @@ -0,0 +1,11 @@
> > +## @file
> > +# CiBuildPlugin used to check license issues for new added files
> > +#
> > +# Copyright (c) 2020, Intel Corporation. All rights reserved.
> > +# SPDX-License-Identifier: BSD-2-Clause-Patent
> > +##
> > +{
> > + "scope": "cibuild",
> > + "name": "License Check Test",
> > + "module": "LicenseCheck"
> > +}
> > diff --git a/.pytool/Plugin/LicenseCheck/Readme.md
> b/.pytool/Plugin/LicenseCheck/Readme.md
> > new file mode 100644
> > index 000000000000..b0e1bbea9e21
> > --- /dev/null
> > +++ b/.pytool/Plugin/LicenseCheck/Readme.md
> > @@ -0,0 +1,17 @@
> > +# License Check Plugin
> > +
> > +This CiBuildPlugin scans all new added files in a package to make sure code
> > +is contributed under BSD-2-Clause-Patent.
> > +
> > +## Configuration
> > +
> > +The plugin can be configured to ignore certain files.
> > +
> > +``` yaml
> > +"LicenseCheck": {
> > + "IgnoreFiles": []
> > +}
> > +```
> > +### IgnoreFiles
> > +
> > +OPTIONAL List of file to ignore.
> > --
> > 2.18.0.windows.1
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH v2 01/15] .pytool/Plugin: Add a plugin LicenseCheck
2020-07-30 3:07 ` Zhang, Shenglei
@ 2020-07-30 3:15 ` Liming Gao
[not found] ` <162669507765F26E.19559@groups.io>
1 sibling, 0 replies; 6+ messages in thread
From: Liming Gao @ 2020-07-30 3:15 UTC (permalink / raw)
To: Zhang, Shenglei, devel@edk2.groups.io
Cc: Sean Brogan, Bret Barkelew, Kinney, Michael D
With this update, Reviewed-by: Liming Gao <liming.gao@intel.com>
If no other comment, I will merge this patch set tomorrow.
Thanks
Liming
-----Original Message-----
From: Zhang, Shenglei <shenglei.zhang@intel.com>
Sent: 2020年7月30日 11:08
To: Gao, Liming <liming.gao@intel.com>; devel@edk2.groups.io
Cc: Sean Brogan <sean.brogan@microsoft.com>; Bret Barkelew <Bret.Barkelew@microsoft.com>; Kinney, Michael D <michael.d.kinney@intel.com>
Subject: RE: [PATCH v2 01/15] .pytool/Plugin: Add a plugin LicenseCheck
> -----Original Message-----
> From: Gao, Liming <liming.gao@intel.com>
> Sent: Wednesday, July 29, 2020 10:02 PM
> To: Zhang, Shenglei <shenglei.zhang@intel.com>; devel@edk2.groups.io
> Cc: Sean Brogan <sean.brogan@microsoft.com>; Bret Barkelew
> <Bret.Barkelew@microsoft.com>; Kinney, Michael D
> <michael.d.kinney@intel.com>
> Subject: RE: [PATCH v2 01/15] .pytool/Plugin: Add a plugin
> LicenseCheck
>
> Shenglei:
>
> > -----Original Message-----
> > From: Zhang, Shenglei <shenglei.zhang@intel.com>
> > Sent: Wednesday, July 29, 2020 8:28 PM
> > To: devel@edk2.groups.io
> > Cc: Sean Brogan <sean.brogan@microsoft.com>; Bret Barkelew
> <Bret.Barkelew@microsoft.com>; Kinney, Michael D
> > <michael.d.kinney@intel.com>; Gao, Liming <liming.gao@intel.com>
> > Subject: [PATCH v2 01/15] .pytool/Plugin: Add a plugin LicenseCheck
> >
> > REF: https://bugzilla.tianocore.org/show_bug.cgi?id=2691
> > REF: https://bugzilla.tianocore.org/show_bug.cgi?id=2833
> > Add a plugin to check license conflict for new added files in a
> > patch. It will report out errors when meeting files which are now
> > contributed under BSD-2-Clause-Patent.
>
> files which are now contributed under BSD-2-Clause-Patent.
> ==>
> files which are not contributed under BSD-2-Clause-Patent.
>
Yes. It's typo.
Thanks,
Shenglei
> Thanks
> Liming
> >
> > Cc: Sean Brogan <sean.brogan@microsoft.com>
> > Cc: Bret Barkelew <Bret.Barkelew@microsoft.com>
> > Cc: Michael D Kinney <michael.d.kinney@intel.com>
> > Cc: Liming Gao <liming.gao@intel.com>
> > Signed-off-by: Shenglei Zhang <shenglei.zhang@intel.com>
> > ---
> > .pytool/Plugin/LicenseCheck/LicenseCheck.py | 115
> ++++++++++++++++++
> > .../LicenseCheck/LicenseCheck_plug_in.yaml | 11 ++
> > .pytool/Plugin/LicenseCheck/Readme.md | 17 +++
> > 3 files changed, 143 insertions(+)
> > create mode 100644 .pytool/Plugin/LicenseCheck/LicenseCheck.py
> > create mode
> 100644 .pytool/Plugin/LicenseCheck/LicenseCheck_plug_in.yaml
> > create mode 100644 .pytool/Plugin/LicenseCheck/Readme.md
> >
> > diff --git a/.pytool/Plugin/LicenseCheck/LicenseCheck.py
> b/.pytool/Plugin/LicenseCheck/LicenseCheck.py
> > new file mode 100644
> > index 000000000000..5733f7bf4ec0
> > --- /dev/null
> > +++ b/.pytool/Plugin/LicenseCheck/LicenseCheck.py
> > @@ -0,0 +1,115 @@
> > +# @file LicenseCheck.py
> > +#
> > +# Copyright (c) 2020, Intel Corporation. All rights reserved.<BR> #
> > +SPDX-License-Identifier: BSD-2-Clause-Patent ##
> > +
> > +import os
> > +import logging
> > +import re
> > +from io import StringIO
> > +from typing import List, Tuple
> > +from edk2toolext.environment.plugintypes.ci_build_plugin import
> ICiBuildPlugin
> > +from edk2toolext.environment.var_dict import VarDict from
> > +edk2toollib.utility_functions import RunCmd
> > +
> > +
> > +class LicenseCheck(ICiBuildPlugin):
> > +
> > + """
> > + A CiBuildPlugin to check the license for new added files.
> > +
> > + Configuration options:
> > + "LicenseCheck": {
> > + "IgnoreFiles": []
> > + },
> > + """
> > +
> > + license_format_preflix = 'SPDX-License-Identifier'
> > +
> > + bsd2_patent = 'BSD-2-Clause-Patent'
> > +
> > + Readdedfileformat = re.compile(r'\+\+\+ b\/(.*)')
> > +
> > + file_extension_list = [".c", ".h", ".inf", ".dsc", ".dec",
> > + ".py", ".bat", ".sh",
> ".uni", ".yaml",
> > + ".fdf", ".inc", "yml", ".asm", ".asm16",
> > + ".asl", ".vfr", ".s", ".S",
> ".aslc",
> > + ".nasm", ".nasmb", ".idf", ".Vfr", ".H"]
> > +
> > + def GetTestName(self, packagename: str, environment: VarDict)
> > + ->
> tuple:
> > + """ Provide the testcase name and classname for use in reporting
> > + testclassname: a descriptive string for the testcase
> > + can include
> whitespace
> > + classname: should be patterned
> <packagename>.<plugin>.<optionally any unique condition>
> > +
> > + Args:
> > + packagename: string containing name of package to build
> > + environment: The VarDict for the test to run in
> > + Returns:
> > + a tuple containing the testcase name and the classname
> > + (testcasename, classname)
> > + """
> > + return ("Check for license for " + packagename, packagename
> > + +
> ".LicenseCheck")
> > +
> > + ##
> > + # External function of plugin. This function is used to
> > + perform the task
> of the ci_build_plugin Plugin
> > + #
> > + # - package is the edk2 path to package. This means
> workspace/packagepath relative.
> > + # - edk2path object configured with workspace and packages path
> > + # - PkgConfig Object (dict) for the pkg
> > + # - EnvConfig Object
> > + # - Plugin Manager Instance
> > + # - Plugin Helper Obj Instance
> > + # - Junit Logger
> > + # - output_stream the StringIO output stream from this plugin via
> logging
> > + def RunBuildPlugin(self, packagename, Edk2pathObj, pkgconfig,
> environment, PLM, PLMHelper, tc, output_stream=None):
> > + return_buffer = StringIO()
> > + params = "diff --unified=0 origin/master HEAD"
> > + RunCmd("git", params, outstream=return_buffer)
> > + p = return_buffer.getvalue().strip()
> > + patch = p.split("\n")
> > + return_buffer.close()
> > +
> > + ignore_files = []
> > + if "IgnoreFiles" in pkgconfig:
> > + ignore_files = pkgconfig["IgnoreFiles"]
> > +
> > + self.ok = True
> > + self.startcheck = False
> > + self.license = True
> > + self.all_file_pass = True
> > + count = len(patch)
> > + line_index = 0
> > + for line in patch:
> > + if line.startswith('--- /dev/null'):
> > + nextline = patch[line_index + 1]
> > + added_file = self.Readdedfileformat.search(nextline).group(1)
> > + added_file_extension = os.path.splitext(added_file)[1]
> > + if added_file_extension in self.file_extension_list
> > + and
> packagename in added_file:
> > + if (self.IsIgnoreFile(added_file, ignore_files)):
> > + line_index = line_index + 1
> > + continue
> > + self.startcheck = True
> > + self.license = False
> > + if self.startcheck and self.license_format_preflix in line:
> > + if self.bsd2_patent in line:
> > + self.license = True
> > + if line_index + 1 == count or patch[line_index +
> > + 1].startswith('diff --')
> and self.startcheck:
> > + if not self.license:
> > + self.all_file_pass = False
> > + error_message = "Invalid license in: " +
> > + added_file + " Hint: Only
> BSD-2-Clause-Patent is accepted."
> > + logging.error(error_message)
> > + self.startcheck = False
> > + self.license = True
> > + line_index = line_index + 1
> > +
> > + if self.all_file_pass:
> > + tc.SetSuccess()
> > + return 0
> > + else:
> > + tc.SetFailed("License Check {0} Failed.
> > + ".format(packagename),
> "LICENSE_CHECK_FAILED")
> > + return 1
> > +
> > + def IsIgnoreFile(self, file: str, ignore_files: List[str]) -> bool:
> > + for f in ignore_files:
> > + if f in file:
> > + return True
> > + return False
> > diff --git a/.pytool/Plugin/LicenseCheck/LicenseCheck_plug_in.yaml
> b/.pytool/Plugin/LicenseCheck/LicenseCheck_plug_in.yaml
> > new file mode 100644
> > index 000000000000..2fdb0aa72359
> > --- /dev/null
> > +++ b/.pytool/Plugin/LicenseCheck/LicenseCheck_plug_in.yaml
> > @@ -0,0 +1,11 @@
> > +## @file
> > +# CiBuildPlugin used to check license issues for new added files #
> > +# Copyright (c) 2020, Intel Corporation. All rights reserved.
> > +# SPDX-License-Identifier: BSD-2-Clause-Patent ## {
> > + "scope": "cibuild",
> > + "name": "License Check Test",
> > + "module": "LicenseCheck"
> > +}
> > diff --git a/.pytool/Plugin/LicenseCheck/Readme.md
> b/.pytool/Plugin/LicenseCheck/Readme.md
> > new file mode 100644
> > index 000000000000..b0e1bbea9e21
> > --- /dev/null
> > +++ b/.pytool/Plugin/LicenseCheck/Readme.md
> > @@ -0,0 +1,17 @@
> > +# License Check Plugin
> > +
> > +This CiBuildPlugin scans all new added files in a package to make
> > +sure code is contributed under BSD-2-Clause-Patent.
> > +
> > +## Configuration
> > +
> > +The plugin can be configured to ignore certain files.
> > +
> > +``` yaml
> > +"LicenseCheck": {
> > + "IgnoreFiles": []
> > +}
> > +```
> > +### IgnoreFiles
> > +
> > +OPTIONAL List of file to ignore.
> > --
> > 2.18.0.windows.1
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [edk2-devel] [PATCH v2 01/15] .pytool/Plugin: Add a plugin LicenseCheck
[not found] ` <162669507765F26E.19559@groups.io>
@ 2020-07-31 3:48 ` Liming Gao
0 siblings, 0 replies; 6+ messages in thread
From: Liming Gao @ 2020-07-31 3:48 UTC (permalink / raw)
To: devel@edk2.groups.io, Gao, Liming, Zhang, Shenglei
Cc: Sean Brogan, Bret Barkelew, Kinney, Michael D
Merged at e848b58d7c85293cd4121287abcea2d22a4f0620..7f79b736b0a57da71d87c987357db0227cd16ac6
Thanks
Liming
-----Original Message-----
From: devel@edk2.groups.io <devel@edk2.groups.io> On Behalf Of Liming Gao
Sent: 2020年7月30日 11:15
To: Zhang, Shenglei <shenglei.zhang@intel.com>; devel@edk2.groups.io
Cc: Sean Brogan <sean.brogan@microsoft.com>; Bret Barkelew <Bret.Barkelew@microsoft.com>; Kinney, Michael D <michael.d.kinney@intel.com>
Subject: Re: [edk2-devel] [PATCH v2 01/15] .pytool/Plugin: Add a plugin LicenseCheck
With this update, Reviewed-by: Liming Gao <liming.gao@intel.com>
If no other comment, I will merge this patch set tomorrow.
Thanks
Liming
-----Original Message-----
From: Zhang, Shenglei <shenglei.zhang@intel.com>
Sent: 2020年7月30日 11:08
To: Gao, Liming <liming.gao@intel.com>; devel@edk2.groups.io
Cc: Sean Brogan <sean.brogan@microsoft.com>; Bret Barkelew <Bret.Barkelew@microsoft.com>; Kinney, Michael D <michael.d.kinney@intel.com>
Subject: RE: [PATCH v2 01/15] .pytool/Plugin: Add a plugin LicenseCheck
> -----Original Message-----
> From: Gao, Liming <liming.gao@intel.com>
> Sent: Wednesday, July 29, 2020 10:02 PM
> To: Zhang, Shenglei <shenglei.zhang@intel.com>; devel@edk2.groups.io
> Cc: Sean Brogan <sean.brogan@microsoft.com>; Bret Barkelew
> <Bret.Barkelew@microsoft.com>; Kinney, Michael D
> <michael.d.kinney@intel.com>
> Subject: RE: [PATCH v2 01/15] .pytool/Plugin: Add a plugin
> LicenseCheck
>
> Shenglei:
>
> > -----Original Message-----
> > From: Zhang, Shenglei <shenglei.zhang@intel.com>
> > Sent: Wednesday, July 29, 2020 8:28 PM
> > To: devel@edk2.groups.io
> > Cc: Sean Brogan <sean.brogan@microsoft.com>; Bret Barkelew
> <Bret.Barkelew@microsoft.com>; Kinney, Michael D
> > <michael.d.kinney@intel.com>; Gao, Liming <liming.gao@intel.com>
> > Subject: [PATCH v2 01/15] .pytool/Plugin: Add a plugin LicenseCheck
> >
> > REF: https://bugzilla.tianocore.org/show_bug.cgi?id=2691
> > REF: https://bugzilla.tianocore.org/show_bug.cgi?id=2833
> > Add a plugin to check license conflict for new added files in a
> > patch. It will report out errors when meeting files which are now
> > contributed under BSD-2-Clause-Patent.
>
> files which are now contributed under BSD-2-Clause-Patent.
> ==>
> files which are not contributed under BSD-2-Clause-Patent.
>
Yes. It's typo.
Thanks,
Shenglei
> Thanks
> Liming
> >
> > Cc: Sean Brogan <sean.brogan@microsoft.com>
> > Cc: Bret Barkelew <Bret.Barkelew@microsoft.com>
> > Cc: Michael D Kinney <michael.d.kinney@intel.com>
> > Cc: Liming Gao <liming.gao@intel.com>
> > Signed-off-by: Shenglei Zhang <shenglei.zhang@intel.com>
> > ---
> > .pytool/Plugin/LicenseCheck/LicenseCheck.py | 115
> ++++++++++++++++++
> > .../LicenseCheck/LicenseCheck_plug_in.yaml | 11 ++
> > .pytool/Plugin/LicenseCheck/Readme.md | 17 +++
> > 3 files changed, 143 insertions(+)
> > create mode 100644 .pytool/Plugin/LicenseCheck/LicenseCheck.py
> > create mode
> 100644 .pytool/Plugin/LicenseCheck/LicenseCheck_plug_in.yaml
> > create mode 100644 .pytool/Plugin/LicenseCheck/Readme.md
> >
> > diff --git a/.pytool/Plugin/LicenseCheck/LicenseCheck.py
> b/.pytool/Plugin/LicenseCheck/LicenseCheck.py
> > new file mode 100644
> > index 000000000000..5733f7bf4ec0
> > --- /dev/null
> > +++ b/.pytool/Plugin/LicenseCheck/LicenseCheck.py
> > @@ -0,0 +1,115 @@
> > +# @file LicenseCheck.py
> > +#
> > +# Copyright (c) 2020, Intel Corporation. All rights reserved.<BR> #
> > +SPDX-License-Identifier: BSD-2-Clause-Patent ##
> > +
> > +import os
> > +import logging
> > +import re
> > +from io import StringIO
> > +from typing import List, Tuple
> > +from edk2toolext.environment.plugintypes.ci_build_plugin import
> ICiBuildPlugin
> > +from edk2toolext.environment.var_dict import VarDict from
> > +edk2toollib.utility_functions import RunCmd
> > +
> > +
> > +class LicenseCheck(ICiBuildPlugin):
> > +
> > + """
> > + A CiBuildPlugin to check the license for new added files.
> > +
> > + Configuration options:
> > + "LicenseCheck": {
> > + "IgnoreFiles": []
> > + },
> > + """
> > +
> > + license_format_preflix = 'SPDX-License-Identifier'
> > +
> > + bsd2_patent = 'BSD-2-Clause-Patent'
> > +
> > + Readdedfileformat = re.compile(r'\+\+\+ b\/(.*)')
> > +
> > + file_extension_list = [".c", ".h", ".inf", ".dsc", ".dec",
> > + ".py", ".bat", ".sh",
> ".uni", ".yaml",
> > + ".fdf", ".inc", "yml", ".asm", ".asm16",
> > + ".asl", ".vfr", ".s", ".S",
> ".aslc",
> > + ".nasm", ".nasmb", ".idf", ".Vfr", ".H"]
> > +
> > + def GetTestName(self, packagename: str, environment: VarDict)
> > + ->
> tuple:
> > + """ Provide the testcase name and classname for use in reporting
> > + testclassname: a descriptive string for the testcase
> > + can include
> whitespace
> > + classname: should be patterned
> <packagename>.<plugin>.<optionally any unique condition>
> > +
> > + Args:
> > + packagename: string containing name of package to build
> > + environment: The VarDict for the test to run in
> > + Returns:
> > + a tuple containing the testcase name and the classname
> > + (testcasename, classname)
> > + """
> > + return ("Check for license for " + packagename, packagename
> > + +
> ".LicenseCheck")
> > +
> > + ##
> > + # External function of plugin. This function is used to
> > + perform the task
> of the ci_build_plugin Plugin
> > + #
> > + # - package is the edk2 path to package. This means
> workspace/packagepath relative.
> > + # - edk2path object configured with workspace and packages path
> > + # - PkgConfig Object (dict) for the pkg
> > + # - EnvConfig Object
> > + # - Plugin Manager Instance
> > + # - Plugin Helper Obj Instance
> > + # - Junit Logger
> > + # - output_stream the StringIO output stream from this plugin via
> logging
> > + def RunBuildPlugin(self, packagename, Edk2pathObj, pkgconfig,
> environment, PLM, PLMHelper, tc, output_stream=None):
> > + return_buffer = StringIO()
> > + params = "diff --unified=0 origin/master HEAD"
> > + RunCmd("git", params, outstream=return_buffer)
> > + p = return_buffer.getvalue().strip()
> > + patch = p.split("\n")
> > + return_buffer.close()
> > +
> > + ignore_files = []
> > + if "IgnoreFiles" in pkgconfig:
> > + ignore_files = pkgconfig["IgnoreFiles"]
> > +
> > + self.ok = True
> > + self.startcheck = False
> > + self.license = True
> > + self.all_file_pass = True
> > + count = len(patch)
> > + line_index = 0
> > + for line in patch:
> > + if line.startswith('--- /dev/null'):
> > + nextline = patch[line_index + 1]
> > + added_file = self.Readdedfileformat.search(nextline).group(1)
> > + added_file_extension = os.path.splitext(added_file)[1]
> > + if added_file_extension in self.file_extension_list
> > + and
> packagename in added_file:
> > + if (self.IsIgnoreFile(added_file, ignore_files)):
> > + line_index = line_index + 1
> > + continue
> > + self.startcheck = True
> > + self.license = False
> > + if self.startcheck and self.license_format_preflix in line:
> > + if self.bsd2_patent in line:
> > + self.license = True
> > + if line_index + 1 == count or patch[line_index +
> > + 1].startswith('diff --')
> and self.startcheck:
> > + if not self.license:
> > + self.all_file_pass = False
> > + error_message = "Invalid license in: " +
> > + added_file + " Hint: Only
> BSD-2-Clause-Patent is accepted."
> > + logging.error(error_message)
> > + self.startcheck = False
> > + self.license = True
> > + line_index = line_index + 1
> > +
> > + if self.all_file_pass:
> > + tc.SetSuccess()
> > + return 0
> > + else:
> > + tc.SetFailed("License Check {0} Failed.
> > + ".format(packagename),
> "LICENSE_CHECK_FAILED")
> > + return 1
> > +
> > + def IsIgnoreFile(self, file: str, ignore_files: List[str]) -> bool:
> > + for f in ignore_files:
> > + if f in file:
> > + return True
> > + return False
> > diff --git a/.pytool/Plugin/LicenseCheck/LicenseCheck_plug_in.yaml
> b/.pytool/Plugin/LicenseCheck/LicenseCheck_plug_in.yaml
> > new file mode 100644
> > index 000000000000..2fdb0aa72359
> > --- /dev/null
> > +++ b/.pytool/Plugin/LicenseCheck/LicenseCheck_plug_in.yaml
> > @@ -0,0 +1,11 @@
> > +## @file
> > +# CiBuildPlugin used to check license issues for new added files #
> > +# Copyright (c) 2020, Intel Corporation. All rights reserved.
> > +# SPDX-License-Identifier: BSD-2-Clause-Patent ## {
> > + "scope": "cibuild",
> > + "name": "License Check Test",
> > + "module": "LicenseCheck"
> > +}
> > diff --git a/.pytool/Plugin/LicenseCheck/Readme.md
> b/.pytool/Plugin/LicenseCheck/Readme.md
> > new file mode 100644
> > index 000000000000..b0e1bbea9e21
> > --- /dev/null
> > +++ b/.pytool/Plugin/LicenseCheck/Readme.md
> > @@ -0,0 +1,17 @@
> > +# License Check Plugin
> > +
> > +This CiBuildPlugin scans all new added files in a package to make
> > +sure code is contributed under BSD-2-Clause-Patent.
> > +
> > +## Configuration
> > +
> > +The plugin can be configured to ignore certain files.
> > +
> > +``` yaml
> > +"LicenseCheck": {
> > + "IgnoreFiles": []
> > +}
> > +```
> > +### IgnoreFiles
> > +
> > +OPTIONAL List of file to ignore.
> > --
> > 2.18.0.windows.1
^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2020-07-31 3:48 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2020-07-29 12:28 [PATCHv v2 00/15] Add a plugin LicenseCheck in open ci Zhang, Shenglei
2020-07-29 12:28 ` [PATCH v2 01/15] .pytool/Plugin: Add a plugin LicenseCheck Zhang, Shenglei
2020-07-29 14:01 ` Liming Gao
2020-07-30 3:07 ` Zhang, Shenglei
2020-07-30 3:15 ` Liming Gao
[not found] ` <162669507765F26E.19559@groups.io>
2020-07-31 3:48 ` [edk2-devel] " Liming Gao
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox