From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mga04.intel.com (mga04.intel.com [192.55.52.120]) by mx.groups.io with SMTP id smtpd.web12.809.1596078470089913847 for ; Wed, 29 Jul 2020 20:07:50 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=@intel.onmicrosoft.com header.s=selector2-intel-onmicrosoft-com header.b=tcXF84C4; spf=pass (domain: intel.com, ip: 192.55.52.120, mailfrom: shenglei.zhang@intel.com) IronPort-SDR: OEU0zZDCeB4xeKSYbfyqM4io9rcnYv9ruxfWAW3Xz3SWbzXUeRWtlFeoScgzFu0jQZHNcqHclN Mnu/0hUGKd6w== X-IronPort-AV: E=McAfee;i="6000,8403,9697"; a="148994590" X-IronPort-AV: E=Sophos;i="5.75,412,1589266800"; d="scan'208";a="148994590" X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga006.jf.intel.com ([10.7.209.51]) by fmsmga104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 29 Jul 2020 20:07:48 -0700 IronPort-SDR: 6gNbZdG6ZkkT/1OP6Chbm7GFokjjm7D+gcS8Ow3wZtu1PvWAAJd45p0ILV9l3G0WLQSEJkhMZc XUq6EQS9fWCw== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.75,412,1589266800"; d="scan'208";a="290736566" Received: from orsmsx107.amr.corp.intel.com ([10.22.240.5]) by orsmga006.jf.intel.com with ESMTP; 29 Jul 2020 20:07:48 -0700 Received: from orsmsx112.amr.corp.intel.com (10.22.240.13) by ORSMSX107.amr.corp.intel.com (10.22.240.5) with Microsoft SMTP Server (TLS) id 14.3.439.0; Wed, 29 Jul 2020 20:07:47 -0700 Received: from ORSEDG001.ED.cps.intel.com (10.7.248.4) by ORSMSX112.amr.corp.intel.com (10.22.240.13) with Microsoft SMTP Server (TLS) id 14.3.439.0; Wed, 29 Jul 2020 20:07:47 -0700 Received: from NAM10-BN7-obe.outbound.protection.outlook.com (104.47.70.107) by edgegateway.intel.com (134.134.137.100) with Microsoft SMTP Server (TLS) id 14.3.439.0; Wed, 29 Jul 2020 20:07:47 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=W7EyUC4AjxXIZm14Sgk22HIWOfGZri6xOt3YR+WVr2y/y1xx53724VPOhWWRoL+q7ipYBS1BbtSGrwoQLXV0ZbQOEqGul9dIoqhOBY7qquZvMGPvZPjOxLjgB/zhKgAMDPVAeTNsw5Wyrn+hm2a9gH8EImMBUoeWdm7b9I0woUSvzXiUJZckQ6g1RHiKKDtd3ADobWAT7F+o68vPS8teguqmJnFoOaXbEBUJyXiLJcspfLBuIU4U9HMn7+ix4MkcwiAHYlC6S2in6X4yl4OgYmpjwNdN2h2INWZHgHkgzCH7Bg/m3yfoTseHky6X19ImECarna+c5jaos3z7dQ7ATA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=g8zEm7CKvKiRtY7l+sXjmlr19rL7NexY4Prmftifdfk=; b=KumOT/btsBNvuSzLimsqhEfQ1jwnUzcioMqFW27PW9rJk0VX7J+3tbSFOXnmpuUOBkCUYOuKjJHFnkdcYrLtJx4MP7DGnQXu8B9CA052zH8FZClaDojJkomq2g9SnI6w+Cy1soNnPxpomMys7/0z34JlPYeLqh5lqIIsGy3tCPguZfuf1yHHJLGXOFoPvVxiPiS+xYkOFkisa+bzQmwKh8Sf9+SbA9S/7NR64wexDQaH//5dPQieHd4JNG24yPhAeX7DRQJT9N/18+TsGn3jyNMZG0Zf/KA8Gak+KSoo73DnlsAHluvTyd1tpRWgHaC1jZjuo7qZZoqicg2vECjVEg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com; dkim=pass header.d=intel.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=intel.onmicrosoft.com; s=selector2-intel-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=g8zEm7CKvKiRtY7l+sXjmlr19rL7NexY4Prmftifdfk=; b=tcXF84C4eGRqEC16X+FJl3Xdt4k2/2Rl1PFUaWvxoDj3fh22H14+9DXpElAr64teYL6RgklCvqIB5XYHfJIEs/KZmTqlOHsmaRuOpGqEqMYG5oTHVOMiWXwJyOy8gm41+fMIaOT6TAe2jQ4yewKED2xxuDDBJlaCBajPnKIYpgQ= Received: from BL0PR11MB3506.namprd11.prod.outlook.com (2603:10b6:208:31::24) by MN2PR11MB4077.namprd11.prod.outlook.com (2603:10b6:208:13f::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3239.17; Thu, 30 Jul 2020 03:07:44 +0000 Received: from BL0PR11MB3506.namprd11.prod.outlook.com ([fe80::b805:cb86:230a:6a1]) by BL0PR11MB3506.namprd11.prod.outlook.com ([fe80::b805:cb86:230a:6a1%6]) with mapi id 15.20.3216.034; Thu, 30 Jul 2020 03:07:44 +0000 From: "Zhang, Shenglei" To: "Gao, Liming" , "devel@edk2.groups.io" CC: Sean Brogan , Bret Barkelew , "Kinney, Michael D" Subject: Re: [PATCH v2 01/15] .pytool/Plugin: Add a plugin LicenseCheck Thread-Topic: [PATCH v2 01/15] .pytool/Plugin: Add a plugin LicenseCheck Thread-Index: AQHWZaPGOLt+1Us2g0+u7v0sVrQer6keljyggADbRtA= Date: Thu, 30 Jul 2020 03:07:43 +0000 Message-ID: References: <20200729122806.21304-1-shenglei.zhang@intel.com> <20200729122806.21304-2-shenglei.zhang@intel.com> In-Reply-To: Accept-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: intel.com; dkim=none (message not signed) header.d=none;intel.com; dmarc=none action=none header.from=intel.com; x-originating-ip: [192.102.204.38] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 8eb05e6e-2677-4208-7e78-08d83435ba8e x-ms-traffictypediagnostic: MN2PR11MB4077: x-ms-exchange-transport-forked: True x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:9508; x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: w0zTk89U3n2SX1b+WWPRsIiOkIJadAqP6uGB07CxsoYDRlKeSLmeR9DDpLDYFEoP8hZdkCaT9XsVGPm+3l9y64u2ee631IBzjZ9Zp5QA72u3RpCA3WSRjnOqTwLiIQisOp2/SWIREScXiTVqJ3vGlHwE8pHskDWUXhXsVVoinLOalrb5TWwlKp29GBPbhgWyryEdwqAihaNgItncWhXDPH01j9eRi+lIc13PtTa1NFIvIf+/oMfNwkmLw/i/TwjLrYA0rJ3PDFimZtWYEJWmi9WKoYM8jG++8spEAdAYVOmjkbIkVzWeai0Yk5JDjSoQfjJ/dCuU7erxEm6pQfA0JsdNSLy7gV4oH/HHyIiYitNFhyIkiWN4KWhkQyjYpQPwBBu058Zeh1eDejL8Aw3OOg== x-forefront-antispam-report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BL0PR11MB3506.namprd11.prod.outlook.com;PTR:;CAT:NONE;SFTY:;SFS:(4636009)(376002)(39860400002)(366004)(136003)(396003)(346002)(8676002)(33656002)(83380400001)(110136005)(8936002)(316002)(966005)(4326008)(54906003)(71200400001)(107886003)(6506007)(26005)(76116006)(52536014)(55016002)(5660300002)(66476007)(86362001)(66446008)(53546011)(66556008)(7696005)(2906002)(478600001)(9686003)(186003)(66946007)(64756008);DIR:OUT;SFP:1102; x-ms-exchange-antispam-messagedata: LiH23S/Ss7PxUMsVN8jf0N0kaiAZl2alOLH61z+VwQrYE4IKWL+g3arMxyKofJuK8jvW60fdBUZc0fUVy5QfEnpFKadIgmm9N+qmShNu/Z1O04lc/HLGM/qDUaPf97T/5O5PoFGLs5YEAqB6qAYoi1uuVmanq02wwnUdR7GQIv5gffQXlPRWG1rA9d7kfD4aGxDktk51HUFJi78xhPl8JgfVRiUUVWcN8mE+fQLV7XcZyJ48nr36GqVRzkdJ0ELZ8u0KCbVB1kWMAlc7KDMq//8Leuf+zK2PbTCbdHEpA7XIZ9iHQEapHXmYA7VcrAtqBgd1JBzd0RlDIxDzbtOB1F6Mrr0gxy2hsdfRvLm/8fxUAQtcibY7O17zlWoacjI9LomOxkCP4eslS3qvW+tPjI2YWEmJrc81ch9riKARKb80vhaC3SDkMzvtLr3ZQlgannIph4v4zsz5xN87stLkLcJv12pe1yqK0TJ02IFrgEL7RR0jjGZ7Mv0FyQAhOO+kikzbmG1UtX8VWAYvpVcrbL1YuIIw5/v5MHBFHD47DOhe2iEYLMsbtir0+G5Ki4SJlcGsMbV63sMn9u/Kc279i8iDnwqh/L9uMpOQUr3sUHD+Bpzv4LbPjeY+thfbgCZqcsz/NpiMfeAmNHABg+N01g== MIME-Version: 1.0 X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: BL0PR11MB3506.namprd11.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: 8eb05e6e-2677-4208-7e78-08d83435ba8e X-MS-Exchange-CrossTenant-originalarrivaltime: 30 Jul 2020 03:07:43.8308 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 46c98d88-e344-4ed4-8496-4ed7712e255d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: dqiYaBGC03EXgsBWnZ41N04q+17/0E2hrjpnC2deyriFuNHs4O6+1JyAGuPENMen0D7zcUIJgQh/9bPifiYaTckdp8XiobCztnbIDc0jxao= X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR11MB4077 Return-Path: shenglei.zhang@intel.com X-OriginatorOrg: intel.com Content-Language: en-US Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable > -----Original Message----- > From: Gao, Liming > Sent: Wednesday, July 29, 2020 10:02 PM > To: Zhang, Shenglei ; devel@edk2.groups.io > Cc: Sean Brogan ; Bret Barkelew > ; Kinney, Michael D > > Subject: RE: [PATCH v2 01/15] .pytool/Plugin: Add a plugin LicenseCheck >=20 > Shenglei: >=20 > > -----Original Message----- > > From: Zhang, Shenglei > > Sent: Wednesday, July 29, 2020 8:28 PM > > To: devel@edk2.groups.io > > Cc: Sean Brogan ; Bret Barkelew > ; Kinney, Michael D > > ; Gao, Liming > > Subject: [PATCH v2 01/15] .pytool/Plugin: Add a plugin LicenseCheck > > > > REF: https://bugzilla.tianocore.org/show_bug.cgi?id=3D2691 > > REF: https://bugzilla.tianocore.org/show_bug.cgi?id=3D2833 > > Add a plugin to check license conflict for new added > > files in a patch. It will report out errors when meeting > > files which are now contributed under BSD-2-Clause-Patent. >=20 > files which are now contributed under BSD-2-Clause-Patent. > =3D=3D> > files which are not contributed under BSD-2-Clause-Patent. >=20 Yes. It's typo. Thanks, Shenglei > Thanks > Liming > > > > Cc: Sean Brogan > > Cc: Bret Barkelew > > Cc: Michael D Kinney > > Cc: Liming Gao > > Signed-off-by: Shenglei Zhang > > --- > > .pytool/Plugin/LicenseCheck/LicenseCheck.py | 115 > ++++++++++++++++++ > > .../LicenseCheck/LicenseCheck_plug_in.yaml | 11 ++ > > .pytool/Plugin/LicenseCheck/Readme.md | 17 +++ > > 3 files changed, 143 insertions(+) > > create mode 100644 .pytool/Plugin/LicenseCheck/LicenseCheck.py > > create mode > 100644 .pytool/Plugin/LicenseCheck/LicenseCheck_plug_in.yaml > > create mode 100644 .pytool/Plugin/LicenseCheck/Readme.md > > > > diff --git a/.pytool/Plugin/LicenseCheck/LicenseCheck.py > b/.pytool/Plugin/LicenseCheck/LicenseCheck.py > > new file mode 100644 > > index 000000000000..5733f7bf4ec0 > > --- /dev/null > > +++ b/.pytool/Plugin/LicenseCheck/LicenseCheck.py > > @@ -0,0 +1,115 @@ > > +# @file LicenseCheck.py > > +# > > +# Copyright (c) 2020, Intel Corporation. All rights reserved.
> > +# SPDX-License-Identifier: BSD-2-Clause-Patent > > +## > > + > > +import os > > +import logging > > +import re > > +from io import StringIO > > +from typing import List, Tuple > > +from edk2toolext.environment.plugintypes.ci_build_plugin import > ICiBuildPlugin > > +from edk2toolext.environment.var_dict import VarDict > > +from edk2toollib.utility_functions import RunCmd > > + > > + > > +class LicenseCheck(ICiBuildPlugin): > > + > > + """ > > + A CiBuildPlugin to check the license for new added files. > > + > > + Configuration options: > > + "LicenseCheck": { > > + "IgnoreFiles": [] > > + }, > > + """ > > + > > + license_format_preflix =3D 'SPDX-License-Identifier' > > + > > + bsd2_patent =3D 'BSD-2-Clause-Patent' > > + > > + Readdedfileformat =3D re.compile(r'\+\+\+ b\/(.*)') > > + > > + file_extension_list =3D [".c", ".h", ".inf", ".dsc", ".dec", ".py"= , ".bat", ".sh", > ".uni", ".yaml", > > + ".fdf", ".inc", "yml", ".asm", ".asm16", ".= asl", ".vfr", ".s", ".S", > ".aslc", > > + ".nasm", ".nasmb", ".idf", ".Vfr", ".H"] > > + > > + def GetTestName(self, packagename: str, environment: VarDict) -> > tuple: > > + """ Provide the testcase name and classname for use in reporti= ng > > + testclassname: a descriptive string for the testcase can i= nclude > whitespace > > + classname: should be patterned > .. > > + > > + Args: > > + packagename: string containing name of package to build > > + environment: The VarDict for the test to run in > > + Returns: > > + a tuple containing the testcase name and the classname > > + (testcasename, classname) > > + """ > > + return ("Check for license for " + packagename, packagename + > ".LicenseCheck") > > + > > + ## > > + # External function of plugin. This function is used to perform t= he task > of the ci_build_plugin Plugin > > + # > > + # - package is the edk2 path to package. This means > workspace/packagepath relative. > > + # - edk2path object configured with workspace and packages path > > + # - PkgConfig Object (dict) for the pkg > > + # - EnvConfig Object > > + # - Plugin Manager Instance > > + # - Plugin Helper Obj Instance > > + # - Junit Logger > > + # - output_stream the StringIO output stream from this plugin vi= a > logging > > + def RunBuildPlugin(self, packagename, Edk2pathObj, pkgconfig, > environment, PLM, PLMHelper, tc, output_stream=3DNone): > > + return_buffer =3D StringIO() > > + params =3D "diff --unified=3D0 origin/master HEAD" > > + RunCmd("git", params, outstream=3Dreturn_buffer) > > + p =3D return_buffer.getvalue().strip() > > + patch =3D p.split("\n") > > + return_buffer.close() > > + > > + ignore_files =3D [] > > + if "IgnoreFiles" in pkgconfig: > > + ignore_files =3D pkgconfig["IgnoreFiles"] > > + > > + self.ok =3D True > > + self.startcheck =3D False > > + self.license =3D True > > + self.all_file_pass =3D True > > + count =3D len(patch) > > + line_index =3D 0 > > + for line in patch: > > + if line.startswith('--- /dev/null'): > > + nextline =3D patch[line_index + 1] > > + added_file =3D self.Readdedfileformat.search(nextline)= .group(1) > > + added_file_extension =3D os.path.splitext(added_file)[= 1] > > + if added_file_extension in self.file_extension_list an= d > packagename in added_file: > > + if (self.IsIgnoreFile(added_file, ignore_files)): > > + line_index =3D line_index + 1 > > + continue > > + self.startcheck =3D True > > + self.license =3D False > > + if self.startcheck and self.license_format_preflix in line= : > > + if self.bsd2_patent in line: > > + self.license =3D True > > + if line_index + 1 =3D=3D count or patch[line_index + 1].st= artswith('diff --') > and self.startcheck: > > + if not self.license: > > + self.all_file_pass =3D False > > + error_message =3D "Invalid license in: " + added_f= ile + " Hint: Only > BSD-2-Clause-Patent is accepted." > > + logging.error(error_message) > > + self.startcheck =3D False > > + self.license =3D True > > + line_index =3D line_index + 1 > > + > > + if self.all_file_pass: > > + tc.SetSuccess() > > + return 0 > > + else: > > + tc.SetFailed("License Check {0} Failed. ".format(packagena= me), > "LICENSE_CHECK_FAILED") > > + return 1 > > + > > + def IsIgnoreFile(self, file: str, ignore_files: List[str]) -> bool= : > > + for f in ignore_files: > > + if f in file: > > + return True > > + return False > > diff --git a/.pytool/Plugin/LicenseCheck/LicenseCheck_plug_in.yaml > b/.pytool/Plugin/LicenseCheck/LicenseCheck_plug_in.yaml > > new file mode 100644 > > index 000000000000..2fdb0aa72359 > > --- /dev/null > > +++ b/.pytool/Plugin/LicenseCheck/LicenseCheck_plug_in.yaml > > @@ -0,0 +1,11 @@ > > +## @file > > +# CiBuildPlugin used to check license issues for new added files > > +# > > +# Copyright (c) 2020, Intel Corporation. All rights reserved. > > +# SPDX-License-Identifier: BSD-2-Clause-Patent > > +## > > +{ > > + "scope": "cibuild", > > + "name": "License Check Test", > > + "module": "LicenseCheck" > > +} > > diff --git a/.pytool/Plugin/LicenseCheck/Readme.md > b/.pytool/Plugin/LicenseCheck/Readme.md > > new file mode 100644 > > index 000000000000..b0e1bbea9e21 > > --- /dev/null > > +++ b/.pytool/Plugin/LicenseCheck/Readme.md > > @@ -0,0 +1,17 @@ > > +# License Check Plugin > > + > > +This CiBuildPlugin scans all new added files in a package to make sure= code > > +is contributed under BSD-2-Clause-Patent. > > + > > +## Configuration > > + > > +The plugin can be configured to ignore certain files. > > + > > +``` yaml > > +"LicenseCheck": { > > + "IgnoreFiles": [] > > +} > > +``` > > +### IgnoreFiles > > + > > +OPTIONAL List of file to ignore. > > -- > > 2.18.0.windows.1