From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mga07.intel.com (mga07.intel.com [134.134.136.100]) by mx.groups.io with SMTP id smtpd.web10.6699.1596031322264001524 for ; Wed, 29 Jul 2020 07:02:02 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=@intel.onmicrosoft.com header.s=selector2-intel-onmicrosoft-com header.b=y56NTdYT; spf=pass (domain: intel.com, ip: 134.134.136.100, mailfrom: liming.gao@intel.com) IronPort-SDR: kKxzd++b6tgpZBsWr62WdX0AEOftA1zmXXoBbonr44BSsI7N9uHHo+9IN0CJsrJi506nvEr4c+ OqKx0kQn2ePA== X-IronPort-AV: E=McAfee;i="6000,8403,9696"; a="215911513" X-IronPort-AV: E=Sophos;i="5.75,410,1589266800"; d="scan'208";a="215911513" X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga008.fm.intel.com ([10.253.24.58]) by orsmga105.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 29 Jul 2020 07:02:00 -0700 IronPort-SDR: xnjt8OQHdpP1CjYm278d5yvkCFZPz19J8hqIHBI3JeaY9D+MBCAwo4J95kz8tOauHnfeK+sAOC AEhXEKgDuCpw== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.75,410,1589266800"; d="scan'208";a="273891929" Received: from orsmsx102.amr.corp.intel.com ([10.22.225.129]) by fmsmga008.fm.intel.com with ESMTP; 29 Jul 2020 07:02:00 -0700 Received: from orsmsx162.amr.corp.intel.com (10.22.240.85) by ORSMSX102.amr.corp.intel.com (10.22.225.129) with Microsoft SMTP Server (TLS) id 14.3.439.0; Wed, 29 Jul 2020 07:02:00 -0700 Received: from ORSEDG002.ED.cps.intel.com (10.7.248.5) by ORSMSX162.amr.corp.intel.com (10.22.240.85) with Microsoft SMTP Server (TLS) id 14.3.439.0; Wed, 29 Jul 2020 07:02:00 -0700 Received: from NAM12-BN8-obe.outbound.protection.outlook.com (104.47.55.177) by edgegateway.intel.com (134.134.137.101) with Microsoft SMTP Server (TLS) id 14.3.439.0; Wed, 29 Jul 2020 07:01:59 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Tgu4pOh2CbYMBvBZsHxNmtTSL279yevLKFr6zdywCzw+OlCO9uaqm7XRQ6yrSfcGZknKOO0cDUJjRESFGbmvQHcHwOhnKnOt0azqtxfBfhKGiyBmyEZZC2X42qjwWeha/dNrNOxgD9ez15HLrfeEgEcrfrQAzGB1clLhvxLpORH2Elzkg5XuqXnXz/L5upx1rA7byuJ+21ld4bwswLrkvazi4q1mxrNvgil1t3gfDUSDuSJ4/OCZX0FPJkQvkgIEU6O2DwDbTeqEGQWV9ld003IP7deci4Uag5IS5Gc03jqEJcSCMNZo4zRp8I/j06pqI1jQFT/tRKLLsuqL0uGGFA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=HPLIGjuQIPS6PO8XUWPYtVkxtQrx32C8HVa/C/Lk+GA=; b=RGE5K6pJ3vKFoS511S2y9icDMt/1K9RSkPE9GKvHgSY0JxYXg1vadyyQbLL8WQZY5f4rk/KmPNLvEqPSfMJs+KD2bkMF1gSs8gowdjKIIZe2OcsmkM4cdESoW1f5090b12dwa2AKCt+ctHME3DxILsisKbiW5X/KztGu+qNUPKGtKlL67YJjFvdkLqA6Xva1D2KaEVYZ8aBiiU0XaHewLcjh/L27iB5Q8Cwdh+563tnvXuA23PiaKEmgJuI7Qly6MkWy/FIphJn5Z4K/+D6zexwDKO2NnfT64SsVvGJYsGBszyuElc2EVVtSZR639wmMqWDSHdZc4Z/fxfZqxz10ag== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com; dkim=pass header.d=intel.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=intel.onmicrosoft.com; s=selector2-intel-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=HPLIGjuQIPS6PO8XUWPYtVkxtQrx32C8HVa/C/Lk+GA=; b=y56NTdYTn5NgSyMsZds/UbDUXnSaHM2MHvuh3h0TKjmiofx9M05NsvOKoms1WtUB9ytn+/6ikkYzbrplNY71YFxqR6yQh6FlJlVhf/tAUXM94AsMeLhuE/4MXFTIqfVes/e8sZ5IORrMGscC9qUV2Hjy67dfu7ihy7T3fL/Hoqk= Received: from CY4PR11MB1526.namprd11.prod.outlook.com (2603:10b6:910:7::10) by CY4PR11MB1831.namprd11.prod.outlook.com (2603:10b6:903:11d::17) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3239.16; Wed, 29 Jul 2020 14:01:58 +0000 Received: from CY4PR11MB1526.namprd11.prod.outlook.com ([fe80::f8c4:711:84e3:4c97]) by CY4PR11MB1526.namprd11.prod.outlook.com ([fe80::f8c4:711:84e3:4c97%4]) with mapi id 15.20.3216.034; Wed, 29 Jul 2020 14:01:58 +0000 From: "Liming Gao" To: "Zhang, Shenglei" , "devel@edk2.groups.io" CC: Sean Brogan , Bret Barkelew , "Kinney, Michael D" Subject: Re: [PATCH v2 01/15] .pytool/Plugin: Add a plugin LicenseCheck Thread-Topic: [PATCH v2 01/15] .pytool/Plugin: Add a plugin LicenseCheck Thread-Index: AQHWZaPGOLt+1Us2g0+u7v0sVrQer6keljyg Date: Wed, 29 Jul 2020 14:01:57 +0000 Message-ID: References: <20200729122806.21304-1-shenglei.zhang@intel.com> <20200729122806.21304-2-shenglei.zhang@intel.com> In-Reply-To: <20200729122806.21304-2-shenglei.zhang@intel.com> Accept-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: dlp-version: 11.2.0.6 dlp-product: dlpe-windows dlp-reaction: no-action authentication-results: intel.com; dkim=none (message not signed) header.d=none;intel.com; dmarc=none action=none header.from=intel.com; x-originating-ip: [192.198.147.210] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: c3a2e8bc-0c66-4ff9-5229-08d833c7f56f x-ms-traffictypediagnostic: CY4PR11MB1831: x-ms-exchange-transport-forked: True x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:9508; x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: o2X1mdbxJtdlnXbQeKRvfzN4GP8wWpFrNtKnNJscLXv9Bx1EVcbAudYuqHblr4JTQUWL4oR0XDxBcQtXfxIVLBlsc7oJ+Da/yQN1kEoOqclHkYLSaX9tUuvt5D5n2z/U7brP1qxq8caMBofc+G7IpNjqRg6v+Xw3578f9MlBkaDrdlp0WRWtDcK7gjgrmu8PQFW8qm/KsG71mQl1aRPeA+W1z9I+JaypjdsYS4g+WH//rfZIeHpZekCRv/OGmmXgmLvgW1u+wH7gz9qZzNJnwdcb6P9D9pfnbVh+vj03xB/dLBKxloywcRtnVPjBlRsE1Ys8uEoyc0kc9U6DKLwrHA3lm456eAu8dLTwnqJPi3pq7lIHySJ3y/k+QU3qwzzBB3IVgGHMAdq/37uGu3GAIA== x-forefront-antispam-report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:CY4PR11MB1526.namprd11.prod.outlook.com;PTR:;CAT:NONE;SFTY:;SFS:(4636009)(39860400002)(366004)(376002)(136003)(396003)(346002)(966005)(83380400001)(53546011)(6506007)(5660300002)(26005)(7696005)(8676002)(186003)(76116006)(110136005)(66946007)(33656002)(66476007)(66446008)(52536014)(71200400001)(86362001)(107886003)(64756008)(4326008)(478600001)(54906003)(2906002)(9686003)(316002)(8936002)(66556008)(55016002);DIR:OUT;SFP:1102; x-ms-exchange-antispam-messagedata: vdC0TnvwoKEzT7fVDnZWqTFZcxcKgAVOuNHQPqPUlaODBrwTAN56mIiE+ariWP0sZLQteanhCCG+4Pc/oH1hsLazGmCPa6v+i6MXmw9P33cD4/gAm8dWLS3umCFukdQ9hj81ibM5AQpZ8WAQk9d5vZOsnqQhMJtzAbj2124LUPXHPySIrr+cZENp+M1ba6zCI8xw11SH/AfX23VdzOdyPqS0FLGv0+D7tDC8Bm/3QoP1isQuMcaXjDH/oBv8Wr7WdYsqf9TxgoV13FkRXwLss0YfGXWVwR4VW58a023Ccc8b1GurunipkhpMQq4CoYk/DIayQrDD3Ii96apK2E1tOWiWtMyRHqtuPAis0InX1jUj+91sgqxKMbWBDMapaXTcS+ACYf3nOZHQHOt/yGNx0oXoUafJoxlZ363c5S4G7/prKAE7x1iw/+8ufTvV+FfaudO1i+D9m6AP/fEeWAaZBLM9OVXPhdDh1PUGUngkBWzmd3+rDvjyAm9SJDp1BbLK/fbzjJKC+Wq43G7fZ9gvIwKnsrE4OXaqfe9it2ZyXTpA6EnU+SjVurySFtBDGFd734hxceHoY92U+Gst+80VRXi4N4rHlFix8y7eFDHMotFdiijklRDaRvHuCp/5PQ285G2JfdVCI87zzfFA10LMAQ== MIME-Version: 1.0 X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: CY4PR11MB1526.namprd11.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: c3a2e8bc-0c66-4ff9-5229-08d833c7f56f X-MS-Exchange-CrossTenant-originalarrivaltime: 29 Jul 2020 14:01:57.9908 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 46c98d88-e344-4ed4-8496-4ed7712e255d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: j/HR6ALfkEprnuDSD89LFt4U1YUy9X0bUpzV+arFfBPwadl9iApUQWqpjhRPHEbRraItxIVdeQvxYqxudp0Azg== X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY4PR11MB1831 Return-Path: liming.gao@intel.com X-OriginatorOrg: intel.com Content-Language: en-US Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Shenglei: > -----Original Message----- > From: Zhang, Shenglei > Sent: Wednesday, July 29, 2020 8:28 PM > To: devel@edk2.groups.io > Cc: Sean Brogan ; Bret Barkelew ; Kinney, Michael D > ; Gao, Liming > Subject: [PATCH v2 01/15] .pytool/Plugin: Add a plugin LicenseCheck >=20 > REF: https://bugzilla.tianocore.org/show_bug.cgi?id=3D2691 > REF: https://bugzilla.tianocore.org/show_bug.cgi?id=3D2833 > Add a plugin to check license conflict for new added > files in a patch. It will report out errors when meeting > files which are now contributed under BSD-2-Clause-Patent. files which are now contributed under BSD-2-Clause-Patent. =3D=3D> files which are not contributed under BSD-2-Clause-Patent. Thanks Liming >=20 > Cc: Sean Brogan > Cc: Bret Barkelew > Cc: Michael D Kinney > Cc: Liming Gao > Signed-off-by: Shenglei Zhang > --- > .pytool/Plugin/LicenseCheck/LicenseCheck.py | 115 ++++++++++++++++++ > .../LicenseCheck/LicenseCheck_plug_in.yaml | 11 ++ > .pytool/Plugin/LicenseCheck/Readme.md | 17 +++ > 3 files changed, 143 insertions(+) > create mode 100644 .pytool/Plugin/LicenseCheck/LicenseCheck.py > create mode 100644 .pytool/Plugin/LicenseCheck/LicenseCheck_plug_in.yaml > create mode 100644 .pytool/Plugin/LicenseCheck/Readme.md >=20 > diff --git a/.pytool/Plugin/LicenseCheck/LicenseCheck.py b/.pytool/Plugin= /LicenseCheck/LicenseCheck.py > new file mode 100644 > index 000000000000..5733f7bf4ec0 > --- /dev/null > +++ b/.pytool/Plugin/LicenseCheck/LicenseCheck.py > @@ -0,0 +1,115 @@ > +# @file LicenseCheck.py > +# > +# Copyright (c) 2020, Intel Corporation. All rights reserved.
> +# SPDX-License-Identifier: BSD-2-Clause-Patent > +## > + > +import os > +import logging > +import re > +from io import StringIO > +from typing import List, Tuple > +from edk2toolext.environment.plugintypes.ci_build_plugin import ICiBuild= Plugin > +from edk2toolext.environment.var_dict import VarDict > +from edk2toollib.utility_functions import RunCmd > + > + > +class LicenseCheck(ICiBuildPlugin): > + > + """ > + A CiBuildPlugin to check the license for new added files. > + > + Configuration options: > + "LicenseCheck": { > + "IgnoreFiles": [] > + }, > + """ > + > + license_format_preflix =3D 'SPDX-License-Identifier' > + > + bsd2_patent =3D 'BSD-2-Clause-Patent' > + > + Readdedfileformat =3D re.compile(r'\+\+\+ b\/(.*)') > + > + file_extension_list =3D [".c", ".h", ".inf", ".dsc", ".dec", ".py", = ".bat", ".sh", ".uni", ".yaml", > + ".fdf", ".inc", "yml", ".asm", ".asm16", ".as= l", ".vfr", ".s", ".S", ".aslc", > + ".nasm", ".nasmb", ".idf", ".Vfr", ".H"] > + > + def GetTestName(self, packagename: str, environment: VarDict) -> tup= le: > + """ Provide the testcase name and classname for use in reporting > + testclassname: a descriptive string for the testcase can inc= lude whitespace > + classname: should be patterned .. > + > + Args: > + packagename: string containing name of package to build > + environment: The VarDict for the test to run in > + Returns: > + a tuple containing the testcase name and the classname > + (testcasename, classname) > + """ > + return ("Check for license for " + packagename, packagename + ".= LicenseCheck") > + > + ## > + # External function of plugin. This function is used to perform the= task of the ci_build_plugin Plugin > + # > + # - package is the edk2 path to package. This means workspace/pac= kagepath relative. > + # - edk2path object configured with workspace and packages path > + # - PkgConfig Object (dict) for the pkg > + # - EnvConfig Object > + # - Plugin Manager Instance > + # - Plugin Helper Obj Instance > + # - Junit Logger > + # - output_stream the StringIO output stream from this plugin via = logging > + def RunBuildPlugin(self, packagename, Edk2pathObj, pkgconfig, enviro= nment, PLM, PLMHelper, tc, output_stream=3DNone): > + return_buffer =3D StringIO() > + params =3D "diff --unified=3D0 origin/master HEAD" > + RunCmd("git", params, outstream=3Dreturn_buffer) > + p =3D return_buffer.getvalue().strip() > + patch =3D p.split("\n") > + return_buffer.close() > + > + ignore_files =3D [] > + if "IgnoreFiles" in pkgconfig: > + ignore_files =3D pkgconfig["IgnoreFiles"] > + > + self.ok =3D True > + self.startcheck =3D False > + self.license =3D True > + self.all_file_pass =3D True > + count =3D len(patch) > + line_index =3D 0 > + for line in patch: > + if line.startswith('--- /dev/null'): > + nextline =3D patch[line_index + 1] > + added_file =3D self.Readdedfileformat.search(nextline).g= roup(1) > + added_file_extension =3D os.path.splitext(added_file)[1] > + if added_file_extension in self.file_extension_list and = packagename in added_file: > + if (self.IsIgnoreFile(added_file, ignore_files)): > + line_index =3D line_index + 1 > + continue > + self.startcheck =3D True > + self.license =3D False > + if self.startcheck and self.license_format_preflix in line: > + if self.bsd2_patent in line: > + self.license =3D True > + if line_index + 1 =3D=3D count or patch[line_index + 1].star= tswith('diff --') and self.startcheck: > + if not self.license: > + self.all_file_pass =3D False > + error_message =3D "Invalid license in: " + added_fil= e + " Hint: Only BSD-2-Clause-Patent is accepted." > + logging.error(error_message) > + self.startcheck =3D False > + self.license =3D True > + line_index =3D line_index + 1 > + > + if self.all_file_pass: > + tc.SetSuccess() > + return 0 > + else: > + tc.SetFailed("License Check {0} Failed. ".format(packagename= ), "LICENSE_CHECK_FAILED") > + return 1 > + > + def IsIgnoreFile(self, file: str, ignore_files: List[str]) -> bool: > + for f in ignore_files: > + if f in file: > + return True > + return False > diff --git a/.pytool/Plugin/LicenseCheck/LicenseCheck_plug_in.yaml b/.pyt= ool/Plugin/LicenseCheck/LicenseCheck_plug_in.yaml > new file mode 100644 > index 000000000000..2fdb0aa72359 > --- /dev/null > +++ b/.pytool/Plugin/LicenseCheck/LicenseCheck_plug_in.yaml > @@ -0,0 +1,11 @@ > +## @file > +# CiBuildPlugin used to check license issues for new added files > +# > +# Copyright (c) 2020, Intel Corporation. All rights reserved. > +# SPDX-License-Identifier: BSD-2-Clause-Patent > +## > +{ > + "scope": "cibuild", > + "name": "License Check Test", > + "module": "LicenseCheck" > +} > diff --git a/.pytool/Plugin/LicenseCheck/Readme.md b/.pytool/Plugin/Licen= seCheck/Readme.md > new file mode 100644 > index 000000000000..b0e1bbea9e21 > --- /dev/null > +++ b/.pytool/Plugin/LicenseCheck/Readme.md > @@ -0,0 +1,17 @@ > +# License Check Plugin > + > +This CiBuildPlugin scans all new added files in a package to make sure c= ode > +is contributed under BSD-2-Clause-Patent. > + > +## Configuration > + > +The plugin can be configured to ignore certain files. > + > +``` yaml > +"LicenseCheck": { > + "IgnoreFiles": [] > +} > +``` > +### IgnoreFiles > + > +OPTIONAL List of file to ignore. > -- > 2.18.0.windows.1