From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mga09.intel.com (mga09.intel.com [134.134.136.24]) by mx.groups.io with SMTP id smtpd.web10.11168.1596167330690851812 for ; Thu, 30 Jul 2020 20:48:50 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=@intel.onmicrosoft.com header.s=selector2-intel-onmicrosoft-com header.b=Tqjzfxt0; spf=pass (domain: intel.com, ip: 134.134.136.24, mailfrom: liming.gao@intel.com) IronPort-SDR: ZEVjPfpi0fs+Nl0fBxnGYOZPSm+zAdU5BuZ8baWMrAu91AImgxVspbRLYCbkPyDBPh0tcvGaCY n74WsUio61HQ== X-IronPort-AV: E=McAfee;i="6000,8403,9698"; a="152938319" X-IronPort-AV: E=Sophos;i="5.75,416,1589266800"; d="scan'208";a="152938319" X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga008.jf.intel.com ([10.7.209.65]) by orsmga102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 30 Jul 2020 20:48:50 -0700 IronPort-SDR: R5ASy4qaAJLf1v/w8qfPdft5M2lUzsZ8htj2dc3AQ582RhTBJ/WTChoLrIBz0Ega5PbnItduVE /iVIh6QyVmWQ== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.75,416,1589266800"; d="scan'208";a="321291225" Received: from fmsmsx105.amr.corp.intel.com ([10.18.124.203]) by orsmga008.jf.intel.com with ESMTP; 30 Jul 2020 20:48:50 -0700 Received: from FMSEDG001.ED.cps.intel.com (10.1.192.133) by FMSMSX105.amr.corp.intel.com (10.18.124.203) with Microsoft SMTP Server (TLS) id 14.3.439.0; Thu, 30 Jul 2020 20:48:49 -0700 Received: from NAM04-SN1-obe.outbound.protection.outlook.com (104.47.44.58) by edgegateway.intel.com (192.55.55.68) with Microsoft SMTP Server (TLS) id 14.3.439.0; Thu, 30 Jul 2020 20:48:49 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=jFAQHMbZ8acmOH2HLt0Z+0aAa9h2o5bMAKHp9Ik60CvwSnhx5HvYbqdxlBf8itLch9pjpc54ijlgmARUl0m5+GeUDGGgLIInwCXHp2o7nbqGPCVdkvEDY47xTk8oxoT9oVluoXk2NCzJrGOlb4QVSJraqH2IPMdwNnZpOqVGGCigKL/ODD2LMbqjGreNDOYPlxnIhSPQz14+6Zie5kGNaikRncGEWxn6gZzdmm+WSzFa4aa1UiFTvawRPV1sw8EsqCZZEI3QnSd19Rrv1CDC3EqF1022Y2R8j9fJtngtKXlTBo+AhvaOsXDCbqDVLYCMlCJMxKLM9SWsoLt3ML8PGw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=RtWpG4eTsQ5zHkmUEil5jqKZ8j/ZtXe11X7c1lXt9Q8=; b=fcqGDIlCDo5LsIkjwVK8+gRW/d1YBEPzYj1+TIerF/1Qcb5kNxqQjFhX+k44hSOjh7Nam5EH7e6kAqF+GZ1kTqK7WklZR8gYwOWY+8rCklp+0lEPbbMkXySk37xRkdVAiphFcbdQJgZZj5g+EvbkUBWnwnPKIuI6WcZYaqiS2zL6qw5j+ZB0dHoanc04boHlDT+I5RzKk9T3YTh/oC/Mj8eOytHx9pFnUdoPftVRSZ+0N+DXySfc7xX/e4UMtwvj5kuny9kXiqCLCXmS0I1QPsVj3eVyyXRheWs6i0G9dFoPAvZpMghJ3MZ9fS7AFdIAdq25H4HBFzsjhwhgJROEoQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com; dkim=pass header.d=intel.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=intel.onmicrosoft.com; s=selector2-intel-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=RtWpG4eTsQ5zHkmUEil5jqKZ8j/ZtXe11X7c1lXt9Q8=; b=Tqjzfxt03kpcUvBOFAFU3BtolyKyqL1bn+ySaqD/UdUTAN7YtNpr7IdJ3Qd79LchR6I6ZqcDCXRQN+B6/g8yo/0xlpxMkzIJAkBv3aYh6sBUD9/6ytrKoydqawvOIDxWU6Pu71OK262R9qnIfRXszNmI80LA3THkkowgTKFWyB0= Received: from CY4PR11MB1526.namprd11.prod.outlook.com (2603:10b6:910:7::10) by CY4PR1101MB2293.namprd11.prod.outlook.com (2603:10b6:910:22::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3216.24; Fri, 31 Jul 2020 03:48:48 +0000 Received: from CY4PR11MB1526.namprd11.prod.outlook.com ([fe80::f8c4:711:84e3:4c97]) by CY4PR11MB1526.namprd11.prod.outlook.com ([fe80::f8c4:711:84e3:4c97%4]) with mapi id 15.20.3216.034; Fri, 31 Jul 2020 03:48:48 +0000 From: "Liming Gao" To: "devel@edk2.groups.io" , "Gao, Liming" , "Zhang, Shenglei" CC: Sean Brogan , Bret Barkelew , "Kinney, Michael D" Subject: Re: [edk2-devel] [PATCH v2 01/15] .pytool/Plugin: Add a plugin LicenseCheck Thread-Topic: [edk2-devel] [PATCH v2 01/15] .pytool/Plugin: Add a plugin LicenseCheck Thread-Index: AQHWZaPGOLt+1Us2g0+u7v0sVrQer6keljyggADbRtCAAAIyQIABm+Ew Date: Fri, 31 Jul 2020 03:48:48 +0000 Message-ID: References: <20200729122806.21304-1-shenglei.zhang@intel.com> <20200729122806.21304-2-shenglei.zhang@intel.com> <162669507765F26E.19559@groups.io> In-Reply-To: <162669507765F26E.19559@groups.io> Accept-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: dlp-version: 11.5.1.3 dlp-product: dlpe-windows dlp-reaction: no-action authentication-results: edk2.groups.io; dkim=none (message not signed) header.d=none;edk2.groups.io; dmarc=none action=none header.from=intel.com; x-originating-ip: [192.198.147.194] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 2b2b0b0b-23de-4326-b169-08d83504a1c9 x-ms-traffictypediagnostic: CY4PR1101MB2293: x-ms-exchange-transport-forked: True x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:9508; x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: noMDjrWRHa25iTrrZwZ1mN7tDwJijsbM6iGsv2TTXuEtC3m3CyMqLiUmBPoq5naCwF5hpswx/2KbzZtJcxjmZzBaqtu+kUaw1v77TdkJnz596W9K4rS4cp56jFCbPdAt4JiixCdfoac2IpE9VfnX5teZH0D0nf4AmhHRTA73AKhEVZ45PwcAcnEQaWFMpztG7P2Hfv2ma5wPPgW/aLwm4r9mX1snGVK5Mx6gCSdYU3kFpbDJKfGfTYSdyaFa698IbyVUJOQjErz8JDoD37x2gc8GdoEeEC08htULsQPYJS8YbO0Q408Y558obUf/cDedrmt0QhBPzy161H+7JanVL3vBz27t1RNPPlvWTh19A+NoLjZSN5qiOVVMQn4MxvDba21mT/6hQdqZ2rQ4Bg3fYw== x-forefront-antispam-report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:CY4PR11MB1526.namprd11.prod.outlook.com;PTR:;CAT:NONE;SFTY:;SFS:(4636009)(136003)(366004)(346002)(396003)(39860400002)(376002)(52536014)(478600001)(186003)(26005)(55016002)(6506007)(107886003)(76116006)(53546011)(7696005)(9686003)(110136005)(66476007)(66946007)(71200400001)(54906003)(6636002)(8676002)(316002)(4326008)(5660300002)(66446008)(966005)(8936002)(64756008)(83380400001)(66556008)(2906002)(86362001)(33656002);DIR:OUT;SFP:1102; x-ms-exchange-antispam-messagedata: 17DKGn6qL5fclwRp9G14d0k/Zd79510pWgWlCscxLFpO1JogZR3NdthQliIPljs95NXirUHDLaFPd95qfwJnXfX4p9ujJ6JZHL9w7SX1EAZ4+oB6R5KKAt+y3Ze2Zf4f579dAptsXxBSJ0LsjnL0S+qS3QvVSVfhIDL9U/IafTZaq3Qo6AYGlYXvfETdk1QYaULXV0AWyFaiGggXXiUEVswn2LcxosHL9C1ax5jlVb6uH6mjZTnPPokTh0A5bKOCu9M5Gn+9JDN7L3Nhj3XZYrSgidxUEl9SHfbwWpl3IFEDAS20CCRkP7E6/i4KH145vTUNfJUJ2e2+IO4ywuU6s/W1M1UxqHd5h13Cg5ftFlHtKaDS1OaUXotr2w8kjFbSlsMiA2eDhyvHtEUxxrBFQIBtYAYWLx5S9EWZvxlbCiSIqzG0tlg31fv0wURV472aIQ3SxWhVXpdlx1kIudiC28cXxAYxj/W8UpYYgRB5q5nhWLyKbw4hE3g8Zi2MXGS2 MIME-Version: 1.0 X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: CY4PR11MB1526.namprd11.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: 2b2b0b0b-23de-4326-b169-08d83504a1c9 X-MS-Exchange-CrossTenant-originalarrivaltime: 31 Jul 2020 03:48:48.1767 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 46c98d88-e344-4ed4-8496-4ed7712e255d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: tOhv96mWJ+Hu0+d/iZEIB8dGo5KnMaiA40Q85E/6mc6f36DRi7CnGS9kxpY+YgQbA3jS4PpPchlHFouEgRzZ6g== X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY4PR1101MB2293 Return-Path: liming.gao@intel.com X-OriginatorOrg: intel.com Content-Language: en-US Content-Type: text/plain; charset="iso-2022-jp" Content-Transfer-Encoding: quoted-printable Merged at e848b58d7c85293cd4121287abcea2d22a4f0620..7f79b736b0a57da71d87c98= 7357db0227cd16ac6 Thanks Liming -----Original Message----- From: devel@edk2.groups.io On Behalf Of Liming Gao Sent: 2020=1B$BG/=1B(B7=1B$B7n=1B(B30=1B$BF|=1B(B 11:15 To: Zhang, Shenglei ; devel@edk2.groups.io Cc: Sean Brogan ; Bret Barkelew ; Kinney, Michael D Subject: Re: [edk2-devel] [PATCH v2 01/15] .pytool/Plugin: Add a plugin Li= censeCheck With this update, Reviewed-by: Liming Gao =20 If no other comment, I will merge this patch set tomorrow.=20 Thanks Liming -----Original Message----- From: Zhang, Shenglei =20 Sent: 2020=1B$BG/=1B(B7=1B$B7n=1B(B30=1B$BF|=1B(B 11:08 To: Gao, Liming ; devel@edk2.groups.io Cc: Sean Brogan ; Bret Barkelew ; Kinney, Michael D Subject: RE: [PATCH v2 01/15] .pytool/Plugin: Add a plugin LicenseCheck > -----Original Message----- > From: Gao, Liming > Sent: Wednesday, July 29, 2020 10:02 PM > To: Zhang, Shenglei ; devel@edk2.groups.io > Cc: Sean Brogan ; Bret Barkelew=20 > ; Kinney, Michael D=20 > > Subject: RE: [PATCH v2 01/15] .pytool/Plugin: Add a plugin=20 > LicenseCheck >=20 > Shenglei: >=20 > > -----Original Message----- > > From: Zhang, Shenglei > > Sent: Wednesday, July 29, 2020 8:28 PM > > To: devel@edk2.groups.io > > Cc: Sean Brogan ; Bret Barkelew > ; Kinney, Michael D > > ; Gao, Liming > > Subject: [PATCH v2 01/15] .pytool/Plugin: Add a plugin LicenseCheck > > > > REF: https://bugzilla.tianocore.org/show_bug.cgi?id=3D2691 > > REF: https://bugzilla.tianocore.org/show_bug.cgi?id=3D2833 > > Add a plugin to check license conflict for new added files in a=20 > > patch. It will report out errors when meeting files which are now=20 > > contributed under BSD-2-Clause-Patent. >=20 > files which are now contributed under BSD-2-Clause-Patent. > =3D=3D> > files which are not contributed under BSD-2-Clause-Patent. >=20 Yes. It's typo. Thanks, Shenglei > Thanks > Liming > > > > Cc: Sean Brogan > > Cc: Bret Barkelew > > Cc: Michael D Kinney > > Cc: Liming Gao > > Signed-off-by: Shenglei Zhang > > --- > > .pytool/Plugin/LicenseCheck/LicenseCheck.py | 115 > ++++++++++++++++++ > > .../LicenseCheck/LicenseCheck_plug_in.yaml | 11 ++ > > .pytool/Plugin/LicenseCheck/Readme.md | 17 +++ > > 3 files changed, 143 insertions(+) > > create mode 100644 .pytool/Plugin/LicenseCheck/LicenseCheck.py > > create mode > 100644 .pytool/Plugin/LicenseCheck/LicenseCheck_plug_in.yaml > > create mode 100644 .pytool/Plugin/LicenseCheck/Readme.md > > > > diff --git a/.pytool/Plugin/LicenseCheck/LicenseCheck.py > b/.pytool/Plugin/LicenseCheck/LicenseCheck.py > > new file mode 100644 > > index 000000000000..5733f7bf4ec0 > > --- /dev/null > > +++ b/.pytool/Plugin/LicenseCheck/LicenseCheck.py > > @@ -0,0 +1,115 @@ > > +# @file LicenseCheck.py > > +# > > +# Copyright (c) 2020, Intel Corporation. All rights reserved.
#= =20 > > +SPDX-License-Identifier: BSD-2-Clause-Patent ## > > + > > +import os > > +import logging > > +import re > > +from io import StringIO > > +from typing import List, Tuple > > +from edk2toolext.environment.plugintypes.ci_build_plugin import > ICiBuildPlugin > > +from edk2toolext.environment.var_dict import VarDict from=20 > > +edk2toollib.utility_functions import RunCmd > > + > > + > > +class LicenseCheck(ICiBuildPlugin): > > + > > + """ > > + A CiBuildPlugin to check the license for new added files. > > + > > + Configuration options: > > + "LicenseCheck": { > > + "IgnoreFiles": [] > > + }, > > + """ > > + > > + license_format_preflix =3D 'SPDX-License-Identifier' > > + > > + bsd2_patent =3D 'BSD-2-Clause-Patent' > > + > > + Readdedfileformat =3D re.compile(r'\+\+\+ b\/(.*)') > > + > > + file_extension_list =3D [".c", ".h", ".inf", ".dsc", ".dec",=20 > > + ".py", ".bat", ".sh", > ".uni", ".yaml", > > + ".fdf", ".inc", "yml", ".asm", ".asm16",= =20 > > + ".asl", ".vfr", ".s", ".S", > ".aslc", > > + ".nasm", ".nasmb", ".idf", ".Vfr", ".H"] > > + > > + def GetTestName(self, packagename: str, environment: VarDict)=20 > > + -> > tuple: > > + """ Provide the testcase name and classname for use in report= ing > > + testclassname: a descriptive string for the testcase=20 > > + can include > whitespace > > + classname: should be patterned > .. > > + > > + Args: > > + packagename: string containing name of package to build > > + environment: The VarDict for the test to run in > > + Returns: > > + a tuple containing the testcase name and the classnam= e > > + (testcasename, classname) > > + """ > > + return ("Check for license for " + packagename, packagename= =20 > > + + > ".LicenseCheck") > > + > > + ## > > + # External function of plugin. This function is used to=20 > > + perform the task > of the ci_build_plugin Plugin > > + # > > + # - package is the edk2 path to package. This means > workspace/packagepath relative. > > + # - edk2path object configured with workspace and packages path > > + # - PkgConfig Object (dict) for the pkg > > + # - EnvConfig Object > > + # - Plugin Manager Instance > > + # - Plugin Helper Obj Instance > > + # - Junit Logger > > + # - output_stream the StringIO output stream from this plugin v= ia > logging > > + def RunBuildPlugin(self, packagename, Edk2pathObj, pkgconfig, > environment, PLM, PLMHelper, tc, output_stream=3DNone): > > + return_buffer =3D StringIO() > > + params =3D "diff --unified=3D0 origin/master HEAD" > > + RunCmd("git", params, outstream=3Dreturn_buffer) > > + p =3D return_buffer.getvalue().strip() > > + patch =3D p.split("\n") > > + return_buffer.close() > > + > > + ignore_files =3D [] > > + if "IgnoreFiles" in pkgconfig: > > + ignore_files =3D pkgconfig["IgnoreFiles"] > > + > > + self.ok =3D True > > + self.startcheck =3D False > > + self.license =3D True > > + self.all_file_pass =3D True > > + count =3D len(patch) > > + line_index =3D 0 > > + for line in patch: > > + if line.startswith('--- /dev/null'): > > + nextline =3D patch[line_index + 1] > > + added_file =3D self.Readdedfileformat.search(nextline= ).group(1) > > + added_file_extension =3D os.path.splitext(added_file)= [1] > > + if added_file_extension in self.file_extension_list= =20 > > + and > packagename in added_file: > > + if (self.IsIgnoreFile(added_file, ignore_files)): > > + line_index =3D line_index + 1 > > + continue > > + self.startcheck =3D True > > + self.license =3D False > > + if self.startcheck and self.license_format_preflix in lin= e: > > + if self.bsd2_patent in line: > > + self.license =3D True > > + if line_index + 1 =3D=3D count or patch[line_index +=20 > > + 1].startswith('diff --') > and self.startcheck: > > + if not self.license: > > + self.all_file_pass =3D False > > + error_message =3D "Invalid license in: " +=20 > > + added_file + " Hint: Only > BSD-2-Clause-Patent is accepted." > > + logging.error(error_message) > > + self.startcheck =3D False > > + self.license =3D True > > + line_index =3D line_index + 1 > > + > > + if self.all_file_pass: > > + tc.SetSuccess() > > + return 0 > > + else: > > + tc.SetFailed("License Check {0} Failed.=20 > > + ".format(packagename), > "LICENSE_CHECK_FAILED") > > + return 1 > > + > > + def IsIgnoreFile(self, file: str, ignore_files: List[str]) -> boo= l: > > + for f in ignore_files: > > + if f in file: > > + return True > > + return False > > diff --git a/.pytool/Plugin/LicenseCheck/LicenseCheck_plug_in.yaml > b/.pytool/Plugin/LicenseCheck/LicenseCheck_plug_in.yaml > > new file mode 100644 > > index 000000000000..2fdb0aa72359 > > --- /dev/null > > +++ b/.pytool/Plugin/LicenseCheck/LicenseCheck_plug_in.yaml > > @@ -0,0 +1,11 @@ > > +## @file > > +# CiBuildPlugin used to check license issues for new added files #=20 > > +# Copyright (c) 2020, Intel Corporation. All rights reserved. > > +# SPDX-License-Identifier: BSD-2-Clause-Patent ## { > > + "scope": "cibuild", > > + "name": "License Check Test", > > + "module": "LicenseCheck" > > +} > > diff --git a/.pytool/Plugin/LicenseCheck/Readme.md > b/.pytool/Plugin/LicenseCheck/Readme.md > > new file mode 100644 > > index 000000000000..b0e1bbea9e21 > > --- /dev/null > > +++ b/.pytool/Plugin/LicenseCheck/Readme.md > > @@ -0,0 +1,17 @@ > > +# License Check Plugin > > + > > +This CiBuildPlugin scans all new added files in a package to make=20 > > +sure code is contributed under BSD-2-Clause-Patent. > > + > > +## Configuration > > + > > +The plugin can be configured to ignore certain files. > > + > > +``` yaml > > +"LicenseCheck": { > > + "IgnoreFiles": [] > > +} > > +``` > > +### IgnoreFiles > > + > > +OPTIONAL List of file to ignore. > > -- > > 2.18.0.windows.1