From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mga03.intel.com (mga03.intel.com [134.134.136.65]) by mx.groups.io with SMTP id smtpd.web11.876.1596078911982128847 for ; Wed, 29 Jul 2020 20:15:12 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=@intel.onmicrosoft.com header.s=selector2-intel-onmicrosoft-com header.b=mAMscN5D; spf=pass (domain: intel.com, ip: 134.134.136.65, mailfrom: liming.gao@intel.com) IronPort-SDR: /E8VdlQfCIfRDTlpYcaD3iTPfgiqTHe2e7UOd+MgAUTupPMLdlGGYBKi6sm/NEWWMNZzyD/fAE aqs8xYRmX0/A== X-IronPort-AV: E=McAfee;i="6000,8403,9697"; a="151511842" X-IronPort-AV: E=Sophos;i="5.75,412,1589266800"; d="scan'208";a="151511842" X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga007.fm.intel.com ([10.253.24.52]) by orsmga103.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 29 Jul 2020 20:15:11 -0700 IronPort-SDR: 4Jc3nYxQtPqQQBx6DTab+aZDyZFnpmufM8TwEKmYmZ6Dq22l+I2Gq0KeYKVjgkWha7chby6tAf h2vU+mlF7ztA== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.75,412,1589266800"; d="scan'208";a="272798114" Received: from orsmsx108.amr.corp.intel.com ([10.22.240.6]) by fmsmga007.fm.intel.com with ESMTP; 29 Jul 2020 20:15:10 -0700 Received: from orsmsx163.amr.corp.intel.com (10.22.240.88) by ORSMSX108.amr.corp.intel.com (10.22.240.6) with Microsoft SMTP Server (TLS) id 14.3.439.0; Wed, 29 Jul 2020 20:15:10 -0700 Received: from ORSEDG001.ED.cps.intel.com (10.7.248.4) by ORSMSX163.amr.corp.intel.com (10.22.240.88) with Microsoft SMTP Server (TLS) id 14.3.439.0; Wed, 29 Jul 2020 20:15:10 -0700 Received: from NAM11-CO1-obe.outbound.protection.outlook.com (104.47.56.176) by edgegateway.intel.com (134.134.137.100) with Microsoft SMTP Server (TLS) id 14.3.439.0; Wed, 29 Jul 2020 20:15:09 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=lWDzuNP05gv48SVNBZ6BdKoRtkfujv4A3IP4Ghk1xO50eZc9uDVoNg/XviFg3YyuFp/GCYjPg83Rg1ueXKj1gd46GfnoaEUt7IrLJNuoa3leTLEvXW2+90IBEwiqi/RNXWhDFrIoUSkhfl7uIzB3C8WQKo8t7C89dxHCWI7ZoJBooOW4xPB95yg6hS4Z6EPvjwb2R2EdiIPFgUrJ9FwPrzbIbFsvGWu63Y3MK0+PtUz5/Npc84eSWUzRq88ndxaeaia2EgremihggaK98RVpE63mlyz29fbOf7ptE8+I02Dk6/qPxAbg7kBdyCTcrD5HNzOUiywhXoSnCooh/hEzmA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=3w6R9nGYwltXYkLuWx8hIC+7zSks8IPMhu7y5iHzf2g=; b=S5knHrHTDzOCB7WryqoFR+VsGs888jc3r89FJ5aqGT6JWIwjCWVlB0ASKkvqepipg3+fDntIbXTdLMKboYaePPbzazPEkp2mpEMT5QJrmslur7s6Gnnw6uMlDAtpzJuqMWvBhsDcucFcP5tuGMwOO4vVnpuidlfwF+LQiXl7BsFjNmgcq28DseiyjtoD5uzUIfhZHmXYfE/fOUrewmgOqCAmYZu5Jw9rKMYYzjYwEvpaUkZ03ZJjDYeqqb5DyJzoMvxPb85VW7M0IscLQQbmOp5STbjhHK8NStxFO8P1XYgkMX40i0Twmt6/2O6kf60/f3x2Q5gOlexzI4nOouVhMw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com; dkim=pass header.d=intel.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=intel.onmicrosoft.com; s=selector2-intel-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=3w6R9nGYwltXYkLuWx8hIC+7zSks8IPMhu7y5iHzf2g=; b=mAMscN5DP7fMnFdeCrG8DclSGxOAx+s+k0eAu/p3udlUoD0Yl5Gm9HKPbm5nnDKN/7XUeZFrZa90xDMwkpf7KiEHIyrT7+f8UBFRoBHkw/GqqlpQFmF++72yMfgreXpvaQpqcLEHalP5HeRrXmQIN1iGZSFdecfjf0F2h/uYnq0= Received: from CY4PR11MB1526.namprd11.prod.outlook.com (2603:10b6:910:7::10) by CY4PR11MB1942.namprd11.prod.outlook.com (2603:10b6:903:125::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3216.23; Thu, 30 Jul 2020 03:15:08 +0000 Received: from CY4PR11MB1526.namprd11.prod.outlook.com ([fe80::f8c4:711:84e3:4c97]) by CY4PR11MB1526.namprd11.prod.outlook.com ([fe80::f8c4:711:84e3:4c97%4]) with mapi id 15.20.3216.034; Thu, 30 Jul 2020 03:15:08 +0000 From: "Liming Gao" To: "Zhang, Shenglei" , "devel@edk2.groups.io" CC: Sean Brogan , Bret Barkelew , "Kinney, Michael D" Subject: Re: [PATCH v2 01/15] .pytool/Plugin: Add a plugin LicenseCheck Thread-Topic: [PATCH v2 01/15] .pytool/Plugin: Add a plugin LicenseCheck Thread-Index: AQHWZaPGOLt+1Us2g0+u7v0sVrQer6keljyggADbRtCAAAIyQA== Date: Thu, 30 Jul 2020 03:15:08 +0000 Message-ID: References: <20200729122806.21304-1-shenglei.zhang@intel.com> <20200729122806.21304-2-shenglei.zhang@intel.com> In-Reply-To: Accept-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: dlp-version: 11.2.0.6 dlp-product: dlpe-windows dlp-reaction: no-action authentication-results: intel.com; dkim=none (message not signed) header.d=none;intel.com; dmarc=none action=none header.from=intel.com; x-originating-ip: [192.198.147.194] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: d0936db5-2d0a-47bd-65ef-08d83436c395 x-ms-traffictypediagnostic: CY4PR11MB1942: x-ms-exchange-transport-forked: True x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:9508; x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: vej5Ru/r6ZhYgS09780hrxFzDbr1Kdxu/tATDrE8zUxldptKgnhn91aV5x5qnS7R8T6l5U70ifceVCIjGBJMtPSxUiUrv29HUtqQbH1onqLLFPE0AWK512JpnIK3P2M095yMZTRx0vrxFvHk+ejwq8AnNAyWqkPXK6fJrQ8ru/tMQtDW6xVkQPSXfumsbYBdFXgM5egO2iwZ2aI5AIn48sTlw2tAlTInqYF0qwJovKRBm+KdV3v3u5dw70hTF3tXWjrVC3XMdWeRP4I9fQZ84whCVLmqwaNqtHf+eRsQcBwAM/mCaOQGGCxCKuVMEQsSSk6OPmdjgloZ7V4gJCd6aSfQJ+qpwPDXKnOxFGSR9Wawum+8zw6FgKFrmSOIzyQCULmbHkU4TVRNd3K9VhSulQ== x-forefront-antispam-report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:CY4PR11MB1526.namprd11.prod.outlook.com;PTR:;CAT:NONE;SFTY:;SFS:(4636009)(376002)(346002)(136003)(396003)(366004)(39860400002)(26005)(5660300002)(478600001)(53546011)(8936002)(6506007)(33656002)(107886003)(186003)(52536014)(7696005)(4326008)(8676002)(110136005)(54906003)(66476007)(66556008)(71200400001)(316002)(966005)(2906002)(9686003)(83380400001)(64756008)(86362001)(55016002)(66446008)(76116006)(66946007);DIR:OUT;SFP:1102; x-ms-exchange-antispam-messagedata: 3F43zbome97Dv1vriVttgLijh24ahchsFxXwcanUaDVS0EUptlMZVwZoRCZA1+uN72xVmd6J/iTV8NSsjPhYJU7G1RrzR0bqiWUKcuL+0upHGxS0MbGQ5AKgSOyPjOPXvXJyEYOYBZE+lT0ip5K9Y9qyxBid0VPj/3J9xD6qjgGj/J1tnPzqOviCosmvHVHWkrXQ+U8dXpBHAPPXwXtHd3bxFH5kSar85ADXKNQynQ2A+W9muhynMj2amu+BLLLFocchftnEPeIvwXJ6ScHNN/CQqAh/dND29foo/2FZOKrrtVmG10GIVPq3+hQg1XGBnGELis1mapkeJILLE7dyX8tZod/u+H2QqbAj06aQmlCur5TWQjmIzUl3BLqduCy2IvlfIm8YrzHD6Iw2doq7ujmx+aNNX18hPJhIj3D9vkw3XNfh0skbkh8btF4wSn0G1poSskdImojGJC2whNO8opATGBmdy23JNvPLGVzwQoJX9ogLpX5Cn+8kQaImJm+8 MIME-Version: 1.0 X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: CY4PR11MB1526.namprd11.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: d0936db5-2d0a-47bd-65ef-08d83436c395 X-MS-Exchange-CrossTenant-originalarrivaltime: 30 Jul 2020 03:15:08.6441 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 46c98d88-e344-4ed4-8496-4ed7712e255d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: HDgbUTrgPWZFyUvdt7uC734yF+u2Rl9bRVI1q8AXyq50n4yUdozAOIrZ9lv8aDoCoBtt6DW+SLl6Jd1qWOi7+w== X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY4PR11MB1942 Return-Path: liming.gao@intel.com X-OriginatorOrg: intel.com Content-Language: en-US Content-Type: text/plain; charset="iso-2022-jp" Content-Transfer-Encoding: quoted-printable With this update, Reviewed-by: Liming Gao =20 If no other comment, I will merge this patch set tomorrow.=20 Thanks Liming -----Original Message----- From: Zhang, Shenglei =20 Sent: 2020=1B$BG/=1B(B7=1B$B7n=1B(B30=1B$BF|=1B(B 11:08 To: Gao, Liming ; devel@edk2.groups.io Cc: Sean Brogan ; Bret Barkelew ; Kinney, Michael D Subject: RE: [PATCH v2 01/15] .pytool/Plugin: Add a plugin LicenseCheck > -----Original Message----- > From: Gao, Liming > Sent: Wednesday, July 29, 2020 10:02 PM > To: Zhang, Shenglei ; devel@edk2.groups.io > Cc: Sean Brogan ; Bret Barkelew=20 > ; Kinney, Michael D=20 > > Subject: RE: [PATCH v2 01/15] .pytool/Plugin: Add a plugin=20 > LicenseCheck >=20 > Shenglei: >=20 > > -----Original Message----- > > From: Zhang, Shenglei > > Sent: Wednesday, July 29, 2020 8:28 PM > > To: devel@edk2.groups.io > > Cc: Sean Brogan ; Bret Barkelew > ; Kinney, Michael D > > ; Gao, Liming > > Subject: [PATCH v2 01/15] .pytool/Plugin: Add a plugin LicenseCheck > > > > REF: https://bugzilla.tianocore.org/show_bug.cgi?id=3D2691 > > REF: https://bugzilla.tianocore.org/show_bug.cgi?id=3D2833 > > Add a plugin to check license conflict for new added files in a=20 > > patch. It will report out errors when meeting files which are now=20 > > contributed under BSD-2-Clause-Patent. >=20 > files which are now contributed under BSD-2-Clause-Patent. > =3D=3D> > files which are not contributed under BSD-2-Clause-Patent. >=20 Yes. It's typo. Thanks, Shenglei > Thanks > Liming > > > > Cc: Sean Brogan > > Cc: Bret Barkelew > > Cc: Michael D Kinney > > Cc: Liming Gao > > Signed-off-by: Shenglei Zhang > > --- > > .pytool/Plugin/LicenseCheck/LicenseCheck.py | 115 > ++++++++++++++++++ > > .../LicenseCheck/LicenseCheck_plug_in.yaml | 11 ++ > > .pytool/Plugin/LicenseCheck/Readme.md | 17 +++ > > 3 files changed, 143 insertions(+) > > create mode 100644 .pytool/Plugin/LicenseCheck/LicenseCheck.py > > create mode > 100644 .pytool/Plugin/LicenseCheck/LicenseCheck_plug_in.yaml > > create mode 100644 .pytool/Plugin/LicenseCheck/Readme.md > > > > diff --git a/.pytool/Plugin/LicenseCheck/LicenseCheck.py > b/.pytool/Plugin/LicenseCheck/LicenseCheck.py > > new file mode 100644 > > index 000000000000..5733f7bf4ec0 > > --- /dev/null > > +++ b/.pytool/Plugin/LicenseCheck/LicenseCheck.py > > @@ -0,0 +1,115 @@ > > +# @file LicenseCheck.py > > +# > > +# Copyright (c) 2020, Intel Corporation. All rights reserved.
#=20 > > +SPDX-License-Identifier: BSD-2-Clause-Patent ## > > + > > +import os > > +import logging > > +import re > > +from io import StringIO > > +from typing import List, Tuple > > +from edk2toolext.environment.plugintypes.ci_build_plugin import > ICiBuildPlugin > > +from edk2toolext.environment.var_dict import VarDict from=20 > > +edk2toollib.utility_functions import RunCmd > > + > > + > > +class LicenseCheck(ICiBuildPlugin): > > + > > + """ > > + A CiBuildPlugin to check the license for new added files. > > + > > + Configuration options: > > + "LicenseCheck": { > > + "IgnoreFiles": [] > > + }, > > + """ > > + > > + license_format_preflix =3D 'SPDX-License-Identifier' > > + > > + bsd2_patent =3D 'BSD-2-Clause-Patent' > > + > > + Readdedfileformat =3D re.compile(r'\+\+\+ b\/(.*)') > > + > > + file_extension_list =3D [".c", ".h", ".inf", ".dsc", ".dec",=20 > > + ".py", ".bat", ".sh", > ".uni", ".yaml", > > + ".fdf", ".inc", "yml", ".asm", ".asm16",=20 > > + ".asl", ".vfr", ".s", ".S", > ".aslc", > > + ".nasm", ".nasmb", ".idf", ".Vfr", ".H"] > > + > > + def GetTestName(self, packagename: str, environment: VarDict)=20 > > + -> > tuple: > > + """ Provide the testcase name and classname for use in reporti= ng > > + testclassname: a descriptive string for the testcase=20 > > + can include > whitespace > > + classname: should be patterned > .. > > + > > + Args: > > + packagename: string containing name of package to build > > + environment: The VarDict for the test to run in > > + Returns: > > + a tuple containing the testcase name and the classname > > + (testcasename, classname) > > + """ > > + return ("Check for license for " + packagename, packagename=20 > > + + > ".LicenseCheck") > > + > > + ## > > + # External function of plugin. This function is used to=20 > > + perform the task > of the ci_build_plugin Plugin > > + # > > + # - package is the edk2 path to package. This means > workspace/packagepath relative. > > + # - edk2path object configured with workspace and packages path > > + # - PkgConfig Object (dict) for the pkg > > + # - EnvConfig Object > > + # - Plugin Manager Instance > > + # - Plugin Helper Obj Instance > > + # - Junit Logger > > + # - output_stream the StringIO output stream from this plugin vi= a > logging > > + def RunBuildPlugin(self, packagename, Edk2pathObj, pkgconfig, > environment, PLM, PLMHelper, tc, output_stream=3DNone): > > + return_buffer =3D StringIO() > > + params =3D "diff --unified=3D0 origin/master HEAD" > > + RunCmd("git", params, outstream=3Dreturn_buffer) > > + p =3D return_buffer.getvalue().strip() > > + patch =3D p.split("\n") > > + return_buffer.close() > > + > > + ignore_files =3D [] > > + if "IgnoreFiles" in pkgconfig: > > + ignore_files =3D pkgconfig["IgnoreFiles"] > > + > > + self.ok =3D True > > + self.startcheck =3D False > > + self.license =3D True > > + self.all_file_pass =3D True > > + count =3D len(patch) > > + line_index =3D 0 > > + for line in patch: > > + if line.startswith('--- /dev/null'): > > + nextline =3D patch[line_index + 1] > > + added_file =3D self.Readdedfileformat.search(nextline)= .group(1) > > + added_file_extension =3D os.path.splitext(added_file)[= 1] > > + if added_file_extension in self.file_extension_list=20 > > + and > packagename in added_file: > > + if (self.IsIgnoreFile(added_file, ignore_files)): > > + line_index =3D line_index + 1 > > + continue > > + self.startcheck =3D True > > + self.license =3D False > > + if self.startcheck and self.license_format_preflix in line= : > > + if self.bsd2_patent in line: > > + self.license =3D True > > + if line_index + 1 =3D=3D count or patch[line_index +=20 > > + 1].startswith('diff --') > and self.startcheck: > > + if not self.license: > > + self.all_file_pass =3D False > > + error_message =3D "Invalid license in: " +=20 > > + added_file + " Hint: Only > BSD-2-Clause-Patent is accepted." > > + logging.error(error_message) > > + self.startcheck =3D False > > + self.license =3D True > > + line_index =3D line_index + 1 > > + > > + if self.all_file_pass: > > + tc.SetSuccess() > > + return 0 > > + else: > > + tc.SetFailed("License Check {0} Failed.=20 > > + ".format(packagename), > "LICENSE_CHECK_FAILED") > > + return 1 > > + > > + def IsIgnoreFile(self, file: str, ignore_files: List[str]) -> bool= : > > + for f in ignore_files: > > + if f in file: > > + return True > > + return False > > diff --git a/.pytool/Plugin/LicenseCheck/LicenseCheck_plug_in.yaml > b/.pytool/Plugin/LicenseCheck/LicenseCheck_plug_in.yaml > > new file mode 100644 > > index 000000000000..2fdb0aa72359 > > --- /dev/null > > +++ b/.pytool/Plugin/LicenseCheck/LicenseCheck_plug_in.yaml > > @@ -0,0 +1,11 @@ > > +## @file > > +# CiBuildPlugin used to check license issues for new added files #=20 > > +# Copyright (c) 2020, Intel Corporation. All rights reserved. > > +# SPDX-License-Identifier: BSD-2-Clause-Patent ## { > > + "scope": "cibuild", > > + "name": "License Check Test", > > + "module": "LicenseCheck" > > +} > > diff --git a/.pytool/Plugin/LicenseCheck/Readme.md > b/.pytool/Plugin/LicenseCheck/Readme.md > > new file mode 100644 > > index 000000000000..b0e1bbea9e21 > > --- /dev/null > > +++ b/.pytool/Plugin/LicenseCheck/Readme.md > > @@ -0,0 +1,17 @@ > > +# License Check Plugin > > + > > +This CiBuildPlugin scans all new added files in a package to make=20 > > +sure code is contributed under BSD-2-Clause-Patent. > > + > > +## Configuration > > + > > +The plugin can be configured to ignore certain files. > > + > > +``` yaml > > +"LicenseCheck": { > > + "IgnoreFiles": [] > > +} > > +``` > > +### IgnoreFiles > > + > > +OPTIONAL List of file to ignore. > > -- > > 2.18.0.windows.1