From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received-SPF: None (no SPF record) identity=mailfrom; client-ip=15.241.48.73; helo=g9t5009.houston.hpe.com; envelope-from=derek.lin2@hpe.com; receiver=edk2-devel@lists.01.org Received: from g9t5009.houston.hpe.com (g9t5009.houston.hpe.com [15.241.48.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ml01.01.org (Postfix) with ESMTPS id 50E92203B8CE9 for ; Wed, 16 May 2018 19:14:55 -0700 (PDT) Received: from G9W8453.americas.hpqcorp.net (exchangepmrr1.us.hpecorp.net [16.216.160.211]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (No client certificate requested) by g9t5009.houston.hpe.com (Postfix) with ESMTPS id 083EC5C for ; Thu, 17 May 2018 02:14:54 +0000 (UTC) Received: from G2W6309.americas.hpqcorp.net (2002:10c5:4033::10c5:4033) by G9W8453.americas.hpqcorp.net (2002:10d8:a0d3::10d8:a0d3) with Microsoft SMTP Server (TLS) id 15.0.1178.4; Thu, 17 May 2018 02:14:53 +0000 Received: from NAM01-BY2-obe.outbound.protection.outlook.com (15.241.52.10) by G2W6309.americas.hpqcorp.net (16.197.64.51) with Microsoft SMTP Server (TLS) id 15.0.1178.4 via Frontend Transport; Thu, 17 May 2018 02:14:53 +0000 Received: from CS1PR8401MB0678.NAMPRD84.PROD.OUTLOOK.COM (10.169.15.11) by CS1PR8401MB0647.NAMPRD84.PROD.OUTLOOK.COM (10.169.14.149) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P256) id 15.20.776.11; Thu, 17 May 2018 02:14:51 +0000 Received: from CS1PR8401MB0678.NAMPRD84.PROD.OUTLOOK.COM ([fe80::197e:4d33:dcf2:33f2]) by CS1PR8401MB0678.NAMPRD84.PROD.OUTLOOK.COM ([fe80::197e:4d33:dcf2:33f2%18]) with mapi id 15.20.0755.018; Thu, 17 May 2018 02:14:51 +0000 From: "Lin, Derek (HPS UEFI Dev)" To: "Carsey, Jaben" , "Zhu, Yonghong" , "edk2-devel@lists.01.org" Thread-Topic: [PATCH] BaseTools: Fix --hash Package and Module hash value. Thread-Index: AQHT53QVfyM26jl8YUiT2x4ShU8beaQnGW7ggArhz1CAAJlpAIAAoSHg Date: Thu, 17 May 2018 02:14:51 +0000 Message-ID: References: <20180509085924.13140-1-derek.lin2@hpe.com> In-Reply-To: Accept-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: spf=none (sender IP is ) smtp.mailfrom=derek.lin2@hpe.com; x-originating-ip: [16.242.247.137] x-ms-publictraffictype: Email x-microsoft-exchange-diagnostics: 1; CS1PR8401MB0647; 7:liGS0vwKdal0wYqu1G5YU+TqKqUlhLHBEVpCqJpLDciZrEjQHFuxzHW6s/4o6sp+Tfn5cumCuNLc/+AlCuk3J45NjWx7KMiXwh7UQLjXAcICEPG68fawai7B9fED0cKqPXzyuTQLcoSSaRRAj3rNz/08wag7h478T2HLrL40MsBELFJL13+4GeghRvkPaZP3xJCnLpslqFnrRg+NHoYLwo0VhkFqYPwS8DSZfomhtRFIhIGV+l06mWxNx4Ce47gA x-ms-exchange-antispam-srfa-diagnostics: SOS; x-ms-office365-filtering-ht: Tenant x-microsoft-antispam: UriScan:(222181515654134); BCL:0; PCL:0; RULEID:(7020095)(4652020)(8989080)(5600026)(48565401081)(4534165)(4627221)(201703031133081)(201702281549075)(8990040)(2017052603328)(7153060)(7193020); SRVR:CS1PR8401MB0647; x-ms-traffictypediagnostic: CS1PR8401MB0647: x-microsoft-antispam-prvs: x-exchange-antispam-report-test: UriScan:(227479698468861)(162533806227266)(222181515654134)(228905959029699); x-ms-exchange-senderadcheck: 1 x-exchange-antispam-report-cfa-test: BCL:0; PCL:0; RULEID:(8211001083)(6040522)(2401047)(8121501046)(5005006)(93006095)(93001095)(3002001)(10201501046)(3231254)(944501410)(52105095)(6055026)(149027)(150027)(6041310)(20161123562045)(201703131423095)(201702281528075)(20161123555045)(201703061421075)(201703061406153)(20161123560045)(20161123558120)(20161123564045)(6072148)(201708071742011)(7699016); SRVR:CS1PR8401MB0647; BCL:0; PCL:0; RULEID:; SRVR:CS1PR8401MB0647; x-forefront-prvs: 067553F396 x-forefront-antispam-report: SFV:NSPM; SFS:(10019020)(346002)(39380400002)(376002)(39860400002)(396003)(366004)(13464003)(199004)(189003)(305945005)(68736007)(93886005)(3280700002)(7696005)(2501003)(5250100002)(6306002)(55016002)(53936002)(6246003)(9686003)(66066001)(229853002)(99286004)(6436002)(316002)(110136005)(2900100001)(86362001)(2906002)(3660700001)(106356001)(105586002)(76176011)(478600001)(11346002)(26005)(6506007)(59450400001)(25786009)(476003)(53546011)(186003)(14454004)(966005)(486006)(102836004)(6116002)(81156014)(8676002)(81166006)(8936002)(3846002)(7736002)(33656002)(446003)(97736004)(5660300001)(74316002); DIR:OUT; SFP:1102; SCL:1; SRVR:CS1PR8401MB0647; H:CS1PR8401MB0678.NAMPRD84.PROD.OUTLOOK.COM; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; A:1; MX:1; received-spf: None (protection.outlook.com: hpe.com does not designate permitted sender hosts) x-microsoft-antispam-message-info: nEgEXYqfB1brH32EzHoB+RbciCwoasqUpQaLPdf5lLiUx2EXJ7CuoqHVB2JSj4wan5aDhx94udAkUPlKEyVqS4apWqlFmcGI6goYHhL6STFZ3F6BMElbriqkk7oq0fd7VWrJkGiTVY9+9AM1EzeE9a6WYHMYOrgYiufe0YstYJPrJDzc7NbMsX6zK/Iv5gge spamdiagnosticoutput: 1:99 spamdiagnosticmetadata: NSPM MIME-Version: 1.0 X-MS-Office365-Filtering-Correlation-Id: e273a1f5-9b9f-439d-a9ae-08d5bb9bf90f X-MS-Exchange-CrossTenant-Network-Message-Id: e273a1f5-9b9f-439d-a9ae-08d5bb9bf90f X-MS-Exchange-CrossTenant-originalarrivaltime: 17 May 2018 02:14:51.3952 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 105b2061-b669-4b31-92ac-24d304d195dc X-MS-Exchange-Transport-CrossTenantHeadersStamped: CS1PR8401MB0647 X-OriginatorOrg: hpe.com Subject: Re: [PATCH] BaseTools: Fix --hash Package and Module hash value. X-BeenThere: edk2-devel@lists.01.org X-Mailman-Version: 2.1.26 Precedence: list List-Id: EDK II Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 17 May 2018 02:14:55 -0000 Content-Language: en-US Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Jaben, You're right, Python list is ordered. So it's because of insertion order, hmm.. but the meta-file parser run in s= ingle thread, not sure why it generate different order in different build..= .never mind. However, I like the idea that changing list to set. I should improve perfor= mance. Thanks, Derek -----Original Message----- From: Carsey, Jaben [mailto:jaben.carsey@intel.com]=20 Sent: Thursday, May 17, 2018 12:20 AM To: Zhu, Yonghong ; Lin, Derek (HPS UEFI Dev) ; edk2-devel@lists.01.org Subject: RE: [PATCH] BaseTools: Fix --hash Package and Module hash value. Derek, The change is good. I have a comment/question on the message. Reviewed-by: Jaben Carsey Order of list enumeration is not arbitrary, it's based on python lists bein= g inherently insertion-ordered objects. If insertion order is irrelevant, should we consider changing those to a di= fferent type? (set?) -Jaben > -----Original Message----- > From: edk2-devel [mailto:edk2-devel-bounces@lists.01.org] On Behalf Of=20 > Zhu, Yonghong > Sent: Wednesday, May 16, 2018 12:12 AM > To: Lin, Derek (HPS UEFI Dev) ; edk2-=20 > devel@lists.01.org > Subject: Re: [edk2] [PATCH] BaseTools: Fix --hash Package and Module=20 > hash value. >=20 > Reviewed-by: Yonghong Zhu >=20 > Best Regards, > Zhu Yonghong >=20 >=20 > -----Original Message----- > From: Lin, Derek (HPS UEFI Dev) [mailto:derek.lin2@hpe.com] > Sent: Wednesday, May 09, 2018 5:03 PM > To: edk2-devel@lists.01.org > Cc: Zhu, Yonghong ; Lin, Derek (HPS UEFI Dev)=20 > > Subject: [PATCH] BaseTools: Fix --hash Package and Module hash value. >=20 > The order of List enumeration is arbitrary. > Need to be sorted while calculating Package/Module hash, otherwise it=20 > generate different hash value even nothing changes. >=20 > Contributed-under: TianoCore Contribution Agreement 1.1 > Signed-off-by: Derek Lin > --- > BaseTools/Source/Python/AutoGen/AutoGen.py | 17 ++++++++++------- > 1 file changed, 10 insertions(+), 7 deletions(-) >=20 > diff --git a/BaseTools/Source/Python/AutoGen/AutoGen.py > b/BaseTools/Source/Python/AutoGen/AutoGen.py > index 54f6b1f173..90704dcae4 100644 > --- a/BaseTools/Source/Python/AutoGen/AutoGen.py > +++ b/BaseTools/Source/Python/AutoGen/AutoGen.py > @@ -2,6 +2,7 @@ > # Generate AutoGen.h, AutoGen.c and *.depex files # # Copyright (c)=20 > 2007 > - 2018, Intel Corporation. All rights reserved.
> +# Copyright (c) 2018, Hewlett Packard Enterprise Development,=20 > +L.P.
> # This program and the accompanying materials # are licensed and=20 > made available under the terms and conditions of the BSD License #=20 > which accompanies this distribution. The full text of the license may=20 > be found at @@ -670,6 +671,9 @@ class WorkspaceAutoGen(AutoGen): > return True >=20 > def _GenPkgLevelHash(self, Pkg): > + if Pkg.PackageName in GlobalData.gPackageHash[Pkg.Arch]: > + return > + > PkgDir =3D os.path.join(self.BuildDir, Pkg.Arch, Pkg.PackageName= ) > CreateDirectory(PkgDir) > HashFile =3D os.path.join(PkgDir, Pkg.PackageName + '.hash') @@= =20 > -681,17 > +685,16 @@ class WorkspaceAutoGen(AutoGen): > m.update(Content) > # Get include files hash value > if Pkg.Includes: > - for inc in Pkg.Includes: > + for inc in sorted(Pkg.Includes, key=3Dlambda x: str(x)): > for Root, Dirs, Files in os.walk(str(inc)): > - for File in Files: > + for File in sorted(Files): > File_Path =3D os.path.join(Root, File) > f =3D open(File_Path, 'r') > Content =3D f.read() > f.close() > m.update(Content) > SaveFileOnChange(HashFile, m.hexdigest(), True) > - if Pkg.PackageName not in GlobalData.gPackageHash[Pkg.Arch]: > - GlobalData.gPackageHash[Pkg.Arch][Pkg.PackageName] =3D > m.hexdigest() > + GlobalData.gPackageHash[Pkg.Arch][Pkg.PackageName] =3D > + m.hexdigest() >=20 > def _GetMetaFiles(self, Target, Toolchain, Arch): > AllWorkSpaceMetaFiles =3D set() @@ -4432,13 +4435,13 @@ class=20 > ModuleAutoGen(AutoGen): > m.update(GlobalData.gPlatformHash) > # Add Package level hash > if self.DependentPackageList: > - for Pkg in self.DependentPackageList: > + for Pkg in sorted(self.DependentPackageList, key=3Dlambda x: > x.PackageName): > if Pkg.PackageName in GlobalData.gPackageHash[self.Arch]= : >=20 > m.update(GlobalData.gPackageHash[self.Arch][Pkg.PackageName]) >=20 > # Add Library hash > if self.LibraryAutoGenList: > - for Lib in self.LibraryAutoGenList: > + for Lib in sorted(self.LibraryAutoGenList, key=3Dlambda x: x= .Name): > if Lib.Name not in GlobalData.gModuleHash[self.Arch]: > Lib.GenModuleHash() > m.update(GlobalData.gModuleHash[self.Arch][Lib.Name]) > @@ -4450,7 +4453,7 @@ class ModuleAutoGen(AutoGen): > m.update(Content) > # Add Module's source files > if self.SourceFileList: > - for File in self.SourceFileList: > + for File in sorted(self.SourceFileList, key=3Dlambda x: str(= x)): > f =3D open(str(File), 'r') > Content =3D f.read() > f.close() > -- > 2.15.1.windows.2 >=20 >=20 > _______________________________________________ > edk2-devel mailing list > edk2-devel@lists.01.org > https://lists.01.org/mailman/listinfo/edk2-devel