From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received-SPF: Pass (sender SPF authorized) identity=mailfrom; client-ip=192.55.52.88; helo=mga01.intel.com; envelope-from=yonghong.zhu@intel.com; receiver=edk2-devel@lists.01.org Received: from mga01.intel.com (mga01.intel.com [192.55.52.88]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ml01.01.org (Postfix) with ESMTPS id 9913721127CD3 for ; Mon, 17 Sep 2018 00:23:14 -0700 (PDT) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga004.jf.intel.com ([10.7.209.38]) by fmsmga101.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 17 Sep 2018 00:23:14 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.53,384,1531810800"; d="scan'208";a="233513339" Received: from fmsmsx105.amr.corp.intel.com ([10.18.124.203]) by orsmga004.jf.intel.com with ESMTP; 17 Sep 2018 00:22:38 -0700 Received: from fmsmsx117.amr.corp.intel.com (10.18.116.17) by FMSMSX105.amr.corp.intel.com (10.18.124.203) with Microsoft SMTP Server (TLS) id 14.3.319.2; Mon, 17 Sep 2018 00:22:37 -0700 Received: from shsmsx101.ccr.corp.intel.com (10.239.4.153) by fmsmsx117.amr.corp.intel.com (10.18.116.17) with Microsoft SMTP Server (TLS) id 14.3.319.2; Mon, 17 Sep 2018 00:22:37 -0700 Received: from shsmsx103.ccr.corp.intel.com ([169.254.4.240]) by SHSMSX101.ccr.corp.intel.com ([169.254.1.205]) with mapi id 14.03.0319.002; Mon, 17 Sep 2018 15:22:31 +0800 From: "Zhu, Yonghong" To: "Zhu, Yonghong" , "edk2-devel@lists.01.org" CC: "Gao, Liming" , "Zhu, Yonghong" Thread-Topic: [edk2] [Patch] BaseTools: remove PyUtility and open the file with unbuffered Thread-Index: AQHUTlG3bRg9kcXQ+0iO4iNTB2nOG6T0EZdQ Date: Mon, 17 Sep 2018 07:22:31 +0000 Message-ID: References: <1537166586-32980-1-git-send-email-yonghong.zhu@intel.com> In-Reply-To: <1537166586-32980-1-git-send-email-yonghong.zhu@intel.com> Accept-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [10.239.127.40] MIME-Version: 1.0 Subject: Re: [Patch] BaseTools: remove PyUtility and open the file with unbuffered X-BeenThere: edk2-devel@lists.01.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: EDK II Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 17 Sep 2018 07:23:14 -0000 Content-Language: en-US Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Please ignore this one, I will separate it into two patches. Best Regards, Zhu Yonghong -----Original Message----- From: edk2-devel [mailto:edk2-devel-bounces@lists.01.org] On Behalf Of Yong= hong Zhu Sent: Monday, September 17, 2018 2:43 PM To: edk2-devel@lists.01.org Cc: Gao, Liming Subject: [edk2] [Patch] BaseTools: remove PyUtility and open the file with = unbuffered the PyUtility is not used, so we remove it. And update the open file with unbuffered to avoid the case that the file is= not wrote but directly used in later when the host's multiple thread is ve= ry strong. Cc: Liming Gao Contributed-under: TianoCore Contribution Agreement 1.1 Signed-off-by: Yonghong Zhu --- BaseTools/Source/C/PyUtility/Makefile | 25 ------- BaseTools/Source/C/PyUtility/PyUtility.c | 106 -----------------------= ---- BaseTools/Source/C/PyUtility/setup.py | 42 ----------- BaseTools/Source/Python/Common/Misc.py | 16 +--- BaseTools/Source/Python/Common/PyUtility.pyd | Bin 6144 -> 0 bytes 5 files changed, 3 insertions(+), 186 deletions(-) delete mode 100644 Bas= eTools/Source/C/PyUtility/Makefile delete mode 100644 BaseTools/Source/C/PyUtility/PyUtility.c delete mode 100644 BaseTools/Source/C/PyUtility/setup.py delete mode 100644 BaseTools/Source/Python/Common/PyUtility.pyd diff --git a/BaseTools/Source/C/PyUtility/Makefile b/BaseTools/Source/C/PyU= tility/Makefile deleted file mode 100644 index 5829070..0000000 --- a/BaseTools/Source/C/PyUtility/Makefile +++ /dev/null @@ -1,25 +0,0 @@ -## @file -# Makefile -# -# Copyright (c) 2007 - 2014, Intel Corporation. All rights reserved.
-= # This program and the accompanying materials -# are licensed and made avai= lable under the terms and conditions of the BSD License -# which accompanies this distribution. The full text of the license may= be found at -# http://opensource.org/licenses/bsd-license.php -# -# THE PROGRAM IS DISTRIBUTED UNDER THE BSD LICENSE ON AN "AS IS" BASIS, -#= WITHOUT WARRANTIES OR REPRESENTATIONS OF ANY KIND, EITHER EXPRESS OR IMPLI= ED. -# - -!INCLUDE ..\Makefiles\ms.common - -APPNAME =3D GenSec - -LIBS =3D $(LIB_PATH)\Common.lib - -OBJECTS =3D PyUtility.obj - -#CFLAGS =3D $(CFLAGS) /nodefaultlib:libc.lib - -!INCLUDE ..\Makefiles\ms.app - diff --git a/BaseTools/Source/C/PyUtility/PyUtility.c b/BaseTools/Source/C/= PyUtility/PyUtility.c deleted file mode 100644 index d14b872..0000000 --- a/BaseTools/Source/C/PyUtility/PyUtility.c +++ /dev/null @@ -1,106 +0,0 @@ -/** @file -Python Utility - -Copyright (c) 2009 - 2018, Intel Corporation. All rights reserved.
-Th= is program and the accompanying materials are licensed and made available -= under the terms and conditions of the BSD License which accompanies this -d= istribution. The full text of the license may be found at -http://opensour= ce.org/licenses/bsd-license.php - -THE PROGRAM IS DISTRIBUTED UNDER THE BSD LICENSE ON AN "AS IS" BASIS, -WIT= HOUT WARRANTIES OR REPRESENTATIONS OF ANY KIND, EITHER EXPRESS OR IMPLIED. - -**/ - -#include -#include -#include - -/* - SaveFileToDisk(FilePath, Content) -*/ -STATIC -PyObject* -SaveFileToDisk ( - PyObject *Self, - PyObject *Args - ) -{ - CHAR8 *File; - UINT8 *Data; - UINTN DataLength; - UINTN WriteBytes; - UINTN Status; - HANDLE FileHandle; - PyObject *ReturnValue =3D Py_False; - - Status =3D PyArg_ParseTuple( - Args, - "ss#", - &File, - &Data, - &DataLength - ); - if (Status =3D=3D 0) { - return NULL; - } - - FileHandle =3D CreateFile( - File, - GENERIC_WRITE, - FILE_SHARE_WRITE|FILE_SHARE_READ|FILE_SHARE_DELETE, - NULL, - CREATE_ALWAYS, - FILE_ATTRIBUTE_NORMAL, - NULL - ); - if (FileHandle =3D=3D INVALID_HANDLE_VALUE) { - PyErr_SetString(PyExc_Exception, "File creation failure"); - return NULL; - } - - while (WriteFile(FileHandle, Data, DataLength, &WriteBytes, NULL)) { - if (DataLength <=3D WriteBytes) { - DataLength =3D 0; - break; - } - - Data +=3D WriteBytes; - DataLength -=3D WriteBytes; - } - - if (DataLength !=3D 0) { - // file saved unsuccessfully - PyErr_SetString(PyExc_Exception, "File write failure"); - goto Done; - } - - // - // Flush buffer may slow down the whole build performance (average 10s s= lower) - // - //if (!FlushFileBuffers(FileHandle)) { - // PyErr_SetString(PyExc_Exception, "File flush failure"); - // goto Done; - //} - - // success! - ReturnValue =3D Py_True; - -Done: - CloseHandle(FileHandle); - return ReturnValue; -} - -STATIC INT8 SaveFileToDiskDocs[] =3D "SaveFileToDisk(): Make sure the file= is saved to disk\n"; - -STATIC PyMethodDef PyUtility_Funcs[] =3D { - {"SaveFileToDisk", (PyCFunction)SaveFileToDisk, METH_VARARGS, SaveFileTo= DiskDocs}, - {NULL, NULL, 0, NULL} -}; - -PyMODINIT_FUNC -initPyUtility(VOID) { - Py_InitModule3("PyUtility", PyUtility_Funcs, "Utilties Module Implemente= d C Language"); -} - - diff --git a/BaseTools/Source/C/PyUtility/setup.py b/BaseTools/Source/C/PyU= tility/setup.py deleted file mode 100644 index e4d407d..0000000 --- a/BaseTools/Source/C/PyUtility/setup.py +++ /dev/null @@ -1,42 +0,0 @@ -## @file -# package and install PyEfiCompressor extension -# -# Copyright (c) 2008,= Intel Corporation. All rights reserved.
-# -# This program and the ac= companying materials -# are licensed and made available under the terms an= d conditions of the BSD License -# which accompanies this distribution. T= he full text of the license may be found at -# http://opensource.org/licen= ses/bsd-license.php -# -# THE PROGRAM IS DISTRIBUTED UNDER THE BSD LICENSE ON AN "AS IS" BASIS, -= # WITHOUT WARRANTIES OR REPRESENTATIONS OF ANY KIND, EITHER EXPRESS OR IMP= LIED. -# - -## -# Import Modules -# -from distutils.core import setup, Extension -import os - -if 'BASE_TOOLS_PATH' not in os.environ: - raise "Please define BASE_TOOLS_PATH to the root of base tools tree" - -BaseToolsDir =3D os.environ['BASE_TOOLS_PATH'] -setup( - name=3D"PyUtility", - version=3D"0.01", - ext_modules=3D[ - Extension( - 'PyUtility', - sources=3D[ - 'PyUtility.c' - ], - include_dirs=3D[ - os.path.join(BaseToolsDir, 'Source', 'C', 'Include'), - os.path.join(BaseToolsDir, 'Source', 'C', 'Include', 'Ia32= '), - os.path.join(BaseToolsDir, 'Source', 'C', 'Common') - ], - ) - ], - ) - diff --git a/BaseTools/Source/Python/Common/Misc.py b/BaseTools/Source/Pyth= on/Common/Misc.py index 430bf6b..3a8e7ca 100644 --- a/BaseTools/Source/Python/Common/Misc.py +++ b/BaseTools/Source/Python/Common/Misc.py @@ -470,23 +470,13 @@ def SaveFileOnChange(File, Content, IsBinaryFile=3DTr= ue): DirName =3D os.getcwd() if not os.access(DirName, os.W_OK): EdkLogger.error(None, PERMISSION_FAILURE, "Do not have write p= ermission on directory %s" % DirName) =20 try: - if GlobalData.gIsWindows: - try: - from .PyUtility import SaveFileToDisk - if not SaveFileToDisk(File, Content): - EdkLogger.error(None, FILE_CREATE_FAILURE, ExtraData= =3DFile) - except: - Fd =3D open(File, "wb") - Fd.write(Content) - Fd.close() - else: - Fd =3D open(File, "wb") - Fd.write(Content) - Fd.close() + Fd =3D open(File, "wb", 0) + Fd.write(Content) + Fd.close() except IOError as X: EdkLogger.error(None, FILE_CREATE_FAILURE, ExtraData=3D'IOError %s= ' % X) =20 return True =20 diff --git a/BaseTools/Source/Python/Common/PyUtility.pyd b/BaseTools/Sourc= e/Python/Common/PyUtility.pyd deleted file mode 100644 index 856b508e4ea39a806fbe52264ac5e4db0956a2de..000000000000000000000000000= 0000000000000 GIT binary patch literal 0 HcmV?d00001 literal 6144 zcmeHLe{hrK9e*LWve_ASI=3Dx2o0fm+ayhs`sRIE ztFXg`&gEXKZh9Oy-AS*z!|n7=3DJsll53AD4Ib%zWqo1NFW>VoaG3PRm#-+i7pDYS^* z?XPWjKDqDnyPsC2^UQ_s)&|ek&^PzraJecO|R>n<%20xWK?Sacd zNTo_cPUOE^HH{l4>VkQyY(nNzM^f0=3D^DycGMrEy6=3D4wJRmF3luxPYKRLjxHNo9MnP zV`}+aAt4)peX@WkTPVkMTNxo+RM(}g6dmayyk_22kf|%x1JQLK4e)E);1XVMHWY(X8jnrM;v&iLBweWenv{@( ziV$r*lSs*^HffZql!0qy^>s`gh4j!1x6Z&oPC(o)peVSiN4#2uH%xJ$t23^ zdt@j!D!*+-7O4s)|Xl4xKok%^ZfZAFh;h=3Dos6h zQ%N~KswUfAQyb68gYevrvv9&E<>b>~wQD3>5g6hQo$KfVkVo^G#Q9S=3DgQfk#&Hf~O zkqx(lAxLW7@*slt_tC6=3DhmIc7u6*zZ)%A#ec5qSBiHl%#rT=3D9x`(H%YpGoPIF?l;k zb#>s&g75%P{^uyyGw70YDJ^cJyo_AO?$L=3D_Xfl{VV^i7Hh~SG_ z*+9{<;_>+#AQTbE?a*T4>r#Ac)5N$*lJYO*Tkh97HA!t%oQh4Tg&eqaA60q&C7@*+ z&5mRfV3s^shj4Wya`!!`P-x2JL1{vA=3Ds?iN_h@4uYfAU%_^Q}yB3zISZPb@6;k;ZT z55^r6DMUO=3D-aL^KD-X&xDumJoCFew}Pscj-VkYa*3NOZ;`nXdUcW9-F(L=3DJRlMTBe zt}Z1+DHo-1m_#=3DXA?uhFwXSshP(z;jdrqCuk)YmhFF*|6K|ZC=3DrB9{pS`g!o5$06f zIT^PNpjzUN$E&L4(2X0@TjQ2bV{dHF2+7^| z7ZQ%Pj>v;ba%FK?9>jgBql^G{<53k8{ z*W@0)cMfx=3DwsV~}zvI%bTctfZsX|x!IiD?U(PsRq^k4kkwXsA|=3D^&qE5-9Ll;sz9X z!72Y^h5lS{xJ1?-BQLUpFPixg=3DsrXY`?@PkslHi}h&` zoXwI$yNu#oDT+&>XhA8J%1p$E)6{7mA@I|k&!R$2vxA*J^#{SNesS4yeg7+XaTA`!EdVl!cmqfBdcU?AnMY@DA-I`g!AV z8#U?jVBAI()O+qZ)hicPV4Ga{6Q!?IdKdcA9Lg%HFZ7!4;l1UI^<}SVd1cX%p6jN!gI>7>@0>SAIa2$MRJht zNh~FYrkh~h>CkrTD86HOQkMHOD?n2b73reH?nxNfpE-^`TSF54nHMor!B!=3D}@s4xY z(nbA@cMM@F8weceE`x}mD#?PcE{KP>21tJ&I@zE_A6rEkh^PgRf%5pNY}ud#4I$QJ zpfTHq5!CCneT6v5fV%)I0E+=3Dyz>o<+1iS#)4`>6l0=3D5Dw0c8Ly;8Or?PcI>45HJE5 z1E_&t0N9WDj{$9fAfO&l30Mgz2HXb70}KFYySSK;_W*wYybSm?-~iy-R(8Wxk7!6~ zs3}jywT%D4npThz_Y6(Sbi)D!kPJoAR*sKM zORCL)=3D=3DI@%;Eedh5Lb-KZWsKKu(f2lFBC$Zsytk{0Us(nt&u=3Dj;CS+&+SB3<;Sa5+ zffvY#rjh3$xp|t;=3DLw70xI^XfakKVm)SdwJKtv14ygK)A#GvwcTcS-Kt}OuH=3D3(Uv zic!JS=3DnL$ke>gmz^48WZ4Xwf*fe^LeAz&IpoHy(d+FLnDZH#!r-WD#3Dd=3DG@?EWCb zBZgsQGuaPrxDp8~;&-V%3U)z;CyjBsYsxpVX2snoyifw@@q5F*5XTpL8X_$%i00iY z>MnfRDK<56yvHAjG!s%z%0rPTSD`SIPE9Qz5I9)Lkv3A!bKcZq4f&mVJ++K#=3D6pyI zq&IP-K;sT^Tq`*;lUg2WY4!5l6(;gJSr>?UcZRrXKP37p13P&y-(HRsL`iiFK*gxAE*E*83kxYh>wUOUW?)@>q=3DAb@f!;{> zy4#}y*HRs5;nqZ{M|4J(X{wRFKp2jt*$Mg8mt~OeYc_{d5v1Ox3NoEMV$h&!zmoi> zdVO?VI>;`bi^5!Tw`0=3D=3DMh1}fsn{^pegtt(DYtKy1l!UWLHO!q2{7* zf6wUjHgm=3D(b{GXe7^y6L1dM{uXA~kvAK38CY4$$6o8hjdKWe*~&aY|IW+A{ujj1zg zT;GDj+=3D9qJYPqozwIq5`PFKCdeq5nSqi-nB-*Rl-H6p)RUM~WOd=3DZi`$NVjGp?RZu zo4LUpH19LNWgasB&HN9u#xl=3DRWLajhTI`m)EN;tIi^n2b9^0MWOt;Dw0w$;{Z+haRoJ7ycUy>EMZ#j`7p zt@z7|PgY!Bv7@B1Bvite#7Z71`9;Z*k}SK{8asvg{gmJzK?YXB*jO_FndWb|3o)`viNC zeU5#JJ<1+u-)7%sKVXS5^;qpO_nA+cbr!wFW!YlcW@)oLZaH8