From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from us-smtp-delivery-1.mimecast.com (us-smtp-delivery-1.mimecast.com [205.139.110.61]) by mx.groups.io with SMTP id smtpd.web12.1665.1575571510141361326 for ; Thu, 05 Dec 2019 10:45:10 -0800 Authentication-Results: mx.groups.io; dkim=pass header.i=@redhat.com header.s=mimecast20190719 header.b=W56A146g; spf=pass (domain: redhat.com, ip: 205.139.110.61, mailfrom: philmd@redhat.com) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1575571509; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=LGDGEMZo1f+SN1dwx9nnplKaHIapikuu0waeWyE2rFM=; b=W56A146gtyeAi9Ip6RSgrKq+aXJiK3ScVs/M3ILXKtYlo6Ze/NhP6Z9VN0X4BMxr/MDCA9 tAD4gfBM5Y3ilqFCIsM8doZ5U1MjPz9UFKXvrdgmRWgs0dB1uYajEKxnPdDKsV8HpI8tjk 7/3GLKSR9uIquIrB0AMl5qVUVPkgU6k= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-434-fddmGAIMNoWdsYVbNs_fdQ-1; Thu, 05 Dec 2019 13:45:05 -0500 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id C12A28017DF; Thu, 5 Dec 2019 18:45:04 +0000 (UTC) Received: from x1w.redhat.com (ovpn-205-76.brq.redhat.com [10.40.205.76]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 0E4815E240; Thu, 5 Dec 2019 18:44:59 +0000 (UTC) From: =?UTF-8?B?UGhpbGlwcGUgTWF0aGlldS1EYXVkw6k=?= To: devel@edk2.groups.io Cc: Zhiju Fan , Philippe Mathieu-Daude , Bob Feng , Liming Gao Subject: [PATCH v2] BaseTools: Fix Python3 encoding issue in TestTools Date: Thu, 5 Dec 2019 19:44:56 +0100 Message-Id: <20191205184456.17799-1-philmd@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 X-MC-Unique: fddmGAIMNoWdsYVbNs_fdQ-1 X-Mimecast-Spam-Score: 0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Under Centos 7.7 we get: Build environment: \ Linux-3.10.0-1062.7.1.el7.x86_64-x86_64-with-centos-7.7.1908-Core [...] =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D ERROR: testRandomDataCycles (TianoCompress.Tests) ---------------------------------------------------------------------- Traceback (most recent call last): File "edk2/BaseTools/Tests/TianoCompress.py", line 60, \ in testRandomDataCycles self.compressionTestCycle(data) File "edk2/BaseTools/Tests/TianoCompress.py", line 46, \ in compressionTestCycle start =3D self.ReadTmpFile('input') File "edk2/BaseTools/Tests/TestTools.py", line 139, in ReadTmpFile data =3D f.read() File "/usr/lib64/python3.6/encodings/ascii.py", line 26, in decode return codecs.ascii_decode(input, self.errors)[0] UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 3: \ ordinal not in range(128) ---------------------------------------------------------------------- Fix by using the 'io' module (per [1]), and specifying the UTF-8 encoding. [1] https://mail.python.org/pipermail/python-list/2015-March/687124.html Cc: Bob Feng Cc: Liming Gao Signed-off-by: Philippe Mathieu-Daude --- v2: Use io, wrap commit message lines per CheckPatch --- BaseTools/Tests/TestTools.py | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/BaseTools/Tests/TestTools.py b/BaseTools/Tests/TestTools.py index 1099fd4eeaea..8aaad3609669 100644 --- a/BaseTools/Tests/TestTools.py +++ b/BaseTools/Tests/TestTools.py @@ -11,6 +11,7 @@ from __future__ import print_function # Import Modules # import base64 +import io import os import os.path import random @@ -18,7 +19,6 @@ import shutil import subprocess import sys import unittest -import codecs =20 TestsDir =3D os.path.realpath(os.path.split(sys.argv[0])[0]) BaseToolsDir =3D os.path.realpath(os.path.join(TestsDir, '..')) @@ -135,7 +135,7 @@ class BaseToolsTest(unittest.TestCase): return open(os.path.join(self.testDir, fileName), mode) =20 def ReadTmpFile(self, fileName): - f =3D open(self.GetTmpFilePath(fileName), 'r') + f =3D io.open(self.GetTmpFilePath(fileName), 'r', encoding=3D'utf-= 8') data =3D f.read() f.close() return data @@ -145,7 +145,7 @@ class BaseToolsTest(unittest.TestCase): with open(self.GetTmpFilePath(fileName), 'wb') as f: f.write(data) else: - with codecs.open(self.GetTmpFilePath(fileName), 'w', encoding= =3D'utf-8') as f: + with io.open(self.GetTmpFilePath(fileName), 'w', encoding=3D'u= tf-8') as f: f.write(data) =20 def GenRandomFileData(self, fileName, minlen =3D None, maxlen =3D None= ): --=20 2.21.0