From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mga14.intel.com (mga14.intel.com [192.55.52.115]) by mx.groups.io with SMTP id smtpd.web11.9345.1597368026711398064 for ; Thu, 13 Aug 2020 18:20:26 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=@intel.onmicrosoft.com header.s=selector2-intel-onmicrosoft-com header.b=zlRI62Fv; spf=pass (domain: intel.com, ip: 192.55.52.115, mailfrom: bob.c.feng@intel.com) IronPort-SDR: zkhHewVBlx7W+sCE7Mk+/j7iVRt43BWThFfkUEnGg/C8MG4FjLFtbYGLvrkwMmxFGYbDlGwmfi RJn/CNp6i/hw== X-IronPort-AV: E=McAfee;i="6000,8403,9712"; a="153570997" X-IronPort-AV: E=Sophos;i="5.76,310,1592895600"; d="scan'208";a="153570997" X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga002.jf.intel.com ([10.7.209.21]) by fmsmga103.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 13 Aug 2020 18:20:25 -0700 IronPort-SDR: rhcCIo19eoy63dTZNcFCBNBgIe1IqO/reXzeVdm/4dRNxJn3n5tK2xqsSfxFpEoZEBUQ8ebFcQ ZNWixy9TqumQ== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.76,310,1592895600"; d="scan'208";a="309242069" Received: from unknown (HELO fmsmsx604.amr.corp.intel.com) ([10.18.84.214]) by orsmga002.jf.intel.com with ESMTP; 13 Aug 2020 18:20:24 -0700 Received: from fmsmsx609.amr.corp.intel.com (10.18.126.89) by fmsmsx604.amr.corp.intel.com (10.18.126.84) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.1713.5; Thu, 13 Aug 2020 18:20:23 -0700 Received: from fmsmsx609.amr.corp.intel.com (10.18.126.89) by fmsmsx609.amr.corp.intel.com (10.18.126.89) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.1713.5; Thu, 13 Aug 2020 18:20:23 -0700 Received: from fmsedg601.ED.cps.intel.com (10.1.192.135) by fmsmsx609.amr.corp.intel.com (10.18.126.89) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.1713.5 via Frontend Transport; Thu, 13 Aug 2020 18:20:23 -0700 Received: from NAM10-DM6-obe.outbound.protection.outlook.com (104.47.58.106) by edgegateway.intel.com (192.55.55.70) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.1.1713.5; Thu, 13 Aug 2020 18:20:23 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=LjlrrokwdIqSGOxx3Ndveknttm4jNbjAWze3zTsM6mcxvLPdTDgHKsa5rjjB3QyvPA37DWJ7Y9ni/pBowaEbWzlK9ZvXDiZHKSYNgaAPP0gbgLoD3+Xf8dU+C1PFPEISnbsAcxXdIGf+poe2d9GkUebb455dIFjlY8+u8TLCJ1N3UiTBSYzNjT66CwQiGOMh3ApzcluHUEAJjKZsGmK9gJzcmu/4UiBrZKgL2AJOALD8mHew5vzfhz4W7Jp9YLju8PIp4BIbM6yb5dSj2LsaCJF7PAoN6ZLq4coJIFdxxNoFR0ojH98UBUEXRbydwFRhn/iJGmlJ8+cZW+fTc2V+rA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=pplb9+Rp5S+DebWeQx4GRgiqbei0RJAXnRYc6HzTr10=; b=PBULddIy8NvM538SfRRf9krntlRaRE6yWr8/IZ90aRK9sK+HTG824GZ9jGYVqerePn1ZvhRePqrrSIvyh1W/YA46HR9vmPjmU5MYlbZQ1KvMzWrq6MFvmvAptAig7X7z9bHPQuimsNxijLlSTQORvZbil+4UWirHwo1xUZHH9ZwW2td+Ky0dfudKHG/TjUrx9p7NpNDjO7kQ17IUQjj4XccdTo9MFssKYd/EjC7mlcGm2XFIwVy7ZcpWjU2T6wewjI6aMfWV8YqrggzXMtAGqJ0uVS6FD/Q6hpdt/6CXNjKkRo3RIuEyRRu6rGe1f2qjJBU7qhfTW1xnuYUx0iW3qQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com; dkim=pass header.d=intel.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=intel.onmicrosoft.com; s=selector2-intel-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=pplb9+Rp5S+DebWeQx4GRgiqbei0RJAXnRYc6HzTr10=; b=zlRI62Fv77XJCGFMh22lGEFgGzzpN0FEPkTM3kjwS6Jo6bPwF57jguo1+6fZ6IQ8eagdb8KSpJDcc44I+/BJSN2fzxcfJPqmUzhwW7lXd8Pq8UID85eOBCprWrxsyMxD76LLhZT7KSsM9y2ivx0bUJaOJWcfo4Rpc3XcKc6vEDw= Received: from DM6PR11MB4073.namprd11.prod.outlook.com (2603:10b6:5:19f::22) by DM6PR11MB3145.namprd11.prod.outlook.com (2603:10b6:5:6e::26) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3261.22; Fri, 14 Aug 2020 01:20:22 +0000 Received: from DM6PR11MB4073.namprd11.prod.outlook.com ([fe80::88bc:d834:df24:ea90]) by DM6PR11MB4073.namprd11.prod.outlook.com ([fe80::88bc:d834:df24:ea90%6]) with mapi id 15.20.3283.020; Fri, 14 Aug 2020 01:20:22 +0000 From: "Bob Feng" To: Cole Robinson , "devel@edk2.groups.io" CC: "Gao, Liming" Subject: Re: [PATCH 1/2] BaseTools: fix ucs-2 lookup on python 3.9 Thread-Topic: [PATCH 1/2] BaseTools: fix ucs-2 lookup on python 3.9 Thread-Index: AQHWcAVhnf1XG7cqlkCrXbEiR2cSoKk20hFw Date: Fri, 14 Aug 2020 01:20:22 +0000 Message-ID: References: <1ded5a5bfc3e3a83eba56fa0875db037e60a7c4e.1597166808.git.crobinso@redhat.com> In-Reply-To: <1ded5a5bfc3e3a83eba56fa0875db037e60a7c4e.1597166808.git.crobinso@redhat.com> Accept-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: redhat.com; dkim=none (message not signed) header.d=none;redhat.com; dmarc=none action=none header.from=intel.com; x-originating-ip: [192.55.46.36] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: a71daf8c-4784-41d6-271f-08d83ff03722 x-ms-traffictypediagnostic: DM6PR11MB3145: x-ms-exchange-transport-forked: True x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:2657; x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: zRxmRemzSQghkqr+qvhDDM3U5YMhoHvlSdg36UtK1gPY24x4KQ//AFVo9bLqSBsaFA+gDx5O5zMV19a85Q14luhvrx6Z60Gt/WhY6OymrarYxvxcHEZhFZJYDvQgkhmi7BPCRQSsoWLDj5G+P9KKpuKnMGvF5yK94Jb//iTxzMg/AKUd9QpURYZM3+09XY8Ten58OKWQJHRt9iaFSdAIPiNHCMiyzobCR6lO0GNVNNhG/ZvjMNS9vY+0DKIelqaWIOTPW86U63oMACSPbfvjsxDfXW8TVtmrz3LkD/OehphVbPuPLo/bmIawIGU+mpe7gygQprNeIAP5gLGRzFHkqwbnxW24+778q0VofdvcqyiXn2qYzFVDiKmN+4BIqMheNU8GAaQ5KsUh3Jdtv6vynw== x-forefront-antispam-report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DM6PR11MB4073.namprd11.prod.outlook.com;PTR:;CAT:NONE;SFTY:;SFS:(4636009)(136003)(396003)(39860400002)(366004)(376002)(346002)(86362001)(4326008)(26005)(76116006)(478600001)(2906002)(9686003)(316002)(71200400001)(66946007)(66446008)(7696005)(64756008)(66556008)(66476007)(8936002)(110136005)(53546011)(8676002)(83380400001)(55016002)(5660300002)(966005)(107886003)(6506007)(52536014)(186003)(33656002)(19627235002);DIR:OUT;SFP:1102; x-ms-exchange-antispam-messagedata: 3WUbwDY1XZNC5nvhr0+gYxtTOE7JXAvf+2+KysT12C53yieYhyyet9GHvI33aQOvA5CnQGe+orNNHNUNS1q+XR40kjgR8qcxxpaYkuL14Fnw1I2AMFhfa1qyUVNYGnylqejc3qNN90lungi0BlllLpF/CjtACURNhNNqV16nt41zYO1BrBG1Q7tCjjhkdgoNLkvE85QMDHCfFjxWfBoKKhvvk7lu6CBCSKBMEKSdJM6rp0Zx0bymrg0Bj6Rh4JtLucV13s8ex1BYcWKBABANVIlrEfh3a735nIxtJTFfrr8/hL3vaCCrWNnh+HU52jFxDmLb9xxpx+iIFaFbB/hSJuiiS1YjA5kHwy8vrolQ806O+Ub8AmJSwSxCKc8NqXv4W1gUdlICy2BtUiZPJ7StVqbZPEZkWfzU3/FIHF5DJHsWOhJee/jWf96peYZQ75tBw6LIPHU/puvrEc+H+qn38w5UUywz3SEjFNr7ahqTWdF0QPl0UnHWv1Zl7n3YHU3i2LLABSrklYNv9mOGOyC66LlFORXfKDeoKyHBcLzpUljI+JcHn8pRXYsBt/9jAA9dG0KXPn8NvnlsCXPel7LSBtZ/sGff6MeWHQ/ELNEUqhBIbFmo9+LnECn2SGsMsdTdV6G0DHXTwQ1O6tO38LUpTg== MIME-Version: 1.0 X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: DM6PR11MB4073.namprd11.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: a71daf8c-4784-41d6-271f-08d83ff03722 X-MS-Exchange-CrossTenant-originalarrivaltime: 14 Aug 2020 01:20:22.0412 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 46c98d88-e344-4ed4-8496-4ed7712e255d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: Oue0paGcj8iECf+yXeas/hWJLfQTeZx4dWZ7L6yBHcEBkjFshr/pQ9cFEGIHPZ3VHuSGLC25yIgFHmoIwVjbpg== X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6PR11MB3145 Return-Path: bob.c.feng@intel.com X-OriginatorOrg: intel.com Content-Language: en-US Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Reviewed-by: Bob Feng -----Original Message----- From: Cole Robinson =20 Sent: Wednesday, August 12, 2020 1:28 AM To: devel@edk2.groups.io Cc: Feng, Bob C ; Gao, Liming ;= Cole Robinson Subject: [PATCH 1/2] BaseTools: fix ucs-2 lookup on python 3.9 python3.9 changed/fixed codec.register behavior to always replace hyphen wi= th underscore for passed in codec names: https://bugs.python.org/issue37751 So the custom Ucs2Search needs to be adapted to handle 'ucs_2' in addition = to existing 'ucs-2' for back compat. This fixes test failures on python3.9, example: =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D FAIL: testUtf16InUniFile (CheckUnicodeSourceFiles.Tests) ---------------------------------------------------------------------- Traceback (most recent call last): File "/builddir/build/BUILD/edk2-edk2-stable202002/BaseTools/Source/Pytho= n/AutoGen/UniClassObject.py", line 375, in PreProcess FileIn =3D UniFileClassObject.OpenUniFile(LongFilePath(File.Path)) File "/builddir/build/BUILD/edk2-edk2-stable202002/BaseTools/Source/Pytho= n/AutoGen/UniClassObject.py", line 303, in OpenUniFile UniFileClassObject.VerifyUcs2Data(FileIn, FileName, Encoding) File "/builddir/build/BUILD/edk2-edk2-stable202002/BaseTools/Source/Pytho= n/AutoGen/UniClassObject.py", line 312, in VerifyUcs2Data Ucs2Info =3D codecs.lookup('ucs-2') LookupError: unknown encoding: ucs-2 Signed-off-by: Cole Robinson --- BaseTools/Source/Python/AutoGen/UniClassObject.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/BaseTools/Source/Python/AutoGen/UniClassObject.py b/BaseTools/= Source/Python/AutoGen/UniClassObject.py index b2895f7e5c..883c2356e0 100644 --- a/BaseTools/Source/Python/AutoGen/UniClassObject.py +++ b/BaseTools/Source/Python/AutoGen/UniClassObject.py @@ -152,7 +152,7 @@ class Ucs2Codec(codecs.Codec): TheUcs2Codec =3D Ucs2Codec() def Ucs2Search(name):- if name =3D=3D 'uc= s-2':+ if name in ['ucs-2', 'ucs_2']: return codecs.CodecInfo( = name=3Dname, encode=3DTheUcs2Codec.encode,--=20 2.26.2