From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received-SPF: Pass (sender SPF authorized) identity=mailfrom; client-ip=40.107.74.90; helo=nam01-bn3-obe.outbound.protection.outlook.com; envelope-from=sean.brogan@microsoft.com; receiver=edk2-devel@lists.01.org Received: from NAM01-BN3-obe.outbound.protection.outlook.com (mail-eopbgr740090.outbound.protection.outlook.com [40.107.74.90]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (No client certificate requested) by ml01.01.org (Postfix) with ESMTPS id 6CED421A00AE6 for ; Wed, 7 Nov 2018 15:00:26 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=DWki0NGsvRTuPsA4UwrcJn3MCNgsIfcFSc8VJiFEqis=; b=H/WnSunw0z7B9bm+Ax1YX/BgeG4Z5L93xNu/ZejBWzLQ+w6q8GTc6RVRhhtmVjbfR/bApIh+0X6iooGH5S54fVsIR1DdcUDfMuEhlJXRrxSo5sEvvWwpqMpLByEZIR6hRR2nWQ7nXojJ7vIjzcmxe+vnKLHxB4JVeP0qQ3A1+K8= Received: from DM5PR21MB0185.namprd21.prod.outlook.com (10.173.173.136) by DM5PR21MB0859.namprd21.prod.outlook.com (10.173.172.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.1339.7; Wed, 7 Nov 2018 23:00:24 +0000 Received: from DM5PR21MB0185.namprd21.prod.outlook.com ([fe80::85ba:98bc:9456:eebf]) by DM5PR21MB0185.namprd21.prod.outlook.com ([fe80::85ba:98bc:9456:eebf%3]) with mapi id 15.20.1339.009; Wed, 7 Nov 2018 23:00:24 +0000 From: Sean Brogan To: "edk2-devel@lists.01.org" Thread-Topic: Edk2 uni file encoding Thread-Index: AdR20C6rSm7ksjUET/Kmuyj3hkLhKA== Date: Wed, 7 Nov 2018 23:00:24 +0000 Message-ID: Accept-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: msip_labels: MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_Enabled=True; MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_SiteId=72f988bf-86f1-41af-91ab-2d7cd011db47; MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_Owner=sebrogan@microsoft.com; MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_SetDate=2018-11-07T23:00:22.7063688Z; MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_Name=General; MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_Application=Microsoft Azure Information Protection; MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_Extended_MSFT_Method=Automatic; Sensitivity=General x-originating-ip: [2001:4898:80e8:0:b95d:1816:5ae8:61e6] x-ms-publictraffictype: Email x-microsoft-exchange-diagnostics: 1; DM5PR21MB0859; 6:7Vk9g26ls17RZ2eSdOZDwq4RNQN8KFgA4d7PJUNiS/hsreVpg+4DD5789TmwNPG2puJ8LxQ8EAbujFQsG9OV3ORS58692dNwnJXCJAtv+nZ1eaAo2RkiVBs36GmqaiDHgbVHMMKdcOZFRzXGytPQjS5MLW7qzLpEAgtiMNfWAqqs/jpH8+qwNIFwkU543vdjDJi8FnRszEmf9O1NMdMOym9FcfvyEWoShNVmMKAuY26nXeT4fb3uFd5+ELTKobhhyw8uoBXhFOLVA9fd+6vqi5ZtwqOTgM3i2UkODKnG69FxPelgV/j6jDRNWJWpI6mWNVony1pqiBI9nGL4w3yGhbWLQs2K6+xuWcpmtIxw/6o0vY4e8/WFRKPSf5xcaJHnTvgEiqKsM2gJURMb0+xrj6OvqkUUJVpwZm4kK8IKRk/acqivZU1ezuUAQc+0LavQpnYxGnVjK9A9tjSOlx/tcA==; 5:B0cdRMYQ2evSJGS1NxyXeqdNPy5bEa8wIO4L/umj/fMU+0+3lDs2K2L85vBiFjspujtH/6RNhxVs2W3j+1J02+3c0lgrvuGd9hhv00fXrnnnzGJYHYsVDpmidqAr1w3MXTcsUlQK4Gdiw0vxFL29Jixx6SbCnkNl3eHWP1cUoVo=; 7:CVx5529IdEQhkjA6FctdIil4evDMQ/PTXKrevkPRHYD06szlIbftWz0ebeTX7YMWERavNQG5o0V37B3J+H2KDUdjvYpSmZovoJsJn6pzbkxTyypmXvvxLenZRY5JmE/ih9//pPp0JhvVrEi83TgeJg== x-ms-exchange-antispam-srfa-diagnostics: SOS; x-ms-office365-filtering-correlation-id: 3a5c3bef-e4e1-456c-4a5c-08d64504ccfd x-ms-office365-filtering-ht: Tenant x-microsoft-antispam: BCL:0; PCL:0; RULEID:(7020095)(4652040)(8989299)(4534185)(4627221)(201703031133081)(201702281549075)(8990200)(5600074)(711020)(4618075)(2017052603328)(7193020); SRVR:DM5PR21MB0859; x-ms-traffictypediagnostic: DM5PR21MB0859: x-ms-exchange-purlcount: 4 authentication-results: spf=none (sender IP is ) smtp.mailfrom=sean.brogan@microsoft.com; x-microsoft-antispam-prvs: x-ms-exchange-senderadcheck: 1 x-exchange-antispam-report-cfa-test: BCL:0; PCL:0; RULEID:(8211001083)(6040522)(8220035)(2401047)(8121501046)(5005006)(3002001)(3231382)(944501410)(2018427008)(93006095)(93001095)(10201501046)(6055026)(148016)(149066)(150057)(6041310)(201703131423095)(201702281528075)(20161123555045)(201703061421075)(201703061406153)(20161123562045)(20161123558120)(20161123560045)(20161123564045)(201708071742011)(7699051)(76991095); SRVR:DM5PR21MB0859; BCL:0; PCL:0; RULEID:; SRVR:DM5PR21MB0859; x-forefront-prvs: 08497C3D99 x-forefront-antispam-report: SFV:NSPM; SFS:(10019020)(366004)(39860400002)(346002)(376002)(136003)(396003)(199004)(189003)(53936002)(6116002)(68736007)(106356001)(561944003)(5640700003)(2351001)(9686003)(55016002)(236005)(105586002)(10090500001)(54896002)(6436002)(8990500004)(81166006)(81156014)(8936002)(74316002)(6916009)(7736002)(6306002)(14454004)(22452003)(5660300001)(486006)(7696005)(6506007)(71200400001)(14444005)(99286004)(71190400001)(186003)(44832011)(46003)(2900100001)(476003)(8676002)(790700001)(2501003)(256004)(10290500003)(102836004)(606006)(316002)(966005)(33656002)(2906002)(478600001)(97736004)(25786009)(86612001)(86362001); DIR:OUT; SFP:1102; SCL:1; SRVR:DM5PR21MB0859; H:DM5PR21MB0185.namprd21.prod.outlook.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; A:1; MX:1; received-spf: None (protection.outlook.com: microsoft.com does not designate permitted sender hosts) x-microsoft-antispam-message-info: laIZt9OmAX0yWJhq9Lt8BAbECGNAAtEK+McuvReQ0bcy75Feclzx7B6NAYuFZ7E7buGcQu3dp0t/7ZeU7oK0awsVI8c+A3mSwD2jg48d2hU7SSjpxd/nW/h+UVCB0ZUO2jBRvXNTWmCI/BZn+PtbcTz3uE262r3zpBdwSpaRUVUxP2GBkXgzY2Kf5GkjPvILunLjkzC8wyRYG8C3DRNKgvdLXsv+hWxartacWGgRUDWP2FiZ+LQX05hcZMQ2QtEA0/GVlTkQ1uANw/ULGxVDKmy/Bp+grypOTcmencuYSsCkhuyWrbnap29tuLavnEz07rkf2u69OHmWbIOQl135VGY4R2p/poWhClTtglOM21k= spamdiagnosticoutput: 1:99 spamdiagnosticmetadata: NSPM MIME-Version: 1.0 X-OriginatorOrg: microsoft.com X-MS-Exchange-CrossTenant-Network-Message-Id: 3a5c3bef-e4e1-456c-4a5c-08d64504ccfd X-MS-Exchange-CrossTenant-originalarrivaltime: 07 Nov 2018 23:00:24.0290 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 72f988bf-86f1-41af-91ab-2d7cd011db47 X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM5PR21MB0859 X-Content-Filtered-By: Mailman/MimeDel 2.1.29 Subject: Edk2 uni file encoding X-BeenThere: edk2-devel@lists.01.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: EDK II Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 07 Nov 2018 23:00:27 -0000 Content-Language: en-US Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Is there a definitive answer for the file encoding for all UNI files in edk= 2? If not I would like to propose one. Incorrect encoding causes tool issues = and is something we can easily check for and fix. Proposal: All UNI files in edk2 should be 1. UTF-8 Or 1. Use a BOM and be UTF-16 https://en.wikipedia.org/wiki/Byte_order_mark Results from searching edk2: 1 - UTF-16 LE BOM file: EdkCompatibilityPkg\Compatibility\FrameworkHiiOnUef= iHiiThunk\Strings.uni 919 - Without BOM and decoded as UTF-8 Thoughts? Future question: Can we make rule for all other standard file types (c, h,= dec, dsc, fdf, inf,)? Thanks Sean