From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received-SPF: Pass (sender SPF authorized) identity=mailfrom; client-ip=104.47.40.109; helo=nam03-co1-obe.outbound.protection.outlook.com; envelope-from=sean.brogan@microsoft.com; receiver=edk2-devel@lists.01.org Received: from NAM03-CO1-obe.outbound.protection.outlook.com (mail-co1nam03on0109.outbound.protection.outlook.com [104.47.40.109]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (No client certificate requested) by ml01.01.org (Postfix) with ESMTPS id DBDAB21B02822 for ; Wed, 7 Nov 2018 23:10:41 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=P0VqRnqVpbjlEG0mWZb3JU4TDITzHfnbhBTQZmr6nc4=; b=OfIx6Mxy2MnSavTHlsqdLo0TOBZh+vks8JPv2iLeLVpIn9CjHgOu/ALMz2fRcL17AzsIHV63shJ7aoijkMlNAdSWnRVvNGldHJY9/07Fd3A8wIOyj9gB8thcW3iFznHAblx47ddFSPXQ6qndcSVajk+pUco/0ekWWiE7Y2+5juk= Received: from DM5PR21MB0185.namprd21.prod.outlook.com (10.173.173.136) by DM5PR21MB0122.namprd21.prod.outlook.com (10.173.173.9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.1339.1; Thu, 8 Nov 2018 07:10:33 +0000 Received: from DM5PR21MB0185.namprd21.prod.outlook.com ([fe80::85ba:98bc:9456:eebf]) by DM5PR21MB0185.namprd21.prod.outlook.com ([fe80::85ba:98bc:9456:eebf%3]) with mapi id 15.20.1339.009; Thu, 8 Nov 2018 07:10:33 +0000 From: Sean Brogan To: "Gao, Liming" CC: "edk2-devel@lists.01.org" Thread-Topic: Edk2 uni file encoding Thread-Index: AdR20C6rSm7ksjUET/Kmuyj3hkLhKAAV0kSAAAKf6tA= Date: Thu, 8 Nov 2018 07:10:32 +0000 Message-ID: References: <4A89E2EF3DFEDB4C8BFDE51014F606A14E366631@SHSMSX104.ccr.corp.intel.com> In-Reply-To: <4A89E2EF3DFEDB4C8BFDE51014F606A14E366631@SHSMSX104.ccr.corp.intel.com> Accept-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [50.35.66.243] x-ms-publictraffictype: Email x-microsoft-exchange-diagnostics: 1; DM5PR21MB0122; 6:RMaqYaXmmT6ea4qJ3PU9ucp4wIKtaJLSOEqpesGXg6zAr79OJ79HUt6JSQ3/RDw6Fh1shvRhu8+x7aUVeG8SpVZhc1QGqp4ftlahgTzCcKc+zwYH7dPp9b8P1qHH+cN1kghziX3+FQGMidKRSe+KOlKzMutck2gDqEMfsZ2j0E7naz/Pyjw7GAAFqaKUvKHji0/mJ604nMta9BsiR3Rnht+bhUtJNEzLVmFOBuVdXTXMtVsJp8bhgbZYcRn/rY8/0qTG6P490dttiTlbJOWVv32pR75rTMEbfb+w9GanXK4evcBUVmNVvkROfMIGnheHVNIZL7rJF0WEtpbLKEEEyFuAtq1FprdL/VXwR9yx55stZ5unCQ3YxfnJbKgDscd60YoM2HI1LzkAkqLBKn1ttzI1tarznYpOy3xfNklTq+dz7XGpVTMq/ExHTDEcv9sJOH8p2o4dJzF6wGcuq8CdNw==; 5:3eHtQct3FeOlJEfOL1/C+pLObmmHWROKy7nHnZDtSt+1Lv+h20zXkiQp2XDYzdA68kQlvY5tlN9aA+2ZqY0sTuTRXXGiJFM5w7QanMrC1ReEYC2MspP7AoGPtRDTr8AA9C7R1do+T1IEVRtW0pNPUQtgtZCwLsHzuEBfrZVHJjk=; 7:OHnjf1TGBG/Ogze19L1KvhJOAqqSDc6vho7o2HAJXSNZ10NBlILNL/xlocW0HfuTZqg5zxu2C87PiA9UF2e6z4mmxHx0BBcCiBar0so36Thj20Ym3uDvmWsEB2LuG0d5hedWkJcFgbI9evW+VjpRyg== x-ms-exchange-antispam-srfa-diagnostics: SOS; x-ms-office365-filtering-correlation-id: f72f27c2-7f01-47e3-e514-08d645494614 x-ms-office365-filtering-ht: Tenant x-microsoft-antispam: BCL:0; PCL:0; RULEID:(7020095)(4652040)(8989299)(5600074)(711020)(4618075)(4534185)(4627221)(201703031133081)(201702281549075)(8990200)(2017052603328)(7193020); SRVR:DM5PR21MB0122; x-ms-traffictypediagnostic: DM5PR21MB0122: x-ms-exchange-purlcount: 4 authentication-results: spf=none (sender IP is ) smtp.mailfrom=sean.brogan@microsoft.com; x-microsoft-antispam-prvs: x-ms-exchange-senderadcheck: 1 x-exchange-antispam-report-cfa-test: BCL:0; PCL:0; RULEID:(8211001083)(6040522)(8220035)(2401047)(8121501046)(5005006)(3231390)(944501410)(2018427008)(93006095)(93001095)(3002001)(10201501046)(6055026)(148016)(149066)(150057)(6041310)(20161123564045)(20161123560045)(201703131423095)(201702281528075)(20161123555045)(201703061421075)(20161123562045)(20161123558120)(201708071742011)(7699051)(76991095); SRVR:DM5PR21MB0122; BCL:0; PCL:0; RULEID:; SRVR:DM5PR21MB0122; x-forefront-prvs: 0850800A29 x-forefront-antispam-report: SFV:NSPM; SFS:(10019020)(366004)(396003)(136003)(376002)(39860400002)(346002)(199004)(189003)(13464003)(186003)(25786009)(229853002)(53936002)(97736004)(966005)(10290500003)(22452003)(508600001)(316002)(8990500004)(446003)(3846002)(6116002)(2906002)(256004)(14444005)(74316002)(10090500001)(71190400001)(71200400001)(305945005)(7736002)(6246003)(68736007)(4326008)(44832011)(476003)(486006)(2900100001)(11346002)(76176011)(9686003)(6306002)(5660300001)(561944003)(7696005)(8936002)(53546011)(86612001)(86362001)(81166006)(81156014)(575784001)(102836004)(6506007)(105586002)(6436002)(106356001)(66066001)(55016002)(33656002)(99286004)(14454004)(8676002)(6916009)(26005); DIR:OUT; SFP:1102; SCL:1; SRVR:DM5PR21MB0122; H:DM5PR21MB0185.namprd21.prod.outlook.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; A:1; MX:1; received-spf: None (protection.outlook.com: microsoft.com does not designate permitted sender hosts) x-microsoft-antispam-message-info: +7MCfzb25kFuicvBEWeVBUDdoLVpXIhRMX/Y2XuvbvRFuQJdEpqoN8xtUSaz8GgT9TYDRi6KAqT8TI4Atf+VbNTAK36zTr14RfTnNgaSnk/R1G2SVqn81FQpLtTmBec3reLg6GdU23GY5FRJFLPu9PI+GH+lV/hxlxAXg+hRpz/HXd5yNkmexoZ+Zed2Jtyx9fxhqKFJrBdiz6e9I5zhefg9QAfWSbP+EGgzzC5yHvFWCG4GgMdA2fosiABNOF5IBwnDgquPjIO++y/JO6J1NbpXvfhiYLcjPLQNKEdNnA9rbelnj3LKkeQqOgKO+7TaaDhkp9tu174LU1hIc1Jyrhoc+jSLFaAm+S3GLl6i1HE= spamdiagnosticoutput: 1:99 spamdiagnosticmetadata: NSPM MIME-Version: 1.0 X-OriginatorOrg: microsoft.com X-MS-Exchange-CrossTenant-Network-Message-Id: f72f27c2-7f01-47e3-e514-08d645494614 X-MS-Exchange-CrossTenant-originalarrivaltime: 08 Nov 2018 07:10:32.8706 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 72f988bf-86f1-41af-91ab-2d7cd011db47 X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM5PR21MB0122 Subject: Re: Edk2 uni file encoding X-BeenThere: edk2-devel@lists.01.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: EDK II Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 08 Nov 2018 07:10:42 -0000 Content-Language: en-US Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Liming, That was exactly what I was looking for. =20 Thanks Sean -----Original Message----- From: Gao, Liming =20 Sent: Wednesday, November 7, 2018 10:01 PM To: Sean Brogan Cc: edk2-devel@lists.01.org Subject: RE: Edk2 uni file encoding Sean: EDKII UNI spec (https://na01.safelinks.protection.outlook.com/?url=3Dhttp= s%3A%2F%2Fgithub.com%2Ftianocore%2Ftianocore.github.io%2Fwiki%2FEDK-II-Spec= ifications&data=3D02%7C01%7Csean.brogan%40microsoft.com%7C5ffeb105737e4= c00150208d6453fa46a%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C6367725369= 83024335&sdata=3Dveov60rbEtr3ub7RcreuFuqJvc4%2BdtAowph7kBGXW54%3D&r= eserved=3D0) Chapter 2 defines UNI file format. EdkCompatibilityPkg is obso= lete. BZ https://na01.safelinks.protection.outlook.com/?url=3Dhttps%3A%2F%2= Fbugzilla.tianocore.org%2Fshow_bug.cgi%3Fid%3D1103&data=3D02%7C01%7Csea= n.brogan%40microsoft.com%7C5ffeb105737e4c00150208d6453fa46a%7C72f988bf86f14= 1af91ab2d7cd011db47%7C1%7C0%7C636772536983024335&sdata=3DLOLezJzuK9kwu8= QK78UM5nnCD%2FZEY5fxr1VQzk8sqY8%3D&reserved=3D0 is submitted to delete = EdkCompatibilityPkg from edk2/master. We will work on it.=20 EDK II Unicode files are used for mapping token names to localized strings = that are identified by an RFC4646 language code. The format for storing EDK= II Unicode files on disk is UTF-8 (without a BOM character) or UTF-16LE (w= ith a BOM character). The character content must be UCS-2. Thanks Liming >-----Original Message----- >From: edk2-devel [mailto:edk2-devel-bounces@lists.01.org] On Behalf Of=20 >Sean Brogan via edk2-devel >Sent: Thursday, November 08, 2018 7:00 AM >To: edk2-devel@lists.01.org >Subject: [edk2] Edk2 uni file encoding > >Is there a definitive answer for the file encoding for all UNI files in ed= k2? >If not I would like to propose one. Incorrect encoding causes tool=20 >issues and is something we can easily check for and fix. > >Proposal: All UNI files in edk2 should be > > > 1. UTF-8 >Or > > 1. Use a BOM and be UTF-16 > >https://na01.safelinks.protection.outlook.com/?url=3Dhttps%3A%2F%2Fen.wik >ipedia.org%2Fwiki%2FByte_order_mark&data=3D02%7C01%7Csean.brogan%40mi >crosoft.com%7C5ffeb105737e4c00150208d6453fa46a%7C72f988bf86f141af91ab2d >7cd011db47%7C1%7C0%7C636772536983024335&sdata=3D1IET4LN5l9FfMscffzgk0 >t7IqYGyYNU9IrZafvi9osU%3D&reserved=3D0 > >Results from searching edk2: >1 - UTF-16 LE BOM file: >EdkCompatibilityPkg\Compatibility\FrameworkHiiOnUefiHiiThunk\Strings.un >i >919 - Without BOM and decoded as UTF-8 > >Thoughts? > >Future question: Can we make rule for all other standard file types=20 >(c, h, dec, dsc, fdf, inf,)? > >Thanks >Sean > > > >_______________________________________________ >edk2-devel mailing list >edk2-devel@lists.01.org >https://na01.safelinks.protection.outlook.com/?url=3Dhttps%3A%2F%2Flists. >01.org%2Fmailman%2Flistinfo%2Fedk2-devel&data=3D02%7C01%7Csean.brogan >%40microsoft.com%7C5ffeb105737e4c00150208d6453fa46a%7C72f988bf86f141af9 >1ab2d7cd011db47%7C1%7C0%7C636772536983024335&sdata=3DHhfPaCyS0sKHu1fF >Gkfh%2FQ4pm34X68YKiaM6IN7%2Fzj0%3D&reserved=3D0