From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mout.gmx.net (mout.gmx.net [212.227.17.21]) by mx.groups.io with SMTP id smtpd.web10.8146.1574590530243743714 for ; Sun, 24 Nov 2019 02:15:30 -0800 Authentication-Results: mx.groups.io; dkim=pass header.i=@gmx.net header.s=badeba3b8450 header.b=DB0AG/M3; spf=pass (domain: gmx.de, ip: 212.227.17.21, mailfrom: xypron.glpk@gmx.de) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=gmx.net; s=badeba3b8450; t=1574590521; bh=MnbkWGMK3nBGNBOwdPOYHCwit/pGOHqeXUjBuvzG8Co=; h=X-UI-Sender-Class:From:To:Cc:Subject:Date; b=DB0AG/M3L8a1ojD+jiJpNOdVMJnKId6HfyeOgKbsHmIK6RcmzP4hiKlKsz0dv9I1e aJyk+l9oaH2UDqbkFXYiw8OLtQZdeJxUrmBJlxFJ3iLZfsKmMYE1Xb4pQuHtqo3at2 /hD3hGKSoXb7AM0us5FaG28NYUN3DTzzX7YFDhko= X-UI-Sender-Class: 01bb95c1-4bf8-414a-932a-4f6e2808ef9c Received: from LT02.fritz.box ([94.114.42.168]) by mail.gmx.com (mrgmx104 [212.227.17.174]) with ESMTPSA (Nemesis) id 1MfYLQ-1ht8RV2vNo-00fzSE; Sun, 24 Nov 2019 11:15:21 +0100 From: "Heinrich Schuchardt" To: EDK II Development , liminig.gao@intel.com Cc: michael.d.kinney@intel.com, Heinrich Schuchardt Subject: [edk] [PATCH] ShellPkg/edit: allow non-ASCII characters in edit Date: Sun, 24 Nov 2019 11:15:12 +0100 Message-Id: <20191124101512.20964-1-xypron.glpk@gmx.de> X-Mailer: git-send-email 2.24.0 MIME-Version: 1.0 X-Provags-ID: V03:K1:FEddyqiArzPoO9p9gU/jqlHwSawYQ3sRBX5rO5MB1kAzcAJfMoO sHz6uwnmPdhH1AcChB+ScU7CFLMh2GEyYchUYqdJ41Rh72h+Zf+QNRDOpdq3aPMOKzOKEuD HqHUx3mMkhGcyHlgQ2GKN4kY+rMfRoSL7/eBPG0djK9KxcLMJASLLESvg+ii5dgFWeSaCix JTMfs4TnB/+u86TAltG2Q== X-Spam-Flag: NO X-UI-Out-Filterresults: notjunk:1;V03:K0:udNQi9UrLYk=:YvBwS36XcMhAMyCWjMgGuH rDfAyva9KKR08QqkFkGWnxAuzdmxLSjdBI1RdDyUEBy7weN03bA0e5xIaorLllzYa8pCK9omq kUnNGibIrfU16gR8oLZgshN6D01rQzulAU8lgOxrTuK3oHvyA67uVmYuStfyEt2VshJeeoz7z UlO+MEujHIKDVhsK/mqzpudejv7e+fvcHGUoC8YKjODLCqp2M8088+7zHzXp9vk4UfdxIzFYs UQ7DFq5rGj/qBkPCNOp0nC2BhKt1DcAHfrNX4qTG2bWe8gqMm9yVrCpweXThMnfkzOBQdseok 4x9Dee2h6ZgjWHx3Dm7o3cmw3PruQfKX4MAEDZfKcb+UgUCEHwttcwoWHIUODObHASZ6hMxUh oVnmMhdS4qrCDSQHC54tZA+LYCZ//rEeysOlBYfhLqA03jiv+wbpU/t+tZ7XqUygdwf9xI9Tx WNrH4LEpoLcR/p+fZz4TT+2pgK1gpw/gWbGZ0K6SFjUGunWQb/IJRwYNyRCYHAo8buNuJt9p7 tJPy0jBDcsLCNVihJw1rr70nOFAAoWg1OnOAqDMD94fLk6d3sAASQBXa/xmJ4SLmYb1s1Lcai hGX/O4I6h3KDebLi7WNZuotykRflJDULP+LC/mFLnFox01k/RCDepUJKcH8HzIBM4IGxS0gbu iC4hiNmUuen/SnqGG5qe5eF6P14Nal2DoVho35l0MEqd+Lzvw55d2jX7ye4QB5upt4Ep4EVFY bdVG8XAKMxWRxZ7f1w+p5Lp1jd0MgKl5LBqL3ttqW8mkZ2reGwGqcjUKXhCqbjyWxqJcNFxDu +XRskQEvhCHpdftOGjneWAuDLnqWvGcp46P4b3xLauaxRQthVp97r/EeJRD56AXKFh9t3jFDa RvDNq2W1B6KOWLdPa4Qdd8MVkhrzyvwju7lY+lDDmNTnpuaadTwq20flKBEDCS3DYItIokxZl yDK5OlqnMgKomAMRARAfaHlBVWuu0w8dJbr2Oyu/VFs1f77uY8E4BhhToCpVrrb7ACK+UZyoF hGs6mHzUSov1G6psJt0BMan9jxxy26PhWf5ViQIYIQf5OAvTJKj0vFr7n5J7n/qjYOH4GWt7E T9d0KFuJhU2eD0fZsnwXzlQSG+emQmdfT/zZnJ+G5NvNM3lC3S8eIgxtjGa6QFpADmYG9GZ5p 4JYvfxEE7Vwdl2TY4Qer/q3jEa9QbjFRKrBzm4+EcS7MVKH5pbRy/gBwVoaoHb0wtRRF9ldiX Klo8Y8fBT6nZkubWkRmHi4xVVRnY48uY0g6Pqy+UUC4MGgiBH7ChwJIjdR5g= Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable REF: https://bugzilla.tianocore.org/show_bug.cgi?id=3D2339 Currently it is not possible to add letters outside the range U+0020 - U+007F to a file using the edit command. In Unicode the following are control characters: * U+0000=E2=80=94U+001F (C0 controls) * U+007F (DEL) * U+0080=E2=80=94U+009F (C1 controls). For reference see: * https://unicode.org/charts/PDF/U0000.pdf * https://unicode.org/charts/PDF/U0080.pdf So the characters we should exclude from the file buffer are: U+0000 - U+001f, U+007f - U009F Allow all other characters as input to the file buffer in Unicode mode. Allow only ASCII characters as input in ASCII mode. When saving a file in ASCII mode replace non-ASCII characters by a question mark ('?'). Signed-off-by: Heinrich Schuchardt --- .../Edit/FileBuffer.c | 21 +++++++++++++++---- 1 file changed, 17 insertions(+), 4 deletions(-) diff --git a/ShellPkg/Library/UefiShellDebug1CommandsLib/Edit/FileBuffer.c = b/ShellPkg/Library/UefiShellDebug1CommandsLib/Edit/FileBuffer.c index fd324cc4a8..12235e4e4b 100644 --- a/ShellPkg/Library/UefiShellDebug1CommandsLib/Edit/FileBuffer.c +++ b/ShellPkg/Library/UefiShellDebug1CommandsLib/Edit/FileBuffer.c @@ -1360,6 +1360,8 @@ GetNewLine ( /**=0D Change a Unicode string to an ASCII string.=0D =0D + Non-ASCII characters are replaced by '?'.=0D +=0D @param[in] UStr The Unicode string.=0D @param[in] Length The maximum size of AStr.=0D @param[out] AStr ASCII string to pass out.=0D @@ -1378,8 +1380,12 @@ UnicodeToAscii ( //=0D // just buffer copy, not character copy=0D //=0D - for (Index =3D 0; Index < Length; Index++) {=0D - *AStr++ =3D (CHAR8) *UStr++;=0D + for (Index =3D 0; Index < Length; Index++, UStr++) {=0D + if (*UStr < 0x80) {=0D + *AStr++ =3D (CHAR8) *UStr;=0D + } else {=0D + *AStr++ =3D '?';=0D + }=0D }=0D =0D return Index;=0D @@ -2154,9 +2160,16 @@ FileBufferDoCharInput ( =0D default:=0D //=0D - // DEAL WITH ASCII CHAR, filter out thing like ctrl+f=0D + // Do not add Unicode control characters to the file buffer:=0D + //=0D + // * U+0000-U+001f (C0 controls)=0D + // * U+007f (DEL)=0D + // * U+0080-U+009f (C1 controls)=0D + //=0D + // Do not add non-ASCII characters in ASCII mode.=0D //=0D - if (Char > 127 || Char < 32) {=0D + if (Char < 0x20 || (Char >=3D 0x7f &&=0D + (Char <=3D 0x9f || FileBuffer.FileType =3D=3D FileTypeAscii))) {=0D Status =3D StatusBarSetStatusString (L"Unknown Command");=0D } else {=0D Status =3D FileBufferAddChar (Char);=0D --=20 2.24.0