From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received-SPF: Pass (sender SPF authorized) identity=mailfrom; client-ip=134.134.136.65; helo=mga03.intel.com; envelope-from=liming.gao@intel.com; receiver=edk2-devel@lists.01.org Received: from mga03.intel.com (mga03.intel.com [134.134.136.65]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ml01.01.org (Postfix) with ESMTPS id 066AE211C385C for ; Thu, 31 Jan 2019 00:07:45 -0800 (PST) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga007.fm.intel.com ([10.253.24.52]) by orsmga103.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 31 Jan 2019 00:07:45 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.56,543,1539673200"; d="scan'208";a="118897683" Received: from fmsmsx103.amr.corp.intel.com ([10.18.124.201]) by fmsmga007.fm.intel.com with ESMTP; 31 Jan 2019 00:07:44 -0800 Received: from FMSMSX109.amr.corp.intel.com (10.18.116.9) by FMSMSX103.amr.corp.intel.com (10.18.124.201) with Microsoft SMTP Server (TLS) id 14.3.408.0; Thu, 31 Jan 2019 00:07:44 -0800 Received: from shsmsx101.ccr.corp.intel.com (10.239.4.153) by fmsmsx109.amr.corp.intel.com (10.18.116.9) with Microsoft SMTP Server (TLS) id 14.3.408.0; Thu, 31 Jan 2019 00:07:43 -0800 Received: from shsmsx104.ccr.corp.intel.com ([169.254.5.102]) by SHSMSX101.ccr.corp.intel.com ([169.254.1.110]) with mapi id 14.03.0415.000; Thu, 31 Jan 2019 16:07:42 +0800 From: "Gao, Liming" To: "Zhang, Shenglei" , "edk2-devel@lists.01.org" CC: Mike Turner , "Kinney, Michael D" Thread-Topic: [PATCH v5] MdePkg/BaseLib: Add Base64Encode() and Base64Decode() Thread-Index: AQHUuTfWXc143etGtkKvISHwE1s52qXJBZqQ Date: Thu, 31 Jan 2019 08:07:41 +0000 Message-ID: <4A89E2EF3DFEDB4C8BFDE51014F606A14E3D4264@SHSMSX104.ccr.corp.intel.com> References: <20190131073724.12420-1-shenglei.zhang@intel.com> In-Reply-To: <20190131073724.12420-1-shenglei.zhang@intel.com> Accept-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ctpclassification: CTP_NT x-titus-metadata-40: eyJDYXRlZ29yeUxhYmVscyI6IiIsIk1ldGFkYXRhIjp7Im5zIjoiaHR0cDpcL1wvd3d3LnRpdHVzLmNvbVwvbnNcL0ludGVsMyIsImlkIjoiZTJjNWEwZWYtYmFjYS00ZjlkLTkwNTAtYjNjYmU1MDQ2ZTBjIiwicHJvcHMiOlt7Im4iOiJDVFBDbGFzc2lmaWNhdGlvbiIsInZhbHMiOlt7InZhbHVlIjoiQ1RQX05UIn1dfV19LCJTdWJqZWN0TGFiZWxzIjpbXSwiVE1DVmVyc2lvbiI6IjE3LjEwLjE4MDQuNDkiLCJUcnVzdGVkTGFiZWxIYXNoIjoibEJBUERRWlR6RHlkV01LV28wQ1ZFQVFlVlwvc0poQjJYTXlhQjNhQzZJeGp0c01QXC9xNTk1Y1BoS1p0R1VwUDdlIn0= dlp-product: dlpe-windows dlp-version: 11.0.400.15 dlp-reaction: no-action x-originating-ip: [10.239.127.40] MIME-Version: 1.0 Subject: Re: [PATCH v5] MdePkg/BaseLib: Add Base64Encode() and Base64Decode() X-BeenThere: edk2-devel@lists.01.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: EDK II Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 31 Jan 2019 08:07:46 -0000 Content-Language: en-US Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Reviewed-by: Liming Gao > -----Original Message----- > From: Zhang, Shenglei > Sent: Thursday, January 31, 2019 3:37 PM > To: edk2-devel@lists.01.org > Cc: Mike Turner ; Kinney, Michael D ; Gao, Liming > Subject: [PATCH v5] MdePkg/BaseLib: Add Base64Encode() and Base64Decode() >=20 > From: Mike Turner >=20 > Introduce public functions Base64Encode and Base64Decode. > https://bugzilla.tianocore.org/show_bug.cgi?id=3D1370 >=20 > v2:1.Remove some white space. > 2.Add unit test with test vectors in RFC 4648. > https://github.com/shenglei10/edk2/tree/encode_test > https://github.com/shenglei10/edk2/tree/decode_test >=20 > v3:1.Align white space. > 2.Update comments of Base64Encode and Base64Decode. > 3.Change the use of macro RETURN_DEVICE_ERROR to > RETURN_INVALID_PARAMETER in string.c. >=20 > v4:Change parameters' names. >=20 > v5:1.Update usage of variables. > 2.Remove debug message in Base64Decode(). >=20 > Cc: Michael D Kinney > Cc: Liming Gao > Contributed-under: TianoCore Contribution Agreement 1.1 > Signed-off-by: Shenglei Zhang > --- > MdePkg/Include/Library/BaseLib.h | 56 ++++++ > MdePkg/Library/BaseLib/String.c | 331 +++++++++++++++++++++++++++++++ > 2 files changed, 387 insertions(+) >=20 > diff --git a/MdePkg/Include/Library/BaseLib.h b/MdePkg/Include/Library/Ba= seLib.h > index 1eb842384e..03173def58 100644 > --- a/MdePkg/Include/Library/BaseLib.h > +++ b/MdePkg/Include/Library/BaseLib.h > @@ -2720,6 +2720,62 @@ AsciiStrnToUnicodeStrS ( > OUT UINTN *DestinationLength > ); >=20 > +/** > + Convert binary data to a Base64 encoded ascii string based on RFC4648. > + > + Produce a Null-terminated Ascii string in the output buffer specified = by Destination and DestinationSize. > + The Ascii string is produced by converting the data string specified b= y Source and SourceLength. > + > + @param Source Input UINT8 data > + @param SourceLength Number of UINT8 bytes of data > + @param Destination Pointer to output string buffer > + @param DestinationSize Size of ascii buffer. Set to 0 to get the size= needed. > + Caller is responsible for passing in buffer of= DestinationSize > + > + @retval RETURN_SUCCESS When ascii buffer is filled in. > + @retval RETURN_INVALID_PARAMETER If Source is NULL or DestinationSiz= e is NULL. > + @retval RETURN_INVALID_PARAMETER If SourceLength or DestinationSize = is bigger than (MAX_ADDRESS - (UINTN)Destination). > + @retval RETURN_BUFFER_TOO_SMALL If SourceLength is 0 and Destinatio= nSize is <1. > + @retval RETURN_BUFFER_TOO_SMALL If Destination is NULL or Destinati= onSize is smaller than required buffersize. > + > +**/ > +RETURN_STATUS > +EFIAPI > +Base64Encode ( > + IN CONST UINT8 *Source, > + IN UINTN SourceLength, > + OUT CHAR8 *Destination OPTIONAL, > + IN OUT UINTN *DestinationSize > + ); > + > +/** > + Convert Base64 ascii string to binary data based on RFC4648. > + > + Produce Null-terminated binary data in the output buffer specified by = Destination and DestinationSize. > + The binary data is produced by converting the Base64 ascii string spec= ified by Source and SourceLength. > + > + @param Source Input ASCII characters > + @param SourceLength Number of ASCII characters > + @param Destination Pointer to output buffer > + @param DestinationSize Caller is responsible for passing in buffer of = at least DestinationSize. > + Set 0 to get the size needed. Set to bytes stor= ed on return. > + > + @retval RETURN_SUCCESS When binary buffer is filled in. > + @retval RETURN_INVALID_PARAMETER If Source is NULL or DestinationSiz= e is NULL. > + @retval RETURN_INVALID_PARAMETER If SourceLength or DestinationSize = is bigger than (MAX_ADDRESS -(UINTN)Destination ). > + @retval RETURN_INVALID_PARAMETER If there is any invalid character i= n input stream. > + @retval RETURN_BUFFER_TOO_SMALL If buffer length is smaller than re= quired buffer size. > + > + **/ > +RETURN_STATUS > +EFIAPI > +Base64Decode ( > + IN CONST CHAR8 *Source, > + IN UINTN SourceLength, > + OUT UINT8 *Destination OPTIONAL, > + IN OUT UINTN *DestinationSize > + ); > + > /** > Converts an 8-bit value to an 8-bit BCD value. >=20 > diff --git a/MdePkg/Library/BaseLib/String.c b/MdePkg/Library/BaseLib/Str= ing.c > index e6df12797d..87d0a9d1dc 100644 > --- a/MdePkg/Library/BaseLib/String.c > +++ b/MdePkg/Library/BaseLib/String.c > @@ -1763,6 +1763,337 @@ AsciiStrToUnicodeStr ( >=20 > #endif >=20 > +// > +// The basis for Base64 encoding is RFC 4686 https://tools.ietf.org/html= /rfc4648 > +// > +// RFC 4686 has a number of MAY and SHOULD cases. This implementation c= hooses > +// the more restrictive versions for security concerns (see RFC 4686 sec= tion 3.3). > +// > +// A invalid character, if encountered during the decode operation, caus= es the data > +// to be rejected. In addition, the '=3D' padding character is only allo= wed at the end > +// of the Base64 encoded string. > +// > +#define BAD_V 99 > + > +STATIC CHAR8 Encoding_table[] =3D "ABCDEFGHIJKLMNOPQRSTUVWXYZ" > + "abcdefghijklmnopqrstuvwxyz" > + "0123456789+/"; > + > +STATIC UINT8 Decoding_table[] =3D { > + // > + // Valid characters ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwx= yz0123456789+/ > + // Also, set '=3D' as a zero for decoding > + // 0 , 1, 2, 3, 4, = 5, 6, 7, 8, > 9, a, b, c, d, e, = f > + BAD_V, BAD_V, BAD_V, BAD_V, BAD_V, BAD_V, BAD_V, BAD_V, BAD_V,= BAD_V, BAD_V, BAD_V, BAD_V, BAD_V, > BAD_V, BAD_V, // 0 > + BAD_V, BAD_V, BAD_V, BAD_V, BAD_V, BAD_V, BAD_V, BAD_V, BAD_V,= BAD_V, BAD_V, BAD_V, BAD_V, BAD_V, > BAD_V, BAD_V, // 10 > + BAD_V, BAD_V, BAD_V, BAD_V, BAD_V, BAD_V, BAD_V, BAD_V, BAD_V,= BAD_V, BAD_V, 62, BAD_V, BAD_V, > BAD_V, 63, // 20 > + 52, 53, 54, 55, 56, 57, 58, 59, 60,= 61, BAD_V, BAD_V, BAD_V, 0, BAD_V, > BAD_V, // 30 > + BAD_V, 0, 1, 2, 3, 4, 5, 6, 7,= 8, 9, 10, 11, 12, 13, > 14, // 40 > + 15, 16, 17, 18, 19, 20, 21, 22, 23,= 24, 25, BAD_V, BAD_V, BAD_V, BAD_V, > BAD_V, // 50 > + BAD_V, 26, 27, 28, 29, 30, 31, 32, 33,= 34, 35, 36, 37, 38, 39, > 40, // 60 > + 41, 42, 43, 44, 45, 46, 47, 48, 49,= 50, 51, BAD_V, BAD_V, BAD_V, BAD_V, > BAD_V, // 70 > + BAD_V, BAD_V, BAD_V, BAD_V, BAD_V, BAD_V, BAD_V, BAD_V, BAD_V,= BAD_V, BAD_V, BAD_V, BAD_V, BAD_V, > BAD_V, BAD_V, // 80 > + BAD_V, BAD_V, BAD_V, BAD_V, BAD_V, BAD_V, BAD_V, BAD_V, BAD_V,= BAD_V, BAD_V, BAD_V, BAD_V, BAD_V, > BAD_V, BAD_V, // 90 > + BAD_V, BAD_V, BAD_V, BAD_V, BAD_V, BAD_V, BAD_V, BAD_V, BAD_V,= BAD_V, BAD_V, BAD_V, BAD_V, BAD_V, > BAD_V, BAD_V, // a0 > + BAD_V, BAD_V, BAD_V, BAD_V, BAD_V, BAD_V, BAD_V, BAD_V, BAD_V,= BAD_V, BAD_V, BAD_V, BAD_V, BAD_V, > BAD_V, BAD_V, // b0 > + BAD_V, BAD_V, BAD_V, BAD_V, BAD_V, BAD_V, BAD_V, BAD_V, BAD_V,= BAD_V, BAD_V, BAD_V, BAD_V, BAD_V, > BAD_V, BAD_V, // c0 > + BAD_V, BAD_V, BAD_V, BAD_V, BAD_V, BAD_V, BAD_V, BAD_V, BAD_V,= BAD_V, BAD_V, BAD_V, BAD_V, BAD_V, > BAD_V, BAD_V, // d0 > + BAD_V, BAD_V, BAD_V, BAD_V, BAD_V, BAD_V, BAD_V, BAD_V, BAD_V,= BAD_V, BAD_V, BAD_V, BAD_V, BAD_V, > BAD_V, BAD_V, // d0 > + BAD_V, BAD_V, BAD_V, BAD_V, BAD_V, BAD_V, BAD_V, BAD_V, BAD_V,= BAD_V, BAD_V, BAD_V, BAD_V, BAD_V, > BAD_V, BAD_V // f0 > +}; > + > +/** > + Convert binary data to a Base64 encoded ascii string based on RFC4648. > + > + Produce a Null-terminated Ascii string in the output buffer specified = by Destination and DestinationSize. > + The Ascii string is produced by converting the data string specified b= y Source and SourceLength. > + > + @param Source Input UINT8 data > + @param SourceLength Number of UINT8 bytes of data > + @param Destination Pointer to output string buffer > + @param DestinationSize Size of ascii buffer. Set to 0 to get the siz= e needed. > + Caller is responsible for passing in buffer o= f DestinationSize > + > + @retval RETURN_SUCCESS When ascii buffer is filled in. > + @retval RETURN_INVALID_PARAMETER If Source is NULL or DestinationSiz= e is NULL. > + @retval RETURN_INVALID_PARAMETER If SourceLength or DestinationSize = is bigger than (MAX_ADDRESS - (UINTN)Destination). > + @retval RETURN_BUFFER_TOO_SMALL If SourceLength is 0 and Destinatio= nSize is <1. > + @retval RETURN_BUFFER_TOO_SMALL If Destination is NULL or Destinati= onSize is smaller than required buffersize. > + > +**/ > +RETURN_STATUS > +EFIAPI > +Base64Encode ( > + IN CONST UINT8 *Source, > + IN UINTN SourceLength, > + OUT CHAR8 *Destination OPTIONAL, > + IN OUT UINTN *DestinationSize > + ) > +{ > + > + UINTN RequiredSize; > + UINTN Left; > + > + // > + // Check pointers, and SourceLength is valid > + // > + if ((Source =3D=3D NULL) || (DestinationSize =3D=3D NULL)) { > + return RETURN_INVALID_PARAMETER; > + } > + > + // > + // Allow for RFC 4648 test vector 1 > + // > + if (SourceLength =3D=3D 0) { > + if (*DestinationSize < 1) { > + *DestinationSize =3D 1; > + return RETURN_BUFFER_TOO_SMALL; > + } > + *DestinationSize =3D 1; > + *Destination =3D '\0'; > + return RETURN_SUCCESS; > + } > + > + // > + // Check if SourceLength or DestinationSize is valid > + // > + if ((SourceLength >=3D (MAX_ADDRESS - (UINTN)Source)) || (*Destination= Size >=3D (MAX_ADDRESS - (UINTN)Destination))){ > + return RETURN_INVALID_PARAMETER; > + } > + > + // > + // 4 ascii per 3 bytes + NULL > + // > + RequiredSize =3D ((SourceLength + 2) / 3) * 4 + 1; > + if ((Destination =3D=3D NULL) || *DestinationSize < RequiredSize) { > + *DestinationSize =3D RequiredSize; > + return RETURN_BUFFER_TOO_SMALL; > + } > + > + Left =3D SourceLength; > + > + // > + // Encode 24 bits (three bytes) into 4 ascii characters > + // > + while (Left >=3D 3) { > + > + *Destination++ =3D Encoding_table[( Source[0] & 0xfc) >> 2 ]; > + *Destination++ =3D Encoding_table[((Source[0] & 0x03) << 4) + ((Sour= ce[1] & 0xf0) >> 4)]; > + *Destination++ =3D Encoding_table[((Source[1] & 0x0f) << 2) + ((Sour= ce[2] & 0xc0) >> 6)]; > + *Destination++ =3D Encoding_table[( Source[2] & 0x3f)]; > + Left -=3D 3; > + Source +=3D 3; > + } > + > + // > + // Handle the remainder, and add padding '=3D' characters as necessary= . > + // > + switch (Left) { > + case 0: > + > + // > + // No bytes Left, done. > + // > + break; > + case 1: > + > + // > + // One more data byte, two pad characters > + // > + *Destination++ =3D Encoding_table[( Source[0] & 0xfc) >> 2]; > + *Destination++ =3D Encoding_table[((Source[0] & 0x03) << 4)]; > + *Destination++ =3D '=3D'; > + *Destination++ =3D '=3D'; > + break; > + case 2: > + > + // > + // Two more data bytes, and one pad character > + // > + *Destination++ =3D Encoding_table[( Source[0] & 0xfc) >> 2]; > + *Destination++ =3D Encoding_table[((Source[0] & 0x03) << 4) + ((So= urce[1] & 0xf0) >> 4)]; > + *Destination++ =3D Encoding_table[((Source[1] & 0x0f) << 2)]; > + *Destination++ =3D '=3D'; > + break; > + } > + // > + // Add terminating NULL > + // > + *Destination =3D '\0'; > + return RETURN_SUCCESS; > +} > + > +/** > + Convert Base64 ascii string to binary data based on RFC4648. > + > + Produce Null-terminated binary data in the output buffer specified by = Destination and DestinationSize. > + The binary data is produced by converting the Base64 ascii string spec= ified by Source and SourceLength. > + > + @param Source Input ASCII characters > + @param SourceLength Number of ASCII characters > + @param Destination Pointer to output buffer > + @param DestinationSize Caller is responsible for passing in buffer o= f at least DestinationSize. > + Set 0 to get the size needed. Set to bytes st= ored on return. > + > + @retval RETURN_SUCCESS When binary buffer is filled in. > + @retval RETURN_INVALID_PARAMETER If Source is NULL or DestinationSiz= e is NULL. > + @retval RETURN_INVALID_PARAMETER If SourceLength or DestinationSize = is bigger than (MAX_ADDRESS -(UINTN)Destination ). > + @retval RETURN_INVALID_PARAMETER If there is any invalid character i= n input stream. > + @retval RETURN_BUFFER_TOO_SMALL If buffer length is smaller than re= quired buffer size. > + **/ > +RETURN_STATUS > +EFIAPI > +Base64Decode ( > + IN CONST CHAR8 *Source, > + IN UINTN SourceLength, > + OUT UINT8 *Destination OPTIONAL, > + IN OUT UINTN *DestinationSize > + ) > +{ > + > + UINT32 Value; > + CHAR8 Chr; > + INTN BufferSize; > + UINTN SourceIndex; > + UINTN DestinationIndex; > + UINTN Index; > + UINTN ActualSourceLength; > + > + // > + // Check pointers are not NULL > + // > + if ((Source =3D=3D NULL) || (DestinationSize =3D=3D NULL)) { > + return RETURN_INVALID_PARAMETER; > + } > + > + // > + // Check if SourceLength or DestinationSize is valid > + // > + if ((SourceLength >=3D (MAX_ADDRESS - (UINTN)Source)) || (*Destination= Size >=3D (MAX_ADDRESS - (UINTN)Destination))){ > + return RETURN_INVALID_PARAMETER; > + } > + > + ActualSourceLength =3D 0; > + BufferSize =3D 0; > + > + // > + // Determine the actual number of valid characters in the string. > + // All invalid characters except selected white space characters, > + // will cause the Base64 string to be rejected. White space to allow > + // properly formatted XML will be ignored. > + // > + // See section 3.3 of RFC 4648. > + // > + for (SourceIndex =3D 0; SourceIndex < SourceLength; SourceIndex++) { > + > + // > + // '=3D' is part of the quantum > + // > + if (Source[SourceIndex] =3D=3D '=3D') { > + ActualSourceLength++; > + BufferSize--; > + > + // > + // Only two '=3D' characters can be valid. > + // > + if (BufferSize < -2) { > + return RETURN_INVALID_PARAMETER; > + } > + } > + else { > + Chr =3D Source[SourceIndex]; > + if (BAD_V !=3D Decoding_table[(UINT8) Chr]) { > + > + // > + // The '=3D' characters are only valid at the end, so any > + // valid character after an '=3D', will be flagged as an error. > + // > + if (BufferSize < 0) { > + return RETURN_INVALID_PARAMETER; > + } > + ActualSourceLength++; > + } > + else { > + > + // > + // The reset of the decoder will ignore all invalid characters a= llowed here. > + // Ignoring selected white space is useful. In this case, the d= ecoder will > + // ignore ' ', '\t', '\n', and '\r'. > + // > + if ((Chr !=3D ' ') &&(Chr !=3D '\t') &&(Chr !=3D '\n') &&(Chr != =3D '\r')) { > + return RETURN_INVALID_PARAMETER; > + } > + } > + } > + } > + > + // > + // The Base64 character string must be a multiple of 4 character quant= ums. > + // > + if (ActualSourceLength % 4 !=3D 0) { > + return RETURN_INVALID_PARAMETER; > + } > + > + BufferSize +=3D ActualSourceLength / 4 * 3; > + if (BufferSize < 0) { > + return RETURN_INVALID_PARAMETER; > + } > + > + // > + // BufferSize is >=3D 0 > + // > + if ((Destination =3D=3D NULL) || (*DestinationSize < (UINTN) BufferSiz= e)) { > + *DestinationSize =3D BufferSize; > + return RETURN_BUFFER_TOO_SMALL; > + } > + > + // > + // If no decodable characters, return a size of zero. RFC 4686 test ve= ctor 1. > + // > + if (ActualSourceLength =3D=3D 0) { > + *DestinationSize =3D 0; > + return RETURN_SUCCESS; > + } > + > + // > + // Input data is verified to be a multiple of 4 valid charcters. Proc= ess four > + // characters at a time. Uncounted (ie. invalid) characters will be i= gnored. > + // > + for (SourceIndex =3D 0, DestinationIndex =3D 0; (SourceIndex < SourceL= ength) && (DestinationIndex < *DestinationSize); ) { > + Value =3D 0; > + > + // > + // Get 24 bits of data from 4 input characters, each character repre= senting 6 bits > + // > + for (Index =3D 0; Index < 4; Index++) { > + do { > + Chr =3D Decoding_table[(UINT8) Source[SourceIndex++]]; > + } while (Chr =3D=3D BAD_V); > + Value <<=3D 6; > + Value |=3D Chr; > + } > + > + // > + // Store 3 bytes of binary data (24 bits) > + // > + *Destination++ =3D (UINT8) (Value >> 16); > + DestinationIndex++; > + > + // > + // Due to the '=3D' special cases for the two bytes at the end, > + // we have to check the length and not store the padding data > + // > + if (DestinationIndex++ < *DestinationSize) { > + *Destination++ =3D (UINT8) (Value >> 8); > + } > + if (DestinationIndex++ < *DestinationSize) { > + *Destination++ =3D (UINT8) Value; > + } > + } > + > + return RETURN_SUCCESS; > +} > + > /** > Converts an 8-bit value to an 8-bit BCD value. >=20 > -- > 2.18.0.windows.1