Re: [Patch] CryptoPkg/TlsLib: Change the return type of TlsInitialize().

["Long, Qin" <qin.long@intel.com>]

Reviewed-by: Long Qin <qin.long@intel.com>


Best Regards & Thanks,
LONG, Qin

-----Original Message-----
From: Wu, Jiaxin 
Sent: Friday, November 17, 2017 11:57 AM
To: edk2-devel@lists.01.org
Cc: Ye, Ting <ting.ye@intel.com>; Long, Qin <qin.long@intel.com>; Fu, Siyuan <siyuan.fu@intel.com>; Wu, Jiaxin <jiaxin.wu@intel.com>
Subject: [Patch] CryptoPkg/TlsLib: Change the return type of TlsInitialize().

Currently, in TlsInitialize(), neither the return status of OPENSSL_init_ssl(0, or 1) nor the return code of RandomSeed (TRUE or FALSE) is not checked. Also VOID is used as the return type of TlsInitialize(), which can't be used to capture the returned value for the error handling.

>From Long Qin (CryptoPkg owner):
The early version of OPENSSL_init_ssl() use the "VOID" as the return value, which was updated to "int" later because the function changes can fail.

So, this patch is to change the return type of TlsInitialize() to follow up the OPENSSL_init_ssl() update.

Cc: Ye Ting <ting.ye@intel.com>
Cc: Long Qin <qin.long@intel.com>
Cc: Fu Siyuan <siyuan.fu@intel.com>
Contributed-under: TianoCore Contribution Agreement 1.0
Signed-off-by: Wu Jiaxin <jiaxin.wu@intel.com>
---
 CryptoPkg/Include/Library/TlsLib.h |  7 +++++--  CryptoPkg/Library/TlsLib/TlsInit.c | 20 ++++++++++++++------
 2 files changed, 19 insertions(+), 8 deletions(-)

diff --git a/CryptoPkg/Include/Library/TlsLib.h b/CryptoPkg/Include/Library/TlsLib.h
index fa6cb99..b69d513 100644
--- a/CryptoPkg/Include/Library/TlsLib.h
+++ b/CryptoPkg/Include/Library/TlsLib.h
@@ -1,9 +1,9 @@
 /** @file
   Defines TLS Library APIs.
 
-Copyright (c) 2016, Intel Corporation. All rights reserved.<BR>
+Copyright (c) 2016 - 2017, Intel Corporation. All rights reserved.<BR>
 This program and the accompanying materials  are licensed and made available under the terms and conditions of the BSD License  which accompanies this distribution.  The full text of the license may be found at  http://opensource.org/licenses/bsd-license.php
 
@@ -20,12 +20,15 @@ WITHOUT WARRANTIES OR REPRESENTATIONS OF ANY KIND, EITHER EXPRESS OR IMPLIED.
 
   This function registers ciphers and digests used directly and indirectly
   by SSL/TLS, and initializes the readable error messages.
   This function must be called before any other action takes places.
 
+  @retval TRUE   The OpenSSL library has been initialized.
+  @retval FALSE  Failed to initialize the OpenSSL library.
+
 **/
-VOID
+BOOLEAN
 EFIAPI
 TlsInitialize (
   VOID
   );
 
diff --git a/CryptoPkg/Library/TlsLib/TlsInit.c b/CryptoPkg/Library/TlsLib/TlsInit.c
index e524647..a530ff7 100644
--- a/CryptoPkg/Library/TlsLib/TlsInit.c
+++ b/CryptoPkg/Library/TlsLib/TlsInit.c
@@ -20,30 +20,38 @@ WITHOUT WARRANTIES OR REPRESENTATIONS OF ANY KIND, EITHER EXPRESS OR IMPLIED.
 
   This function registers ciphers and digests used directly and indirectly
   by SSL/TLS, and initializes the readable error messages.
   This function must be called before any other action takes places.
 
+  @retval TRUE   The OpenSSL library has been initialized.
+  @retval FALSE  Failed to initialize the OpenSSL library.
+
 **/
-VOID
+BOOLEAN
 EFIAPI
 TlsInitialize (
   VOID
   )
 {
+  INTN            Ret;
+
   //
   // Performs initialization of crypto and ssl library, and loads required
   // algorithms.
   //
-  OPENSSL_init_ssl (
-    OPENSSL_INIT_LOAD_SSL_STRINGS | OPENSSL_INIT_LOAD_CRYPTO_STRINGS,
-    NULL
-    );
+  Ret = OPENSSL_init_ssl (
+          OPENSSL_INIT_LOAD_SSL_STRINGS | OPENSSL_INIT_LOAD_CRYPTO_STRINGS,
+          NULL
+          );
+  if (Ret != 1) {
+    return FALSE;
+  }
 
   //
   // Initialize the pseudorandom number generator.
   //
-  RandomSeed (NULL, 0);
+  return RandomSeed (NULL, 0);
 }
 
 /**
   Free an allocated SSL_CTX object.
 
--
1.9.5.msysgit.1