From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from us-smtp-1.mimecast.com (us-smtp-1.mimecast.com [207.211.31.120]) by mx.groups.io with SMTP id smtpd.web12.1541.1600272307003099329 for ; Wed, 16 Sep 2020 09:05:07 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=@redhat.com header.s=mimecast20190719 header.b=fRp7x4k/; spf=pass (domain: redhat.com, ip: 207.211.31.120, mailfrom: lersek@redhat.com) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1600272306; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=iMCiSvvFiOAvK0L3ToubSnbpIpPKmhAGDVHvoKNwLRQ=; b=fRp7x4k/yOswZQXCefrDBCycEoX423sp/JtRRykXs+laKBEfRkq8ifM81NAMWTuDpQ9Dtn YudPL1QLp0ohK1Jt+u6zg1fskqQe+MJib8PrP0t6eddVqZkrVp0nsiDjRwSJWvmsxeuNZe tumivfzgkUl8MlTJwrv5IdEKi0I9igM= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-501-FemUFIUROEiLoTUxC5YIXA-1; Wed, 16 Sep 2020 12:04:56 -0400 X-MC-Unique: FemUFIUROEiLoTUxC5YIXA-1 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id A20A21021D31; Wed, 16 Sep 2020 16:04:50 +0000 (UTC) Received: from lacos-laptop-7.usersys.redhat.com (ovpn-113-131.ams2.redhat.com [10.36.113.131]) by smtp.corp.redhat.com (Postfix) with ESMTP id 4C25A19D6C; Wed, 16 Sep 2020 16:04:49 +0000 (UTC) Subject: Re: [edk2-devel] [PATCH v3 1/3] CryptoPkg/BaseCryptLib: Add EVP (Envelope) Digest interface To: "Yao, Jiewen" , "devel@edk2.groups.io" , "Zurcher, Christopher J" Cc: "Wang, Jian J" , "Lu, XiaoyuX" References: <20200916005902.6114-1-christopher.j.zurcher@intel.com> <20200916005902.6114-2-christopher.j.zurcher@intel.com> <6ffd9a53-0fde-cf4b-240a-6703a3dfd48e@redhat.com> From: "Laszlo Ersek" Message-ID: Date: Wed, 16 Sep 2020 18:04:48 +0200 MIME-Version: 1.0 In-Reply-To: X-Scanned-By: MIMEDefang 2.84 on 10.5.11.23 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=lersek@redhat.com X-Mimecast-Spam-Score: 0.002 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit Content-Language: en-US On 09/16/20 16:17, Yao, Jiewen wrote: > Thank you Laszlo. > You forced me to read the code again and more carefully. :-) > > I believe I understand why you use NULL to free the context now - to support error path. > > First, I did have some new thought for EVP API. Let’s consider 3 cases, 1) Hash all data one time, 2) Hash update data multiple times, 3) Error during update. > > A. In current design, the API sequence is: > 1) EvpMdHashAll (Digest) > 2) EvpMdInit, EvpMdUpdate, EvpMdUpdate, ... , EvpMdFinal (Digest) > 3) EvpMdInit, EvpMdUpdate->ERROR, EvpMdFinal (NULL) > > B. We can auto free context in EvpMdUpdate in error path - the API sequence is: > 1) EvpMdHashAll (Digest) > 2) EvpMdInit, EvpMdUpdate, EvpMdUpdate, ... , EvpMdFinal (Digest) > 3) EvpMdInit, EvpMdUpdate->ERROR I don't like "B" because in the loop body where you call EvpMdUpdate(), you may encounter an error from a different source than EvpMdUpdate() itself. For example, you may be reading data from the network or a disk file. If fetching the next chunk fails, we'd still want to clean up the EVP MD context. Therefore, if EvpMdUpdate() would itself invalidate the context, if it failed, then that would *complicate* the error path. (= Clean up the context after the loop body *only* if something different from EvpMdUpdate() failed.) > > C: We can use New/Free style - similar to HMAC > 1) EvpMdHashAll (Digest) > 2) EvpMdNew, EvpMdInit, EvpMdUpdate, EvpMdUpdate, ... , EvpMdFinal (Digest), EvpMdFree > 3) EvpMdNew, EvpMdInit, EvpMdUpdate->ERROR, EvpMdFree Yes, this was the first pattern, the one that caused me to say "please don't do this". In this pattern, the context may exist between "New" and "Init", and also between "Final" and "Free". Those two states are useless and only good for causing confusion. For example, are you allowed to Duplicate a context in those states? It's best to prevent even the *asking* of that question. > > I checked below APIs: > 1) openssl (EVP_MD_CTX_new, EVP_DigestInit_ex, EVP_DigestUpdate, EVP_DigestFinal_ex, EVP_MD_CTX_free) > 2) mbedtls (mbedtls_sha256_init, mbedtls_sha256_starts_ret, mbedtls_sha256_update_ret, mbedtls_sha256_finish_ret, mbedtls_sha256_free) > 3) BaseCryptoLib HMAC (HmacSha256New, HmacSha256SetKey, HmacSha256Update, HmacSha256Final, HmacSha256Free) > > All APIs include free function to free the context, I don’t think it is a bad idea to have EvpMdFree() API. > I would like to propose option - C. - I cannot comment on mbedtls (2). - I think the current BaseCryptLib HMAC APIs (3) are not great, and we should use this opportunity to improve upon them. - Regarding openssl (1), as I understand it, the situation is different from edk2. In OpenSSL, EVP_MD_CTX is *not* an opaque type (expressed in C language terms, it is not an "incomplete" type, but a "complete" type). Therefore, an expression such as sizeof(EVP_MD_CTX) works. The consequence is that OpenSSL code itself can use the following style: void func(void) { EVP_MD_CTX ctx; EVP_DigestInit_ex(&ctx, ...); ... EVP_DigestFinal_ex(&ctx, ...) } In other words, OpenSSL-native code is permitted to know the internals of EVP_MD_CTX, therefore the OpenSSL-internal code may allocate EVP_MD_CTX with means that are *different* from EVP_MD_CTX_new() and EVP_MD_CTX_free(). The caller of (Init, Final) may use (New, Free) for memory management, or may use something completely different (for example, local variables). Therefore, offering the (Init, Final) APIs separately from (New, Free) is meaningful. But the situation in edk2 -- or any other OpenSSL *consumer* -- is different. In edk2, EVP_MD_CTX is an opaque type -- in C language terms: an "incomplete" type. OpenSSL deliberately hides the internals of EVP_MD_CTX. See "CryptoPkg/Library/OpensslLib/openssl/include/openssl/ossl_typ.h": > typedef struct evp_md_ctx_st EVP_MD_CTX; and the "evp_md_ctx_st" structure type is only defined in "CryptoPkg/Library/OpensslLib/openssl/crypto/evp/evp_local.h", whose contents -- I think! -- OpenSSL client code cannot, or *should not*, refer to. This means that OpenSSL consumer code can *only* rely on EVP_MD_CTX_new() and EVP_MD_CTX_free(), for allocating and releasing the context, respectively. "sizeof(EVP_MD_CTX)" is not permitted, and defining a local or global variable of type EVP_MD_CTX is also not valid. This means that the only reason for separating (Init, Final) from (New, Free) falls away, in OpenSSL consumer code. In OpenSSL consumer code, there is no reason to keep *both* pairs of APIs. Please note that, this (very prudent) information hiding / encapsulation in OpenSSL used to be violated by edk2 in the past, intentionally. That's what the HmacXxxGetContextSize() APIs were about -- those APIs forcefully leaked the context sizes to client code, so that client code in edk2 could perform its own allocation. But in , we finally eliminated HmacXxxGetContextSize(). As a part of that, we also simplified the HMAC API -- note that Init was replaced by SetKey. And because we were reworking an existent API, I didn't propose very intrusive changes -- I didn't propose fusing Final and Free, for example. Now, the EVP MD APIs are just being introduced to edk2. So we have a chance at getting them right, and making them minimal. > Second, I believe EVP is just something in openssl. It does not mean that we *have to* design API to follow openssl. > After rethink the size impact, I do have concern to merge all Hash implementation together in one function. > It might means nothing if the crypto library is based upon openssl. > But if the cryptolib implementation is based upon other crypto, such as Intel IPP (https://github.com/slimbootloader/slimbootloader/tree/master/BootloaderCommonPkg/Library/IppCryptoLib) or mbedTls (https://github.com/jyao1/edk2/tree/DeviceSecurity/CryptoMbedTlsPkg), then we can NOT get size benefit by separating the hash algorithm. > > > I would like to propose separate EvpMdxxx. > EvpMdNew -> Sha256New, Sha384New > EvpMdInit -> Sha256Init, Sha384Init > EvpMdUpdate -> Sha256Update, Sha384Update > EvpMdFinal -> Sha256Final, Sha384Final > EvpMdFree -> Sha256Free, Sha384Free I have no comment on this. > We can do similar thing with Hmac, to deprecate Sha256GetContextSize() API, Yes, good idea. > and replace caller with Sha256New() / Sha384Free() Indeed. But then -- there's no reason left for keeping (New, Free) separate from (Init, Final). Thanks Laszlo