Re: [edk2-devel] [PATCH v3 1/3] CryptoPkg/BaseCryptLib: Add EVP (Envelope) Digest interface

["Laszlo Ersek" <lersek@redhat.com>]

On 09/16/20 16:17, Yao, Jiewen wrote:
> Thank you Laszlo.
> You forced me to read the code again and more carefully. :-)
> 
> I believe I understand why you use NULL to free the context now - to support error path.
> 
> First, I did have some new thought for EVP API. Let’s consider 3 cases, 1) Hash all data one time, 2) Hash update data multiple times, 3) Error during update.
> 
> A. In current design, the API sequence is:
> 1) EvpMdHashAll (Digest)
> 2) EvpMdInit, EvpMdUpdate, EvpMdUpdate, ... , EvpMdFinal (Digest)
> 3) EvpMdInit, EvpMdUpdate->ERROR, EvpMdFinal (NULL)
> 
> B. We can auto free context in EvpMdUpdate in error path  - the API sequence is:
> 1) EvpMdHashAll (Digest)
> 2) EvpMdInit, EvpMdUpdate, EvpMdUpdate, ... , EvpMdFinal (Digest)
> 3) EvpMdInit, EvpMdUpdate->ERROR

I don't like "B" because in the loop body where you call EvpMdUpdate(),
you may encounter an error from a different source than EvpMdUpdate()
itself. For example, you may be reading data from the network or a disk
file. If fetching the next chunk fails, we'd still want to clean up the
EVP MD context. Therefore, if EvpMdUpdate() would itself invalidate the
context, if it failed, then that would *complicate* the error path. (=
Clean up the context after the loop body *only* if something different
from EvpMdUpdate() failed.)

> 
> C: We can use New/Free style - similar to HMAC
> 1) EvpMdHashAll (Digest)
> 2) EvpMdNew, EvpMdInit, EvpMdUpdate, EvpMdUpdate, ... , EvpMdFinal (Digest), EvpMdFree
> 3) EvpMdNew, EvpMdInit, EvpMdUpdate->ERROR, EvpMdFree

Yes, this was the first pattern, the one that caused me to say "please
don't do this". In this pattern, the context may exist between "New" and
"Init", and also between "Final" and "Free". Those two states are
useless and only good for causing confusion.

For example, are you allowed to Duplicate a context in those states?
It's best to prevent even the *asking* of that question.

> 
> I checked below APIs:
> 1) openssl (EVP_MD_CTX_new, EVP_DigestInit_ex, EVP_DigestUpdate, EVP_DigestFinal_ex, EVP_MD_CTX_free)
> 2) mbedtls (mbedtls_sha256_init, mbedtls_sha256_starts_ret, mbedtls_sha256_update_ret, mbedtls_sha256_finish_ret, mbedtls_sha256_free)
> 3) BaseCryptoLib HMAC (HmacSha256New, HmacSha256SetKey, HmacSha256Update, HmacSha256Final, HmacSha256Free)
> 
> All APIs include free function to free the context, I don’t think it is a bad idea to have EvpMdFree() API.
> I would like to propose option - C.

- I cannot comment on mbedtls (2).

- I think the current BaseCryptLib HMAC APIs (3) are not great, and we
should use this opportunity to improve upon them.

- Regarding openssl (1), as I understand it, the situation is different
from edk2. In OpenSSL, EVP_MD_CTX is *not* an opaque type (expressed in
C language terms, it is not an "incomplete" type, but a "complete"
type). Therefore, an expression such as sizeof(EVP_MD_CTX) works.

The consequence is that OpenSSL code itself can use the following style:

void func(void)
{
  EVP_MD_CTX ctx;

  EVP_DigestInit_ex(&ctx, ...);
  ...
  EVP_DigestFinal_ex(&ctx, ...)
}

In other words, OpenSSL-native code is permitted to know the internals
of EVP_MD_CTX, therefore the OpenSSL-internal code may allocate
EVP_MD_CTX with means that are *different* from EVP_MD_CTX_new() and
EVP_MD_CTX_free().

The caller of (Init, Final) may use (New, Free) for memory management,
or may use something completely different (for example, local variables).

Therefore, offering the (Init, Final) APIs separately from (New, Free)
is meaningful.

But the situation in edk2 -- or any other OpenSSL *consumer* -- is
different. In edk2, EVP_MD_CTX is an opaque type -- in C language terms:
an "incomplete" type. OpenSSL deliberately hides the internals of
EVP_MD_CTX.

See "CryptoPkg/Library/OpensslLib/openssl/include/openssl/ossl_typ.h":

> typedef struct evp_md_ctx_st EVP_MD_CTX;

and the "evp_md_ctx_st" structure type is only defined in
"CryptoPkg/Library/OpensslLib/openssl/crypto/evp/evp_local.h", whose
contents -- I think! -- OpenSSL client code cannot, or *should not*,
refer to.

This means that OpenSSL consumer code can *only* rely on
EVP_MD_CTX_new() and EVP_MD_CTX_free(), for allocating and releasing the
context, respectively. "sizeof(EVP_MD_CTX)" is not permitted, and
defining a local or global variable of type EVP_MD_CTX is also not valid.

This means that the only reason for separating (Init, Final) from (New,
Free) falls away, in OpenSSL consumer code. In OpenSSL consumer code,
there is no reason to keep *both* pairs of APIs.


Please note that, this (very prudent) information hiding / encapsulation
in OpenSSL used to be violated by edk2 in the past, intentionally.
That's what the HmacXxxGetContextSize() APIs were about -- those APIs
forcefully leaked the context sizes to client code, so that client code
in edk2 could perform its own allocation.

But in <https://bugzilla.tianocore.org/show_bug.cgi?id=1792>, we finally
eliminated HmacXxxGetContextSize(). As a part of that, we also
simplified the HMAC API -- note that Init was replaced by SetKey. And
because we were reworking an existent API, I didn't propose very
intrusive changes -- I didn't propose fusing Final and Free, for example.

Now, the EVP MD APIs are just being introduced to edk2. So we have a
chance at getting them right, and making them minimal.

> Second, I believe EVP is just something in openssl. It does not mean that we *have to* design API to follow openssl.
> After rethink the size impact, I do have concern to merge all Hash implementation together in one function.
> It might means nothing if the crypto library is based upon openssl.
> But if the cryptolib implementation is based upon other crypto, such as Intel IPP (https://github.com/slimbootloader/slimbootloader/tree/master/BootloaderCommonPkg/Library/IppCryptoLib) or mbedTls (https://github.com/jyao1/edk2/tree/DeviceSecurity/CryptoMbedTlsPkg), then we can NOT get size benefit by separating the hash algorithm.
> 
> 
> I would like to propose separate EvpMdxxx.
> EvpMdNew -> Sha256New, Sha384New
> EvpMdInit -> Sha256Init, Sha384Init
> EvpMdUpdate -> Sha256Update, Sha384Update
> EvpMdFinal -> Sha256Final, Sha384Final
> EvpMdFree -> Sha256Free, Sha384Free

I have no comment on this.

> We can do similar thing with Hmac, to deprecate Sha256GetContextSize() API,

Yes, good idea.

> and replace caller with Sha256New() / Sha384Free()

Indeed.

But then -- there's no reason left for keeping (New, Free) separate from
(Init, Final).

Thanks
Laszlo