public inbox for devel@edk2.groups.io
 help / color / mirror / Atom feed
* [PATCH 00/24] CryptoPkg/openssl: update openssl submodule to v3.0
@ 2021-12-03 16:07 Gerd Hoffmann
  2021-12-03 16:07 ` [PATCH 01/24] CryptoPkg/openssl: update submodule to 3.0 Gerd Hoffmann
                   ` (24 more replies)
  0 siblings, 25 replies; 63+ messages in thread
From: Gerd Hoffmann @ 2021-12-03 16:07 UTC (permalink / raw)
  To: devel
  Cc: Jiewen Yao, Jian J Wang, Guomin Jiang, Pawel Polawski,
	Philippe Mathieu-Daudé, Xiaoyu Lu, Gerd Hoffmann

Very first take on updating openssl to 3.0.

Some hacks are in there still, only limited testing
(no CI runs), so cleary not complete yet.  Review
comments and other hints are welcome nevertheless.

take care,
  Gerd

Gerd Hoffmann (24):
  CryptoPkg/openssl: update submodule to 3.0
  CryptoPkg/openssl: process_files.pl: drop UefiAsm.conf
  CryptoPkg/openssl: process_files.pl: expand *.a
  CryptoPkg/openssl: process_files.pl: set api to 1.1.1
  CryptoPkg/openssl: process_files.pl: change config header handling
  CryptoPkg/openssl: process_files.pl: provider headers
  CryptoPkg/openssl: process_files.pl: skip unused files
  CryptoPkg/openssl: process_files.pl: clean up when done
  CryptoPkg/openssl: process_files.pl: filter out crypto/buildinf.h
  CryptoPkg/openssl: update generated files
  CryptoPkg/BaseCryptLib: no openssl deprecation warnings please
  CryptoPkg/BaseCryptLib; adapt CryptSm3.c to openssl 3.0 changes.
  CryptoPkg/BaseCryptLib: add more bio print dummies
  CryptoPkg/openssl: adapt rand_pool.c to openssl 3.0 changes
  CryptoPkg/openssl: add dummy file store
  CryptoPkg/openssl: move compiler_flags to buildinf.c
  CryptoPkg/CrtLibSupport: add fcntl.h
  CryptoPkg/CrtLibSupport: add strstr()
  CryptoPkg/CrtLibSupport: add INT_MIN
  CryptoPkg/CrtLibSupport: add UINT_MAX
  CryptoPkg/CrtLibSupport: add MODULESDIR
  CryptoPkg/openssl: process_files.pl: copy generated der/*.c source
    files.
  CryptoPkg/openssl: add generated files der source files
  [hack] turn off -Werror

 CryptoPkg/Library/OpensslLib/OpensslLib.inf   | 1305 +++++----
 .../Library/OpensslLib/OpensslLibCrypto.inf   | 1220 +++++---
 .../Library/OpensslLib/OpensslLibX64.inf      |    1 +
 .../Library/OpensslLib/OpensslLibX64Gcc.inf   |    1 +
 .../Library/BaseCryptLib/InternalCryptLib.h   |    2 +
 CryptoPkg/Library/Include/CrtLibSupport.h     |    4 +
 CryptoPkg/Library/Include/crypto/bn_conf.h    |   29 +
 CryptoPkg/Library/Include/crypto/dso_conf.h   |    8 +-
 CryptoPkg/Library/Include/fcntl.h             |    9 +
 CryptoPkg/Library/Include/openssl/asn1.h      | 1128 +++++++
 CryptoPkg/Library/Include/openssl/asn1t.h     |  946 ++++++
 CryptoPkg/Library/Include/openssl/bio.h       |  884 ++++++
 CryptoPkg/Library/Include/openssl/cmp.h       |  592 ++++
 CryptoPkg/Library/Include/openssl/cms.h       |  493 ++++
 CryptoPkg/Library/Include/openssl/conf.h      |  211 ++
 .../Library/Include/openssl/configuration.h   |  286 ++
 CryptoPkg/Library/Include/openssl/crmf.h      |  227 ++
 CryptoPkg/Library/Include/openssl/crypto.h    |  556 ++++
 CryptoPkg/Library/Include/openssl/ct.h        |  573 ++++
 CryptoPkg/Library/Include/openssl/err.h       |  492 ++++
 CryptoPkg/Library/Include/openssl/ess.h       |  128 +
 CryptoPkg/Library/Include/openssl/fipskey.h   |   36 +
 CryptoPkg/Library/Include/openssl/lhash.h     |  288 ++
 CryptoPkg/Library/Include/openssl/ocsp.h      |  483 +++
 .../Library/Include/openssl/opensslconf.h     |  348 ---
 CryptoPkg/Library/Include/openssl/opensslv.h  |  114 +
 CryptoPkg/Library/Include/openssl/pkcs12.h    |  350 +++
 CryptoPkg/Library/Include/openssl/pkcs7.h     |  427 +++
 CryptoPkg/Library/Include/openssl/safestack.h |  297 ++
 CryptoPkg/Library/Include/openssl/srp.h       |  285 ++
 CryptoPkg/Library/Include/openssl/ssl.h       | 2585 +++++++++++++++++
 CryptoPkg/Library/Include/openssl/ui.h        |  407 +++
 CryptoPkg/Library/Include/openssl/x509.h      | 1276 ++++++++
 CryptoPkg/Library/Include/openssl/x509_vfy.h  |  894 ++++++
 CryptoPkg/Library/Include/openssl/x509v3.h    | 1450 +++++++++
 CryptoPkg/Library/Include/prov/bio.h          |   32 +
 CryptoPkg/Library/Include/prov/blake2.h       |  120 +
 CryptoPkg/Library/Include/prov/ciphercommon.h |  361 +++
 .../Library/Include/prov/ciphercommon_aead.h  |   47 +
 .../Library/Include/prov/ciphercommon_ccm.h   |  100 +
 .../Library/Include/prov/ciphercommon_gcm.h   |  129 +
 CryptoPkg/Library/Include/prov/der_digests.h  |  160 +
 CryptoPkg/Library/Include/prov/der_dsa.h      |   94 +
 CryptoPkg/Library/Include/prov/der_ec.h       |  286 ++
 CryptoPkg/Library/Include/prov/der_ecx.h      |   50 +
 CryptoPkg/Library/Include/prov/der_rsa.h      |  187 ++
 CryptoPkg/Library/Include/prov/der_sm2.h      |   37 +
 CryptoPkg/Library/Include/prov/der_wrap.h     |   46 +
 CryptoPkg/Library/Include/prov/digestcommon.h |  123 +
 .../Library/Include/prov/implementations.h    |  516 ++++
 CryptoPkg/Library/Include/prov/kdfexchange.h  |   24 +
 CryptoPkg/Library/Include/prov/macsignature.h |   30 +
 CryptoPkg/Library/Include/prov/md5_sha1.h     |   36 +
 CryptoPkg/Library/Include/prov/names.h        |  327 +++
 CryptoPkg/Library/Include/prov/proverr.h      |   27 +
 CryptoPkg/Library/Include/prov/provider_ctx.h |   40 +
 .../Library/Include/prov/provider_util.h      |  138 +
 .../Library/Include/prov/providercommon.h     |   24 +
 .../Library/Include/prov/securitycheck.h      |   30 +
 CryptoPkg/Library/Include/prov/seeding.h      |   41 +
 CryptoPkg/Library/OpensslLib/buildinf.h       |    2 +-
 .../Library/BaseCryptLib/Hash/CryptSm3.c      |   14 +-
 .../Library/BaseCryptLib/SysCall/CrtWrapper.c |   10 +
 .../OpensslLib/{buildinf.h => buildinf.c}     |    3 +-
 .../Library/OpensslLib/der_digests_gen.c      |  160 +
 CryptoPkg/Library/OpensslLib/der_rsa_gen.c    |  174 ++
 CryptoPkg/Library/OpensslLib/der_wrap_gen.c   |   46 +
 CryptoPkg/Library/OpensslLib/ossl_store.c     |   11 +
 CryptoPkg/Library/OpensslLib/rand_pool.c      |   20 +-
 CryptoPkg/Library/OpensslLib/openssl          |    2 +-
 CryptoPkg/Library/OpensslLib/process_files.pl |   79 +-
 71 files changed, 20510 insertions(+), 1351 deletions(-)
 create mode 100644 CryptoPkg/Library/Include/crypto/bn_conf.h
 create mode 100644 CryptoPkg/Library/Include/fcntl.h
 create mode 100644 CryptoPkg/Library/Include/openssl/asn1.h
 create mode 100644 CryptoPkg/Library/Include/openssl/asn1t.h
 create mode 100644 CryptoPkg/Library/Include/openssl/bio.h
 create mode 100644 CryptoPkg/Library/Include/openssl/cmp.h
 create mode 100644 CryptoPkg/Library/Include/openssl/cms.h
 create mode 100644 CryptoPkg/Library/Include/openssl/conf.h
 create mode 100644 CryptoPkg/Library/Include/openssl/configuration.h
 create mode 100644 CryptoPkg/Library/Include/openssl/crmf.h
 create mode 100644 CryptoPkg/Library/Include/openssl/crypto.h
 create mode 100644 CryptoPkg/Library/Include/openssl/ct.h
 create mode 100644 CryptoPkg/Library/Include/openssl/err.h
 create mode 100644 CryptoPkg/Library/Include/openssl/ess.h
 create mode 100644 CryptoPkg/Library/Include/openssl/fipskey.h
 create mode 100644 CryptoPkg/Library/Include/openssl/lhash.h
 create mode 100644 CryptoPkg/Library/Include/openssl/ocsp.h
 delete mode 100644 CryptoPkg/Library/Include/openssl/opensslconf.h
 create mode 100644 CryptoPkg/Library/Include/openssl/opensslv.h
 create mode 100644 CryptoPkg/Library/Include/openssl/pkcs12.h
 create mode 100644 CryptoPkg/Library/Include/openssl/pkcs7.h
 create mode 100644 CryptoPkg/Library/Include/openssl/safestack.h
 create mode 100644 CryptoPkg/Library/Include/openssl/srp.h
 create mode 100644 CryptoPkg/Library/Include/openssl/ssl.h
 create mode 100644 CryptoPkg/Library/Include/openssl/ui.h
 create mode 100644 CryptoPkg/Library/Include/openssl/x509.h
 create mode 100644 CryptoPkg/Library/Include/openssl/x509_vfy.h
 create mode 100644 CryptoPkg/Library/Include/openssl/x509v3.h
 create mode 100644 CryptoPkg/Library/Include/prov/bio.h
 create mode 100644 CryptoPkg/Library/Include/prov/blake2.h
 create mode 100644 CryptoPkg/Library/Include/prov/ciphercommon.h
 create mode 100644 CryptoPkg/Library/Include/prov/ciphercommon_aead.h
 create mode 100644 CryptoPkg/Library/Include/prov/ciphercommon_ccm.h
 create mode 100644 CryptoPkg/Library/Include/prov/ciphercommon_gcm.h
 create mode 100644 CryptoPkg/Library/Include/prov/der_digests.h
 create mode 100644 CryptoPkg/Library/Include/prov/der_dsa.h
 create mode 100644 CryptoPkg/Library/Include/prov/der_ec.h
 create mode 100644 CryptoPkg/Library/Include/prov/der_ecx.h
 create mode 100644 CryptoPkg/Library/Include/prov/der_rsa.h
 create mode 100644 CryptoPkg/Library/Include/prov/der_sm2.h
 create mode 100644 CryptoPkg/Library/Include/prov/der_wrap.h
 create mode 100644 CryptoPkg/Library/Include/prov/digestcommon.h
 create mode 100644 CryptoPkg/Library/Include/prov/implementations.h
 create mode 100644 CryptoPkg/Library/Include/prov/kdfexchange.h
 create mode 100644 CryptoPkg/Library/Include/prov/macsignature.h
 create mode 100644 CryptoPkg/Library/Include/prov/md5_sha1.h
 create mode 100644 CryptoPkg/Library/Include/prov/names.h
 create mode 100644 CryptoPkg/Library/Include/prov/proverr.h
 create mode 100644 CryptoPkg/Library/Include/prov/provider_ctx.h
 create mode 100644 CryptoPkg/Library/Include/prov/provider_util.h
 create mode 100644 CryptoPkg/Library/Include/prov/providercommon.h
 create mode 100644 CryptoPkg/Library/Include/prov/securitycheck.h
 create mode 100644 CryptoPkg/Library/Include/prov/seeding.h
 copy CryptoPkg/Library/OpensslLib/{buildinf.h => buildinf.c} (50%)
 create mode 100644 CryptoPkg/Library/OpensslLib/der_digests_gen.c
 create mode 100644 CryptoPkg/Library/OpensslLib/der_rsa_gen.c
 create mode 100644 CryptoPkg/Library/OpensslLib/der_wrap_gen.c

-- 
2.33.1


^ permalink raw reply	[flat|nested] 63+ messages in thread
* Re: [edk2-devel] [PATCH 00/24] CryptoPkg/openssl: update openssl submodule to v3.0
@ 2022-01-28 14:07 Gerd Hoffmann
  2022-01-28 14:14 ` Gerd Hoffmann
                   ` (2 more replies)
  0 siblings, 3 replies; 63+ messages in thread
From: Gerd Hoffmann @ 2022-01-28 14:07 UTC (permalink / raw)
  To: devel

  Hi,

Oops, dropped the list by mistake, forwarding ...

----- Forwarded message from "kraxel@redhat.com" <kraxel@redhat.com> -----

Date: Fri, 28 Jan 2022 10:35:10 +0100
Subject: Re: [edk2-devel] [PATCH 00/24] CryptoPkg/openssl: update openssl
 submodule to v3.0
From: "kraxel@redhat.com" <kraxel@redhat.com>
To: Kilian Kegel <kilian_kegel@outlook.com>
Message-ID: <20220128093510.atupc4ly6bvwinlk@sirius.home.kraxel.org>
Content-Type: text/plain; charset=utf-8

  Hi,

> On my 32Bit Ubuntu standard installation I ran
> 
>   1.  cc - Xlinker -Map=static.map hello.c -static
>   2.  cc  -Xlinker -Map=shared.map hello.c
> 
> The first .OBJ file mentioned in the .MAP file is in both cases:
> /usr/lib/gcc/i686-linux-gnu/6/libgcc.a(_udivdi3.o)

Yes, you are correct.  gcc provides both shared and static intrinsics.
There is a command line switch to pick which one you want
(-static-libgcc, -shared-libgcc).

> It seems to me that GNU holds the intrinsic functions in a separate library
> that can be used without any change, and is always correct by definition.

>   1.  add libgcc.a as a search library, adjust the conf\tools_def.txt like:
> 
> DEBUG_GCCxx_IA32_DLINK_FLAGS   = …predefined parameter … /usr/lib/gcc/i686-linux-gnu/6/libgcc.a

gcc documentation suggests to use just '-lgcc' (should pick the correct
library no matter what the compiler version and architecture is), so I
tried this:

-DEFINE GCC_DLINK2_FLAGS_COMMON     = -Wl,--script=$(EDK_TOOLS_PATH)/Scripts/GccBase.lds
+DEFINE GCC_DLINK2_FLAGS_COMMON     = -Wl,--script=$(EDK_TOOLS_PATH)/Scripts/GccBase.lds -lgcc

Build doesn't come very far.  Looks like the gcc intrinsics are not
free-standing but want call into libc:

/usr/bin/ld: /usr/lib/gcc/x86_64-redhat-linux/11/libgcc.a(_absvsi2.o): in function `__absvdi2':
(.text+0x18): undefined reference to `abort'
/usr/bin/ld: /usr/lib/gcc/x86_64-redhat-linux/11/libgcc.a(_absvsi2.o): in function `__absvsi2':
(.text+0x32): undefined reference to `abort'
/usr/bin/ld: /usr/lib/gcc/x86_64-redhat-linux/11/libgcc.a(_absvdi2.o): in function `__absvti2.cold':
(.text.unlikely+0x2): undefined reference to `abort'
/usr/bin/ld: /usr/lib/gcc/x86_64-redhat-linux/11/libgcc.a(_addvsi3.o): in function `__addvdi3':
(.text+0xf): undefined reference to `abort'
/usr/bin/ld: /usr/lib/gcc/x86_64-redhat-linux/11/libgcc.a(_addvsi3.o): in function `__addvsi3':
(.text+0x2d): undefined reference to `abort'
/usr/bin/ld: /usr/lib/gcc/x86_64-redhat-linux/11/libgcc.a(_addvdi3.o):(.text.unlikely+0x2): more undefined references to `abort' follow
/usr/bin/ld: /usr/lib/gcc/x86_64-redhat-linux/11/libgcc.a(_eprintf.o): in function `__eprintf':
(.text+0x8): undefined reference to `stderr'
/usr/bin/ld: (.text+0x1d): undefined reference to `fprintf'
/usr/bin/ld: (.text+0x25): undefined reference to `fflush'
/usr/bin/ld: (.text+0x2a): undefined reference to `abort'
/usr/bin/ld: /usr/lib/gcc/x86_64-redhat-linux/11/libgcc.a(generic-morestack.o): in function `__morestack_fail':
(.text+0xbc): undefined reference to `writev'
[ ... more errors snipped ... ]

The generic-morestack.o issues should be solvable, that shouldn't be
something which tianocore actually needs.  Not sure why the linker tries
to resolve symbols for object files which should not be needed in the
first place.  Possibly something else is fishy here, any hints are
welcome.  Something in the linker script maybe?

But the math intrinsics apparently having error paths which print a
message and abort doesn't look very promising to me.

Also: When trying arm cross-builds I run into the ABI problem already
mentioned elsewhere in this thread:

/usr/bin/arm-linux-gnu-ld: error: /usr/lib/gcc/arm-linux-gnueabi/11/libgcc.a(_muldi3.o) uses VFP register arguments, /home/kraxel/projects/edk2/Build/ArmVirtQemu-ARM/DEBUG_GCC5/ARM/OvmfPkg/VirtioBlkDxe/VirtioBlk/DEBUG/VirtioBlkDxe.dll does not

Patches are here:
  https://github.com/kraxel/edk2/commits/intrinsics-playground

> >* I have my doubts that compiler's builtin libraries are optimized for
> >   size, so I'd suspect we would see a noticeable size grow from that.
> Please check the size of __udivdi3() and whether the tianocore reimplementation is smaller or not

I'll rather check the size of the final binaries, but I can only do that
once the build works ...

> The intrinsic library belongs to the compiler not to the build system.

I'm open to explore that path, but apparently we have a number of road
blocks along the way.  Seems neither gcc nor xcode (see other reply)
provide a usable free-standing intrinsic library ...

take care,
  Gerd


^ permalink raw reply	[flat|nested] 63+ messages in thread

end of thread, other threads:[~2022-02-02 12:07 UTC | newest]

Thread overview: 63+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2021-12-03 16:07 [PATCH 00/24] CryptoPkg/openssl: update openssl submodule to v3.0 Gerd Hoffmann
2021-12-03 16:07 ` [PATCH 01/24] CryptoPkg/openssl: update submodule to 3.0 Gerd Hoffmann
2021-12-03 16:07 ` [PATCH 02/24] CryptoPkg/openssl: process_files.pl: drop UefiAsm.conf Gerd Hoffmann
2021-12-03 16:07 ` [PATCH 03/24] CryptoPkg/openssl: process_files.pl: expand *.a Gerd Hoffmann
2021-12-03 16:07 ` [PATCH 04/24] CryptoPkg/openssl: process_files.pl: set api to 1.1.1 Gerd Hoffmann
2021-12-03 16:07 ` [PATCH 05/24] CryptoPkg/openssl: process_files.pl: change config header handling Gerd Hoffmann
2021-12-03 16:07 ` [PATCH 06/24] CryptoPkg/openssl: process_files.pl: provider headers Gerd Hoffmann
2021-12-03 16:07 ` [PATCH 07/24] CryptoPkg/openssl: process_files.pl: skip unused files Gerd Hoffmann
2021-12-03 16:07 ` [PATCH 08/24] CryptoPkg/openssl: process_files.pl: clean up when done Gerd Hoffmann
2021-12-03 16:07 ` [PATCH 09/24] CryptoPkg/openssl: process_files.pl: filter out crypto/buildinf.h Gerd Hoffmann
2021-12-03 16:07 ` [PATCH 10/24] CryptoPkg/openssl: update generated files Gerd Hoffmann
2021-12-03 16:07 ` [PATCH 11/24] CryptoPkg/BaseCryptLib: no openssl deprecation warnings please Gerd Hoffmann
2021-12-03 16:07 ` [PATCH 12/24] CryptoPkg/BaseCryptLib; adapt CryptSm3.c to openssl 3.0 changes Gerd Hoffmann
2021-12-03 16:07 ` [PATCH 13/24] CryptoPkg/BaseCryptLib: add more bio print dummies Gerd Hoffmann
2021-12-03 16:07 ` [PATCH 14/24] CryptoPkg/openssl: adapt rand_pool.c to openssl 3.0 changes Gerd Hoffmann
2021-12-03 16:07 ` [PATCH 15/24] CryptoPkg/openssl: add dummy file store Gerd Hoffmann
2021-12-03 16:07 ` [PATCH 16/24] CryptoPkg/openssl: move compiler_flags to buildinf.c Gerd Hoffmann
2021-12-03 16:07 ` [PATCH 17/24] CryptoPkg/CrtLibSupport: add fcntl.h Gerd Hoffmann
2021-12-03 16:07 ` [PATCH 18/24] CryptoPkg/CrtLibSupport: add strstr() Gerd Hoffmann
2021-12-03 16:07 ` [PATCH 19/24] CryptoPkg/CrtLibSupport: add INT_MIN Gerd Hoffmann
2021-12-03 16:07 ` [PATCH 20/24] CryptoPkg/CrtLibSupport: add UINT_MAX Gerd Hoffmann
2021-12-03 16:07 ` [PATCH 21/24] CryptoPkg/CrtLibSupport: add MODULESDIR Gerd Hoffmann
2021-12-03 16:07 ` [PATCH 22/24] CryptoPkg/openssl: process_files.pl: copy generated der/*.c source files Gerd Hoffmann
2021-12-03 16:07 ` [PATCH 23/24] CryptoPkg/openssl: add generated files der " Gerd Hoffmann
2021-12-03 16:07 ` [PATCH 24/24] [hack] turn off -Werror Gerd Hoffmann
2021-12-03 16:27   ` [edk2-devel] " Michael D Kinney
2021-12-03 17:57     ` Pedro Falcato
2021-12-03 18:38       ` Michael D Kinney
2021-12-06  7:38         ` Gerd Hoffmann
2021-12-06  7:23     ` Gerd Hoffmann
2021-12-08  8:06     ` Gerd Hoffmann
2021-12-03 16:32 ` [edk2-devel] [PATCH 00/24] CryptoPkg/openssl: update openssl submodule to v3.0 Michael D Kinney
2021-12-03 16:42   ` Yao, Jiewen
2022-01-17 11:46     ` Gerd Hoffmann
2022-01-18 11:12       ` Yao, Jiewen
2022-01-18 16:12         ` Michael D Kinney
2022-01-21  8:33           ` Gerd Hoffmann
2022-01-21 16:34             ` Michael D Kinney
2022-01-21  8:30         ` Gerd Hoffmann
2022-01-21 16:38           ` Michael D Kinney
2022-01-24 16:24             ` Kilian Kegel
2022-01-24 17:28               ` Michael D Kinney
2022-01-24 19:58                 ` Pedro Falcato
2022-01-26 11:02                   ` Gerd Hoffmann
2022-01-27 22:26                     ` Kilian Kegel
2022-01-28  0:55                       ` Andrew Fish
2022-01-28  9:06                         ` Pedro Falcato
2022-01-28 10:14                           ` Gerd Hoffmann
2022-01-28 11:23                             ` Pedro Falcato
2022-01-28  9:51                         ` Gerd Hoffmann
2022-01-30 20:17                         ` Kilian Kegel
2022-02-01  9:55                           ` Gerd Hoffmann
2022-02-02 12:07                             ` Kilian Kegel
2022-01-25 20:05                 ` Kilian Kegel
2022-01-23  8:41           ` Yao, Jiewen
2021-12-06  8:05   ` Gerd Hoffmann
  -- strict thread matches above, loose matches on Subject: below --
2022-01-28 14:07 Gerd Hoffmann
2022-01-28 14:14 ` Gerd Hoffmann
2022-01-28 15:54 ` Pedro Falcato
2022-02-01  9:39   ` Gerd Hoffmann
2022-01-28 16:00 ` Pedro Falcato
2022-01-28 16:12   ` Kilian Kegel
2022-02-01  9:50   ` Gerd Hoffmann

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox