From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [63.128.21.124]) by mx.groups.io with SMTP id smtpd.web08.17214.1611087164438460674 for ; Tue, 19 Jan 2021 12:12:44 -0800 Authentication-Results: mx.groups.io; dkim=pass header.i=@redhat.com header.s=mimecast20190719 header.b=cLP0Mo3a; spf=pass (domain: redhat.com, ip: 63.128.21.124, mailfrom: lersek@redhat.com) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1611087163; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=sFY6dcpHHU19jPjYDdyaMuOCL7Ui7Ykr6cQ/xjXfZQs=; b=cLP0Mo3aGyTnvLSLyQlmDvGiQZ45vKTR7pbVvAeHxI4xVQk/mjIaKabyPH/4NYunFcbp8M 7FKm0gqE80lbvWst+OYp8fktn2pCqYz5KC46T3xkJWw1+KnTKTTPyfzQRM+axCXguzpA+U U/izR6l/ovnHeuZqe5BL6rPLhXxI4uQ= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-500-0lGNObIpPOOAugVGEM7QiQ-1; Tue, 19 Jan 2021 15:12:34 -0500 X-MC-Unique: 0lGNObIpPOOAugVGEM7QiQ-1 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 0C6E8C73A1; Tue, 19 Jan 2021 20:12:33 +0000 (UTC) Received: from lacos-laptop-7.usersys.redhat.com (ovpn-114-176.ams2.redhat.com [10.36.114.176]) by smtp.corp.redhat.com (Postfix) with ESMTP id BEEEF19CB2; Tue, 19 Jan 2021 20:12:30 +0000 (UTC) Subject: Re: [EXTERNAL] [edk2-devel] building the shell for edk2-stable202102 To: devel@edk2.groups.io, bret.barkelew@microsoft.com, "Liming Gao (Byosoft address)" , "Leif Lindholm (Nuvia address)" , "Ard Biesheuvel (TianoCore)" , "Ni, Ray" , Zhichao Gao , Abner Chang , Michael Kinney References: <660228dd-649f-2fd9-e4c6-d8d206194020@redhat.com> From: "Laszlo Ersek" Message-ID: Date: Tue, 19 Jan 2021 21:12:29 +0100 MIME-Version: 1.0 In-Reply-To: X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=lersek@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset=windows-1252 Content-Language: en-US Content-Transfer-Encoding: 7bit +Mike On 01/19/21 20:34, Bret Barkelew via groups.io wrote: > We've definitely build release pipelines for binaries internally and > would be happy to help with this. Thanks! > > I like the idea of kicking a new release after a release is tagged. Do > you have any thoughts about how we're possibly re-release if there > were changes to one of the new stable/* branches, or would it be > easier to just cross that bridge when we get there? If I remember correctly, any given stable branch also carries tags, so the tagging action could be used to kick off the build process there as well. Please refer to section (4) in the following RFC -- "Release requirements for supported branches": [edk2-rfc] [RFC V3] Create supported branch from edk2-stable* tag (Required to address critical bug BZ3111) https://edk2.groups.io/g/rfc/message/470 > Do you have opinions about Nuget vs other binary release mechanisms > (our experience is with Nuget, but I know there are other feed types > that we could publish to)? I think we should primarily release via the github.com Assets drop-downs -- as long as we publish our sources there, anyway. Additional release channels could be enabled (if they are convenient for some users). ... I do think it's quite important to offer bare https URLs (at least in the primary location), which nuget.org doesn't seem to do (correct me if I'm wrong please). > Ideally it would be something that could be authenticated or compared > somehow. Yep, that's messy. I vaguely recall discussing this, namely that github.com generates *some* of the Assets (namely the edk2 source zip files?) on-demand, and that leads to different results e.g. whenever the central github.com compression settings change. Hm... this was a topic from Leif and Mike, in the December 2020 stewards' meeting. References (from the minutes): https://github.com/abseil/federation-hello/issues/3 https://github.com/spack/spack/issues/5411 FWIW, the submodule source tarballs are not affected, because those are prepared by Liming manually, according to the method originally proposed here: . In the end, Mike removed said re-generation manually, for past releases: https://bugzilla.tianocore.org/show_bug.cgi?id=3099 But I think we'd benefit from an automatism that doesn't leave anything to github.com or to manual processes, and generates all the artifacts / assets internally. Perhaps the final upload could be left to the release manager (for final human supervision). Summary (of my opinion): - Tags look suitable for kicking off release builds, both on the master and the stable branches. - I'd prefer releasing all assets through primarily, for now; additional channels are welcome. - Reproducible builds are important; they appear available on github.com too (only every tarball etc has to be uploaded as a custom binary file, disabling on-demand generation). - Cryptographic signing for assets: probably not (needs a whole separate organization and infrastructure for handling private keys, AFAICT). Anyway, let's not allow "perfect" to get in the way of "good" -- this deserves its own RFC thread probably; for now I'd just like to add a permanent reminder "somewhere" about the need to rebuild the shell, for edk2-stable202102. Thanks! Laszlo > > - Bret > > From: Laszlo Ersek via groups.io > Sent: Tuesday, January 19, 2021 11:30 AM > To: Liming Gao (Byosoft address); > Leif Lindholm (Nuvia address); Ard > Biesheuvel (TianoCore); Ni, > Ray; Zhichao > Gao; Abner > Chang > Cc: edk2-devel-groups-io > Subject: [EXTERNAL] [edk2-devel] building the shell for > edk2-stable202102 > > Ouch, I totally forgot to add the mailing list to the address list! > Doing that now. Apologies. > > --o-- > > Hi All, > > we've last built the UEFI shell binary for edk2-stable202002 (i.e., > almost 1 year ago): > > https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2%2Freleases%2Ftag%2Fedk2-stable202002&data=04%7C01%7Cbret.barkelew%40microsoft.com%7C8e0e9a9a97be429c1b6c08d8bcb0bcb0%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637466814561378976%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=ObFHM4vr8SuNSLwSwBD95qIvZjt7wRCA54xinkWZeLY%3D&reserved=0 > > Note "ShellBinPkg.zip" under Assets -- there is no stable tag that is > (a) more recent and (b) whose Assets contain "ShellBinPkg.zip". > > Contents: > >> Archive: ShellBinPkg.zip >> Length Date Time Name >> --------- ---------- ----- ---- >> 0 03-06-2020 22:43 ShellBinPkg/MinUefiShell/ >> 0 03-06-2020 22:41 ShellBinPkg/MinUefiShell/AArch64/ >> 380928 03-06-2020 17:39 ShellBinPkg/MinUefiShell/AArch64/Shell.efi >> 0 03-06-2020 22:41 ShellBinPkg/MinUefiShell/Arm/ >> 321568 03-06-2020 17:38 ShellBinPkg/MinUefiShell/Arm/Shell.efi >> 0 03-05-2020 09:01 ShellBinPkg/MinUefiShell/Ia32/ >> 339424 03-05-2020 09:01 ShellBinPkg/MinUefiShell/Ia32/Shell.efi >> 643 03-06-2020 22:43 ShellBinPkg/MinUefiShell/MinUefiShell.inf >> 0 03-05-2020 09:01 ShellBinPkg/MinUefiShell/X64/ >> 392352 03-05-2020 09:01 ShellBinPkg/MinUefiShell/X64/Shell.efi >> 0 03-06-2020 22:43 ShellBinPkg/UefiShell/ >> 0 03-06-2020 22:41 ShellBinPkg/UefiShell/AArch64/ >> 892928 03-06-2020 17:40 ShellBinPkg/UefiShell/AArch64/Shell.efi >> 0 03-06-2020 22:41 ShellBinPkg/UefiShell/Arm/ >> 791360 03-06-2020 17:39 ShellBinPkg/UefiShell/Arm/Shell.efi >> 0 03-05-2020 09:01 ShellBinPkg/UefiShell/Ia32/ >> 825184 03-05-2020 09:00 ShellBinPkg/UefiShell/Ia32/Shell.efi >> 643 03-06-2020 22:43 ShellBinPkg/UefiShell/UefiShell.inf >> 0 03-05-2020 09:01 ShellBinPkg/UefiShell/X64/ >> 939648 03-05-2020 09:01 ShellBinPkg/UefiShell/X64/Shell.efi >> 0 03-06-2020 22:40 ShellBinPkg/ >> --------- ------- >> 4884678 21 files > > I propose that we rebuild the shell for edk2-stable202102. Reasons: > > (1) There are two small shell features minimally in the latest > development cycle: > > https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Ftianocore.github.io%2Fwiki%2FEDK-II-Release-Planning&data=04%7C01%7Cbret.barkelew%40microsoft.com%7C8e0e9a9a97be429c1b6c08d8bcb0bcb0%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637466814561378976%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=sH5nHKgFwcykHgNKm6Nu2esq05F1dKt3t%2BEncNyRap8%3D&reserved=0 > > * add file buffering to the UEFI shell's COMP command > https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fbugzilla.tianocore.org%2Fshow_bug.cgi%3Fid%3D3123&data=04%7C01%7Cbret.barkelew%40microsoft.com%7C8e0e9a9a97be429c1b6c08d8bcb0bcb0%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637466814561378976%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=Po1VBWyzo7YPioV1FUJZ2t1rB%2FhwgoofX8EH9YKJPBc%3D&reserved=0 > > * Shell: pathname / filename sorting > https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fbugzilla.tianocore.org%2Fshow_bug.cgi%3Fid%3D3151&data=04%7C01%7Cbret.barkelew%40microsoft.com%7C8e0e9a9a97be429c1b6c08d8bcb0bcb0%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637466814561378976%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=Qz58BSZ7iwNpyD%2BvOayA0VaA0IlDWNz14zwCjxsS6aU%3D&reserved=0 > > (2) The zip file listed above does not contain a RISC-V binary, and > RISC-V has been an official UEFI and edk2 platform minimally since > edk2-stable202005 / > . > > In particular, the following two platforms in edk2-platforms include > the shell (SUPPORTED_ARCHITECTURES = RISCV64): > > Platform/SiFive/U5SeriesPkg/FreedomU500VC707Board/U500.dsc > Platform/SiFive/U5SeriesPkg/FreedomU540HiFiveUnleashedBoard/U540.dsc > > However, as of this writing (@ 6e5586863148), we only have the > following list in "Maintainers.txt": > >> UEFI Shell Binaries (ShellBinPkg.zip) from EDK II Releases: >> ----------------------------------------------------------- >> W: https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftianocore%2Fedk2%2Freleases%2F&data=04%7C01%7Cbret.barkelew%40microsoft.com%7C8e0e9a9a97be429c1b6c08d8bcb0bcb0%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637466814561388970%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=ikDhWcETk90pMfREpPuvAWzobu41W2vUQT9wrRRBhzM%3D&reserved=0 >> M: Ray Ni (Ia32/X64) >> M: Zhichao Gao (Ia32/X64) >> M: Leif Lindholm (ARM/AArch64) >> M: Ard Biesheuvel (ARM/AArch64) > > I think that (a) Abner should be added to this list, and (b) we > should include a RISC-V shell binary in the upcoming assets. > > Abner, can you send a patch for "Maintainers.txt" please? > > Questions: > > - I'm not clear on how we intend to build the shell binaries -- will we > retrieve them from CI / Azure somehow, or is it a manual process? > > - Given that this is a release activity, I'm unsure where I could file a > reminder about it -- clearly, the binaries should be built right after > the tag has been made. > > Should I perhaps file a new reminder BZ for the "N/A" Package, and > maybe assign it to Liming (our release manager)? > > Thanks, > Laszlo > > > > > > > > > > > >