From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by mx.groups.io with SMTP id smtpd.web10.56898.1674678095125371683 for ; Wed, 25 Jan 2023 12:21:35 -0800 Authentication-Results: mx.groups.io; dkim=pass header.i=@redhat.com header.s=mimecast20190719 header.b=dIFy1vEE; spf=pass (domain: redhat.com, ip: 170.10.133.124, mailfrom: osteffen@redhat.com) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1674678094; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=ReLMXrVmENUaDUln5Z9BUKZIK7DkTnYU+HGC9t9r1Bc=; b=dIFy1vEEaUXHzwZi++5BP3H3spqRvbDLUOdDCXQCKnT9TGNCLVpaHPGFE1VH6fYq0Li0vj IHygSstJgKknltzz7X7FryBPNPBCBSjOEP2zYPdj/9TsopgIFiDBCN+j94YdgQ5To6afM3 IkHpJ47DnRRqOHSUxA5HWDid0xdgkhg= Received: from mail-lj1-f200.google.com (mail-lj1-f200.google.com [209.85.208.200]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_128_GCM_SHA256) id us-mta-278-E4J5ccvoNQO3wcag1OR8zQ-1; Wed, 25 Jan 2023 15:21:32 -0500 X-MC-Unique: E4J5ccvoNQO3wcag1OR8zQ-1 Received: by mail-lj1-f200.google.com with SMTP id e10-20020a2ea54a000000b0028bb7bdae44so4402704ljn.5 for ; Wed, 25 Jan 2023 12:21:32 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=ReLMXrVmENUaDUln5Z9BUKZIK7DkTnYU+HGC9t9r1Bc=; b=vkZZMhHSEkAbil3tutjQik9NA313EjDCQ28HrrLMUFKxR2JiGfawQdOX0QEnTn3UGF jrErxaBnhGegqF/VFOdVe1HtTo7EN50nQkrnqoNJRJBnI2enriv2C3moPWlX5aPasyze AeAEYdwCXQQWEkREbkWuCktJ3/Sk5iHi2ApUxfNumr+LA6BTvYyDIi6PnaQg9kwZwGki hzrDfSmBY5sovh14voxiFIifrBRwiDkno+a1OgcPYBmGZzMxbuN2YxXHQNbRIyyZvqJ+ 1SFVF9HGJlUOuJdaQRFg4yTVRU+MkX0tyKyTSVHjL6huhIOI6MASTf6+h3LzTPtddg6D 0TcA== X-Gm-Message-State: AFqh2kqwlZVQvtPhN9R5T/CWiQ6gnYsdxOB6KRODAzmF/jrnNSTVwHLW 9r/4SH1iFAYHv2o67UPGQhOwdqAw2xHd3cSlCBBBERYJbOht010QOcwI3LQnJ3e3oq2GVVnr3FA JCJkfQR7r655Y5eZcQMvRmkMZk0OKeQ== X-Received: by 2002:a05:6512:ad4:b0:4d5:79e9:ece0 with SMTP id n20-20020a0565120ad400b004d579e9ece0mr1545594lfu.400.1674678090646; Wed, 25 Jan 2023 12:21:30 -0800 (PST) X-Google-Smtp-Source: AMrXdXvVv8IvV+UlyU+ZgnyH67QuWscSHt6walo88dcd9EaysPFHNMFDFNAWsKCZsP2Zx4/U+int8LItfhU/HBZjPPw= X-Received: by 2002:a05:6512:ad4:b0:4d5:79e9:ece0 with SMTP id n20-20020a0565120ad400b004d579e9ece0mr1545587lfu.400.1674678090342; Wed, 25 Jan 2023 12:21:30 -0800 (PST) MIME-Version: 1.0 References: <20230125165556.tvp7cgjsj2kie7b6@sirius.home.kraxel.org> In-Reply-To: From: "Oliver Steffen" Date: Wed, 25 Jan 2023 21:21:18 +0100 Message-ID: Subject: Re: [edk2-devel] arm64 support for stuart To: "Kinney, Michael D" Cc: "devel@edk2.groups.io" , Gerd Hoffmann , Ard Biesheuvel , Michael Kubacki , Sean Brogan , "Yao, Jiewen" X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: multipart/alternative; boundary="000000000000b042e305f31c602a" --000000000000b042e305f31c602a Content-Type: text/plain; charset="UTF-8" On Wed, Jan 25, 2023 at 8:00 PM Kinney, Michael D < michael.d.kinney@intel.com> wrote: > Sounds like a reasonable feature request to disable install of all > external tools. Pytools uses GitHub issues, so a feature request like > this can be entered there. > There was a discussion about this (https://github.com/tianocore/edk2-pytool-extensions/discussions/323), where some possible approaches were described. Back then we were under the time pressure of getting the CI patches in before Dec 1st (the day the ubuntu-18 image was expected to disappear from Azure Pipelines). In the end the solution of deleting the ext_dep files was accepted. I opened a feature request here: https://github.com/tianocore/edk2-pytool-extensions/issues/416 > > One caution on this approach is that you make see passing conditions using > your local versions of dependent tools, > > but when you submit a PR and the CI tools run with potentially different > tools, there may be failures. > > > > If a failure it detected, then you will likely need to install the same > version of tools the CI uses to reproduce, > > debug, and resolve. > Sure. But at least there would be a good way to bring your own tools. - Oliver > > Mike > > > > *From:* devel@edk2.groups.io *On Behalf Of *Oliver > Steffen > *Sent:* Wednesday, January 25, 2023 10:28 AM > *To:* Gerd Hoffmann > *Cc:* Ard Biesheuvel ; Michael Kubacki < > mikuback@linux.microsoft.com>; Sean Brogan ; > Kinney, Michael D ; edk2-devel-groups-io < > devel@edk2.groups.io>; Yao, Jiewen > *Subject:* Re: [edk2-devel] arm64 support for stuart > > > > > > > > On Wed, Jan 25, 2023 at 5:56 PM Gerd Hoffmann wrote: > > Hi, > > > Given that nasm is x86 specific, we should be able to work around this > > by moving the nasm_ext_dep.yaml file into the right place. > > Overall stuart feels kind of alien to linux. It just goes download > stuff from the internet, even in case the tools are already available > locally. Oliver fixed some (but not all) of these when moving CI over > to using containers. IIRC at least the cross compilers are just the > standard fedora cross compiler packages now. > > While being at it: edk2-pytool-library fails to build with network > access turned off[1] because it tries to download vswhere.exe from the > internet. Even when building on linux. > > > Then, if/when mu_nasm for arm64 becomes available, we will also be > > able to build OVMF from arm64 (although I am probably the only person > > in the world who does that regularly.) > > Fedora build system does that too. We have a patch to make x86 cross > builds work like arm cross builds, by just setting GCC5_${ARCH}_PREFIX > environment variable: > > https://github.com/kraxel/edk2/commit/6b2ca6f01bb76a3b9632e902b4bf0ef9e912ce40 > > Guess I should submit that one for upstream inclusion ;) > > > iasl is a different matter, as we need it to build for arm64 as well. > > iasl is already available in the arm64 distros, so as I see it, there > > are 3 options here: > > - build iasl for Linux/arm64 and add it to the nuget repo > > - allow a fallback to system-wide iasl (how?) > > Just use the system-wide tools is the best option IMHO. The packages > are available in Fedora (other distros should be have them too), on both > x86_64 and aarch64, we only need to add them to the CI container image. > So why bother adding nuget builds? > > That is also less fragile than downloading them on each CI run and have > checks fail now and then due to network problems. > > Is it possible to run github actions (used to build containers) and > azure pipelines on aarch64 systems? So we could move ArmVirt CI from > x86 cross builds to native arm builds? > > > > Not natively, AFAIK, at least not without self-hosting a runner or using > 3rd-party > > services. > > Most suggestions I came across suggest running in Qemu (tcg)... > > > > > take care, > Gerd > > [1] Offline builds are standard for linux distro builds to make sure > all sources needed are to produce the binary package are actually > included in the source package. > > > --000000000000b042e305f31c602a Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable


On Wed, Jan 25, 2023 at 8:00 PM Kinney, Michael D &= lt;michael.d.kinney@intel.com= > wrote:
=

Sounds like a reasonable feature request to di= sable install of all external tools.=C2=A0 Pytools uses GitH= ub issues, so a feature request like this can be entered there.

<= /div>
There was a discussion about this
appro= aches were described.
Back then we were under the time pressure of getting t= he CI patches in before Dec 1st (the
day the ubuntu-18 image was expected to = disappear from Azure Pipelines).
In the end the solution of deleting the ext_dep file= s was accepted.

=C2=A0

One caution on this approach is that you make = see passing conditions using your local versions of dependent tools,=

but when you submit a PR and the CI tools run = with potentially different tools, there may be failures.

=C2=A0

If a failure it detected, then you will likely= need to install the same version of tools the CI uses to reproduce,=

debug, and resolve.

=C2=A0
Sure.=C2=A0 But at least there would be a good way to bring your own = tools.

- Oliver

=C2=A0

Mike

=C2=A0

From: devel@edk2.groups.io <devel@edk2.groups.io= > On Behalf Of Oliver Steffen
Sent: Wednesday, January 25, 2023 10:28 AM
To: Gerd Hoffmann <kraxel@redhat.com>
Cc: Ard Biesheuvel <ardb@kernel.org>; Michael Kubacki <mikuback@linux.microsoft.com>; Sean Brogan <sean.brogan@microsoft.com>; Kinney, Michael D <michael.d.kinney@= intel.com>; edk2-devel-groups-io <devel@edk2.groups.io>; Yao, Jiewen <jiewen.yao@intel.co= m>
Subject: Re: [edk2-devel] arm64 support for stuart

=C2=A0

=C2=A0<= /span>

=C2=A0

On Wed, Jan 25, 2023 at 5:56 PM Gerd Hoffmann <kraxel@redhat.com&= gt; wrote:

=C2=A0 Hi,

> Given that nasm is x86 specific, we should be able to work around this=
> by moving the nasm_ext_dep.yaml file into the right place.

Overall stuart feels kind of alien to linux.=C2=A0 It just goes download stuff from the internet, even in case the tools are already available
locally.=C2=A0 Oliver fixed some (but not all) of these when moving CI over=
to using containers.=C2=A0 IIRC at least the cross compilers are just the standard fedora cross compiler packages now.

While being at it:=C2=A0 edk2-pytool-library fails to build with network access turned off[1] because it tries to download vswhere.exe from the
internet.=C2=A0 Even when building on linux.

> Then, if/when mu_nasm for arm64 becomes available, we will also be
> able to build OVMF from arm64 (although I am probably the only person<= br> > in the world who does that regularly.)

Fedora build system does that too.=C2=A0 We have a patch to make x86 cross<= br> builds work like arm cross builds, by just setting GCC5_${ARCH}_PREFIX
environment variable:
=C2=A0 https://github.com/kraxel/edk2/commit/6b2ca6f01bb76a3b9632e902b4bf0ef9e912c= e40

Guess I should submit that one for upstream inclusion ;)

> iasl is a different matter, as we need it to build for arm64 as well.<= br> > iasl is already available in the arm64 distros, so as I see it, there<= br> > are 3 options here:
> - build iasl for Linux/arm64 and add it to the nuget repo
> - allow a fallback to system-wide iasl (how?)

Just use the system-wide tools is the best option IMHO.=C2=A0 The packages<= br> are available in Fedora (other distros should be have them too), on both x86_64 and aarch64, we only need to add them to the CI container image.
So why bother adding nuget builds?

That is also less fragile than downloading them on each CI run and have
checks fail now and then due to network problems.

Is it possible to run github actions (used to build containers) and
azure pipelines on aarch64 systems?=C2=A0 So we could move ArmVirt CI from<= br> x86 cross builds to native arm builds?

=C2=A0

Not natively, AFAIK, at least not without self= -hosting a runner or using 3rd-party

services.

Most suggestions I came across suggest running= in Qemu (tcg)...

=C2=A0


take care,
=C2=A0 Gerd

[1] Offline builds are standard for linux distro builds to make sure
=C2=A0 =C2=A0 all sources needed are to produce the binary package are actu= ally
=C2=A0 =C2=A0 included in the source package.

--000000000000b042e305f31c602a--