[Patch][edk2-stable202011 0/3] Address EDK II CI Failures

[Patch][edk2-stable202011 0/3] Address EDK II CI Failures

[Michael D Kinney]

* Always fetch the target branch of the PR (usually origin/master)
  so diffs between PR and the target branch can be performed.  There
  is no guarantee that the target branch is fetched when a PR is 
  evaluated by a CI agent.
* QEMU release for Windows from Nov 20, 2020 is installed into 
  wrong directory.  Use previous QEMU for Windows release from
  Aug 14, 2020.

Cc: Sean Brogan <sean.brogan@microsoft.com>
Cc: Bret Barkelew <Bret.Barkelew@microsoft.com>
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Cc: Jordan Justen <jordan.l.justen@intel.com>
Cc: Laszlo Ersek <lersek@redhat.com>
Cc: Ard Biesheuvel <ard.biesheuvel@arm.com>
Cc: Andrew Fish <afish@apple.com>
Cc: Leif Lindholm <leif@nuviainc.com>
Signed-off-by: Michael D Kinney <michael.d.kinney@intel.com>

Michael D Kinney (1):
  OvmfPkg/PlatformCI/.azurepiplines: Use stable release of QEMU

Sean Brogan (2):
  .azurepipelines/templates/pr-gate-steps.yml: Fetch target branch
  .azurepipelines/templates/platform-build-run-steps.yml: Fetch target
    branch

 .azurepipelines/templates/platform-build-run-steps.yml | 6 ++++++
 .azurepipelines/templates/pr-gate-steps.yml            | 6 ++++++
 OvmfPkg/PlatformCI/.azurepipelines/Windows-VS2019.yml  | 2 +-
 3 files changed, 13 insertions(+), 1 deletion(-)

-- 
2.29.2.windows.2

[Patch][edk2-stable202011 1/3] .azurepipelines/templates/pr-gate-steps.yml: Fetch target branch

[Michael D Kinney]

From: Sean Brogan <sean.brogan@microsoft.com>

Add step to fetch target branch so that PR_EVAL can resolve diff

Cc: Bret Barkelew <Bret.Barkelew@microsoft.com>
Cc: Michael D Kinney <michael.d.kinney@intel.com>
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Signed-off-by: Sean Brogan <sean.brogan@microsoft.com>
Reviewed-by: Michael D Kinney <michael.d.kinney@intel.com>
---
 .azurepipelines/templates/pr-gate-steps.yml | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/.azurepipelines/templates/pr-gate-steps.yml b/.azurepipelines/templates/pr-gate-steps.yml
index c1c0c04d6c17..70c19a462194 100644
--- a/.azurepipelines/templates/pr-gate-steps.yml
+++ b/.azurepipelines/templates/pr-gate-steps.yml
@@ -31,6 +31,12 @@ steps:
     echo "##vso[task.setvariable variable=pkgs_to_build]${{ parameters.build_pkgs }}"
     echo "##vso[task.setvariable variable=pkg_count]${{ 1 }}"
 
+# Fetch the target branch so that pr_eval can diff them.
+# Seems like azure pipelines/github changed checkout process in nov 2020.
+- script: git fetch origin $(System.PullRequest.targetBranch)
+  displayName: fetch target branch
+  condition: eq(variables['Build.Reason'], 'PullRequest')
+
 # trim the package list if this is a PR
 - task: CmdLine@1
   displayName: Check if ${{ parameters.build_pkgs }} need testing
-- 
2.29.2.windows.2

[Patch][edk2-stable202011 2/3] .azurepipelines/templates/platform-build-run-steps.yml: Fetch target branch

[Michael D Kinney]

From: Sean Brogan <sean.brogan@microsoft.com>

Update Platform build steps process to fetch target branch

Cc: Bret Barkelew <Bret.Barkelew@microsoft.com>
Cc: Michael D Kinney <michael.d.kinney@intel.com>
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Signed-off-by: Sean Brogan <sean.brogan@microsoft.com>
Reviewed-by: Michael D Kinney <michael.d.kinney@intel.com>
---
 .azurepipelines/templates/platform-build-run-steps.yml | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/.azurepipelines/templates/platform-build-run-steps.yml b/.azurepipelines/templates/platform-build-run-steps.yml
index ebf674bfc7d8..97e7faa26682 100644
--- a/.azurepipelines/templates/platform-build-run-steps.yml
+++ b/.azurepipelines/templates/platform-build-run-steps.yml
@@ -51,6 +51,12 @@ steps:
 # Set default
 - bash: echo "##vso[task.setvariable variable=pkg_count]${{ 1 }}"
 
+# Fetch the target branch so that pr_eval can diff them.
+# Seems like azure pipelines/github changed checkout process in nov 2020.
+- script: git fetch origin $(System.PullRequest.targetBranch)
+  displayName: fetch target branch
+  condition: eq(variables['Build.Reason'], 'PullRequest')
+
 # trim the package list if this is a PR
 - task: CmdLine@1
   displayName: Check if ${{ parameters.build_pkg }} need testing
-- 
2.29.2.windows.2

[Patch][edk2-stable202011 3/3] OvmfPkg/PlatformCI/.azurepiplines: Use stable release of QEMU

[Michael D Kinney]

Install the 2020.08.14 release of QEMU for Windows.

The QEMU release from 2020.11.20 is installed into the incorrect
directory and is causing EDK II CI failures in the run to shell
step.

Cc: Sean Brogan <sean.brogan@microsoft.com>
Cc: Bret Barkelew <Bret.Barkelew@microsoft.com>
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Cc: Jordan Justen <jordan.l.justen@intel.com>
Cc: Laszlo Ersek <lersek@redhat.com>
Cc: Ard Biesheuvel <ard.biesheuvel@arm.com>
Signed-off-by: Michael D Kinney <michael.d.kinney@intel.com>
---
 OvmfPkg/PlatformCI/.azurepipelines/Windows-VS2019.yml | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/OvmfPkg/PlatformCI/.azurepipelines/Windows-VS2019.yml b/OvmfPkg/PlatformCI/.azurepipelines/Windows-VS2019.yml
index 02ed233fdb60..dd19fff50066 100644
--- a/OvmfPkg/PlatformCI/.azurepipelines/Windows-VS2019.yml
+++ b/OvmfPkg/PlatformCI/.azurepipelines/Windows-VS2019.yml
@@ -132,7 +132,7 @@ jobs:
         build_flags: $(Build.Flags)
         run_flags: $(Run.Flags)
         extra_install_step:
-        - powershell: choco install qemu; Write-Host "##vso[task.prependpath]c:\Program Files\qemu"
+        - powershell: choco install qemu --version=2020.08.14; Write-Host "##vso[task.prependpath]c:\Program Files\qemu"
           displayName: Install QEMU and Set QEMU on path # friendly name displayed in the UI
           condition: and(gt(variables.pkg_count, 0), succeeded())
 
-- 
2.29.2.windows.2

Re: [edk2-devel] [Patch][edk2-stable202011 0/3] Address EDK II CI Failures

[Michael D Kinney]

A test PR was sent through CI with these changes.

    https://github.com/tianocore/edk2/pull/1143

All checks passed and the PR was auto closed.

This does not address the 2nd issue related to the CryptoPkg 
unit tests for MD5.  These are not run for a PR unless there are
changes to the CryptoPkg in the PR.  The post commit checks 
will run when this commit is made and the CryptoPkg unit tests
are expected to fail.

The CryptoPkg unit test fix is still being investigated.

Mike

> -----Original Message-----
> From: devel@edk2.groups.io <devel@edk2.groups.io> On Behalf Of Michael D Kinney
> Sent: Tuesday, November 24, 2020 6:12 PM
> To: devel@edk2.groups.io
> Cc: Sean Brogan <sean.brogan@microsoft.com>; Bret Barkelew <Bret.Barkelew@microsoft.com>; Liming Gao
> <gaoliming@byosoft.com.cn>; Justen, Jordan L <jordan.l.justen@intel.com>; Laszlo Ersek <lersek@redhat.com>; Ard Biesheuvel
> <ard.biesheuvel@arm.com>; Andrew Fish <afish@apple.com>; Leif Lindholm <leif@nuviainc.com>
> Subject: [edk2-devel] [Patch][edk2-stable202011 0/3] Address EDK II CI Failures
> 
> * Always fetch the target branch of the PR (usually origin/master)
>   so diffs between PR and the target branch can be performed.  There
>   is no guarantee that the target branch is fetched when a PR is
>   evaluated by a CI agent.
> * QEMU release for Windows from Nov 20, 2020 is installed into
>   wrong directory.  Use previous QEMU for Windows release from
>   Aug 14, 2020.
> 
> Cc: Sean Brogan <sean.brogan@microsoft.com>
> Cc: Bret Barkelew <Bret.Barkelew@microsoft.com>
> Cc: Liming Gao <gaoliming@byosoft.com.cn>
> Cc: Jordan Justen <jordan.l.justen@intel.com>
> Cc: Laszlo Ersek <lersek@redhat.com>
> Cc: Ard Biesheuvel <ard.biesheuvel@arm.com>
> Cc: Andrew Fish <afish@apple.com>
> Cc: Leif Lindholm <leif@nuviainc.com>
> Signed-off-by: Michael D Kinney <michael.d.kinney@intel.com>
> 
> Michael D Kinney (1):
>   OvmfPkg/PlatformCI/.azurepiplines: Use stable release of QEMU
> 
> Sean Brogan (2):
>   .azurepipelines/templates/pr-gate-steps.yml: Fetch target branch
>   .azurepipelines/templates/platform-build-run-steps.yml: Fetch target
>     branch
> 
>  .azurepipelines/templates/platform-build-run-steps.yml | 6 ++++++
>  .azurepipelines/templates/pr-gate-steps.yml            | 6 ++++++
>  OvmfPkg/PlatformCI/.azurepipelines/Windows-VS2019.yml  | 2 +-
>  3 files changed, 13 insertions(+), 1 deletion(-)
> 
> --
> 2.29.2.windows.2
> 
> 
> 
> 
> 

回复: [edk2-devel] [Patch][edk2-stable202011 0/3] Address EDK II CI Failures

[gaoliming]

Mike and Sean:
  Thanks for your quick fix. I agree to merge them for edk2-stable202011. So far, there are two patches to be merged for edk2-stable202011. With these fixes, the second one can be merged. The first one may still be blocked. Is it right? 

https://edk2.groups.io/g/devel/message/67823 [PATCH v2 1/1] CryptoPkg: BaseCryptLib: Fix buffer double free in CryptPkcs7VerifyEku
https://edk2.groups.io/g/devel/message/67853 [PATCH 1/1] Revert "SecurityPkg: Add RPMC Index to the RpmcLib"

Thanks
Liming
> -----邮件原件-----
> 发件人: bounce+27952+67923+4905953+8761045@groups.io
> <bounce+27952+67923+4905953+8761045@groups.io> 代表 Michael D
> Kinney
> 发送时间: 2020年11月25日 10:17
> 收件人: devel@edk2.groups.io; Kinney, Michael D
> <michael.d.kinney@intel.com>
> 抄送: Sean Brogan <sean.brogan@microsoft.com>; Bret Barkelew
> <Bret.Barkelew@microsoft.com>; Liming Gao <gaoliming@byosoft.com.cn>;
> Justen, Jordan L <jordan.l.justen@intel.com>; Laszlo Ersek
> <lersek@redhat.com>; Ard Biesheuvel <ard.biesheuvel@arm.com>; Andrew
> Fish <afish@apple.com>; Leif Lindholm <leif@nuviainc.com>
> 主题: Re: [edk2-devel] [Patch][edk2-stable202011 0/3] Address EDK II CI
> Failures
> 
> A test PR was sent through CI with these changes.
> 
>     https://github.com/tianocore/edk2/pull/1143
> 
> All checks passed and the PR was auto closed.
> 
> This does not address the 2nd issue related to the CryptoPkg
> unit tests for MD5.  These are not run for a PR unless there are
> changes to the CryptoPkg in the PR.  The post commit checks
> will run when this commit is made and the CryptoPkg unit tests
> are expected to fail.
> 
> The CryptoPkg unit test fix is still being investigated.
> 
> Mike
> 
> > -----Original Message-----
> > From: devel@edk2.groups.io <devel@edk2.groups.io> On Behalf Of Michael
> D Kinney
> > Sent: Tuesday, November 24, 2020 6:12 PM
> > To: devel@edk2.groups.io
> > Cc: Sean Brogan <sean.brogan@microsoft.com>; Bret Barkelew
> <Bret.Barkelew@microsoft.com>; Liming Gao
> > <gaoliming@byosoft.com.cn>; Justen, Jordan L <jordan.l.justen@intel.com>;
> Laszlo Ersek <lersek@redhat.com>; Ard Biesheuvel
> > <ard.biesheuvel@arm.com>; Andrew Fish <afish@apple.com>; Leif
> Lindholm <leif@nuviainc.com>
> > Subject: [edk2-devel] [Patch][edk2-stable202011 0/3] Address EDK II CI
> Failures
> >
> > * Always fetch the target branch of the PR (usually origin/master)
> >   so diffs between PR and the target branch can be performed.  There
> >   is no guarantee that the target branch is fetched when a PR is
> >   evaluated by a CI agent.
> > * QEMU release for Windows from Nov 20, 2020 is installed into
> >   wrong directory.  Use previous QEMU for Windows release from
> >   Aug 14, 2020.
> >
> > Cc: Sean Brogan <sean.brogan@microsoft.com>
> > Cc: Bret Barkelew <Bret.Barkelew@microsoft.com>
> > Cc: Liming Gao <gaoliming@byosoft.com.cn>
> > Cc: Jordan Justen <jordan.l.justen@intel.com>
> > Cc: Laszlo Ersek <lersek@redhat.com>
> > Cc: Ard Biesheuvel <ard.biesheuvel@arm.com>
> > Cc: Andrew Fish <afish@apple.com>
> > Cc: Leif Lindholm <leif@nuviainc.com>
> > Signed-off-by: Michael D Kinney <michael.d.kinney@intel.com>
> >
> > Michael D Kinney (1):
> >   OvmfPkg/PlatformCI/.azurepiplines: Use stable release of QEMU
> >
> > Sean Brogan (2):
> >   .azurepipelines/templates/pr-gate-steps.yml: Fetch target branch
> >   .azurepipelines/templates/platform-build-run-steps.yml: Fetch target
> >     branch
> >
> >  .azurepipelines/templates/platform-build-run-steps.yml | 6 ++++++
> >  .azurepipelines/templates/pr-gate-steps.yml            | 6 ++++++
> >  OvmfPkg/PlatformCI/.azurepipelines/Windows-VS2019.yml  | 2 +-
> >  3 files changed, 13 insertions(+), 1 deletion(-)
> >
> > --
> > 2.29.2.windows.2
> >
> >
> >
> >
> >
> 
> 
> 
> 
> 

Re: [edk2-devel] [Patch][edk2-stable202011 0/3] Address EDK II CI Failures

[Michael D Kinney]

Hi Liming,

Yes.  A PR for CryptoPkg will be blocked.

However, both will generate a failed status on the post commit checks.

Mike

> -----Original Message-----
> From: gaoliming <gaoliming@byosoft.com.cn>
> Sent: Tuesday, November 24, 2020 7:02 PM
> To: devel@edk2.groups.io; Kinney, Michael D <michael.d.kinney@intel.com>
> Cc: 'Sean Brogan' <sean.brogan@microsoft.com>; 'Bret Barkelew' <Bret.Barkelew@microsoft.com>; Justen, Jordan L
> <jordan.l.justen@intel.com>; 'Laszlo Ersek' <lersek@redhat.com>; 'Ard Biesheuvel' <ard.biesheuvel@arm.com>; 'Andrew Fish'
> <afish@apple.com>; 'Leif Lindholm' <leif@nuviainc.com>
> Subject: 回复: [edk2-devel] [Patch][edk2-stable202011 0/3] Address EDK II CI Failures
> 
> Mike and Sean:
>   Thanks for your quick fix. I agree to merge them for edk2-stable202011. So far, there are two patches to be merged for
> edk2-stable202011. With these fixes, the second one can be merged. The first one may still be blocked. Is it right?
> 
> https://edk2.groups.io/g/devel/message/67823 [PATCH v2 1/1] CryptoPkg: BaseCryptLib: Fix buffer double free in
> CryptPkcs7VerifyEku
> https://edk2.groups.io/g/devel/message/67853 [PATCH 1/1] Revert "SecurityPkg: Add RPMC Index to the RpmcLib"
> 
> Thanks
> Liming
> > -----邮件原件-----
> > 发件人: bounce+27952+67923+4905953+8761045@groups.io
> > <bounce+27952+67923+4905953+8761045@groups.io> 代表 Michael D
> > Kinney
> > 发送时间: 2020年11月25日 10:17
> > 收件人: devel@edk2.groups.io; Kinney, Michael D
> > <michael.d.kinney@intel.com>
> > 抄送: Sean Brogan <sean.brogan@microsoft.com>; Bret Barkelew
> > <Bret.Barkelew@microsoft.com>; Liming Gao <gaoliming@byosoft.com.cn>;
> > Justen, Jordan L <jordan.l.justen@intel.com>; Laszlo Ersek
> > <lersek@redhat.com>; Ard Biesheuvel <ard.biesheuvel@arm.com>; Andrew
> > Fish <afish@apple.com>; Leif Lindholm <leif@nuviainc.com>
> > 主题: Re: [edk2-devel] [Patch][edk2-stable202011 0/3] Address EDK II CI
> > Failures
> >
> > A test PR was sent through CI with these changes.
> >
> >     https://github.com/tianocore/edk2/pull/1143
> >
> > All checks passed and the PR was auto closed.
> >
> > This does not address the 2nd issue related to the CryptoPkg
> > unit tests for MD5.  These are not run for a PR unless there are
> > changes to the CryptoPkg in the PR.  The post commit checks
> > will run when this commit is made and the CryptoPkg unit tests
> > are expected to fail.
> >
> > The CryptoPkg unit test fix is still being investigated.
> >
> > Mike
> >
> > > -----Original Message-----
> > > From: devel@edk2.groups.io <devel@edk2.groups.io> On Behalf Of Michael
> > D Kinney
> > > Sent: Tuesday, November 24, 2020 6:12 PM
> > > To: devel@edk2.groups.io
> > > Cc: Sean Brogan <sean.brogan@microsoft.com>; Bret Barkelew
> > <Bret.Barkelew@microsoft.com>; Liming Gao
> > > <gaoliming@byosoft.com.cn>; Justen, Jordan L <jordan.l.justen@intel.com>;
> > Laszlo Ersek <lersek@redhat.com>; Ard Biesheuvel
> > > <ard.biesheuvel@arm.com>; Andrew Fish <afish@apple.com>; Leif
> > Lindholm <leif@nuviainc.com>
> > > Subject: [edk2-devel] [Patch][edk2-stable202011 0/3] Address EDK II CI
> > Failures
> > >
> > > * Always fetch the target branch of the PR (usually origin/master)
> > >   so diffs between PR and the target branch can be performed.  There
> > >   is no guarantee that the target branch is fetched when a PR is
> > >   evaluated by a CI agent.
> > > * QEMU release for Windows from Nov 20, 2020 is installed into
> > >   wrong directory.  Use previous QEMU for Windows release from
> > >   Aug 14, 2020.
> > >
> > > Cc: Sean Brogan <sean.brogan@microsoft.com>
> > > Cc: Bret Barkelew <Bret.Barkelew@microsoft.com>
> > > Cc: Liming Gao <gaoliming@byosoft.com.cn>
> > > Cc: Jordan Justen <jordan.l.justen@intel.com>
> > > Cc: Laszlo Ersek <lersek@redhat.com>
> > > Cc: Ard Biesheuvel <ard.biesheuvel@arm.com>
> > > Cc: Andrew Fish <afish@apple.com>
> > > Cc: Leif Lindholm <leif@nuviainc.com>
> > > Signed-off-by: Michael D Kinney <michael.d.kinney@intel.com>
> > >
> > > Michael D Kinney (1):
> > >   OvmfPkg/PlatformCI/.azurepiplines: Use stable release of QEMU
> > >
> > > Sean Brogan (2):
> > >   .azurepipelines/templates/pr-gate-steps.yml: Fetch target branch
> > >   .azurepipelines/templates/platform-build-run-steps.yml: Fetch target
> > >     branch
> > >
> > >  .azurepipelines/templates/platform-build-run-steps.yml | 6 ++++++
> > >  .azurepipelines/templates/pr-gate-steps.yml            | 6 ++++++
> > >  OvmfPkg/PlatformCI/.azurepipelines/Windows-VS2019.yml  | 2 +-
> > >  3 files changed, 13 insertions(+), 1 deletion(-)
> > >
> > > --
> > > 2.29.2.windows.2
> > >
> > >
> > >
> > >
> > >
> >
> >
> >
> > 
> >
> 
> 

Re: [Patch][edk2-stable202011 3/3] OvmfPkg/PlatformCI/.azurepiplines: Use stable release of QEMU

[Laszlo Ersek]

On 11/25/20 03:11, Michael D Kinney wrote:
> Install the 2020.08.14 release of QEMU for Windows.
> 
> The QEMU release from 2020.11.20 is installed into the incorrect
> directory and is causing EDK II CI failures in the run to shell
> step.
> 
> Cc: Sean Brogan <sean.brogan@microsoft.com>
> Cc: Bret Barkelew <Bret.Barkelew@microsoft.com>
> Cc: Liming Gao <gaoliming@byosoft.com.cn>
> Cc: Jordan Justen <jordan.l.justen@intel.com>
> Cc: Laszlo Ersek <lersek@redhat.com>
> Cc: Ard Biesheuvel <ard.biesheuvel@arm.com>
> Signed-off-by: Michael D Kinney <michael.d.kinney@intel.com>
> ---
>  OvmfPkg/PlatformCI/.azurepipelines/Windows-VS2019.yml | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/OvmfPkg/PlatformCI/.azurepipelines/Windows-VS2019.yml b/OvmfPkg/PlatformCI/.azurepipelines/Windows-VS2019.yml
> index 02ed233fdb60..dd19fff50066 100644
> --- a/OvmfPkg/PlatformCI/.azurepipelines/Windows-VS2019.yml
> +++ b/OvmfPkg/PlatformCI/.azurepipelines/Windows-VS2019.yml
> @@ -132,7 +132,7 @@ jobs:
>          build_flags: $(Build.Flags)
>          run_flags: $(Run.Flags)
>          extra_install_step:
> -        - powershell: choco install qemu; Write-Host "##vso[task.prependpath]c:\Program Files\qemu"
> +        - powershell: choco install qemu --version=2020.08.14; Write-Host "##vso[task.prependpath]c:\Program Files\qemu"
>            displayName: Install QEMU and Set QEMU on path # friendly name displayed in the UI
>            condition: and(gt(variables.pkg_count, 0), succeeded())
>  
> 

Acked-by: Laszlo Ersek <lersek@redhat.com>

I agree this is eligible for edk2-stable202011.

Thanks
Laszlo