Split up MdeModulePkg and CryptoPkg CodeQL CI checks

Split up MdeModulePkg and CryptoPkg CodeQL CI checks

[Michael D Kinney]

Hi Michael,

I am looking at the CI agent execution times since we resolved 
several issued the last few days.

We have CodeQL running in GitHub Actions and the rest of the
Checks running in Azure Pipelines.

Looking at only CodeQL, I see that the 2 longest jobs are
on the MdeModulePkg and CryptoPkg and they build for both
IA32 and X64 to get 32-bit and 64-bit coverage.

* stuart_ci_build -c .pytool/CISettings.py -p CryptoPkg -t DEBUG -a IA32,X64 TOOL_CHAIN_TAG=VS2019
* stuart_ci_build -c .pytool/CISettings.py -p MdeModulePkg -t DEBUG -a IA32,X64 TOOL_CHAIN_TAG=VS2019

Is it possible to split these two jobs into 4 jobs to check
IA32 and X64 independently?

* stuart_ci_build -c .pytool/CISettings.py -p CryptoPkg -t DEBUG -a IA32 TOOL_CHAIN_TAG=VS2019
* stuart_ci_build -c .pytool/CISettings.py -p CryptoPkg -t DEBUG -a X64 TOOL_CHAIN_TAG=VS2019
* stuart_ci_build -c .pytool/CISettings.py -p MdeModulePkg -t DEBUG -a IA32 TOOL_CHAIN_TAG=VS2019
* stuart_ci_build -c .pytool/CISettings.py -p MdeModulePkg -t DEBUG -a X64 TOOL_CHAIN_TAG=VS2019

Thanks,

Mike

Re: Split up MdeModulePkg and CryptoPkg CodeQL CI checks

[Michael D Kinney]

Hi Michael,

On the Azure Piplines side, the same 2 packages have the longest
build time.  The DEBUG and RELEASE builds take about the same
time each.  The NO-TARGET and NOOPT combined are less than 
DEBUG or RELEASE.

stuart_ci_build -c .pytool/CISettings.py -p CryptoPkg -t DEBUG,RELEASE,NO-TARGET,NOOPT -a IA32,X64,ARM,AARCH64,RISCV64,LOONGARCH64 TOOL_CHAIN_TAG=GCC5

Could we split the CryptoPkg and MdeModulePkg jobs into 3 jobs each?
* DEBUG
* RELEASE
* NO-TARGET, NOOPT

Alternatively, split into 2 jobs

* DEBUG, NO-TARGET
* RELEASE, NOOPT

Thanks,

Mike


> -----Original Message-----
> From: Kinney, Michael D <michael.d.kinney@intel.com>
> Sent: Saturday, April 1, 2023 3:31 PM
> To: devel@edk2.groups.io; Michael Kubacki <mikuback@linux.microsoft.com>
> Cc: Kinney, Michael D <michael.d.kinney@intel.com>
> Subject: Split up MdeModulePkg and CryptoPkg CodeQL CI checks
> 
> Hi Michael,
> 
> I am looking at the CI agent execution times since we resolved
> several issued the last few days.
> 
> We have CodeQL running in GitHub Actions and the rest of the
> Checks running in Azure Pipelines.
> 
> Looking at only CodeQL, I see that the 2 longest jobs are
> on the MdeModulePkg and CryptoPkg and they build for both
> IA32 and X64 to get 32-bit and 64-bit coverage.
> 
> * stuart_ci_build -c .pytool/CISettings.py -p CryptoPkg -t DEBUG -a IA32,X64 TOOL_CHAIN_TAG=VS2019
> * stuart_ci_build -c .pytool/CISettings.py -p MdeModulePkg -t DEBUG -a IA32,X64 TOOL_CHAIN_TAG=VS2019
> 
> Is it possible to split these two jobs into 4 jobs to check
> IA32 and X64 independently?
> 
> * stuart_ci_build -c .pytool/CISettings.py -p CryptoPkg -t DEBUG -a IA32 TOOL_CHAIN_TAG=VS2019
> * stuart_ci_build -c .pytool/CISettings.py -p CryptoPkg -t DEBUG -a X64 TOOL_CHAIN_TAG=VS2019
> * stuart_ci_build -c .pytool/CISettings.py -p MdeModulePkg -t DEBUG -a IA32 TOOL_CHAIN_TAG=VS2019
> * stuart_ci_build -c .pytool/CISettings.py -p MdeModulePkg -t DEBUG -a X64 TOOL_CHAIN_TAG=VS2019
> 
> Thanks,
> 
> Mike
> 

Re: Split up MdeModulePkg and CryptoPkg CodeQL CI checks

[Michael Kubacki]

Hi Mike,

The changes in this PR look reasonable 
https://github.com/tianocore/edk2/pull/4233 if you'd like to send a patch.

Thanks,
Michael

On 4/1/2023 6:53 PM, Kinney, Michael D wrote:
> Hi Michael,
> 
> On the Azure Piplines side, the same 2 packages have the longest
> build time.  The DEBUG and RELEASE builds take about the same
> time each.  The NO-TARGET and NOOPT combined are less than
> DEBUG or RELEASE.
> 
> stuart_ci_build -c .pytool/CISettings.py -p CryptoPkg -t DEBUG,RELEASE,NO-TARGET,NOOPT -a IA32,X64,ARM,AARCH64,RISCV64,LOONGARCH64 TOOL_CHAIN_TAG=GCC5
> 
> Could we split the CryptoPkg and MdeModulePkg jobs into 3 jobs each?
> * DEBUG
> * RELEASE
> * NO-TARGET, NOOPT
> 
> Alternatively, split into 2 jobs
> 
> * DEBUG, NO-TARGET
> * RELEASE, NOOPT
> 
> Thanks,
> 
> Mike
> 
> 
>> -----Original Message-----
>> From: Kinney, Michael D <michael.d.kinney@intel.com>
>> Sent: Saturday, April 1, 2023 3:31 PM
>> To: devel@edk2.groups.io; Michael Kubacki <mikuback@linux.microsoft.com>
>> Cc: Kinney, Michael D <michael.d.kinney@intel.com>
>> Subject: Split up MdeModulePkg and CryptoPkg CodeQL CI checks
>>
>> Hi Michael,
>>
>> I am looking at the CI agent execution times since we resolved
>> several issued the last few days.
>>
>> We have CodeQL running in GitHub Actions and the rest of the
>> Checks running in Azure Pipelines.
>>
>> Looking at only CodeQL, I see that the 2 longest jobs are
>> on the MdeModulePkg and CryptoPkg and they build for both
>> IA32 and X64 to get 32-bit and 64-bit coverage.
>>
>> * stuart_ci_build -c .pytool/CISettings.py -p CryptoPkg -t DEBUG -a IA32,X64 TOOL_CHAIN_TAG=VS2019
>> * stuart_ci_build -c .pytool/CISettings.py -p MdeModulePkg -t DEBUG -a IA32,X64 TOOL_CHAIN_TAG=VS2019
>>
>> Is it possible to split these two jobs into 4 jobs to check
>> IA32 and X64 independently?
>>
>> * stuart_ci_build -c .pytool/CISettings.py -p CryptoPkg -t DEBUG -a IA32 TOOL_CHAIN_TAG=VS2019
>> * stuart_ci_build -c .pytool/CISettings.py -p CryptoPkg -t DEBUG -a X64 TOOL_CHAIN_TAG=VS2019
>> * stuart_ci_build -c .pytool/CISettings.py -p MdeModulePkg -t DEBUG -a IA32 TOOL_CHAIN_TAG=VS2019
>> * stuart_ci_build -c .pytool/CISettings.py -p MdeModulePkg -t DEBUG -a X64 TOOL_CHAIN_TAG=VS2019
>>
>> Thanks,
>>
>> Mike
>>
>