[PATCH v2 0/2] Enable Initial CodeQL Support

[PATCH v2 0/2] Enable Initial CodeQL Support

[Michael Kubacki]

From: Michael Kubacki <michael.kubacki@microsoft.com>

REF:https://bugzilla.tianocore.org/show_bug.cgi?id=4115

This patch series enables initial CodeQL support within the edk2
repository.

After this patch, a CodeQL Analyze step will run on a subset of
edk2 packages in Pull Requests. The plan to scale additional CodeQL
queries is documented in the RFC and it is recommeneded to read
that for more detail:

https://github.com/tianocore/edk2/discussions/3258#discussioncomment-3682099

In summary, this change will:

- Main workflow file - .github/workflows/codeql-analysis.yml

1. Add a code scanning workflow with the CodeQL Analysis GitHub action
2. Build packages on Windows + VS2019
3. Perform CodeQL analysis on the build results

- Custom configuration file - .github/codeql/codeql-config.yml

1. Specify the edk2 CodeQL query set
2. Apply a query filter to exclude errors, warnings, and
   recommendations

- edk2 query set file - .github/codeql/edk2.qls

1. Enable a single query: cpp/conditionallyuninitializedvariable

Per the RFC, this enables CodeQL but does not allow any alerts
by suppressing all of the severity levels.

When the code changes necessary to resolve problems found with
cpp/conditionallyuninitializedvariable are checked in, the severity
filter can be adjusted such that query is enabled.

V2 Changes:

1. Switch from Ubuntu + GCC5

As noted in the v1 series, there is an occassional issue with
filesystem paths on Ubuntu + GCC at the moment that prevents this
change from being checked in. A bug has been filed against CodeQL
and  the CodeQL team has confirmed this is a bug. They were not
aware of the bug until this report.

https://github.com/github/codeql-action/issues/1338

In the meantime, this v2 patch series places the workflow on
Windows with VS2019 which has been reliable across testing.

Cc: Andrew Fish <afish@apple.com>
Cc: Leif Lindholm <quic_llindhol@quicinc.com>
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Cc: Michael D Kinney <michael.d.kinney@intel.com>
Cc: Sean Brogan <sean.brogan@microsoft.com>
Signed-off-by: Michael Kubacki <michael.kubacki@microsoft.com>

Michael Kubacki (2):
  Maintainers.txt: Add .github maintainers and reviewers
  .github: Add initial CodeQL config and workflow files

 .github/codeql/codeql-config.yml      | 30 ++++++
 .github/codeql/edk2.qls               | 12 +++
 .github/workflows/codeql-analysis.yml | 99 ++++++++++++++++++++
 Maintainers.txt                       |  6 ++
 4 files changed, 147 insertions(+)
 create mode 100644 .github/codeql/codeql-config.yml
 create mode 100644 .github/codeql/edk2.qls
 create mode 100644 .github/workflows/codeql-analysis.yml

-- 
2.28.0.windows.1

[PATCH v2 1/2] Maintainers.txt: Add .github maintainers and reviewers

[Michael Kubacki]

From: Michael Kubacki <michael.kubacki@microsoft.com>

Adds the maintainers and reviewers for the new .github directory
being added to hold GitHub workflows files.

Cc: Andrew Fish <afish@apple.com>
Cc: Leif Lindholm <quic_llindhol@quicinc.com>
Cc: Michael D Kinney <michael.d.kinney@intel.com>
Cc: Sean Brogan <sean.brogan@microsoft.com>
Signed-off-by: Michael Kubacki <michael.kubacki@microsoft.com>
---
 Maintainers.txt | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/Maintainers.txt b/Maintainers.txt
index 889990fa566f..a2727fa7bd42 100644
--- a/Maintainers.txt
+++ b/Maintainers.txt
@@ -117,6 +117,12 @@ M: Michael Kubacki <mikuback@linux.microsoft.com> [makubacki]
 R: Michael D Kinney <michael.d.kinney@intel.com> [mdkinney]
 R: Liming Gao <gaoliming@byosoft.com.cn> [lgao4]
 
+.github/
+F: .github/
+M: Sean Brogan <sean.brogan@microsoft.com> [spbrogan]
+M: Michael Kubacki <mikuback@linux.microsoft.com> [makubacki]
+R: Michael D Kinney <michael.d.kinney@intel.com> [mdkinney]
+
 .mergify/
 F: .mergify/
 M: Michael D Kinney <michael.d.kinney@intel.com> [mdkinney]
-- 
2.28.0.windows.1

Re: [edk2-devel] [PATCH v2 1/2] Maintainers.txt: Add .github maintainers and reviewers

[Michael D Kinney]

Reviewed-by: Michael D Kinney <michael.d.kinney@intel.com>

> -----Original Message-----
> From: devel@edk2.groups.io <devel@edk2.groups.io> On Behalf Of Michael Kubacki
> Sent: Thursday, November 3, 2022 2:40 PM
> To: devel@edk2.groups.io
> Cc: Andrew Fish <afish@apple.com>; Leif Lindholm <quic_llindhol@quicinc.com>; Kinney, Michael D <michael.d.kinney@intel.com>;
> Sean Brogan <sean.brogan@microsoft.com>
> Subject: [edk2-devel] [PATCH v2 1/2] Maintainers.txt: Add .github maintainers and reviewers
> 
> From: Michael Kubacki <michael.kubacki@microsoft.com>
> 
> Adds the maintainers and reviewers for the new .github directory
> being added to hold GitHub workflows files.
> 
> Cc: Andrew Fish <afish@apple.com>
> Cc: Leif Lindholm <quic_llindhol@quicinc.com>
> Cc: Michael D Kinney <michael.d.kinney@intel.com>
> Cc: Sean Brogan <sean.brogan@microsoft.com>
> Signed-off-by: Michael Kubacki <michael.kubacki@microsoft.com>
> ---
>  Maintainers.txt | 6 ++++++
>  1 file changed, 6 insertions(+)
> 
> diff --git a/Maintainers.txt b/Maintainers.txt
> index 889990fa566f..a2727fa7bd42 100644
> --- a/Maintainers.txt
> +++ b/Maintainers.txt
> @@ -117,6 +117,12 @@ M: Michael Kubacki <mikuback@linux.microsoft.com> [makubacki]
>  R: Michael D Kinney <michael.d.kinney@intel.com> [mdkinney]
>  R: Liming Gao <gaoliming@byosoft.com.cn> [lgao4]
> 
> +.github/
> +F: .github/
> +M: Sean Brogan <sean.brogan@microsoft.com> [spbrogan]
> +M: Michael Kubacki <mikuback@linux.microsoft.com> [makubacki]
> +R: Michael D Kinney <michael.d.kinney@intel.com> [mdkinney]
> +
>  .mergify/
>  F: .mergify/
>  M: Michael D Kinney <michael.d.kinney@intel.com> [mdkinney]
> --
> 2.28.0.windows.1
> 
> 
> 
> -=-=-=-=-=-=
> Groups.io Links: You receive all messages sent to this group.
> View/Reply Online (#95907): https://edk2.groups.io/g/devel/message/95907
> Mute This Topic: https://groups.io/mt/94793993/1643496
> Group Owner: devel+owner@edk2.groups.io
> Unsubscribe: https://edk2.groups.io/g/devel/unsub [michael.d.kinney@intel.com]
> -=-=-=-=-=-=
> 

[PATCH v2 2/2] .github: Add initial CodeQL config and workflow files

[Michael Kubacki]

From: Michael Kubacki <michael.kubacki@microsoft.com>

REF:https://bugzilla.tianocore.org/show_bug.cgi?id=4115

Adds initial support for enabling CodeQL Code Scanning in this
repository per the RFC:

  https://github.com/tianocore/edk2/discussions/3258

Adds the following new files:
  - .github/workflows/codql-analysis.yml - The main GitHub workflow
    file used to setup CodeQL in the repo.
  - .github/codeql/codeql-config.yml - The main CodeQL configuration
    file used to customize the queries and other resources the repo
    is using for CodeQL.

Cc: Sean Brogan <sean.brogan@microsoft.com>
Cc: Michael D Kinney <michael.d.kinney@intel.com>
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Signed-off-by: Michael Kubacki <michael.kubacki@microsoft.com>
---
 .github/codeql/codeql-config.yml      | 30 ++++++
 .github/codeql/edk2.qls               | 12 +++
 .github/workflows/codeql-analysis.yml | 99 ++++++++++++++++++++
 3 files changed, 141 insertions(+)

diff --git a/.github/codeql/codeql-config.yml b/.github/codeql/codeql-config.yml
new file mode 100644
index 000000000000..3e27c2fb0d28
--- /dev/null
+++ b/.github/codeql/codeql-config.yml
@@ -0,0 +1,30 @@
+## @file
+# CodeQL configuration file for edk2.
+#
+# Copyright (c) Microsoft Corporation.
+# SPDX-License-Identifier: BSD-2-Clause-Patent
+##
+
+name: "CodeQL config"
+
+# The following line disables the default queries. This is used because we want to enable on query at a time by
+# explicitly specifying each query in a "queries" array as they are enabled.
+#
+# See the following for more information about adding custom queries:
+# https://docs.github.com/en/code-security/code-scanning/automatically-scanning-your-code-for-vulnerabilities-and-errors/configuring-code-scanning#using-a-custom-configuration-file
+
+#disable-default-queries: true
+
+queries:
+  - name: EDK2 CodeQL Query List
+    uses: ./.github/codeql/edk2.qls
+
+# We must specify a query for CodeQL to run. Until the first query is enabled, enable the security query suite but
+# exclude all problem levels from impacting the results. After the first query is enabled, this filter can be relaxed
+# to find the level of problems desired from the query.
+query-filters:
+- exclude:
+    problem.severity:
+      - error
+      - warning
+      - recommendation
diff --git a/.github/codeql/edk2.qls b/.github/codeql/edk2.qls
new file mode 100644
index 000000000000..0efc7dca52db
--- /dev/null
+++ b/.github/codeql/edk2.qls
@@ -0,0 +1,12 @@
+---
+- description: EDK2 (C++) queries
+
+# Bring in all queries from the official cpp-queries suite so individual queries can be explicitly enabled.
+
+- queries: '.'
+  from: codeql/cpp-queries
+
+# Enable individual queries below.
+
+- include:
+    id: cpp/conditionallyuninitializedvariable
diff --git a/.github/workflows/codeql-analysis.yml b/.github/workflows/codeql-analysis.yml
new file mode 100644
index 000000000000..4ab8be04ecbe
--- /dev/null
+++ b/.github/workflows/codeql-analysis.yml
@@ -0,0 +1,99 @@
+# @file
+# GitHub Workflow for CodeQL Analysis
+#
+# Copyright (c) Microsoft Corporation.
+#
+# SPDX-License-Identifier: BSD-2-Clause-Patent
+##
+
+name: "CodeQL"
+
+on:
+  push:
+    branches:
+      - master
+  pull_request:
+    branches:
+      - master
+    paths-ignore:
+      - '**/*.bat'
+      - '**/*.md'
+      - '**/*.py'
+      - '**/*.rst'
+      - '**/*.sh'
+      - '**/*.txt'
+
+  schedule:
+    # https://crontab.guru/#20_23_*_*_4
+    - cron: '20 23 * * 4'
+
+jobs:
+  analyze:
+    name: Analyze
+    runs-on: windows-2019
+    permissions:
+      actions: read
+      contents: read
+      security-events: write
+
+    strategy:
+      fail-fast: false
+      matrix:
+        package: [
+          "ArmPkg",
+          "CryptoPkg",
+          "DynamicTablesPkg",
+          "FatPkg",
+          "FmpDevicePkg",
+          "IntelFsp2Pkg",
+          "IntelFsp2WrapperPkg",
+          "MdeModulePkg",
+          "MdePkg",
+          "PcAtChipsetPkg",
+          "PrmPkg",
+          "SecurityPkg",
+          "ShellPkg",
+          "SourceLevelDebugPkg",
+          "StandaloneMmPkg",
+          "UefiCpuPkg",
+          "UnitTestFrameworkPkg"]
+
+    steps:
+    - name: Checkout repository
+      uses: actions/checkout@v3
+
+    # Initializes the CodeQL tools for scanning.
+    - name: Initialize CodeQL
+      uses: github/codeql-action/init@v2
+      with:
+        languages: 'cpp'
+        # CodeQL supports [ 'cpp', 'csharp', 'go', 'java', 'javascript', 'python', 'ruby' ]
+        # Learn more about CodeQL language support at https://codeql.github.com/docs/codeql-overview/supported-languages-and-frameworks/
+        config-file: ./.github/codeql/codeql-config.yml
+        # Note: Add new queries to codeql-config.yml file as they are enabled.
+
+    - name: Install/Upgrade pip Modules
+      run: pip install -r pip-requirements.txt --upgrade
+
+    - name: Use Node.js 19.x
+      uses: actions/setup-node@v3
+      with:
+          node-version: 19.x
+
+    - name: Install cspell npm
+      run: npm install -g cspell@5.20.0
+
+    - name: Setup
+      run: stuart_setup -c .pytool/CISettings.py -t DEBUG -a IA32,X64 TOOL_CHAIN_TAG=VS2019
+
+    - name: Update
+      run: stuart_update -c .pytool/CISettings.py -t DEBUG -a IA32,X64 TOOL_CHAIN_TAG=VS2019
+
+    - name: Build Tools From Source
+      run: python BaseTools/Edk2ToolsBuild.py -t VS2019
+
+    - name: CI Build
+      run: stuart_ci_build -c .pytool/CISettings.py -p ${{ matrix.package }} -t DEBUG -a IA32,X64 TOOL_CHAIN_TAG=VS2019
+
+    - name: Perform CodeQL Analysis
+      uses: github/codeql-action/analyze@v2
-- 
2.28.0.windows.1

Re: [PATCH v2 2/2] .github: Add initial CodeQL config and workflow files

[Michael D Kinney]

Glad to see this works on Windows agents.  I know it will be good to switch back to Linux agents when stable.

See comments below.

Mike

> -----Original Message-----
> From: mikuback@linux.microsoft.com <mikuback@linux.microsoft.com>
> Sent: Thursday, November 3, 2022 2:41 PM
> To: devel@edk2.groups.io
> Cc: Sean Brogan <sean.brogan@microsoft.com>; Kinney, Michael D <michael.d.kinney@intel.com>; Gao, Liming
> <gaoliming@byosoft.com.cn>
> Subject: [PATCH v2 2/2] .github: Add initial CodeQL config and workflow files
> 
> From: Michael Kubacki <michael.kubacki@microsoft.com>
> 
> REF:https://bugzilla.tianocore.org/show_bug.cgi?id=4115
> 
> Adds initial support for enabling CodeQL Code Scanning in this
> repository per the RFC:
> 
>   https://github.com/tianocore/edk2/discussions/3258
> 
> Adds the following new files:
>   - .github/workflows/codql-analysis.yml - The main GitHub workflow
>     file used to setup CodeQL in the repo.
>   - .github/codeql/codeql-config.yml - The main CodeQL configuration
>     file used to customize the queries and other resources the repo
>     is using for CodeQL.
> 
> Cc: Sean Brogan <sean.brogan@microsoft.com>
> Cc: Michael D Kinney <michael.d.kinney@intel.com>
> Cc: Liming Gao <gaoliming@byosoft.com.cn>
> Signed-off-by: Michael Kubacki <michael.kubacki@microsoft.com>
> ---
>  .github/codeql/codeql-config.yml      | 30 ++++++
>  .github/codeql/edk2.qls               | 12 +++
>  .github/workflows/codeql-analysis.yml | 99 ++++++++++++++++++++
>  3 files changed, 141 insertions(+)
> 
> diff --git a/.github/codeql/codeql-config.yml b/.github/codeql/codeql-config.yml
> new file mode 100644
> index 000000000000..3e27c2fb0d28
> --- /dev/null
> +++ b/.github/codeql/codeql-config.yml
> @@ -0,0 +1,30 @@
> +## @file
> +# CodeQL configuration file for edk2.
> +#
> +# Copyright (c) Microsoft Corporation.
> +# SPDX-License-Identifier: BSD-2-Clause-Patent
> +##
> +
> +name: "CodeQL config"
> +
> +# The following line disables the default queries. This is used because we want to enable on query at a time by
> +# explicitly specifying each query in a "queries" array as they are enabled.
> +#
> +# See the following for more information about adding custom queries:
> +# https://docs.github.com/en/code-security/code-scanning/automatically-scanning-your-code-for-vulnerabilities-and-
> errors/configuring-code-scanning#using-a-custom-configuration-file
> +
> +#disable-default-queries: true
> +
> +queries:
> +  - name: EDK2 CodeQL Query List
> +    uses: ./.github/codeql/edk2.qls
> +
> +# We must specify a query for CodeQL to run. Until the first query is enabled, enable the security query suite but
> +# exclude all problem levels from impacting the results. After the first query is enabled, this filter can be relaxed
> +# to find the level of problems desired from the query.
> +query-filters:
> +- exclude:
> +    problem.severity:
> +      - error
> +      - warning
> +      - recommendation
> diff --git a/.github/codeql/edk2.qls b/.github/codeql/edk2.qls
> new file mode 100644
> index 000000000000..0efc7dca52db
> --- /dev/null
> +++ b/.github/codeql/edk2.qls
> @@ -0,0 +1,12 @@
> +---
> +- description: EDK2 (C++) queries
> +
> +# Bring in all queries from the official cpp-queries suite so individual queries can be explicitly enabled.
> +
> +- queries: '.'
> +  from: codeql/cpp-queries
> +
> +# Enable individual queries below.
> +
> +- include:
> +    id: cpp/conditionallyuninitializedvariable
> diff --git a/.github/workflows/codeql-analysis.yml b/.github/workflows/codeql-analysis.yml
> new file mode 100644
> index 000000000000..4ab8be04ecbe
> --- /dev/null
> +++ b/.github/workflows/codeql-analysis.yml
> @@ -0,0 +1,99 @@
> +# @file
> +# GitHub Workflow for CodeQL Analysis
> +#
> +# Copyright (c) Microsoft Corporation.
> +#
> +# SPDX-License-Identifier: BSD-2-Clause-Patent
> +##
> +
> +name: "CodeQL"
> +
> +on:
> +  push:
> +    branches:
> +      - master
> +  pull_request:
> +    branches:
> +      - master
> +    paths-ignore:
> +      - '**/*.bat'
> +      - '**/*.md'
> +      - '**/*.py'
> +      - '**/*.rst'
> +      - '**/*.sh'
> +      - '**/*.txt'
> +
> +  schedule:
> +    # https://crontab.guru/#20_23_*_*_4
> +    - cron: '20 23 * * 4'
> +
> +jobs:
> +  analyze:
> +    name: Analyze
> +    runs-on: windows-2019
> +    permissions:
> +      actions: read
> +      contents: read
> +      security-events: write
> +
> +    strategy:
> +      fail-fast: false
> +      matrix:
> +        package: [
> +          "ArmPkg",
> +          "CryptoPkg",
> +          "DynamicTablesPkg",
> +          "FatPkg",
> +          "FmpDevicePkg",
> +          "IntelFsp2Pkg",
> +          "IntelFsp2WrapperPkg",
> +          "MdeModulePkg",
> +          "MdePkg",
> +          "PcAtChipsetPkg",
> +          "PrmPkg",
> +          "SecurityPkg",
> +          "ShellPkg",
> +          "SourceLevelDebugPkg",
> +          "StandaloneMmPkg",
> +          "UefiCpuPkg",
> +          "UnitTestFrameworkPkg"]
> +
> +    steps:
> +    - name: Checkout repository
> +      uses: actions/checkout@v3
> +
> +    # Initializes the CodeQL tools for scanning.
> +    - name: Initialize CodeQL
> +      uses: github/codeql-action/init@v2
> +      with:
> +        languages: 'cpp'
> +        # CodeQL supports [ 'cpp', 'csharp', 'go', 'java', 'javascript', 'python', 'ruby' ]
> +        # Learn more about CodeQL language support at https://codeql.github.com/docs/codeql-overview/supported-languages-and-
> frameworks/
> +        config-file: ./.github/codeql/codeql-config.yml
> +        # Note: Add new queries to codeql-config.yml file as they are enabled.
> +
> +    - name: Install/Upgrade pip Modules
> +      run: pip install -r pip-requirements.txt --upgrade
> +
> +    - name: Use Node.js 19.x
> +      uses: actions/setup-node@v3
> +      with:
> +          node-version: 19.x
> +

Is this only required for cspell?

> +    - name: Install cspell npm
> +      run: npm install -g cspell@5.20.0

Do you have to install cspell to run CodeQL analysis?

> +
> +    - name: Setup
> +      run: stuart_setup -c .pytool/CISettings.py -t DEBUG -a IA32,X64 TOOL_CHAIN_TAG=VS2019
> +
> +    - name: Update
> +      run: stuart_update -c .pytool/CISettings.py -t DEBUG -a IA32,X64 TOOL_CHAIN_TAG=VS2019
> +
> +    - name: Build Tools From Source
> +      run: python BaseTools/Edk2ToolsBuild.py -t VS2019
> +
> +    - name: CI Build
> +      run: stuart_ci_build -c .pytool/CISettings.py -p ${{ matrix.package }} -t DEBUG -a IA32,X64 TOOL_CHAIN_TAG=VS2019
> +
> +    - name: Perform CodeQL Analysis
> +      uses: github/codeql-action/analyze@v2
> --
> 2.28.0.windows.1

Re: [edk2-devel] [PATCH v2 2/2] .github: Add initial CodeQL config and workflow files

[Michael Kubacki]

[-- Attachment #1: Type: text/plain, Size: 385 bytes --]

Hi Mike,

Node.js and cspell are not needed. I confirmed the GitHub workflow with them removed here:
Enable CodeQL · tianocore/edk2@ad62416 (github.com) ( https://github.com/tianocore/edk2/actions/runs/3390100498 )

v3 has been sent with that change:
[PATCH v3 0/2] Enable Initial CodeQL Support (groups.io) ( https://edk2.groups.io/g/devel/message/95927 )

Thanks,
Michael

[-- Attachment #2: Type: text/html, Size: 447 bytes --]