[edk2-devel] [PATCH v1 1/1] .github/workflows/codeql.yml: Add emacs output

[edk2-devel] [PATCH v1 1/1] .github/workflows/codeql.yml: Add emacs output

[Michael Kubacki]

From: Michael Kubacki <michael.kubacki@microsoft.com>

Updates the workflow to also output files that can be loaded in emacs
to show CodeQL issues (in addition to the existing SARIF output for
standard SARIF viewers).

The emacs files are in the SARIF zip file attached to each "CodeQL"
run (https://github.com/tianocore/edk2/actions/workflows/codeql.yml).

The file name ends with "-emacs.txt". An MdePkg example:
  "codeql-db-mdepkg-debug-0-emacs.txt".

Cc: Joey Vagedes <joey.vagedes@gmail.com>
Cc: Laszlo Ersek <lersek@redhat.com>
Cc: Michael D Kinney <michael.d.kinney@intel.com>
Cc: Sean Brogan <sean.brogan@microsoft.com>
Signed-off-by: Michael Kubacki <michael.kubacki@microsoft.com>
---

Notes:
    An example CodeQL run with this change:
    
    https://github.com/tianocore/edk2/actions/runs/7035482184

 .github/workflows/codeql.yml | 20 ++++++++++++++++----
 1 file changed, 16 insertions(+), 4 deletions(-)

diff --git a/.github/workflows/codeql.yml b/.github/workflows/codeql.yml
index 72ece9dcb446..e826e67eb912 100644
--- a/.github/workflows/codeql.yml
+++ b/.github/workflows/codeql.yml
@@ -92,7 +92,7 @@ jobs:
         git config --system core.longpaths true
 
     - name: Install/Upgrade pip Modules
-      run: pip install -r pip-requirements.txt --upgrade requests
+      run: pip install -r pip-requirements.txt --upgrade requests sarif-tools
 
     - name: Determine CI Settings File Supported Operations
       id: get_ci_file_operations
@@ -304,16 +304,26 @@ jobs:
         PACKAGE_NAME: ${{ matrix.Package }}
       shell: python
       run: |
+        import logging
         import os
+        from edk2toollib.utility_functions import RunCmd
+        from io import StringIO
+        from pathlib import Path
 
         package = os.environ['PACKAGE_NAME'].strip().lower()
         directory_name = 'codeql-analysis-' + package + '-debug'
         file_name = 'codeql-db-' + package + '-debug-0.sarif'
-        sarif_path = os.path.join('Build', directory_name, file_name)
+        sarif_path = Path('Build', directory_name, file_name)
 
         with open(os.environ['GITHUB_OUTPUT'], 'a') as fh:
-            if os.path.isfile(sarif_path):
+            if sarif_path.is_file():
+                emacs_file_path = sarif_path.with_name(sarif_path.stem + "-emacs.txt")
+                out_stream_buffer = StringIO()
+                exit_code = RunCmd("sarif", f"emacs {sarif_path} --output {emacs_file_path}",
+                                   outstream=out_stream_buffer,
+                                   logging_level=logging.NOTSET)
                 print(f'upload_sarif_file=true', file=fh)
+                print(f'emacs_file_path={emacs_file_path}', file=fh)
                 print(f'sarif_file_path={sarif_path}', file=fh)
             else:
                 print(f'upload_sarif_file=false', file=fh)
@@ -323,7 +333,9 @@ jobs:
       if: steps.env_data.outputs.upload_sarif_file == 'true'
       with:
         name: ${{ matrix.Package }}-CodeQL-SARIF
-        path: ${{ steps.env_data.outputs.sarif_file_path }}
+        path: |
+          ${{ steps.env_data.outputs.emacs_file_path }}
+          ${{ steps.env_data.outputs.sarif_file_path }}
         retention-days: 14
         if-no-files-found: warn
 
-- 
2.43.0.windows.1



-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.
View/Reply Online (#111852): https://edk2.groups.io/g/devel/message/111852
Mute This Topic: https://groups.io/mt/102875700/7686176
Group Owner: devel+owner@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub [rebecca@openfw.io]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [edk2-devel] [PATCH v1 1/1] .github/workflows/codeql.yml: Add emacs output

[Joey Vagedes via groups.io]

[-- Attachment #1: Type: text/plain, Size: 4007 bytes --]

Reviewed-by: Joey Vagedes <joey.vagedes@gmail.com>

On Wed, Nov 29, 2023 at 9:01 AM <mikuback@linux.microsoft.com> wrote:

> From: Michael Kubacki <michael.kubacki@microsoft.com>
>
> Updates the workflow to also output files that can be loaded in emacs
> to show CodeQL issues (in addition to the existing SARIF output for
> standard SARIF viewers).
>
> The emacs files are in the SARIF zip file attached to each "CodeQL"
> run (https://github.com/tianocore/edk2/actions/workflows/codeql.yml).
>
> The file name ends with "-emacs.txt". An MdePkg example:
>   "codeql-db-mdepkg-debug-0-emacs.txt".
>
> Cc: Joey Vagedes <joey.vagedes@gmail.com>
> Cc: Laszlo Ersek <lersek@redhat.com>
> Cc: Michael D Kinney <michael.d.kinney@intel.com>
> Cc: Sean Brogan <sean.brogan@microsoft.com>
> Signed-off-by: Michael Kubacki <michael.kubacki@microsoft.com>
> ---
>
> Notes:
>     An example CodeQL run with this change:
>
>     https://github.com/tianocore/edk2/actions/runs/7035482184
>
>  .github/workflows/codeql.yml | 20 ++++++++++++++++----
>  1 file changed, 16 insertions(+), 4 deletions(-)
>
> diff --git a/.github/workflows/codeql.yml b/.github/workflows/codeql.yml
> index 72ece9dcb446..e826e67eb912 100644
> --- a/.github/workflows/codeql.yml
> +++ b/.github/workflows/codeql.yml
> @@ -92,7 +92,7 @@ jobs:
>          git config --system core.longpaths true
>
>      - name: Install/Upgrade pip Modules
> -      run: pip install -r pip-requirements.txt --upgrade requests
> +      run: pip install -r pip-requirements.txt --upgrade requests
> sarif-tools
>
>      - name: Determine CI Settings File Supported Operations
>        id: get_ci_file_operations
> @@ -304,16 +304,26 @@ jobs:
>          PACKAGE_NAME: ${{ matrix.Package }}
>        shell: python
>        run: |
> +        import logging
>          import os
> +        from edk2toollib.utility_functions import RunCmd
> +        from io import StringIO
> +        from pathlib import Path
>
>          package = os.environ['PACKAGE_NAME'].strip().lower()
>          directory_name = 'codeql-analysis-' + package + '-debug'
>          file_name = 'codeql-db-' + package + '-debug-0.sarif'
> -        sarif_path = os.path.join('Build', directory_name, file_name)
> +        sarif_path = Path('Build', directory_name, file_name)
>
>          with open(os.environ['GITHUB_OUTPUT'], 'a') as fh:
> -            if os.path.isfile(sarif_path):
> +            if sarif_path.is_file():
> +                emacs_file_path = sarif_path.with_name(sarif_path.stem +
> "-emacs.txt")
> +                out_stream_buffer = StringIO()
> +                exit_code = RunCmd("sarif", f"emacs {sarif_path} --output
> {emacs_file_path}",
> +                                   outstream=out_stream_buffer,
> +                                   logging_level=logging.NOTSET)
>                  print(f'upload_sarif_file=true', file=fh)
> +                print(f'emacs_file_path={emacs_file_path}', file=fh)
>                  print(f'sarif_file_path={sarif_path}', file=fh)
>              else:
>                  print(f'upload_sarif_file=false', file=fh)
> @@ -323,7 +333,9 @@ jobs:
>        if: steps.env_data.outputs.upload_sarif_file == 'true'
>        with:
>          name: ${{ matrix.Package }}-CodeQL-SARIF
> -        path: ${{ steps.env_data.outputs.sarif_file_path }}
> +        path: |
> +          ${{ steps.env_data.outputs.emacs_file_path }}
> +          ${{ steps.env_data.outputs.sarif_file_path }}
>          retention-days: 14
>          if-no-files-found: warn
>
> --
> 2.43.0.windows.1
>
>


-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.
View/Reply Online (#111853): https://edk2.groups.io/g/devel/message/111853
Mute This Topic: https://groups.io/mt/102875700/7686176
Group Owner: devel+owner@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub [rebecca@openfw.io]
-=-=-=-=-=-=-=-=-=-=-=-



[-- Attachment #2: Type: text/html, Size: 6124 bytes --]

Re: [edk2-devel] [PATCH v1 1/1] .github/workflows/codeql.yml: Add emacs output

[Michael D Kinney]

Acked-by: Michael D Kinney <michael.d.kinney@intel.com>

> -----Original Message-----
> From: mikuback@linux.microsoft.com <mikuback@linux.microsoft.com>
> Sent: Wednesday, November 29, 2023 9:01 AM
> To: devel@edk2.groups.io
> Cc: Joey Vagedes <joey.vagedes@gmail.com>; Laszlo Ersek
> <lersek@redhat.com>; Kinney, Michael D <michael.d.kinney@intel.com>; Sean
> Brogan <sean.brogan@microsoft.com>
> Subject: [PATCH v1 1/1] .github/workflows/codeql.yml: Add emacs output
> 
> From: Michael Kubacki <michael.kubacki@microsoft.com>
> 
> Updates the workflow to also output files that can be loaded in emacs
> to show CodeQL issues (in addition to the existing SARIF output for
> standard SARIF viewers).
> 
> The emacs files are in the SARIF zip file attached to each "CodeQL"
> run (https://github.com/tianocore/edk2/actions/workflows/codeql.yml).
> 
> The file name ends with "-emacs.txt". An MdePkg example:
>   "codeql-db-mdepkg-debug-0-emacs.txt".
> 
> Cc: Joey Vagedes <joey.vagedes@gmail.com>
> Cc: Laszlo Ersek <lersek@redhat.com>
> Cc: Michael D Kinney <michael.d.kinney@intel.com>
> Cc: Sean Brogan <sean.brogan@microsoft.com>
> Signed-off-by: Michael Kubacki <michael.kubacki@microsoft.com>
> ---
> 
> Notes:
>     An example CodeQL run with this change:
> 
>     https://github.com/tianocore/edk2/actions/runs/7035482184
> 
>  .github/workflows/codeql.yml | 20 ++++++++++++++++----
>  1 file changed, 16 insertions(+), 4 deletions(-)
> 
> diff --git a/.github/workflows/codeql.yml b/.github/workflows/codeql.yml
> index 72ece9dcb446..e826e67eb912 100644
> --- a/.github/workflows/codeql.yml
> +++ b/.github/workflows/codeql.yml
> @@ -92,7 +92,7 @@ jobs:
>          git config --system core.longpaths true
> 
>      - name: Install/Upgrade pip Modules
> -      run: pip install -r pip-requirements.txt --upgrade requests
> +      run: pip install -r pip-requirements.txt --upgrade requests sarif-
> tools
> 
>      - name: Determine CI Settings File Supported Operations
>        id: get_ci_file_operations
> @@ -304,16 +304,26 @@ jobs:
>          PACKAGE_NAME: ${{ matrix.Package }}
>        shell: python
>        run: |
> +        import logging
>          import os
> +        from edk2toollib.utility_functions import RunCmd
> +        from io import StringIO
> +        from pathlib import Path
> 
>          package = os.environ['PACKAGE_NAME'].strip().lower()
>          directory_name = 'codeql-analysis-' + package + '-debug'
>          file_name = 'codeql-db-' + package + '-debug-0.sarif'
> -        sarif_path = os.path.join('Build', directory_name, file_name)
> +        sarif_path = Path('Build', directory_name, file_name)
> 
>          with open(os.environ['GITHUB_OUTPUT'], 'a') as fh:
> -            if os.path.isfile(sarif_path):
> +            if sarif_path.is_file():
> +                emacs_file_path = sarif_path.with_name(sarif_path.stem +
> "-emacs.txt")
> +                out_stream_buffer = StringIO()
> +                exit_code = RunCmd("sarif", f"emacs {sarif_path} --output
> {emacs_file_path}",
> +                                   outstream=out_stream_buffer,
> +                                   logging_level=logging.NOTSET)
>                  print(f'upload_sarif_file=true', file=fh)
> +                print(f'emacs_file_path={emacs_file_path}', file=fh)
>                  print(f'sarif_file_path={sarif_path}', file=fh)
>              else:
>                  print(f'upload_sarif_file=false', file=fh)
> @@ -323,7 +333,9 @@ jobs:
>        if: steps.env_data.outputs.upload_sarif_file == 'true'
>        with:
>          name: ${{ matrix.Package }}-CodeQL-SARIF
> -        path: ${{ steps.env_data.outputs.sarif_file_path }}
> +        path: |
> +          ${{ steps.env_data.outputs.emacs_file_path }}
> +          ${{ steps.env_data.outputs.sarif_file_path }}
>          retention-days: 14
>          if-no-files-found: warn
> 
> --
> 2.43.0.windows.1



-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.
View/Reply Online (#111854): https://edk2.groups.io/g/devel/message/111854
Mute This Topic: https://groups.io/mt/102875700/7686176
Group Owner: devel+owner@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/leave/12367111/7686176/1913456212/xyzzy [rebecca@openfw.io]
-=-=-=-=-=-=-=-=-=-=-=-