Re: RFC v2: Static Analysis in edk2 CI

Re: RFC v2: Static Analysis in edk2 CI

[Michael D Kinney]

+devel@edk2.groups.io

Mike

> -----Original Message-----
> From: rfc@edk2.groups.io <rfc@edk2.groups.io> On Behalf Of Felix Polyudov via groups.io
> Sent: Monday, June 13, 2022 10:48 AM
> To: rfc@edk2.groups.io
> Cc: Kinney, Michael D <michael.d.kinney@intel.com>
> Subject: [edk2-rfc] RFC v2: Static Analysis in edk2 CI
> 
> This is version 2 of the proposal that provides additional details regarding the bring up process.
> 
> The initial version is at https://edk2.groups.io/g/rfc/message/696
> 
> The goal of the proposal is integration of the static analysis (SA) into the edk2 workflow.
> 
> - Use Open Coverity SA service to scan edk2 repository. The service is free for open source projects.
>     edk2 Open Coverity project: https://scan.coverity.com/projects/tianocore-edk2
> - Update edk2 CI scripts to run analysis once a week
>    - Perform analysis on all the edk2 packages using package DSC files that are used for CI build tests
>    (Coverity analysis is executed in the course of a specially instrumented project build).
>    - SA results are uploaded to scan.coverity.com. To access them one would need to register on the site and request tianocore-
> edk2 project access. The site can be used to triage the reported issues. Confirmed issues can be addressed using a standard edk2
> process (Bugzilla, mailing list).
> - During the initial bring up period, access to the SA results is restricted to stewards, maintainers, and members of the
> TianoCore InfoSec group, who are encouraged to review reported issues with the primary goal of identifying security-related
> issues. All such issues should be handled in accordance with the following guidelines:
>   https://github.com/tianocore/tianocore.github.io/wiki/Reporting-Security-Issues
> - The initial bring up period ends when embargo for all the identified security issues ends or after 30 days if no security
> issues have been identified
> - Once brig up period is over, SA results access is open to everybody.
> - The package maintainers should monitor weekly scan results for a newly reported issues and reach back to original patch
> submitters to resolve them. Package maintainers can revert the patch if no action is taken by the submitter.
> 
> -The information contained in this message may be confidential and proprietary to American Megatrends (AMI). This communication
> is intended to be read only by the individual or entity to whom it is addressed or by their designee. If the reader of this
> message is not the intended recipient, you are on notice that any distribution of this message, in any form, is strictly
> prohibited. Please promptly notify the sender by reply e-mail or by telephone at 770-246-8600, and then delete or destroy all
> copies of the transmission.
> 
> 
> 
> 

Re: [edk2-rfc] RFC v2: Static Analysis in edk2 CI

[Pedro Falcato]

[-- Attachment #1: Type: text/plain, Size: 3377 bytes --]

(Replying under Mike for devel visibility)

Felix,

Why coverity? I feel like we could run something akin to LLVM's clang-tidy
+ scan-build; it's open source (transparent *and* we can improve it or add
UEFI quirks) and doesn't rely on a third-party service. I'm sure we could
figure something out for hosting the thing. Otherwise, looks good to me.

Thanks,
Pedro

On Mon, Jun 13, 2022 at 7:54 PM Michael D Kinney <michael.d.kinney@intel.com>
wrote:

> +devel@edk2.groups.io
>
> Mike
>
> > -----Original Message-----
> > From: rfc@edk2.groups.io <rfc@edk2.groups.io> On Behalf Of Felix
> Polyudov via groups.io
> > Sent: Monday, June 13, 2022 10:48 AM
> > To: rfc@edk2.groups.io
> > Cc: Kinney, Michael D <michael.d.kinney@intel.com>
> > Subject: [edk2-rfc] RFC v2: Static Analysis in edk2 CI
> >
> > This is version 2 of the proposal that provides additional details
> regarding the bring up process.
> >
> > The initial version is at https://edk2.groups.io/g/rfc/message/696
> >
> > The goal of the proposal is integration of the static analysis (SA) into
> the edk2 workflow.
> >
> > - Use Open Coverity SA service to scan edk2 repository. The service is
> free for open source projects.
> >     edk2 Open Coverity project:
> https://scan.coverity.com/projects/tianocore-edk2
> > - Update edk2 CI scripts to run analysis once a week
> >    - Perform analysis on all the edk2 packages using package DSC files
> that are used for CI build tests
> >    (Coverity analysis is executed in the course of a specially
> instrumented project build).
> >    - SA results are uploaded to scan.coverity.com. To access them one
> would need to register on the site and request tianocore-
> > edk2 project access. The site can be used to triage the reported issues.
> Confirmed issues can be addressed using a standard edk2
> > process (Bugzilla, mailing list).
> > - During the initial bring up period, access to the SA results is
> restricted to stewards, maintainers, and members of the
> > TianoCore InfoSec group, who are encouraged to review reported issues
> with the primary goal of identifying security-related
> > issues. All such issues should be handled in accordance with the
> following guidelines:
> >
> https://github.com/tianocore/tianocore.github.io/wiki/Reporting-Security-Issues
> > - The initial bring up period ends when embargo for all the identified
> security issues ends or after 30 days if no security
> > issues have been identified
> > - Once brig up period is over, SA results access is open to everybody.
> > - The package maintainers should monitor weekly scan results for a newly
> reported issues and reach back to original patch
> > submitters to resolve them. Package maintainers can revert the patch if
> no action is taken by the submitter.
> >
> > -The information contained in this message may be confidential and
> proprietary to American Megatrends (AMI). This communication
> > is intended to be read only by the individual or entity to whom it is
> addressed or by their designee. If the reader of this
> > message is not the intended recipient, you are on notice that any
> distribution of this message, in any form, is strictly
> > prohibited. Please promptly notify the sender by reply e-mail or by
> telephone at 770-246-8600, and then delete or destroy all
> > copies of the transmission.
> >
> >
> >
> >
>
>
>
> 
>
>
>

-- 
Pedro Falcato

[-- Attachment #2: Type: text/html, Size: 4870 bytes --]

Re: [edk2-rfc] RFC v2: Static Analysis in edk2 CI

[Rebecca Cran]

LLVM's tools also appear to be much easier to review, for other people 
to run etc. I'd suggest at least starting with clang-tidy + scan-build 
and possibly adding Coverity later.

I've found the Coverity tools, while very powerful, tend to get ignored 
after a while because it's quite a process to keep it running, go 
through the issues it detects and keep the database up-to-date etc.


-- 

Rebecca Cran


On 6/13/22 15:54, Pedro Falcato wrote:
> (Replying under Mike for devel visibility)
>
> Felix,
>
> Why coverity? I feel like we could run something akin to LLVM's clang-tidy
> + scan-build; it's open source (transparent *and* we can improve it or add
> UEFI quirks) and doesn't rely on a third-party service. I'm sure we could
> figure something out for hosting the thing. Otherwise, looks good to me.
>
> Thanks,
> Pedro
>
> On Mon, Jun 13, 2022 at 7:54 PM Michael D Kinney <michael.d.kinney@intel.com>
> wrote:
>
>> +devel@edk2.groups.io
>>
>> Mike
>>
>>> -----Original Message-----
>>> From: rfc@edk2.groups.io <rfc@edk2.groups.io> On Behalf Of Felix
>> Polyudov via groups.io
>>> Sent: Monday, June 13, 2022 10:48 AM
>>> To: rfc@edk2.groups.io
>>> Cc: Kinney, Michael D <michael.d.kinney@intel.com>
>>> Subject: [edk2-rfc] RFC v2: Static Analysis in edk2 CI
>>>
>>> This is version 2 of the proposal that provides additional details
>> regarding the bring up process.
>>> The initial version is at https://edk2.groups.io/g/rfc/message/696
>>>
>>> The goal of the proposal is integration of the static analysis (SA) into
>> the edk2 workflow.
>>> - Use Open Coverity SA service to scan edk2 repository. The service is
>> free for open source projects.
>>>      edk2 Open Coverity project:
>> https://scan.coverity.com/projects/tianocore-edk2
>>> - Update edk2 CI scripts to run analysis once a week
>>>     - Perform analysis on all the edk2 packages using package DSC files
>> that are used for CI build tests
>>>     (Coverity analysis is executed in the course of a specially
>> instrumented project build).
>>>     - SA results are uploaded to scan.coverity.com. To access them one
>> would need to register on the site and request tianocore-
>>> edk2 project access. The site can be used to triage the reported issues.
>> Confirmed issues can be addressed using a standard edk2
>>> process (Bugzilla, mailing list).
>>> - During the initial bring up period, access to the SA results is
>> restricted to stewards, maintainers, and members of the
>>> TianoCore InfoSec group, who are encouraged to review reported issues
>> with the primary goal of identifying security-related
>>> issues. All such issues should be handled in accordance with the
>> following guidelines:
>> https://github.com/tianocore/tianocore.github.io/wiki/Reporting-Security-Issues
>>> - The initial bring up period ends when embargo for all the identified
>> security issues ends or after 30 days if no security
>>> issues have been identified
>>> - Once brig up period is over, SA results access is open to everybody.
>>> - The package maintainers should monitor weekly scan results for a newly
>> reported issues and reach back to original patch
>>> submitters to resolve them. Package maintainers can revert the patch if
>> no action is taken by the submitter.
>>> -The information contained in this message may be confidential and
>> proprietary to American Megatrends (AMI). This communication
>>> is intended to be read only by the individual or entity to whom it is
>> addressed or by their designee. If the reader of this
>>> message is not the intended recipient, you are on notice that any
>> distribution of this message, in any form, is strictly
>>> prohibited. Please promptly notify the sender by reply e-mail or by
>> telephone at 770-246-8600, and then delete or destroy all
>>> copies of the transmission.
>>>
>>>
>>>
>>>
>>
>>
>>
>>
>>
>>

Re: [edk2-rfc] RFC v2: Static Analysis in edk2 CI

[Pedro Falcato]

[-- Attachment #1: Type: text/plain, Size: 1177 bytes --]

(Re-adding devel@ since Felix dropped it)

On Tue, Jun 14, 2022 at 8:59 PM Pedro Falcato <pedro.falcato@gmail.com>
wrote:

> Just want to note that if we want to go ahead with fuzzing (I detailed a
> possible plan to do so in the mailing list a month or so ago) we will
> definitely need somewhere to run fuzzing (even if it's Google's syzbot).
> Getting somewhere where we can run static analysis, fuzzing just makes
> sense IMO (hell, who knows, maybe even CI or something like Gerrit for
> mailing list-less code reviews).
>
> On Tue, Jun 14, 2022 at 7:43 PM Felix Polyudov via groups.io <felixp=
> ami.com@groups.io> wrote:
>
>> Yes, LLVM/CLANG Static Analyzer is another possibility. I've mentioned it
>> in the first version of the RFC.
>> CodeChecker (https://codechecker.readthedocs.io/en/latest/) is an open
>> source front-end for the scan-build and clang-tidy.
>> It simplifies analyzer configuration and provides web-based report
>> storage. However, it has to be hosted somewhere.
>> If somebody has an idea on how edk2 community can host the CodeChecker,
>> that's definitely an option to consider.
>>
>>
>> 
>>
>>
>>
>
> --
> Pedro Falcato
>


-- 
Pedro Falcato

[-- Attachment #2: Type: text/html, Size: 2069 bytes --]

Re: [edk2-rfc] RFC v2: Static Analysis in edk2 CI

[Michael D Kinney]

I have Coverity scan builds running in a GitHub Action and then uploaded to Coverity.

We should be able to configure a GitHub Action to run other analyzers.

Mike

> -----Original Message-----
> From: rfc@edk2.groups.io <rfc@edk2.groups.io> On Behalf Of Pedro Falcato
> Sent: Tuesday, June 14, 2022 1:00 PM
> To: rfc@edk2.groups.io; POLUDOV, FELIX <felixp@ami.com>
> Cc: Rebecca Cran <rebecca@bsdio.com>; edk2-devel-groups-io <devel@edk2.groups.io>
> Subject: Re: [edk2-rfc] RFC v2: Static Analysis in edk2 CI
> 
> (Re-adding devel@ since Felix dropped it)
> 
> On Tue, Jun 14, 2022 at 8:59 PM Pedro Falcato <pedro.falcato@gmail.com>
> wrote:
> 
> > Just want to note that if we want to go ahead with fuzzing (I detailed a
> > possible plan to do so in the mailing list a month or so ago) we will
> > definitely need somewhere to run fuzzing (even if it's Google's syzbot).
> > Getting somewhere where we can run static analysis, fuzzing just makes
> > sense IMO (hell, who knows, maybe even CI or something like Gerrit for
> > mailing list-less code reviews).
> >
> > On Tue, Jun 14, 2022 at 7:43 PM Felix Polyudov via groups.io <felixp=
> > ami.com@groups.io> wrote:
> >
> >> Yes, LLVM/CLANG Static Analyzer is another possibility. I've mentioned it
> >> in the first version of the RFC.
> >> CodeChecker (https://codechecker.readthedocs.io/en/latest/) is an open
> >> source front-end for the scan-build and clang-tidy.
> >> It simplifies analyzer configuration and provides web-based report
> >> storage. However, it has to be hosted somewhere.
> >> If somebody has an idea on how edk2 community can host the CodeChecker,
> >> that's definitely an option to consider.
> >>
> >>
> >>
> >>
> >>
> >>
> >
> > --
> > Pedro Falcato
> >
> 
> 
> --
> Pedro Falcato
> 
> 
> 
> 

Re: [edk2-rfc] RFC v2: Static Analysis in edk2 CI

[Felix Polyudov]

Yes, we can run other analyzer; however, in case of CodeChecker we also need a server to upload the result to.

> -----Original Message-----
> From: rfc@edk2.groups.io <rfc@edk2.groups.io> On Behalf Of Michael D
> Kinney via groups.io
> Sent: Thursday, June 23, 2022 9:30 PM
> To: rfc@edk2.groups.io; pedro.falcato@gmail.com; Felix Polyudov
> <Felixp@ami.com>; Kinney, Michael D <michael.d.kinney@intel.com>
> Cc: Rebecca Cran <rebecca@bsdio.com>; edk2-devel-groups-io
> <devel@edk2.groups.io>
> Subject: [EXTERNAL] Re: [edk2-rfc] RFC v2: Static Analysis in edk2 CI
>
>
> **CAUTION: The e-mail below is from an external source. Please exercise
> caution before opening attachments, clicking links, or following guidance.**
>
> I have Coverity scan builds running in a GitHub Action and then uploaded to
> Coverity.
>
> We should be able to configure a GitHub Action to run other analyzers.
>
> Mike
>
> > -----Original Message-----
> > From: rfc@edk2.groups.io <rfc@edk2.groups.io> On Behalf Of Pedro
> > Falcato
> > Sent: Tuesday, June 14, 2022 1:00 PM
> > To: rfc@edk2.groups.io; POLUDOV, FELIX <felixp@ami.com>
> > Cc: Rebecca Cran <rebecca@bsdio.com>; edk2-devel-groups-io
> > <devel@edk2.groups.io>
> > Subject: Re: [edk2-rfc] RFC v2: Static Analysis in edk2 CI
> >
> > (Re-adding devel@ since Felix dropped it)
> >
> > On Tue, Jun 14, 2022 at 8:59 PM Pedro Falcato
> > <pedro.falcato@gmail.com>
> > wrote:
> >
> > > Just want to note that if we want to go ahead with fuzzing (I
> > > detailed a possible plan to do so in the mailing list a month or so
> > > ago) we will definitely need somewhere to run fuzzing (even if it's Google's
> syzbot).
> > > Getting somewhere where we can run static analysis, fuzzing just
> > > makes sense IMO (hell, who knows, maybe even CI or something like
> > > Gerrit for mailing list-less code reviews).
> > >
> > > On Tue, Jun 14, 2022 at 7:43 PM Felix Polyudov via groups.io
> > > <felixp= ami.com@groups.io> wrote:
> > >
> > >> Yes, LLVM/CLANG Static Analyzer is another possibility. I've
> > >> mentioned it in the first version of the RFC.
> > >> CodeChecker
> > >>
> (https://codechecker.readthedocs.io/en/latest/) is an open source front-end
> for the scan-build and clang-tidy.
> > >> It simplifies analyzer configuration and provides web-based report
> > >> storage. However, it has to be hosted somewhere.
> > >> If somebody has an idea on how edk2 community can host the
> > >> CodeChecker, that's definitely an option to consider.
> > >>
> > >>
> > >>
> > >>
> > >>
> > >>
> > >
> > > --
> > > Pedro Falcato

-The information contained in this message may be confidential and proprietary to American Megatrends (AMI). This communication is intended to be read only by the individual or entity to whom it is addressed or by their designee. If the reader of this message is not the intended recipient, you are on notice that any distribution of this message, in any form, is strictly prohibited. Please promptly notify the sender by reply e-mail or by telephone at 770-246-8600, and then delete or destroy all copies of the transmission.