* [Patch v3 01/22] Maintainers.txt: Add continuous integration(CI) directories
2019-10-29 19:54 [Patch v3 00/22] Enable Phase 1 of EDK II CI Michael D Kinney
@ 2019-10-29 19:54 ` Michael D Kinney
2019-10-30 2:34 ` Leif Lindholm
2019-10-29 19:54 ` [Patch v3 02/22] .gitignore: Ignore python compiled files, extdeps, and vscode Michael D Kinney
` (21 subsequent siblings)
22 siblings, 1 reply; 56+ messages in thread
From: Michael D Kinney @ 2019-10-29 19:54 UTC (permalink / raw)
To: devel; +Cc: Andrew Fish, Laszlo Ersek, Leif Lindholm
https://bugzilla.tianocore.org/show_bug.cgi?id=2315
Add maintainers and reviewers for the directories associated
with continuous integration steps.
* .azurepipelines
* .mergify
* .pytool
Cc: Andrew Fish <afish@apple.com>
Cc: Laszlo Ersek <lersek@redhat.com>
Cc: Leif Lindholm <leif.lindholm@linaro.org>
Signed-off-by: Michael D Kinney <michael.d.kinney@intel.com>
---
Maintainers.txt | 23 +++++++++++++++++++++++
1 file changed, 23 insertions(+)
diff --git a/Maintainers.txt b/Maintainers.txt
index 71a34a2845..1fb38ddc5b 100644
--- a/Maintainers.txt
+++ b/Maintainers.txt
@@ -460,3 +460,26 @@ F: StandaloneMmPkg/
M: Achin Gupta <achin.gupta@arm.com>
M: Jiewen Yao <jiewen.yao@intel.com>
R: Supreeth Venkatesh <supreeth.venkatesh@arm.com>
+
+EDK II Continuous Integration:
+------------------------------
+.azurepipelines/
+F: .azurepipelines/
+M: Sean Brogan <sean.brogan@microsoft.com>
+M: Bret Barkelew <Bret.Barkelew@microsoft.com>
+R: Michael D Kinney <michael.d.kinney@intel.com>
+R: Liming Gao <liming.gao@intel.com>
+
+.mergify/
+F: .mergify/
+M: Michael D Kinney <michael.d.kinney@intel.com>
+M: Liming Gao <liming.gao@intel.com>
+R: Sean Brogan <sean.brogan@microsoft.com>
+R: Bret Barkelew <Bret.Barkelew@microsoft.com>
+
+.pytool/
+F: .pytool/
+M: Sean Brogan <sean.brogan@microsoft.com>
+M: Bret Barkelew <Bret.Barkelew@microsoft.com>
+R: Michael D Kinney <michael.d.kinney@intel.com>
+R: Liming Gao <liming.gao@intel.com>
--
2.21.0.windows.1
^ permalink raw reply related [flat|nested] 56+ messages in thread* Re: [Patch v3 01/22] Maintainers.txt: Add continuous integration(CI) directories
2019-10-29 19:54 ` [Patch v3 01/22] Maintainers.txt: Add continuous integration(CI) directories Michael D Kinney
@ 2019-10-30 2:34 ` Leif Lindholm
2019-10-31 9:03 ` Laszlo Ersek
0 siblings, 1 reply; 56+ messages in thread
From: Leif Lindholm @ 2019-10-30 2:34 UTC (permalink / raw)
To: Michael D Kinney; +Cc: devel, Andrew Fish, Laszlo Ersek
On Tue, Oct 29, 2019 at 12:54:56PM -0700, Michael D Kinney wrote:
> https://bugzilla.tianocore.org/show_bug.cgi?id=2315
>
> Add maintainers and reviewers for the directories associated
> with continuous integration steps.
> * .azurepipelines
> * .mergify
> * .pytool
>
> Cc: Andrew Fish <afish@apple.com>
> Cc: Laszlo Ersek <lersek@redhat.com>
> Cc: Leif Lindholm <leif.lindholm@linaro.org>
> Signed-off-by: Michael D Kinney <michael.d.kinney@intel.com>
Reviewed-by: Leif Lindholm <leif.lindholm@linaro.org>
With the added preference that the addition be moved before the EDK II
Packages: section (since C comes before P). But it is just a preference.
/
Leif
> ---
> Maintainers.txt | 23 +++++++++++++++++++++++
> 1 file changed, 23 insertions(+)
>
> diff --git a/Maintainers.txt b/Maintainers.txt
> index 71a34a2845..1fb38ddc5b 100644
> --- a/Maintainers.txt
> +++ b/Maintainers.txt
> @@ -460,3 +460,26 @@ F: StandaloneMmPkg/
> M: Achin Gupta <achin.gupta@arm.com>
> M: Jiewen Yao <jiewen.yao@intel.com>
> R: Supreeth Venkatesh <supreeth.venkatesh@arm.com>
> +
> +EDK II Continuous Integration:
> +------------------------------
> +.azurepipelines/
> +F: .azurepipelines/
> +M: Sean Brogan <sean.brogan@microsoft.com>
> +M: Bret Barkelew <Bret.Barkelew@microsoft.com>
> +R: Michael D Kinney <michael.d.kinney@intel.com>
> +R: Liming Gao <liming.gao@intel.com>
> +
> +.mergify/
> +F: .mergify/
> +M: Michael D Kinney <michael.d.kinney@intel.com>
> +M: Liming Gao <liming.gao@intel.com>
> +R: Sean Brogan <sean.brogan@microsoft.com>
> +R: Bret Barkelew <Bret.Barkelew@microsoft.com>
> +
> +.pytool/
> +F: .pytool/
> +M: Sean Brogan <sean.brogan@microsoft.com>
> +M: Bret Barkelew <Bret.Barkelew@microsoft.com>
> +R: Michael D Kinney <michael.d.kinney@intel.com>
> +R: Liming Gao <liming.gao@intel.com>
> --
> 2.21.0.windows.1
>
^ permalink raw reply [flat|nested] 56+ messages in thread* Re: [Patch v3 01/22] Maintainers.txt: Add continuous integration(CI) directories
2019-10-30 2:34 ` Leif Lindholm
@ 2019-10-31 9:03 ` Laszlo Ersek
0 siblings, 0 replies; 56+ messages in thread
From: Laszlo Ersek @ 2019-10-31 9:03 UTC (permalink / raw)
To: Leif Lindholm, Michael D Kinney; +Cc: devel, Andrew Fish
On 10/30/19 03:34, Leif Lindholm wrote:
> On Tue, Oct 29, 2019 at 12:54:56PM -0700, Michael D Kinney wrote:
>> https://bugzilla.tianocore.org/show_bug.cgi?id=2315
>>
>> Add maintainers and reviewers for the directories associated
>> with continuous integration steps.
>> * .azurepipelines
>> * .mergify
>> * .pytool
>>
>> Cc: Andrew Fish <afish@apple.com>
>> Cc: Laszlo Ersek <lersek@redhat.com>
>> Cc: Leif Lindholm <leif.lindholm@linaro.org>
>> Signed-off-by: Michael D Kinney <michael.d.kinney@intel.com>
>
> Reviewed-by: Leif Lindholm <leif.lindholm@linaro.org>
Reviewed-by: Laszlo Ersek <lersek@redhat.com>
> With the added preference that the addition be moved before the EDK II
> Packages: section (since C comes before P). But it is just a preference.
Good idea!
Thanks
Laszlo
>
> /
> Leif
>
>> ---
>> Maintainers.txt | 23 +++++++++++++++++++++++
>> 1 file changed, 23 insertions(+)
>>
>> diff --git a/Maintainers.txt b/Maintainers.txt
>> index 71a34a2845..1fb38ddc5b 100644
>> --- a/Maintainers.txt
>> +++ b/Maintainers.txt
>> @@ -460,3 +460,26 @@ F: StandaloneMmPkg/
>> M: Achin Gupta <achin.gupta@arm.com>
>> M: Jiewen Yao <jiewen.yao@intel.com>
>> R: Supreeth Venkatesh <supreeth.venkatesh@arm.com>
>> +
>> +EDK II Continuous Integration:
>> +------------------------------
>> +.azurepipelines/
>> +F: .azurepipelines/
>> +M: Sean Brogan <sean.brogan@microsoft.com>
>> +M: Bret Barkelew <Bret.Barkelew@microsoft.com>
>> +R: Michael D Kinney <michael.d.kinney@intel.com>
>> +R: Liming Gao <liming.gao@intel.com>
>> +
>> +.mergify/
>> +F: .mergify/
>> +M: Michael D Kinney <michael.d.kinney@intel.com>
>> +M: Liming Gao <liming.gao@intel.com>
>> +R: Sean Brogan <sean.brogan@microsoft.com>
>> +R: Bret Barkelew <Bret.Barkelew@microsoft.com>
>> +
>> +.pytool/
>> +F: .pytool/
>> +M: Sean Brogan <sean.brogan@microsoft.com>
>> +M: Bret Barkelew <Bret.Barkelew@microsoft.com>
>> +R: Michael D Kinney <michael.d.kinney@intel.com>
>> +R: Liming Gao <liming.gao@intel.com>
>> --
>> 2.21.0.windows.1
>>
^ permalink raw reply [flat|nested] 56+ messages in thread
* [Patch v3 02/22] .gitignore: Ignore python compiled files, extdeps, and vscode
2019-10-29 19:54 [Patch v3 00/22] Enable Phase 1 of EDK II CI Michael D Kinney
2019-10-29 19:54 ` [Patch v3 01/22] Maintainers.txt: Add continuous integration(CI) directories Michael D Kinney
@ 2019-10-29 19:54 ` Michael D Kinney
2019-10-30 2:29 ` Leif Lindholm
2019-10-29 19:54 ` [Patch v3 03/22] requirements.txt: Add python pip requirements file Michael D Kinney
` (20 subsequent siblings)
22 siblings, 1 reply; 56+ messages in thread
From: Michael D Kinney @ 2019-10-29 19:54 UTC (permalink / raw)
To: devel; +Cc: Sean Brogan, Andrew Fish, Laszlo Ersek, Leif Lindholm, Kinney
From: Sean Brogan <sean.brogan@microsoft.com>
https://bugzilla.tianocore.org/show_bug.cgi?id=2315
Update .gitignore to ignore .pyc files and __pycache__
directories. Python based plugins can be added to any
package or platform, so these files and directories may
be present outside of BaseTools.
Ignore _extdep directories that are generated by the
pytool external dependency feature.
Ignore .vscode directories generated by the VS Code
editor.
Cc: Andrew Fish <afish@apple.com>
Cc: Laszlo Ersek <lersek@redhat.com>
Cc: Leif Lindholm <leif.lindholm@linaro.org>
Signed-off-by: Kinney <michael.d.kinney@intel.com>
---
.gitignore | 5 +++++
1 file changed, 5 insertions(+)
diff --git a/.gitignore b/.gitignore
index 97f22c348c..71679478ac 100644
--- a/.gitignore
+++ b/.gitignore
@@ -1,3 +1,8 @@
Build/
tags/
.DS_Store
+*.pyc
+__pycache__/
+*_extdep/
+
+.vscode/
--
2.21.0.windows.1
^ permalink raw reply related [flat|nested] 56+ messages in thread* Re: [Patch v3 02/22] .gitignore: Ignore python compiled files, extdeps, and vscode
2019-10-29 19:54 ` [Patch v3 02/22] .gitignore: Ignore python compiled files, extdeps, and vscode Michael D Kinney
@ 2019-10-30 2:29 ` Leif Lindholm
2019-10-31 4:43 ` Michael D Kinney
0 siblings, 1 reply; 56+ messages in thread
From: Leif Lindholm @ 2019-10-30 2:29 UTC (permalink / raw)
To: Michael D Kinney; +Cc: devel, Sean Brogan, Andrew Fish, Laszlo Ersek
On Tue, Oct 29, 2019 at 12:54:57PM -0700, Michael D Kinney wrote:
> From: Sean Brogan <sean.brogan@microsoft.com>
>
> https://bugzilla.tianocore.org/show_bug.cgi?id=2315
>
> Update .gitignore to ignore .pyc files and __pycache__
> directories. Python based plugins can be added to any
> package or platform, so these files and directories may
> be present outside of BaseTools.
>
> Ignore _extdep directories that are generated by the
> pytool external dependency feature.
>
> Ignore .vscode directories generated by the VS Code
> editor.
>
> Cc: Andrew Fish <afish@apple.com>
> Cc: Laszlo Ersek <lersek@redhat.com>
> Cc: Leif Lindholm <leif.lindholm@linaro.org>
> Signed-off-by: Kinney <michael.d.kinney@intel.com>
Reviewed-by: Leif Lindholm <leif.lindholm@linaro.org>
(This one could be merged independently of rest of set, it's useful on
its own.)
> ---
> .gitignore | 5 +++++
> 1 file changed, 5 insertions(+)
>
> diff --git a/.gitignore b/.gitignore
> index 97f22c348c..71679478ac 100644
> --- a/.gitignore
> +++ b/.gitignore
> @@ -1,3 +1,8 @@
> Build/
> tags/
> .DS_Store
> +*.pyc
> +__pycache__/
> +*_extdep/
> +
> +.vscode/
> --
> 2.21.0.windows.1
>
^ permalink raw reply [flat|nested] 56+ messages in thread
* Re: [Patch v3 02/22] .gitignore: Ignore python compiled files, extdeps, and vscode
2019-10-30 2:29 ` Leif Lindholm
@ 2019-10-31 4:43 ` Michael D Kinney
2019-10-31 9:02 ` Laszlo Ersek
0 siblings, 1 reply; 56+ messages in thread
From: Michael D Kinney @ 2019-10-31 4:43 UTC (permalink / raw)
To: Leif Lindholm, Kinney, Michael D
Cc: devel@edk2.groups.io, Sean Brogan, Andrew Fish, Laszlo Ersek
Leif,
Thanks. I agree. I will split it out.
Mike
> -----Original Message-----
> From: Leif Lindholm <leif.lindholm@linaro.org>
> Sent: Tuesday, October 29, 2019 7:30 PM
> To: Kinney, Michael D <michael.d.kinney@intel.com>
> Cc: devel@edk2.groups.io; Sean Brogan
> <sean.brogan@microsoft.com>; Andrew Fish
> <afish@apple.com>; Laszlo Ersek <lersek@redhat.com>
> Subject: Re: [Patch v3 02/22] .gitignore: Ignore python
> compiled files, extdeps, and vscode
>
> On Tue, Oct 29, 2019 at 12:54:57PM -0700, Michael D
> Kinney wrote:
> > From: Sean Brogan <sean.brogan@microsoft.com>
> >
> > https://bugzilla.tianocore.org/show_bug.cgi?id=2315
> >
> > Update .gitignore to ignore .pyc files and __pycache__
> directories.
> > Python based plugins can be added to any package or
> platform, so these
> > files and directories may be present outside of
> BaseTools.
> >
> > Ignore _extdep directories that are generated by the
> pytool external
> > dependency feature.
> >
> > Ignore .vscode directories generated by the VS Code
> editor.
> >
> > Cc: Andrew Fish <afish@apple.com>
> > Cc: Laszlo Ersek <lersek@redhat.com>
> > Cc: Leif Lindholm <leif.lindholm@linaro.org>
> > Signed-off-by: Kinney <michael.d.kinney@intel.com>
>
> Reviewed-by: Leif Lindholm <leif.lindholm@linaro.org>
>
> (This one could be merged independently of rest of set,
> it's useful on its own.)
>
> > ---
> > .gitignore | 5 +++++
> > 1 file changed, 5 insertions(+)
> >
> > diff --git a/.gitignore b/.gitignore
> > index 97f22c348c..71679478ac 100644
> > --- a/.gitignore
> > +++ b/.gitignore
> > @@ -1,3 +1,8 @@
> > Build/
> > tags/
> > .DS_Store
> > +*.pyc
> > +__pycache__/
> > +*_extdep/
> > +
> > +.vscode/
> > --
> > 2.21.0.windows.1
> >
^ permalink raw reply [flat|nested] 56+ messages in thread
* Re: [Patch v3 02/22] .gitignore: Ignore python compiled files, extdeps, and vscode
2019-10-31 4:43 ` Michael D Kinney
@ 2019-10-31 9:02 ` Laszlo Ersek
2019-11-04 16:02 ` [edk2-devel] " Philippe Mathieu-Daudé
0 siblings, 1 reply; 56+ messages in thread
From: Laszlo Ersek @ 2019-10-31 9:02 UTC (permalink / raw)
To: Kinney, Michael D, Leif Lindholm
Cc: devel@edk2.groups.io, Sean Brogan, Andrew Fish
On 10/31/19 05:43, Kinney, Michael D wrote:
> Leif,
>
> Thanks. I agree. I will split it out.
I would further suggest alphabetical sorting for the .gitignore lines.
Thanks!
Laszlo
>
> Mike
>
>> -----Original Message-----
>> From: Leif Lindholm <leif.lindholm@linaro.org>
>> Sent: Tuesday, October 29, 2019 7:30 PM
>> To: Kinney, Michael D <michael.d.kinney@intel.com>
>> Cc: devel@edk2.groups.io; Sean Brogan
>> <sean.brogan@microsoft.com>; Andrew Fish
>> <afish@apple.com>; Laszlo Ersek <lersek@redhat.com>
>> Subject: Re: [Patch v3 02/22] .gitignore: Ignore python
>> compiled files, extdeps, and vscode
>>
>> On Tue, Oct 29, 2019 at 12:54:57PM -0700, Michael D
>> Kinney wrote:
>>> From: Sean Brogan <sean.brogan@microsoft.com>
>>>
>>> https://bugzilla.tianocore.org/show_bug.cgi?id=2315
>>>
>>> Update .gitignore to ignore .pyc files and __pycache__
>> directories.
>>> Python based plugins can be added to any package or
>> platform, so these
>>> files and directories may be present outside of
>> BaseTools.
>>>
>>> Ignore _extdep directories that are generated by the
>> pytool external
>>> dependency feature.
>>>
>>> Ignore .vscode directories generated by the VS Code
>> editor.
>>>
>>> Cc: Andrew Fish <afish@apple.com>
>>> Cc: Laszlo Ersek <lersek@redhat.com>
>>> Cc: Leif Lindholm <leif.lindholm@linaro.org>
>>> Signed-off-by: Kinney <michael.d.kinney@intel.com>
>>
>> Reviewed-by: Leif Lindholm <leif.lindholm@linaro.org>
>>
>> (This one could be merged independently of rest of set,
>> it's useful on its own.)
>>
>>> ---
>>> .gitignore | 5 +++++
>>> 1 file changed, 5 insertions(+)
>>>
>>> diff --git a/.gitignore b/.gitignore
>>> index 97f22c348c..71679478ac 100644
>>> --- a/.gitignore
>>> +++ b/.gitignore
>>> @@ -1,3 +1,8 @@
>>> Build/
>>> tags/
>>> .DS_Store
>>> +*.pyc
>>> +__pycache__/
>>> +*_extdep/
>>> +
>>> +.vscode/
>>> --
>>> 2.21.0.windows.1
>>>
^ permalink raw reply [flat|nested] 56+ messages in thread
* Re: [edk2-devel] [Patch v3 02/22] .gitignore: Ignore python compiled files, extdeps, and vscode
2019-10-31 9:02 ` Laszlo Ersek
@ 2019-11-04 16:02 ` Philippe Mathieu-Daudé
0 siblings, 0 replies; 56+ messages in thread
From: Philippe Mathieu-Daudé @ 2019-11-04 16:02 UTC (permalink / raw)
To: devel, lersek, Kinney, Michael D, Leif Lindholm; +Cc: Sean Brogan, Andrew Fish
On 10/31/19 10:02 AM, Laszlo Ersek wrote:
> On 10/31/19 05:43, Kinney, Michael D wrote:
>> Leif,
>>
>> Thanks. I agree. I will split it out.
>
> I would further suggest alphabetical sorting for the .gitignore lines.
Reviewed-by: Philippe Mathieu-Daude <philmd@redhat.com>
>>> -----Original Message-----
>>> From: Leif Lindholm <leif.lindholm@linaro.org>
>>> Sent: Tuesday, October 29, 2019 7:30 PM
>>> To: Kinney, Michael D <michael.d.kinney@intel.com>
>>> Cc: devel@edk2.groups.io; Sean Brogan
>>> <sean.brogan@microsoft.com>; Andrew Fish
>>> <afish@apple.com>; Laszlo Ersek <lersek@redhat.com>
>>> Subject: Re: [Patch v3 02/22] .gitignore: Ignore python
>>> compiled files, extdeps, and vscode
>>>
>>> On Tue, Oct 29, 2019 at 12:54:57PM -0700, Michael D
>>> Kinney wrote:
>>>> From: Sean Brogan <sean.brogan@microsoft.com>
>>>>
>>>> https://bugzilla.tianocore.org/show_bug.cgi?id=2315
>>>>
>>>> Update .gitignore to ignore .pyc files and __pycache__
>>> directories.
>>>> Python based plugins can be added to any package or
>>> platform, so these
>>>> files and directories may be present outside of
>>> BaseTools.
>>>>
>>>> Ignore _extdep directories that are generated by the
>>> pytool external
>>>> dependency feature.
>>>>
>>>> Ignore .vscode directories generated by the VS Code
>>> editor.
>>>>
>>>> Cc: Andrew Fish <afish@apple.com>
>>>> Cc: Laszlo Ersek <lersek@redhat.com>
>>>> Cc: Leif Lindholm <leif.lindholm@linaro.org>
>>>> Signed-off-by: Kinney <michael.d.kinney@intel.com>
>>>
>>> Reviewed-by: Leif Lindholm <leif.lindholm@linaro.org>
>>>
>>> (This one could be merged independently of rest of set,
>>> it's useful on its own.)
>>>
>>>> ---
>>>> .gitignore | 5 +++++
>>>> 1 file changed, 5 insertions(+)
>>>>
>>>> diff --git a/.gitignore b/.gitignore
>>>> index 97f22c348c..71679478ac 100644
>>>> --- a/.gitignore
>>>> +++ b/.gitignore
>>>> @@ -1,3 +1,8 @@
>>>> Build/
>>>> tags/
>>>> .DS_Store
>>>> +*.pyc
>>>> +__pycache__/
>>>> +*_extdep/
>>>> +
>>>> +.vscode/
>>>> --
>>>> 2.21.0.windows.1
>>>>
>
>
>
>
^ permalink raw reply [flat|nested] 56+ messages in thread
* [Patch v3 03/22] requirements.txt: Add python pip requirements file
2019-10-29 19:54 [Patch v3 00/22] Enable Phase 1 of EDK II CI Michael D Kinney
2019-10-29 19:54 ` [Patch v3 01/22] Maintainers.txt: Add continuous integration(CI) directories Michael D Kinney
2019-10-29 19:54 ` [Patch v3 02/22] .gitignore: Ignore python compiled files, extdeps, and vscode Michael D Kinney
@ 2019-10-29 19:54 ` Michael D Kinney
2019-10-30 2:27 ` Leif Lindholm
2019-10-29 19:54 ` [Patch v3 04/22] BaseTools/tools_def.template: Add RC_PATH define Michael D Kinney
` (19 subsequent siblings)
22 siblings, 1 reply; 56+ messages in thread
From: Michael D Kinney @ 2019-10-29 19:54 UTC (permalink / raw)
To: devel; +Cc: Sean Brogan, Andrew Fish, Laszlo Ersek, Leif Lindholm
From: Sean Brogan <sean.brogan@microsoft.com>
https://bugzilla.tianocore.org/show_bug.cgi?id=2315
Add pip requirements file that is used to install the
python pip modules build from the edk2-pytool-library and
edk2-pytool-extensions repositories.
These python modules provide the extensions required to
perform EDK II Continuous Integration(CI) builds.
Cc: Andrew Fish <afish@apple.com>
Cc: Laszlo Ersek <lersek@redhat.com>
Cc: Leif Lindholm <leif.lindholm@linaro.org>
Signed-off-by: Michael D Kinney <michael.d.kinney@intel.com>
---
requirements.txt | 2 ++
1 file changed, 2 insertions(+)
create mode 100644 requirements.txt
diff --git a/requirements.txt b/requirements.txt
new file mode 100644
index 0000000000..d4610f81c5
--- /dev/null
+++ b/requirements.txt
@@ -0,0 +1,2 @@
+edk2-pytool-library==0.10.*
+edk2-pytool-extensions==0.12.*
--
2.21.0.windows.1
^ permalink raw reply related [flat|nested] 56+ messages in thread* Re: [Patch v3 03/22] requirements.txt: Add python pip requirements file
2019-10-29 19:54 ` [Patch v3 03/22] requirements.txt: Add python pip requirements file Michael D Kinney
@ 2019-10-30 2:27 ` Leif Lindholm
2019-10-30 2:52 ` [edk2-devel] " Sean
2019-10-31 8:59 ` Laszlo Ersek
0 siblings, 2 replies; 56+ messages in thread
From: Leif Lindholm @ 2019-10-30 2:27 UTC (permalink / raw)
To: Michael D Kinney; +Cc: devel, Sean Brogan, Andrew Fish, Laszlo Ersek
On Tue, Oct 29, 2019 at 12:54:58PM -0700, Michael D Kinney wrote:
> From: Sean Brogan <sean.brogan@microsoft.com>
>
> https://bugzilla.tianocore.org/show_bug.cgi?id=2315
>
> Add pip requirements file that is used to install the
> python pip modules build from the edk2-pytool-library and
> edk2-pytool-extensions repositories.
>
> These python modules provide the extensions required to
> perform EDK II Continuous Integration(CI) builds.
>
> Cc: Andrew Fish <afish@apple.com>
> Cc: Laszlo Ersek <lersek@redhat.com>
> Cc: Leif Lindholm <leif.lindholm@linaro.org>
> Signed-off-by: Michael D Kinney <michael.d.kinney@intel.com>
> ---
> requirements.txt | 2 ++
This ia an extremely generic name for a file with very specific
content. Is this a magic name, or could we rename it (for example)
pip-requirements.txt?
/
Leif
> 1 file changed, 2 insertions(+)
> create mode 100644 requirements.txt
>
> diff --git a/requirements.txt b/requirements.txt
> new file mode 100644
> index 0000000000..d4610f81c5
> --- /dev/null
> +++ b/requirements.txt
> @@ -0,0 +1,2 @@
> +edk2-pytool-library==0.10.*
> +edk2-pytool-extensions==0.12.*
> --
> 2.21.0.windows.1
>
^ permalink raw reply [flat|nested] 56+ messages in thread* Re: [edk2-devel] [Patch v3 03/22] requirements.txt: Add python pip requirements file
2019-10-30 2:27 ` Leif Lindholm
@ 2019-10-30 2:52 ` Sean
2019-10-30 3:03 ` Leif Lindholm
2019-10-31 8:59 ` Laszlo Ersek
1 sibling, 1 reply; 56+ messages in thread
From: Sean @ 2019-10-30 2:52 UTC (permalink / raw)
To: Leif Lindholm, devel
[-- Attachment #1: Type: text/plain, Size: 672 bytes --]
It is a convention for projects using python. It definitely isn't required but there are some features that come for free when using that filename.
https://github.blog/2018-07-12-security-vulnerability-alerts-for-python/
and
https://help.github.com/en/github/visualizing-repository-data-with-graphs/listing-the-packages-that-a-repository-depends-on
You can also get more insight from github into dependencies and dependents.
example: https://github.com/tianocore/edk2-pytool-extensions/network/dependencies
I did notice on the example from above that github picked up the requirements.publisher.txt so i don't know what their pattern matching does exactly.
[-- Attachment #2: Type: text/html, Size: 1072 bytes --]
^ permalink raw reply [flat|nested] 56+ messages in thread
* Re: [edk2-devel] [Patch v3 03/22] requirements.txt: Add python pip requirements file
2019-10-30 2:52 ` [edk2-devel] " Sean
@ 2019-10-30 3:03 ` Leif Lindholm
2019-10-31 4:39 ` Michael D Kinney
0 siblings, 1 reply; 56+ messages in thread
From: Leif Lindholm @ 2019-10-30 3:03 UTC (permalink / raw)
To: devel, sean.brogan
OK, if it makes a difference for tools (and security updates), let's
try to keep it. (*grumble*)
*But* given its too-generic name, can we add a big bold comment header
to the file explaining what it is?
Some quick searching suggests lines starting with # are ignored, so
hopefully this should be possible?
Best Regards,
Leif
On Tue, Oct 29, 2019 at 07:52:48PM -0700, Sean via Groups.Io wrote:
> It is a convention for projects using python. It definitely isn't required but there are some features that come for free when using that filename.
>
> https://github.blog/2018-07-12-security-vulnerability-alerts-for-python/
> and
> https://help.github.com/en/github/visualizing-repository-data-with-graphs/listing-the-packages-that-a-repository-depends-on
>
> You can also get more insight from github into dependencies and dependents.
> example: https://github.com/tianocore/edk2-pytool-extensions/network/dependencies
>
> I did notice on the example from above that github picked up the requirements.publisher.txt so i don't know what their pattern matching does exactly.
>
>
>
^ permalink raw reply [flat|nested] 56+ messages in thread
* Re: [edk2-devel] [Patch v3 03/22] requirements.txt: Add python pip requirements file
2019-10-30 3:03 ` Leif Lindholm
@ 2019-10-31 4:39 ` Michael D Kinney
2019-10-31 4:58 ` Leif Lindholm
0 siblings, 1 reply; 56+ messages in thread
From: Michael D Kinney @ 2019-10-31 4:39 UTC (permalink / raw)
To: devel@edk2.groups.io, leif.lindholm@linaro.org,
sean.brogan@microsoft.com, Kinney, Michael D
Hi Leif,
I will see if we can add a proper file header with a description.
I may break this out into its own patch since it is useful to
install these pip modules to run CI tests and builds locally.
The pip modules are not only use in a CI agent.
Mike
> -----Original Message-----
> From: devel@edk2.groups.io <devel@edk2.groups.io> On
> Behalf Of Leif Lindholm
> Sent: Tuesday, October 29, 2019 8:03 PM
> To: devel@edk2.groups.io; sean.brogan@microsoft.com
> Subject: Re: [edk2-devel] [Patch v3 03/22]
> requirements.txt: Add python pip requirements file
>
> OK, if it makes a difference for tools (and security
> updates), let's try to keep it. (*grumble*)
>
> *But* given its too-generic name, can we add a big bold
> comment header to the file explaining what it is?
> Some quick searching suggests lines starting with # are
> ignored, so hopefully this should be possible?
>
> Best Regards,
>
> Leif
>
> On Tue, Oct 29, 2019 at 07:52:48PM -0700, Sean via
> Groups.Io wrote:
> > It is a convention for projects using python. It
> definitely isn't required but there are some features
> that come for free when using that filename.
> >
> > https://github.blog/2018-07-12-security-vulnerability-
> alerts-for-pytho
> > n/
> > and
> > https://help.github.com/en/github/visualizing-
> repository-data-with-gra
> > phs/listing-the-packages-that-a-repository-depends-on
> >
> > You can also get more insight from github into
> dependencies and dependents.
> > example:
> > https://github.com/tianocore/edk2-pytool-
> extensions/network/dependenci
> > es
> >
> > I did notice on the example from above that github
> picked up the requirements.publisher.txt so i don't know
> what their pattern matching does exactly.
> >
> >
> >
>
>
^ permalink raw reply [flat|nested] 56+ messages in thread
* Re: [edk2-devel] [Patch v3 03/22] requirements.txt: Add python pip requirements file
2019-10-31 4:39 ` Michael D Kinney
@ 2019-10-31 4:58 ` Leif Lindholm
2019-10-31 7:18 ` Sean
0 siblings, 1 reply; 56+ messages in thread
From: Leif Lindholm @ 2019-10-31 4:58 UTC (permalink / raw)
To: Kinney, Michael D; +Cc: edk2-devel-groups-io, Sean Brogan
[-- Attachment #1: Type: text/plain, Size: 1946 bytes --]
Agreed.
On Thu, 31 Oct 2019, 12:39 Kinney, Michael D, <michael.d.kinney@intel.com>
wrote:
> Hi Leif,
>
> I will see if we can add a proper file header with a description.
>
> I may break this out into its own patch since it is useful to
> install these pip modules to run CI tests and builds locally.
> The pip modules are not only use in a CI agent.
>
> Mike
>
> > -----Original Message-----
> > From: devel@edk2.groups.io <devel@edk2.groups.io> On
> > Behalf Of Leif Lindholm
> > Sent: Tuesday, October 29, 2019 8:03 PM
> > To: devel@edk2.groups.io; sean.brogan@microsoft.com
> > Subject: Re: [edk2-devel] [Patch v3 03/22]
> > requirements.txt: Add python pip requirements file
> >
> > OK, if it makes a difference for tools (and security
> > updates), let's try to keep it. (*grumble*)
> >
> > *But* given its too-generic name, can we add a big bold
> > comment header to the file explaining what it is?
> > Some quick searching suggests lines starting with # are
> > ignored, so hopefully this should be possible?
> >
> > Best Regards,
> >
> > Leif
> >
> > On Tue, Oct 29, 2019 at 07:52:48PM -0700, Sean via
> > Groups.Io wrote:
> > > It is a convention for projects using python. It
> > definitely isn't required but there are some features
> > that come for free when using that filename.
> > >
> > > https://github.blog/2018-07-12-security-vulnerability-
> > alerts-for-pytho
> > > n/
> > > and
> > > https://help.github.com/en/github/visualizing-
> > repository-data-with-gra
> > > phs/listing-the-packages-that-a-repository-depends-on
> > >
> > > You can also get more insight from github into
> > dependencies and dependents.
> > > example:
> > > https://github.com/tianocore/edk2-pytool-
> > extensions/network/dependenci
> > > es
> > >
> > > I did notice on the example from above that github
> > picked up the requirements.publisher.txt so i don't know
> > what their pattern matching does exactly.
> > >
> > >
> > >
> >
> >
>
>
[-- Attachment #2: Type: text/html, Size: 3243 bytes --]
^ permalink raw reply [flat|nested] 56+ messages in thread
* Re: [Patch v3 03/22] requirements.txt: Add python pip requirements file
2019-10-30 2:27 ` Leif Lindholm
2019-10-30 2:52 ` [edk2-devel] " Sean
@ 2019-10-31 8:59 ` Laszlo Ersek
1 sibling, 0 replies; 56+ messages in thread
From: Laszlo Ersek @ 2019-10-31 8:59 UTC (permalink / raw)
To: Leif Lindholm, Michael D Kinney; +Cc: devel, Sean Brogan, Andrew Fish
On 10/30/19 03:27, Leif Lindholm wrote:
> On Tue, Oct 29, 2019 at 12:54:58PM -0700, Michael D Kinney wrote:
>> From: Sean Brogan <sean.brogan@microsoft.com>
>>
>> https://bugzilla.tianocore.org/show_bug.cgi?id=2315
>>
>> Add pip requirements file that is used to install the
>> python pip modules build from the edk2-pytool-library and
>> edk2-pytool-extensions repositories.
>>
>> These python modules provide the extensions required to
>> perform EDK II Continuous Integration(CI) builds.
>>
>> Cc: Andrew Fish <afish@apple.com>
>> Cc: Laszlo Ersek <lersek@redhat.com>
>> Cc: Leif Lindholm <leif.lindholm@linaro.org>
>> Signed-off-by: Michael D Kinney <michael.d.kinney@intel.com>
>> ---
>> requirements.txt | 2 ++
>
> This ia an extremely generic name for a file with very specific
> content. Is this a magic name, or could we rename it (for example)
> pip-requirements.txt?
+1, I find the name of the file, especially with the file placed in the
project root dir, very confusing.
Thanks
Laszlo
>
> /
> Leif
>
>> 1 file changed, 2 insertions(+)
>> create mode 100644 requirements.txt
>>
>> diff --git a/requirements.txt b/requirements.txt
>> new file mode 100644
>> index 0000000000..d4610f81c5
>> --- /dev/null
>> +++ b/requirements.txt
>> @@ -0,0 +1,2 @@
>> +edk2-pytool-library==0.10.*
>> +edk2-pytool-extensions==0.12.*
>> --
>> 2.21.0.windows.1
>>
^ permalink raw reply [flat|nested] 56+ messages in thread
* [Patch v3 04/22] BaseTools/tools_def.template: Add RC_PATH define
2019-10-29 19:54 [Patch v3 00/22] Enable Phase 1 of EDK II CI Michael D Kinney
` (2 preceding siblings ...)
2019-10-29 19:54 ` [Patch v3 03/22] requirements.txt: Add python pip requirements file Michael D Kinney
@ 2019-10-29 19:54 ` Michael D Kinney
2019-11-05 8:18 ` Liming Gao
2019-10-29 19:55 ` [Patch v3 05/22] BaseTools: Add YAML files with path env and tool extdeps Michael D Kinney
` (18 subsequent siblings)
22 siblings, 1 reply; 56+ messages in thread
From: Michael D Kinney @ 2019-10-29 19:54 UTC (permalink / raw)
To: devel; +Cc: Sean Brogan, Bob Feng, Liming Gao, Kinney
From: Sean Brogan <sean.brogan@microsoft.com>
https://bugzilla.tianocore.org/show_bug.cgi?id=2315
Add use of RC_PATH define that provides the path to the resource
compiler that is typically provided in a Windows SDK. The path
changes with different Windows SDK releases. This define is set
to the WINSDK_PATH_FOR_RC_EXE environment variable. This
environment variable must be set to the path to the currently
installed resource compiler (rc.exe).
Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Signed-off-by: Kinney <michael.d.kinney@intel.com>
---
BaseTools/Conf/tools_def.template | 18 ++++++++++++++++--
1 file changed, 16 insertions(+), 2 deletions(-)
diff --git a/BaseTools/Conf/tools_def.template b/BaseTools/Conf/tools_def.template
index dce6c5875a..afa13a7cae 100755
--- a/BaseTools/Conf/tools_def.template
+++ b/BaseTools/Conf/tools_def.template
@@ -4,9 +4,18 @@
# Portions copyright (c) 2011 - 2014, ARM Ltd. All rights reserved.<BR>
# Copyright (c) 2015, Hewlett-Packard Development Company, L.P.<BR>
# (C) Copyright 2016 Hewlett Packard Enterprise Development LP<BR>
+# Copyright (c) Microsoft Corporation
#
# SPDX-License-Identifier: BSD-2-Clause-Patent
#
+#
+# Increase this version tag any time you want user to get warning about updating this
+# file in the Conf dir. By default it does not do update existing conf dirs.
+#
+# 2.00 - Initial version with changes for CI
+# - Change RC path to use plugin
+#
+#!VERSION=2.00
IDENTIFIER = Default TOOL_CHAIN_CONF
@@ -71,6 +80,11 @@ DEFINE VS2019_BIN_X64 = DEF(VS2019_BIN)\HostDEF(VS_HOST)\x64
DEFINE VS2019_BIN_ARM = DEF(VS2019_BIN)\HostDEF(VS_HOST)\arm
DEFINE VS2019_BIN_AARCH64 = DEF(VS2019_BIN)\HostDEF(VS_HOST)\arm64
+#
+# Resource compiler
+#
+DEFINE RC_PATH = ENV(WINSDK_PATH_FOR_RC_EXE)\rc.exe
+
DEFINE WINSDK_BIN = ENV(WINSDK_PREFIX)
DEFINE WINSDKx86_BIN = ENV(WINSDKx86_PREFIX)
@@ -1566,7 +1580,7 @@ NOOPT_VS2015x86_X64_DLINK_FLAGS = /NOLOGO /NODEFAULTLIB /IGNORE:4001 /OPT:REF
*_VS2017_*_MAKE_PATH = DEF(VS2017_BIN_HOST)\nmake.exe
*_VS2017_*_MAKE_FLAG = /nologo
-*_VS2017_*_RC_PATH = DEF(WINSDK10_BIN)\rc.exe
+*_VS2017_*_RC_PATH = DEF(RC_PATH)
*_VS2017_*_MAKE_FLAGS = /nologo
*_VS2017_*_SLINK_FLAGS = /NOLOGO /LTCG
@@ -1729,7 +1743,7 @@ NOOPT_VS2017_AARCH64_DLINK_FLAGS = /NOLOGO /NODEFAULTLIB /IGNORE:4001 /OPT:REF
*_VS2019_*_MAKE_PATH = DEF(VS2019_BIN_HOST)\nmake.exe
*_VS2019_*_MAKE_FLAG = /nologo
-*_VS2019_*_RC_PATH = DEF(WINSDK10_BIN)\rc.exe
+*_VS2019_*_RC_PATH = DEF(RC_PATH)
*_VS2019_*_MAKE_FLAGS = /nologo
*_VS2019_*_SLINK_FLAGS = /NOLOGO /LTCG
--
2.21.0.windows.1
^ permalink raw reply related [flat|nested] 56+ messages in thread* Re: [Patch v3 04/22] BaseTools/tools_def.template: Add RC_PATH define
2019-10-29 19:54 ` [Patch v3 04/22] BaseTools/tools_def.template: Add RC_PATH define Michael D Kinney
@ 2019-11-05 8:18 ` Liming Gao
2019-11-07 0:32 ` Michael D Kinney
0 siblings, 1 reply; 56+ messages in thread
From: Liming Gao @ 2019-11-05 8:18 UTC (permalink / raw)
To: Kinney, Michael D, devel@edk2.groups.io; +Cc: Sean Brogan, Feng, Bob C
Mike:
To avoid the impact of the developer, we should update edk2\BaseTools\set_vsprefix_envs.bat to auto set WINSDK_PATH_FOR_RC_EXE env.
Thanks
Liming
>-----Original Message-----
>From: Kinney, Michael D
>Sent: Wednesday, October 30, 2019 3:55 AM
>To: devel@edk2.groups.io
>Cc: Sean Brogan <sean.brogan@microsoft.com>; Feng, Bob C
><bob.c.feng@intel.com>; Gao, Liming <liming.gao@intel.com>; Kinney,
>Michael D <michael.d.kinney@intel.com>
>Subject: [Patch v3 04/22] BaseTools/tools_def.template: Add RC_PATH define
>
>From: Sean Brogan <sean.brogan@microsoft.com>
>
>https://bugzilla.tianocore.org/show_bug.cgi?id=2315
>
>Add use of RC_PATH define that provides the path to the resource
>compiler that is typically provided in a Windows SDK. The path
>changes with different Windows SDK releases. This define is set
>to the WINSDK_PATH_FOR_RC_EXE environment variable. This
>environment variable must be set to the path to the currently
>installed resource compiler (rc.exe).
>
>Cc: Bob Feng <bob.c.feng@intel.com>
>Cc: Liming Gao <liming.gao@intel.com>
>Signed-off-by: Kinney <michael.d.kinney@intel.com>
>---
> BaseTools/Conf/tools_def.template | 18 ++++++++++++++++--
> 1 file changed, 16 insertions(+), 2 deletions(-)
>
>diff --git a/BaseTools/Conf/tools_def.template
>b/BaseTools/Conf/tools_def.template
>index dce6c5875a..afa13a7cae 100755
>--- a/BaseTools/Conf/tools_def.template
>+++ b/BaseTools/Conf/tools_def.template
>@@ -4,9 +4,18 @@
> # Portions copyright (c) 2011 - 2014, ARM Ltd. All rights reserved.<BR>
> # Copyright (c) 2015, Hewlett-Packard Development Company, L.P.<BR>
> # (C) Copyright 2016 Hewlett Packard Enterprise Development LP<BR>
>+# Copyright (c) Microsoft Corporation
> #
> # SPDX-License-Identifier: BSD-2-Clause-Patent
> #
>+#
>+# Increase this version tag any time you want user to get warning about
>updating this
>+# file in the Conf dir. By default it does not do update existing conf dirs.
>+#
>+# 2.00 - Initial version with changes for CI
>+# - Change RC path to use plugin
>+#
>+#!VERSION=2.00
>
> IDENTIFIER = Default TOOL_CHAIN_CONF
>
>@@ -71,6 +80,11 @@ DEFINE VS2019_BIN_X64 =
>DEF(VS2019_BIN)\HostDEF(VS_HOST)\x64
> DEFINE VS2019_BIN_ARM = DEF(VS2019_BIN)\HostDEF(VS_HOST)\arm
> DEFINE VS2019_BIN_AARCH64 =
>DEF(VS2019_BIN)\HostDEF(VS_HOST)\arm64
>
>+#
>+# Resource compiler
>+#
>+DEFINE RC_PATH = ENV(WINSDK_PATH_FOR_RC_EXE)\rc.exe
>+
> DEFINE WINSDK_BIN = ENV(WINSDK_PREFIX)
> DEFINE WINSDKx86_BIN = ENV(WINSDKx86_PREFIX)
>
>@@ -1566,7 +1580,7 @@ NOOPT_VS2015x86_X64_DLINK_FLAGS =
>/NOLOGO /NODEFAULTLIB /IGNORE:4001 /OPT:REF
>
> *_VS2017_*_MAKE_PATH = DEF(VS2017_BIN_HOST)\nmake.exe
> *_VS2017_*_MAKE_FLAG = /nologo
>-*_VS2017_*_RC_PATH = DEF(WINSDK10_BIN)\rc.exe
>+*_VS2017_*_RC_PATH = DEF(RC_PATH)
>
> *_VS2017_*_MAKE_FLAGS = /nologo
> *_VS2017_*_SLINK_FLAGS = /NOLOGO /LTCG
>@@ -1729,7 +1743,7 @@ NOOPT_VS2017_AARCH64_DLINK_FLAGS =
>/NOLOGO /NODEFAULTLIB /IGNORE:4001 /OPT:REF
>
> *_VS2019_*_MAKE_PATH = DEF(VS2019_BIN_HOST)\nmake.exe
> *_VS2019_*_MAKE_FLAG = /nologo
>-*_VS2019_*_RC_PATH = DEF(WINSDK10_BIN)\rc.exe
>+*_VS2019_*_RC_PATH = DEF(RC_PATH)
>
> *_VS2019_*_MAKE_FLAGS = /nologo
> *_VS2019_*_SLINK_FLAGS = /NOLOGO /LTCG
>--
>2.21.0.windows.1
^ permalink raw reply [flat|nested] 56+ messages in thread
* Re: [Patch v3 04/22] BaseTools/tools_def.template: Add RC_PATH define
2019-11-05 8:18 ` Liming Gao
@ 2019-11-07 0:32 ` Michael D Kinney
0 siblings, 0 replies; 56+ messages in thread
From: Michael D Kinney @ 2019-11-07 0:32 UTC (permalink / raw)
To: Gao, Liming, devel@edk2.groups.io, Kinney, Michael D
Cc: Sean Brogan, Feng, Bob C
Liming,
Thanks. I have addressed this in a V4 branch I have prepared.
Mike
> -----Original Message-----
> From: Gao, Liming <liming.gao@intel.com>
> Sent: Tuesday, November 5, 2019 12:18 AM
> To: Kinney, Michael D <michael.d.kinney@intel.com>;
> devel@edk2.groups.io
> Cc: Sean Brogan <sean.brogan@microsoft.com>; Feng, Bob C
> <bob.c.feng@intel.com>
> Subject: RE: [Patch v3 04/22]
> BaseTools/tools_def.template: Add RC_PATH define
>
> Mike:
> To avoid the impact of the developer, we should update
> edk2\BaseTools\set_vsprefix_envs.bat to auto set
> WINSDK_PATH_FOR_RC_EXE env.
>
> Thanks
> Liming
> >-----Original Message-----
> >From: Kinney, Michael D
> >Sent: Wednesday, October 30, 2019 3:55 AM
> >To: devel@edk2.groups.io
> >Cc: Sean Brogan <sean.brogan@microsoft.com>; Feng, Bob
> C
> ><bob.c.feng@intel.com>; Gao, Liming
> <liming.gao@intel.com>; Kinney,
> >Michael D <michael.d.kinney@intel.com>
> >Subject: [Patch v3 04/22] BaseTools/tools_def.template:
> Add RC_PATH
> >define
> >
> >From: Sean Brogan <sean.brogan@microsoft.com>
> >
> >https://bugzilla.tianocore.org/show_bug.cgi?id=2315
> >
> >Add use of RC_PATH define that provides the path to the
> resource
> >compiler that is typically provided in a Windows SDK.
> The path changes
> >with different Windows SDK releases. This define is
> set to the
> >WINSDK_PATH_FOR_RC_EXE environment variable. This
> environment variable
> >must be set to the path to the currently installed
> resource compiler
> >(rc.exe).
> >
> >Cc: Bob Feng <bob.c.feng@intel.com>
> >Cc: Liming Gao <liming.gao@intel.com>
> >Signed-off-by: Kinney <michael.d.kinney@intel.com>
> >---
> > BaseTools/Conf/tools_def.template | 18
> ++++++++++++++++--
> > 1 file changed, 16 insertions(+), 2 deletions(-)
> >
> >diff --git a/BaseTools/Conf/tools_def.template
> >b/BaseTools/Conf/tools_def.template
> >index dce6c5875a..afa13a7cae 100755
> >--- a/BaseTools/Conf/tools_def.template
> >+++ b/BaseTools/Conf/tools_def.template
> >@@ -4,9 +4,18 @@
> > # Portions copyright (c) 2011 - 2014, ARM Ltd. All
> rights
> >reserved.<BR> # Copyright (c) 2015, Hewlett-Packard
> Development
> >Company, L.P.<BR> # (C) Copyright 2016 Hewlett
> Packard Enterprise
> >Development LP<BR>
> >+# Copyright (c) Microsoft Corporation
> > #
> > # SPDX-License-Identifier: BSD-2-Clause-Patent #
> >+#
> >+# Increase this version tag any time you want user to
> get warning
> >+about
> >updating this
> >+# file in the Conf dir. By default it does not do
> update existing conf dirs.
> >+#
> >+# 2.00 - Initial version with changes for CI
> >+# - Change RC path to use plugin
> >+#
> >+#!VERSION=2.00
> >
> > IDENTIFIER = Default TOOL_CHAIN_CONF
> >
> >@@ -71,6 +80,11 @@ DEFINE VS2019_BIN_X64 =
> >DEF(VS2019_BIN)\HostDEF(VS_HOST)\x64
> > DEFINE VS2019_BIN_ARM =
> DEF(VS2019_BIN)\HostDEF(VS_HOST)\arm
> > DEFINE VS2019_BIN_AARCH64 =
> >DEF(VS2019_BIN)\HostDEF(VS_HOST)\arm64
> >
> >+#
> >+# Resource compiler
> >+#
> >+DEFINE RC_PATH = ENV(WINSDK_PATH_FOR_RC_EXE)\rc.exe
> >+
> > DEFINE WINSDK_BIN = ENV(WINSDK_PREFIX)
> > DEFINE WINSDKx86_BIN = ENV(WINSDKx86_PREFIX)
> >
> >@@ -1566,7 +1580,7 @@ NOOPT_VS2015x86_X64_DLINK_FLAGS
> =
> >/NOLOGO /NODEFAULTLIB /IGNORE:4001 /OPT:REF
> >
> > *_VS2017_*_MAKE_PATH =
> DEF(VS2017_BIN_HOST)\nmake.exe
> > *_VS2017_*_MAKE_FLAG = /nologo
> >-*_VS2017_*_RC_PATH = DEF(WINSDK10_BIN)\rc.exe
> >+*_VS2017_*_RC_PATH = DEF(RC_PATH)
> >
> > *_VS2017_*_MAKE_FLAGS = /nologo
> > *_VS2017_*_SLINK_FLAGS = /NOLOGO /LTCG
> >@@ -1729,7 +1743,7 @@ NOOPT_VS2017_AARCH64_DLINK_FLAGS
> =
> >/NOLOGO /NODEFAULTLIB /IGNORE:4001 /OPT:REF
> >
> > *_VS2019_*_MAKE_PATH =
> DEF(VS2019_BIN_HOST)\nmake.exe
> > *_VS2019_*_MAKE_FLAG = /nologo
> >-*_VS2019_*_RC_PATH = DEF(WINSDK10_BIN)\rc.exe
> >+*_VS2019_*_RC_PATH = DEF(RC_PATH)
> >
> > *_VS2019_*_MAKE_FLAGS = /nologo
> > *_VS2019_*_SLINK_FLAGS = /NOLOGO /LTCG
> >--
> >2.21.0.windows.1
^ permalink raw reply [flat|nested] 56+ messages in thread
* [Patch v3 05/22] BaseTools: Add YAML files with path env and tool extdeps
2019-10-29 19:54 [Patch v3 00/22] Enable Phase 1 of EDK II CI Michael D Kinney
` (3 preceding siblings ...)
2019-10-29 19:54 ` [Patch v3 04/22] BaseTools/tools_def.template: Add RC_PATH define Michael D Kinney
@ 2019-10-29 19:55 ` Michael D Kinney
2019-10-29 19:55 ` [Patch v3 06/22] BaseTools: Add BaseTools plugins to support CI Michael D Kinney
` (17 subsequent siblings)
22 siblings, 0 replies; 56+ messages in thread
From: Michael D Kinney @ 2019-10-29 19:55 UTC (permalink / raw)
To: devel; +Cc: Sean Brogan, Bob Feng, Liming Gao
From: Sean Brogan <sean.brogan@microsoft.com>
https://bugzilla.tianocore.org/show_bug.cgi?id=2315
Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Signed-off-by: Michael D Kinney <michael.d.kinney@intel.com>
---
BaseTools/Bin/gcc_aarch64_linux_ext_dep.yaml | 21 +++
BaseTools/Bin/gcc_arm_linux_ext_dep.yaml | 21 +++
BaseTools/Bin/iasl_ext_dep.yaml | 21 +++
BaseTools/Bin/nasm_ext_dep.yaml | 18 ++
.../BinWrappers/PosixLike/posix_path_env.yaml | 10 ++
.../WindowsLike/win_build_tools_path_env.yaml | 10 ++
BaseTools/Edk2ToolsBuild.py | 163 ++++++++++++++++++
.../basetool_tiano_python_path_env.yaml | 11 ++
BaseTools/basetools_calling_path_env.yaml | 11 ++
BaseTools/basetools_path_env.yaml | 11 ++
10 files changed, 297 insertions(+)
create mode 100644 BaseTools/Bin/gcc_aarch64_linux_ext_dep.yaml
create mode 100644 BaseTools/Bin/gcc_arm_linux_ext_dep.yaml
create mode 100644 BaseTools/Bin/iasl_ext_dep.yaml
create mode 100644 BaseTools/Bin/nasm_ext_dep.yaml
create mode 100644 BaseTools/BinWrappers/PosixLike/posix_path_env.yaml
create mode 100644 BaseTools/BinWrappers/WindowsLike/win_build_tools_path_env.yaml
create mode 100644 BaseTools/Edk2ToolsBuild.py
create mode 100644 BaseTools/Source/Python/basetool_tiano_python_path_env.yaml
create mode 100644 BaseTools/basetools_calling_path_env.yaml
create mode 100644 BaseTools/basetools_path_env.yaml
diff --git a/BaseTools/Bin/gcc_aarch64_linux_ext_dep.yaml b/BaseTools/Bin/gcc_aarch64_linux_ext_dep.yaml
new file mode 100644
index 0000000000..69648c89d8
--- /dev/null
+++ b/BaseTools/Bin/gcc_aarch64_linux_ext_dep.yaml
@@ -0,0 +1,21 @@
+##
+# Download GCC AARCH64 compiler from Linaro's release site
+# Set shell variable GCC5_AARCH64_INSTALL to this folder
+#
+# This is only downloaded when a build activates scope gcc_aarch64_linux
+#
+# Copyright (c) 2019, Microsoft Corporation
+# SPDX-License-Identifier: BSD-2-Clause-Patent
+##
+{
+ "scope": "gcc_aarch64_linux",
+ "type": "web",
+ "name": "gcc_aarch64_linux",
+ "source": "http://releases.linaro.org/components/toolchain/binaries/7.4-2019.02/aarch64-linux-gnu/gcc-linaro-7.4.1-2019.02-x86_64_aarch64-linux-gnu.tar.xz",
+ "version": "7.4.1",
+ "sha256": "27f1dc2c491ed61ae8f0d4b0c11de59cd2f7dd9c94761ee7153006fcac1bf9ab",
+ "compression_type": "tar",
+ "internal_path": "/gcc-linaro-7.4.1-2019.02-x86_64_aarch64-linux-gnu/",
+ "flags": ["set_shell_var", ],
+ "var_name": "GCC5_AARCH64_INSTALL"
+}
diff --git a/BaseTools/Bin/gcc_arm_linux_ext_dep.yaml b/BaseTools/Bin/gcc_arm_linux_ext_dep.yaml
new file mode 100644
index 0000000000..e108aaad2a
--- /dev/null
+++ b/BaseTools/Bin/gcc_arm_linux_ext_dep.yaml
@@ -0,0 +1,21 @@
+##
+# Download GCC ARM compiler from Linaro's release site
+# Set shell variable GCC5_ARM_INSTALL to this folder
+#
+# This is only downloaded when a build activates scope gcc_arm_linux
+#
+# Copyright (c) 2019, Microsoft Corporation
+# SPDX-License-Identifier: BSD-2-Clause-Patent
+##
+{
+ "scope": "gcc_arm_linux",
+ "type": "web",
+ "name": "gcc_arm_linux",
+ "source": "https://releases.linaro.org/components/toolchain/binaries/7.4-2019.02/arm-linux-gnueabihf/gcc-linaro-7.4.1-2019.02-x86_64_arm-linux-gnueabihf.tar.xz",
+ "version": "7.4.1",
+ "sha256": "3C951CF1941D0FA06D64CC0D5E88612B209D8123B273FA26C16D70BD7BC6B163",
+ "compression_type": "tar",
+ "internal_path": "/gcc-linaro-7.4.1-2019.02-x86_64_arm-linux-gnueabihf/",
+ "flags": ["set_shell_var", ],
+ "var_name": "GCC5_ARM_INSTALL"
+}
diff --git a/BaseTools/Bin/iasl_ext_dep.yaml b/BaseTools/Bin/iasl_ext_dep.yaml
new file mode 100644
index 0000000000..1c63921367
--- /dev/null
+++ b/BaseTools/Bin/iasl_ext_dep.yaml
@@ -0,0 +1,21 @@
+##
+# Download iasl executable tool from a nuget.org package
+# - package contains different binaries based on host
+# Add the folder with the tool to the path
+#
+# This is only downloaded for scope cibuild thus
+# should have no impact on the asl compiler used by any
+# given platform to build.
+#
+# Copyright (c) 2019, Microsoft Corporation
+# SPDX-License-Identifier: BSD-2-Clause-Patent
+##
+{
+ "id": "iasl-ci-1",
+ "scope": "cibuild",
+ "type": "nuget",
+ "name": "iasl",
+ "source": "https://api.nuget.org/v3/index.json",
+ "version": "20190215.0.0",
+ "flags": ["set_path", "host_specific"]
+}
diff --git a/BaseTools/Bin/nasm_ext_dep.yaml b/BaseTools/Bin/nasm_ext_dep.yaml
new file mode 100644
index 0000000000..8894516b5a
--- /dev/null
+++ b/BaseTools/Bin/nasm_ext_dep.yaml
@@ -0,0 +1,18 @@
+##
+# Download nasm x86 assembler executable tool from a nuget.org package
+# - package contains different binaries based on host
+# Put on the tool on the path
+#
+#
+# Copyright (c) 2019, Microsoft Corporation
+# SPDX-License-Identifier: BSD-2-Clause-Patent
+##
+{
+ "id": "nasm-1",
+ "scope": "edk2-build",
+ "type": "nuget",
+ "name": "mu_nasm",
+ "source": "https://api.nuget.org/v3/index.json",
+ "version": "2.14.02",
+ "flags": ["set_path", "host_specific"]
+}
diff --git a/BaseTools/BinWrappers/PosixLike/posix_path_env.yaml b/BaseTools/BinWrappers/PosixLike/posix_path_env.yaml
new file mode 100644
index 0000000000..eec65ed7db
--- /dev/null
+++ b/BaseTools/BinWrappers/PosixLike/posix_path_env.yaml
@@ -0,0 +1,10 @@
+##
+# Set this folder on the path for all linux builds
+#
+# Copyright (c) 2019, Microsoft Corporation
+# SPDX-License-Identifier: BSD-2-Clause-Patent
+##
+{
+ "scope": "global-nix",
+ "flags": ["set_path"]
+}
diff --git a/BaseTools/BinWrappers/WindowsLike/win_build_tools_path_env.yaml b/BaseTools/BinWrappers/WindowsLike/win_build_tools_path_env.yaml
new file mode 100644
index 0000000000..340724f39f
--- /dev/null
+++ b/BaseTools/BinWrappers/WindowsLike/win_build_tools_path_env.yaml
@@ -0,0 +1,10 @@
+##
+# Add this folder to the path on Windows
+#
+# Copyright (c) 2019, Microsoft Corporation
+# SPDX-License-Identifier: BSD-2-Clause-Patent
+##
+{
+ "scope": "global-win",
+ "flags": ["set_path"]
+}
diff --git a/BaseTools/Edk2ToolsBuild.py b/BaseTools/Edk2ToolsBuild.py
new file mode 100644
index 0000000000..057d2e9e06
--- /dev/null
+++ b/BaseTools/Edk2ToolsBuild.py
@@ -0,0 +1,163 @@
+# @file Edk2ToolsBuild.py
+# Invocable class that builds the basetool c files.
+#
+# Supports VS2017, VS2019, and GCC5
+##
+# Copyright (c) Microsoft Corporation
+#
+# SPDX-License-Identifier: BSD-2-Clause-Patent
+##
+import os
+import sys
+import logging
+import argparse
+from edk2toolext import edk2_logging
+from edk2toolext.environment import self_describing_environment
+from edk2toolext.base_abstract_invocable import BaseAbstractInvocable
+from edk2toollib.utility_functions import RunCmd
+from edk2toollib.windows.locate_tools import QueryVcVariables
+
+
+class Edk2ToolsBuild(BaseAbstractInvocable):
+
+ def ParseCommandLineOptions(self):
+ ''' parse arguments '''
+ ParserObj = argparse.ArgumentParser()
+ ParserObj.add_argument("-t", "--tool_chain_tag", dest="tct", default="VS2017",
+ help="Set the toolchain used to compile the build tools")
+ args = ParserObj.parse_args()
+ self.tool_chain_tag = args.tct
+
+ def GetWorkspaceRoot(self):
+ ''' Return the workspace root for initializing the SDE '''
+
+ # this is the bastools dir...not the traditional EDK2 workspace root
+ return os.path.dirname(os.path.abspath(__file__))
+
+ def GetActiveScopes(self):
+ ''' return tuple containing scopes that should be active for this process '''
+
+ # for now don't use scopes
+ return ('global',)
+
+ def GetLoggingLevel(self, loggerType):
+ ''' Get the logging level for a given type (return Logging.Level)
+ base == lowest logging level supported
+ con == Screen logging
+ txt == plain text file logging
+ md == markdown file logging
+ '''
+ if(loggerType == "con"):
+ return logging.ERROR
+ else:
+ return logging.DEBUG
+
+ def GetLoggingFolderRelativeToRoot(self):
+ ''' Return a path to folder for log files '''
+ return "BaseToolsBuild"
+
+ def GetVerifyCheckRequired(self):
+ ''' Will call self_describing_environment.VerifyEnvironment if this returns True '''
+ return True
+
+ def GetLoggingFileName(self, loggerType):
+ ''' Get the logging file name for the type.
+ Return None if the logger shouldn't be created
+
+ base == lowest logging level supported
+ con == Screen logging
+ txt == plain text file logging
+ md == markdown file logging
+ '''
+ return "BASETOOLS_BUILD"
+
+ def WritePathEnvFile(self, OutputDir):
+ ''' Write a PyTool path env file for future PyTool based edk2 builds'''
+ content = '''##
+# Set shell variable EDK_TOOLS_BIN to this folder
+#
+# Autogenerated by Edk2ToolsBuild.py
+#
+# Copyright (c), Microsoft Corporation
+# SPDX-License-Identifier: BSD-2-Clause-Patent
+##
+{
+ "id": "You-Built-BaseTools",
+ "scope": "edk2-build",
+ "flags": ["set_shell_var", "set_path"],
+ "var_name": "EDK_TOOLS_BIN"
+}
+'''
+ with open(os.path.join(OutputDir, "basetoolsbin_path_env.yaml"), "w") as f:
+ f.write(content)
+
+ def Go(self):
+ logging.info("Running Python version: " + str(sys.version_info))
+
+ (build_env, shell_env) = self_describing_environment.BootstrapEnvironment(
+ self.GetWorkspaceRoot(), self.GetActiveScopes())
+
+ # # Bind our current execution environment into the shell vars.
+ ph = os.path.dirname(sys.executable)
+ if " " in ph:
+ ph = '"' + ph + '"'
+ shell_env.set_shell_var("PYTHON_HOME", ph)
+ # PYTHON_COMMAND is required to be set for using edk2 python builds.
+ pc = sys.executable
+ if " " in pc:
+ pc = '"' + pc + '"'
+ shell_env.set_shell_var("PYTHON_COMMAND", pc)
+
+ if self.tool_chain_tag.lower().startswith("vs"):
+
+ # # Update environment with required VC vars.
+ interesting_keys = ["ExtensionSdkDir", "INCLUDE", "LIB"]
+ interesting_keys.extend(
+ ["LIBPATH", "Path", "UniversalCRTSdkDir", "UCRTVersion", "WindowsLibPath", "WindowsSdkBinPath"])
+ interesting_keys.extend(
+ ["WindowsSdkDir", "WindowsSdkVerBinPath", "WindowsSDKVersion", "VCToolsInstallDir"])
+ vc_vars = QueryVcVariables(
+ interesting_keys, 'x86', vs_version=self.tool_chain_tag.lower())
+ for key in vc_vars.keys():
+ logging.debug(f"Var - {key} = {vc_vars[key]}")
+ if key.lower() == 'path':
+ shell_env.insert_path(vc_vars[key])
+ else:
+ shell_env.set_shell_var(key, vc_vars[key])
+
+ self.OutputDir = os.path.join(
+ shell_env.get_shell_var("EDK_TOOLS_PATH"), "Bin", "Win32")
+
+ # compiled tools need to be added to path because antlr is referenced
+ shell_env.insert_path(self.OutputDir)
+
+ # Actually build the tools.
+ ret = RunCmd('nmake.exe', None,
+ workingdir=shell_env.get_shell_var("EDK_TOOLS_PATH"))
+ if ret != 0:
+ raise Exception("Failed to build.")
+
+ self.WritePathEnvFile(self.OutputDir)
+ return ret
+
+ elif self.tool_chain_tag.lower().startswith("gcc"):
+ ret = RunCmd("make", "-C .", workingdir=shell_env.get_shell_var("EDK_TOOLS_PATH"))
+ if ret != 0:
+ raise Exception("Failed to build.")
+
+ self.OutputDir = os.path.join(
+ shell_env.get_shell_var("EDK_TOOLS_PATH"), "Source", "C", "bin")
+
+ self.WritePathEnvFile(self.OutputDir)
+ return ret
+
+ logging.critical("Tool Chain not supported")
+ return -1
+
+
+def main():
+ Edk2ToolsBuild().Invoke()
+
+
+if __name__ == "__main__":
+ main()
diff --git a/BaseTools/Source/Python/basetool_tiano_python_path_env.yaml b/BaseTools/Source/Python/basetool_tiano_python_path_env.yaml
new file mode 100644
index 0000000000..1edc8810ce
--- /dev/null
+++ b/BaseTools/Source/Python/basetool_tiano_python_path_env.yaml
@@ -0,0 +1,11 @@
+##
+# Add this folder to the pypath so modules can be easily
+# loaded
+#
+# Copyright (c) 2019, Microsoft Corporation
+# SPDX-License-Identifier: BSD-2-Clause-Patent
+##
+{
+ "scope": "global",
+ "flags": ["set_pypath"]
+}
diff --git a/BaseTools/basetools_calling_path_env.yaml b/BaseTools/basetools_calling_path_env.yaml
new file mode 100644
index 0000000000..cfdbcea104
--- /dev/null
+++ b/BaseTools/basetools_calling_path_env.yaml
@@ -0,0 +1,11 @@
+##
+# Set shell variable BASE_TOOL_PATH to this folder
+#
+# Copyright (c) 2019, Microsoft Corporation
+# SPDX-License-Identifier: BSD-2-Clause-Patent
+##
+{
+ "scope": "global",
+ "flags": ["set_shell_var"],
+ "var_name": "BASE_TOOLS_PATH"
+}
diff --git a/BaseTools/basetools_path_env.yaml b/BaseTools/basetools_path_env.yaml
new file mode 100644
index 0000000000..f64a3b8d45
--- /dev/null
+++ b/BaseTools/basetools_path_env.yaml
@@ -0,0 +1,11 @@
+##
+# Set shell variable EDK_TOOL_PATH to this folder
+#
+# Copyright (c) 2019, Microsoft Corporation
+# SPDX-License-Identifier: BSD-2-Clause-Patent
+##
+{
+ "scope": "global",
+ "flags": ["set_shell_var", "set_build_var"],
+ "var_name": "EDK_TOOLS_PATH"
+}
--
2.21.0.windows.1
^ permalink raw reply related [flat|nested] 56+ messages in thread* [Patch v3 06/22] BaseTools: Add BaseTools plugins to support CI
2019-10-29 19:54 [Patch v3 00/22] Enable Phase 1 of EDK II CI Michael D Kinney
` (4 preceding siblings ...)
2019-10-29 19:55 ` [Patch v3 05/22] BaseTools: Add YAML files with path env and tool extdeps Michael D Kinney
@ 2019-10-29 19:55 ` Michael D Kinney
2019-10-29 19:55 ` [Patch v3 07/22] .pytool/Plugin: Add CI plugins Michael D Kinney
` (16 subsequent siblings)
22 siblings, 0 replies; 56+ messages in thread
From: Michael D Kinney @ 2019-10-29 19:55 UTC (permalink / raw)
To: devel; +Cc: Sean Brogan, Bob Feng, Liming Gao
From: Sean Brogan <sean.brogan@microsoft.com>
https://bugzilla.tianocore.org/show_bug.cgi?id=2315
Add the following plugins that are required to support
EDK II Continuous Integration (CI) builds. These plugins
are added to BaseTools because that support EDK II BaseTools
features.
* BuildToolsReportGenerator
* LinuxGcc5ToolChain
* WindowsResourceCompiler
* WindowsVsToolChain
Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Liming Gao <liming.gao@intel.com>
Signed-off-by: Michael D Kinney <michael.d.kinney@intel.com>
---
.../BuildToolsReportGenerator.py | 69 ++++++++++
.../BuildToolsReportGenerator_plug_in.yaml | 12 ++
.../BuildToolsReport_Template.html | 126 ++++++++++++++++++
.../LinuxGcc5ToolChain/LinuxGcc5ToolChain.py | 85 ++++++++++++
.../LinuxGcc5ToolChain_plug_in.yaml | 12 ++
.../WindowsResourceCompiler/WinRcPath.py | 29 ++++
.../WinRcPath_plug_in.yaml | 13 ++
.../WindowsVsToolChain/WindowsVsToolChain.py | 126 ++++++++++++++++++
.../WindowsVsToolChain_plug_in.yaml | 11 ++
9 files changed, 483 insertions(+)
create mode 100644 BaseTools/Plugin/BuildToolsReport/BuildToolsReportGenerator.py
create mode 100644 BaseTools/Plugin/BuildToolsReport/BuildToolsReportGenerator_plug_in.yaml
create mode 100644 BaseTools/Plugin/BuildToolsReport/BuildToolsReport_Template.html
create mode 100644 BaseTools/Plugin/LinuxGcc5ToolChain/LinuxGcc5ToolChain.py
create mode 100644 BaseTools/Plugin/LinuxGcc5ToolChain/LinuxGcc5ToolChain_plug_in.yaml
create mode 100644 BaseTools/Plugin/WindowsResourceCompiler/WinRcPath.py
create mode 100644 BaseTools/Plugin/WindowsResourceCompiler/WinRcPath_plug_in.yaml
create mode 100644 BaseTools/Plugin/WindowsVsToolChain/WindowsVsToolChain.py
create mode 100644 BaseTools/Plugin/WindowsVsToolChain/WindowsVsToolChain_plug_in.yaml
diff --git a/BaseTools/Plugin/BuildToolsReport/BuildToolsReportGenerator.py b/BaseTools/Plugin/BuildToolsReport/BuildToolsReportGenerator.py
new file mode 100644
index 0000000000..d5df9f2f9b
--- /dev/null
+++ b/BaseTools/Plugin/BuildToolsReport/BuildToolsReportGenerator.py
@@ -0,0 +1,69 @@
+##
+# Copyright (c) Microsoft Corporation. All rights reserved.
+# SPDX-License-Identifier: BSD-2-Clause-Patent
+##
+import os
+import logging
+import json
+
+try:
+ from edk2toolext.environment.plugintypes.uefi_build_plugin import IUefiBuildPlugin
+
+ class BuildToolsReportGenerator(IUefiBuildPlugin):
+ def do_report(self, thebuilder):
+ try:
+ from edk2toolext.environment import version_aggregator
+ except ImportError:
+ logging.critical("Loading BuildToolsReportGenerator failed, please update your Edk2-PyTool-Extensions")
+ return 0
+
+ OutputReport = os.path.join(thebuilder.env.GetValue("BUILD_OUTPUT_BASE"), "BUILD_TOOLS_REPORT")
+ OutputReport = os.path.normpath(OutputReport)
+ if not os.path.isdir(os.path.dirname(OutputReport)):
+ os.makedirs(os.path.dirname(OutputReport))
+
+ Report = BuildToolsReport()
+ Report.MakeReport(version_aggregator.GetVersionAggregator().GetAggregatedVersionInformation(), OutputReport=OutputReport)
+
+ def do_pre_build(self, thebuilder):
+ self.do_report(thebuilder)
+ return 0
+
+ def do_post_build(self, thebuilder):
+ self.do_report(thebuilder)
+ return 0
+
+except ImportError:
+ pass
+
+
+class BuildToolsReport(object):
+ MY_FOLDER = os.path.dirname(os.path.realpath(__file__))
+ VERSION = "1.00"
+
+ def __init__(self):
+ pass
+
+ def MakeReport(self, BuildTools, OutputReport="BuildToolsReport"):
+ logging.info("Writing BuildToolsReports to {0}".format(OutputReport))
+ versions_list = []
+ for key, value in BuildTools.items():
+ versions_list.append(value)
+ versions_list = sorted(versions_list, key=lambda k: k['type'])
+ json_dict = {"modules": versions_list,
+ "PluginVersion": BuildToolsReport.VERSION}
+
+ htmlfile = open(OutputReport + ".html", "w")
+ jsonfile = open(OutputReport + ".json", "w")
+ template = open(os.path.join(BuildToolsReport.MY_FOLDER, "BuildToolsReport_Template.html"), "r")
+
+ for line in template.readlines():
+ if "%TO_BE_FILLED_IN_BY_PYTHON_SCRIPT%" in line:
+ line = line.replace("%TO_BE_FILLED_IN_BY_PYTHON_SCRIPT%", json.dumps(json_dict))
+ htmlfile.write(line)
+
+ jsonfile.write(json.dumps(versions_list, indent=4))
+
+ jsonfile.close()
+ template.close()
+ htmlfile.close()
diff --git a/BaseTools/Plugin/BuildToolsReport/BuildToolsReportGenerator_plug_in.yaml b/BaseTools/Plugin/BuildToolsReport/BuildToolsReportGenerator_plug_in.yaml
new file mode 100644
index 0000000000..50afd53926
--- /dev/null
+++ b/BaseTools/Plugin/BuildToolsReport/BuildToolsReportGenerator_plug_in.yaml
@@ -0,0 +1,12 @@
+##
+# Build Plugin used to output html report of all versions collected
+# during the build
+#
+# Copyright (c) 2019, Microsoft Corporation
+# SPDX-License-Identifier: BSD-2-Clause-Patent
+##
+{
+ "scope": "global",
+ "name": "Build Tools Report Generator",
+ "module": "BuildToolsReportGenerator"
+}
diff --git a/BaseTools/Plugin/BuildToolsReport/BuildToolsReport_Template.html b/BaseTools/Plugin/BuildToolsReport/BuildToolsReport_Template.html
new file mode 100644
index 0000000000..26350ab425
--- /dev/null
+++ b/BaseTools/Plugin/BuildToolsReport/BuildToolsReport_Template.html
@@ -0,0 +1,126 @@
+<!doctype html>
+<html lang="en">
+<head>
+ <meta charset="utf-8">
+ <meta http-equiv="X-UA-Compatible">
+ <title>Build Tools Report</title>
+ <meta name="viewport" content="width=device-width, initial-scale=1">
+ <link rel="stylesheet" type="text/css" href="https://ajax.aspnetcdn.com/ajax/bootstrap/3.3.7/css/bootstrap.min.css" />
+ <link rel="stylesheet" type="text/css" href="https://cdn.datatables.net/1.10.15/css/dataTables.bootstrap.min.css" />
+ <style>
+ div.attribution {
+ border: 1px solid #ddd;
+ background-color: #bbb;
+ padding-left: 20px;
+ }
+ </style>
+</head>
+<body>
+ <div class="container-fluid">
+ <h1>Build Tools Report</h1>
+ <ul class="nav nav-tabs">
+ <li class="active"><a data-toggle="tab" href="#tabs-1">Tools</a></li>
+ <li><a data-toggle="tab" href="#tabs-2">About</a></li>
+ </ul>
+ <div class="tab-content">
+ <div id="tabs-1" class="tab-pane fade in active">
+ <table id="modinfo" class="table table-striped table-bordered table-hover" cellspacing="0">
+ <thead>
+ <tr>
+ <th>Key</th>
+ <th>Value</th>
+ <th>Type</th>
+ </tr>
+ </thead>
+ <tbody></tbody>
+ </table>
+ </div>
+ <div id="tabs-2" class="tab-pane">
+ <div class="row">
+ <div class="col-xs-7">
+ <p></p>
+ <p>
+ Build Tools Report Template Version: <span id="ReportTemplateVersion">1.00</span><br />
+ Build Tools Report Plugin Version: <span id='ReportToolVersion'></span><br />
+ </p>
+ <h3>License</h3>
+ <hr />
+ <div id="ToolLicenseContent">
+ <p>
+ <span class="copyright">Copyright (c) Microsoft Corporation. All rights reserved.</span><br />
+ <span class="license">
+ SPDX-License-Identifier: BSD-2-Clause-Patent
+ </span>
+ </p>
+ </div>
+ </div>
+ <div id="AttributionListWrapper" class="col-xs-5">
+ <h3>External Licenses</h3>
+ </div>
+ </div>
+ </div>
+ </div>
+ </div>
+
+ <!-- Javascript libraries -->
+ <script type="text/javascript" charset="utf8" src="https://ajax.aspnetcdn.com/ajax/jQuery/jquery-3.2.1.min.js"></script>
+ <script type="text/javascript" charset="utf8" src="https://cdn.datatables.net/1.10.15/js/jquery.dataTables.min.js"></script>
+ <script type="text/javascript" charset="utf8" src="https://ajax.aspnetcdn.com/ajax/bootstrap/3.3.7/bootstrap.min.js"></script>
+ <script type="text/javascript" charset="utf8" src="https://cdn.datatables.net/1.10.15/js/dataTables.bootstrap.min.js"></script>
+
+ <script>
+ var EmbeddedJd = %TO_BE_FILLED_IN_BY_PYTHON_SCRIPT%;
+ </script>
+ <!-- Add javascript here -->
+ <script>
+ var MODULE_TABLE_OFFSET = 350; //Space needed for other stuff besides the Table
+ $(document).ready(function () {
+ $('span#ReportToolVersion').text(EmbeddedJd.PluginVersion);
+ //To support tabs and correct column width we need this change
+ $('a[data-toggle="tab"][href="#tabs-1"]').on('shown.bs.tab', function (e) {
+ $.fn.dataTable.tables({ visible: true, api: true }).columns.adjust();
+ });
+ //table for modules
+ var mTable = $('table#modinfo').dataTable({
+ "aaData": EmbeddedJd.modules,
+ "paginate": false,
+ "autoWidth": false,
+ "scrollY": ($(window).height() - MODULE_TABLE_OFFSET) + "px",
+ "aaSorting": [[2, "asc"]],
+ "aoColumnDefs": [
+ {
+ "mData": "name",
+ "aTargets": [0]
+ },
+
+ {
+ "mData": "version",
+ "aTargets": [1]
+ },
+ {
+ "mData": "type",
+ "aTargets": [2],
+ }
+ ] //end of column def
+ }); //end of modules table
+
+ //
+ // Create Attribution List for all external libraries used
+ //
+ [
+ { Title: "JQuery", Copyright: "Copyright 2017 The jQuery Foundation", Version: $.fn.jquery, LicenseType: "MIT", LicenseLink: "https://jquery.org/license/" },
+ { Title: "DataTables", Copyright: "DataTables designed and created by SpryMedia Ltd Copyright 2007-2017", Version: $.fn.dataTable.version, LicenseType: "MIT", LicenseLink: "https://datatables.net/license/mit" },
+ { Title: "BootStrap", Copyright: "Code and documentation copyright 2011-2017 the Bootstrap Authors and Twitter, Inc.", Version: "3.3.7", LicenseType: "MIT", LicenseLink: "https://github.com/twbs/bootstrap/blob/master/LICENSE" }
+ ].forEach(function (element) {
+ $("<div class='attribution'><h4>" + element.Title + "</h4><p>Version: <span class='version'>" + element.Version + "</span><br /><span class='copyright'>" +
+ element.Copyright + "</span><br />License: <a class='license' href='" + element.LicenseLink + "'>" + element.LicenseType + "</a></p></div>").appendTo("div#AttributionListWrapper");
+ });
+ });
+ $(window).resize(function() {
+ $.fn.dataTable.tables({ visible: true, api: true }).columns.adjust();
+ });
+
+
+ </script>
+</body>
+</html>
diff --git a/BaseTools/Plugin/LinuxGcc5ToolChain/LinuxGcc5ToolChain.py b/BaseTools/Plugin/LinuxGcc5ToolChain/LinuxGcc5ToolChain.py
new file mode 100644
index 0000000000..c31641e931
--- /dev/null
+++ b/BaseTools/Plugin/LinuxGcc5ToolChain/LinuxGcc5ToolChain.py
@@ -0,0 +1,85 @@
+# @file LinuxGcc5ToolChain.py
+# Plugin to configures paths for GCC5 ARM/AARCH64 Toolchain
+##
+# This plugin works in conjuncture with the tools_def
+#
+# Copyright (c) Microsoft Corporation
+# SPDX-License-Identifier: BSD-2-Clause-Patent
+##
+import os
+import logging
+from edk2toolext.environment.plugintypes.uefi_build_plugin import IUefiBuildPlugin
+from edk2toolext.environment import shell_environment
+
+
+class LinuxGcc5ToolChain(IUefiBuildPlugin):
+
+ def do_post_build(self, thebuilder):
+ return 0
+
+ def do_pre_build(self, thebuilder):
+ self.Logger = logging.getLogger("LinuxGcc5ToolChain")
+
+ #
+ # GCC5 - The ARM and AARCH64 compilers need their paths set if available
+ if thebuilder.env.GetValue("TOOL_CHAIN_TAG") == "GCC5":
+
+ # Start with AARACH64 compiler
+ ret = self._check_aarch64()
+ if ret != 0:
+ self.Logger.critical("Failed in check aarch64")
+ return ret
+
+ # Check arm compiler
+ ret = self._check_arm()
+ if ret != 0:
+ self.Logger.critical("Failed in check arm")
+ return ret
+
+ return 0
+
+ def _check_arm(self):
+ # check to see if full path already configured
+ if shell_environment.GetEnvironment().get_shell_var("GCC5_ARM_PREFIX") is not None:
+ self.Logger.info("GCC5_ARM_PREFIX is already set.")
+
+ else:
+ # now check for install dir. If set then set the Prefix
+ install_path = shell_environment.GetEnvironment().get_shell_var("GCC5_ARM_INSTALL")
+ if install_path is None:
+ return 0
+
+ # make GCC5_ARM_PREFIX to align with tools_def.txt
+ prefix = os.path.join(install_path, "bin", "arm-linux-gnueabihf-")
+ shell_environment.GetEnvironment().set_shell_var("GCC5_ARM_PREFIX", prefix)
+
+ # now confirm it exists
+ if not os.path.exists(shell_environment.GetEnvironment().get_shell_var("GCC5_ARM_PREFIX") + "gcc"):
+ self.Logger.error("Path for GCC5_ARM_PREFIX toolchain is invalid")
+ return -2
+
+ return 0
+
+ def _check_aarch64(self):
+ # check to see if full path already configured
+ if shell_environment.GetEnvironment().get_shell_var("GCC5_AARCH64_PREFIX") is not None:
+ self.Logger.info("GCC5_AARCH64_PREFIX is already set.")
+
+ else:
+ # now check for install dir. If set then set the Prefix
+ install_path = shell_environment.GetEnvironment(
+ ).get_shell_var("GCC5_AARCH64_INSTALL")
+ if install_path is None:
+ return 0
+
+ # make GCC5_AARCH64_PREFIX to align with tools_def.txt
+ prefix = os.path.join(install_path, "bin", "aarch64-linux-gnu-")
+ shell_environment.GetEnvironment().set_shell_var("GCC5_AARCH64_PREFIX", prefix)
+
+ # now confirm it exists
+ if not os.path.exists(shell_environment.GetEnvironment().get_shell_var("GCC5_AARCH64_PREFIX") + "gcc"):
+ self.Logger.error(
+ "Path for GCC5_AARCH64_PREFIX toolchain is invalid")
+ return -2
+
+ return 0
diff --git a/BaseTools/Plugin/LinuxGcc5ToolChain/LinuxGcc5ToolChain_plug_in.yaml b/BaseTools/Plugin/LinuxGcc5ToolChain/LinuxGcc5ToolChain_plug_in.yaml
new file mode 100644
index 0000000000..91cfcfbb28
--- /dev/null
+++ b/BaseTools/Plugin/LinuxGcc5ToolChain/LinuxGcc5ToolChain_plug_in.yaml
@@ -0,0 +1,12 @@
+##
+# Build Plugin used to set the path
+# for the GCC5 ARM/AARCH64 downloaded compilers
+#
+# Copyright (c) 2019, Microsoft Corporation
+# SPDX-License-Identifier: BSD-2-Clause-Patent
+##
+{
+ "scope": "global-nix",
+ "name": "Linux GCC5 Tool Chain Support",
+ "module": "LinuxGcc5ToolChain"
+}
diff --git a/BaseTools/Plugin/WindowsResourceCompiler/WinRcPath.py b/BaseTools/Plugin/WindowsResourceCompiler/WinRcPath.py
new file mode 100644
index 0000000000..ec2f2d1298
--- /dev/null
+++ b/BaseTools/Plugin/WindowsResourceCompiler/WinRcPath.py
@@ -0,0 +1,29 @@
+## @file WinRcPath.py
+# Plugin to find Windows SDK Resource Compiler rc.exe
+##
+# This plugin works in conjuncture with the tools_def to support rc.exe
+#
+# Copyright (c) Microsoft Corporation
+# SPDX-License-Identifier: BSD-2-Clause-Patent
+##
+import os
+from edk2toolext.environment.plugintypes.uefi_build_plugin import IUefiBuildPlugin
+import edk2toollib.windows.locate_tools as locate_tools
+from edk2toolext.environment import shell_environment
+from edk2toolext.environment import version_aggregator
+
+class WinRcPath(IUefiBuildPlugin):
+
+ def do_post_build(self, thebuilder):
+ return 0
+
+ def do_pre_build(self, thebuilder):
+ #get the locate tools module
+ path = locate_tools.FindToolInWinSdk("rc.exe")
+ if path is None:
+ thebuilder.logging.warning("Failed to find rc.exe")
+ else:
+ p = os.path.abspath(os.path.dirname(path))
+ shell_environment.GetEnvironment().set_shell_var("WINSDK_PATH_FOR_RC_EXE", p)
+ version_aggregator.GetVersionAggregator().ReportVersion("WINSDK_PATH_FOR_RC_EXE", p, version_aggregator.VersionTypes.INFO)
+ return 0
diff --git a/BaseTools/Plugin/WindowsResourceCompiler/WinRcPath_plug_in.yaml b/BaseTools/Plugin/WindowsResourceCompiler/WinRcPath_plug_in.yaml
new file mode 100644
index 0000000000..030ebe776c
--- /dev/null
+++ b/BaseTools/Plugin/WindowsResourceCompiler/WinRcPath_plug_in.yaml
@@ -0,0 +1,13 @@
+##
+# Build Plugin used to set the path to rc.exe on windows.
+# The plugin is able to use python to locate the tool as to avoid
+# hard-coding the path
+#
+# Copyright (c) 2019, Microsoft Corporation
+# SPDX-License-Identifier: BSD-2-Clause-Patent
+##
+{
+ "scope": "global-win",
+ "name": "Windows RC Path Support",
+ "module": "WinRcPath"
+}
diff --git a/BaseTools/Plugin/WindowsVsToolChain/WindowsVsToolChain.py b/BaseTools/Plugin/WindowsVsToolChain/WindowsVsToolChain.py
new file mode 100644
index 0000000000..a8202e5992
--- /dev/null
+++ b/BaseTools/Plugin/WindowsVsToolChain/WindowsVsToolChain.py
@@ -0,0 +1,126 @@
+## @file WindowsVsToolChain.py
+# Plugin to configures paths for the VS2017 and VS2019 tool chain
+##
+# This plugin works in conjuncture with the tools_def
+#
+# Copyright (c) Microsoft Corporation
+# SPDX-License-Identifier: BSD-2-Clause-Patent
+##
+import os
+import logging
+from edk2toolext.environment.plugintypes.uefi_build_plugin import IUefiBuildPlugin
+import edk2toollib.windows.locate_tools as locate_tools
+from edk2toollib.windows.locate_tools import FindWithVsWhere
+from edk2toolext.environment import shell_environment
+from edk2toolext.environment import version_aggregator
+
+class WindowsVsToolChain(IUefiBuildPlugin):
+
+ def do_post_build(self, thebuilder):
+ return 0
+
+ def do_pre_build(self, thebuilder):
+ self.Logger = logging.getLogger("WindowsVsToolChain")
+
+#
+ # VS2017 - Follow VS2017 where there is potential for many versions of the tools.
+ # If a specific version is required then the user must set both env variables:
+ ## VS150INSTALLPATH: base install path on system to VC install dir. Here you will find the VC folder, etc
+ ## VS150TOOLVER: version number for the VC compiler tools
+ ## VS2017_PREFIX: path to MSVC compiler folder with trailing slash (can be used instead of two vars above)
+ if thebuilder.env.GetValue("TOOL_CHAIN_TAG") == "VS2017":
+
+ # check to see if full path already configured
+ if shell_environment.GetEnvironment().get_shell_var("VS2017_PREFIX") != None:
+ self.Logger.info("VS2017_PREFIX is already set.")
+
+ else:
+ install_path = self._get_vs_install_path("VS2017".lower(), "VS150INSTALLPATH")
+ vc_ver = self._get_vc_version(install_path, "VS150TOOLVER")
+
+ if install_path is None or vc_ver is None:
+ self.Logger.error("Failed to configure environment for VS2017")
+ return -1
+
+ version_aggregator.GetVersionAggregator().ReportVersion(
+ "Visual Studio Install Path", install_path, version_aggregator.VersionTypes.INFO)
+ version_aggregator.GetVersionAggregator().ReportVersion(
+ "VC Version", vc_ver, version_aggregator.VersionTypes.TOOL)
+
+ #make VS2017_PREFIX to align with tools_def.txt
+ prefix = os.path.join(install_path, "VC", "Tools", "MSVC", vc_ver)
+ prefix = prefix + os.path.sep
+ shell_environment.GetEnvironment().set_shell_var("VS2017_PREFIX", prefix)
+
+ # now confirm it exists
+ if not os.path.exists(shell_environment.GetEnvironment().get_shell_var("VS2017_PREFIX")):
+ self.Logger.error("Path for VS2017 toolchain is invalid")
+ return -2
+
+ #
+ # VS2019 - Follow VS2019 where there is potential for many versions of the tools.
+ # If a specific version is required then the user must set both env variables:
+ ## VS160INSTALLPATH: base install path on system to VC install dir. Here you will find the VC folder, etc
+ ## VS160TOOLVER: version number for the VC compiler tools
+ ## VS2019_PREFIX: path to MSVC compiler folder with trailing slash (can be used instead of two vars above)
+ elif thebuilder.env.GetValue("TOOL_CHAIN_TAG") == "VS2019":
+
+ # check to see if full path already configured
+ if shell_environment.GetEnvironment().get_shell_var("VS2019_PREFIX") != None:
+ self.Logger.info("VS2019_PREFIX is already set.")
+
+ else:
+ install_path = self._get_vs_install_path("VS2019".lower(), "VS160INSTALLPATH")
+ vc_ver = self._get_vc_version(install_path, "VS160TOOLVER")
+
+ if install_path is None or vc_ver is None:
+ self.Logger.error("Failed to configure environment for VS2019")
+ return -1
+
+ version_aggregator.GetVersionAggregator().ReportVersion(
+ "Visual Studio Install Path", install_path, version_aggregator.VersionTypes.INFO)
+ version_aggregator.GetVersionAggregator().ReportVersion(
+ "VC Version", vc_ver, version_aggregator.VersionTypes.TOOL)
+
+ #make VS2019_PREFIX to align with tools_def.txt
+ prefix = os.path.join(install_path, "VC", "Tools", "MSVC", vc_ver)
+ prefix = prefix + os.path.sep
+ shell_environment.GetEnvironment().set_shell_var("VS2019_PREFIX", prefix)
+
+ # now confirm it exists
+ if not os.path.exists(shell_environment.GetEnvironment().get_shell_var("VS2019_PREFIX")):
+ self.Logger.error("Path for VS2019 toolchain is invalid")
+ return -2
+
+ return 0
+
+ def _get_vs_install_path(self, vs_version, varname):
+ # check if already specified
+ path = shell_environment.GetEnvironment().get_shell_var(varname)
+ if(path is None):
+ # Not specified...find latest
+ (rc, path) = FindWithVsWhere(vs_version=vs_version)
+ if rc == 0 and path is not None and os.path.exists(path):
+ self.Logger.debug("Found VS instance for %s", vs_version)
+ else:
+ self.Logger.error("Failed to find VS instance with VsWhere (%d)" % rc)
+ return path
+
+ def _get_vc_version(self, path, varname):
+ # check if already specified
+ vc_ver = shell_environment.GetEnvironment().get_shell_var(varname)
+ if (path is None):
+ self.Logger.critical("Failed to find Visual Studio tools. Might need to check for VS install")
+ return vc_ver
+ if(vc_ver is None):
+ # Not specified...find latest
+ p2 = os.path.join(path, "VC", "Tools", "MSVC")
+ if not os.path.isdir(p2):
+ self.Logger.critical(
+ "Failed to find VC tools. Might need to check for VS install")
+ return vc_ver
+ vc_ver = os.listdir(p2)[-1].strip() # get last in list
+ self.Logger.debug("Found VC Tool version is %s" % vc_ver)
+ return vc_ver
+
+
diff --git a/BaseTools/Plugin/WindowsVsToolChain/WindowsVsToolChain_plug_in.yaml b/BaseTools/Plugin/WindowsVsToolChain/WindowsVsToolChain_plug_in.yaml
new file mode 100644
index 0000000000..0d7acc2e6d
--- /dev/null
+++ b/BaseTools/Plugin/WindowsVsToolChain/WindowsVsToolChain_plug_in.yaml
@@ -0,0 +1,11 @@
+##
+# Build Plugin used to set the path to the visual studio tools chain
+#
+# Copyright (c) 2019, Microsoft Corporation
+# SPDX-License-Identifier: BSD-2-Clause-Patent
+##
+{
+ "scope": "global-win",
+ "name": "Windows Visual Studio Tool Chain Support",
+ "module": "WindowsVsToolChain"
+}
--
2.21.0.windows.1
^ permalink raw reply related [flat|nested] 56+ messages in thread* [Patch v3 07/22] .pytool/Plugin: Add CI plugins
2019-10-29 19:54 [Patch v3 00/22] Enable Phase 1 of EDK II CI Michael D Kinney
` (5 preceding siblings ...)
2019-10-29 19:55 ` [Patch v3 06/22] BaseTools: Add BaseTools plugins to support CI Michael D Kinney
@ 2019-10-29 19:55 ` Michael D Kinney
2019-10-29 19:55 ` [Patch v3 08/22] CryptoPkg: Add YAML file for CI builds Michael D Kinney
` (15 subsequent siblings)
22 siblings, 0 replies; 56+ messages in thread
From: Michael D Kinney @ 2019-10-29 19:55 UTC (permalink / raw)
To: devel; +Cc: Sean Brogan, Bret Barkelew, Liming Gao
From: Sean Brogan <sean.brogan@microsoft.com>
https://bugzilla.tianocore.org/show_bug.cgi?id=2315
Add .pytool directory to the edk2 repository with the
following plugins. These plugins are in a top level
directory because that can be used with all packages
and platforms.
* CharEncodingCheck
* CompilerPlugin
* DependencyCheck
* DscCompleteCheck
* GuidCheck
* LibraryClassCheck
* SpellCheck
Cc: Sean Brogan <sean.brogan@microsoft.com>
Cc: Bret Barkelew <Bret.Barkelew@microsoft.com>
Cc: Liming Gao <liming.gao@intel.com>
Signed-off-by: Michael D Kinney <michael.d.kinney@intel.com>
---
.../CharEncodingCheck/CharEncodingCheck.py | 118 ++++++++
.../CharEncodingCheck_plug_in.yaml | 11 +
.pytool/Plugin/CharEncodingCheck/Readme.md | 13 +
.../Plugin/CompilerPlugin/CompilerPlugin.py | 102 +++++++
.../CompilerPlugin/Compiler_plug_in.yaml | 11 +
.../Plugin/DependencyCheck/DependencyCheck.py | 120 +++++++++
.../DependencyCheck_plug_in.yaml | 13 +
.../DscCompleteCheck/DscCompleteCheck.py | 118 ++++++++
.../DscCompleteCheck_plug_in.yaml | 12 +
.pytool/Plugin/DscCompleteCheck/readme.md | 22 ++
.pytool/Plugin/GuidCheck/GuidCheck.py | 251 ++++++++++++++++++
.../Plugin/GuidCheck/GuidCheck_plug_in.yaml | 11 +
.pytool/Plugin/GuidCheck/Readme.md | 60 +++++
.../LibraryClassCheck/LibraryClassCheck.py | 153 +++++++++++
.../LibraryClassCheck_plug_in.yaml | 11 +
.pytool/Plugin/LibraryClassCheck/readme.md | 22 ++
.pytool/Plugin/SpellCheck/Readme.md | 100 +++++++
.pytool/Plugin/SpellCheck/SpellCheck.py | 216 +++++++++++++++
.../Plugin/SpellCheck/SpellCheck_plug_in.yaml | 11 +
.pytool/Plugin/SpellCheck/cspell.base.yaml | 165 ++++++++++++
20 files changed, 1540 insertions(+)
create mode 100644 .pytool/Plugin/CharEncodingCheck/CharEncodingCheck.py
create mode 100644 .pytool/Plugin/CharEncodingCheck/CharEncodingCheck_plug_in.yaml
create mode 100644 .pytool/Plugin/CharEncodingCheck/Readme.md
create mode 100644 .pytool/Plugin/CompilerPlugin/CompilerPlugin.py
create mode 100644 .pytool/Plugin/CompilerPlugin/Compiler_plug_in.yaml
create mode 100644 .pytool/Plugin/DependencyCheck/DependencyCheck.py
create mode 100644 .pytool/Plugin/DependencyCheck/DependencyCheck_plug_in.yaml
create mode 100644 .pytool/Plugin/DscCompleteCheck/DscCompleteCheck.py
create mode 100644 .pytool/Plugin/DscCompleteCheck/DscCompleteCheck_plug_in.yaml
create mode 100644 .pytool/Plugin/DscCompleteCheck/readme.md
create mode 100644 .pytool/Plugin/GuidCheck/GuidCheck.py
create mode 100644 .pytool/Plugin/GuidCheck/GuidCheck_plug_in.yaml
create mode 100644 .pytool/Plugin/GuidCheck/Readme.md
create mode 100644 .pytool/Plugin/LibraryClassCheck/LibraryClassCheck.py
create mode 100644 .pytool/Plugin/LibraryClassCheck/LibraryClassCheck_plug_in.yaml
create mode 100644 .pytool/Plugin/LibraryClassCheck/readme.md
create mode 100644 .pytool/Plugin/SpellCheck/Readme.md
create mode 100644 .pytool/Plugin/SpellCheck/SpellCheck.py
create mode 100644 .pytool/Plugin/SpellCheck/SpellCheck_plug_in.yaml
create mode 100644 .pytool/Plugin/SpellCheck/cspell.base.yaml
diff --git a/.pytool/Plugin/CharEncodingCheck/CharEncodingCheck.py b/.pytool/Plugin/CharEncodingCheck/CharEncodingCheck.py
new file mode 100644
index 0000000000..54a2424875
--- /dev/null
+++ b/.pytool/Plugin/CharEncodingCheck/CharEncodingCheck.py
@@ -0,0 +1,118 @@
+# @file CharEncodingCheck.py
+#
+# Copyright (c) Microsoft Corporation. All rights reserved.
+# SPDX-License-Identifier: BSD-2-Clause-Patent
+##
+
+
+import os
+import logging
+from edk2toolext.environment.plugintypes.ci_build_plugin import ICiBuildPlugin
+from edk2toolext.environment.var_dict import VarDict
+
+##
+# map
+##
+EcodingMap = {
+ ".md": 'utf-8',
+ ".dsc": 'utf-8',
+ ".dec": 'utf-8',
+ ".c": 'utf-8',
+ ".h": 'utf-8',
+ ".asm": 'utf-8',
+ ".masm": 'utf-8',
+ ".nasm": 'utf-8',
+ ".s": 'utf-8',
+ ".inf": 'utf-8',
+ ".asl": 'utf-8',
+ ".uni": 'utf-8',
+ ".py": 'utf-8'
+}
+
+
+class CharEncodingCheck(ICiBuildPlugin):
+ """
+ A CiBuildPlugin that scans each file in the code tree and confirms the encoding is correct.
+
+ Configuration options:
+ "CharEncodingCheck": {
+ "IgnoreFiles": []
+ }
+ """
+
+ def GetTestName(self, packagename: str, environment: VarDict) -> tuple:
+ """ Provide the testcase name and classname for use in reporting
+ testclassname: a descriptive string for the testcase can include whitespace
+ classname: should be patterned <packagename>.<plugin>.<optionally any unique condition>
+
+ Args:
+ packagename: string containing name of package to build
+ environment: The VarDict for the test to run in
+ Returns:
+ a tuple containing the testcase name and the classname
+ (testcasename, classname)
+ """
+ return ("Check for valid file encoding for " + packagename, packagename + ".CharEncodingCheck")
+
+ ##
+ # External function of plugin. This function is used to perform the task of the ci_build_plugin Plugin
+ #
+ # - package is the edk2 path to package. This means workspace/packagepath relative.
+ # - edk2path object configured with workspace and packages path
+ # - PkgConfig Object (dict) for the pkg
+ # - EnvConfig Object
+ # - Plugin Manager Instance
+ # - Plugin Helper Obj Instance
+ # - Junit Logger
+ # - output_stream the StringIO output stream from this plugin via logging
+ def RunBuildPlugin(self, packagename, Edk2pathObj, pkgconfig, environment, PLM, PLMHelper, tc, output_stream=None):
+ overall_status = 0
+ files_tested = 0
+
+ abs_pkg_path = Edk2pathObj.GetAbsolutePathOnThisSytemFromEdk2RelativePath(packagename)
+
+ if abs_pkg_path is None:
+ tc.SetSkipped()
+ tc.LogStdError("No Package folder {0}".format(abs_pkg_path))
+ return 0
+
+ for (ext, enc) in EcodingMap.items():
+ files = self.WalkDirectoryForExtension([ext], abs_pkg_path)
+ files = [Edk2pathObj.GetEdk2RelativePathFromAbsolutePath(x) for x in files] # make edk2relative path so can process ignores
+
+ if "IgnoreFiles" in pkgconfig:
+ for a in pkgconfig["IgnoreFiles"]:
+ a = a.replace(os.sep, "/")
+ try:
+ tc.LogStdOut("Ignoring File {0}".format(a))
+ files.remove(a)
+ except:
+ tc.LogStdError("CharEncodingCheck.IgnoreInf -> {0} not found in filesystem. Invalid ignore file".format(a))
+ logging.info("CharEncodingCheck.IgnoreInf -> {0} not found in filesystem. Invalid ignore file".format(a))
+
+ files = [Edk2pathObj.GetAbsolutePathOnThisSytemFromEdk2RelativePath(x) for x in files]
+ for a in files:
+ files_tested += 1
+ if(self.TestEncodingOk(a, enc)):
+ logging.debug("File {0} Passed Encoding Check {1}".format(a, enc))
+ else:
+ tc.LogStdError("Encoding Failure in {0}. Not {1}".format(a, enc))
+ overall_status += 1
+
+ tc.LogStdOut("Tested Encoding on {0} files".format(files_tested))
+ if overall_status is not 0:
+ tc.SetFailed("CharEncoding {0} Failed. Errors {1}".format(packagename, overall_status), "CHAR_ENCODING_CHECK_FAILED")
+ else:
+ tc.SetSuccess()
+ return overall_status
+
+ def TestEncodingOk(self, apath, encodingValue):
+ try:
+ with open(apath, "rb") as fobj:
+ fobj.read().decode(encodingValue)
+ except Exception as exp:
+ logging.error("Encoding failure: file: {0} type: {1}".format(apath, encodingValue))
+ logging.debug("EXCEPTION: while processing {1} - {0}".format(exp, apath))
+ return False
+
+ return True
diff --git a/.pytool/Plugin/CharEncodingCheck/CharEncodingCheck_plug_in.yaml b/.pytool/Plugin/CharEncodingCheck/CharEncodingCheck_plug_in.yaml
new file mode 100644
index 0000000000..915d3f4fdb
--- /dev/null
+++ b/.pytool/Plugin/CharEncodingCheck/CharEncodingCheck_plug_in.yaml
@@ -0,0 +1,11 @@
+##
+# CiBuildPlugin used to check char encoding
+#
+# Copyright (c) 2019, Microsoft Corporation
+# SPDX-License-Identifier: BSD-2-Clause-Patent
+##
+{
+ "scope": "cibuild",
+ "name": "Char Encoding Check Test",
+ "module": "CharEncodingCheck"
+}
diff --git a/.pytool/Plugin/CharEncodingCheck/Readme.md b/.pytool/Plugin/CharEncodingCheck/Readme.md
new file mode 100644
index 0000000000..8350542966
--- /dev/null
+++ b/.pytool/Plugin/CharEncodingCheck/Readme.md
@@ -0,0 +1,13 @@
+# Character Encoding Check Plugin
+
+This CiBuildPlugin scans all the files in a package to make sure each file is correctly encoded and all characters can be read. Improper encoding causes tools to fail in some situations especially in different locals.
+
+## Configuration
+
+The plugin can be configured to ignore certain files.
+
+``` yaml
+"CharEncodingCheck": {
+ "IgnoreFiles": [] # optional - list of files to ignore
+}
+```
diff --git a/.pytool/Plugin/CompilerPlugin/CompilerPlugin.py b/.pytool/Plugin/CompilerPlugin/CompilerPlugin.py
new file mode 100644
index 0000000000..0a357309a4
--- /dev/null
+++ b/.pytool/Plugin/CompilerPlugin/CompilerPlugin.py
@@ -0,0 +1,102 @@
+# @file HostUnitTestCompiler_plugin.py
+##
+# Copyright (c) Microsoft Corporation. All rights reserved.
+# SPDX-License-Identifier: BSD-2-Clause-Patent
+##
+
+import logging
+import os
+import re
+from edk2toollib.uefi.edk2.parsers.dsc_parser import DscParser
+from edk2toolext.environment.plugintypes.ci_build_plugin import ICiBuildPlugin
+from edk2toolext.environment.uefi_build import UefiBuilder
+from edk2toolext import edk2_logging
+from edk2toolext.environment.var_dict import VarDict
+
+
+class CompilerPlugin(ICiBuildPlugin):
+ """
+ A CiBuildPlugin that compiles the package dsc
+ from the package being tested.
+
+ Configuration options:
+ "CompilerPlugin": {
+ "DscPath": "<path to dsc from root of pkg>"
+ }
+ """
+
+ def GetTestName(self, packagename: str, environment: VarDict) -> tuple:
+ """ Provide the testcase name and classname for use in reporting
+
+ Args:
+ packagename: string containing name of package to build
+ environment: The VarDict for the test to run in
+ Returns:
+ a tuple containing the testcase name and the classname
+ (testcasename, classname)
+ """
+ target = environment.GetValue("TARGET")
+ return ("Compile " + packagename + " " + target, packagename + ".Compiler." + target)
+
+ def RunsOnTargetList(self):
+ return ["DEBUG", "RELEASE"]
+
+ ##
+ # External function of plugin. This function is used to perform the task of the MuBuild Plugin
+ #
+ # - package is the edk2 path to package. This means workspace/packagepath relative.
+ # - edk2path object configured with workspace and packages path
+ # - PkgConfig Object (dict) for the pkg
+ # - EnvConfig Object
+ # - Plugin Manager Instance
+ # - Plugin Helper Obj Instance
+ # - Junit Logger
+ # - output_stream the StringIO output stream from this plugin via logging
+ def RunBuildPlugin(self, packagename, Edk2pathObj, pkgconfig, environment, PLM, PLMHelper, tc, output_stream=None):
+ self._env = environment
+
+ # Parse the config for required DscPath element
+ if "DscPath" not in pkgconfig:
+ tc.SetSkipped()
+ tc.LogStdError("DscPath not found in config file. Nothing to compile.")
+ return -1
+
+ AP = Edk2pathObj.GetAbsolutePathOnThisSytemFromEdk2RelativePath(packagename)
+
+ APDSC = os.path.join(AP, pkgconfig["DscPath"].strip())
+ AP_Path = Edk2pathObj.GetEdk2RelativePathFromAbsolutePath(APDSC)
+ if AP is None or AP_Path is None or not os.path.isfile(APDSC):
+ tc.SetSkipped()
+ tc.LogStdError("Package Dsc not found.")
+ return -1
+
+ logging.info("Building {0}".format(AP_Path))
+ self._env.SetValue("ACTIVE_PLATFORM", AP_Path, "Set in Compiler Plugin")
+
+ # Parse DSC to check for SUPPORTED_ARCHITECTURES
+ dp = DscParser()
+ dp.SetBaseAbsPath(Edk2pathObj.WorkspacePath)
+ dp.SetPackagePaths(Edk2pathObj.PackagePathList)
+ dp.ParseFile(AP_Path)
+ if "SUPPORTED_ARCHITECTURES" in dp.LocalVars:
+ SUPPORTED_ARCHITECTURES = dp.LocalVars["SUPPORTED_ARCHITECTURES"].split('|')
+ TARGET_ARCHITECTURES = environment.GetValue("TARGET_ARCH").split(' ')
+
+ # Skip if there is no intersection between SUPPORTED_ARCHITECTURES and TARGET_ARCHITECTURES
+ if len(set(SUPPORTED_ARCHITECTURES) & set(TARGET_ARCHITECTURES)) == 0:
+ tc.SetSkipped()
+ tc.LogStdError("No supported architecutres to build")
+ return -1
+
+ uefiBuilder = UefiBuilder()
+ # do all the steps
+ # WorkSpace, PackagesPath, PInHelper, PInManager
+ ret = uefiBuilder.Go(Edk2pathObj.WorkspacePath, os.pathsep.join(Edk2pathObj.PackagePathList), PLMHelper, PLM)
+ if ret != 0: # failure:
+ tc.SetFailed("Compile failed for {0}".format(packagename), "Compile_FAILED")
+ tc.LogStdError("{0} Compile failed with error code {1} ".format(AP_Path, ret))
+ return 1
+
+ else:
+ tc.SetSuccess()
+ return 0
diff --git a/.pytool/Plugin/CompilerPlugin/Compiler_plug_in.yaml b/.pytool/Plugin/CompilerPlugin/Compiler_plug_in.yaml
new file mode 100644
index 0000000000..4f9b3d3113
--- /dev/null
+++ b/.pytool/Plugin/CompilerPlugin/Compiler_plug_in.yaml
@@ -0,0 +1,11 @@
+##
+# CiBuildPlugin used to compile each package
+#
+# Copyright (c) 2019, Microsoft Corporation
+# SPDX-License-Identifier: BSD-2-Clause-Patent
+##
+{
+ "scope": "cibuild",
+ "name": "Compiler Plugin",
+ "module": "CompilerPlugin"
+}
diff --git a/.pytool/Plugin/DependencyCheck/DependencyCheck.py b/.pytool/Plugin/DependencyCheck/DependencyCheck.py
new file mode 100644
index 0000000000..497914cf3a
--- /dev/null
+++ b/.pytool/Plugin/DependencyCheck/DependencyCheck.py
@@ -0,0 +1,120 @@
+# @file dependency_check.py
+#
+# Copyright (c) Microsoft Corporation. All rights reserved.
+# SPDX-License-Identifier: BSD-2-Clause-Patent
+##
+
+import logging
+import os
+from edk2toolext.environment.plugintypes.ci_build_plugin import ICiBuildPlugin
+from edk2toollib.uefi.edk2.parsers.inf_parser import InfParser
+from edk2toolext.environment.var_dict import VarDict
+
+
+class DependencyCheck(ICiBuildPlugin):
+ """
+ A CiBuildPlugin that finds all modules (inf files) in a package and reviews the packages used
+ to confirm they are acceptable. This is to help enforce layering and identify improper
+ dependencies between packages.
+
+ Configuration options:
+ "DependencyCheck": {
+ "AcceptableDependencies": [], # Package dec files that are allowed in all INFs. Example: MdePkg/MdePkg.dec
+ "AcceptableDependencies-<MODULE_TYPE>": [], # OPTIONAL Package dependencies for INFs that are HOST_APPLICATION
+ "AcceptableDependencies-HOST_APPLICATION": [], # EXAMPLE Package dependencies for INFs that are HOST_APPLICATION
+ "IgnoreInf": [] # Ignore INF if found in filesystem
+ }
+ """
+
+ def GetTestName(self, packagename: str, environment: VarDict) -> tuple:
+ """ Provide the testcase name and classname for use in reporting
+
+ Args:
+ packagename: string containing name of package to build
+ environment: The VarDict for the test to run in
+ Returns:
+ a tuple containing the testcase name and the classname
+ (testcasename, classname)
+ testclassname: a descriptive string for the testcase can include whitespace
+ classname: should be patterned <packagename>.<plugin>.<optionally any unique condition>
+ """
+ return ("Test Package Dependencies for modules in " + packagename, packagename + ".DependencyCheck")
+
+ ##
+ # External function of plugin. This function is used to perform the task of the MuBuild Plugin
+ #
+ # - package is the edk2 path to package. This means workspace/packagepath relative.
+ # - edk2path object configured with workspace and packages path
+ # - PkgConfig Object (dict) for the pkg
+ # - EnvConfig Object
+ # - Plugin Manager Instance
+ # - Plugin Helper Obj Instance
+ # - Junit Logger
+ # - output_stream the StringIO output stream from this plugin via logging
+ def RunBuildPlugin(self, packagename, Edk2pathObj, pkgconfig, environment, PLM, PLMHelper, tc, output_stream=None):
+ overall_status = 0
+
+ # Get current platform
+ abs_pkg_path = Edk2pathObj.GetAbsolutePathOnThisSytemFromEdk2RelativePath(packagename)
+
+ # Get INF Files
+ INFFiles = self.WalkDirectoryForExtension([".inf"], abs_pkg_path)
+ INFFiles = [Edk2pathObj.GetEdk2RelativePathFromAbsolutePath(x) for x in INFFiles] # make edk2relative path so can compare with Ignore List
+
+ # Remove ignored INFs
+ if "IgnoreInf" in pkgconfig:
+ for a in pkgconfig["IgnoreInf"]:
+ a = a.replace(os.sep, "/") ## convert path sep in case ignore list is bad. Can't change case
+ try:
+ INFFiles.remove(a)
+ tc.LogStdOut("IgnoreInf {0}".format(a))
+ except:
+ logging.info("DependencyConfig.IgnoreInf -> {0} not found in filesystem. Invalid ignore file".format(a))
+ tc.LogStdError("DependencyConfig.IgnoreInf -> {0} not found in filesystem. Invalid ignore file".format(a))
+
+
+ # Get the AccpetableDependencies list
+ if "AcceptableDependencies" not in pkgconfig:
+ logging.info("DependencyCheck Skipped. No Acceptable Dependencies defined.")
+ tc.LogStdOut("DependencyCheck Skipped. No Acceptable Dependencies defined.")
+ tc.SetSkipped()
+ return -1
+
+ # Log dependencies
+ for k in pkgconfig.keys():
+ if k.startswith("AcceptableDependencies"):
+ pkgstring = "\n".join(pkgconfig[k])
+ if ("-" in k):
+ _, _, mod_type = k.partition("-")
+ tc.LogStdOut(f"Additional dependencies for MODULE_TYPE {mod_type}:\n {pkgstring}")
+ else:
+ tc.LogStdOut(f"Acceptable Dependencies:\n {pkgstring}")
+
+ # For each INF file
+ for file in INFFiles:
+ ip = InfParser()
+ logging.debug("Parsing " + file)
+ ip.SetBaseAbsPath(Edk2pathObj.WorkspacePath).SetPackagePaths(Edk2pathObj.PackagePathList).ParseFile(file)
+
+ if("MODULE_TYPE" not in ip.Dict):
+ tc.LogStdOut("Ignoring INF. Missing key for MODULE_TYPE {0}".format(file))
+ continue
+
+ mod_type = ip.Dict["MODULE_TYPE"].upper()
+ for p in ip.PackagesUsed:
+ if p not in pkgconfig["AcceptableDependencies"]:
+ # If not in the main acceptable dependencies list then check module specific
+ mod_specific_key = "AcceptableDependencies-" + mod_type
+ if mod_specific_key in pkgconfig and p in pkgconfig[mod_specific_key]:
+ continue
+
+ logging.error("Dependency Check: Invalid Dependency INF: {0} depends on pkg {1}".format(file, p))
+ tc.LogStdError("Dependency Check: Invalid Dependency INF: {0} depends on pkg {1}".format(file, p))
+ overall_status += 1
+
+ # If XML object exists, add results
+ if overall_status is not 0:
+ tc.SetFailed("Failed with {0} errors".format(overall_status), "DEPENDENCYCHECK_FAILED")
+ else:
+ tc.SetSuccess()
+ return overall_status
diff --git a/.pytool/Plugin/DependencyCheck/DependencyCheck_plug_in.yaml b/.pytool/Plugin/DependencyCheck/DependencyCheck_plug_in.yaml
new file mode 100644
index 0000000000..fdb96d625b
--- /dev/null
+++ b/.pytool/Plugin/DependencyCheck/DependencyCheck_plug_in.yaml
@@ -0,0 +1,13 @@
+##
+# CiBuildPlugin used to check all infs within a package
+# to confirm the packagesdependency are on the configured list of acceptable
+# dependencies.
+#
+# Copyright (c) 2019, Microsoft Corporation
+# SPDX-License-Identifier: BSD-2-Clause-Patent
+##
+{
+ "scope": "cibuild",
+ "name": "Dependency Check Test",
+ "module": "DependencyCheck"
+}
diff --git a/.pytool/Plugin/DscCompleteCheck/DscCompleteCheck.py b/.pytool/Plugin/DscCompleteCheck/DscCompleteCheck.py
new file mode 100644
index 0000000000..dcd8946ca6
--- /dev/null
+++ b/.pytool/Plugin/DscCompleteCheck/DscCompleteCheck.py
@@ -0,0 +1,118 @@
+# @file DscCompleteCheck.py
+#
+# Copyright (c) Microsoft Corporation. All rights reserved.
+# SPDX-License-Identifier: BSD-2-Clause-Patent
+##
+import logging
+import os
+from edk2toolext.environment.plugintypes.ci_build_plugin import ICiBuildPlugin
+from edk2toollib.uefi.edk2.parsers.dsc_parser import DscParser
+from edk2toollib.uefi.edk2.parsers.inf_parser import InfParser
+from edk2toolext.environment.var_dict import VarDict
+
+
+class DscCompleteCheck(ICiBuildPlugin):
+ """
+ A CiBuildPlugin that scans the package dsc file and confirms all modules (inf files) are
+ listed in the components sections.
+
+ Configuration options:
+ "DscCompleteCheck": {
+ "DscPath": "<path to dsc from root of pkg>"
+ "IgnoreInf": [] # Ignore INF if found in filesystem by not dsc
+ }
+ """
+
+ def GetTestName(self, packagename: str, environment: VarDict) -> tuple:
+ """ Provide the testcase name and classname for use in reporting
+
+ Args:
+ packagename: string containing name of package to build
+ environment: The VarDict for the test to run in
+ Returns:
+ a tuple containing the testcase name and the classname
+ (testcasename, classname)
+ testclassname: a descriptive string for the testcase can include whitespace
+ classname: should be patterned <packagename>.<plugin>.<optionally any unique condition>
+ """
+ return ("Check the " + packagename + " DSC for a being complete", packagename + ".DscCompleteCheck")
+
+ ##
+ # External function of plugin. This function is used to perform the task of the MuBuild Plugin
+ #
+ # - package is the edk2 path to package. This means workspace/packagepath relative.
+ # - edk2path object configured with workspace and packages path
+ # - PkgConfig Object (dict) for the pkg
+ # - VarDict containing the shell environment Build Vars
+ # - Plugin Manager Instance
+ # - Plugin Helper Obj Instance
+ # - Junit Logger
+ # - output_stream the StringIO output stream from this plugin via logging
+ def RunBuildPlugin(self, packagename, Edk2pathObj, pkgconfig, environment, PLM, PLMHelper, tc, output_stream=None):
+ overall_status = 0
+
+ # Parse the config for required DscPath element
+ if "DscPath" not in pkgconfig:
+ tc.SetSkipped()
+ tc.LogStdError("DscPath not found in config file. Nothing to check.")
+ return -1
+
+ abs_pkg_path = Edk2pathObj.GetAbsolutePathOnThisSytemFromEdk2RelativePath(packagename)
+ abs_dsc_path = os.path.join(abs_pkg_path, pkgconfig["DscPath"].strip())
+ wsr_dsc_path = Edk2pathObj.GetEdk2RelativePathFromAbsolutePath(abs_dsc_path)
+
+ if abs_dsc_path is None or wsr_dsc_path is "" or not os.path.isfile(abs_dsc_path):
+ tc.SetSkipped()
+ tc.LogStdError("Package Dsc not found")
+ return 0
+
+ # Get INF Files
+ INFFiles = self.WalkDirectoryForExtension([".inf"], abs_pkg_path)
+ INFFiles = [Edk2pathObj.GetEdk2RelativePathFromAbsolutePath(x) for x in INFFiles] # make edk2relative path so can compare with DSC
+
+ # remove ignores
+
+ if "IgnoreInf" in pkgconfig:
+ for a in pkgconfig["IgnoreInf"]:
+ a = a.replace(os.sep, "/")
+ try:
+ tc.LogStdOut("Ignoring INF {0}".format(a))
+ INFFiles.remove(a)
+ except:
+ tc.LogStdError("DscCompleteCheck.IgnoreInf -> {0} not found in filesystem. Invalid ignore file".format(a))
+ logging.info("DscCompleteCheck.IgnoreInf -> {0} not found in filesystem. Invalid ignore file".format(a))
+
+ # DSC Parser
+ dp = DscParser()
+ dp.SetBaseAbsPath(Edk2pathObj.WorkspacePath)
+ dp.SetPackagePaths(Edk2pathObj.PackagePathList)
+ dp.SetInputVars(environment.GetAllBuildKeyValues())
+ dp.ParseFile(wsr_dsc_path)
+
+ # Check if INF in component section
+ for INF in INFFiles:
+ if not any(INF.strip() in x for x in dp.ThreeMods) and \
+ not any(INF.strip() in x for x in dp.SixMods) and \
+ not any(INF.strip() in x for x in dp.OtherMods):
+
+ infp = InfParser().SetBaseAbsPath(Edk2pathObj.WorkspacePath)
+ infp.SetPackagePaths(Edk2pathObj.PackagePathList)
+ infp.ParseFile(INF)
+ if("MODULE_TYPE" not in infp.Dict):
+ tc.LogStdOut("Ignoring INF. Missing key for MODULE_TYPE {0}".format(INF))
+ continue
+
+ if(infp.Dict["MODULE_TYPE"] == "HOST_APPLICATION"):
+ tc.LogStdOut("Ignoring INF. Module type is HOST_APPLICATION {0}".format(INF))
+ continue
+
+ logging.critical(INF + " not in " + wsr_dsc_path)
+ tc.LogStdError("{0} not in {1}".format(INF, wsr_dsc_path))
+ overall_status = overall_status + 1
+
+ # If XML object exists, add result
+ if overall_status is not 0:
+ tc.SetFailed("DscCompleteCheck {0} Failed. Errors {1}".format(wsr_dsc_path, overall_status), "CHECK_FAILED")
+ else:
+ tc.SetSuccess()
+ return overall_status
diff --git a/.pytool/Plugin/DscCompleteCheck/DscCompleteCheck_plug_in.yaml b/.pytool/Plugin/DscCompleteCheck/DscCompleteCheck_plug_in.yaml
new file mode 100644
index 0000000000..d84d57d973
--- /dev/null
+++ b/.pytool/Plugin/DscCompleteCheck/DscCompleteCheck_plug_in.yaml
@@ -0,0 +1,12 @@
+##
+# CiBuildPlugin used to confirm all INFs are listed in
+# the components section of package dsc
+#
+# Copyright (c) 2019, Microsoft Corporation
+# SPDX-License-Identifier: BSD-2-Clause-Patent
+##
+{
+ "scope": "cibuild",
+ "name": "Dsc Complete Check Test",
+ "module": "DscCompleteCheck"
+}
diff --git a/.pytool/Plugin/DscCompleteCheck/readme.md b/.pytool/Plugin/DscCompleteCheck/readme.md
new file mode 100644
index 0000000000..17e542b8d6
--- /dev/null
+++ b/.pytool/Plugin/DscCompleteCheck/readme.md
@@ -0,0 +1,22 @@
+# Dsc Complete Check Plugin
+
+This CiBuildPlugin scans all INF files from a package and confirms they are listed in the package level DSC file. The test considers it an error if any INF does not appear in the `Components` section of the package-level DSC (indicating that it would not be built if the package were built). This is critical because much of the CI infrastructure assumes that all modules will be listed in the DSC and compiled.
+
+## Configuration
+
+The plugin has a few configuration options to support the UEFI codebase.
+
+``` yaml
+"DscCompleteCheck": {
+ "DscPath": "", # Path to dsc from root of package
+ "IgnoreInf": [] # Ignore INF if found in filesystem by not dsc
+ }
+```
+
+### DscPath
+
+Path to DSC to consider platform dsc
+
+### IgnoreInf
+
+Ignore error if Inf file is not listed in DSC file
diff --git a/.pytool/Plugin/GuidCheck/GuidCheck.py b/.pytool/Plugin/GuidCheck/GuidCheck.py
new file mode 100644
index 0000000000..467e17f3e8
--- /dev/null
+++ b/.pytool/Plugin/GuidCheck/GuidCheck.py
@@ -0,0 +1,251 @@
+# @file GuidCheck.py
+#
+# Copyright (c) Microsoft Corporation. All rights reserved.
+# SPDX-License-Identifier: BSD-2-Clause-Patent
+##
+import logging
+from edk2toolext.environment.plugintypes.ci_build_plugin import ICiBuildPlugin
+from edk2toollib.uefi.edk2.guid_list import GuidList
+from edk2toolext.environment.var_dict import VarDict
+
+
+class GuidCheck(ICiBuildPlugin):
+ """
+ A CiBuildPlugin that scans the code tree and looks for duplicate guids
+ from the package being tested.
+
+ Configuration options:
+ "GuidCheck": {
+ "IgnoreGuidName": [], # provide in format guidname=guidvalue or just guidname
+ "IgnoreGuidValue": [],
+ "IgnoreFoldersAndFiles": [],
+ "IgnoreDuplicates": [] # Provide in format guidname=guidname=guidname...
+ }
+ """
+
+ def GetTestName(self, packagename: str, environment: VarDict) -> tuple:
+ """ Provide the testcase name and classname for use in reporting
+
+ Args:
+ packagename: string containing name of package to build
+ environment: The VarDict for the test to run in
+ Returns:
+ a tuple containing the testcase name and the classname
+ (testcasename, classname)
+ testclassname: a descriptive string for the testcase can include whitespace
+ classname: should be patterned <packagename>.<plugin>.<optionally any unique condition>
+ """
+ return ("Confirm GUIDs are unique in " + packagename, packagename + ".GuidCheck")
+
+ def _FindConflictingGuidValues(self, guidlist: list) -> list:
+ """ Find all duplicate guids by guid value and report them as errors
+ """
+ # Sort the list by guid
+ guidsorted = sorted(
+ guidlist, key=lambda x: x.guid.upper(), reverse=True)
+
+ previous = None # Store previous entry for comparison
+ error = None
+ errors = []
+ for index in range(len(guidsorted)):
+ i = guidsorted[index]
+ if(previous is not None):
+ if i.guid == previous.guid: # Error
+ if(error is None):
+ # Catch errors with more than 1 conflict
+ error = ErrorEntry("guid")
+ error.entries.append(previous)
+ errors.append(error)
+ error.entries.append(i)
+ else:
+ # no match. clear error
+ error = None
+ previous = i
+ return errors
+
+ def _FindConflictingGuidNames(self, guidlist: list) -> list:
+ """ Find all duplicate guids by name and if they are not all
+ from inf files report them as errors. It is ok to have
+ BASE_NAME duplication.
+
+ Is this useful? It would catch two same named guids in dec file
+ that resolve to different values.
+ """
+ # Sort the list by guid
+ namesorted = sorted(guidlist, key=lambda x: x.name.upper())
+
+ previous = None # Store previous entry for comparison
+ error = None
+ errors = []
+ for index in range(len(namesorted)):
+ i = namesorted[index]
+ if(previous is not None):
+ # If name matches
+ if i.name == previous.name:
+ if(error is None):
+ # Catch errors with more than 1 conflict
+ error = ErrorEntry("name")
+ error.entries.append(previous)
+ errors.append(error)
+ error.entries.append(i)
+ else:
+ # no match. clear error
+ error = None
+ previous = i
+
+ # Loop thru and remove any errors where all files are infs as it is ok if
+ # they have the same inf base name.
+ for e in errors[:]:
+ if len( [en for en in e.entries if not en.absfilepath.lower().endswith(".inf")]) == 0:
+ errors.remove(e)
+
+ return errors
+
+ ##
+ # External function of plugin. This function is used to perform the task of the MuBuild Plugin
+ #
+ # - package is the edk2 path to package. This means workspace/packagepath relative.
+ # - edk2path object configured with workspace and packages path
+ # - PkgConfig Object (dict) for the pkg
+ # - EnvConfig Object
+ # - Plugin Manager Instance
+ # - Plugin Helper Obj Instance
+ # - Junit Logger
+ # - output_stream the StringIO output stream from this plugin via logging
+
+ def RunBuildPlugin(self, packagename, Edk2pathObj, pkgconfig, environment, PLM, PLMHelper, tc, output_stream=None):
+ Errors = []
+
+ abs_pkg_path = Edk2pathObj.GetAbsolutePathOnThisSytemFromEdk2RelativePath(
+ packagename)
+
+ if abs_pkg_path is None:
+ tc.SetSkipped()
+ tc.LogStdError("No package {0}".format(packagename))
+ return -1
+
+ All_Ignores = ["/Build", "/Conf"]
+ # Parse the config for other ignores
+ if "IgnoreFoldersAndFiles" in pkgconfig:
+ All_Ignores.extend(pkgconfig["IgnoreFoldersAndFiles"])
+
+ # Parse the workspace for all GUIDs
+ gs = GuidList.guidlist_from_filesystem(
+ Edk2pathObj.WorkspacePath, ignore_lines=All_Ignores)
+
+ # Remove ignored guidvalue
+ if "IgnoreGuidValue" in pkgconfig:
+ for a in pkgconfig["IgnoreGuidValue"]:
+ try:
+ tc.LogStdOut("Ignoring Guid {0}".format(a.upper()))
+ for b in gs[:]:
+ if b.guid == a.upper():
+ gs.remove(b)
+ except:
+ tc.LogStdError("GuidCheck.IgnoreGuid -> {0} not found. Invalid ignore guid".format(a.upper()))
+ logging.info("GuidCheck.IgnoreGuid -> {0} not found. Invalid ignore guid".format(a.upper()))
+
+ # Remove ignored guidname
+ if "IgnoreGuidName" in pkgconfig:
+ for a in pkgconfig["IgnoreGuidName"]:
+ entry = a.split("=")
+ if(len(entry) > 2):
+ tc.LogStdError("GuidCheck.IgnoreGuidName -> {0} Invalid Format.".format(a))
+ logging.info("GuidCheck.IgnoreGuidName -> {0} Invalid Format.".format(a))
+ continue
+ try:
+ tc.LogStdOut("Ignoring Guid {0}".format(a))
+ for b in gs[:]:
+ if b.name == entry[0]:
+ if(len(entry) == 1):
+ gs.remove(b)
+ elif(len(entry) == 2 and b.guid.upper() == entry[1].upper()):
+ gs.remove(b)
+ else:
+ c.LogStdError("GuidCheck.IgnoreGuidName -> {0} incomplete match. Invalid ignore guid".format(a))
+
+ except:
+ tc.LogStdError("GuidCheck.IgnoreGuidName -> {0} not found. Invalid ignore name".format(a))
+ logging.info("GuidCheck.IgnoreGuidName -> {0} not found. Invalid ignore name".format(a))
+
+ # Find conflicting Guid Values
+ Errors.extend(self._FindConflictingGuidValues(gs))
+
+ # Check if there are expected duplicates and remove it from the error list
+ if "IgnoreDuplicates" in pkgconfig:
+ for a in pkgconfig["IgnoreDuplicates"]:
+ names = a.split("=")
+ if len(names) < 2:
+ tc.LogStdError("GuidCheck.IgnoreDuplicates -> {0} invalid format".format(a))
+ logging.info("GuidCheck.IgnoreDuplicates -> {0} invalid format".format(a))
+ continue
+
+ for b in Errors[:]:
+ if b.type != "guid":
+ continue
+ ## Make a list of the names that are not in the names list. If there
+ ## are any in the list then this error should not be ignored.
+ t = [x for x in b.entries if x.name not in names]
+ if(len(t) == len(b.entries)):
+ ## did not apply to any entry
+ continue
+ elif(len(t) == 0):
+ ## full match - ignore duplicate
+ tc.LogStdOut("GuidCheck.IgnoreDuplicates -> {0}".format(a))
+ Errors.remove(b)
+ elif(len(t) < len(b.entries)):
+ ## partial match
+ tc.LogStdOut("GuidCheck.IgnoreDuplicates -> {0} incomplete match".format(a))
+ logging.info("GuidCheck.IgnoreDuplicates -> {0} incomplete match".format(a))
+ else:
+ tc.LogStdOut("GuidCheck.IgnoreDuplicates -> {0} unknown error.".format(a))
+ logging.info("GuidCheck.IgnoreDuplicates -> {0} unknown error".format(a))
+
+
+
+ # Find conflicting Guid Names
+ Errors.extend(self._FindConflictingGuidNames(gs))
+
+ # Log errors for anything within the package under test
+ for er in Errors[:]:
+ InMyPackage = False
+ for a in er.entries:
+ if abs_pkg_path in a.absfilepath:
+ InMyPackage = True
+ break
+ if(not InMyPackage):
+ Errors.remove(er)
+ else:
+ logging.error(str(er))
+ tc.LogStdError(str(er))
+
+ # add result to test case
+ overall_status = len(Errors)
+ if overall_status is not 0:
+ tc.SetFailed("GuidCheck {0} Failed. Errors {1}".format(
+ packagename, overall_status), "CHECK_FAILED")
+ else:
+ tc.SetSuccess()
+ return overall_status
+
+
+class ErrorEntry():
+ """ Custom/private class for reporting errors in the GuidList
+ """
+
+ def __init__(self, errortype):
+ self.type = errortype # 'guid' or 'name' depending on error type
+ self.entries = [] # GuidListEntry that are in error condition
+
+ def __str__(self):
+ a = f"Error Duplicate {self.type}: "
+ if(self.type == "guid"):
+ a += f" {self.entries[0].guid}"
+ elif(self.type == "name"):
+ a += f" {self.entries[0].name}"
+
+ a += f" ({len(self.entries)})\n"
+
+ for e in self.entries:
+ a += "\t" + str(e) + "\n"
+ return a
diff --git a/.pytool/Plugin/GuidCheck/GuidCheck_plug_in.yaml b/.pytool/Plugin/GuidCheck/GuidCheck_plug_in.yaml
new file mode 100644
index 0000000000..531efb7885
--- /dev/null
+++ b/.pytool/Plugin/GuidCheck/GuidCheck_plug_in.yaml
@@ -0,0 +1,11 @@
+##
+# CiBuildPlugin used to check guid uniqueness
+#
+# Copyright (c) 2019, Microsoft Corporation
+# SPDX-License-Identifier: BSD-2-Clause-Patent
+##
+{
+ "scope": "cibuild",
+ "name": "Guid Check Test",
+ "module": "GuidCheck"
+}
diff --git a/.pytool/Plugin/GuidCheck/Readme.md b/.pytool/Plugin/GuidCheck/Readme.md
new file mode 100644
index 0000000000..c1bf3d728e
--- /dev/null
+++ b/.pytool/Plugin/GuidCheck/Readme.md
@@ -0,0 +1,60 @@
+# Guid Check Plugin
+
+This CiBuildPlugin scans all the files in a code tree to find all the GUID definitions. After collection it will then look for duplication in the package under test. Uniqueness of all GUIDs are critical within the UEFI environment. Duplication can cause numerous issues including locating the wrong data structure, calling the wrong function, or decoding the wrong data members.
+
+Currently Scanned:
+
+* INF files are scanned for there Module guid
+* DEC files are scanned for all of their Protocols, PPIs, and Guids as well as the one package GUID.
+
+Any GUID value being equal to two names or even just defined in two files is considered an error unless in the ignore list.
+
+Any GUID name that is found more than once is an error unless all occurrences are Module GUIDs. Since the Module GUID is assigned to the Module name it is common to have numerous versions of the same module named the same.
+
+## Configuration
+
+The plugin has numerous configuration options to support the UEFI codebase.
+
+``` yaml
+"GuidCheck": {
+ "IgnoreGuidName": [],
+ "IgnoreGuidValue": [],
+ "IgnoreFoldersAndFiles": [],
+ "IgnoreDuplicates": []
+ }
+```
+
+### IgnoreGuidName
+
+This list allows strings in two formats.
+
+* _GuidName_
+ * This will remove any entry with this GuidName from the list of GUIDs therefore ignoring any error associated with this name.
+* _GuidName=GuidValue_
+ * This will also ignore the GUID by name but only if the value equals the GuidValue.
+ * GuidValue should be in registry format.
+ * This is the suggested format to use as it will limit the ignore to only the defined case.
+
+### IgnoreGuidValue
+
+This list allows strings in guid registry format _GuidValue_.
+
+* This will remove any entry with this GuidValue from the list of GUIDs therefore ignoring any error associated with this value.
+* GuidValue must be in registry format xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
+
+### IgnoreFoldersAndFiles
+
+This supports .gitignore file and folder matching strings including wildcards
+
+* Any folder or file ignored will not be parsed and therefore any GUID defined will be ignored.
+* The plugin will always ignores the following ["/Build", "/Conf"]
+
+### IgnoreDuplicates
+
+This supports strings in the format of _GuidName_=_GuidName_=_GuidName_
+
+* For the error with the GuidNames to be ignored the list must match completely with what is found during the code scan.
+ * For example if there are two GUIDs that are by design equal within the code tree then it should be _GuidName_=_GuidName_
+ * If instead there are three GUIDs then it must be _GuidName_=_GuidName_=_GuidName_
+* This is the best ignore list to use because it is the most strict and will catch new problems when new conflicts are introduced.
+* There are numerous places in the UEFI specification in which two GUID names are assigned the same value. These names should be set in this ignore list so that they don't cause an error but any additional duplication would still be caught.
diff --git a/.pytool/Plugin/LibraryClassCheck/LibraryClassCheck.py b/.pytool/Plugin/LibraryClassCheck/LibraryClassCheck.py
new file mode 100644
index 0000000000..33745dff11
--- /dev/null
+++ b/.pytool/Plugin/LibraryClassCheck/LibraryClassCheck.py
@@ -0,0 +1,153 @@
+# @file LibraryClassCheck.py
+#
+# Copyright (c) Microsoft Corporation. All rights reserved.
+# SPDX-License-Identifier: BSD-2-Clause-Patent
+##
+import logging
+import os
+from edk2toolext.environment.plugintypes.ci_build_plugin import ICiBuildPlugin
+from edk2toollib.uefi.edk2.parsers.dec_parser import DecParser
+from edk2toollib.uefi.edk2.parsers.inf_parser import InfParser
+from edk2toolext.environment.var_dict import VarDict
+
+
+class LibraryClassCheck(ICiBuildPlugin):
+ """
+ A CiBuildPlugin that scans the code tree and library classes for undeclared
+ files
+
+ Configuration options:
+ "LibraryClassCheck": {
+ IgnoreHeaderFile: [], # Ignore a file found on disk
+ IgnoreLibraryClass: [] # Ignore a declaration found in dec file
+ }
+ """
+
+ def GetTestName(self, packagename: str, environment: VarDict) -> tuple:
+ """ Provide the testcase name and classname for use in reporting
+ testclassname: a descriptive string for the testcase can include whitespace
+ classname: should be patterned <packagename>.<plugin>.<optionally any unique condition>
+
+ Args:
+ packagename: string containing name of package to build
+ environment: The VarDict for the test to run in
+ Returns:
+ a tuple containing the testcase name and the classname
+ (testcasename, classname)
+ """
+ return ("Check library class declarations in " + packagename, packagename + ".LibraryClassCheck")
+
+ def __GetPkgDec(self, rootpath):
+ try:
+ allEntries = os.listdir(rootpath)
+ for entry in allEntries:
+ if entry.lower().endswith(".dec"):
+ return(os.path.join(rootpath, entry))
+ except Exception:
+ logging.error("Unable to find DEC for package:{0}".format(rootpath))
+
+ return None
+
+ ##
+ # External function of plugin. This function is used to perform the task of the MuBuild Plugin
+ #
+ # - package is the edk2 path to package. This means workspace/packagepath relative.
+ # - edk2path object configured with workspace and packages path
+ # - PkgConfig Object (dict) for the pkg
+ # - EnvConfig Object
+ # - Plugin Manager Instance
+ # - Plugin Helper Obj Instance
+ # - Junit Logger
+ # - output_stream the StringIO output stream from this plugin via logging
+ def RunBuildPlugin(self, packagename, Edk2pathObj, pkgconfig, environment, PLM, PLMHelper, tc, output_stream=None):
+ overall_status = 0
+ LibraryClassIgnore = []
+
+ abs_pkg_path = Edk2pathObj.GetAbsolutePathOnThisSytemFromEdk2RelativePath(packagename)
+ abs_dec_path = self.__GetPkgDec(abs_pkg_path)
+ wsr_dec_path = Edk2pathObj.GetEdk2RelativePathFromAbsolutePath(abs_dec_path)
+
+ if abs_dec_path is None or wsr_dec_path is "" or not os.path.isfile(abs_dec_path):
+ tc.SetSkipped()
+ tc.LogStdError("No DEC file {0} in package {1}".format(abs_dec_path, abs_pkg_path))
+ return -1
+
+ # Get all include folders
+ dec = DecParser()
+ dec.SetBaseAbsPath(Edk2pathObj.WorkspacePath).SetPackagePaths(Edk2pathObj.PackagePathList)
+ dec.ParseFile(wsr_dec_path)
+
+ AllHeaderFiles = []
+
+ for includepath in dec.IncludePaths:
+ ## Get all header files in the library folder
+ AbsLibraryIncludePath = os.path.join(abs_pkg_path, includepath, "Library")
+ if(not os.path.isdir(AbsLibraryIncludePath)):
+ continue
+
+ hfiles = self.WalkDirectoryForExtension([".h"], AbsLibraryIncludePath)
+ hfiles = [os.path.relpath(x,abs_pkg_path) for x in hfiles] # make package root relative path
+ hfiles = [x.replace("\\", "/") for x in hfiles] # make package relative path
+
+ AllHeaderFiles.extend(hfiles)
+
+ if len(AllHeaderFiles) == 0:
+ tc.SetSkipped()
+ tc.LogStdError(f"No Library include folder in any Include path")
+ return -1
+
+ # Remove ignored paths
+ if "IgnoreHeaderFile" in pkgconfig:
+ for a in pkgconfig["IgnoreHeaderFile"]:
+ try:
+ tc.LogStdOut("Ignoring Library Header File {0}".format(a))
+ AllHeaderFiles.remove(a)
+ except:
+ tc.LogStdError("LibraryClassCheck.IgnoreHeaderFile -> {0} not found. Invalid Header File".format(a))
+ logging.info("LibraryClassCheck.IgnoreHeaderFile -> {0} not found. Invalid Header File".format(a))
+
+ if "IgnoreLibraryClass" in pkgconfig:
+ LibraryClassIgnore = pkgconfig["IgnoreLibraryClass"]
+
+
+ ## Attempt to find library classes
+ for lcd in dec.LibraryClasses:
+ ## Check for correct file path separator
+ if "\\" in lcd.path:
+ tc.LogStdError("LibraryClassCheck.DecFilePathSeparator -> {0} invalid.".format(lcd.path))
+ logging.error("LibraryClassCheck.DecFilePathSeparator -> {0} invalid.".format(lcd.path))
+ overall_status += 1
+ continue
+
+ if lcd.name in LibraryClassIgnore:
+ tc.LogStdOut("Ignoring Library Class Name {0}".format(lcd.name))
+ LibraryClassIgnore.remove(lcd.name)
+ continue
+
+ logging.debug(f"Looking for Library Class {lcd.path}")
+ try:
+ AllHeaderFiles.remove(lcd.path)
+
+ except ValueError:
+ tc.LogStdError(f"Library {lcd.name} with path {lcd.path} not found in package filesystem")
+ logging.error(f"Library {lcd.name} with path {lcd.path} not found in package filesystem")
+ overall_status += 1
+
+ ## any remaining AllHeaderFiles are not described in DEC
+ for h in AllHeaderFiles:
+ tc.LogStdError(f"Library Header File {h} not declared in package DEC {wsr_dec_path}")
+ logging.error(f"Library Header File {h} not declared in package DEC {wsr_dec_path}")
+ overall_status += 1
+
+ ## Warn about any invalid library class names in the ignore list
+ for r in LibraryClassIgnore:
+ tc.LogStdError("LibraryClassCheck.IgnoreLibraryClass -> {0} not found. Library Class not found".format(r))
+ logging.info("LibraryClassCheck.IgnoreLibraryClass -> {0} not found. Library Class not found".format(r))
+
+
+ # If XML object exists, add result
+ if overall_status is not 0:
+ tc.SetFailed("LibraryClassCheck {0} Failed. Errors {1}".format(wsr_dec_path, overall_status), "CHECK_FAILED")
+ else:
+ tc.SetSuccess()
+ return overall_status
diff --git a/.pytool/Plugin/LibraryClassCheck/LibraryClassCheck_plug_in.yaml b/.pytool/Plugin/LibraryClassCheck/LibraryClassCheck_plug_in.yaml
new file mode 100644
index 0000000000..9174453a86
--- /dev/null
+++ b/.pytool/Plugin/LibraryClassCheck/LibraryClassCheck_plug_in.yaml
@@ -0,0 +1,11 @@
+##
+# CiBuildPlugin used to check that all library classes are declared correctly in dec file
+#
+# Copyright (c) 2019, Microsoft Corporation
+# SPDX-License-Identifier: BSD-2-Clause-Patent
+##
+{
+ "scope": "cibuild",
+ "name": "Library Class Check Test",
+ "module": "LibraryClassCheck"
+}
diff --git a/.pytool/Plugin/LibraryClassCheck/readme.md b/.pytool/Plugin/LibraryClassCheck/readme.md
new file mode 100644
index 0000000000..dedee16988
--- /dev/null
+++ b/.pytool/Plugin/LibraryClassCheck/readme.md
@@ -0,0 +1,22 @@
+# Library Class Check Plugin
+
+This CiBuildPlugin scans at all library header files found in the `Library` folders in all of the package's declared include directories and ensures that all files have a matching LibraryClass declaration in the DEC file for the package. Any missing declarations will cause a failure.
+
+## Configuration
+
+The plugin has a few configuration options to support the UEFI codebase.
+
+``` yaml
+"LibraryClassCheck": {
+ IgnoreHeaderFile: [], # Ignore a file found on disk
+ IgnoreLibraryClass: [] # Ignore a declaration found in dec file
+}
+```
+
+### IgnoreHeaderFile
+
+Ignore a file found on disk
+
+### IgnoreLibraryClass
+
+Ignore a declaration found in dec file
diff --git a/.pytool/Plugin/SpellCheck/Readme.md b/.pytool/Plugin/SpellCheck/Readme.md
new file mode 100644
index 0000000000..e0ac835191
--- /dev/null
+++ b/.pytool/Plugin/SpellCheck/Readme.md
@@ -0,0 +1,100 @@
+# Spell Check Plugin
+
+This CiBuildPlugin scans all the files in a given package and checks for spelling errors.
+
+This plugin requires NodeJs and cspell. If the plugin doesn't find its required tools then it will mark the test as skipped.
+
+* NodeJS: https://nodejs.org/en/
+* cspell: https://www.npmjs.com/package/cspell
+ * Src and doc available: https://github.com/streetsidesoftware/cspell
+
+## Configuration
+
+The plugin has a few configuration options to support the UEFI codebase.
+
+``` yaml
+ "SpellCheck": {
+ "AuditOnly": False, # If True, log all errors and then mark as skipped
+ "IgnoreFiles": [], # use gitignore syntax to ignore errors in matching files
+ "ExtendWords": [], # words to extend to the dictionary for this package
+ "IgnoreStandardPaths": [], # Standard Plugin defined paths that should be ignore
+ "AdditionalIncludePaths": [] # Additional paths to spell check (wildcards supported)
+ }
+```
+
+### AuditOnly
+
+Boolean - Default is False.
+If True run the test in an Audit only mode which will log all errors but instead of failing the build it will set the test as skipped. This allows visibility into the failures without breaking the build.
+
+### IgnoreFiles
+
+This supports .gitignore file and folder matching strings including wildcards
+
+* All files will be parsed regardless but then any spelling errors found within ignored files will not be reported as an error.
+* Errors in ignored files will still be output to the test results as informational comments.
+
+### ExtendWords
+
+This list allows words to be added to the dictionary for the spell checker when this package is tested. These follow the rules of the cspell config words field.
+
+### IgnoreStandardPaths
+
+This plugin by default will check the below standard paths. If the package would like to ignore any of them list that here.
+
+```python
+[
+# C source
+"*.c",
+"*.h",
+
+# Assembly files
+"*.nasm",
+"*.asm",
+"*.masm",
+"*.s",
+
+# ACPI source language
+"*.asl",
+
+# Edk2 build files
+"*.dsc", "*.dec", "*.fdf", "*.inf",
+
+# Documentation files
+"*.md", "*.txt"
+]
+```
+
+### AdditionalIncludePaths
+
+If the package would to add additional path patterns to be included in spellchecking they can be defined here.
+
+## Other configuration
+
+In the cspell.base.json there are numerous other settings configured. There is no support to override these on a per package basis but future features could make this available. One interesting configuration option is `minWordLength`. Currently it is set to _5_ which means all 2,3, and 4 letter words will be ignored. This helps minimize the number of technical acronyms, register names, and other UEFI specific values that must be ignored .
+
+## False positives
+
+The cspell dictionary is not perfect and there are cases where technical words or acronyms are not found in the dictionary. There are three ways to resolve false positives and the choice for which method should be based on how broadly the word should be accepted.
+
+### CSpell Base Config file
+
+If the change should apply to all UEFI code and documentation then it should be added to the base config file `words` section. The base config file is adjacent to this file and titled `cspell.base.json`. This is a list of accepted words for all spell checking operations on all packages.
+
+### Package Config
+
+In the package `*.ci.yaml` file there is a `SpellCheck` config section. This section allows files to be ignored as well as words that should be considered valid for all files within this package. Add the desired words to the "ExtendedWords" member.
+
+### In-line File
+
+CSpell supports numerous methods to annotate your files to ignore words, sections, etc. This can be found in CSpell documentation. Suggestion here is to use a c-style comment at the top of the file to add words that should be ignored just for this file. Obviously this has the highest maintenance cost so it should only be used for file unique words.
+
+``` c
+// spell-checker:ignore unenroll, word2, word3
+```
+
+or
+
+```ini
+# spell-checker:ignore unenroll, word2, word3
+```
diff --git a/.pytool/Plugin/SpellCheck/SpellCheck.py b/.pytool/Plugin/SpellCheck/SpellCheck.py
new file mode 100644
index 0000000000..94ca4cd071
--- /dev/null
+++ b/.pytool/Plugin/SpellCheck/SpellCheck.py
@@ -0,0 +1,216 @@
+# @file SpellCheck.py
+#
+# An edk2-pytool based plugin wrapper for cspell
+#
+# Copyright (c) Microsoft Corporation. All rights reserved.
+# SPDX-License-Identifier: BSD-2-Clause-Patent
+##
+import logging
+import json
+import yaml
+from io import StringIO
+import os
+from edk2toolext.environment.plugintypes.ci_build_plugin import ICiBuildPlugin
+from edk2toollib.utility_functions import RunCmd
+from edk2toolext.environment.var_dict import VarDict
+from edk2toollib.gitignore_parser import parse_gitignore_lines
+from edk2toolext.environment import version_aggregator
+
+
+class SpellCheck(ICiBuildPlugin):
+ """
+ A CiBuildPlugin that uses the cspell node module to scan the files
+ from the package being tested for spelling errors. The plugin contains
+ the base cspell.json file then thru the configuration options other settings
+ can be changed or extended.
+
+ Configuration options:
+ "SpellCheck": {
+ "AuditOnly": False, # Don't fail the build if there are errors. Just log them
+ "IgnoreFiles": [], # use gitignore syntax to ignore errors in matching files
+ "ExtendWords": [], # words to extend to the dictionary for this package
+ "IgnoreStandardPaths": [], # Standard Plugin defined paths that should be ignore
+ "AdditionalIncludePaths": [] # Additional paths to spell check (wildcards supported)
+ }
+ """
+
+ #
+ # A package can remove any of these using IgnoreStandardPaths
+ #
+ STANDARD_PLUGIN_DEFINED_PATHS = ["*.c", "*.h",
+ "*.nasm", "*.asm", "*.masm", "*.s",
+ "*.asl",
+ "*.dsc", "*.dec", "*.fdf", "*.inf",
+ "*.md", "*.txt"
+ ]
+
+ def GetTestName(self, packagename: str, environment: VarDict) -> tuple:
+ """ Provide the testcase name and classname for use in reporting
+
+ Args:
+ packagename: string containing name of package to build
+ environment: The VarDict for the test to run in
+ Returns:
+ a tuple containing the testcase name and the classname
+ (testcasename, classname)
+ testclassname: a descriptive string for the testcase can include whitespace
+ classname: should be patterned <packagename>.<plugin>.<optionally any unique condition>
+ """
+ return ("Spell check files in " + packagename, packagename + ".SpellCheck")
+
+ ##
+ # External function of plugin. This function is used to perform the task of the CiBuild Plugin
+ #
+ # - package is the edk2 path to package. This means workspace/packagepath relative.
+ # - edk2path object configured with workspace and packages path
+ # - PkgConfig Object (dict) for the pkg
+ # - EnvConfig Object
+ # - Plugin Manager Instance
+ # - Plugin Helper Obj Instance
+ # - Junit Logger
+ # - output_stream the StringIO output stream from this plugin via logging
+
+ def RunBuildPlugin(self, packagename, Edk2pathObj, pkgconfig, environment, PLM, PLMHelper, tc, output_stream=None):
+ Errors = []
+
+ abs_pkg_path = Edk2pathObj.GetAbsolutePathOnThisSytemFromEdk2RelativePath(
+ packagename)
+
+ if abs_pkg_path is None:
+ tc.SetSkipped()
+ tc.LogStdError("No package {0}".format(packagename))
+ return -1
+
+ # check for node
+ return_buffer = StringIO()
+ ret = RunCmd("node", "--version", outstream=return_buffer)
+ if (ret != 0):
+ tc.SetSkipped()
+ tc.LogStdError("NodeJs not installed. Test can't run")
+ logging.warning("NodeJs not installed. Test can't run")
+ return -1
+ node_version = return_buffer.getvalue().strip() # format vXX.XX.XX
+ tc.LogStdOut(f"Node version: {node_version}")
+ version_aggregator.GetVersionAggregator().ReportVersion(
+ "NodeJs", node_version, version_aggregator.VersionTypes.INFO)
+
+ # Check for cspell
+ return_buffer = StringIO()
+ ret = RunCmd("cspell", "--version", outstream=return_buffer)
+ if (ret != 0):
+ tc.SetSkipped()
+ tc.LogStdError("cspell not installed. Test can't run")
+ logging.warning("cspell not installed. Test can't run")
+ return -1
+ cspell_version = return_buffer.getvalue().strip() # format XX.XX.XX
+ tc.LogStdOut(f"CSpell version: {cspell_version}")
+ version_aggregator.GetVersionAggregator().ReportVersion(
+ "CSpell", cspell_version, version_aggregator.VersionTypes.INFO)
+
+ package_relative_paths_to_spell_check = SpellCheck.STANDARD_PLUGIN_DEFINED_PATHS
+
+ #
+ # Allow the ci.yaml to remove any of the above standard paths
+ #
+ if("IgnoreStandardPaths" in pkgconfig):
+ for a in pkgconfig["IgnoreStandardPaths"]:
+ if(a in package_relative_paths_to_spell_check):
+ tc.LogStdOut(
+ f"ignoring standard path due to ci.yaml ignore: {a}")
+ package_relative_paths_to_spell_check.remove(a)
+ else:
+ tc.LogStdOut(f"Invalid IgnoreStandardPaths value: {a}")
+
+ #
+ # check for any additional include paths defined by package config
+ #
+ if("AdditionalIncludePaths" in pkgconfig):
+ package_relative_paths_to_spell_check.extend(
+ pkgconfig["AdditionalIncludePaths"])
+
+ #
+ # Make the path string for cspell to check
+ #
+ relpath = os.path.relpath(abs_pkg_path)
+ cpsell_paths = " ".join(
+ [f"{relpath}/**/{x}" for x in package_relative_paths_to_spell_check])
+
+ # Make the config file
+ config_file_path = os.path.join(
+ Edk2pathObj.WorkspacePath, "Build", packagename, "cspell_actual_config.json")
+ mydir = os.path.dirname(os.path.abspath(__file__))
+ # load as yaml so it can have comments
+ base = os.path.join(mydir, "cspell.base.yaml")
+ with open(base, "r") as i:
+ config = yaml.safe_load(i)
+
+ if("ExtendWords" in pkgconfig):
+ config["words"].extend(pkgconfig["ExtendWords"])
+ with open(config_file_path, "w") as o:
+ json.dump(config, o) # output as json so compat with cspell
+
+ All_Ignores = []
+ # Parse the config for other ignores
+ if "IgnoreFiles" in pkgconfig:
+ All_Ignores.extend(pkgconfig["IgnoreFiles"])
+
+ # spell check all the files
+ ignore = parse_gitignore_lines(All_Ignores, os.path.join(
+ abs_pkg_path, "nofile.txt"), abs_pkg_path)
+
+ # result is a list of strings like this
+ # C:\src\sp-edk2\edk2\FmpDevicePkg\FmpDevicePkg.dec:53:9 - Unknown word (Capule)
+ EasyFix = []
+ results = self._check_spelling(cpsell_paths, config_file_path)
+ for r in results:
+ path, _, word = r.partition(" - Unknown word ")
+ if len(word) == 0:
+ # didn't find pattern
+ continue
+
+ pathinfo = path.rsplit(":", 2) # remove the line no info
+ if(ignore(pathinfo[0])): # check against ignore list
+ tc.LogStdOut(f"ignoring error due to ci.yaml ignore: {r}")
+ continue
+
+ # real error
+ EasyFix.append(word.strip().strip("()"))
+ Errors.append(r)
+
+ # Log all errors tc StdError
+ for l in Errors:
+ tc.LogStdError(l.strip())
+
+ # Helper - Log the syntax needed to add these words to dictionary
+ if len(EasyFix) > 0:
+ EasyFix = sorted(set(a.lower() for a in EasyFix))
+ tc.LogStdOut("\n Easy fix:")
+ OneString = "If these are not errors add this to your ci.yaml file.\n"
+ OneString += '"SpellCheck": {\n "ExtendWords": ['
+ for a in EasyFix:
+ tc.LogStdOut(f'\n"{a}",')
+ OneString += f'\n "{a}",'
+ logging.info(OneString.rstrip(",") + '\n ]\n}')
+
+ # add result to test case
+ overall_status = len(Errors)
+ if overall_status != 0:
+ if "AuditOnly" in pkgconfig and pkgconfig["AuditOnly"]:
+ # set as skipped if AuditOnly
+ tc.SetSkipped()
+ return -1
+ else:
+ tc.SetFailed("SpellCheck {0} Failed. Errors {1}".format(
+ packagename, overall_status), "CHECK_FAILED")
+ else:
+ tc.SetSuccess()
+ return overall_status
+
+ def _check_spelling(self, abs_file_to_check: str, abs_config_file_to_use: str) -> []:
+ output = StringIO()
+ ret = RunCmd(
+ "cspell", f"--config {abs_config_file_to_use} {abs_file_to_check}", outstream=output)
+ if ret == 0:
+ return []
+ else:
+ return output.getvalue().strip().splitlines()
diff --git a/.pytool/Plugin/SpellCheck/SpellCheck_plug_in.yaml b/.pytool/Plugin/SpellCheck/SpellCheck_plug_in.yaml
new file mode 100644
index 0000000000..161045e19e
--- /dev/null
+++ b/.pytool/Plugin/SpellCheck/SpellCheck_plug_in.yaml
@@ -0,0 +1,11 @@
+##
+# CiBuildPlugin used to check spelling
+#
+# Copyright (c) 2019, Microsoft Corporation
+# SPDX-License-Identifier: BSD-2-Clause-Patent
+##
+{
+ "scope": "cibuild",
+ "name": "Spell Check Test",
+ "module": "SpellCheck"
+}
diff --git a/.pytool/Plugin/SpellCheck/cspell.base.yaml b/.pytool/Plugin/SpellCheck/cspell.base.yaml
new file mode 100644
index 0000000000..da6c5e5da7
--- /dev/null
+++ b/.pytool/Plugin/SpellCheck/cspell.base.yaml
@@ -0,0 +1,165 @@
+##
+# CSpell configuration
+#
+# Copyright (c) Microsoft Corporation
+# SPDX-License-Identifier: BSD-2-Clause-Patent
+##
+{
+ "version": "0.1",
+ "language": "en",
+ "dictionaries": [
+ "companies ",
+ "softwareTerms",
+ "python",
+ "cpp"
+ ],
+ "ignorePaths": [
+ "*.pdb",
+ "**/*_extdep/**",
+ "*.pdf",
+ "*.exe",
+ "*.jpg"
+ ],
+ "minWordLength": 5,
+ "allowCompoundWords": false,
+ "ignoreWords": [
+ "muchange"
+ ],
+ "words": [
+ "MTRRs",
+ "Microarchitecture",
+ "Goldmont",
+ "cpuid",
+ "mwait",
+ "cstate",
+ "smram",
+ "scrtm",
+ "smbus",
+ "selftest",
+ "socket",
+ "MMRAM",
+ "qword",
+ "ENDBR",
+ "SMBASE",
+ "FXSAVE",
+ "FXRSTOR",
+ "RDRAND",
+ "IOAPIC",
+ "ATAPI",
+ "movsb",
+ "iretw",
+ "XENSTORE",
+ "cdrom",
+ "oprom",
+ "oproms",
+ "varstore",
+ "EKU",
+ "ascii",
+ "nmake",
+ "NVDIMM",
+ "nasmb",
+ "Mtftp",
+ "Hypercall",
+ "hypercalls",
+ "IOMMU",
+ "QEMU",
+ "qemus",
+ "OVMF",
+ "tiano",
+ "tianocore",
+ "edkii",
+ "coreboot",
+ "uefipayload",
+ "bootloader",
+ "bootloaders",
+ "mdepkg",
+ "skuid",
+ "dxefv",
+ "toolchain",
+ "libraryclass",
+ "preboot",
+ "pythonpath",
+ "cygpath",
+ "nuget",
+ "basetools",
+ "prepi",
+ "OPTEE",
+ "stringid",
+ "peims",
+ "memmap",
+ "guids",
+ "uuids",
+ "smbios",
+ "certdb",
+ "certdbv",
+ "EfiSigList",
+ "depex",
+ "IHANDLE",
+ "Virtio",
+ "Mbytes",
+ "Citrix",
+ "initrd",
+ "semihost",
+ "Semihosting",
+ "Trustzone",
+ "Fastboot",
+ "framebuffer",
+ "genfw",
+ "TTYTERM",
+ "miniport",
+ "LFENCE",
+ "PCANSI",
+ "submodule",
+ "submodules",
+ "brotli",
+ "PCCTS",
+ "softfloat",
+ "whitepaper",
+ "ACPICA",
+ "plugfest",
+ "bringup",
+ "formset", #VFR
+ "ideqvallist",
+ "numberof",
+ "oneof",
+ "endformset",
+ "endnumeric",
+ "endoneof",
+ "disableif",
+ "guidid",
+ "classguid",
+ "efivarstore",
+ "formsetguid",
+ "formid",
+ "suppressif",
+ "grayoutif",
+ "ideqval",
+ "endform",
+ "endcheckbox",
+ "questionid",
+ "questionref",
+ "enddate",
+ "endstring",
+ "guidop",
+ "endguidop",
+ "langdef",
+ "dynamicex",
+ "tokenspace",
+ "tokenguid",
+ "pcd's", #seems like cspell bug
+ "peim's",
+ "autogen",
+ "Disasm",
+ "Torito",
+ "SRIOV",
+ "MRIOV",
+ "UARTs",
+ "Consplitter", # common module in UEFI
+ "FIFOs",
+ "ACPINVS",
+ "Endof", # due to of not being uppercase
+ "bootability",
+ "Sdhci",
+ "inmodule",
+ ]
+}
--
2.21.0.windows.1
^ permalink raw reply related [flat|nested] 56+ messages in thread* [Patch v3 08/22] CryptoPkg: Add YAML file for CI builds
2019-10-29 19:54 [Patch v3 00/22] Enable Phase 1 of EDK II CI Michael D Kinney
` (6 preceding siblings ...)
2019-10-29 19:55 ` [Patch v3 07/22] .pytool/Plugin: Add CI plugins Michael D Kinney
@ 2019-10-29 19:55 ` Michael D Kinney
2019-10-29 19:55 ` [Patch v3 09/22] FatPkg: " Michael D Kinney
` (14 subsequent siblings)
22 siblings, 0 replies; 56+ messages in thread
From: Michael D Kinney @ 2019-10-29 19:55 UTC (permalink / raw)
To: devel; +Cc: Jian J Wang, Xiaoyu Lu, Kinney
https://bugzilla.tianocore.org/show_bug.cgi?id=2315
Add YAML file to the package directory with the
configuration of the checks to perform during a
CI build.
Cc: Jian J Wang <jian.j.wang@intel.com>
Cc: Xiaoyu Lu <xiaoyux.lu@intel.com>
Signed-off-by: Kinney <michael.d.kinney@intel.com>
---
CryptoPkg/CryptoPkg.ci.yaml | 48 +++++++++++++++++++++++++++++++++++++
1 file changed, 48 insertions(+)
create mode 100644 CryptoPkg/CryptoPkg.ci.yaml
diff --git a/CryptoPkg/CryptoPkg.ci.yaml b/CryptoPkg/CryptoPkg.ci.yaml
new file mode 100644
index 0000000000..a20ac79d0e
--- /dev/null
+++ b/CryptoPkg/CryptoPkg.ci.yaml
@@ -0,0 +1,48 @@
+##
+# CI configuration for CryptoPkg
+#
+# Copyright (c) Microsoft Corporation
+# SPDX-License-Identifier: BSD-2-Clause-Patent
+##
+{
+ "CompilerPlugin": {
+ "DscPath": "CryptoPkg.dsc"
+ },
+ "CharEncodingCheck": {
+ "IgnoreFiles": []
+ },
+ "DependencyCheck": {
+ "AcceptableDependencies": [
+ "MdePkg/MdePkg.dec",
+ "MdeModulePkg/MdeModulePkg.dec",
+ "CryptoPkg/CryptoPkg.dec",
+ #"SecurityPkg/SecurityPkg.dec"
+ ],
+ # For host based unit tests
+ "AcceptableDependencies-HOST_APPLICATION":[],
+ # For UEFI shell based apps
+ "AcceptableDependencies-UEFI_APPLICATION":[],
+ "IgnoreInf": []
+ },
+ "DscCompleteCheck": {
+ "DscPath": "CryptoPkg.dsc",
+ "IgnoreInf": []
+ },
+ "GuidCheck": {
+ "IgnoreGuidName": [],
+ "IgnoreGuidValue": [],
+ "IgnoreFoldersAndFiles": []
+ },
+ "LibraryClassCheck": {
+ "IgnoreHeaderFile": []
+ },
+
+ ## options defined ci/Plugin/SpellCheck
+ "SpellCheck": {
+ "skip": True,
+ "IgnoreFiles": [], # use gitignore syntax to ignore errors in matching files
+ "ExtendWords": [], # words to extend to the dictionary for this package
+ "IgnoreStandardPaths": [], # Standard Plugin defined paths that should be ignore
+ "AdditionalIncludePaths": [] # Additional paths to spell check (wildcards supported)
+ }
+}
--
2.21.0.windows.1
^ permalink raw reply related [flat|nested] 56+ messages in thread* [Patch v3 09/22] FatPkg: Add YAML file for CI builds
2019-10-29 19:54 [Patch v3 00/22] Enable Phase 1 of EDK II CI Michael D Kinney
` (7 preceding siblings ...)
2019-10-29 19:55 ` [Patch v3 08/22] CryptoPkg: Add YAML file for CI builds Michael D Kinney
@ 2019-10-29 19:55 ` Michael D Kinney
2019-10-29 19:55 ` [Patch v3 10/22] FmpDevicePkg: " Michael D Kinney
` (13 subsequent siblings)
22 siblings, 0 replies; 56+ messages in thread
From: Michael D Kinney @ 2019-10-29 19:55 UTC (permalink / raw)
To: devel; +Cc: Ray Ni
https://bugzilla.tianocore.org/show_bug.cgi?id=2315
Add YAML file to the package directory with the
configuration of the checks to perform during a
CI build.
Cc: Ray Ni <ray.ni@intel.com>
Signed-off-by: Michael D Kinney <michael.d.kinney@intel.com>
---
FatPkg/FatPkg.ci.yaml | 50 +++++++++++++++++++++++++++++++++++++++++++
1 file changed, 50 insertions(+)
create mode 100644 FatPkg/FatPkg.ci.yaml
diff --git a/FatPkg/FatPkg.ci.yaml b/FatPkg/FatPkg.ci.yaml
new file mode 100644
index 0000000000..d584eea0a7
--- /dev/null
+++ b/FatPkg/FatPkg.ci.yaml
@@ -0,0 +1,50 @@
+##
+# CI configuration for FatPkg
+#
+# Copyright (c) Microsoft Corporation
+# SPDX-License-Identifier: BSD-2-Clause-Patent
+##
+{
+ "CompilerPlugin": {
+ "DscPath": "FatPkg.dsc"
+ },
+ "CharEncodingCheck": {
+ "IgnoreFiles": []
+ },
+ "DependencyCheck": {
+ "AcceptableDependencies": [
+ "MdePkg/MdePkg.dec",
+ "MdeModulePkg/MdeModulePkg.dec",
+ ],
+ # For host based unit tests
+ "AcceptableDependencies-HOST_APPLICATION":[],
+ # For UEFI shell based apps
+ "AcceptableDependencies-UEFI_APPLICATION":[],
+ "IgnoreInf": []
+ },
+ "DscCompleteCheck": {
+ "IgnoreInf": [],
+ "DscPath": "FatPkg.dsc"
+ },
+ "GuidCheck": {
+ "IgnoreGuidName": [],
+ "IgnoreGuidValue": [],
+ "IgnoreFoldersAndFiles": []
+ },
+ "LibraryClassCheck": {
+ "IgnoreHeaderFile": []
+ },
+ "SpellCheck": {
+ "ExtendWords": [
+ "ELTORITO",
+ "FHAND",
+ "IFILE",
+ "OFILE",
+ "FDISKed",
+ "Lfnbuffer",
+ "FFFFFFFFL",
+ "CDVOL",
+ "DMDEPKG"
+ ]
+ }
+}
--
2.21.0.windows.1
^ permalink raw reply related [flat|nested] 56+ messages in thread* [Patch v3 10/22] FmpDevicePkg: Add YAML file for CI builds
2019-10-29 19:54 [Patch v3 00/22] Enable Phase 1 of EDK II CI Michael D Kinney
` (8 preceding siblings ...)
2019-10-29 19:55 ` [Patch v3 09/22] FatPkg: " Michael D Kinney
@ 2019-10-29 19:55 ` Michael D Kinney
2019-10-29 19:55 ` [Patch v3 11/22] MdeModulePkg: " Michael D Kinney
` (12 subsequent siblings)
22 siblings, 0 replies; 56+ messages in thread
From: Michael D Kinney @ 2019-10-29 19:55 UTC (permalink / raw)
To: devel; +Cc: Liming Gao
https://bugzilla.tianocore.org/show_bug.cgi?id=2315
Add YAML file to the package directory with the
configuration of the checks to perform during a
CI build.
Use BaseCryptLibNull for package CI builds to reduce package
build times. Enabled with CONTINUOUS_INTEGRATION in YAML
files. By default CONTINUOUS_INTEGRATION is not defined,
and the original lib mappings are preserved.
Cc: Liming Gao <liming.gao@intel.com>
Signed-off-by: Michael D Kinney <michael.d.kinney@intel.com>
---
FmpDevicePkg/FmpDevicePkg.ci.yaml | 43 +++++++++++++++++++++++++++++++
FmpDevicePkg/FmpDevicePkg.dsc | 6 ++++-
2 files changed, 48 insertions(+), 1 deletion(-)
create mode 100644 FmpDevicePkg/FmpDevicePkg.ci.yaml
diff --git a/FmpDevicePkg/FmpDevicePkg.ci.yaml b/FmpDevicePkg/FmpDevicePkg.ci.yaml
new file mode 100644
index 0000000000..f089932556
--- /dev/null
+++ b/FmpDevicePkg/FmpDevicePkg.ci.yaml
@@ -0,0 +1,43 @@
+##
+# CI configuration for FmpDevicePkg
+#
+# Copyright (c) Microsoft Corporation
+# SPDX-License-Identifier: BSD-2-Clause-Patent
+##
+{
+ "CompilerPlugin": {
+ "DscPath": "FmpDevicePkg.dsc"
+ },
+ "CharEncodingCheck": {
+ "IgnoreFiles": []
+ },
+ "DependencyCheck": {
+ "AcceptableDependencies": [
+ "MdePkg/MdePkg.dec",
+ "MdeModulePkg/MdeModulePkg.dec",
+ "FmpDevicePkg/FmpDevicePkg.dec",
+ "CryptoPkg/CryptoPkg.dec"
+ ],
+ "IgnoreInf": []
+ },
+ "DscCompleteCheck": {
+ "DscPath": "FmpDevicePkg.dsc",
+ "IgnoreInf": []
+ },
+ "GuidCheck": {
+ "IgnoreGuidName": [],
+ "IgnoreGuidValue": [],
+ "IgnoreFoldersAndFiles": []
+ },
+ "LibraryClassCheck": {
+ "IgnoreHeaderFile": []
+ },
+ "SpellCheck": {
+ "ExtendWords": [
+ "FMPSTATE",
+ ]
+ },
+ "Defines": {
+ "BLD_*_CONTINUOUS_INTEGRATION": "TRUE",
+ }
+}
diff --git a/FmpDevicePkg/FmpDevicePkg.dsc b/FmpDevicePkg/FmpDevicePkg.dsc
index 67b5aa7a32..f4093d3837 100644
--- a/FmpDevicePkg/FmpDevicePkg.dsc
+++ b/FmpDevicePkg/FmpDevicePkg.dsc
@@ -48,9 +48,13 @@ [LibraryClasses]
DebugLib|MdePkg/Library/UefiDebugLibStdErr/UefiDebugLibStdErr.inf
DebugPrintErrorLevelLib|MdePkg/Library/BaseDebugPrintErrorLevelLib/BaseDebugPrintErrorLevelLib.inf
PcdLib|MdePkg/Library/BasePcdLibNull/BasePcdLibNull.inf
- OpensslLib|CryptoPkg/Library/OpensslLib/OpensslLib.inf
+!ifdef CONTINUOUS_INTEGRATION
+ BaseCryptLib|CryptoPkg/Library/BaseCryptLibNull/BaseCryptLibNull.inf
+!else
IntrinsicLib|CryptoPkg/Library/IntrinsicLib/IntrinsicLib.inf
+ OpensslLib|CryptoPkg/Library/OpensslLib/OpensslLib.inf
BaseCryptLib|CryptoPkg/Library/BaseCryptLib/BaseCryptLib.inf
+!endif
FmpAuthenticationLib|SecurityPkg/Library/FmpAuthenticationLibPkcs7/FmpAuthenticationLibPkcs7.inf
CapsuleUpdatePolicyLib|FmpDevicePkg/Library/CapsuleUpdatePolicyLibNull/CapsuleUpdatePolicyLibNull.inf
FmpPayloadHeaderLib|FmpDevicePkg/Library/FmpPayloadHeaderLibV1/FmpPayloadHeaderLibV1.inf
--
2.21.0.windows.1
^ permalink raw reply related [flat|nested] 56+ messages in thread* [Patch v3 11/22] MdeModulePkg: Add YAML file for CI builds
2019-10-29 19:54 [Patch v3 00/22] Enable Phase 1 of EDK II CI Michael D Kinney
` (9 preceding siblings ...)
2019-10-29 19:55 ` [Patch v3 10/22] FmpDevicePkg: " Michael D Kinney
@ 2019-10-29 19:55 ` Michael D Kinney
2019-10-29 19:55 ` [Patch v3 12/22] MdePkg: " Michael D Kinney
` (11 subsequent siblings)
22 siblings, 0 replies; 56+ messages in thread
From: Michael D Kinney @ 2019-10-29 19:55 UTC (permalink / raw)
To: devel; +Cc: Jian J Wang, Hao A Wu
https://bugzilla.tianocore.org/show_bug.cgi?id=2315
Add YAML file to the package directory with the
configuration of the checks to perform during a
CI build.
Cc: Jian J Wang <jian.j.wang@intel.com>
Cc: Hao A Wu <hao.a.wu@intel.com>
Signed-off-by: Michael D Kinney <michael.d.kinney@intel.com>
---
MdeModulePkg/MdeModulePkg.ci.yaml | 81 +++++++++++++++++++++++++++++++
1 file changed, 81 insertions(+)
create mode 100644 MdeModulePkg/MdeModulePkg.ci.yaml
diff --git a/MdeModulePkg/MdeModulePkg.ci.yaml b/MdeModulePkg/MdeModulePkg.ci.yaml
new file mode 100644
index 0000000000..6ae58d5379
--- /dev/null
+++ b/MdeModulePkg/MdeModulePkg.ci.yaml
@@ -0,0 +1,81 @@
+##
+# CI configuration for MdeModulePkg
+#
+# Copyright (c) Microsoft Corporation
+# SPDX-License-Identifier: BSD-2-Clause-Patent
+##
+{
+ ## options defined ci/Plugin/CompilerPlugin
+ "CompilerPlugin": {
+ "DscPath": "MdeModulePkg.dsc"
+ },
+
+ ## options defined ci/Plugin/CharEncodingCheck
+ "CharEncodingCheck": {
+ "IgnoreFiles": []
+ },
+
+ ## options defined ci/Plugin/DependencyCheck
+ "DependencyCheck": {
+ "AcceptableDependencies": [
+ "MdePkg/MdePkg.dec",
+ "MdeModulePkg/MdeModulePkg.dec",
+ "StandaloneMmPkg/StandaloneMmPkg.dec",
+ "ArmPkg/ArmPkg.dec" # this should be fixed by promoting an abstraction
+ ],
+ # For host based unit tests
+ "AcceptableDependencies-HOST_APPLICATION":[],
+ # For UEFI shell based apps
+ "AcceptableDependencies-UEFI_APPLICATION":[],
+ "IgnoreInf": []
+ },
+
+ ## options defined ci/Plugin/DscCompleteCheck
+ "DscCompleteCheck": {
+ "IgnoreInf": [],
+ "DscPath": "MdeModulePkg.dsc"
+ },
+
+ ## options defined ci/Plugin/GuidCheck
+ "GuidCheck": {
+ "IgnoreGuidName": [],
+ "IgnoreGuidValue": ["00000000-0000-0000-0000-000000000000"],
+ "IgnoreFoldersAndFiles": [],
+ "IgnoreDuplicates": [
+ "gEdkiiFormBrowserExProtocolGuid=gEfiFormBrowserExProtocolGuid",
+ "gEfiPeiMmAccessPpiGuid=gPeiSmmAccessPpiGuid",
+ "gPeiSmmControlPpiGuid=gEfiPeiMmControlPpiGuid",
+ "gEdkiiSerialPortLibVendorGuid=SerialDxe" # Is this a bug????
+ ]
+ },
+
+ ## options defined ci/Plugin/LibraryClassCheck
+ "LibraryClassCheck": {
+ "IgnoreHeaderFile": []
+ },
+
+ ## options defined ci/Plugin/SpellCheck
+ "SpellCheck": {
+ "AuditOnly": True, # Fails test but run in AuditOnly mode to collect log
+ "IgnoreStandardPaths": [ # Standard Plugin defined paths that should be ignore
+ "*.c", "*.asm", "*.h", "*.nasm", "*.s", "*.asl", "*.inf"
+ ],
+ "IgnoreFiles": [ # use gitignore syntax to ignore errors in matching files
+ "Library/LzmaCustomDecompressLib/Sdk/DOC/*"
+ ],
+ "ExtendWords": [ # words to extend to the dictionary for this package
+ "LIGHTGRAY",
+ "DARKGRAY",
+ "LIGHTBLUE",
+ "LIGHTGREEN",
+ "LIGHTCYAN",
+ "LIGHTRED",
+ "LIGHTMAGENTA",
+ "FVMAIN",
+ "VARCHECKPCD",
+ "Getxx",
+ "lzturbo"
+ ],
+ "AdditionalIncludePaths": [] # Additional paths to spell check relative to package root (wildcards supported)
+ }
+}
--
2.21.0.windows.1
^ permalink raw reply related [flat|nested] 56+ messages in thread* [Patch v3 12/22] MdePkg: Add YAML file for CI builds
2019-10-29 19:54 [Patch v3 00/22] Enable Phase 1 of EDK II CI Michael D Kinney
` (10 preceding siblings ...)
2019-10-29 19:55 ` [Patch v3 11/22] MdeModulePkg: " Michael D Kinney
@ 2019-10-29 19:55 ` Michael D Kinney
2019-11-05 8:22 ` Liming Gao
2019-10-29 19:55 ` [Patch v3 13/22] NetworkPkg: " Michael D Kinney
` (10 subsequent siblings)
22 siblings, 1 reply; 56+ messages in thread
From: Michael D Kinney @ 2019-10-29 19:55 UTC (permalink / raw)
To: devel; +Cc: Liming Gao
https://bugzilla.tianocore.org/show_bug.cgi?id=2315
Add YAML file to the package directory with the
configuration of the checks to perform during a
CI build.
Cc: Liming Gao <liming.gao@intel.com>
Signed-off-by: Michael D Kinney <michael.d.kinney@intel.com>
---
MdePkg/MdePkg.ci.yaml | 90 +++++++++++++++++++++++++++++++++++++++++++
1 file changed, 90 insertions(+)
create mode 100644 MdePkg/MdePkg.ci.yaml
diff --git a/MdePkg/MdePkg.ci.yaml b/MdePkg/MdePkg.ci.yaml
new file mode 100644
index 0000000000..4015c07f16
--- /dev/null
+++ b/MdePkg/MdePkg.ci.yaml
@@ -0,0 +1,90 @@
+##
+# CI configuration for MdePkg
+#
+# Copyright (c) Microsoft Corporation
+# SPDX-License-Identifier: BSD-2-Clause-Patent
+##
+{
+ ## options defined ci/Plugin/CompilerPlugin
+ "CompilerPlugin": {
+ "DscPath": "MdePkg.dsc"
+ },
+
+ ## options defined ci/Plugin/CharEncodingCheck
+"CharEncodingCheck": {
+ "IgnoreFiles": []
+ },
+
+ ## options defined ci/Plugin/DependencyCheck
+ "DependencyCheck": {
+ "AcceptableDependencies": [
+ "MdePkg/MdePkg.dec"
+ ],
+ # For host based unit tests
+ "AcceptableDependencies-HOST_APPLICATION":[],
+ # For UEFI shell based apps
+ "AcceptableDependencies-UEFI_APPLICATION":[],
+ "IgnoreInf": []
+ },
+
+ ## options defined ci/Plugin/DscCompleteCheck
+ "DscCompleteCheck": {
+ "IgnoreInf": [],
+ "DscPath": "MdePkg.dsc"
+ },
+
+ ## options defined ci/Plugin/GuidCheck
+ "GuidCheck": {
+ "IgnoreGuidName": [
+ "gEfiFirmwareVolumeTopFileGuid" # sec modules must be set to this guid
+ ],
+ "IgnoreGuidValue": ["00000000-0000-0000-0000-000000000000"],
+ "IgnoreFoldersAndFiles": [],
+ "IgnoreDuplicates": [
+ "gEfiFirmwareVolumeBlockProtocolGuid=gEfiFirmwareVolumeBlock2ProtocolGuid",
+ "gEfiMmAccessProtocolGuid=gEfiSmmAccess2ProtocolGuid",
+ "gEfiSmmCommunicationProtocolGuid=gEfiMmCommunicationProtocolGuid",
+ "gEfiMmPciRootBridgeIoProtocolGuid=gEfiSmmPciRootBridgeIoProtocolGuid",
+ "gEfiTrEEProtocolGuid=gEfiTcg2ProtocolGuid",
+ "gEfiSmmPowerButtonDispatch2ProtocolGuid=gEfiMmPowerButtonDispatchProtocolGuid",
+ "gEfiSmmBase2ProtocolGuid=gEfiMmBaseProtocolGuid",
+ "gEfiSmmUsbDispatch2ProtocolGuid=gEfiMmUsbDispatchProtocolGuid",
+ "gEfiSmmCpuProtocolGuid=gEfiMmCpuProtocolGuid",
+ "gEfiAcpiTableGuid=gEfiAcpi20TableGuid",
+ "gEfiSmmControl2ProtocolGuid=gEfiMmControlProtocolGuid",
+ "gEfiSmmStandbyButtonDispatch2ProtocolGuid=gEfiMmStandbyButtonDispatchProtocolGuid",
+ "gEfiSmmStatusCodeProtocolGuid=gEfiMmStatusCodeProtocolGuid",
+ "gEfiDxeSmmReadyToLockProtocolGuid=gEfiDxeMmReadyToLockProtocolGuid",
+ "gEfiSmmIoTrapDispatch2ProtocolGuid=gEfiMmIoTrapDispatchProtocolGuid",
+ "gEfiSmmReadyToLockProtocolGuid=gEfiMmReadyToLockProtocolGuid",
+ "gEfiSmmSxDispatch2ProtocolGuid=gEfiMmSxDispatchProtocolGuid",
+ "gEfiPeiCapsulePpiGuid=gPeiCapsulePpiGuid",
+ "gEfiSmmCpuIo2ProtocolGuid=gEfiMmCpuIoProtocolGuid",
+ "gEfiSmmRscHandlerProtocolGuid=gEfiMmRscHandlerProtocolGuid",
+ "gEfiSmmConfigurationProtocolGuid=gEfiMmConfigurationProtocolGuid",
+ "gEfiSmmGpiDispatch2ProtocolGuid=gEfiMmGpiDispatchProtocolGuid",
+ "gEfiSmmEndOfDxeProtocolGuid=gEfiMmEndOfDxeProtocolGuid",
+ "gEfiSmmSwDispatch2ProtocolGuid=gEfiMmSwDispatchProtocolGuid",
+ "gEfiDebugPortProtocolGuid=gEfiDebugPortVariableGuid=gEfiDebugPortDevicePathGuid", ## is this a bug
+ "gEfiProcessorSpecificErrorSectionGuid=gEfiIa32X64ProcessorErrorSectionGuid", ## is this a bug
+ "gEfiSmmPeriodicTimerDispatch2ProtocolGuid=gEfiMmPeriodicTimerDispatchProtocolGuid",
+ "gEdkiiFormBrowserExProtocolGuid=gEfiFormBrowserExProtocolGuid",
+ "gEfiPeiMmAccessPpiGuid=gPeiSmmAccessPpiGuid",
+ "gPeiSmmControlPpiGuid=gEfiPeiMmControlPpiGuid"
+ ]
+ },
+
+ ## options defined ci/Plugin/LibraryClassCheck
+ "LibraryClassCheck": {
+ "IgnoreHeaderFile": []
+ },
+
+ ## options defined ci/Plugin/SpellCheck
+ "SpellCheck": {
+ "AuditOnly": True, # Fails test but run in AuditOnly mode to collect log
+ "IgnoreFiles": [], # use gitignore syntax to ignore errors in matching files
+ "ExtendWords": [], # words to extend to the dictionary for this package
+ "IgnoreStandardPaths": [], # Standard Plugin defined paths that should be ignore
+ "AdditionalIncludePaths": [] # Additional paths to spell check (wildcards supported)
+ }
+}
--
2.21.0.windows.1
^ permalink raw reply related [flat|nested] 56+ messages in thread* Re: [Patch v3 12/22] MdePkg: Add YAML file for CI builds
2019-10-29 19:55 ` [Patch v3 12/22] MdePkg: " Michael D Kinney
@ 2019-11-05 8:22 ` Liming Gao
2019-11-05 17:04 ` Michael D Kinney
0 siblings, 1 reply; 56+ messages in thread
From: Liming Gao @ 2019-11-05 8:22 UTC (permalink / raw)
To: Kinney, Michael D, devel@edk2.groups.io
Mike:
Is there some basic introduction for YAML file syntax?
Thanks
Liming
>-----Original Message-----
>From: Kinney, Michael D
>Sent: Wednesday, October 30, 2019 3:55 AM
>To: devel@edk2.groups.io
>Cc: Gao, Liming <liming.gao@intel.com>
>Subject: [Patch v3 12/22] MdePkg: Add YAML file for CI builds
>
>https://bugzilla.tianocore.org/show_bug.cgi?id=2315
>
>Add YAML file to the package directory with the
>configuration of the checks to perform during a
>CI build.
>
>Cc: Liming Gao <liming.gao@intel.com>
>Signed-off-by: Michael D Kinney <michael.d.kinney@intel.com>
>---
> MdePkg/MdePkg.ci.yaml | 90
>+++++++++++++++++++++++++++++++++++++++++++
> 1 file changed, 90 insertions(+)
> create mode 100644 MdePkg/MdePkg.ci.yaml
>
>diff --git a/MdePkg/MdePkg.ci.yaml b/MdePkg/MdePkg.ci.yaml
>new file mode 100644
>index 0000000000..4015c07f16
>--- /dev/null
>+++ b/MdePkg/MdePkg.ci.yaml
>@@ -0,0 +1,90 @@
>+##
>+# CI configuration for MdePkg
>+#
>+# Copyright (c) Microsoft Corporation
>+# SPDX-License-Identifier: BSD-2-Clause-Patent
>+##
>+{
>+ ## options defined ci/Plugin/CompilerPlugin
>+ "CompilerPlugin": {
>+ "DscPath": "MdePkg.dsc"
>+ },
>+
>+ ## options defined ci/Plugin/CharEncodingCheck
>+"CharEncodingCheck": {
>+ "IgnoreFiles": []
>+ },
>+
>+ ## options defined ci/Plugin/DependencyCheck
>+ "DependencyCheck": {
>+ "AcceptableDependencies": [
>+ "MdePkg/MdePkg.dec"
>+ ],
>+ # For host based unit tests
>+ "AcceptableDependencies-HOST_APPLICATION":[],
>+ # For UEFI shell based apps
>+ "AcceptableDependencies-UEFI_APPLICATION":[],
>+ "IgnoreInf": []
>+ },
>+
>+ ## options defined ci/Plugin/DscCompleteCheck
>+ "DscCompleteCheck": {
>+ "IgnoreInf": [],
>+ "DscPath": "MdePkg.dsc"
>+ },
>+
>+ ## options defined ci/Plugin/GuidCheck
>+ "GuidCheck": {
>+ "IgnoreGuidName": [
>+ "gEfiFirmwareVolumeTopFileGuid" # sec modules must be set to this
>guid
>+ ],
>+ "IgnoreGuidValue": ["00000000-0000-0000-0000-000000000000"],
>+ "IgnoreFoldersAndFiles": [],
>+ "IgnoreDuplicates": [
>+
>"gEfiFirmwareVolumeBlockProtocolGuid=gEfiFirmwareVolumeBlock2Protocol
>Guid",
>+ "gEfiMmAccessProtocolGuid=gEfiSmmAccess2ProtocolGuid",
>+
>"gEfiSmmCommunicationProtocolGuid=gEfiMmCommunicationProtocolGuid",
>+
>"gEfiMmPciRootBridgeIoProtocolGuid=gEfiSmmPciRootBridgeIoProtocolGuid",
>+ "gEfiTrEEProtocolGuid=gEfiTcg2ProtocolGuid",
>+
>"gEfiSmmPowerButtonDispatch2ProtocolGuid=gEfiMmPowerButtonDispatch
>ProtocolGuid",
>+ "gEfiSmmBase2ProtocolGuid=gEfiMmBaseProtocolGuid",
>+
>"gEfiSmmUsbDispatch2ProtocolGuid=gEfiMmUsbDispatchProtocolGuid",
>+ "gEfiSmmCpuProtocolGuid=gEfiMmCpuProtocolGuid",
>+ "gEfiAcpiTableGuid=gEfiAcpi20TableGuid",
>+ "gEfiSmmControl2ProtocolGuid=gEfiMmControlProtocolGuid",
>+
>"gEfiSmmStandbyButtonDispatch2ProtocolGuid=gEfiMmStandbyButtonDispat
>chProtocolGuid",
>+ "gEfiSmmStatusCodeProtocolGuid=gEfiMmStatusCodeProtocolGuid",
>+
>"gEfiDxeSmmReadyToLockProtocolGuid=gEfiDxeMmReadyToLockProtocolGui
>d",
>+
>"gEfiSmmIoTrapDispatch2ProtocolGuid=gEfiMmIoTrapDispatchProtocolGuid",
>+
>"gEfiSmmReadyToLockProtocolGuid=gEfiMmReadyToLockProtocolGuid",
>+ "gEfiSmmSxDispatch2ProtocolGuid=gEfiMmSxDispatchProtocolGuid",
>+ "gEfiPeiCapsulePpiGuid=gPeiCapsulePpiGuid",
>+ "gEfiSmmCpuIo2ProtocolGuid=gEfiMmCpuIoProtocolGuid",
>+ "gEfiSmmRscHandlerProtocolGuid=gEfiMmRscHandlerProtocolGuid",
>+
>"gEfiSmmConfigurationProtocolGuid=gEfiMmConfigurationProtocolGuid",
>+
>"gEfiSmmGpiDispatch2ProtocolGuid=gEfiMmGpiDispatchProtocolGuid",
>+ "gEfiSmmEndOfDxeProtocolGuid=gEfiMmEndOfDxeProtocolGuid",
>+ "gEfiSmmSwDispatch2ProtocolGuid=gEfiMmSwDispatchProtocolGuid",
>+
>"gEfiDebugPortProtocolGuid=gEfiDebugPortVariableGuid=gEfiDebugPortDevi
>cePathGuid", ## is this a bug
>+
>"gEfiProcessorSpecificErrorSectionGuid=gEfiIa32X64ProcessorErrorSectionGui
>d", ## is this a bug
>+
>"gEfiSmmPeriodicTimerDispatch2ProtocolGuid=gEfiMmPeriodicTimerDispatch
>ProtocolGuid",
>+ "gEdkiiFormBrowserExProtocolGuid=gEfiFormBrowserExProtocolGuid",
>+ "gEfiPeiMmAccessPpiGuid=gPeiSmmAccessPpiGuid",
>+ "gPeiSmmControlPpiGuid=gEfiPeiMmControlPpiGuid"
>+ ]
>+ },
>+
>+ ## options defined ci/Plugin/LibraryClassCheck
>+ "LibraryClassCheck": {
>+ "IgnoreHeaderFile": []
>+ },
>+
>+ ## options defined ci/Plugin/SpellCheck
>+ "SpellCheck": {
>+ "AuditOnly": True, # Fails test but run in AuditOnly mode to collect
>log
>+ "IgnoreFiles": [], # use gitignore syntax to ignore errors in matching
>files
>+ "ExtendWords": [], # words to extend to the dictionary for this
>package
>+ "IgnoreStandardPaths": [], # Standard Plugin defined paths that should
>be ignore
>+ "AdditionalIncludePaths": [] # Additional paths to spell check (wildcards
>supported)
>+ }
>+}
>--
>2.21.0.windows.1
^ permalink raw reply [flat|nested] 56+ messages in thread* Re: [Patch v3 12/22] MdePkg: Add YAML file for CI builds
2019-11-05 8:22 ` Liming Gao
@ 2019-11-05 17:04 ` Michael D Kinney
2019-11-06 0:20 ` Liming Gao
0 siblings, 1 reply; 56+ messages in thread
From: Michael D Kinney @ 2019-11-05 17:04 UTC (permalink / raw)
To: Gao, Liming, devel@edk2.groups.io, Kinney, Michael D, Sean Brogan
Hi Liming,
There are two types of YAML files in this patch series.
1) Azure Piplelines .yml files.
https://docs.microsoft.com/en-us/azure/devops/pipelines/yaml-schema?view=azure-devops&tabs=schema
2) EDK II Package .yaml files. These specify the set of pytool extensions to
use to make sure an EDK II package passes all the required checks. The
contents of these files are JSON format and the syntax for each extension
is defined by each pytool plugin. JSON is considered a subset of YAML.
https://github.com/tianocore/edk2-staging/tree/edk2-ci/.pytool
https://github.com/tianocore/edk2-staging/tree/edk2-ci/.pytool/Plugin/CharEncodingCheck
https://github.com/tianocore/edk2-staging/tree/edk2-ci/.pytool/Plugin/CompilerPlugin
https://github.com/tianocore/edk2-staging/tree/edk2-ci/.pytool/Plugin/DependencyCheck
https://github.com/tianocore/edk2-staging/tree/edk2-ci/.pytool/Plugin/DscCompleteCheck
https://github.com/tianocore/edk2-staging/tree/edk2-ci/.pytool/Plugin/GuidCheck
https://github.com/tianocore/edk2-staging/tree/edk2-ci/.pytool/Plugin/LibraryClassCheck
https://github.com/tianocore/edk2-staging/tree/edk2-ci/.pytool/Plugin/SpellCheck
Please let us know if the plugin documentation including the YAML syntax is not clear.
Best regards,
Mike
> -----Original Message-----
> From: Gao, Liming <liming.gao@intel.com>
> Sent: Tuesday, November 5, 2019 12:22 AM
> To: Kinney, Michael D <michael.d.kinney@intel.com>;
> devel@edk2.groups.io
> Subject: RE: [Patch v3 12/22] MdePkg: Add YAML file for
> CI builds
>
> Mike:
> Is there some basic introduction for YAML file
> syntax?
>
> Thanks
> Liming
> >-----Original Message-----
> >From: Kinney, Michael D
> >Sent: Wednesday, October 30, 2019 3:55 AM
> >To: devel@edk2.groups.io
> >Cc: Gao, Liming <liming.gao@intel.com>
> >Subject: [Patch v3 12/22] MdePkg: Add YAML file for CI
> builds
> >
> >https://bugzilla.tianocore.org/show_bug.cgi?id=2315
> >
> >Add YAML file to the package directory with the
> configuration of the
> >checks to perform during a CI build.
> >
> >Cc: Liming Gao <liming.gao@intel.com>
> >Signed-off-by: Michael D Kinney
> <michael.d.kinney@intel.com>
> >---
> > MdePkg/MdePkg.ci.yaml | 90
> >+++++++++++++++++++++++++++++++++++++++++++
> > 1 file changed, 90 insertions(+)
> > create mode 100644 MdePkg/MdePkg.ci.yaml
> >
> >diff --git a/MdePkg/MdePkg.ci.yaml
> b/MdePkg/MdePkg.ci.yaml new file
> >mode 100644 index 0000000000..4015c07f16
> >--- /dev/null
> >+++ b/MdePkg/MdePkg.ci.yaml
> >@@ -0,0 +1,90 @@
> >+##
> >+# CI configuration for MdePkg
> >+#
> >+# Copyright (c) Microsoft Corporation
> >+# SPDX-License-Identifier: BSD-2-Clause-Patent ## {
> >+ ## options defined ci/Plugin/CompilerPlugin
> >+ "CompilerPlugin": {
> >+ "DscPath": "MdePkg.dsc"
> >+ },
> >+
> >+ ## options defined ci/Plugin/CharEncodingCheck
> >+"CharEncodingCheck": {
> >+ "IgnoreFiles": []
> >+ },
> >+
> >+ ## options defined ci/Plugin/DependencyCheck
> >+ "DependencyCheck": {
> >+ "AcceptableDependencies": [
> >+ "MdePkg/MdePkg.dec"
> >+ ],
> >+ # For host based unit tests
> >+ "AcceptableDependencies-HOST_APPLICATION":[],
> >+ # For UEFI shell based apps
> >+ "AcceptableDependencies-UEFI_APPLICATION":[],
> >+ "IgnoreInf": []
> >+ },
> >+
> >+ ## options defined ci/Plugin/DscCompleteCheck
> >+ "DscCompleteCheck": {
> >+ "IgnoreInf": [],
> >+ "DscPath": "MdePkg.dsc"
> >+ },
> >+
> >+ ## options defined ci/Plugin/GuidCheck
> >+ "GuidCheck": {
> >+ "IgnoreGuidName": [
> >+ "gEfiFirmwareVolumeTopFileGuid" # sec
> modules must be set
> >+ to this
> >guid
> >+ ],
> >+ "IgnoreGuidValue": ["00000000-0000-0000-0000-
> 000000000000"],
> >+ "IgnoreFoldersAndFiles": [],
> >+ "IgnoreDuplicates": [
> >+
> >"gEfiFirmwareVolumeBlockProtocolGuid=gEfiFirmwareVolume
> Block2Protocol
> >Guid",
> >+
> "gEfiMmAccessProtocolGuid=gEfiSmmAccess2ProtocolGuid",
> >+
> >"gEfiSmmCommunicationProtocolGuid=gEfiMmCommunicationPr
> otocolGuid",
> >+
> >"gEfiMmPciRootBridgeIoProtocolGuid=gEfiSmmPciRootBridge
> IoProtocolGuid",
> >+
> "gEfiTrEEProtocolGuid=gEfiTcg2ProtocolGuid",
> >+
> >"gEfiSmmPowerButtonDispatch2ProtocolGuid=gEfiMmPowerBut
> tonDispatch
> >ProtocolGuid",
> >+
> "gEfiSmmBase2ProtocolGuid=gEfiMmBaseProtocolGuid",
> >+
> >"gEfiSmmUsbDispatch2ProtocolGuid=gEfiMmUsbDispatchProto
> colGuid",
> >+
> "gEfiSmmCpuProtocolGuid=gEfiMmCpuProtocolGuid",
> >+ "gEfiAcpiTableGuid=gEfiAcpi20TableGuid",
> >+
> "gEfiSmmControl2ProtocolGuid=gEfiMmControlProtocolGuid",
> >+
> >"gEfiSmmStandbyButtonDispatch2ProtocolGuid=gEfiMmStandb
> yButtonDispat
> >chProtocolGuid",
> >+
> >+
> "gEfiSmmStatusCodeProtocolGuid=gEfiMmStatusCodeProtocolG
> uid",
> >+
> >"gEfiDxeSmmReadyToLockProtocolGuid=gEfiDxeMmReadyToLock
> ProtocolGui
> >d",
> >+
> >"gEfiSmmIoTrapDispatch2ProtocolGuid=gEfiMmIoTrapDispatc
> hProtocolGuid",
> >+
> >"gEfiSmmReadyToLockProtocolGuid=gEfiMmReadyToLockProtoc
> olGuid",
> >+
> "gEfiSmmSxDispatch2ProtocolGuid=gEfiMmSxDispatchProtocol
> Guid",
> >+
> "gEfiPeiCapsulePpiGuid=gPeiCapsulePpiGuid",
> >+
> "gEfiSmmCpuIo2ProtocolGuid=gEfiMmCpuIoProtocolGuid",
> >+
> >+
> "gEfiSmmRscHandlerProtocolGuid=gEfiMmRscHandlerProtocolG
> uid",
> >+
> >"gEfiSmmConfigurationProtocolGuid=gEfiMmConfigurationPr
> otocolGuid",
> >+
> >"gEfiSmmGpiDispatch2ProtocolGuid=gEfiMmGpiDispatchProto
> colGuid",
> >+
> "gEfiSmmEndOfDxeProtocolGuid=gEfiMmEndOfDxeProtocolGuid"
> ,
> >+
> >+
> "gEfiSmmSwDispatch2ProtocolGuid=gEfiMmSwDispatchProtocol
> Guid",
> >+
> >"gEfiDebugPortProtocolGuid=gEfiDebugPortVariableGuid=gE
> fiDebugPortDevi
> >cePathGuid", ## is this a bug
> >+
> >"gEfiProcessorSpecificErrorSectionGuid=gEfiIa32X64Proce
> ssorErrorSection
> >Gui
> >d", ## is this a bug
> >+
> >"gEfiSmmPeriodicTimerDispatch2ProtocolGuid=gEfiMmPeriod
> icTimerDispatch
> >ProtocolGuid",
> >+
> "gEdkiiFormBrowserExProtocolGuid=gEfiFormBrowserExProtoc
> olGuid",
> >+
> "gEfiPeiMmAccessPpiGuid=gPeiSmmAccessPpiGuid",
> >+
> "gPeiSmmControlPpiGuid=gEfiPeiMmControlPpiGuid"
> >+ ]
> >+ },
> >+
> >+ ## options defined ci/Plugin/LibraryClassCheck
> >+ "LibraryClassCheck": {
> >+ "IgnoreHeaderFile": []
> >+ },
> >+
> >+ ## options defined ci/Plugin/SpellCheck
> >+ "SpellCheck": {
> >+ "AuditOnly": True, # Fails test but
> run in AuditOnly mode to collect
> >log
> >+ "IgnoreFiles": [], # use gitignore
> syntax to ignore errors in matching
> >files
> >+ "ExtendWords": [], # words to extend
> to the dictionary for this
> >package
> >+ "IgnoreStandardPaths": [], # Standard Plugin
> defined paths that should
> >be ignore
> >+ "AdditionalIncludePaths": [] # Additional
> paths to spell check
> >+ (wildcards
> >supported)
> >+ }
> >+}
> >--
> >2.21.0.windows.1
^ permalink raw reply [flat|nested] 56+ messages in thread* Re: [Patch v3 12/22] MdePkg: Add YAML file for CI builds
2019-11-05 17:04 ` Michael D Kinney
@ 2019-11-06 0:20 ` Liming Gao
2019-11-06 0:29 ` Michael D Kinney
0 siblings, 1 reply; 56+ messages in thread
From: Liming Gao @ 2019-11-06 0:20 UTC (permalink / raw)
To: Kinney, Michael D, devel@edk2.groups.io, Sean Brogan
Mike:
So, github plugin (.azurepipelines, .mergify) uses .yml files. Edk2 pytool plugin uses .yaml. Right?
Thanks
Liming
>-----Original Message-----
>From: Kinney, Michael D
>Sent: Wednesday, November 06, 2019 1:05 AM
>To: Gao, Liming <liming.gao@intel.com>; devel@edk2.groups.io; Kinney,
>Michael D <michael.d.kinney@intel.com>; Sean Brogan
><sean.brogan@microsoft.com>
>Subject: RE: [Patch v3 12/22] MdePkg: Add YAML file for CI builds
>
>Hi Liming,
>
>There are two types of YAML files in this patch series.
>
>1) Azure Piplelines .yml files.
>
> https://docs.microsoft.com/en-us/azure/devops/pipelines/yaml-
>schema?view=azure-devops&tabs=schema
>
>2) EDK II Package .yaml files. These specify the set of pytool extensions to
> use to make sure an EDK II package passes all the required checks. The
> contents of these files are JSON format and the syntax for each extension
> is defined by each pytool plugin. JSON is considered a subset of YAML.
>
>https://github.com/tianocore/edk2-staging/tree/edk2-ci/.pytool
>https://github.com/tianocore/edk2-staging/tree/edk2-
>ci/.pytool/Plugin/CharEncodingCheck
>https://github.com/tianocore/edk2-staging/tree/edk2-
>ci/.pytool/Plugin/CompilerPlugin
>https://github.com/tianocore/edk2-staging/tree/edk2-
>ci/.pytool/Plugin/DependencyCheck
>https://github.com/tianocore/edk2-staging/tree/edk2-
>ci/.pytool/Plugin/DscCompleteCheck
>https://github.com/tianocore/edk2-staging/tree/edk2-
>ci/.pytool/Plugin/GuidCheck
>https://github.com/tianocore/edk2-staging/tree/edk2-
>ci/.pytool/Plugin/LibraryClassCheck
>https://github.com/tianocore/edk2-staging/tree/edk2-
>ci/.pytool/Plugin/SpellCheck
>
>Please let us know if the plugin documentation including the YAML syntax is
>not clear.
>
>Best regards,
>
>Mike
>
>> -----Original Message-----
>> From: Gao, Liming <liming.gao@intel.com>
>> Sent: Tuesday, November 5, 2019 12:22 AM
>> To: Kinney, Michael D <michael.d.kinney@intel.com>;
>> devel@edk2.groups.io
>> Subject: RE: [Patch v3 12/22] MdePkg: Add YAML file for
>> CI builds
>>
>> Mike:
>> Is there some basic introduction for YAML file
>> syntax?
>>
>> Thanks
>> Liming
>> >-----Original Message-----
>> >From: Kinney, Michael D
>> >Sent: Wednesday, October 30, 2019 3:55 AM
>> >To: devel@edk2.groups.io
>> >Cc: Gao, Liming <liming.gao@intel.com>
>> >Subject: [Patch v3 12/22] MdePkg: Add YAML file for CI
>> builds
>> >
>> >https://bugzilla.tianocore.org/show_bug.cgi?id=2315
>> >
>> >Add YAML file to the package directory with the
>> configuration of the
>> >checks to perform during a CI build.
>> >
>> >Cc: Liming Gao <liming.gao@intel.com>
>> >Signed-off-by: Michael D Kinney
>> <michael.d.kinney@intel.com>
>> >---
>> > MdePkg/MdePkg.ci.yaml | 90
>> >+++++++++++++++++++++++++++++++++++++++++++
>> > 1 file changed, 90 insertions(+)
>> > create mode 100644 MdePkg/MdePkg.ci.yaml
>> >
>> >diff --git a/MdePkg/MdePkg.ci.yaml
>> b/MdePkg/MdePkg.ci.yaml new file
>> >mode 100644 index 0000000000..4015c07f16
>> >--- /dev/null
>> >+++ b/MdePkg/MdePkg.ci.yaml
>> >@@ -0,0 +1,90 @@
>> >+##
>> >+# CI configuration for MdePkg
>> >+#
>> >+# Copyright (c) Microsoft Corporation
>> >+# SPDX-License-Identifier: BSD-2-Clause-Patent ## {
>> >+ ## options defined ci/Plugin/CompilerPlugin
>> >+ "CompilerPlugin": {
>> >+ "DscPath": "MdePkg.dsc"
>> >+ },
>> >+
>> >+ ## options defined ci/Plugin/CharEncodingCheck
>> >+"CharEncodingCheck": {
>> >+ "IgnoreFiles": []
>> >+ },
>> >+
>> >+ ## options defined ci/Plugin/DependencyCheck
>> >+ "DependencyCheck": {
>> >+ "AcceptableDependencies": [
>> >+ "MdePkg/MdePkg.dec"
>> >+ ],
>> >+ # For host based unit tests
>> >+ "AcceptableDependencies-HOST_APPLICATION":[],
>> >+ # For UEFI shell based apps
>> >+ "AcceptableDependencies-UEFI_APPLICATION":[],
>> >+ "IgnoreInf": []
>> >+ },
>> >+
>> >+ ## options defined ci/Plugin/DscCompleteCheck
>> >+ "DscCompleteCheck": {
>> >+ "IgnoreInf": [],
>> >+ "DscPath": "MdePkg.dsc"
>> >+ },
>> >+
>> >+ ## options defined ci/Plugin/GuidCheck
>> >+ "GuidCheck": {
>> >+ "IgnoreGuidName": [
>> >+ "gEfiFirmwareVolumeTopFileGuid" # sec
>> modules must be set
>> >+ to this
>> >guid
>> >+ ],
>> >+ "IgnoreGuidValue": ["00000000-0000-0000-0000-
>> 000000000000"],
>> >+ "IgnoreFoldersAndFiles": [],
>> >+ "IgnoreDuplicates": [
>> >+
>> >"gEfiFirmwareVolumeBlockProtocolGuid=gEfiFirmwareVolume
>> Block2Protocol
>> >Guid",
>> >+
>> "gEfiMmAccessProtocolGuid=gEfiSmmAccess2ProtocolGuid",
>> >+
>> >"gEfiSmmCommunicationProtocolGuid=gEfiMmCommunicationPr
>> otocolGuid",
>> >+
>> >"gEfiMmPciRootBridgeIoProtocolGuid=gEfiSmmPciRootBridge
>> IoProtocolGuid",
>> >+
>> "gEfiTrEEProtocolGuid=gEfiTcg2ProtocolGuid",
>> >+
>> >"gEfiSmmPowerButtonDispatch2ProtocolGuid=gEfiMmPowerBut
>> tonDispatch
>> >ProtocolGuid",
>> >+
>> "gEfiSmmBase2ProtocolGuid=gEfiMmBaseProtocolGuid",
>> >+
>> >"gEfiSmmUsbDispatch2ProtocolGuid=gEfiMmUsbDispatchProto
>> colGuid",
>> >+
>> "gEfiSmmCpuProtocolGuid=gEfiMmCpuProtocolGuid",
>> >+ "gEfiAcpiTableGuid=gEfiAcpi20TableGuid",
>> >+
>> "gEfiSmmControl2ProtocolGuid=gEfiMmControlProtocolGuid",
>> >+
>> >"gEfiSmmStandbyButtonDispatch2ProtocolGuid=gEfiMmStandb
>> yButtonDispat
>> >chProtocolGuid",
>> >+
>> >+
>> "gEfiSmmStatusCodeProtocolGuid=gEfiMmStatusCodeProtocolG
>> uid",
>> >+
>> >"gEfiDxeSmmReadyToLockProtocolGuid=gEfiDxeMmReadyToLock
>> ProtocolGui
>> >d",
>> >+
>> >"gEfiSmmIoTrapDispatch2ProtocolGuid=gEfiMmIoTrapDispatc
>> hProtocolGuid",
>> >+
>> >"gEfiSmmReadyToLockProtocolGuid=gEfiMmReadyToLockProtoc
>> olGuid",
>> >+
>> "gEfiSmmSxDispatch2ProtocolGuid=gEfiMmSxDispatchProtocol
>> Guid",
>> >+
>> "gEfiPeiCapsulePpiGuid=gPeiCapsulePpiGuid",
>> >+
>> "gEfiSmmCpuIo2ProtocolGuid=gEfiMmCpuIoProtocolGuid",
>> >+
>> >+
>> "gEfiSmmRscHandlerProtocolGuid=gEfiMmRscHandlerProtocolG
>> uid",
>> >+
>> >"gEfiSmmConfigurationProtocolGuid=gEfiMmConfigurationPr
>> otocolGuid",
>> >+
>> >"gEfiSmmGpiDispatch2ProtocolGuid=gEfiMmGpiDispatchProto
>> colGuid",
>> >+
>> "gEfiSmmEndOfDxeProtocolGuid=gEfiMmEndOfDxeProtocolGuid"
>> ,
>> >+
>> >+
>> "gEfiSmmSwDispatch2ProtocolGuid=gEfiMmSwDispatchProtocol
>> Guid",
>> >+
>> >"gEfiDebugPortProtocolGuid=gEfiDebugPortVariableGuid=gE
>> fiDebugPortDevi
>> >cePathGuid", ## is this a bug
>> >+
>> >"gEfiProcessorSpecificErrorSectionGuid=gEfiIa32X64Proce
>> ssorErrorSection
>> >Gui
>> >d", ## is this a bug
>> >+
>> >"gEfiSmmPeriodicTimerDispatch2ProtocolGuid=gEfiMmPeriod
>> icTimerDispatch
>> >ProtocolGuid",
>> >+
>> "gEdkiiFormBrowserExProtocolGuid=gEfiFormBrowserExProtoc
>> olGuid",
>> >+
>> "gEfiPeiMmAccessPpiGuid=gPeiSmmAccessPpiGuid",
>> >+
>> "gPeiSmmControlPpiGuid=gEfiPeiMmControlPpiGuid"
>> >+ ]
>> >+ },
>> >+
>> >+ ## options defined ci/Plugin/LibraryClassCheck
>> >+ "LibraryClassCheck": {
>> >+ "IgnoreHeaderFile": []
>> >+ },
>> >+
>> >+ ## options defined ci/Plugin/SpellCheck
>> >+ "SpellCheck": {
>> >+ "AuditOnly": True, # Fails test but
>> run in AuditOnly mode to collect
>> >log
>> >+ "IgnoreFiles": [], # use gitignore
>> syntax to ignore errors in matching
>> >files
>> >+ "ExtendWords": [], # words to extend
>> to the dictionary for this
>> >package
>> >+ "IgnoreStandardPaths": [], # Standard Plugin
>> defined paths that should
>> >be ignore
>> >+ "AdditionalIncludePaths": [] # Additional
>> paths to spell check
>> >+ (wildcards
>> >supported)
>> >+ }
>> >+}
>> >--
>> >2.21.0.windows.1
^ permalink raw reply [flat|nested] 56+ messages in thread* Re: [Patch v3 12/22] MdePkg: Add YAML file for CI builds
2019-11-06 0:20 ` Liming Gao
@ 2019-11-06 0:29 ` Michael D Kinney
0 siblings, 0 replies; 56+ messages in thread
From: Michael D Kinney @ 2019-11-06 0:29 UTC (permalink / raw)
To: Gao, Liming, devel@edk2.groups.io, Sean Brogan, Kinney, Michael D
Yes.
Mike
> -----Original Message-----
> From: Gao, Liming <liming.gao@intel.com>
> Sent: Tuesday, November 5, 2019 4:21 PM
> To: Kinney, Michael D <michael.d.kinney@intel.com>;
> devel@edk2.groups.io; Sean Brogan
> <sean.brogan@microsoft.com>
> Subject: RE: [Patch v3 12/22] MdePkg: Add YAML file for
> CI builds
>
> Mike:
> So, github plugin (.azurepipelines, .mergify) uses
> .yml files. Edk2 pytool plugin uses .yaml. Right?
>
> Thanks
> Liming
> >-----Original Message-----
> >From: Kinney, Michael D
> >Sent: Wednesday, November 06, 2019 1:05 AM
> >To: Gao, Liming <liming.gao@intel.com>;
> devel@edk2.groups.io; Kinney,
> >Michael D <michael.d.kinney@intel.com>; Sean Brogan
> ><sean.brogan@microsoft.com>
> >Subject: RE: [Patch v3 12/22] MdePkg: Add YAML file for
> CI builds
> >
> >Hi Liming,
> >
> >There are two types of YAML files in this patch series.
> >
> >1) Azure Piplelines .yml files.
> >
> > https://docs.microsoft.com/en-
> us/azure/devops/pipelines/yaml-
> >schema?view=azure-devops&tabs=schema
> >
> >2) EDK II Package .yaml files. These specify the set
> of pytool extensions to
> > use to make sure an EDK II package passes all the
> required checks. The
> > contents of these files are JSON format and the
> syntax for each extension
> > is defined by each pytool plugin. JSON is
> considered a subset of YAML.
> >
> >https://github.com/tianocore/edk2-staging/tree/edk2-
> ci/.pytool
> >https://github.com/tianocore/edk2-staging/tree/edk2-
> >ci/.pytool/Plugin/CharEncodingCheck
> >https://github.com/tianocore/edk2-staging/tree/edk2-
> >ci/.pytool/Plugin/CompilerPlugin
> >https://github.com/tianocore/edk2-staging/tree/edk2-
> >ci/.pytool/Plugin/DependencyCheck
> >https://github.com/tianocore/edk2-staging/tree/edk2-
> >ci/.pytool/Plugin/DscCompleteCheck
> >https://github.com/tianocore/edk2-staging/tree/edk2-
> >ci/.pytool/Plugin/GuidCheck
> >https://github.com/tianocore/edk2-staging/tree/edk2-
> >ci/.pytool/Plugin/LibraryClassCheck
> >https://github.com/tianocore/edk2-staging/tree/edk2-
> >ci/.pytool/Plugin/SpellCheck
> >
> >Please let us know if the plugin documentation
> including the YAML
> >syntax is not clear.
> >
> >Best regards,
> >
> >Mike
> >
> >> -----Original Message-----
> >> From: Gao, Liming <liming.gao@intel.com>
> >> Sent: Tuesday, November 5, 2019 12:22 AM
> >> To: Kinney, Michael D <michael.d.kinney@intel.com>;
> >> devel@edk2.groups.io
> >> Subject: RE: [Patch v3 12/22] MdePkg: Add YAML file
> for CI builds
> >>
> >> Mike:
> >> Is there some basic introduction for YAML file
> syntax?
> >>
> >> Thanks
> >> Liming
> >> >-----Original Message-----
> >> >From: Kinney, Michael D
> >> >Sent: Wednesday, October 30, 2019 3:55 AM
> >> >To: devel@edk2.groups.io
> >> >Cc: Gao, Liming <liming.gao@intel.com>
> >> >Subject: [Patch v3 12/22] MdePkg: Add YAML file for
> CI
> >> builds
> >> >
> >> >https://bugzilla.tianocore.org/show_bug.cgi?id=2315
> >> >
> >> >Add YAML file to the package directory with the
> >> configuration of the
> >> >checks to perform during a CI build.
> >> >
> >> >Cc: Liming Gao <liming.gao@intel.com>
> >> >Signed-off-by: Michael D Kinney
> >> <michael.d.kinney@intel.com>
> >> >---
> >> > MdePkg/MdePkg.ci.yaml | 90
> >> >+++++++++++++++++++++++++++++++++++++++++++
> >> > 1 file changed, 90 insertions(+)
> >> > create mode 100644 MdePkg/MdePkg.ci.yaml
> >> >
> >> >diff --git a/MdePkg/MdePkg.ci.yaml
> >> b/MdePkg/MdePkg.ci.yaml new file
> >> >mode 100644 index 0000000000..4015c07f16
> >> >--- /dev/null
> >> >+++ b/MdePkg/MdePkg.ci.yaml
> >> >@@ -0,0 +1,90 @@
> >> >+##
> >> >+# CI configuration for MdePkg
> >> >+#
> >> >+# Copyright (c) Microsoft Corporation # SPDX-
> License-Identifier:
> >> >+BSD-2-Clause-Patent ## {
> >> >+ ## options defined ci/Plugin/CompilerPlugin
> >> >+ "CompilerPlugin": {
> >> >+ "DscPath": "MdePkg.dsc"
> >> >+ },
> >> >+
> >> >+ ## options defined ci/Plugin/CharEncodingCheck
> >> >+"CharEncodingCheck": {
> >> >+ "IgnoreFiles": []
> >> >+ },
> >> >+
> >> >+ ## options defined ci/Plugin/DependencyCheck
> >> >+ "DependencyCheck": {
> >> >+ "AcceptableDependencies": [
> >> >+ "MdePkg/MdePkg.dec"
> >> >+ ],
> >> >+ # For host based unit tests
> >> >+ "AcceptableDependencies-
> HOST_APPLICATION":[],
> >> >+ # For UEFI shell based apps
> >> >+ "AcceptableDependencies-
> UEFI_APPLICATION":[],
> >> >+ "IgnoreInf": []
> >> >+ },
> >> >+
> >> >+ ## options defined ci/Plugin/DscCompleteCheck
> >> >+ "DscCompleteCheck": {
> >> >+ "IgnoreInf": [],
> >> >+ "DscPath": "MdePkg.dsc"
> >> >+ },
> >> >+
> >> >+ ## options defined ci/Plugin/GuidCheck
> >> >+ "GuidCheck": {
> >> >+ "IgnoreGuidName": [
> >> >+ "gEfiFirmwareVolumeTopFileGuid" # sec
> >> modules must be set
> >> >+ to this
> >> >guid
> >> >+ ],
> >> >+ "IgnoreGuidValue": ["00000000-0000-0000-
> 0000-
> >> 000000000000"],
> >> >+ "IgnoreFoldersAndFiles": [],
> >> >+ "IgnoreDuplicates": [
> >> >+
> >>
> >"gEfiFirmwareVolumeBlockProtocolGuid=gEfiFirmwareVolume
> >> Block2Protocol
> >> >Guid",
> >> >+
> >>
> "gEfiMmAccessProtocolGuid=gEfiSmmAccess2ProtocolGuid",
> >> >+
> >>
> >"gEfiSmmCommunicationProtocolGuid=gEfiMmCommunicationPr
> >> otocolGuid",
> >> >+
> >>
> >"gEfiMmPciRootBridgeIoProtocolGuid=gEfiSmmPciRootBridge
> >> IoProtocolGuid",
> >> >+
> >> "gEfiTrEEProtocolGuid=gEfiTcg2ProtocolGuid",
> >> >+
> >>
> >"gEfiSmmPowerButtonDispatch2ProtocolGuid=gEfiMmPowerBut
> >> tonDispatch
> >> >ProtocolGuid",
> >> >+
> >> "gEfiSmmBase2ProtocolGuid=gEfiMmBaseProtocolGuid",
> >> >+
> >>
> >"gEfiSmmUsbDispatch2ProtocolGuid=gEfiMmUsbDispatchProto
> >> colGuid",
> >> >+
> >> "gEfiSmmCpuProtocolGuid=gEfiMmCpuProtocolGuid",
> >> >+
> "gEfiAcpiTableGuid=gEfiAcpi20TableGuid",
> >> >+
> >>
> "gEfiSmmControl2ProtocolGuid=gEfiMmControlProtocolGuid",
> >> >+
> >>
> >"gEfiSmmStandbyButtonDispatch2ProtocolGuid=gEfiMmStandb
> >> yButtonDispat
> >> >chProtocolGuid",
> >> >+
> >> >+
> >>
> "gEfiSmmStatusCodeProtocolGuid=gEfiMmStatusCodeProtocolG
> >> uid",
> >> >+
> >>
> >"gEfiDxeSmmReadyToLockProtocolGuid=gEfiDxeMmReadyToLock
> >> ProtocolGui
> >> >d",
> >> >+
> >>
> >"gEfiSmmIoTrapDispatch2ProtocolGuid=gEfiMmIoTrapDispatc
> >> hProtocolGuid",
> >> >+
> >>
> >"gEfiSmmReadyToLockProtocolGuid=gEfiMmReadyToLockProtoc
> >> olGuid",
> >> >+
> >>
> "gEfiSmmSxDispatch2ProtocolGuid=gEfiMmSxDispatchProtocol
> >> Guid",
> >> >+
> >> "gEfiPeiCapsulePpiGuid=gPeiCapsulePpiGuid",
> >> >+
> >> "gEfiSmmCpuIo2ProtocolGuid=gEfiMmCpuIoProtocolGuid",
> >> >+
> >> >+
> >>
> "gEfiSmmRscHandlerProtocolGuid=gEfiMmRscHandlerProtocolG
> >> uid",
> >> >+
> >>
> >"gEfiSmmConfigurationProtocolGuid=gEfiMmConfigurationPr
> >> otocolGuid",
> >> >+
> >>
> >"gEfiSmmGpiDispatch2ProtocolGuid=gEfiMmGpiDispatchProto
> >> colGuid",
> >> >+
> >>
> "gEfiSmmEndOfDxeProtocolGuid=gEfiMmEndOfDxeProtocolGuid"
> >> ,
> >> >+
> >> >+
> >>
> "gEfiSmmSwDispatch2ProtocolGuid=gEfiMmSwDispatchProtocol
> >> Guid",
> >> >+
> >>
> >"gEfiDebugPortProtocolGuid=gEfiDebugPortVariableGuid=gE
> >> fiDebugPortDevi
> >> >cePathGuid", ## is this a bug
> >> >+
> >>
> >"gEfiProcessorSpecificErrorSectionGuid=gEfiIa32X64Proce
> >> ssorErrorSection
> >> >Gui
> >> >d", ## is this a bug
> >> >+
> >>
> >"gEfiSmmPeriodicTimerDispatch2ProtocolGuid=gEfiMmPeriod
> >> icTimerDispatch
> >> >ProtocolGuid",
> >> >+
> >>
> "gEdkiiFormBrowserExProtocolGuid=gEfiFormBrowserExProtoc
> >> olGuid",
> >> >+
> >> "gEfiPeiMmAccessPpiGuid=gPeiSmmAccessPpiGuid",
> >> >+
> >> "gPeiSmmControlPpiGuid=gEfiPeiMmControlPpiGuid"
> >> >+ ]
> >> >+ },
> >> >+
> >> >+ ## options defined ci/Plugin/LibraryClassCheck
> >> >+ "LibraryClassCheck": {
> >> >+ "IgnoreHeaderFile": []
> >> >+ },
> >> >+
> >> >+ ## options defined ci/Plugin/SpellCheck
> >> >+ "SpellCheck": {
> >> >+ "AuditOnly": True, # Fails test
> but
> >> run in AuditOnly mode to collect
> >> >log
> >> >+ "IgnoreFiles": [], # use
> gitignore
> >> syntax to ignore errors in matching
> >> >files
> >> >+ "ExtendWords": [], # words to
> extend
> >> to the dictionary for this
> >> >package
> >> >+ "IgnoreStandardPaths": [], # Standard
> Plugin
> >> defined paths that should
> >> >be ignore
> >> >+ "AdditionalIncludePaths": [] # Additional
> >> paths to spell check
> >> >+ (wildcards
> >> >supported)
> >> >+ }
> >> >+}
> >> >--
> >> >2.21.0.windows.1
^ permalink raw reply [flat|nested] 56+ messages in thread
* [Patch v3 13/22] NetworkPkg: Add YAML file for CI builds
2019-10-29 19:54 [Patch v3 00/22] Enable Phase 1 of EDK II CI Michael D Kinney
` (11 preceding siblings ...)
2019-10-29 19:55 ` [Patch v3 12/22] MdePkg: " Michael D Kinney
@ 2019-10-29 19:55 ` Michael D Kinney
2019-10-29 19:55 ` [Patch v3 14/22] PcAtChipsetPkg: Add YAML files " Michael D Kinney
` (9 subsequent siblings)
22 siblings, 0 replies; 56+ messages in thread
From: Michael D Kinney @ 2019-10-29 19:55 UTC (permalink / raw)
To: devel; +Cc: Jiaxin Wu, Siyuan Fu
https://bugzilla.tianocore.org/show_bug.cgi?id=2315
Add YAML file to the package directory with the
configuration of the checks to perform during a
CI build.
Use BaseCryptLibNull and TlsLibNull for package CI
builds to reduce package build times. Enabled with
CONTINUOUS_INTEGRATION in YAML files. By default
CONTINUOUS_INTEGRATION is not defined, and the
original lib mappings are preserved.
Cc: Jiaxin Wu <jiaxin.wu@intel.com>
Cc: Siyuan Fu <siyuan.fu@intel.com>
Signed-off-by: Michael D Kinney <michael.d.kinney@intel.com>
---
NetworkPkg/NetworkPkg.ci.yaml | 62 +++++++++++++++++++++++++++++++++++
NetworkPkg/NetworkPkg.dsc | 5 +++
2 files changed, 67 insertions(+)
create mode 100644 NetworkPkg/NetworkPkg.ci.yaml
diff --git a/NetworkPkg/NetworkPkg.ci.yaml b/NetworkPkg/NetworkPkg.ci.yaml
new file mode 100644
index 0000000000..ea41c9db66
--- /dev/null
+++ b/NetworkPkg/NetworkPkg.ci.yaml
@@ -0,0 +1,62 @@
+##
+# CI configuration for NetworkPkg
+#
+# Copyright (c) Microsoft Corporation
+# SPDX-License-Identifier: BSD-2-Clause-Patent
+##
+{
+ "CompilerPlugin": {
+ "DscPath": "NetworkPkg.dsc"
+ },
+ "CharEncodingCheck": {
+ "IgnoreFiles": []
+ },
+ "DependencyCheck": {
+ "AcceptableDependencies": [
+ "MdePkg/MdePkg.dec",
+ "MdeModulePkg/MdeModulePkg.dec",
+ "NetworkPkg/NetworkPkg.dec",
+ "CryptoPkg/CryptoPkg.dec"
+ ],
+ # For host based unit tests
+ "AcceptableDependencies-HOST_APPLICATION":[],
+ # For UEFI shell based apps
+ "AcceptableDependencies-UEFI_APPLICATION":[
+ "ShellPkg/ShellPkg.dec"
+ ],
+ "IgnoreInf": []
+ },
+ "DscCompleteCheck": {
+ "DscPath": "NetworkPkg.dsc",
+ "IgnoreInf": []
+ },
+ "GuidCheck": {
+ "IgnoreGuidName": [],
+ "IgnoreGuidValue": [],
+ "IgnoreFoldersAndFiles": []
+ },
+ "LibraryClassCheck": {
+ "IgnoreHeaderFile": []
+ },
+
+ ## options defined ci/Plugin/SpellCheck
+ "SpellCheck": {
+ "AuditOnly": True, # Fails test but run in AuditOnly mode to collect log
+ "IgnoreFiles": [], # use gitignore syntax to ignore errors in matching files
+ "ExtendWords": [], # words to extend to the dictionary for this package
+ "IgnoreStandardPaths": [], # Standard Plugin defined paths that should be ignore
+ "AdditionalIncludePaths": [] # Additional paths to spell check (wildcards supported)
+ },
+
+ "Defines": {
+ "BLD_*_CONTINUOUS_INTEGRATION": "TRUE",
+ "BLD_*_NETWORK_ENABLE": "TRUE",
+ "BLD_*_NETWORK_SNP_ENABLE": "TRUE",
+ "BLD_*_NETWORK_VLAN_ENABLE": "TRUE",
+ "BLD_*_NETWORK_IP4_ENABLE": "TRUE",
+ "BLD_*_NETWORK_IP6_ENABLE": "TRUE",
+ "BLD_*_NETWORK_TLS_ENABLE": "TRUE",
+ "BLD_*_NETWORK_HTTP_BOOT_ENABLE": "TRUE",
+ "BLD_*_NETWORK_ISCSI_ENABLE": "TRUE",
+ }
+}
diff --git a/NetworkPkg/NetworkPkg.dsc b/NetworkPkg/NetworkPkg.dsc
index 11a2981261..b149453d26 100644
--- a/NetworkPkg/NetworkPkg.dsc
+++ b/NetworkPkg/NetworkPkg.dsc
@@ -41,10 +41,15 @@ [LibraryClasses]
DxeServicesTableLib|MdePkg/Library/DxeServicesTableLib/DxeServicesTableLib.inf
SafeIntLib|MdePkg/Library/BaseSafeIntLib/BaseSafeIntLib.inf
+!ifdef CONTINUOUS_INTEGRATION
+ BaseCryptLib|CryptoPkg/Library/BaseCryptLibNull/BaseCryptLibNull.inf
+ TlsLib|CryptoPkg/Library/TlsLibNull/TlsLibNull.inf
+!else
BaseCryptLib|CryptoPkg/Library/BaseCryptLib/BaseCryptLib.inf
OpensslLib|CryptoPkg/Library/OpensslLib/OpensslLib.inf
IntrinsicLib|CryptoPkg/Library/IntrinsicLib/IntrinsicLib.inf
TlsLib|CryptoPkg/Library/TlsLib/TlsLib.inf
+!endif
DebugPrintErrorLevelLib|MdePkg/Library/BaseDebugPrintErrorLevelLib/BaseDebugPrintErrorLevelLib.inf
FileHandleLib|MdePkg/Library/UefiFileHandleLib/UefiFileHandleLib.inf
FileExplorerLib|MdeModulePkg/Library/FileExplorerLib/FileExplorerLib.inf
--
2.21.0.windows.1
^ permalink raw reply related [flat|nested] 56+ messages in thread* [Patch v3 14/22] PcAtChipsetPkg: Add YAML files for CI builds
2019-10-29 19:54 [Patch v3 00/22] Enable Phase 1 of EDK II CI Michael D Kinney
` (12 preceding siblings ...)
2019-10-29 19:55 ` [Patch v3 13/22] NetworkPkg: " Michael D Kinney
@ 2019-10-29 19:55 ` Michael D Kinney
2019-10-29 19:55 ` [Patch v3 15/22] SecurityPkg: " Michael D Kinney
` (8 subsequent siblings)
22 siblings, 0 replies; 56+ messages in thread
From: Michael D Kinney @ 2019-10-29 19:55 UTC (permalink / raw)
To: devel; +Cc: Ray Ni
https://bugzilla.tianocore.org/show_bug.cgi?id=2315
Add YAML file to the package directory with the
configuration of the checks to perform during a
CI build.
Cc: Ray Ni <ray.ni@intel.com>
Signed-off-by: Michael D Kinney <michael.d.kinney@intel.com>
---
PcAtChipsetPkg/PcAtChipsetPkg.ci.yaml | 46 +++++++++++++++++++++++++++
1 file changed, 46 insertions(+)
create mode 100644 PcAtChipsetPkg/PcAtChipsetPkg.ci.yaml
diff --git a/PcAtChipsetPkg/PcAtChipsetPkg.ci.yaml b/PcAtChipsetPkg/PcAtChipsetPkg.ci.yaml
new file mode 100644
index 0000000000..6ab9f11761
--- /dev/null
+++ b/PcAtChipsetPkg/PcAtChipsetPkg.ci.yaml
@@ -0,0 +1,46 @@
+##
+# CI configuration for PcAtChipsetPkg
+#
+# Copyright (c) Microsoft Corporation
+# SPDX-License-Identifier: BSD-2-Clause-Patent
+##
+{
+ "CompilerPlugin": {
+ "DscPath": "PcAtChipsetPkg.dsc"
+ },
+ "CharEncodingCheck": {
+ "IgnoreFiles": []
+ },
+ "DependencyCheck": {
+ "AcceptableDependencies": [
+ "MdePkg/MdePkg.dec",
+ "PcAtChipsetPkg/PcAtChipsetPkg.dec",
+ "UefiCpuPkg/UefiCpuPkg.dec"
+ ],
+ # For host based unit tests
+ "AcceptableDependencies-HOST_APPLICATION":[],
+ # For UEFI shell based apps
+ "AcceptableDependencies-UEFI_APPLICATION":[],
+ "IgnoreInf": []
+ },
+ "DscCompleteCheck": {
+ "DscPath": "PcAtChipsetPkg.dsc",
+ "IgnoreInf": []
+ },
+ "GuidCheck": {
+ "IgnoreGuidName": [],
+ "IgnoreGuidValue": [],
+ "IgnoreFoldersAndFiles": [],
+ "IgnoreDuplicates": []
+ },
+ "LibraryClassCheck": {
+ "IgnoreHeaderFile": []
+ },
+ "SpellCheck": {
+ "ExtendWords": [
+ "ENUMER", # this is part of an IDE enum
+ "PCATCHIPSET",
+ "TXRDY"
+ ]
+ }
+}
--
2.21.0.windows.1
^ permalink raw reply related [flat|nested] 56+ messages in thread* [Patch v3 15/22] SecurityPkg: Add YAML files for CI builds
2019-10-29 19:54 [Patch v3 00/22] Enable Phase 1 of EDK II CI Michael D Kinney
` (13 preceding siblings ...)
2019-10-29 19:55 ` [Patch v3 14/22] PcAtChipsetPkg: Add YAML files " Michael D Kinney
@ 2019-10-29 19:55 ` Michael D Kinney
2019-10-29 19:55 ` [Patch v3 16/22] ShellPkg: Add YAML file " Michael D Kinney
` (7 subsequent siblings)
22 siblings, 0 replies; 56+ messages in thread
From: Michael D Kinney @ 2019-10-29 19:55 UTC (permalink / raw)
To: devel; +Cc: Jiewen Yao, Jian J Wang, Chao Zhang
https://bugzilla.tianocore.org/show_bug.cgi?id=2315
Add YAML file to the package directory with the
configuration of the checks to perform during a
CI build.
Use BaseCryptLibNull for package CI builds to reduce package
build times. Enabled with CONTINUOUS_INTEGRATION in YAML
files. By default CONTINUOUS_INTEGRATION is not defined,
and the original lib mappings are preserved.
Cc: Jiewen Yao <jiewen.yao@intel.com>
Cc: Jian J Wang <jian.j.wang@intel.com>
Cc: Chao Zhang <chao.b.zhang@intel.com>
Signed-off-by: Michael D Kinney <michael.d.kinney@intel.com>
---
SecurityPkg/SecurityPkg.ci.yaml | 80 +++++++++++++++++++++++++++++++++
SecurityPkg/SecurityPkg.dsc | 32 ++++++++++++-
2 files changed, 110 insertions(+), 2 deletions(-)
create mode 100644 SecurityPkg/SecurityPkg.ci.yaml
diff --git a/SecurityPkg/SecurityPkg.ci.yaml b/SecurityPkg/SecurityPkg.ci.yaml
new file mode 100644
index 0000000000..c7e309fcda
--- /dev/null
+++ b/SecurityPkg/SecurityPkg.ci.yaml
@@ -0,0 +1,80 @@
+##
+# CI configuration for SecurityPkg
+#
+# Copyright (c) Microsoft Corporation
+# SPDX-License-Identifier: BSD-2-Clause-Patent
+##
+{
+ "CompilerPlugin": {
+ "DscPath": "SecurityPkg.dsc"
+ },
+ "CharEncodingCheck": {
+ "IgnoreFiles": []
+ },
+ "DependencyCheck": {
+ "AcceptableDependencies": [
+ "MdePkg/MdePkg.dec",
+ "MdeModulePkg/MdeModulePkg.dec",
+ "SecurityPkg/SecurityPkg.dec",
+ "CryptoPkg/CryptoPkg.dec"
+ ],
+ # For host based unit tests
+ "AcceptableDependencies-HOST_APPLICATION":[],
+ # For UEFI shell based apps
+ "AcceptableDependencies-UEFI_APPLICATION":[],
+ "IgnoreInf": []
+ },
+ "DscCompleteCheck": {
+ "DscPath": "SecurityPkg.dsc",
+ "IgnoreInf": []
+ },
+ "GuidCheck": {
+ "IgnoreGuidName": [],
+ "IgnoreGuidValue": ["00000000-0000-0000-0000-000000000000"],
+ "IgnoreFoldersAndFiles": [],
+ "IgnoreDuplicates": [
+ "Tpm2InstanceLibDTpm=gEfiTpmDeviceInstanceTpm20DtpmGuid", # by design
+ ]
+ },
+ "LibraryClassCheck": {
+ "IgnoreHeaderFile": []
+ },
+
+ ## options defined ci/Plugin/SpellCheck
+ "SpellCheck": {
+ "AuditOnly": True, # Fails test but run in AuditOnly mode to collect log
+ "ExtendWords": [ # words to extend to the dictionary for this package
+ "shortformed", # tpm acpi
+ "autodetect",
+ "blocksid",
+ "comid",
+ "cpinsidpin", #OpalSScV2
+ "ecdsa", # TPM
+ "ecschnorr", # TPM
+ "eisaid", # ACPI
+ "harddisk",
+ "hashall",
+ "hashto",
+ "kek's",
+ "lfanew", # PE/COFF
+ "pcrindex",
+ "pkglength",
+ "ppuser",
+ "preos",
+ "stclear",
+ "toctou",
+ "tpm's",
+ "tpmcmdbuflength",
+ "tpmcommlib",
+ "tpmnvvaluelength",
+ "wrlocked",
+ "xored"
+ ],
+ "IgnoreStandardPaths": [], # Standard Plugin defined paths that should be ignore
+ "AdditionalIncludePaths": [] # Additional paths to spell check (wildcards supported)
+ },
+
+ "Defines": {
+ "BLD_*_CONTINUOUS_INTEGRATION": "TRUE",
+ }
+}
diff --git a/SecurityPkg/SecurityPkg.dsc b/SecurityPkg/SecurityPkg.dsc
index 9a254087a3..a2eeadda7a 100644
--- a/SecurityPkg/SecurityPkg.dsc
+++ b/SecurityPkg/SecurityPkg.dsc
@@ -44,8 +44,6 @@ [LibraryClasses]
HiiLib|MdeModulePkg/Library/UefiHiiLib/UefiHiiLib.inf
UefiHiiServicesLib|MdeModulePkg/Library/UefiHiiServicesLib/UefiHiiServicesLib.inf
PcdLib|MdePkg/Library/BasePcdLibNull/BasePcdLibNull.inf
- IntrinsicLib|CryptoPkg/Library/IntrinsicLib/IntrinsicLib.inf
- OpensslLib|CryptoPkg/Library/OpensslLib/OpensslLib.inf
IoLib|MdePkg/Library/BaseIoLibIntrinsic/BaseIoLibIntrinsic.inf
TpmCommLib|SecurityPkg/Library/TpmCommLib/TpmCommLib.inf
PlatformSecureLib|SecurityPkg/Library/PlatformSecureLibNull/PlatformSecureLibNull.inf
@@ -84,7 +82,13 @@ [LibraryClasses.common.PEIM]
PeiServicesTablePointerLib|MdePkg/Library/PeiServicesTablePointerLib/PeiServicesTablePointerLib.inf
HobLib|MdePkg/Library/PeiHobLib/PeiHobLib.inf
MemoryAllocationLib|MdePkg/Library/PeiMemoryAllocationLib/PeiMemoryAllocationLib.inf
+!ifdef CONTINUOUS_INTEGRATION
+ BaseCryptLib|CryptoPkg/Library/BaseCryptLibNull/BaseCryptLibNull.inf
+!else
+ IntrinsicLib|CryptoPkg/Library/IntrinsicLib/IntrinsicLib.inf
+ OpensslLib|CryptoPkg/Library/OpensslLib/OpensslLib.inf
BaseCryptLib|CryptoPkg/Library/BaseCryptLib/PeiCryptLib.inf
+!endif
HashLib|SecurityPkg/Library/HashLibBaseCryptoRouter/HashLibBaseCryptoRouterPei.inf
ReportStatusCodeLib|MdeModulePkg/Library/PeiReportStatusCodeLib/PeiReportStatusCodeLib.inf
Tpm12DeviceLib|SecurityPkg/Library/Tpm12DeviceLibDTpm/Tpm12DeviceLibDTpm.inf
@@ -95,7 +99,13 @@ [LibraryClasses.common.PEIM]
[LibraryClasses.common.DXE_DRIVER]
HobLib|MdePkg/Library/DxeHobLib/DxeHobLib.inf
ReportStatusCodeLib|MdeModulePkg/Library/DxeReportStatusCodeLib/DxeReportStatusCodeLib.inf
+!ifdef CONTINUOUS_INTEGRATION
+ BaseCryptLib|CryptoPkg/Library/BaseCryptLibNull/BaseCryptLibNull.inf
+!else
+ IntrinsicLib|CryptoPkg/Library/IntrinsicLib/IntrinsicLib.inf
+ OpensslLib|CryptoPkg/Library/OpensslLib/OpensslLib.inf
BaseCryptLib|CryptoPkg/Library/BaseCryptLib/BaseCryptLib.inf
+!endif
HashLib|SecurityPkg/Library/HashLibBaseCryptoRouter/HashLibBaseCryptoRouterDxe.inf
Tpm12DeviceLib|SecurityPkg/Library/Tpm12DeviceLibTcg/Tpm12DeviceLibTcg.inf
Tpm2DeviceLib|SecurityPkg/Library/Tpm2DeviceLibTcg2/Tpm2DeviceLibTcg2.inf
@@ -110,13 +120,25 @@ [LibraryClasses.common.UEFI_DRIVER, LibraryClasses.common.DXE_RUNTIME_DRIVER, Li
[LibraryClasses.common.DXE_RUNTIME_DRIVER]
ReportStatusCodeLib|MdeModulePkg/Library/RuntimeDxeReportStatusCodeLib/RuntimeDxeReportStatusCodeLib.inf
+!ifdef CONTINUOUS_INTEGRATION
+ BaseCryptLib|CryptoPkg/Library/BaseCryptLibNull/BaseCryptLibNull.inf
+!else
+ IntrinsicLib|CryptoPkg/Library/IntrinsicLib/IntrinsicLib.inf
+ OpensslLib|CryptoPkg/Library/OpensslLib/OpensslLib.inf
BaseCryptLib|CryptoPkg/Library/BaseCryptLib/RuntimeCryptLib.inf
+!endif
HashLib|SecurityPkg/Library/HashLibBaseCryptoRouter/HashLibBaseCryptoRouterDxe.inf
Tpm12DeviceLib|SecurityPkg/Library/Tpm12DeviceLibTcg/Tpm12DeviceLibTcg.inf
Tpm2DeviceLib|SecurityPkg/Library/Tpm2DeviceLibTcg2/Tpm2DeviceLibTcg2.inf
[LibraryClasses.common.UEFI_DRIVER, LibraryClasses.common.UEFI_APPLICATION]
+!ifdef CONTINUOUS_INTEGRATION
+ BaseCryptLib|CryptoPkg/Library/BaseCryptLibNull/BaseCryptLibNull.inf
+!else
+ IntrinsicLib|CryptoPkg/Library/IntrinsicLib/IntrinsicLib.inf
+ OpensslLib|CryptoPkg/Library/OpensslLib/OpensslLib.inf
BaseCryptLib|CryptoPkg/Library/BaseCryptLib/BaseCryptLib.inf
+!endif
HashLib|SecurityPkg/Library/HashLibBaseCryptoRouter/HashLibBaseCryptoRouterDxe.inf
Tpm12DeviceLib|SecurityPkg/Library/Tpm12DeviceLibTcg/Tpm12DeviceLibTcg.inf
Tpm2DeviceLib|SecurityPkg/Library/Tpm2DeviceLibTcg2/Tpm2DeviceLibTcg2.inf
@@ -127,7 +149,13 @@ [LibraryClasses.common.DXE_SMM_DRIVER]
MemoryAllocationLib|MdePkg/Library/SmmMemoryAllocationLib/SmmMemoryAllocationLib.inf
ReportStatusCodeLib|MdeModulePkg/Library/SmmReportStatusCodeLib/SmmReportStatusCodeLib.inf
SmmMemLib|MdePkg/Library/SmmMemLib/SmmMemLib.inf
+!ifdef CONTINUOUS_INTEGRATION
+ BaseCryptLib|CryptoPkg/Library/BaseCryptLibNull/BaseCryptLibNull.inf
+!else
+ IntrinsicLib|CryptoPkg/Library/IntrinsicLib/IntrinsicLib.inf
+ OpensslLib|CryptoPkg/Library/OpensslLib/OpensslLib.inf
BaseCryptLib|CryptoPkg/Library/BaseCryptLib/SmmCryptLib.inf
+!endif
Tpm12DeviceLib|SecurityPkg/Library/Tpm12DeviceLibTcg/Tpm12DeviceLibTcg.inf
Tpm2DeviceLib|SecurityPkg/Library/Tpm2DeviceLibTcg2/Tpm2DeviceLibTcg2.inf
Tcg2PhysicalPresenceLib|SecurityPkg/Library/SmmTcg2PhysicalPresenceLib/SmmTcg2PhysicalPresenceLib.inf
--
2.21.0.windows.1
^ permalink raw reply related [flat|nested] 56+ messages in thread* [Patch v3 16/22] ShellPkg: Add YAML file for CI builds
2019-10-29 19:54 [Patch v3 00/22] Enable Phase 1 of EDK II CI Michael D Kinney
` (14 preceding siblings ...)
2019-10-29 19:55 ` [Patch v3 15/22] SecurityPkg: " Michael D Kinney
@ 2019-10-29 19:55 ` Michael D Kinney
2019-11-05 2:19 ` Gao, Zhichao
2019-10-29 19:55 ` [Patch v3 17/22] UefiCpuPkg: " Michael D Kinney
` (6 subsequent siblings)
22 siblings, 1 reply; 56+ messages in thread
From: Michael D Kinney @ 2019-10-29 19:55 UTC (permalink / raw)
To: devel; +Cc: Ray Ni, Zhichao Gao
https://bugzilla.tianocore.org/show_bug.cgi?id=2315
Add YAML file to the package directory with the
configuration of the checks to perform during a
CI build.
Cc: Ray Ni <ray.ni@intel.com>
Cc: Zhichao Gao <zhichao.gao@intel.com>
Signed-off-by: Michael D Kinney <michael.d.kinney@intel.com>
---
ShellPkg/ShellPkg.ci.yaml | 55 +++++++++++++++++++++++++++++++++++++++
1 file changed, 55 insertions(+)
create mode 100644 ShellPkg/ShellPkg.ci.yaml
diff --git a/ShellPkg/ShellPkg.ci.yaml b/ShellPkg/ShellPkg.ci.yaml
new file mode 100644
index 0000000000..5bf7330c1a
--- /dev/null
+++ b/ShellPkg/ShellPkg.ci.yaml
@@ -0,0 +1,55 @@
+##
+# CI configuration for ShellPkg
+#
+# Copyright (c) Microsoft Corporation
+# SPDX-License-Identifier: BSD-2-Clause-Patent
+##
+{
+ "CompilerPlugin": {
+ "DscPath": "ShellPkg.dsc"
+ },
+ "CharEncodingCheck": {
+ "IgnoreFiles": []
+ },
+ "DependencyCheck": {
+ "AcceptableDependencies": [
+ "MdePkg/MdePkg.dec",
+ "MdeModulePkg/MdeModulePkg.dec",
+ "ShellPkg/ShellPkg.dec",
+ "NetworkPkg/NetworkPkg.dec"
+ ],
+ # For host based unit tests
+ "AcceptableDependencies-HOST_APPLICATION":[],
+ # For UEFI shell based apps
+ "AcceptableDependencies-UEFI_APPLICATION":[],
+ "IgnoreInf": []
+ },
+ "DscCompleteCheck": {
+ "DscPath": "ShellPkg.dsc",
+ "IgnoreInf": [
+ "ShellPkg/Application/ShellCTestApp/ShellCTestApp.inf",
+ "ShellPkg/Application/ShellExecTestApp/SA.inf",
+ "ShellPkg/Application/ShellSortTestApp/ShellSortTestApp.inf"
+ ]
+ },
+ "GuidCheck": {
+ "IgnoreGuidName": [],
+ "IgnoreGuidValue": [],
+ "IgnoreFoldersAndFiles": [],
+ "IgnoreDuplicates": [
+ "Shell=gUefiShellFileGuid", # by design
+ ]
+ },
+ "LibraryClassCheck": {
+ "IgnoreHeaderFile": []
+ },
+
+ ## options defined ci/Plugin/SpellCheck
+ "SpellCheck": {
+ "AuditOnly": True, # Fails test but run in AuditOnly mode to collect log
+ "IgnoreFiles": [], # use gitignore syntax to ignore errors in matching files
+ "ExtendWords": [], # words to extend to the dictionary for this package
+ "IgnoreStandardPaths": [], # Standard Plugin defined paths that should be ignore
+ "AdditionalIncludePaths": [] # Additional paths to spell check (wildcards supported)
+ }
+}
--
2.21.0.windows.1
^ permalink raw reply related [flat|nested] 56+ messages in thread* Re: [Patch v3 16/22] ShellPkg: Add YAML file for CI builds
2019-10-29 19:55 ` [Patch v3 16/22] ShellPkg: Add YAML file " Michael D Kinney
@ 2019-11-05 2:19 ` Gao, Zhichao
0 siblings, 0 replies; 56+ messages in thread
From: Gao, Zhichao @ 2019-11-05 2:19 UTC (permalink / raw)
To: Kinney, Michael D, devel@edk2.groups.io; +Cc: Ni, Ray
Acked-by: Zhichao Gao <zhichao.gao@intel.com>
> -----Original Message-----
> From: Kinney, Michael D
> Sent: Wednesday, October 30, 2019 3:55 AM
> To: devel@edk2.groups.io
> Cc: Ni, Ray <ray.ni@intel.com>; Gao, Zhichao <zhichao.gao@intel.com>
> Subject: [Patch v3 16/22] ShellPkg: Add YAML file for CI builds
>
> https://bugzilla.tianocore.org/show_bug.cgi?id=2315
>
> Add YAML file to the package directory with the configuration of the checks
> to perform during a CI build.
>
> Cc: Ray Ni <ray.ni@intel.com>
> Cc: Zhichao Gao <zhichao.gao@intel.com>
> Signed-off-by: Michael D Kinney <michael.d.kinney@intel.com>
> ---
> ShellPkg/ShellPkg.ci.yaml | 55
> +++++++++++++++++++++++++++++++++++++++
> 1 file changed, 55 insertions(+)
> create mode 100644 ShellPkg/ShellPkg.ci.yaml
>
> diff --git a/ShellPkg/ShellPkg.ci.yaml b/ShellPkg/ShellPkg.ci.yaml new file
> mode 100644 index 0000000000..5bf7330c1a
> --- /dev/null
> +++ b/ShellPkg/ShellPkg.ci.yaml
> @@ -0,0 +1,55 @@
> +##
> +# CI configuration for ShellPkg
> +#
> +# Copyright (c) Microsoft Corporation
> +# SPDX-License-Identifier: BSD-2-Clause-Patent ## {
> + "CompilerPlugin": {
> + "DscPath": "ShellPkg.dsc"
> + },
> + "CharEncodingCheck": {
> + "IgnoreFiles": []
> + },
> + "DependencyCheck": {
> + "AcceptableDependencies": [
> + "MdePkg/MdePkg.dec",
> + "MdeModulePkg/MdeModulePkg.dec",
> + "ShellPkg/ShellPkg.dec",
> + "NetworkPkg/NetworkPkg.dec"
> + ],
> + # For host based unit tests
> + "AcceptableDependencies-HOST_APPLICATION":[],
> + # For UEFI shell based apps
> + "AcceptableDependencies-UEFI_APPLICATION":[],
> + "IgnoreInf": []
> + },
> + "DscCompleteCheck": {
> + "DscPath": "ShellPkg.dsc",
> + "IgnoreInf": [
> + "ShellPkg/Application/ShellCTestApp/ShellCTestApp.inf",
> + "ShellPkg/Application/ShellExecTestApp/SA.inf",
> + "ShellPkg/Application/ShellSortTestApp/ShellSortTestApp.inf"
> + ]
> + },
> + "GuidCheck": {
> + "IgnoreGuidName": [],
> + "IgnoreGuidValue": [],
> + "IgnoreFoldersAndFiles": [],
> + "IgnoreDuplicates": [
> + "Shell=gUefiShellFileGuid", # by design
> + ]
> + },
> + "LibraryClassCheck": {
> + "IgnoreHeaderFile": []
> + },
> +
> + ## options defined ci/Plugin/SpellCheck
> + "SpellCheck": {
> + "AuditOnly": True, # Fails test but run in AuditOnly mode to collect
> log
> + "IgnoreFiles": [], # use gitignore syntax to ignore errors in matching
> files
> + "ExtendWords": [], # words to extend to the dictionary for this
> package
> + "IgnoreStandardPaths": [], # Standard Plugin defined paths that should
> be ignore
> + "AdditionalIncludePaths": [] # Additional paths to spell check (wildcards
> supported)
> + }
> +}
> --
> 2.21.0.windows.1
^ permalink raw reply [flat|nested] 56+ messages in thread
* [Patch v3 17/22] UefiCpuPkg: Add YAML file for CI builds
2019-10-29 19:54 [Patch v3 00/22] Enable Phase 1 of EDK II CI Michael D Kinney
` (15 preceding siblings ...)
2019-10-29 19:55 ` [Patch v3 16/22] ShellPkg: Add YAML file " Michael D Kinney
@ 2019-10-29 19:55 ` Michael D Kinney
2019-10-31 8:55 ` Laszlo Ersek
2019-10-29 19:55 ` [Patch v3 18/22] SignedCapsulePkg: Use BaseCryptLibNull to reduce package CI time Michael D Kinney
` (5 subsequent siblings)
22 siblings, 1 reply; 56+ messages in thread
From: Michael D Kinney @ 2019-10-29 19:55 UTC (permalink / raw)
To: devel; +Cc: Eric Dong, Ray Ni, Laszlo Ersek
https://bugzilla.tianocore.org/show_bug.cgi?id=2315
Add YAML file to the package directory with the
configuration of the checks to perform during a
CI build.
Cc: Eric Dong <eric.dong@intel.com>
Cc: Ray Ni <ray.ni@intel.com>
Cc: Laszlo Ersek <lersek@redhat.com>
Signed-off-by: Michael D Kinney <michael.d.kinney@intel.com>
---
UefiCpuPkg/UefiCpuPkg.ci.yaml | 51 +++++++++++++++++++++++++++++++++++
1 file changed, 51 insertions(+)
create mode 100644 UefiCpuPkg/UefiCpuPkg.ci.yaml
diff --git a/UefiCpuPkg/UefiCpuPkg.ci.yaml b/UefiCpuPkg/UefiCpuPkg.ci.yaml
new file mode 100644
index 0000000000..81da29b878
--- /dev/null
+++ b/UefiCpuPkg/UefiCpuPkg.ci.yaml
@@ -0,0 +1,51 @@
+##
+# CI configuration for UefiCpuPkg
+#
+# Copyright (c) Microsoft Corporation
+# SPDX-License-Identifier: BSD-2-Clause-Patent
+##
+{
+ "CompilerPlugin": {
+ "DscPath": "UefiCpuPkg.dsc"
+ },
+ "CharEncodingCheck": {
+ "IgnoreFiles": []
+ },
+ "DependencyCheck": {
+ "AcceptableDependencies": [
+ "MdePkg/MdePkg.dec",
+ "MdeModulePkg/MdeModulePkg.dec",
+ "UefiCpuPkg/UefiCpuPkg.dec"
+ ],
+ # For host based unit tests
+ "AcceptableDependencies-HOST_APPLICATION":[],
+ # For UEFI shell based apps
+ "AcceptableDependencies-UEFI_APPLICATION":[],
+ "IgnoreInf": []
+ },
+ "DscCompleteCheck": {
+ "DscPath": "UefiCpuPkg.dsc",
+ "IgnoreInf": [
+ "UefiCpuPkg/ResetVector/FixupVtf/Vtf.inf",
+ "UefiCpuPkg/ResetVector/Vtf0/Vtf0.inf"
+ ]
+ },
+ "GuidCheck": {
+ "IgnoreGuidName": ["SecCore", "ResetVector"], # Expected duplication for gEfiFirmwareVolumeTopFileGuid
+ "IgnoreGuidValue": [],
+ "IgnoreFoldersAndFiles": [],
+ "IgnoreDuplicates": []
+ },
+ "LibraryClassCheck": {
+ "IgnoreHeaderFile": []
+ },
+
+ ## options defined ci/Plugin/SpellCheck
+ "SpellCheck": {
+ "AuditOnly": True, # Fails test but run in AuditOnly mode to collect log
+ "IgnoreFiles": [], # use gitignore syntax to ignore errors in matching files
+ "ExtendWords": [], # words to extend to the dictionary for this package
+ "IgnoreStandardPaths": [], # Standard Plugin defined paths that should be ignore
+ "AdditionalIncludePaths": [] # Additional paths to spell check (wildcards supported)
+ }
+}
--
2.21.0.windows.1
^ permalink raw reply related [flat|nested] 56+ messages in thread* Re: [Patch v3 17/22] UefiCpuPkg: Add YAML file for CI builds
2019-10-29 19:55 ` [Patch v3 17/22] UefiCpuPkg: " Michael D Kinney
@ 2019-10-31 8:55 ` Laszlo Ersek
0 siblings, 0 replies; 56+ messages in thread
From: Laszlo Ersek @ 2019-10-31 8:55 UTC (permalink / raw)
To: Michael D Kinney, devel; +Cc: Eric Dong, Ray Ni
On 10/29/19 20:55, Michael D Kinney wrote:
> https://bugzilla.tianocore.org/show_bug.cgi?id=2315
>
> Add YAML file to the package directory with the
> configuration of the checks to perform during a
> CI build.
>
> Cc: Eric Dong <eric.dong@intel.com>
> Cc: Ray Ni <ray.ni@intel.com>
> Cc: Laszlo Ersek <lersek@redhat.com>
> Signed-off-by: Michael D Kinney <michael.d.kinney@intel.com>
> ---
> UefiCpuPkg/UefiCpuPkg.ci.yaml | 51 +++++++++++++++++++++++++++++++++++
> 1 file changed, 51 insertions(+)
> create mode 100644 UefiCpuPkg/UefiCpuPkg.ci.yaml
>
> diff --git a/UefiCpuPkg/UefiCpuPkg.ci.yaml b/UefiCpuPkg/UefiCpuPkg.ci.yaml
> new file mode 100644
> index 0000000000..81da29b878
> --- /dev/null
> +++ b/UefiCpuPkg/UefiCpuPkg.ci.yaml
> @@ -0,0 +1,51 @@
> +##
> +# CI configuration for UefiCpuPkg
> +#
> +# Copyright (c) Microsoft Corporation
> +# SPDX-License-Identifier: BSD-2-Clause-Patent
> +##
> +{
> + "CompilerPlugin": {
> + "DscPath": "UefiCpuPkg.dsc"
> + },
> + "CharEncodingCheck": {
> + "IgnoreFiles": []
> + },
> + "DependencyCheck": {
> + "AcceptableDependencies": [
> + "MdePkg/MdePkg.dec",
> + "MdeModulePkg/MdeModulePkg.dec",
> + "UefiCpuPkg/UefiCpuPkg.dec"
> + ],
> + # For host based unit tests
> + "AcceptableDependencies-HOST_APPLICATION":[],
> + # For UEFI shell based apps
> + "AcceptableDependencies-UEFI_APPLICATION":[],
> + "IgnoreInf": []
> + },
> + "DscCompleteCheck": {
> + "DscPath": "UefiCpuPkg.dsc",
> + "IgnoreInf": [
> + "UefiCpuPkg/ResetVector/FixupVtf/Vtf.inf",
> + "UefiCpuPkg/ResetVector/Vtf0/Vtf0.inf"
> + ]
> + },
> + "GuidCheck": {
> + "IgnoreGuidName": ["SecCore", "ResetVector"], # Expected duplication for gEfiFirmwareVolumeTopFileGuid
> + "IgnoreGuidValue": [],
> + "IgnoreFoldersAndFiles": [],
> + "IgnoreDuplicates": []
> + },
> + "LibraryClassCheck": {
> + "IgnoreHeaderFile": []
> + },
> +
> + ## options defined ci/Plugin/SpellCheck
> + "SpellCheck": {
> + "AuditOnly": True, # Fails test but run in AuditOnly mode to collect log
> + "IgnoreFiles": [], # use gitignore syntax to ignore errors in matching files
> + "ExtendWords": [], # words to extend to the dictionary for this package
> + "IgnoreStandardPaths": [], # Standard Plugin defined paths that should be ignore
> + "AdditionalIncludePaths": [] # Additional paths to spell check (wildcards supported)
> + }
> +}
>
Acked-by: Laszlo Ersek <lersek@redhat.com>
^ permalink raw reply [flat|nested] 56+ messages in thread
* [Patch v3 18/22] SignedCapsulePkg: Use BaseCryptLibNull to reduce package CI time
2019-10-29 19:54 [Patch v3 00/22] Enable Phase 1 of EDK II CI Michael D Kinney
` (16 preceding siblings ...)
2019-10-29 19:55 ` [Patch v3 17/22] UefiCpuPkg: " Michael D Kinney
@ 2019-10-29 19:55 ` Michael D Kinney
2019-11-05 8:23 ` [edk2-devel] " Liming Gao
2019-10-29 19:55 ` [Patch v3 19/22] .pytool: Add CISettings.py and Readme.md Michael D Kinney
` (4 subsequent siblings)
22 siblings, 1 reply; 56+ messages in thread
From: Michael D Kinney @ 2019-10-29 19:55 UTC (permalink / raw)
To: devel; +Cc: Jiewen Yao, Chao Zhang
https://bugzilla.tianocore.org/show_bug.cgi?id=2315
Use BaseCryptLibNull for package CI builds to reduce package
build times. Enabled with PYTOOL_CONTINUOUS_INTEGRATION in YAML
files. By default PYTOOL_CONTINUOUS_INTEGRATION is not defined,
and the original lib mappings are preserved.
Cc: Jiewen Yao <jiewen.yao@intel.com>
Cc: Chao Zhang <chao.b.zhang@intel.com>
Signed-off-by: Michael D Kinney <michael.d.kinney@intel.com>
---
SignedCapsulePkg/SignedCapsulePkg.dsc | 38 +++++++++++++++++++++++++--
1 file changed, 36 insertions(+), 2 deletions(-)
diff --git a/SignedCapsulePkg/SignedCapsulePkg.dsc b/SignedCapsulePkg/SignedCapsulePkg.dsc
index 03f714f9a7..c5080ec1dd 100644
--- a/SignedCapsulePkg/SignedCapsulePkg.dsc
+++ b/SignedCapsulePkg/SignedCapsulePkg.dsc
@@ -87,8 +87,6 @@ [LibraryClasses]
AuthVariableLib|MdeModulePkg/Library/AuthVariableLibNull/AuthVariableLibNull.inf
VarCheckLib|MdeModulePkg/Library/VarCheckLib/VarCheckLib.inf
FileExplorerLib|MdeModulePkg/Library/FileExplorerLib/FileExplorerLib.inf
- IntrinsicLib|CryptoPkg/Library/IntrinsicLib/IntrinsicLib.inf
- OpensslLib|CryptoPkg/Library/OpensslLib/OpensslLib.inf
EdkiiSystemCapsuleLib|SignedCapsulePkg/Library/EdkiiSystemCapsuleLib/EdkiiSystemCapsuleLib.inf
IniParsingLib|SignedCapsulePkg/Library/IniParsingLib/IniParsingLib.inf
@@ -116,7 +114,13 @@ [LibraryClasses.common.PEIM]
HobLib|MdePkg/Library/PeiHobLib/PeiHobLib.inf
MemoryAllocationLib|MdePkg/Library/PeiMemoryAllocationLib/PeiMemoryAllocationLib.inf
LockBoxLib|MdeModulePkg/Library/SmmLockBoxLib/SmmLockBoxPeiLib.inf
+!ifdef CONTINUOUS_INTEGRATION
+ BaseCryptLib|CryptoPkg/Library/BaseCryptLibNull/BaseCryptLibNull.inf
+!else
+ IntrinsicLib|CryptoPkg/Library/IntrinsicLib/IntrinsicLib.inf
+ OpensslLib|CryptoPkg/Library/OpensslLib/OpensslLib.inf
BaseCryptLib|CryptoPkg/Library/BaseCryptLib/PeiCryptLib.inf
+!endif
[LibraryClasses.common.DXE_CORE]
HobLib|MdePkg/Library/DxeCoreHobLib/DxeCoreHobLib.inf
@@ -125,7 +129,13 @@ [LibraryClasses.common.DXE_CORE]
[LibraryClasses.common.DXE_DRIVER]
HobLib|MdePkg/Library/DxeHobLib/DxeHobLib.inf
MemoryAllocationLib|MdePkg/Library/UefiMemoryAllocationLib/UefiMemoryAllocationLib.inf
+!ifdef CONTINUOUS_INTEGRATION
+ BaseCryptLib|CryptoPkg/Library/BaseCryptLibNull/BaseCryptLibNull.inf
+!else
+ IntrinsicLib|CryptoPkg/Library/IntrinsicLib/IntrinsicLib.inf
+ OpensslLib|CryptoPkg/Library/OpensslLib/OpensslLib.inf
BaseCryptLib|CryptoPkg/Library/BaseCryptLib/BaseCryptLib.inf
+!endif
CapsuleLib|MdeModulePkg/Library/DxeCapsuleLibFmp/DxeCapsuleLib.inf
@@ -133,7 +143,13 @@ [LibraryClasses.common.DXE_RUNTIME_DRIVER]
HobLib|MdePkg/Library/DxeHobLib/DxeHobLib.inf
MemoryAllocationLib|MdePkg/Library/UefiMemoryAllocationLib/UefiMemoryAllocationLib.inf
DebugLib|MdePkg/Library/UefiDebugLibConOut/UefiDebugLibConOut.inf
+!ifdef CONTINUOUS_INTEGRATION
+ BaseCryptLib|CryptoPkg/Library/BaseCryptLibNull/BaseCryptLibNull.inf
+!else
+ IntrinsicLib|CryptoPkg/Library/IntrinsicLib/IntrinsicLib.inf
+ OpensslLib|CryptoPkg/Library/OpensslLib/OpensslLib.inf
BaseCryptLib|CryptoPkg/Library/BaseCryptLib/RuntimeCryptLib.inf
+!endif
CapsuleLib|MdeModulePkg/Library/DxeCapsuleLibFmp/DxeRuntimeCapsuleLib.inf
@@ -150,19 +166,37 @@ [LibraryClasses.common.DXE_SMM_DRIVER]
MemoryAllocationLib|MdePkg/Library/SmmMemoryAllocationLib/SmmMemoryAllocationLib.inf
SmmServicesTableLib|MdePkg/Library/SmmServicesTableLib/SmmServicesTableLib.inf
SmmMemLib|MdePkg/Library/SmmMemLib/SmmMemLib.inf
+!ifdef CONTINUOUS_INTEGRATION
+ BaseCryptLib|CryptoPkg/Library/BaseCryptLibNull/BaseCryptLibNull.inf
+!else
+ IntrinsicLib|CryptoPkg/Library/IntrinsicLib/IntrinsicLib.inf
+ OpensslLib|CryptoPkg/Library/OpensslLib/OpensslLib.inf
BaseCryptLib|CryptoPkg/Library/BaseCryptLib/SmmCryptLib.inf
+!endif
[LibraryClasses.common.UEFI_DRIVER]
HobLib|MdePkg/Library/DxeHobLib/DxeHobLib.inf
MemoryAllocationLib|MdePkg/Library/UefiMemoryAllocationLib/UefiMemoryAllocationLib.inf
DebugLib|MdePkg/Library/UefiDebugLibConOut/UefiDebugLibConOut.inf
+!ifdef CONTINUOUS_INTEGRATION
+ BaseCryptLib|CryptoPkg/Library/BaseCryptLibNull/BaseCryptLibNull.inf
+!else
+ IntrinsicLib|CryptoPkg/Library/IntrinsicLib/IntrinsicLib.inf
+ OpensslLib|CryptoPkg/Library/OpensslLib/OpensslLib.inf
BaseCryptLib|CryptoPkg/Library/BaseCryptLib/BaseCryptLib.inf
+!endif
[LibraryClasses.common.UEFI_APPLICATION]
HobLib|MdePkg/Library/DxeHobLib/DxeHobLib.inf
MemoryAllocationLib|MdePkg/Library/UefiMemoryAllocationLib/UefiMemoryAllocationLib.inf
DebugLib|MdePkg/Library/UefiDebugLibStdErr/UefiDebugLibStdErr.inf
+!ifdef CONTINUOUS_INTEGRATION
+ BaseCryptLib|CryptoPkg/Library/BaseCryptLibNull/BaseCryptLibNull.inf
+!else
+ IntrinsicLib|CryptoPkg/Library/IntrinsicLib/IntrinsicLib.inf
+ OpensslLib|CryptoPkg/Library/OpensslLib/OpensslLib.inf
BaseCryptLib|CryptoPkg/Library/BaseCryptLib/BaseCryptLib.inf
+!endif
[PcdsFixedAtBuild]
gEfiMdePkgTokenSpaceGuid.PcdDebugPropertyMask|0x0f
--
2.21.0.windows.1
^ permalink raw reply related [flat|nested] 56+ messages in thread* Re: [edk2-devel] [Patch v3 18/22] SignedCapsulePkg: Use BaseCryptLibNull to reduce package CI time
2019-10-29 19:55 ` [Patch v3 18/22] SignedCapsulePkg: Use BaseCryptLibNull to reduce package CI time Michael D Kinney
@ 2019-11-05 8:23 ` Liming Gao
2019-11-05 17:05 ` Michael D Kinney
0 siblings, 1 reply; 56+ messages in thread
From: Liming Gao @ 2019-11-05 8:23 UTC (permalink / raw)
To: devel@edk2.groups.io, Kinney, Michael D; +Cc: Yao, Jiewen, Zhang, Chao B
Mike:
Is there the package CI YAML file for SignedCapsulePkg?
Thanks
Liming
>-----Original Message-----
>From: devel@edk2.groups.io [mailto:devel@edk2.groups.io] On Behalf Of
>Michael D Kinney
>Sent: Wednesday, October 30, 2019 3:55 AM
>To: devel@edk2.groups.io
>Cc: Yao, Jiewen <jiewen.yao@intel.com>; Zhang, Chao B
><chao.b.zhang@intel.com>
>Subject: [edk2-devel] [Patch v3 18/22] SignedCapsulePkg: Use
>BaseCryptLibNull to reduce package CI time
>
>https://bugzilla.tianocore.org/show_bug.cgi?id=2315
>
>Use BaseCryptLibNull for package CI builds to reduce package
>build times. Enabled with PYTOOL_CONTINUOUS_INTEGRATION in YAML
>files. By default PYTOOL_CONTINUOUS_INTEGRATION is not defined,
>and the original lib mappings are preserved.
>
>Cc: Jiewen Yao <jiewen.yao@intel.com>
>Cc: Chao Zhang <chao.b.zhang@intel.com>
>Signed-off-by: Michael D Kinney <michael.d.kinney@intel.com>
>---
> SignedCapsulePkg/SignedCapsulePkg.dsc | 38
>+++++++++++++++++++++++++--
> 1 file changed, 36 insertions(+), 2 deletions(-)
>
>diff --git a/SignedCapsulePkg/SignedCapsulePkg.dsc
>b/SignedCapsulePkg/SignedCapsulePkg.dsc
>index 03f714f9a7..c5080ec1dd 100644
>--- a/SignedCapsulePkg/SignedCapsulePkg.dsc
>+++ b/SignedCapsulePkg/SignedCapsulePkg.dsc
>@@ -87,8 +87,6 @@ [LibraryClasses]
>
>AuthVariableLib|MdeModulePkg/Library/AuthVariableLibNull/AuthVariableLi
>bNull.inf
> VarCheckLib|MdeModulePkg/Library/VarCheckLib/VarCheckLib.inf
> FileExplorerLib|MdeModulePkg/Library/FileExplorerLib/FileExplorerLib.inf
>- IntrinsicLib|CryptoPkg/Library/IntrinsicLib/IntrinsicLib.inf
>- OpensslLib|CryptoPkg/Library/OpensslLib/OpensslLib.inf
>
>
>EdkiiSystemCapsuleLib|SignedCapsulePkg/Library/EdkiiSystemCapsuleLib/Edk
>iiSystemCapsuleLib.inf
> IniParsingLib|SignedCapsulePkg/Library/IniParsingLib/IniParsingLib.inf
>@@ -116,7 +114,13 @@ [LibraryClasses.common.PEIM]
> HobLib|MdePkg/Library/PeiHobLib/PeiHobLib.inf
>
>MemoryAllocationLib|MdePkg/Library/PeiMemoryAllocationLib/PeiMemory
>AllocationLib.inf
> LockBoxLib|MdeModulePkg/Library/SmmLockBoxLib/SmmLockBoxPeiLib.inf
>+!ifdef CONTINUOUS_INTEGRATION
>+ BaseCryptLib|CryptoPkg/Library/BaseCryptLibNull/BaseCryptLibNull.inf
>+!else
>+ IntrinsicLib|CryptoPkg/Library/IntrinsicLib/IntrinsicLib.inf
>+ OpensslLib|CryptoPkg/Library/OpensslLib/OpensslLib.inf
> BaseCryptLib|CryptoPkg/Library/BaseCryptLib/PeiCryptLib.inf
>+!endif
>
> [LibraryClasses.common.DXE_CORE]
> HobLib|MdePkg/Library/DxeCoreHobLib/DxeCoreHobLib.inf
>@@ -125,7 +129,13 @@ [LibraryClasses.common.DXE_CORE]
> [LibraryClasses.common.DXE_DRIVER]
> HobLib|MdePkg/Library/DxeHobLib/DxeHobLib.inf
>
>MemoryAllocationLib|MdePkg/Library/UefiMemoryAllocationLib/UefiMemor
>yAllocationLib.inf
>+!ifdef CONTINUOUS_INTEGRATION
>+ BaseCryptLib|CryptoPkg/Library/BaseCryptLibNull/BaseCryptLibNull.inf
>+!else
>+ IntrinsicLib|CryptoPkg/Library/IntrinsicLib/IntrinsicLib.inf
>+ OpensslLib|CryptoPkg/Library/OpensslLib/OpensslLib.inf
> BaseCryptLib|CryptoPkg/Library/BaseCryptLib/BaseCryptLib.inf
>+!endif
>
> CapsuleLib|MdeModulePkg/Library/DxeCapsuleLibFmp/DxeCapsuleLib.inf
>
>@@ -133,7 +143,13 @@ [LibraryClasses.common.DXE_RUNTIME_DRIVER]
> HobLib|MdePkg/Library/DxeHobLib/DxeHobLib.inf
>
>MemoryAllocationLib|MdePkg/Library/UefiMemoryAllocationLib/UefiMemor
>yAllocationLib.inf
> DebugLib|MdePkg/Library/UefiDebugLibConOut/UefiDebugLibConOut.inf
>+!ifdef CONTINUOUS_INTEGRATION
>+ BaseCryptLib|CryptoPkg/Library/BaseCryptLibNull/BaseCryptLibNull.inf
>+!else
>+ IntrinsicLib|CryptoPkg/Library/IntrinsicLib/IntrinsicLib.inf
>+ OpensslLib|CryptoPkg/Library/OpensslLib/OpensslLib.inf
> BaseCryptLib|CryptoPkg/Library/BaseCryptLib/RuntimeCryptLib.inf
>+!endif
>
>
>CapsuleLib|MdeModulePkg/Library/DxeCapsuleLibFmp/DxeRuntimeCapsuleL
>ib.inf
>
>@@ -150,19 +166,37 @@ [LibraryClasses.common.DXE_SMM_DRIVER]
>
>MemoryAllocationLib|MdePkg/Library/SmmMemoryAllocationLib/SmmMem
>oryAllocationLib.inf
>
>SmmServicesTableLib|MdePkg/Library/SmmServicesTableLib/SmmServicesTa
>bleLib.inf
> SmmMemLib|MdePkg/Library/SmmMemLib/SmmMemLib.inf
>+!ifdef CONTINUOUS_INTEGRATION
>+ BaseCryptLib|CryptoPkg/Library/BaseCryptLibNull/BaseCryptLibNull.inf
>+!else
>+ IntrinsicLib|CryptoPkg/Library/IntrinsicLib/IntrinsicLib.inf
>+ OpensslLib|CryptoPkg/Library/OpensslLib/OpensslLib.inf
> BaseCryptLib|CryptoPkg/Library/BaseCryptLib/SmmCryptLib.inf
>+!endif
>
> [LibraryClasses.common.UEFI_DRIVER]
> HobLib|MdePkg/Library/DxeHobLib/DxeHobLib.inf
>
>MemoryAllocationLib|MdePkg/Library/UefiMemoryAllocationLib/UefiMemor
>yAllocationLib.inf
> DebugLib|MdePkg/Library/UefiDebugLibConOut/UefiDebugLibConOut.inf
>+!ifdef CONTINUOUS_INTEGRATION
>+ BaseCryptLib|CryptoPkg/Library/BaseCryptLibNull/BaseCryptLibNull.inf
>+!else
>+ IntrinsicLib|CryptoPkg/Library/IntrinsicLib/IntrinsicLib.inf
>+ OpensslLib|CryptoPkg/Library/OpensslLib/OpensslLib.inf
> BaseCryptLib|CryptoPkg/Library/BaseCryptLib/BaseCryptLib.inf
>+!endif
>
> [LibraryClasses.common.UEFI_APPLICATION]
> HobLib|MdePkg/Library/DxeHobLib/DxeHobLib.inf
>
>MemoryAllocationLib|MdePkg/Library/UefiMemoryAllocationLib/UefiMemor
>yAllocationLib.inf
> DebugLib|MdePkg/Library/UefiDebugLibStdErr/UefiDebugLibStdErr.inf
>+!ifdef CONTINUOUS_INTEGRATION
>+ BaseCryptLib|CryptoPkg/Library/BaseCryptLibNull/BaseCryptLibNull.inf
>+!else
>+ IntrinsicLib|CryptoPkg/Library/IntrinsicLib/IntrinsicLib.inf
>+ OpensslLib|CryptoPkg/Library/OpensslLib/OpensslLib.inf
> BaseCryptLib|CryptoPkg/Library/BaseCryptLib/BaseCryptLib.inf
>+!endif
>
> [PcdsFixedAtBuild]
> gEfiMdePkgTokenSpaceGuid.PcdDebugPropertyMask|0x0f
>--
>2.21.0.windows.1
>
>
>
^ permalink raw reply [flat|nested] 56+ messages in thread
* Re: [edk2-devel] [Patch v3 18/22] SignedCapsulePkg: Use BaseCryptLibNull to reduce package CI time
2019-11-05 8:23 ` [edk2-devel] " Liming Gao
@ 2019-11-05 17:05 ` Michael D Kinney
0 siblings, 0 replies; 56+ messages in thread
From: Michael D Kinney @ 2019-11-05 17:05 UTC (permalink / raw)
To: Gao, Liming, devel@edk2.groups.io, Kinney, Michael D
Cc: Yao, Jiewen, Zhang, Chao B
Liming,
Not yet. We will add more packages in a future series.
Mike
> -----Original Message-----
> From: Gao, Liming <liming.gao@intel.com>
> Sent: Tuesday, November 5, 2019 12:23 AM
> To: devel@edk2.groups.io; Kinney, Michael D
> <michael.d.kinney@intel.com>
> Cc: Yao, Jiewen <jiewen.yao@intel.com>; Zhang, Chao B
> <chao.b.zhang@intel.com>
> Subject: RE: [edk2-devel] [Patch v3 18/22]
> SignedCapsulePkg: Use BaseCryptLibNull to reduce package
> CI time
>
> Mike:
> Is there the package CI YAML file for
> SignedCapsulePkg?
>
> Thanks
> Liming
> >-----Original Message-----
> >From: devel@edk2.groups.io
> [mailto:devel@edk2.groups.io] On Behalf Of
> >Michael D Kinney
> >Sent: Wednesday, October 30, 2019 3:55 AM
> >To: devel@edk2.groups.io
> >Cc: Yao, Jiewen <jiewen.yao@intel.com>; Zhang, Chao B
> ><chao.b.zhang@intel.com>
> >Subject: [edk2-devel] [Patch v3 18/22]
> SignedCapsulePkg: Use
> >BaseCryptLibNull to reduce package CI time
> >
> >https://bugzilla.tianocore.org/show_bug.cgi?id=2315
> >
> >Use BaseCryptLibNull for package CI builds to reduce
> package build
> >times. Enabled with PYTOOL_CONTINUOUS_INTEGRATION in
> YAML files. By
> >default PYTOOL_CONTINUOUS_INTEGRATION is not defined,
> and the original
> >lib mappings are preserved.
> >
> >Cc: Jiewen Yao <jiewen.yao@intel.com>
> >Cc: Chao Zhang <chao.b.zhang@intel.com>
> >Signed-off-by: Michael D Kinney
> <michael.d.kinney@intel.com>
> >---
> > SignedCapsulePkg/SignedCapsulePkg.dsc | 38
> >+++++++++++++++++++++++++--
> > 1 file changed, 36 insertions(+), 2 deletions(-)
> >
> >diff --git a/SignedCapsulePkg/SignedCapsulePkg.dsc
> >b/SignedCapsulePkg/SignedCapsulePkg.dsc
> >index 03f714f9a7..c5080ec1dd 100644
> >--- a/SignedCapsulePkg/SignedCapsulePkg.dsc
> >+++ b/SignedCapsulePkg/SignedCapsulePkg.dsc
> >@@ -87,8 +87,6 @@ [LibraryClasses]
> >
> >AuthVariableLib|MdeModulePkg/Library/AuthVariableLibNul
> l/AuthVariableLi
> >bNull.inf
> >
> VarCheckLib|MdeModulePkg/Library/VarCheckLib/VarCheckLib
> .inf
> >
> >FileExplorerLib|MdeModulePkg/Library/FileExplorerLib/Fi
> leExplorerLib.in
> >f
> >-
> IntrinsicLib|CryptoPkg/Library/IntrinsicLib/IntrinsicLib
> .inf
> >-
> OpensslLib|CryptoPkg/Library/OpensslLib/OpensslLib.inf
> >
> >
> >EdkiiSystemCapsuleLib|SignedCapsulePkg/Library/EdkiiSys
> temCapsuleLib/Ed
> >EdkiiSystemCapsuleLib|k
> >iiSystemCapsuleLib.inf
> >
> >IniParsingLib|SignedCapsulePkg/Library/IniParsingLib/In
> iParsingLib.inf
> >@@ -116,7 +114,13 @@ [LibraryClasses.common.PEIM]
> > HobLib|MdePkg/Library/PeiHobLib/PeiHobLib.inf
> >
> >MemoryAllocationLib|MdePkg/Library/PeiMemoryAllocationL
> ib/PeiMemory
> >AllocationLib.inf
> >
> LockBoxLib|MdeModulePkg/Library/SmmLockBoxLib/SmmLockBox
> PeiLib.inf
> >+!ifdef CONTINUOUS_INTEGRATION
> >+
> BaseCryptLib|CryptoPkg/Library/BaseCryptLibNull/BaseCryp
> tLibNull.inf
> >+!else
> >+
> IntrinsicLib|CryptoPkg/Library/IntrinsicLib/IntrinsicLib
> .inf
> >+
> OpensslLib|CryptoPkg/Library/OpensslLib/OpensslLib.inf
> >
> BaseCryptLib|CryptoPkg/Library/BaseCryptLib/PeiCryptLib.
> inf
> >+!endif
> >
> > [LibraryClasses.common.DXE_CORE]
> >
> HobLib|MdePkg/Library/DxeCoreHobLib/DxeCoreHobLib.inf
> >@@ -125,7 +129,13 @@ [LibraryClasses.common.DXE_CORE]
> >[LibraryClasses.common.DXE_DRIVER]
> > HobLib|MdePkg/Library/DxeHobLib/DxeHobLib.inf
> >
> >MemoryAllocationLib|MdePkg/Library/UefiMemoryAllocation
> Lib/UefiMemor
> >yAllocationLib.inf
> >+!ifdef CONTINUOUS_INTEGRATION
> >+
> BaseCryptLib|CryptoPkg/Library/BaseCryptLibNull/BaseCryp
> tLibNull.inf
> >+!else
> >+
> IntrinsicLib|CryptoPkg/Library/IntrinsicLib/IntrinsicLib
> .inf
> >+
> OpensslLib|CryptoPkg/Library/OpensslLib/OpensslLib.inf
> >
> BaseCryptLib|CryptoPkg/Library/BaseCryptLib/BaseCryptLib
> .inf
> >+!endif
> >
> >
> CapsuleLib|MdeModulePkg/Library/DxeCapsuleLibFmp/DxeCaps
> uleLib.inf
> >
> >@@ -133,7 +143,13 @@
> [LibraryClasses.common.DXE_RUNTIME_DRIVER]
> > HobLib|MdePkg/Library/DxeHobLib/DxeHobLib.inf
> >
> >MemoryAllocationLib|MdePkg/Library/UefiMemoryAllocation
> Lib/UefiMemor
> >yAllocationLib.inf
> >
> DebugLib|MdePkg/Library/UefiDebugLibConOut/UefiDebugLibC
> onOut.inf
> >+!ifdef CONTINUOUS_INTEGRATION
> >+
> BaseCryptLib|CryptoPkg/Library/BaseCryptLibNull/BaseCryp
> tLibNull.inf
> >+!else
> >+
> IntrinsicLib|CryptoPkg/Library/IntrinsicLib/IntrinsicLib
> .inf
> >+
> OpensslLib|CryptoPkg/Library/OpensslLib/OpensslLib.inf
> >
> BaseCryptLib|CryptoPkg/Library/BaseCryptLib/RuntimeCrypt
> Lib.inf
> >+!endif
> >
> >
> >CapsuleLib|MdeModulePkg/Library/DxeCapsuleLibFmp/DxeRun
> timeCapsuleL
> >ib.inf
> >
> >@@ -150,19 +166,37 @@
> [LibraryClasses.common.DXE_SMM_DRIVER]
> >
> >MemoryAllocationLib|MdePkg/Library/SmmMemoryAllocationL
> ib/SmmMem
> >oryAllocationLib.inf
> >
> >SmmServicesTableLib|MdePkg/Library/SmmServicesTableLib/
> SmmServicesTa
> >bleLib.inf
> > SmmMemLib|MdePkg/Library/SmmMemLib/SmmMemLib.inf
> >+!ifdef CONTINUOUS_INTEGRATION
> >+
> BaseCryptLib|CryptoPkg/Library/BaseCryptLibNull/BaseCryp
> tLibNull.inf
> >+!else
> >+
> IntrinsicLib|CryptoPkg/Library/IntrinsicLib/IntrinsicLib
> .inf
> >+
> OpensslLib|CryptoPkg/Library/OpensslLib/OpensslLib.inf
> >
> BaseCryptLib|CryptoPkg/Library/BaseCryptLib/SmmCryptLib.
> inf
> >+!endif
> >
> > [LibraryClasses.common.UEFI_DRIVER]
> > HobLib|MdePkg/Library/DxeHobLib/DxeHobLib.inf
> >
> >MemoryAllocationLib|MdePkg/Library/UefiMemoryAllocation
> Lib/UefiMemor
> >yAllocationLib.inf
> >
> DebugLib|MdePkg/Library/UefiDebugLibConOut/UefiDebugLibC
> onOut.inf
> >+!ifdef CONTINUOUS_INTEGRATION
> >+
> BaseCryptLib|CryptoPkg/Library/BaseCryptLibNull/BaseCryp
> tLibNull.inf
> >+!else
> >+
> IntrinsicLib|CryptoPkg/Library/IntrinsicLib/IntrinsicLib
> .inf
> >+
> OpensslLib|CryptoPkg/Library/OpensslLib/OpensslLib.inf
> >
> BaseCryptLib|CryptoPkg/Library/BaseCryptLib/BaseCryptLib
> .inf
> >+!endif
> >
> > [LibraryClasses.common.UEFI_APPLICATION]
> > HobLib|MdePkg/Library/DxeHobLib/DxeHobLib.inf
> >
> >MemoryAllocationLib|MdePkg/Library/UefiMemoryAllocation
> Lib/UefiMemor
> >yAllocationLib.inf
> >
> DebugLib|MdePkg/Library/UefiDebugLibStdErr/UefiDebugLibS
> tdErr.inf
> >+!ifdef CONTINUOUS_INTEGRATION
> >+
> BaseCryptLib|CryptoPkg/Library/BaseCryptLibNull/BaseCryp
> tLibNull.inf
> >+!else
> >+
> IntrinsicLib|CryptoPkg/Library/IntrinsicLib/IntrinsicLib
> .inf
> >+
> OpensslLib|CryptoPkg/Library/OpensslLib/OpensslLib.inf
> >
> BaseCryptLib|CryptoPkg/Library/BaseCryptLib/BaseCryptLib
> .inf
> >+!endif
> >
> > [PcdsFixedAtBuild]
> > gEfiMdePkgTokenSpaceGuid.PcdDebugPropertyMask|0x0f
> >--
> >2.21.0.windows.1
> >
> >
> >
^ permalink raw reply [flat|nested] 56+ messages in thread
* [Patch v3 19/22] .pytool: Add CISettings.py and Readme.md
2019-10-29 19:54 [Patch v3 00/22] Enable Phase 1 of EDK II CI Michael D Kinney
` (17 preceding siblings ...)
2019-10-29 19:55 ` [Patch v3 18/22] SignedCapsulePkg: Use BaseCryptLibNull to reduce package CI time Michael D Kinney
@ 2019-10-29 19:55 ` Michael D Kinney
2019-10-29 19:55 ` [Patch v3 20/22] .azurepipelines: Add Azure Pipelines YML configuration files Michael D Kinney
` (3 subsequent siblings)
22 siblings, 0 replies; 56+ messages in thread
From: Michael D Kinney @ 2019-10-29 19:55 UTC (permalink / raw)
To: devel; +Cc: Sean Brogan, Bret Barkelew, Liming Gao
From: Sean Brogan <sean.brogan@microsoft.com>
https://bugzilla.tianocore.org/show_bug.cgi?id=2315
Add main python script for EDK II Continuous Integration (CI)
builds along with a Readme.md that provides a summary of the
packages, platforms, and checks performs during a CI build.
Cc: Sean Brogan <sean.brogan@microsoft.com>
Cc: Bret Barkelew <Bret.Barkelew@microsoft.com>
Cc: Liming Gao <liming.gao@intel.com>
Signed-off-by: Michael D Kinney <michael.d.kinney@intel.com>
---
.pytool/CISettings.py | 173 ++++++++++++++++++++++++++++++++++++++++++
.pytool/Readme.md | 128 +++++++++++++++++++++++++++++++
2 files changed, 301 insertions(+)
create mode 100644 .pytool/CISettings.py
create mode 100644 .pytool/Readme.md
diff --git a/.pytool/CISettings.py b/.pytool/CISettings.py
new file mode 100644
index 0000000000..a78e8b974c
--- /dev/null
+++ b/.pytool/CISettings.py
@@ -0,0 +1,173 @@
+# @file
+#
+# Copyright (c) 2018, Microsoft Corporation
+# SPDX-License-Identifier: BSD-2-Clause-Patent
+##
+import os
+import logging
+from edk2toolext.environment import shell_environment
+from edk2toolext.invocables.edk2_ci_build import CiBuildSettingsManager
+from edk2toolext.invocables.edk2_setup import SetupSettingsManager, RequiredSubmodule
+from edk2toolext.invocables.edk2_update import UpdateSettingsManager
+from edk2toolext.invocables.edk2_pr_eval import PrEvalSettingsManager
+from edk2toollib.utility_functions import GetHostInfo
+
+
+class Settings(CiBuildSettingsManager, UpdateSettingsManager, SetupSettingsManager, PrEvalSettingsManager):
+
+ def __init__(self):
+ self.ActualPackages = []
+ self.ActualTargets = []
+ self.ActualArchitectures = []
+ self.ActualToolChainTag = ""
+
+ # ####################################################################################### #
+ # Extra CmdLine configuration #
+ # ####################################################################################### #
+
+ def AddCommandLineOptions(self, parserObj):
+ pass
+
+ def RetrieveCommandLineOptions(self, args):
+ pass
+
+ # ####################################################################################### #
+ # Default Support for this Ci Build #
+ # ####################################################################################### #
+
+ def GetPackagesSupported(self):
+ ''' return iterable of edk2 packages supported by this build.
+ These should be edk2 workspace relative paths '''
+
+ return ("MdePkg",
+ "MdeModulePkg",
+ "NetworkPkg",
+ "PcAtChipsetPkg",
+ "SecurityPkg",
+ "UefiCpuPkg",
+ "FmpDevicePkg",
+ "ShellPkg",
+ "FatPkg",
+ "CryptoPkg"
+ )
+
+ def GetArchitecturesSupported(self):
+ ''' return iterable of edk2 architectures supported by this build '''
+ return ("IA32",
+ "X64",
+ "ARM",
+ "AARCH64")
+
+ def GetTargetsSupported(self):
+ ''' return iterable of edk2 target tags supported by this build '''
+ return ("DEBUG", "RELEASE", "NO-TARGET", "NOOPT")
+
+ # ####################################################################################### #
+ # Verify and Save requested Ci Build Config #
+ # ####################################################################################### #
+
+ def SetPackages(self, list_of_requested_packages):
+ ''' Confirm the requested package list is valid and configure SettingsManager
+ to build the requested packages.
+
+ Raise UnsupportedException if a requested_package is not supported
+ '''
+ unsupported = set(list_of_requested_packages) - \
+ set(self.GetPackagesSupported())
+ if(len(unsupported) > 0):
+ logging.critical(
+ "Unsupported Package Requested: " + " ".join(unsupported))
+ raise Exception("Unsupported Package Requested: " +
+ " ".join(unsupported))
+ self.ActualPackages = list_of_requested_packages
+
+ def SetArchitectures(self, list_of_requested_architectures):
+ ''' Confirm the requests architecture list is valid and configure SettingsManager
+ to run only the requested architectures.
+
+ Raise Exception if a list_of_requested_architectures is not supported
+ '''
+ unsupported = set(list_of_requested_architectures) - \
+ set(self.GetArchitecturesSupported())
+ if(len(unsupported) > 0):
+ logging.critical(
+ "Unsupported Architecture Requested: " + " ".join(unsupported))
+ raise Exception(
+ "Unsupported Architecture Requested: " + " ".join(unsupported))
+ self.ActualArchitectures = list_of_requested_architectures
+
+ def SetTargets(self, list_of_requested_target):
+ ''' Confirm the request target list is valid and configure SettingsManager
+ to run only the requested targets.
+
+ Raise UnsupportedException if a requested_target is not supported
+ '''
+ unsupported = set(list_of_requested_target) - \
+ set(self.GetTargetsSupported())
+ if(len(unsupported) > 0):
+ logging.critical(
+ "Unsupported Targets Requested: " + " ".join(unsupported))
+ raise Exception("Unsupported Targets Requested: " +
+ " ".join(unsupported))
+ self.ActualTargets = list_of_requested_target
+
+ # ####################################################################################### #
+ # Actual Configuration for Ci Build #
+ # ####################################################################################### #
+
+ def GetActiveScopes(self):
+ ''' return tuple containing scopes that should be active for this process '''
+ scopes = ("cibuild","edk2-build")
+
+ self.ActualToolChainTag = shell_environment.GetBuildVars().GetValue("TOOL_CHAIN_TAG", "")
+
+ if GetHostInfo().os.upper() == "LINUX" and self.ActualToolChainTag.upper().startswith("GCC"):
+ if "AARCH64" in self.ActualArchitectures:
+ scopes += ("gcc_aarch64_linux",)
+ if "ARM" in self.ActualArchitectures:
+ scopes += ("gcc_arm_linux",)
+
+ return scopes
+
+ def GetRequiredSubmodules(self):
+ ''' return iterable containing RequiredSubmodule objects.
+ If no RequiredSubmodules return an empty iterable
+ '''
+ rs=[]
+ rs.append(RequiredSubmodule(
+ "ArmPkg/Library/ArmSoftFloatLib/berkeley-softfloat-3", False))
+ rs.append(RequiredSubmodule(
+ "CryptoPkg/Library/OpensslLib/openssl", False))
+ return rs
+
+ def GetName(self):
+ return "Edk2"
+
+ def GetDependencies(self):
+ return []
+
+ def GetPackagesPath(self):
+ return ()
+
+ def GetWorkspaceRoot(self):
+ ''' get WorkspacePath '''
+ return os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
+
+ def FilterPackagesToTest(self, changedFilesList: list, potentialPackagesList: list) -> list:
+ ''' Filter potential packages to test based on changed files. '''
+ build_these_packages=[]
+ possible_packages=potentialPackagesList.copy()
+ for f in changedFilesList:
+ nodes=f.split("/") # split each part of path for comparison later
+
+ # python file change in .pytool folder causes building all
+ if f.endswith(".py") and ".pytool" in nodes:
+ build_these_packages = possible_packages
+ break
+
+ # BaseTools files that might change the build
+ if "BaseTools" in nodes:
+ if os.path.splitext(f) not in [".txt", ".md"]:
+ build_these_packages = possible_packages
+ break
+ return build_these_packages
diff --git a/.pytool/Readme.md b/.pytool/Readme.md
new file mode 100644
index 0000000000..5c1b107213
--- /dev/null
+++ b/.pytool/Readme.md
@@ -0,0 +1,128 @@
+# Edk2 Continuous Integration
+
+## Basic Status
+
+| Package | Windows VS2019 | Ubuntu GCC5 | Known Issues |
+| :---- | :----- | :---- | :--- |
+|ArmPkg | | |
+|ArmPlatformPkg |||
+|ArmVirtPkg |||
+|CryptoPkg | :heavy_check_mark: | :heavy_check_mark: | New changes for NULL basecryptlib and tlslib, add IntrinsicLib.inf to dsc|
+|DynamicTablesPkg |||
+|EmbeddedPkg |||
+|EmulatorPkg |||
+|FatPkg|:heavy_check_mark:|:heavy_check_mark:||
+|FmpDevicePkg|:heavy_check_mark:|:heavy_check_mark:||
+|IntelFsp2Pkg|||
+|IntelFsp2WrapperPkg|||
+| MdeModulePkg | :heavy_check_mark: |:heavy_check_mark: |DxeIpl dependency on ArmPkg, Missing Visual Studio AARCH64/ARM support for EBC |
+|MdePkg | :heavy_check_mark: |:heavy_check_mark: | Update DSC to add UefiFileHandleLib, Update BaseIoLibIntrinsic to support MSFT ARM/AARCH64|
+|NetworkPkg | :heavy_check_mark: |:heavy_check_mark:| Libraries missing from components section for package dsc|
+|OvmfPkg |||
+|PcAtChipsetPkg | :heavy_check_mark: |:heavy_check_mark:|
+|SecurityPkg|:heavy_check_mark:|:heavy_check_mark:| Incorrect libraryclass in dec|
+|ShellPkg | :heavy_check_mark: |:heavy_check_mark:||
+|SignedCapsulePkg|||
+|SourceLevelDebugPkg|||
+|StandaloneMmPkg|||
+|UefiCpuPkg | :heavy_check_mark: |:heavy_check_mark: ||
+|UefiPayloadPkg|||
+
+For more detailed status look at the test results of the latest CI run on the repo readme.
+
+## Background
+
+While a number of CI solutions exist, this proposal will focus on the usage of Azure Dev Ops and Build Pipelines. For demonstration, a sample [TianoCore repo](https://github.com/tianocore/edk2-staging.git) (branch master) and [Dev Ops Pipeline](https://dev.azure.com/tianocore/edk2-ci-play/_build?definitionId=14) have been set up.
+
+Furthermore, this proposal will leverage the TianoCore python tools PIP modules: [library](https://pypi.org/project/edk2-pytool-library/) and [extensions](https://pypi.org/project/edk2-pytool-extensions/) (with repos located [here](https://github.com/tianocore/edk2-pytool-library) and [here](https://github.com/tianocore/edk2-pytool-extensions)).
+
+The primary execution flows can be found in the `ci/AzurePipelines/Windows-VS2019.yml` and `ci/AzurePipelines/Ubuntu-GCC5.yml` files. These YAML files are consumed by the Azure Dev Ops Build Pipeline and dictate what server resources should be used, how they should be configured, and what processes should be run on them. An overview of this schema can be found [here](https://docs.microsoft.com/en-us/azure/devops/pipelines/yaml-schema?view=azure-devops&tabs=schema).
+
+Inspection of these files reveals the EDKII Tools commands that make up the primary processes for the CI build: 'stuart_setup', 'stuart_update', and 'stuart_ci_build'. These commands come from the EDKII Tools PIP modules and are configured as described below. More documentation on the stuart tools can be found [here](https://github.com/tianocore/edk2-pytool-extensions/blob/master/docs/using.md) and [here](https://github.com/tianocore/edk2-pytool-extensions/blob/master/docs/features/feature_invocables.md).
+
+## Configuration
+
+Configuration of the CI process consists of (in order of precedence):
+
+* command-line arguments passed in via the Pipeline YAML
+* a per-package configuration file (e.g. `<package-name>.ci.yaml`) that is detected by the CI system in EDKII Tools.
+* a global configuration Python module (e.g. `CISetting.py`) passed in via the command-line
+
+The global configuration file is described in [this readme](https://github.com/tianocore/edk2-pytool-extensions/blob/master/docs/usability/using_settings_manager.md) from the EDKII Tools documentation. This configuration is written as a Python module so that decisions can be made dynamically based on command line parameters and codebase state.
+
+The per-package configuration file can override most settings in the global configuration file, but is not dynamic. This file can be used to skip or customize tests that may be incompatible with a specific package. By default, the global configuration will try to run all tests on all packages.
+
+## Current PyTool Test Capabilities
+
+All CI tests are instances of EDKII Tools plugins. Documentation on the plugin system can be found [here](https://github.com/tianocore/edk2-pytool-extensions/blob/master/docs/usability/using_plugin_manager.md) and [here](https://github.com/tianocore/edk2-pytool-extensions/blob/master/docs/features/feature_plugin_manager.md). Upon invocation, each plugin will be passed the path to the current package under test and a dictionary containing its targeted configuration, as assembled from the command line, per-package configuration, and global configuration.
+
+Note: CI plugins are considered unique from build plugins and helper plugins, even though some CI plugins may execute steps of a build.
+
+In the example, these plugins live alongside the code under test (in the `ci/Plugin` directory), but may be moved to the 'edk2-test' repo if that location makes more sense for the community.
+
+### Module Inclusion Test - DscCompleteCheck
+
+This test scans all available modules (via INF files) and compares them to the package-level DSC file for the package each module is contained within. The test considers it an error if any module does not appear in the `Components` section of at least one package-level DSC (indicating that it would not be built if the package were built).
+
+### Code Compilation Test - CompilerPlugin
+
+Once the Module Inclusion Test has verified that all modules would be built if all package-level DSCs were built, the Code Compilation Test simply runs through and builds every package-level DSC on every toolchain and for every architecture that is supported. Any module that fails to build is considered an error.
+
+### GUID Uniqueness Test - GuidCheck
+
+This test works on the collection of all packages rather than an individual package. It looks at all FILE_GUIDs and GUIDs declared in DEC files and ensures that they are unique for the codebase. This prevents, for example, accidental duplication of GUIDs when using an existing INF as a template for a new module.
+
+### Cross-Package Dependency Test - DependencyCheck
+
+This test compares the list of all packages used in INFs files for a given package against a list of "allowed dependencies" in plugin configuration for that package. Any module that depends on a disallowed package will cause a test failure.
+
+### Library Declaration Test - LibraryClassCheck
+
+This test scans at all library header files found in the `Library` folders in all of the package's declared include directories and ensures that all files have a matching LibraryClass declaration in the DEC file for the package. Any missing declarations will cause a failure.
+
+### Invalid Character Test - CharEncodingCheck
+
+This test scans all files in a package to make sure that there are no invalid Unicode characters that may cause build errors in some character sets/localizations.
+
+### Spell Checking - cspell
+
+This test runs a spell checker on all files within the package. This is done using the NodeJs cspell tool. For details check `ci/Plugin/SpellCheck`. For this plugin to run during ci you must install nodejs and cspell and have both available to the command line when running your CI.
+
+Install
+
+* Install nodejs from https://nodejs.org/en/
+* Install cspell
+ 1. Open cmd prompt with access to node and npm
+ 2. Run `npm install -g cspell`
+
+ More cspell info: https://github.com/streetsidesoftware/cspell
+
+## Current Azure Pipeline Tests
+
+When adding a test it can be added as either a *PyTool* test or just added to the CI build process. This should be a deliberate choice. Any change added as a pipeline test is not as easily run on a private/local workspace. But there are times where this is still the preferred method.
+
+## PyTool Scopes
+
+Scopes are how the PyTool ext_dep, path_env, and plugins are activated. Meaning that if an invocable process has a scope active then those ext_dep and path_env will be active. To allow easy integration of PyTools capabilities there are a few standard scopes.
+
+| Scope | Invocable | Description |
+| :---- | :----- | :---- |
+| global | edk2_invocable++ - should be base_abstract_invocable | Running an invocables |
+| global-win | edk2_invocable++ | Running on Microsoft Windows |
+| global-nix | edk2_invocable++ | Running on Linux based OS |
+| edk2-build | | This indicates that an invocable is building EDK2 based UEFI code |
+| cibuild | set in .pytool/CISettings.py | Suggested target for edk2 continuous integration builds. Tools used for CiBuilds can use this scope. Example: asl compiler |
+
+## Future investments
+
+* PatchCheck tests as plugins
+* MacOS/xcode support
+* Clang/LLVM support
+* Visual Studio AARCH64 and ARM support
+* BaseTools C tools CI/PR and binary release process
+* BaseTools Python tools CI/PR process
+* Host based unit testing
+* Extensible private/closed source platform reporting
+* Platform builds, validation
+* UEFI SCTs
+* Other automation
--
2.21.0.windows.1
^ permalink raw reply related [flat|nested] 56+ messages in thread* [Patch v3 20/22] .azurepipelines: Add Azure Pipelines YML configuration files
2019-10-29 19:54 [Patch v3 00/22] Enable Phase 1 of EDK II CI Michael D Kinney
` (18 preceding siblings ...)
2019-10-29 19:55 ` [Patch v3 19/22] .pytool: Add CISettings.py and Readme.md Michael D Kinney
@ 2019-10-29 19:55 ` Michael D Kinney
2019-10-29 19:55 ` [Patch v3 21/22] .merify: Add Megify YML pull request rules configuration file Michael D Kinney
` (2 subsequent siblings)
22 siblings, 0 replies; 56+ messages in thread
From: Michael D Kinney @ 2019-10-29 19:55 UTC (permalink / raw)
To: devel; +Cc: Sean Brogan, Bret Barkelew, Liming Gao
From: Sean Brogan <sean.brogan@microsoft.com>
https://bugzilla.tianocore.org/show_bug.cgi?id=2315
Add YML configuration files used to run the EDK II Continuous
Integration (CI) checks on Azure Pipelines agents.
Cc: Sean Brogan <sean.brogan@microsoft.com>
Cc: Bret Barkelew <Bret.Barkelew@microsoft.com>
Cc: Liming Gao <liming.gao@intel.com>
Signed-off-by: Michael D Kinney <michael.d.kinney@intel.com>
---
.azurepipelines/Ubuntu-GCC5.yml | 18 +++
.azurepipelines/Ubuntu-PatchCheck.yml | 35 +++++
.azurepipelines/Windows-VS2019.yml | 18 +++
.../templates/basetools-build-steps.yml | 36 +++++
.../templates/pr-gate-build-job.yml | 60 ++++++++
.azurepipelines/templates/pr-gate-steps.yml | 129 ++++++++++++++++++
.../templates/spell-check-prereq-steps.yml | 21 +++
7 files changed, 317 insertions(+)
create mode 100644 .azurepipelines/Ubuntu-GCC5.yml
create mode 100644 .azurepipelines/Ubuntu-PatchCheck.yml
create mode 100644 .azurepipelines/Windows-VS2019.yml
create mode 100644 .azurepipelines/templates/basetools-build-steps.yml
create mode 100644 .azurepipelines/templates/pr-gate-build-job.yml
create mode 100644 .azurepipelines/templates/pr-gate-steps.yml
create mode 100644 .azurepipelines/templates/spell-check-prereq-steps.yml
diff --git a/.azurepipelines/Ubuntu-GCC5.yml b/.azurepipelines/Ubuntu-GCC5.yml
new file mode 100644
index 0000000000..191bda8a91
--- /dev/null
+++ b/.azurepipelines/Ubuntu-GCC5.yml
@@ -0,0 +1,18 @@
+##
+# Azure Pipeline build file for a build using ubuntu and GCC5
+#
+# Copyright (c) 2019, Microsoft Corporation
+# SPDX-License-Identifier: BSD-2-Clause-Patent
+##
+trigger:
+- master
+pr:
+- master
+
+jobs:
+- template: templates/pr-gate-build-job.yml
+ parameters:
+ tool_chain_tag: 'GCC5'
+ vm_image: 'ubuntu-latest'
+ arch_list: "IA32,X64,ARM,AARCH64"
+
diff --git a/.azurepipelines/Ubuntu-PatchCheck.yml b/.azurepipelines/Ubuntu-PatchCheck.yml
new file mode 100644
index 0000000000..dff8f579b6
--- /dev/null
+++ b/.azurepipelines/Ubuntu-PatchCheck.yml
@@ -0,0 +1,35 @@
+## @file
+# Azure Pipielines YML file that evalues the patch series in a PR using the
+# python script BaseTools/Scripts/PatchCheck.py.
+#
+# NOTE: This example monitors pull requests against the edk2-ci branch. Most
+# environments would replace 'edk2-ci' with 'master'.
+#
+# Copyright (c) 2019, Intel Corporation. All rights reserved.<BR>
+# SPDX-License-Identifier: BSD-2-Clause-Patent
+#
+# https://github.com/tianocore
+#
+##
+
+trigger: none
+
+pr:
+- master
+
+pool:
+ vmImage: 'ubuntu-latest'
+
+steps:
+- checkout: self
+ clean: true
+
+- task: UsePythonVersion@0
+ inputs:
+ versionSpec: '3.7.x'
+ architecture: 'x64'
+
+- script: |
+ git fetch origin $(System.PullRequest.TargetBranch):$(System.PullRequest.TargetBranch)
+ python BaseTools/Scripts/PatchCheck.py $(System.PullRequest.TargetBranch)..$(System.PullRequest.SourceCommitId)
+ displayName: 'Use PatchCheck.py to verify patch series in pull request'
diff --git a/.azurepipelines/Windows-VS2019.yml b/.azurepipelines/Windows-VS2019.yml
new file mode 100644
index 0000000000..2143162e02
--- /dev/null
+++ b/.azurepipelines/Windows-VS2019.yml
@@ -0,0 +1,18 @@
+##
+# Azure Pipeline build file for a build using Windows and VS2019
+#
+# Copyright (c) 2019, Microsoft Corporation
+# SPDX-License-Identifier: BSD-2-Clause-Patent
+##
+trigger:
+- master
+
+pr:
+- master
+
+jobs:
+- template: templates/pr-gate-build-job.yml
+ parameters:
+ tool_chain_tag: 'VS2019'
+ vm_image: 'windows-latest'
+ arch_list: "IA32,X64"
diff --git a/.azurepipelines/templates/basetools-build-steps.yml b/.azurepipelines/templates/basetools-build-steps.yml
new file mode 100644
index 0000000000..dc1fdffd64
--- /dev/null
+++ b/.azurepipelines/templates/basetools-build-steps.yml
@@ -0,0 +1,36 @@
+# File templates/basetools-build-job.yml
+#
+# template file to build basetools
+#
+# Copyright (c) 2019, Microsoft Corporation
+# SPDX-License-Identifier: BSD-2-Clause-Patent
+##
+parameters:
+ tool_chain_tag: ''
+
+steps:
+- ${{ if contains(parameters.tool_chain_tag, 'GCC') }}:
+ - bash: sudo apt-get update
+ displayName: Update apt
+ condition: and(gt(variables.pkg_count, 0), succeeded())
+
+ - bash: sudo apt-get install gcc g++ make uuid-dev
+ displayName: Install required tools
+ condition: and(gt(variables.pkg_count, 0), succeeded())
+
+- task: CmdLine@1
+ displayName: Build Base Tools from source
+ inputs:
+ filename: python
+ arguments: BaseTools/Edk2ToolsBuild.py -t ${{ parameters.tool_chain_tag }}
+ condition: and(gt(variables.pkg_count, 0), succeeded())
+
+- task: CopyFiles@2
+ displayName: "Copy base tools build log"
+ inputs:
+ targetFolder: '$(Build.ArtifactStagingDirectory)'
+ SourceFolder: 'BaseTools/BaseToolsBuild'
+ contents: |
+ BASETOOLS_BUILD*.*
+ flattenFolders: true
+ condition: succeededOrFailed()
diff --git a/.azurepipelines/templates/pr-gate-build-job.yml b/.azurepipelines/templates/pr-gate-build-job.yml
new file mode 100644
index 0000000000..98e0ee7877
--- /dev/null
+++ b/.azurepipelines/templates/pr-gate-build-job.yml
@@ -0,0 +1,60 @@
+# File templates/pr-gate-build-job.yml
+#
+# template file used to build supported packages.
+#
+# Copyright (c) 2019, Microsoft Corporation
+# SPDX-License-Identifier: BSD-2-Clause-Patent
+##
+
+parameters:
+ tool_chain_tag: ''
+ vm_image: ''
+ arch_list: ''
+
+# Build step
+jobs:
+
+- job: Build_${{ parameters.tool_chain_tag }}
+
+ #Use matrix to speed up the build process
+ strategy:
+ matrix:
+ TARGET_MDE_CPU:
+ Build.Pkgs: 'MdePkg,UefiCpuPkg'
+ Build.Targets: 'DEBUG,RELEASE,NO-TARGET'
+ TARGET_MDEMODULE_DEBUG:
+ Build.Pkgs: 'MdeModulePkg'
+ Build.Targets: 'DEBUG'
+ TARGET_MDEMODULE_RELEASE:
+ Build.Pkgs: 'MdeModulePkg'
+ Build.Targets: 'RELEASE,NO-TARGET'
+ TARGET_NETWORK:
+ Build.Pkgs: 'NetworkPkg'
+ Build.Targets: 'DEBUG,RELEASE,NO-TARGET'
+ TARGET_OTHER:
+ Build.Pkgs: 'PcAtChipsetPkg,ShellPkg'
+ Build.Targets: 'DEBUG,RELEASE,NO-TARGET'
+ TARGET_FMP:
+ Build.Pkgs: 'FmpDevicePkg,FatPkg'
+ Build.Targets: 'DEBUG,RELEASE,NO-TARGET'
+ TARGET_CRYPTO:
+ Build.Pkgs: 'CryptoPkg'
+ Build.Targets: 'DEBUG,RELEASE,NO-TARGET'
+ TARGET_SECURITY:
+ Build.Pkgs: 'SecurityPkg'
+ Build.Targets: 'DEBUG,RELEASE,NO-TARGET'
+
+ workspace:
+ clean: all
+
+ pool:
+ vmImage: ${{ parameters.vm_image }}
+
+ steps:
+ - template: pr-gate-steps.yml
+ parameters:
+ tool_chain_tag: ${{ parameters.tool_chain_tag }}
+ build_pkgs: $(Build.Pkgs)
+ build_targets: $(Build.Targets)
+ build_archs: ${{ parameters.arch_list }}
+
diff --git a/.azurepipelines/templates/pr-gate-steps.yml b/.azurepipelines/templates/pr-gate-steps.yml
new file mode 100644
index 0000000000..d3c7eec4ad
--- /dev/null
+++ b/.azurepipelines/templates/pr-gate-steps.yml
@@ -0,0 +1,129 @@
+# File templates/pr-gate-steps.yml
+#
+# template file containing the steps to build
+#
+# Copyright (c) 2019, Microsoft Corporation
+# SPDX-License-Identifier: BSD-2-Clause-Patent
+##
+
+parameters:
+ tool_chain_tag: ''
+ build_pkgs: ''
+ build_targets: ''
+ build_archs: ''
+
+steps:
+- checkout: self
+ clean: true
+ fetchDepth: 1
+
+- task: UsePythonVersion@0
+ inputs:
+ versionSpec: '3.7.x'
+ architecture: 'x64'
+
+- script: pip install -r requirements.txt --upgrade
+ displayName: 'Install/Upgrade pip modules'
+
+# Set default
+- bash: |
+ echo "##vso[task.setvariable variable=pkgs_to_build]${{ parameters.build_pkgs }}"
+ echo "##vso[task.setvariable variable=pkg_count]${{ 1 }}"
+
+# trim the package list if this is a PR
+- task: CmdLine@1
+ displayName: Check if ${{ parameters.build_pkgs }} need testing
+ inputs:
+ filename: stuart_pr_eval
+ arguments: -c .pytool/CISettings.py -p ${{ parameters.build_pkgs }} --pr-target origin/$(System.PullRequest.targetBranch) --output-csv-format-string "##vso[task.setvariable variable=pkgs_to_build;isOutpout=true]{pkgcsv}" --output-count-format-string "##vso[task.setvariable variable=pkg_count;isOutpout=true]{pkgcount}"
+ condition: eq(variables['Build.Reason'], 'PullRequest')
+
+# build basetools
+- template: basetools-build-steps.yml
+ parameters:
+ tool_chain_tag: ${{ parameters.tool_chain_tag }}
+
+# install spell check prereqs
+- template: spell-check-prereq-steps.yml
+
+# Build repo
+- task: CmdLine@1
+ displayName: Setup ${{ parameters.build_pkgs }} ${{ parameters.build_archs}}
+ inputs:
+ filename: stuart_setup
+ arguments: -c .pytool/CISettings.py -p $(pkgs_to_build) -t ${{ parameters.build_targets}} -a ${{ parameters.build_archs}} TOOL_CHAIN_TAG=${{ parameters.tool_chain_tag}}
+ condition: and(gt(variables.pkg_count, 0), succeeded())
+
+- task: CmdLine@1
+ displayName: Update ${{ parameters.build_pkgs }} ${{ parameters.build_archs}}
+ inputs:
+ filename: stuart_update
+ arguments: -c .pytool/CISettings.py -p $(pkgs_to_build) -t ${{ parameters.build_targets}} -a ${{ parameters.build_archs}} TOOL_CHAIN_TAG=${{ parameters.tool_chain_tag}}
+ condition: and(gt(variables.pkg_count, 0), succeeded())
+
+- task: CmdLine@1
+ displayName: Build and Test ${{ parameters.build_pkgs }} ${{ parameters.build_archs}}
+ inputs:
+ filename: stuart_ci_build
+ arguments: -c .pytool/CISettings.py -p $(pkgs_to_build) -t ${{ parameters.build_targets}} -a ${{ parameters.build_archs}} TOOL_CHAIN_TAG=${{ parameters.tool_chain_tag}}
+ condition: and(gt(variables.pkg_count, 0), succeeded())
+
+# Publish Test Results to Azure Pipelines/TFS
+- task: PublishTestResults@2
+ displayName: 'Publish junit test results'
+ continueOnError: true
+ condition: and( succeededOrFailed(),gt(variables.pkg_count, 0))
+ inputs:
+ testResultsFormat: 'JUnit' # Options: JUnit, NUnit, VSTest, xUnit
+ testResultsFiles: 'Build/TestSuites.xml'
+ #searchFolder: '$(System.DefaultWorkingDirectory)' # Optional
+ mergeTestResults: true # Optional
+ testRunTitle: $(System.JobName) # Optional
+ #buildPlatform: # Optional
+ #buildConfiguration: # Optional
+ publishRunAttachments: true # Optional
+
+# Publish Test Results to Azure Pipelines/TFS
+- task: PublishTestResults@2
+ displayName: 'Publish host based test results for $(System.JobName)'
+ continueOnError: true
+ condition: and( succeededOrFailed(), gt(variables.pkg_count, 0))
+ inputs:
+ testResultsFormat: 'JUnit' # Options: JUnit, NUnit, VSTest, xUnit
+ testResultsFiles: 'Build/**/*.result.xml'
+ #searchFolder: '$(System.DefaultWorkingDirectory)' # Optional
+ mergeTestResults: false # Optional
+ testRunTitle: ${{ parameters.build_pkgs }} # Optional
+ #buildPlatform: # Optional
+ #buildConfiguration: # Optional
+ publishRunAttachments: true # Optional
+
+# Copy the build logs to the artifact staging directory
+- task: CopyFiles@2
+ displayName: "Copy build logs"
+ inputs:
+ targetFolder: '$(Build.ArtifactStagingDirectory)'
+ SourceFolder: 'Build'
+ contents: |
+ BUILDLOG_*.txt
+ BUILDLOG_*.md
+ CI_*.txt
+ CI_*.md
+ CISETUP.txt
+ SETUPLOG.txt
+ UPDATE_LOG.txt
+ PREVALLOG.txt
+ TestSuites.xml
+ **/BUILD_TOOLS_REPORT.html
+ **/OVERRIDELOG.TXT
+ flattenFolders: true
+ condition: succeededOrFailed()
+
+# Publish build artifacts to Azure Artifacts/TFS or a file share
+- task: PublishBuildArtifacts@1
+ continueOnError: true
+ displayName: "Publish build logs"
+ inputs:
+ pathtoPublish: '$(Build.ArtifactStagingDirectory)'
+ artifactName: 'Build Logs $(System.JobName)'
+ condition: succeededOrFailed()
diff --git a/.azurepipelines/templates/spell-check-prereq-steps.yml b/.azurepipelines/templates/spell-check-prereq-steps.yml
new file mode 100644
index 0000000000..7d7797ffed
--- /dev/null
+++ b/.azurepipelines/templates/spell-check-prereq-steps.yml
@@ -0,0 +1,21 @@
+# File templates/spell-check-prereq-steps.yml
+#
+# template file used to install spell checking prerequisits
+#
+# Copyright (c) 2019, Microsoft Corporation
+# SPDX-License-Identifier: BSD-2-Clause-Patent
+##
+
+parameters:
+ none: ''
+
+steps:
+- task: NodeTool@0
+ inputs:
+ versionSpec: '10.x'
+ #checkLatest: false # Optional
+ condition: and(gt(variables.pkg_count, 0), succeeded())
+
+- script: npm install -g cspell
+ displayName: 'Install cspell npm'
+ condition: and(gt(variables.pkg_count, 0), succeeded())
--
2.21.0.windows.1
^ permalink raw reply related [flat|nested] 56+ messages in thread* [Patch v3 21/22] .merify: Add Megify YML pull request rules configuration file
2019-10-29 19:54 [Patch v3 00/22] Enable Phase 1 of EDK II CI Michael D Kinney
` (19 preceding siblings ...)
2019-10-29 19:55 ` [Patch v3 20/22] .azurepipelines: Add Azure Pipelines YML configuration files Michael D Kinney
@ 2019-10-29 19:55 ` Michael D Kinney
2019-10-29 19:55 ` [Patch v3 22/22] Readme.md: Add CI build status badges Michael D Kinney
2019-10-31 9:54 ` [edk2-devel] [Patch v3 00/22] Enable Phase 1 of EDK II CI Laszlo Ersek
22 siblings, 0 replies; 56+ messages in thread
From: Michael D Kinney @ 2019-10-29 19:55 UTC (permalink / raw)
To: devel; +Cc: Liming Gao, Sean Brogan, Bret Barkelew
https://bugzilla.tianocore.org/show_bug.cgi?id=2315
Add directory for the Mergify YML configuration files that
provides rules and actions used to process a pull request.
* Auto commit a PR from EDK II Maintainer with 'push' label
set and all CI checks pass
* Auto close a PR from any developers without 'push' label
set and all CI checks pass.
* Auto close a PR from a non EDK II Maintainer that has
the 'push' label set.
* Post a comment to a PR that has a merge conflict.
Submitter can resolved conflicts and reopen the PR.
* Post a comment to a PR that fails PatchCheck.py
Submitter can resolve PatchCheck.py issues and
reopen the PR.
Cc: Liming Gao <liming.gao@intel.com>
Cc: Sean Brogan <sean.brogan@microsoft.com>
Cc: Bret Barkelew <Bret.Barkelew@microsoft.com>
Signed-off-by: Michael D Kinney <michael.d.kinney@intel.com>
---
.mergify/config.yml | 71 +++++++++++++++++++++++++++++++++++++++++++++
1 file changed, 71 insertions(+)
create mode 100644 .mergify/config.yml
diff --git a/.mergify/config.yml b/.mergify/config.yml
new file mode 100644
index 0000000000..86674ce366
--- /dev/null
+++ b/.mergify/config.yml
@@ -0,0 +1,71 @@
+## @file
+# Mergify YML file that automatically merges a GitHub pull request against
+# edk2-ci if all of the GitHub branch protections have passed. It also
+# contains rules to:
+# * auto close branches that are not from an EDK II Maintainer
+# * post a comment on pull requests that have merge conflicts.
+# * post a comment on pull requests that have PatchCheck.py errors.
+#
+# Configuration Notes:
+# * Update the 'base=edk2-ci' statements with the name of the branch to merge
+# pull requests.
+#
+# * Update the 'status-failure' statement with the name of the name of the Azure
+# Pipelines Build that performs the EDK II Maintainer check.
+#
+# * This file must be checked into the 'default' branch of a repo. Copies
+# of this file on other branches of a repo are ignored by Mergify.
+#
+# Copyright (c) 2019, Intel Corporation. All rights reserved.<BR>
+# SPDX-License-Identifier: BSD-2-Clause-Patent
+#
+# https://github.com/apps/mergify
+# https://doc.mergify.io/
+#
+##
+
+pull_request_rules:
+ - name: automatic merge when GitHub branch protection passes and 'push' label is present
+ conditions:
+ - base=master
+ - label=push
+ - author=@tianocore/edk-ii-maintainers
+ - status-success=tianocore.PatchCheck
+ - status-success=Ubuntu GCC5 PR
+ - status-success=Windows VS2019 PR
+ actions:
+ merge:
+ strict: true
+ method: rebase
+ - name: automatic close when GitHub branch protection passes and 'push' label is not present
+ conditions:
+ - base=master
+ - label!=push
+ - status-success=tianocore.PatchCheck
+ - status-success=Ubuntu GCC5 PR
+ - status-success=Windows VS2019 PR
+ actions:
+ close:
+ message: All checks passed. Auto close personal build.
+ - name: post a comment on a PR that can not be merged due to a merge conflict
+ conditions:
+ - base=master
+ - conflict
+ actions:
+ comment:
+ message: PR can not be merged due to conflict. Please rebase and resubmit
+ - name: automatically close a PR that fails the EDK II Maintainers membership check and 'push' label is present
+ conditions:
+ - base=master
+ - label=push
+ - -author=@tianocore/edk-ii-maintainers
+ actions:
+ close:
+ message: PR submitter is not a member of the Tianocore EDK II Maintainers team
+ - name: post a comment on a PR that does not pass PatchCheck.py checks
+ conditions:
+ - base=master
+ - status-failure=tianocore.PatchCheck
+ actions:
+ comment:
+ message: PR can not be merged due to PatchCheck errors. Please resolve and resubmit
--
2.21.0.windows.1
^ permalink raw reply related [flat|nested] 56+ messages in thread* [Patch v3 22/22] Readme.md: Add CI build status badges
2019-10-29 19:54 [Patch v3 00/22] Enable Phase 1 of EDK II CI Michael D Kinney
` (20 preceding siblings ...)
2019-10-29 19:55 ` [Patch v3 21/22] .merify: Add Megify YML pull request rules configuration file Michael D Kinney
@ 2019-10-29 19:55 ` Michael D Kinney
2019-10-30 2:58 ` Leif Lindholm
2019-11-06 0:21 ` [edk2-devel] " Liming Gao
2019-10-31 9:54 ` [edk2-devel] [Patch v3 00/22] Enable Phase 1 of EDK II CI Laszlo Ersek
22 siblings, 2 replies; 56+ messages in thread
From: Michael D Kinney @ 2019-10-29 19:55 UTC (permalink / raw)
To: devel; +Cc: Andrew Fish, Laszlo Ersek, Leif Lindholm
https://bugzilla.tianocore.org/show_bug.cgi?id=2315
Update Readme.md with status badges from the most recent
EDK II Continuous Integration(CI) build.
Cc: Andrew Fish <afish@apple.com>
Cc: Laszlo Ersek <lersek@redhat.com>
Cc: Leif Lindholm <leif.lindholm@linaro.org>
Signed-off-by: Michael D Kinney <michael.d.kinney@intel.com>
---
Readme.md | 29 ++++++++++++++++++++++-------
1 file changed, 22 insertions(+), 7 deletions(-)
diff --git a/Readme.md b/Readme.md
index 27e4ce0771..37183c3265 100644
--- a/Readme.md
+++ b/Readme.md
@@ -3,6 +3,19 @@
A modern, feature-rich, cross-platform firmware development environment
for the UEFI and PI specifications from www.uefi.org.
+## Build Status
+
+| Host Type | Toolchain | Branch | Build Status | Test Status | Code Coverage |
+| :-------- | :-------- | :---- | :----- | :---- | :--- |
+| Windows | VS2019 | master | [](https://dev.azure.com/tianocore/edk2-ci-play/_build/latest?definitionId=14&branchName=master) | [](https://dev.azure.com/tianocore/edk2-ci-play/_build/latest?definitionId=14&branchName=master) | [](https://dev.azure.com/tianocore/edk2-ci-play/_build/latest?definitionId=14&branchName=master)|
+| Ubuntu | GCC5 | master | [](https://dev.azure.com/tianocore/edk2-ci-play/_build/latest?definitionId=15&branchName=master) | [](https://dev.azure.com/tianocore/edk2-ci-play/_build/latest?definitionId=15&branchName=master) | [](https://dev.azure.com/tianocore/edk2-ci-play/_build/latest?definitionId=15&branchName=master) |
+
+[More CI Build information](ci/readme.md)
+
+## License Details
+
+[](License.txt)
+
The majority of the content in the EDK II open source project uses a
[BSD-2-Clause Plus Patent License](License.txt). The EDK II open source project
contains the following components that are covered by additional licenses:
@@ -20,7 +33,8 @@ contains the following components that are covered by additional licenses:
The EDK II Project is composed of packages. The maintainers for each package
are listed in [Maintainers.txt](Maintainers.txt).
-# Resources
+## Resources
+
* [TianoCore](http://www.tianocore.org)
* [EDK II](https://github.com/tianocore/tianocore.github.io/wiki/EDK-II)
* [Getting Started with EDK II](https://github.com/tianocore/tianocore.github.io/wiki/Getting-Started-with-EDK-II)
@@ -29,7 +43,8 @@ are listed in [Maintainers.txt](Maintainers.txt).
* [How To Contribute](https://github.com/tianocore/tianocore.github.io/wiki/How-To-Contribute)
* [Release Planning](https://github.com/tianocore/tianocore.github.io/wiki/EDK-II-Release-Planning)
-# Code Contributions
+## Code Contributions
+
To make a contribution to a TianoCore project, follow these steps.
1. Create a change description in the format specified below to
use in the source control commit log.
@@ -57,7 +72,7 @@ To make a contribution to a TianoCore project, follow these steps.
Contributions using other licenses might be accepted, but further
review will be required.
-# Developer Certificate of Origin
+## Developer Certificate of Origin
Your change description should use the standard format for a
commit message, and must include your `Signed-off-by` signature.
@@ -104,7 +119,7 @@ By making a contribution to this project, I certify that:
this project or the open source license(s) involved.
```
-# Sample Change Description / Commit Message
+## Sample Change Description / Commit Message
```
From: Contributor Name <contributor@example.com>
@@ -115,14 +130,14 @@ Full-commit-message
Signed-off-by: Contributor Name <contributor@example.com>
```
-## Notes for sample patch email
+### Notes for sample patch email
* The first line of commit message is taken from the email's subject
line following `[Repository/Branch PATCH]`. The remaining portion of the
commit message is the email's content.
* `git format-patch` is one way to create this format
-## Definitions for sample patch email
+### Definitions for sample patch email
* `Repository` is the identifier of the repository the patch applies.
This identifier should only be provided for repositories other than
@@ -141,7 +156,7 @@ Signed-off-by: Contributor Name <contributor@example.com>
* `Signed-off-by` is the contributor's signature identifying them
by their real/legal name and their email address.
-# Submodules
+## Submodules
Submodule in EDK II is allowed but submodule chain should be avoided
as possible as we can. Currently EDK II contains two submodules
--
2.21.0.windows.1
^ permalink raw reply related [flat|nested] 56+ messages in thread* Re: [Patch v3 22/22] Readme.md: Add CI build status badges
2019-10-29 19:55 ` [Patch v3 22/22] Readme.md: Add CI build status badges Michael D Kinney
@ 2019-10-30 2:58 ` Leif Lindholm
2019-10-31 4:42 ` Michael D Kinney
2019-11-06 0:21 ` [edk2-devel] " Liming Gao
1 sibling, 1 reply; 56+ messages in thread
From: Leif Lindholm @ 2019-10-30 2:58 UTC (permalink / raw)
To: Michael D Kinney; +Cc: devel, Andrew Fish, Laszlo Ersek
On Tue, Oct 29, 2019 at 12:55:17PM -0700, Michael D Kinney wrote:
> https://bugzilla.tianocore.org/show_bug.cgi?id=2315
>
> Update Readme.md with status badges from the most recent
> EDK II Continuous Integration(CI) build.
>
> Cc: Andrew Fish <afish@apple.com>
> Cc: Laszlo Ersek <lersek@redhat.com>
> Cc: Leif Lindholm <leif.lindholm@linaro.org>
> Signed-off-by: Michael D Kinney <michael.d.kinney@intel.com>
> ---
> Readme.md | 29 ++++++++++++++++++++++-------
> 1 file changed, 22 insertions(+), 7 deletions(-)
>
> diff --git a/Readme.md b/Readme.md
> index 27e4ce0771..37183c3265 100644
> --- a/Readme.md
> +++ b/Readme.md
> @@ -3,6 +3,19 @@
> A modern, feature-rich, cross-platform firmware development environment
> for the UEFI and PI specifications from www.uefi.org.
>
> +## Build Status
> +
> +| Host Type | Toolchain | Branch | Build Status | Test Status | Code Coverage |
This may sound like nitpicking, but I think it's important for
messaging: we aren't listing tolchains, we are listing toolchain
profiles. For VS, we are actually adding new profiles for each
version, so that statement is not misleading. However, GCC5 is highly
misleading (it is the profile we use for anything gcc5 onwards).
I don't think we can usefully pull in information about the toolchain
actually used, so I think the name of the column should really be
toolchain profile.
Even given that, I think it is time we create an abstracted
GCC_CURRENT toolchain profile, pointing to GCC5. But if we rename the
column, that may not be a prerequisite for this set.
> +| :-------- | :-------- | :---- | :----- | :---- | :--- |
> +| Windows | VS2019 | master | [](https://dev.azure.com/tianocore/edk2-ci-play/_build/latest?definitionId=14&branchName=master) | [](https://dev.azure.com/tianocore/edk2-ci-play/_build/latest?definitionId=14&branchName=master) | [](https://dev.azure.com/tianocore/edk2-ci-play/_build/latest?definitionId=14&branchName=master)|
> +| Ubuntu | GCC5 | master | [](https://dev.azure.com/tianocore/edk2-ci-play/_build/latest?definitionId=15&branchName=master) | [](https://dev.azure.com/tianocore/edk2-ci-play/_build/latest?definitionId=15&branchName=master) | [](https://dev.azure.com/tianocore/edk2-ci-play/_build/latest?definitionId=15&branchName=master) |
I have one problem with the above, which is that it turns an otherwise
very readable in text mode file (which is half the point of .md) into
what looks a little bit like a transmission error.
While I agree there is value in having this on the default loaded
github webpage, do we have other options?
> +
> +[More CI Build information](ci/readme.md)
I don't see this file in the tree?
Which is good, beacuse it doesn't follow name standards.
> +
> +## License Details
> +
> +[](License.txt)
> +
I don't mind this addition, but I think it should be a separate patch.
I also think it would be best to hold this back until we've submoduled
some of those otherwise licensed components.
There is a very real likelihood of people skimming, seeing the image
and ignoring the text.
> The majority of the content in the EDK II open source project uses a
> [BSD-2-Clause Plus Patent License](License.txt). The EDK II open source project
> contains the following components that are covered by additional licenses:
> @@ -20,7 +33,8 @@ contains the following components that are covered by additional licenses:
> The EDK II Project is composed of packages. The maintainers for each package
> are listed in [Maintainers.txt](Maintainers.txt).
>
> -# Resources
> +## Resources
> +
No objections to change, but it is unrelated to this patch.
> * [TianoCore](http://www.tianocore.org)
> * [EDK II](https://github.com/tianocore/tianocore.github.io/wiki/EDK-II)
> * [Getting Started with EDK II](https://github.com/tianocore/tianocore.github.io/wiki/Getting-Started-with-EDK-II)
> @@ -29,7 +43,8 @@ are listed in [Maintainers.txt](Maintainers.txt).
> * [How To Contribute](https://github.com/tianocore/tianocore.github.io/wiki/How-To-Contribute)
> * [Release Planning](https://github.com/tianocore/tianocore.github.io/wiki/EDK-II-Release-Planning)
>
> -# Code Contributions
> +## Code Contributions
> +
No objection to change, but it is unrelated to this patch.
> To make a contribution to a TianoCore project, follow these steps.
> 1. Create a change description in the format specified below to
> use in the source control commit log.
> @@ -57,7 +72,7 @@ To make a contribution to a TianoCore project, follow these steps.
> Contributions using other licenses might be accepted, but further
> review will be required.
>
> -# Developer Certificate of Origin
> +## Developer Certificate of Origin
No objection to change, but it is unrelated to this patch.
>
> Your change description should use the standard format for a
> commit message, and must include your `Signed-off-by` signature.
> @@ -104,7 +119,7 @@ By making a contribution to this project, I certify that:
> this project or the open source license(s) involved.
> ```
>
> -# Sample Change Description / Commit Message
> +## Sample Change Description / Commit Message
No objection to change, but it is unrelated to this patch.
>
> ```
> From: Contributor Name <contributor@example.com>
> @@ -115,14 +130,14 @@ Full-commit-message
> Signed-off-by: Contributor Name <contributor@example.com>
> ```
>
> -## Notes for sample patch email
> +### Notes for sample patch email
No objection to change, but it is unrelated to this patch.
>
> * The first line of commit message is taken from the email's subject
> line following `[Repository/Branch PATCH]`. The remaining portion of the
> commit message is the email's content.
> * `git format-patch` is one way to create this format
>
> -## Definitions for sample patch email
> +### Definitions for sample patch email
No objection to change, but it is unrelated to this patch.
>
> * `Repository` is the identifier of the repository the patch applies.
> This identifier should only be provided for repositories other than
> @@ -141,7 +156,7 @@ Signed-off-by: Contributor Name <contributor@example.com>
> * `Signed-off-by` is the contributor's signature identifying them
> by their real/legal name and their email address.
>
> -# Submodules
> +## Submodules
No objection to change, but it is unrelated to this patch.
/
Leif
>
> Submodule in EDK II is allowed but submodule chain should be avoided
> as possible as we can. Currently EDK II contains two submodules
> --
> 2.21.0.windows.1
>
^ permalink raw reply [flat|nested] 56+ messages in thread* Re: [Patch v3 22/22] Readme.md: Add CI build status badges
2019-10-30 2:58 ` Leif Lindholm
@ 2019-10-31 4:42 ` Michael D Kinney
0 siblings, 0 replies; 56+ messages in thread
From: Michael D Kinney @ 2019-10-31 4:42 UTC (permalink / raw)
To: Leif Lindholm, Kinney, Michael D
Cc: devel@edk2.groups.io, Andrew Fish, Laszlo Ersek
Hi Leif,
Thanks for noticing the reference to the ci dir. That content
Has been moved to .azurepipelines directory and the Readme.md
filename is correct there.
I will split out the format changes to this Readme.md into its
own patch so it will not be mixed with the CI features.
I agree that the table with CI status is not very readable.
This is true in MD files for most long hyperlinks, especially
when they are put into a table format. We can see if we can
improve the readability to make it easier to maintain, but
will likely still be long lines.
Mike
> -----Original Message-----
> From: Leif Lindholm <leif.lindholm@linaro.org>
> Sent: Tuesday, October 29, 2019 7:58 PM
> To: Kinney, Michael D <michael.d.kinney@intel.com>
> Cc: devel@edk2.groups.io; Andrew Fish <afish@apple.com>;
> Laszlo Ersek <lersek@redhat.com>
> Subject: Re: [Patch v3 22/22] Readme.md: Add CI build
> status badges
>
> On Tue, Oct 29, 2019 at 12:55:17PM -0700, Michael D
> Kinney wrote:
> > https://bugzilla.tianocore.org/show_bug.cgi?id=2315
> >
> > Update Readme.md with status badges from the most
> recent EDK II
> > Continuous Integration(CI) build.
> >
> > Cc: Andrew Fish <afish@apple.com>
> > Cc: Laszlo Ersek <lersek@redhat.com>
> > Cc: Leif Lindholm <leif.lindholm@linaro.org>
> > Signed-off-by: Michael D Kinney
> <michael.d.kinney@intel.com>
> > ---
> > Readme.md | 29 ++++++++++++++++++++++-------
> > 1 file changed, 22 insertions(+), 7 deletions(-)
> >
> > diff --git a/Readme.md b/Readme.md
> > index 27e4ce0771..37183c3265 100644
> > --- a/Readme.md
> > +++ b/Readme.md
> > @@ -3,6 +3,19 @@
> > A modern, feature-rich, cross-platform firmware
> development
> > environment for the UEFI and PI specifications from
> www.uefi.org.
> >
> > +## Build Status
> > +
> > +| Host Type | Toolchain | Branch | Build Status |
> Test Status | Code
> > +| Coverage |
>
> This may sound like nitpicking, but I think it's
> important for
> messaging: we aren't listing tolchains, we are listing
> toolchain profiles. For VS, we are actually adding new
> profiles for each version, so that statement is not
> misleading. However, GCC5 is highly misleading (it is
> the profile we use for anything gcc5 onwards).
>
> I don't think we can usefully pull in information about
> the toolchain actually used, so I think the name of the
> column should really be toolchain profile.
>
> Even given that, I think it is time we create an
> abstracted GCC_CURRENT toolchain profile, pointing to
> GCC5. But if we rename the column, that may not be a
> prerequisite for this set.
>
> > +| :-------- | :-------- | :---- | :----- | :---- | :-
> -- | Windows |
> > +| VS2019 | master | [](ht
> tps://dev.a
> > +| zure.com/tianocore/edk2-ci-
> play/_build/latest?definitionId=14&branc
> > +| hName=master) | [](https://dev.azure.com/tianocore/edk2-
> ci-play/_build/l
> > +| atest?definitionId=14&branchName=master) | [](
> > +| https://dev.azure.com/tianocore/edk2-ci-
> play/_build/latest?definiti
> > +| onId=14&branchName=master)| Ubuntu | GCC5 | master
> | [](https
> ://dev.azur
> > +| e.com/tianocore/edk2-ci-
> play/_build/latest?definitionId=15&branchNa
> > +| me=master) | [](https://dev.azure.com/tianocore/edk2-
> ci-play/_build/l
> > +| atest?definitionId=15&branchName=master) | [](
> > +| https://dev.azure.com/tianocore/edk2-ci-
> play/_build/latest?definiti
> > +| onId=15&branchName=master) |
>
> I have one problem with the above, which is that it
> turns an otherwise very readable in text mode file
> (which is half the point of .md) into what looks a
> little bit like a transmission error.
>
> While I agree there is value in having this on the
> default loaded github webpage, do we have other options?
>
> > +
> > +[More CI Build information](ci/readme.md)
>
> I don't see this file in the tree?
> Which is good, beacuse it doesn't follow name standards.
>
> > +
> > +## License Details
> > +
> > +[](License.txt)
> > +
>
> I don't mind this addition, but I think it should be a
> separate patch.
> I also think it would be best to hold this back until
> we've submoduled some of those otherwise licensed
> components.
> There is a very real likelihood of people skimming,
> seeing the image and ignoring the text.
>
> > The majority of the content in the EDK II open source
> project uses a
> > [BSD-2-Clause Plus Patent License](License.txt). The
> EDK II open
> > source project contains the following components that
> are covered by additional licenses:
> > @@ -20,7 +33,8 @@ contains the following components
> that are covered by additional licenses:
> > The EDK II Project is composed of packages. The
> maintainers for each
> > package are listed in
> [Maintainers.txt](Maintainers.txt).
> >
> > -# Resources
> > +## Resources
> > +
>
> No objections to change, but it is unrelated to this
> patch.
>
> > * [TianoCore](http://www.tianocore.org)
> > * [EDK
> >
> II](https://github.com/tianocore/tianocore.github.io/wik
> i/EDK-II)
> > * [Getting Started with EDK
> >
> II](https://github.com/tianocore/tianocore.github.io/wik
> i/Getting-Star
> > ted-with-EDK-II) @@ -29,7 +43,8 @@ are listed in
> > [Maintainers.txt](Maintainers.txt).
> > * [How To
> >
> Contribute](https://github.com/tianocore/tianocore.githu
> b.io/wiki/How-
> > To-Contribute)
> > * [Release
> >
> Planning](https://github.com/tianocore/tianocore.github.
> io/wiki/EDK-II
> > -Release-Planning)
> >
> > -# Code Contributions
> > +## Code Contributions
> > +
>
> No objection to change, but it is unrelated to this
> patch.
>
> > To make a contribution to a TianoCore project, follow
> these steps.
> > 1. Create a change description in the format
> specified below to
> > use in the source control commit log.
> > @@ -57,7 +72,7 @@ To make a contribution to a
> TianoCore project, follow these steps.
> > Contributions using other licenses might be
> accepted, but further
> > review will be required.
> >
> > -# Developer Certificate of Origin
> > +## Developer Certificate of Origin
>
> No objection to change, but it is unrelated to this
> patch.
>
> >
> > Your change description should use the standard
> format for a commit
> > message, and must include your `Signed-off-by`
> signature.
> > @@ -104,7 +119,7 @@ By making a contribution to this
> project, I certify that:
> > this project or the open source license(s)
> involved.
> > ```
> >
> > -# Sample Change Description / Commit Message
> > +## Sample Change Description / Commit Message
>
> No objection to change, but it is unrelated to this
> patch.
>
> >
> > ```
> > From: Contributor Name <contributor@example.com> @@ -
> 115,14 +130,14
> > @@ Full-commit-message
> > Signed-off-by: Contributor Name
> <contributor@example.com> ```
> >
> > -## Notes for sample patch email
> > +### Notes for sample patch email
>
> No objection to change, but it is unrelated to this
> patch.
>
> >
> > * The first line of commit message is taken from the
> email's subject
> > line following `[Repository/Branch PATCH]`. The
> remaining portion of the
> > commit message is the email's content.
> > * `git format-patch` is one way to create this format
> >
> > -## Definitions for sample patch email
> > +### Definitions for sample patch email
>
> No objection to change, but it is unrelated to this
> patch.
>
> >
> > * `Repository` is the identifier of the repository
> the patch applies.
> > This identifier should only be provided for
> repositories other than
> > @@ -141,7 +156,7 @@ Signed-off-by: Contributor Name
> > <contributor@example.com>
> > * `Signed-off-by` is the contributor's signature
> identifying them
> > by their real/legal name and their email address.
> >
> > -# Submodules
> > +## Submodules
>
> No objection to change, but it is unrelated to this
> patch.
>
> /
> Leif
>
> >
> > Submodule in EDK II is allowed but submodule chain
> should be avoided
> > as possible as we can. Currently EDK II contains two
> submodules
> > --
> > 2.21.0.windows.1
> >
^ permalink raw reply [flat|nested] 56+ messages in thread
* Re: [edk2-devel] [Patch v3 22/22] Readme.md: Add CI build status badges
2019-10-29 19:55 ` [Patch v3 22/22] Readme.md: Add CI build status badges Michael D Kinney
2019-10-30 2:58 ` Leif Lindholm
@ 2019-11-06 0:21 ` Liming Gao
2019-11-06 0:34 ` Michael D Kinney
1 sibling, 1 reply; 56+ messages in thread
From: Liming Gao @ 2019-11-06 0:21 UTC (permalink / raw)
To: devel@edk2.groups.io, Kinney, Michael D
Cc: Andrew Fish, Laszlo Ersek, Leif Lindholm
Mike:
How to know Ubuntu OS version?
Thanks
Liming
>-----Original Message-----
>From: devel@edk2.groups.io [mailto:devel@edk2.groups.io] On Behalf Of
>Michael D Kinney
>Sent: Wednesday, October 30, 2019 3:55 AM
>To: devel@edk2.groups.io
>Cc: Andrew Fish <afish@apple.com>; Laszlo Ersek <lersek@redhat.com>; Leif
>Lindholm <leif.lindholm@linaro.org>
>Subject: [edk2-devel] [Patch v3 22/22] Readme.md: Add CI build status
>badges
>
>https://bugzilla.tianocore.org/show_bug.cgi?id=2315
>
>Update Readme.md with status badges from the most recent
>EDK II Continuous Integration(CI) build.
>
>Cc: Andrew Fish <afish@apple.com>
>Cc: Laszlo Ersek <lersek@redhat.com>
>Cc: Leif Lindholm <leif.lindholm@linaro.org>
>Signed-off-by: Michael D Kinney <michael.d.kinney@intel.com>
>---
> Readme.md | 29 ++++++++++++++++++++++-------
> 1 file changed, 22 insertions(+), 7 deletions(-)
>
>diff --git a/Readme.md b/Readme.md
>index 27e4ce0771..37183c3265 100644
>--- a/Readme.md
>+++ b/Readme.md
>@@ -3,6 +3,19 @@
> A modern, feature-rich, cross-platform firmware development environment
> for the UEFI and PI specifications from www.uefi.org.
>
>+## Build Status
>+
>+| Host Type | Toolchain | Branch | Build Status | Test Status | Code Coverage
>|
>+| :-------- | :-------- | :---- | :----- | :---- | :--- |
>+| Windows | VS2019 | master | [](https://dev.azure.com/tianocore/edk2-ci-
>play/_build/latest?definitionId=14&branchName=master) | [](https://dev.azure.com/tianocore/edk2-ci-
>play/_build/latest?definitionId=14&branchName=master) | [](https://dev.azure.com/tianocore/edk2-ci-
>play/_build/latest?definitionId=14&branchName=master)|
>+| Ubuntu | GCC5 | master | [](https://dev.azure.com/tianocore/edk2-ci-
>play/_build/latest?definitionId=15&branchName=master) | [](https://dev.azure.com/tianocore/edk2-ci-
>play/_build/latest?definitionId=15&branchName=master) | [](https://dev.azure.com/tianocore/edk2-ci-
>play/_build/latest?definitionId=15&branchName=master) |
>+
>+[More CI Build information](ci/readme.md)
>+
>+## License Details
>+
>+[](License.txt)
>+
> The majority of the content in the EDK II open source project uses a
> [BSD-2-Clause Plus Patent License](License.txt). The EDK II open source
>project
> contains the following components that are covered by additional licenses:
>@@ -20,7 +33,8 @@ contains the following components that are covered by
>additional licenses:
> The EDK II Project is composed of packages. The maintainers for each
>package
> are listed in [Maintainers.txt](Maintainers.txt).
>
>-# Resources
>+## Resources
>+
> * [TianoCore](http://www.tianocore.org)
> * [EDK II](https://github.com/tianocore/tianocore.github.io/wiki/EDK-II)
> * [Getting Started with EDK
>II](https://github.com/tianocore/tianocore.github.io/wiki/Getting-Started-
>with-EDK-II)
>@@ -29,7 +43,8 @@ are listed in [Maintainers.txt](Maintainers.txt).
> * [How To
>Contribute](https://github.com/tianocore/tianocore.github.io/wiki/How-To-
>Contribute)
> * [Release
>Planning](https://github.com/tianocore/tianocore.github.io/wiki/EDK-II-
>Release-Planning)
>
>-# Code Contributions
>+## Code Contributions
>+
> To make a contribution to a TianoCore project, follow these steps.
> 1. Create a change description in the format specified below to
> use in the source control commit log.
>@@ -57,7 +72,7 @@ To make a contribution to a TianoCore project, follow
>these steps.
> Contributions using other licenses might be accepted, but further
> review will be required.
>
>-# Developer Certificate of Origin
>+## Developer Certificate of Origin
>
> Your change description should use the standard format for a
> commit message, and must include your `Signed-off-by` signature.
>@@ -104,7 +119,7 @@ By making a contribution to this project, I certify that:
> this project or the open source license(s) involved.
> ```
>
>-# Sample Change Description / Commit Message
>+## Sample Change Description / Commit Message
>
> ```
> From: Contributor Name <contributor@example.com>
>@@ -115,14 +130,14 @@ Full-commit-message
> Signed-off-by: Contributor Name <contributor@example.com>
> ```
>
>-## Notes for sample patch email
>+### Notes for sample patch email
>
> * The first line of commit message is taken from the email's subject
> line following `[Repository/Branch PATCH]`. The remaining portion of the
> commit message is the email's content.
> * `git format-patch` is one way to create this format
>
>-## Definitions for sample patch email
>+### Definitions for sample patch email
>
> * `Repository` is the identifier of the repository the patch applies.
> This identifier should only be provided for repositories other than
>@@ -141,7 +156,7 @@ Signed-off-by: Contributor Name
><contributor@example.com>
> * `Signed-off-by` is the contributor's signature identifying them
> by their real/legal name and their email address.
>
>-# Submodules
>+## Submodules
>
> Submodule in EDK II is allowed but submodule chain should be avoided
> as possible as we can. Currently EDK II contains two submodules
>--
>2.21.0.windows.1
>
>
>
^ permalink raw reply [flat|nested] 56+ messages in thread
* Re: [edk2-devel] [Patch v3 22/22] Readme.md: Add CI build status badges
2019-11-06 0:21 ` [edk2-devel] " Liming Gao
@ 2019-11-06 0:34 ` Michael D Kinney
0 siblings, 0 replies; 56+ messages in thread
From: Michael D Kinney @ 2019-11-06 0:34 UTC (permalink / raw)
To: Gao, Liming, devel@edk2.groups.io, Kinney, Michael D
Cc: Andrew Fish, Laszlo Ersek, Leif Lindholm
Liming,
The Azure Pipelines config is
vm_image: 'ubuntu-latest'
The documentation on available vm_image settings is here:
https://docs.microsoft.com/en-us/azure/devops/pipelines/agents/hosted?view=azure-devops
Currently ubuntu-latest is same as ubuntu-18.04.
Mike
> -----Original Message-----
> From: Gao, Liming <liming.gao@intel.com>
> Sent: Tuesday, November 5, 2019 4:21 PM
> To: devel@edk2.groups.io; Kinney, Michael D
> <michael.d.kinney@intel.com>
> Cc: Andrew Fish <afish@apple.com>; Laszlo Ersek
> <lersek@redhat.com>; Leif Lindholm
> <leif.lindholm@linaro.org>
> Subject: RE: [edk2-devel] [Patch v3 22/22] Readme.md:
> Add CI build status badges
>
> Mike:
> How to know Ubuntu OS version?
>
> Thanks
> Liming
> >-----Original Message-----
> >From: devel@edk2.groups.io
> [mailto:devel@edk2.groups.io] On Behalf Of
> >Michael D Kinney
> >Sent: Wednesday, October 30, 2019 3:55 AM
> >To: devel@edk2.groups.io
> >Cc: Andrew Fish <afish@apple.com>; Laszlo Ersek
> <lersek@redhat.com>;
> >Leif Lindholm <leif.lindholm@linaro.org>
> >Subject: [edk2-devel] [Patch v3 22/22] Readme.md: Add
> CI build status
> >badges
> >
> >https://bugzilla.tianocore.org/show_bug.cgi?id=2315
> >
> >Update Readme.md with status badges from the most
> recent EDK II
> >Continuous Integration(CI) build.
> >
> >Cc: Andrew Fish <afish@apple.com>
> >Cc: Laszlo Ersek <lersek@redhat.com>
> >Cc: Leif Lindholm <leif.lindholm@linaro.org>
> >Signed-off-by: Michael D Kinney
> <michael.d.kinney@intel.com>
> >---
> > Readme.md | 29 ++++++++++++++++++++++-------
> > 1 file changed, 22 insertions(+), 7 deletions(-)
> >
> >diff --git a/Readme.md b/Readme.md
> >index 27e4ce0771..37183c3265 100644
> >--- a/Readme.md
> >+++ b/Readme.md
> >@@ -3,6 +3,19 @@
> > A modern, feature-rich, cross-platform firmware
> development
> >environment for the UEFI and PI specifications from
> www.uefi.org.
> >
> >+## Build Status
> >+
> >+| Host Type | Toolchain | Branch | Build Status | Test
> Status | Code
> >+| Coverage
> >|
> >+| :-------- | :-------- | :---- | :----- | :---- | :--
> - | Windows |
> >+| VS2019 | master | [](https://dev.azure.com/tianocore/edk2-ci-
> >play/_build/latest?definitionId=14&branchName=master) |
> [](https://dev.azure.com/tianocore/edk2-ci-
> >play/_build/latest?definitionId=14&branchName=master) |
> [](https://dev.azure.com/tianocore/edk2-ci-
> >play/_build/latest?definitionId=14&branchName=master)|
> >+| Ubuntu | GCC5 | master | [](https://dev.azure.com/tianocore/edk2-ci-
> >play/_build/latest?definitionId=15&branchName=master) |
> [](https://dev.azure.com/tianocore/edk2-ci-
> >play/_build/latest?definitionId=15&branchName=master)
> | [](https://dev.azure.com/tianocore/edk2-ci-
> >play/_build/latest?definitionId=15&branchName=master) |
> >+
> >+[More CI Build information](ci/readme.md)
> >+
> >+## License Details
> >+
> >+[](License.txt)
> >+
> > The majority of the content in the EDK II open source
> project uses a
> >[BSD-2-Clause Plus Patent License](License.txt). The
> EDK II open
> >source project contains the following components that
> are covered by
> >additional licenses:
> >@@ -20,7 +33,8 @@ contains the following components
> that are covered by
> >additional licenses:
> > The EDK II Project is composed of packages. The
> maintainers for each
> >package are listed in
> [Maintainers.txt](Maintainers.txt).
> >
> >-# Resources
> >+## Resources
> >+
> > * [TianoCore](http://www.tianocore.org)
> > * [EDK
> >II](https://github.com/tianocore/tianocore.github.io/wi
> ki/EDK-II)
> > * [Getting Started with EDK
> >II](https://github.com/tianocore/tianocore.github.io/wi
> ki/Getting-Start
> >ed-
> >with-EDK-II)
> >@@ -29,7 +43,8 @@ are listed in
> [Maintainers.txt](Maintainers.txt).
> > * [How To
> >Contribute](https://github.com/tianocore/tianocore.gith
> ub.io/wiki/How-T
> >o-
> >Contribute)
> > * [Release
> >Planning](https://github.com/tianocore/tianocore.github
> .io/wiki/EDK-II-
> >Release-Planning)
> >
> >-# Code Contributions
> >+## Code Contributions
> >+
> > To make a contribution to a TianoCore project, follow
> these steps.
> > 1. Create a change description in the format specified
> below to
> > use in the source control commit log.
> >@@ -57,7 +72,7 @@ To make a contribution to a TianoCore
> project, follow
> >these steps.
> > Contributions using other licenses might be
> accepted, but further
> > review will be required.
> >
> >-# Developer Certificate of Origin
> >+## Developer Certificate of Origin
> >
> > Your change description should use the standard format
> for a commit
> >message, and must include your `Signed-off-by`
> signature.
> >@@ -104,7 +119,7 @@ By making a contribution to this
> project, I certify that:
> > this project or the open source license(s)
> involved.
> > ```
> >
> >-# Sample Change Description / Commit Message
> >+## Sample Change Description / Commit Message
> >
> > ```
> > From: Contributor Name <contributor@example.com> @@ -
> 115,14 +130,14 @@
> >Full-commit-message
> > Signed-off-by: Contributor Name
> <contributor@example.com> ```
> >
> >-## Notes for sample patch email
> >+### Notes for sample patch email
> >
> > * The first line of commit message is taken from the
> email's subject
> > line following `[Repository/Branch PATCH]`. The
> remaining portion of the
> > commit message is the email's content.
> > * `git format-patch` is one way to create this format
> >
> >-## Definitions for sample patch email
> >+### Definitions for sample patch email
> >
> > * `Repository` is the identifier of the repository the
> patch applies.
> > This identifier should only be provided for
> repositories other than
> >@@ -141,7 +156,7 @@ Signed-off-by: Contributor Name
> ><contributor@example.com>
> > * `Signed-off-by` is the contributor's signature
> identifying them
> > by their real/legal name and their email address.
> >
> >-# Submodules
> >+## Submodules
> >
> > Submodule in EDK II is allowed but submodule chain
> should be avoided
> >as possible as we can. Currently EDK II contains two
> submodules
> >--
> >2.21.0.windows.1
> >
> >
> >
^ permalink raw reply [flat|nested] 56+ messages in thread
* Re: [edk2-devel] [Patch v3 00/22] Enable Phase 1 of EDK II CI
2019-10-29 19:54 [Patch v3 00/22] Enable Phase 1 of EDK II CI Michael D Kinney
` (21 preceding siblings ...)
2019-10-29 19:55 ` [Patch v3 22/22] Readme.md: Add CI build status badges Michael D Kinney
@ 2019-10-31 9:54 ` Laszlo Ersek
2019-11-01 5:03 ` Michael D Kinney
22 siblings, 1 reply; 56+ messages in thread
From: Laszlo Ersek @ 2019-10-31 9:54 UTC (permalink / raw)
To: devel, michael.d.kinney
Cc: Sean Brogan, Bret Barkelew, Liming Gao, Bob Feng, Andrew Fish,
Leif Lindholm, Jian J Wang, Xiaoyu Lu, Ray Ni, Hao A Wu,
Jiaxin Wu, Siyuan Fu, Jiewen Yao, Chao Zhang, Zhichao Gao,
Eric Dong
On 10/29/19 20:54, Michael D Kinney wrote:
> Active branch for testing/evaluation:
> * https://github.com/tianocore/edk2-staging/tree/edk2-ci
> * To test, fork edk2-staging repo, create a branch with a change, and submit
> a pull request targeting edk2-staging/edk2-ci. NOTE: the default branch for
> the edk2-staging is 'about'. You must select the 'edk2-ci' branch when
> a pull request is opened. Set the 'push' label to require commit if all
> checks pass.
The edk2-staging repository has been added as a "git remote" to my local
edk2 clone for a long while now. Using the local identifier "staging".
(This makes perfect sense as edk2-staging is itself a fork of edk2, with
branches that are supposed to be rebased to edk2/master periodically.)
Furthermore, the identifier by which I refer to the remote at
<https://github.com/lersek/edk2.git> is "lersek".
I've now run the following commands:
$ git fetch staging
$ git checkout -b ci-test-1 staging/edk2-ci
[modify "SampleFile.txt"]
$ git add -p
$ git commit
$ git push lersek ci-test-1
Questions:
(a) How can I submit a pull request for the staging repo's edk2-ci
branch using the command line (and set the "push" label)?
(b) How can I submit a PR for the staging repo's edk2-ci branch
(regardless of command line vs. WebUI usage) against my
<https://github.com/lersek/edk2.git> repository?
When I go to the WebUI, the PR view does not offer
"tianocore/edk2-staging" as "base repository", it only offers
"tianocore/edk2". I thought I'd be able to pick any destination
repository at all.
By the instruction "fork edk2-staging repo", did you mean we should fork
edk2-staging *on github*? (Using the WebUI?)
Thanks!
Laszlo
^ permalink raw reply [flat|nested] 56+ messages in thread* Re: [edk2-devel] [Patch v3 00/22] Enable Phase 1 of EDK II CI
2019-10-31 9:54 ` [edk2-devel] [Patch v3 00/22] Enable Phase 1 of EDK II CI Laszlo Ersek
@ 2019-11-01 5:03 ` Michael D Kinney
2019-11-01 22:39 ` Laszlo Ersek
0 siblings, 1 reply; 56+ messages in thread
From: Michael D Kinney @ 2019-11-01 5:03 UTC (permalink / raw)
To: Laszlo Ersek, devel@edk2.groups.io, Kinney, Michael D
Cc: Sean Brogan, Bret Barkelew, Gao, Liming, Feng, Bob C, Andrew Fish,
Leif Lindholm, Wang, Jian J, Lu, XiaoyuX, Ni, Ray, Wu, Hao A,
Wu, Jiaxin, Fu, Siyuan, Yao, Jiewen, Zhang, Chao B, Gao, Zhichao,
Dong, Eric
Hi Laszlo,
===================================================
Note for all reviewers:
---------------------------------------------------
Pull requests against edk2-staging/edk2-ci are not
being processed right now. We are working on some
configuration changes after noticing that all the
checks were not being shown on the Web UI. We will
let you know when it is back up.
===================================================
Yes. You need to create a fork of the tianocore/edk2-staging repo.
You can do this with the WebUI or the hub command line utility.
* https://github.com/github/hub
* https://github.com/github/hub/releases
* https://hub.github.com/hub.1.html
The hub command also supports creating a pull request.
I have used it extensively to write some unit tests
for edk2-ci this week.
Once you create a branch with changes to submit in your
own fork of tianocore/edk2-staging the WebUI will show
that this pull request is possible and guide you through
it.
The use of edk2-staging/edk2-ci is only for the review
and unit testing. Once the review is approved, it will
be enabled on edk2/master and you will be able to use
your own fork of edk2 to make branches and submit pull
requests.
Mike
> -----Original Message-----
> From: Laszlo Ersek <lersek@redhat.com>
> Sent: Thursday, October 31, 2019 2:55 AM
> To: devel@edk2.groups.io; Kinney, Michael D
> <michael.d.kinney@intel.com>
> Cc: Sean Brogan <sean.brogan@microsoft.com>; Bret
> Barkelew <Bret.Barkelew@microsoft.com>; Gao, Liming
> <liming.gao@intel.com>; Feng, Bob C
> <bob.c.feng@intel.com>; Andrew Fish <afish@apple.com>;
> Leif Lindholm <leif.lindholm@linaro.org>; Wang, Jian J
> <jian.j.wang@intel.com>; Lu, XiaoyuX
> <xiaoyux.lu@intel.com>; Ni, Ray <ray.ni@intel.com>; Wu,
> Hao A <hao.a.wu@intel.com>; Wu, Jiaxin
> <jiaxin.wu@intel.com>; Fu, Siyuan <siyuan.fu@intel.com>;
> Yao, Jiewen <jiewen.yao@intel.com>; Zhang, Chao B
> <chao.b.zhang@intel.com>; Gao, Zhichao
> <zhichao.gao@intel.com>; Dong, Eric
> <eric.dong@intel.com>
> Subject: Re: [edk2-devel] [Patch v3 00/22] Enable Phase
> 1 of EDK II CI
>
> On 10/29/19 20:54, Michael D Kinney wrote:
>
> > Active branch for testing/evaluation:
> > * https://github.com/tianocore/edk2-staging/tree/edk2-
> ci
> > * To test, fork edk2-staging repo, create a branch
> with a change, and submit
> > a pull request targeting edk2-staging/edk2-ci.
> NOTE: the default branch for
> > the edk2-staging is 'about'. You must select the
> 'edk2-ci' branch when
> > a pull request is opened. Set the 'push' label to
> require commit if all
> > checks pass.
>
> The edk2-staging repository has been added as a "git
> remote" to my local
> edk2 clone for a long while now. Using the local
> identifier "staging".
> (This makes perfect sense as edk2-staging is itself a
> fork of edk2, with branches that are supposed to be
> rebased to edk2/master periodically.)
>
> Furthermore, the identifier by which I refer to the
> remote at <https://github.com/lersek/edk2.git> is
> "lersek".
>
> I've now run the following commands:
>
> $ git fetch staging
> $ git checkout -b ci-test-1 staging/edk2-ci
>
> [modify "SampleFile.txt"]
>
> $ git add -p
> $ git commit
> $ git push lersek ci-test-1
>
> Questions:
>
> (a) How can I submit a pull request for the staging
> repo's edk2-ci branch using the command line (and set
> the "push" label)?
>
> (b) How can I submit a PR for the staging repo's edk2-ci
> branch (regardless of command line vs. WebUI usage)
> against my <https://github.com/lersek/edk2.git>
> repository?
>
> When I go to the WebUI, the PR view does not offer
> "tianocore/edk2-staging" as "base repository", it only
> offers "tianocore/edk2". I thought I'd be able to pick
> any destination repository at all.
>
> By the instruction "fork edk2-staging repo", did you
> mean we should fork edk2-staging *on github*? (Using the
> WebUI?)
>
> Thanks!
> Laszlo
^ permalink raw reply [flat|nested] 56+ messages in thread
* Re: [edk2-devel] [Patch v3 00/22] Enable Phase 1 of EDK II CI
2019-11-01 5:03 ` Michael D Kinney
@ 2019-11-01 22:39 ` Laszlo Ersek
2019-11-01 23:22 ` Laszlo Ersek
2019-11-02 0:32 ` Michael D Kinney
0 siblings, 2 replies; 56+ messages in thread
From: Laszlo Ersek @ 2019-11-01 22:39 UTC (permalink / raw)
To: Kinney, Michael D, devel@edk2.groups.io
Cc: Sean Brogan, Bret Barkelew, Gao, Liming, Feng, Bob C, Andrew Fish,
Leif Lindholm, Wang, Jian J, Lu, XiaoyuX, Ni, Ray, Wu, Hao A,
Wu, Jiaxin, Fu, Siyuan, Yao, Jiewen, Zhang, Chao B, Gao, Zhichao,
Dong, Eric
Hi Mike,
On 11/01/19 06:03, Kinney, Michael D wrote:
> Hi Laszlo,
>
> ===================================================
> Note for all reviewers:
> ---------------------------------------------------
> Pull requests against edk2-staging/edk2-ci are not
> being processed right now. We are working on some
> configuration changes after noticing that all the
> checks were not being shown on the Web UI. We will
> let you know when it is back up.
> ===================================================
>
> Yes. You need to create a fork of the tianocore/edk2-staging repo.
>
> You can do this with the WebUI or the hub command line utility.
>
> * https://github.com/github/hub
> * https://github.com/github/hub/releases
> * https://hub.github.com/hub.1.html
>
> The hub command also supports creating a pull request.
> I have used it extensively to write some unit tests
> for edk2-ci this week.
>
> Once you create a branch with changes to submit in your
> own fork of tianocore/edk2-staging the WebUI will show
> that this pull request is possible and guide you through
> it.
>
> The use of edk2-staging/edk2-ci is only for the review
> and unit testing. Once the review is approved, it will
> be enabled on edk2/master and you will be able to use
> your own fork of edk2 to make branches and submit pull
> requests.
* For now, I submitted a pull request using the WebUi.
https://github.com/tianocore/edk2-staging/pull/87
I can see the processing is paused at the moment, as you state above.
* When I submitted PR#87, there had been no conflicts; the PR was simply
blocked on the CI tasks. Now, there are conflicts (I assume due to
intervening pushes, while my PR#87 was blocked). It's useful that the
WebUI points out this change in the status of the PR (i.e., "now
conflicting" vs. "no conflicts just waiting for CI"). However, this
status change has not been emailed to me.
When this occurs to a PR (for example due to an intervening PR that is
merged), what happens to the pre-empted PR? Does it remain suspended
forever? How does the submitter learn about it?
* Regarding my question (b) below, the answer is "I can't". I cannot
submit a PR against tianocore/edk2-staging.git with reference to
lersek/edk2.git, given that the latter is a not a "github-level fork" of
the former.
This is quite regrettable, but once the CI system is in production, it
will be OK in practice (I will only submit PRs against tianocore/edk2.git).
* I've explored the "hub" command a bit. (This relates to my question
(a); thanks for your suggestion.)
On the positive side, it is packaged in EPEL7, therefore I could readily
install it on my RHEL7 laptop, from a distribution-level package.
On the negative side, I quickly noticed two issues with "hub". I went
looking in the "hub" issue tracker, and to my relief, others had
reported the same problems before me (so I only commented on, and
subscribed to, the reports):
https://github.com/github/hub/issues/2338#issuecomment-548952413
https://github.com/github/hub/issues/2222#issuecomment-548967446
Issue#2338 is more technically challenging, but in practice (once CI is
enabled for edk2 itself), it's not really going to affect me.
Issue#2222 should be easy to solve, technically speaking, but until it's
solved, I find it quite disappointing.
Thanks!
Laszlo
>> -----Original Message-----
>> From: Laszlo Ersek <lersek@redhat.com>
>> Sent: Thursday, October 31, 2019 2:55 AM
>> To: devel@edk2.groups.io; Kinney, Michael D
>> <michael.d.kinney@intel.com>
>> Cc: Sean Brogan <sean.brogan@microsoft.com>; Bret
>> Barkelew <Bret.Barkelew@microsoft.com>; Gao, Liming
>> <liming.gao@intel.com>; Feng, Bob C
>> <bob.c.feng@intel.com>; Andrew Fish <afish@apple.com>;
>> Leif Lindholm <leif.lindholm@linaro.org>; Wang, Jian J
>> <jian.j.wang@intel.com>; Lu, XiaoyuX
>> <xiaoyux.lu@intel.com>; Ni, Ray <ray.ni@intel.com>; Wu,
>> Hao A <hao.a.wu@intel.com>; Wu, Jiaxin
>> <jiaxin.wu@intel.com>; Fu, Siyuan <siyuan.fu@intel.com>;
>> Yao, Jiewen <jiewen.yao@intel.com>; Zhang, Chao B
>> <chao.b.zhang@intel.com>; Gao, Zhichao
>> <zhichao.gao@intel.com>; Dong, Eric
>> <eric.dong@intel.com>
>> Subject: Re: [edk2-devel] [Patch v3 00/22] Enable Phase
>> 1 of EDK II CI
>>
>> On 10/29/19 20:54, Michael D Kinney wrote:
>>
>>> Active branch for testing/evaluation:
>>> * https://github.com/tianocore/edk2-staging/tree/edk2-
>> ci
>>> * To test, fork edk2-staging repo, create a branch
>> with a change, and submit
>>> a pull request targeting edk2-staging/edk2-ci.
>> NOTE: the default branch for
>>> the edk2-staging is 'about'. You must select the
>> 'edk2-ci' branch when
>>> a pull request is opened. Set the 'push' label to
>> require commit if all
>>> checks pass.
>>
>> The edk2-staging repository has been added as a "git
>> remote" to my local
>> edk2 clone for a long while now. Using the local
>> identifier "staging".
>> (This makes perfect sense as edk2-staging is itself a
>> fork of edk2, with branches that are supposed to be
>> rebased to edk2/master periodically.)
>>
>> Furthermore, the identifier by which I refer to the
>> remote at <https://github.com/lersek/edk2.git> is
>> "lersek".
>>
>> I've now run the following commands:
>>
>> $ git fetch staging
>> $ git checkout -b ci-test-1 staging/edk2-ci
>>
>> [modify "SampleFile.txt"]
>>
>> $ git add -p
>> $ git commit
>> $ git push lersek ci-test-1
>>
>> Questions:
>>
>> (a) How can I submit a pull request for the staging
>> repo's edk2-ci branch using the command line (and set
>> the "push" label)?
>>
>> (b) How can I submit a PR for the staging repo's edk2-ci
>> branch (regardless of command line vs. WebUI usage)
>> against my <https://github.com/lersek/edk2.git>
>> repository?
>>
>> When I go to the WebUI, the PR view does not offer
>> "tianocore/edk2-staging" as "base repository", it only
>> offers "tianocore/edk2". I thought I'd be able to pick
>> any destination repository at all.
>>
>> By the instruction "fork edk2-staging repo", did you
>> mean we should fork edk2-staging *on github*? (Using the
>> WebUI?)
>>
>> Thanks!
>> Laszlo
>
^ permalink raw reply [flat|nested] 56+ messages in thread
* Re: [edk2-devel] [Patch v3 00/22] Enable Phase 1 of EDK II CI
2019-11-01 22:39 ` Laszlo Ersek
@ 2019-11-01 23:22 ` Laszlo Ersek
2019-11-02 0:32 ` Michael D Kinney
1 sibling, 0 replies; 56+ messages in thread
From: Laszlo Ersek @ 2019-11-01 23:22 UTC (permalink / raw)
To: Kinney, Michael D, devel@edk2.groups.io
Cc: Sean Brogan, Bret Barkelew, Gao, Liming, Feng, Bob C, Andrew Fish,
Leif Lindholm, Wang, Jian J, Lu, XiaoyuX, Ni, Ray, Wu, Hao A,
Wu, Jiaxin, Fu, Siyuan, Yao, Jiewen, Zhang, Chao B, Gao, Zhichao,
Dong, Eric
On 11/01/19 23:39, Laszlo Ersek wrote:
> * When I submitted PR#87, there had been no conflicts; the PR was simply
> blocked on the CI tasks. Now, there are conflicts (I assume due to
> intervening pushes, while my PR#87 was blocked). It's useful that the
> WebUI points out this change in the status of the PR (i.e., "now
> conflicting" vs. "no conflicts just waiting for CI"). However, this
> status change has not been emailed to me.
>
> When this occurs to a PR (for example due to an intervening PR that is
> merged), what happens to the pre-empted PR? Does it remain suspended
> forever? How does the submitter learn about it?
Correction: I did get an email stating "PR can not be merged due to
conflict. Please rebase and resubmit".
I missed it temporarily because it was apparently only sent to:
tianocore/edk2-staging <edk2-staging@noreply.github.com>
and so it got filed into one of my list folders, not in my inbox.
I think this is OK after all.
Thanks!
Laszlo
^ permalink raw reply [flat|nested] 56+ messages in thread
* Re: [edk2-devel] [Patch v3 00/22] Enable Phase 1 of EDK II CI
2019-11-01 22:39 ` Laszlo Ersek
2019-11-01 23:22 ` Laszlo Ersek
@ 2019-11-02 0:32 ` Michael D Kinney
2019-11-02 10:49 ` Laszlo Ersek
1 sibling, 1 reply; 56+ messages in thread
From: Michael D Kinney @ 2019-11-02 0:32 UTC (permalink / raw)
To: Laszlo Ersek, devel@edk2.groups.io, Kinney, Michael D
Cc: Sean Brogan, Bret Barkelew, Gao, Liming, Feng, Bob C, Andrew Fish,
Leif Lindholm, Wang, Jian J, Lu, XiaoyuX, Ni, Ray, Wu, Hao A,
Wu, Jiaxin, Fu, Siyuan, Yao, Jiewen, Zhang, Chao B, Gao, Zhichao,
Dong, Eric
Hi Laszlo,
Everything is back online now. Will be doing more extensive
CI unit tests next.
If there is a merge conflict at the time the PR is submitted
you should be notified by email. I think you ran into what
should be an impossible case when the service is running
correctly. The fact it was not auto merging push requests
at the time you sent your PR allowed other pushes to make
it in.
You should have since received an email that says:
"PR can not be merged due to conflict. Please rebase and resubmit"
This came from the Mergify CI service when it was re-activated
And noticed the conflict.
Mike
> -----Original Message-----
> From: Laszlo Ersek <lersek@redhat.com>
> Sent: Friday, November 1, 2019 3:40 PM
> To: Kinney, Michael D <michael.d.kinney@intel.com>;
> devel@edk2.groups.io
> Cc: Sean Brogan <sean.brogan@microsoft.com>; Bret
> Barkelew <Bret.Barkelew@microsoft.com>; Gao, Liming
> <liming.gao@intel.com>; Feng, Bob C
> <bob.c.feng@intel.com>; Andrew Fish <afish@apple.com>;
> Leif Lindholm <leif.lindholm@linaro.org>; Wang, Jian J
> <jian.j.wang@intel.com>; Lu, XiaoyuX
> <xiaoyux.lu@intel.com>; Ni, Ray <ray.ni@intel.com>; Wu,
> Hao A <hao.a.wu@intel.com>; Wu, Jiaxin
> <jiaxin.wu@intel.com>; Fu, Siyuan <siyuan.fu@intel.com>;
> Yao, Jiewen <jiewen.yao@intel.com>; Zhang, Chao B
> <chao.b.zhang@intel.com>; Gao, Zhichao
> <zhichao.gao@intel.com>; Dong, Eric
> <eric.dong@intel.com>
> Subject: Re: [edk2-devel] [Patch v3 00/22] Enable Phase
> 1 of EDK II CI
>
> Hi Mike,
>
> On 11/01/19 06:03, Kinney, Michael D wrote:
> > Hi Laszlo,
> >
> > ===================================================
> > Note for all reviewers:
> > ---------------------------------------------------
> > Pull requests against edk2-staging/edk2-ci are not
> being processed
> > right now. We are working on some configuration
> changes after
> > noticing that all the checks were not being shown on
> the Web UI. We
> > will let you know when it is back up.
> > ===================================================
> >
> > Yes. You need to create a fork of the tianocore/edk2-
> staging repo.
> >
> > You can do this with the WebUI or the hub command line
> utility.
> >
> > * https://github.com/github/hub
> > * https://github.com/github/hub/releases
> > * https://hub.github.com/hub.1.html
> >
> > The hub command also supports creating a pull request.
> > I have used it extensively to write some unit tests
> for edk2-ci this
> > week.
> >
> > Once you create a branch with changes to submit in
> your own fork of
> > tianocore/edk2-staging the WebUI will show that this
> pull request is
> > possible and guide you through it.
> >
> > The use of edk2-staging/edk2-ci is only for the review
> and unit
> > testing. Once the review is approved, it will be
> enabled on
> > edk2/master and you will be able to use your own fork
> of edk2 to make
> > branches and submit pull requests.
>
> * For now, I submitted a pull request using the WebUi.
>
> https://github.com/tianocore/edk2-staging/pull/87
>
> I can see the processing is paused at the moment, as you
> state above.
>
>
> * When I submitted PR#87, there had been no conflicts;
> the PR was simply blocked on the CI tasks. Now, there
> are conflicts (I assume due to intervening pushes, while
> my PR#87 was blocked). It's useful that the WebUI points
> out this change in the status of the PR (i.e., "now
> conflicting" vs. "no conflicts just waiting for CI").
> However, this status change has not been emailed to me.
>
> When this occurs to a PR (for example due to an
> intervening PR that is merged), what happens to the pre-
> empted PR? Does it remain suspended forever? How does
> the submitter learn about it?
>
>
> * Regarding my question (b) below, the answer is "I
> can't". I cannot submit a PR against tianocore/edk2-
> staging.git with reference to lersek/edk2.git, given
> that the latter is a not a "github-level fork" of the
> former.
>
> This is quite regrettable, but once the CI system is in
> production, it will be OK in practice (I will only
> submit PRs against tianocore/edk2.git).
>
>
> * I've explored the "hub" command a bit. (This relates
> to my question (a); thanks for your suggestion.)
>
> On the positive side, it is packaged in EPEL7, therefore
> I could readily install it on my RHEL7 laptop, from a
> distribution-level package.
>
> On the negative side, I quickly noticed two issues with
> "hub". I went looking in the "hub" issue tracker, and to
> my relief, others had reported the same problems before
> me (so I only commented on, and subscribed to, the
> reports):
>
> https://github.com/github/hub/issues/2338#issuecomment-
> 548952413
> https://github.com/github/hub/issues/2222#issuecomment-
> 548967446
>
> Issue#2338 is more technically challenging, but in
> practice (once CI is enabled for edk2 itself), it's not
> really going to affect me.
>
> Issue#2222 should be easy to solve, technically
> speaking, but until it's solved, I find it quite
> disappointing.
>
> Thanks!
> Laszlo
>
> >> -----Original Message-----
> >> From: Laszlo Ersek <lersek@redhat.com>
> >> Sent: Thursday, October 31, 2019 2:55 AM
> >> To: devel@edk2.groups.io; Kinney, Michael D
> >> <michael.d.kinney@intel.com>
> >> Cc: Sean Brogan <sean.brogan@microsoft.com>; Bret
> Barkelew
> >> <Bret.Barkelew@microsoft.com>; Gao, Liming
> <liming.gao@intel.com>;
> >> Feng, Bob C <bob.c.feng@intel.com>; Andrew Fish
> <afish@apple.com>;
> >> Leif Lindholm <leif.lindholm@linaro.org>; Wang, Jian
> J
> >> <jian.j.wang@intel.com>; Lu, XiaoyuX
> <xiaoyux.lu@intel.com>; Ni, Ray
> >> <ray.ni@intel.com>; Wu, Hao A <hao.a.wu@intel.com>;
> Wu, Jiaxin
> >> <jiaxin.wu@intel.com>; Fu, Siyuan
> <siyuan.fu@intel.com>; Yao, Jiewen
> >> <jiewen.yao@intel.com>; Zhang, Chao B
> <chao.b.zhang@intel.com>; Gao,
> >> Zhichao <zhichao.gao@intel.com>; Dong, Eric
> <eric.dong@intel.com>
> >> Subject: Re: [edk2-devel] [Patch v3 00/22] Enable
> Phase
> >> 1 of EDK II CI
> >>
> >> On 10/29/19 20:54, Michael D Kinney wrote:
> >>
> >>> Active branch for testing/evaluation:
> >>> * https://github.com/tianocore/edk2-
> staging/tree/edk2-
> >> ci
> >>> * To test, fork edk2-staging repo, create a branch
> >> with a change, and submit
> >>> a pull request targeting edk2-staging/edk2-ci.
> >> NOTE: the default branch for
> >>> the edk2-staging is 'about'. You must select the
> >> 'edk2-ci' branch when
> >>> a pull request is opened. Set the 'push' label to
> >> require commit if all
> >>> checks pass.
> >>
> >> The edk2-staging repository has been added as a "git
> remote" to my
> >> local
> >> edk2 clone for a long while now. Using the local
> identifier
> >> "staging".
> >> (This makes perfect sense as edk2-staging is itself a
> fork of edk2,
> >> with branches that are supposed to be rebased to
> edk2/master
> >> periodically.)
> >>
> >> Furthermore, the identifier by which I refer to the
> remote at
> >> <https://github.com/lersek/edk2.git> is "lersek".
> >>
> >> I've now run the following commands:
> >>
> >> $ git fetch staging
> >> $ git checkout -b ci-test-1 staging/edk2-ci
> >>
> >> [modify "SampleFile.txt"]
> >>
> >> $ git add -p
> >> $ git commit
> >> $ git push lersek ci-test-1
> >>
> >> Questions:
> >>
> >> (a) How can I submit a pull request for the staging
> repo's edk2-ci
> >> branch using the command line (and set the "push"
> label)?
> >>
> >> (b) How can I submit a PR for the staging repo's
> edk2-ci branch
> >> (regardless of command line vs. WebUI usage) against
> my
> >> <https://github.com/lersek/edk2.git>
> >> repository?
> >>
> >> When I go to the WebUI, the PR view does not offer
> >> "tianocore/edk2-staging" as "base repository", it
> only offers
> >> "tianocore/edk2". I thought I'd be able to pick any
> destination
> >> repository at all.
> >>
> >> By the instruction "fork edk2-staging repo", did you
> mean we should
> >> fork edk2-staging *on github*? (Using the
> >> WebUI?)
> >>
> >> Thanks!
> >> Laszlo
> >
^ permalink raw reply [flat|nested] 56+ messages in thread
* Re: [edk2-devel] [Patch v3 00/22] Enable Phase 1 of EDK II CI
2019-11-02 0:32 ` Michael D Kinney
@ 2019-11-02 10:49 ` Laszlo Ersek
0 siblings, 0 replies; 56+ messages in thread
From: Laszlo Ersek @ 2019-11-02 10:49 UTC (permalink / raw)
To: Kinney, Michael D, devel@edk2.groups.io
Cc: Sean Brogan, Bret Barkelew, Gao, Liming, Feng, Bob C, Andrew Fish,
Leif Lindholm, Wang, Jian J, Lu, XiaoyuX, Ni, Ray, Wu, Hao A,
Wu, Jiaxin, Fu, Siyuan, Yao, Jiewen, Zhang, Chao B, Gao, Zhichao,
Dong, Eric
On 11/02/19 01:32, Kinney, Michael D wrote:
> Hi Laszlo,
>
> Everything is back online now. Will be doing more extensive
> CI unit tests next.
Thanks! I've submitted
<https://github.com/tianocore/edk2-staging/pull/99> now.
> If there is a merge conflict at the time the PR is submitted
> you should be notified by email. I think you ran into what
> should be an impossible case when the service is running
> correctly. The fact it was not auto merging push requests
> at the time you sent your PR allowed other pushes to make
> it in.
>
> You should have since received an email that says:
>
> "PR can not be merged due to conflict. Please rebase and resubmit"
That's indeed the case. (See my other email.)
> This came from the Mergify CI service when it was re-activated
> And noticed the conflict.
Thanks!
Laszlo
>> -----Original Message-----
>> From: Laszlo Ersek <lersek@redhat.com>
>> Sent: Friday, November 1, 2019 3:40 PM
>> To: Kinney, Michael D <michael.d.kinney@intel.com>;
>> devel@edk2.groups.io
>> Cc: Sean Brogan <sean.brogan@microsoft.com>; Bret
>> Barkelew <Bret.Barkelew@microsoft.com>; Gao, Liming
>> <liming.gao@intel.com>; Feng, Bob C
>> <bob.c.feng@intel.com>; Andrew Fish <afish@apple.com>;
>> Leif Lindholm <leif.lindholm@linaro.org>; Wang, Jian J
>> <jian.j.wang@intel.com>; Lu, XiaoyuX
>> <xiaoyux.lu@intel.com>; Ni, Ray <ray.ni@intel.com>; Wu,
>> Hao A <hao.a.wu@intel.com>; Wu, Jiaxin
>> <jiaxin.wu@intel.com>; Fu, Siyuan <siyuan.fu@intel.com>;
>> Yao, Jiewen <jiewen.yao@intel.com>; Zhang, Chao B
>> <chao.b.zhang@intel.com>; Gao, Zhichao
>> <zhichao.gao@intel.com>; Dong, Eric
>> <eric.dong@intel.com>
>> Subject: Re: [edk2-devel] [Patch v3 00/22] Enable Phase
>> 1 of EDK II CI
>>
>> Hi Mike,
>>
>> On 11/01/19 06:03, Kinney, Michael D wrote:
>>> Hi Laszlo,
>>>
>>> ===================================================
>>> Note for all reviewers:
>>> ---------------------------------------------------
>>> Pull requests against edk2-staging/edk2-ci are not
>> being processed
>>> right now. We are working on some configuration
>> changes after
>>> noticing that all the checks were not being shown on
>> the Web UI. We
>>> will let you know when it is back up.
>>> ===================================================
>>>
>>> Yes. You need to create a fork of the tianocore/edk2-
>> staging repo.
>>>
>>> You can do this with the WebUI or the hub command line
>> utility.
>>>
>>> * https://github.com/github/hub
>>> * https://github.com/github/hub/releases
>>> * https://hub.github.com/hub.1.html
>>>
>>> The hub command also supports creating a pull request.
>>> I have used it extensively to write some unit tests
>> for edk2-ci this
>>> week.
>>>
>>> Once you create a branch with changes to submit in
>> your own fork of
>>> tianocore/edk2-staging the WebUI will show that this
>> pull request is
>>> possible and guide you through it.
>>>
>>> The use of edk2-staging/edk2-ci is only for the review
>> and unit
>>> testing. Once the review is approved, it will be
>> enabled on
>>> edk2/master and you will be able to use your own fork
>> of edk2 to make
>>> branches and submit pull requests.
>>
>> * For now, I submitted a pull request using the WebUi.
>>
>> https://github.com/tianocore/edk2-staging/pull/87
>>
>> I can see the processing is paused at the moment, as you
>> state above.
>>
>>
>> * When I submitted PR#87, there had been no conflicts;
>> the PR was simply blocked on the CI tasks. Now, there
>> are conflicts (I assume due to intervening pushes, while
>> my PR#87 was blocked). It's useful that the WebUI points
>> out this change in the status of the PR (i.e., "now
>> conflicting" vs. "no conflicts just waiting for CI").
>> However, this status change has not been emailed to me.
>>
>> When this occurs to a PR (for example due to an
>> intervening PR that is merged), what happens to the pre-
>> empted PR? Does it remain suspended forever? How does
>> the submitter learn about it?
>>
>>
>> * Regarding my question (b) below, the answer is "I
>> can't". I cannot submit a PR against tianocore/edk2-
>> staging.git with reference to lersek/edk2.git, given
>> that the latter is a not a "github-level fork" of the
>> former.
>>
>> This is quite regrettable, but once the CI system is in
>> production, it will be OK in practice (I will only
>> submit PRs against tianocore/edk2.git).
>>
>>
>> * I've explored the "hub" command a bit. (This relates
>> to my question (a); thanks for your suggestion.)
>>
>> On the positive side, it is packaged in EPEL7, therefore
>> I could readily install it on my RHEL7 laptop, from a
>> distribution-level package.
>>
>> On the negative side, I quickly noticed two issues with
>> "hub". I went looking in the "hub" issue tracker, and to
>> my relief, others had reported the same problems before
>> me (so I only commented on, and subscribed to, the
>> reports):
>>
>> https://github.com/github/hub/issues/2338#issuecomment-
>> 548952413
>> https://github.com/github/hub/issues/2222#issuecomment-
>> 548967446
>>
>> Issue#2338 is more technically challenging, but in
>> practice (once CI is enabled for edk2 itself), it's not
>> really going to affect me.
>>
>> Issue#2222 should be easy to solve, technically
>> speaking, but until it's solved, I find it quite
>> disappointing.
>>
>> Thanks!
>> Laszlo
>>
>>>> -----Original Message-----
>>>> From: Laszlo Ersek <lersek@redhat.com>
>>>> Sent: Thursday, October 31, 2019 2:55 AM
>>>> To: devel@edk2.groups.io; Kinney, Michael D
>>>> <michael.d.kinney@intel.com>
>>>> Cc: Sean Brogan <sean.brogan@microsoft.com>; Bret
>> Barkelew
>>>> <Bret.Barkelew@microsoft.com>; Gao, Liming
>> <liming.gao@intel.com>;
>>>> Feng, Bob C <bob.c.feng@intel.com>; Andrew Fish
>> <afish@apple.com>;
>>>> Leif Lindholm <leif.lindholm@linaro.org>; Wang, Jian
>> J
>>>> <jian.j.wang@intel.com>; Lu, XiaoyuX
>> <xiaoyux.lu@intel.com>; Ni, Ray
>>>> <ray.ni@intel.com>; Wu, Hao A <hao.a.wu@intel.com>;
>> Wu, Jiaxin
>>>> <jiaxin.wu@intel.com>; Fu, Siyuan
>> <siyuan.fu@intel.com>; Yao, Jiewen
>>>> <jiewen.yao@intel.com>; Zhang, Chao B
>> <chao.b.zhang@intel.com>; Gao,
>>>> Zhichao <zhichao.gao@intel.com>; Dong, Eric
>> <eric.dong@intel.com>
>>>> Subject: Re: [edk2-devel] [Patch v3 00/22] Enable
>> Phase
>>>> 1 of EDK II CI
>>>>
>>>> On 10/29/19 20:54, Michael D Kinney wrote:
>>>>
>>>>> Active branch for testing/evaluation:
>>>>> * https://github.com/tianocore/edk2-
>> staging/tree/edk2-
>>>> ci
>>>>> * To test, fork edk2-staging repo, create a branch
>>>> with a change, and submit
>>>>> a pull request targeting edk2-staging/edk2-ci.
>>>> NOTE: the default branch for
>>>>> the edk2-staging is 'about'. You must select the
>>>> 'edk2-ci' branch when
>>>>> a pull request is opened. Set the 'push' label to
>>>> require commit if all
>>>>> checks pass.
>>>>
>>>> The edk2-staging repository has been added as a "git
>> remote" to my
>>>> local
>>>> edk2 clone for a long while now. Using the local
>> identifier
>>>> "staging".
>>>> (This makes perfect sense as edk2-staging is itself a
>> fork of edk2,
>>>> with branches that are supposed to be rebased to
>> edk2/master
>>>> periodically.)
>>>>
>>>> Furthermore, the identifier by which I refer to the
>> remote at
>>>> <https://github.com/lersek/edk2.git> is "lersek".
>>>>
>>>> I've now run the following commands:
>>>>
>>>> $ git fetch staging
>>>> $ git checkout -b ci-test-1 staging/edk2-ci
>>>>
>>>> [modify "SampleFile.txt"]
>>>>
>>>> $ git add -p
>>>> $ git commit
>>>> $ git push lersek ci-test-1
>>>>
>>>> Questions:
>>>>
>>>> (a) How can I submit a pull request for the staging
>> repo's edk2-ci
>>>> branch using the command line (and set the "push"
>> label)?
>>>>
>>>> (b) How can I submit a PR for the staging repo's
>> edk2-ci branch
>>>> (regardless of command line vs. WebUI usage) against
>> my
>>>> <https://github.com/lersek/edk2.git>
>>>> repository?
>>>>
>>>> When I go to the WebUI, the PR view does not offer
>>>> "tianocore/edk2-staging" as "base repository", it
>> only offers
>>>> "tianocore/edk2". I thought I'd be able to pick any
>> destination
>>>> repository at all.
>>>>
>>>> By the instruction "fork edk2-staging repo", did you
>> mean we should
>>>> fork edk2-staging *on github*? (Using the
>>>> WebUI?)
>>>>
>>>> Thanks!
>>>> Laszlo
>>>
>
^ permalink raw reply [flat|nested] 56+ messages in thread
