public inbox for devel@edk2.groups.io
 help / color / mirror / Atom feed
* Improvements to build system etc. for edk2-platforms devel-MinnowBoard3?
@ 2017-02-22  1:20 Rebecca Cran
  2017-02-22  4:50 ` Gao, Liming
  0 siblings, 1 reply; 12+ messages in thread
From: Rebecca Cran @ 2017-02-22  1:20 UTC (permalink / raw)
  To: edk2-devel@lists.01.org
  Cc: david.wei, mang.guo, vincent.zimmer, mike.wu, shifeix.a.lu

I'm a bit of a build system geek and noticed there are some improvements 
(e.g. improving build speed) that could be made to the 
devel-MinnowBoard3 branch in edk2-platforms.  I know it's still in heavy 
development, so I'm wondering if my contributions might be welcome, or 
if I should hold off for a few more months?


-- 

Rebecca



^ permalink raw reply	[flat|nested] 12+ messages in thread

* Re: Improvements to build system etc. for edk2-platforms devel-MinnowBoard3?
  2017-02-22  1:20 Improvements to build system etc. for edk2-platforms devel-MinnowBoard3? Rebecca Cran
@ 2017-02-22  4:50 ` Gao, Liming
  2017-02-22  5:11   ` Andrew Fish
  2017-02-22  5:48   ` Rebecca Cran
  0 siblings, 2 replies; 12+ messages in thread
From: Gao, Liming @ 2017-02-22  4:50 UTC (permalink / raw)
  To: Rebecca Cran, edk2-devel@lists.01.org
  Cc: Lu, ShifeiX A, Zimmer, Vincent, Wei, David

Rebecca:
  Could you introduce what change will be done for build improvement? I am also interested in this topic. 

Thanks
Liming
>-----Original Message-----
>From: edk2-devel [mailto:edk2-devel-bounces@lists.01.org] On Behalf Of
>Rebecca Cran
>Sent: Wednesday, February 22, 2017 9:21 AM
>To: edk2-devel@lists.01.org
>Cc: Lu, ShifeiX A <shifeix.a.lu@intel.com>; Zimmer, Vincent
><vincent.zimmer@intel.com>; Wei, David <david.wei@intel.com>
>Subject: [edk2] Improvements to build system etc. for edk2-platforms devel-
>MinnowBoard3?
>
>I'm a bit of a build system geek and noticed there are some improvements
>(e.g. improving build speed) that could be made to the
>devel-MinnowBoard3 branch in edk2-platforms.  I know it's still in heavy
>development, so I'm wondering if my contributions might be welcome, or
>if I should hold off for a few more months?
>
>
>--
>
>Rebecca
>
>_______________________________________________
>edk2-devel mailing list
>edk2-devel@lists.01.org
>https://lists.01.org/mailman/listinfo/edk2-devel


^ permalink raw reply	[flat|nested] 12+ messages in thread

* Re: Improvements to build system etc. for edk2-platforms devel-MinnowBoard3?
  2017-02-22  4:50 ` Gao, Liming
@ 2017-02-22  5:11   ` Andrew Fish
  2017-02-22  5:48   ` Rebecca Cran
  1 sibling, 0 replies; 12+ messages in thread
From: Andrew Fish @ 2017-02-22  5:11 UTC (permalink / raw)
  To: Gao, Liming
  Cc: Rebecca Cran, edk2-devel@lists.01.org, Lu, ShifeiX A,
	Vincent Zimmer, Wei, David


> On Feb 21, 2017, at 8:50 PM, Gao, Liming <liming.gao@intel.com> wrote:
> 
> Rebecca:
>  Could you introduce what change will be done for build improvement? I am also interested in this topic. 
> 

Me too.

Thanks,

Andrew Fish

> Thanks
> Liming
>> -----Original Message-----
>> From: edk2-devel [mailto:edk2-devel-bounces@lists.01.org] On Behalf Of
>> Rebecca Cran
>> Sent: Wednesday, February 22, 2017 9:21 AM
>> To: edk2-devel@lists.01.org
>> Cc: Lu, ShifeiX A <shifeix.a.lu@intel.com>; Zimmer, Vincent
>> <vincent.zimmer@intel.com>; Wei, David <david.wei@intel.com>
>> Subject: [edk2] Improvements to build system etc. for edk2-platforms devel-
>> MinnowBoard3?
>> 
>> I'm a bit of a build system geek and noticed there are some improvements
>> (e.g. improving build speed) that could be made to the
>> devel-MinnowBoard3 branch in edk2-platforms.  I know it's still in heavy
>> development, so I'm wondering if my contributions might be welcome, or
>> if I should hold off for a few more months?
>> 
>> 
>> --
>> 
>> Rebecca
>> 
>> _______________________________________________
>> edk2-devel mailing list
>> edk2-devel@lists.01.org
>> https://lists.01.org/mailman/listinfo/edk2-devel
> _______________________________________________
> edk2-devel mailing list
> edk2-devel@lists.01.org
> https://lists.01.org/mailman/listinfo/edk2-devel



^ permalink raw reply	[flat|nested] 12+ messages in thread

* Re: Improvements to build system etc. for edk2-platforms devel-MinnowBoard3?
  2017-02-22  4:50 ` Gao, Liming
  2017-02-22  5:11   ` Andrew Fish
@ 2017-02-22  5:48   ` Rebecca Cran
  2017-02-22  7:35     ` Gao, Liming
  2017-02-22 16:34     ` Richardson, Brian
  1 sibling, 2 replies; 12+ messages in thread
From: Rebecca Cran @ 2017-02-22  5:48 UTC (permalink / raw)
  To: Gao, Liming, edk2-devel@lists.01.org
  Cc: Lu, ShifeiX A, Zimmer, Vincent, Wei, David, Andrew Fish,
	Gao, Liming

On 2/21/2017 9:50 PM, Gao, Liming wrote:

>    Could you introduce what change will be done for build improvement? I am also interested in this topic.

The first change I'd make is to set 'buildthreads' in BuildBIOS to 
%NUMBER_OF_PROCESSORS% - that by itself reduces the build time from 
around 7 minutes to 2.5 on my system. I also have some changes to the 
ReadMe.MD file to convert it to be a MD file instead of plain text, 
which massively increases readability on GitHub.  I've also been 
wondering about allowing more parameters to be passed through to 'build' 
such as reducing the verbosity to make compiler warnings more apparent. 
Finally, and this is likely more controversial - I've been wondering if 
the build scripts like BuildBIOS should be in python instead of Windows 
.cmd/.bat since python is already used elsewhere and is more flexible.

-- 
Rebecca


^ permalink raw reply	[flat|nested] 12+ messages in thread

* Re: Improvements to build system etc. for edk2-platforms devel-MinnowBoard3?
  2017-02-22  5:48   ` Rebecca Cran
@ 2017-02-22  7:35     ` Gao, Liming
  2017-02-22 16:34     ` Richardson, Brian
  1 sibling, 0 replies; 12+ messages in thread
From: Gao, Liming @ 2017-02-22  7:35 UTC (permalink / raw)
  To: Rebecca Cran, edk2-devel@lists.01.org
  Cc: Lu, ShifeiX A, Zimmer, Vincent, Wei, David, Andrew Fish

Rebecca:
  Yes. It is good to enable multiple thread and update ReadMe.MD. On .cmd/.bat vs python, .cmd/.bat doesn't require to install Python in windows OS. edksetup.bat is just a case. 

Thanks
Liming
>-----Original Message-----
>From: Rebecca Cran [mailto:rebecca@bluestop.org]
>Sent: Wednesday, February 22, 2017 1:48 PM
>To: Gao, Liming <liming.gao@intel.com>; edk2-devel@lists.01.org
>Cc: Lu, ShifeiX A <shifeix.a.lu@intel.com>; Zimmer, Vincent
><vincent.zimmer@intel.com>; Wei, David <david.wei@intel.com>; Andrew
>Fish <afish@apple.com>; Gao, Liming <liming.gao@intel.com>
>Subject: Re: [edk2] Improvements to build system etc. for edk2-platforms
>devel-MinnowBoard3?
>
>On 2/21/2017 9:50 PM, Gao, Liming wrote:
>
>>    Could you introduce what change will be done for build improvement? I
>am also interested in this topic.
>
>The first change I'd make is to set 'buildthreads' in BuildBIOS to
>%NUMBER_OF_PROCESSORS% - that by itself reduces the build time from
>around 7 minutes to 2.5 on my system. I also have some changes to the
>ReadMe.MD file to convert it to be a MD file instead of plain text,
>which massively increases readability on GitHub.  I've also been
>wondering about allowing more parameters to be passed through to 'build'
>such as reducing the verbosity to make compiler warnings more apparent.
>Finally, and this is likely more controversial - I've been wondering if
>the build scripts like BuildBIOS should be in python instead of Windows
>.cmd/.bat since python is already used elsewhere and is more flexible.
>
>--
>Rebecca


^ permalink raw reply	[flat|nested] 12+ messages in thread

* Re: Improvements to build system etc. for edk2-platforms devel-MinnowBoard3?
  2017-02-22  5:48   ` Rebecca Cran
  2017-02-22  7:35     ` Gao, Liming
@ 2017-02-22 16:34     ` Richardson, Brian
  2017-02-22 16:52       ` Rebecca Cran
  1 sibling, 1 reply; 12+ messages in thread
From: Richardson, Brian @ 2017-02-22 16:34 UTC (permalink / raw)
  To: Rebecca Cran, Gao, Liming, edk2-devel@lists.01.org
  Cc: Lu, ShifeiX A, Zimmer, Vincent, Andrew Fish, Gao, Liming,
	Wei, David

Thanks for the input. For future reference, you can use the TianoCore Bugzilla to report issues on any EDK II feature/platform. https://github.com/tianocore/tianocore.github.io/wiki/Reporting-Issues   

I agree the readme.md should be present, and use markup instead of plain text to work better in github. You can open an issue on this in Bugzilla.

Normally, we ask folks to change the number of processor threads based on their system configuration. We don't add a larger thread number by default, but it might be good to set it '5' by default (assuming a dual core processor with hyperthreading) instead of '1' (assuming a single core system w/o threading). I don't know if this will cause any compatibility issues on older systems, but it's worth a check.

At this time, MinnowBoard 3 build is only validated in Windows. That's why there is no equivalent .sh file for BuildBIOS yet, but it will be added once Linux build is verified and checked in.

Thanks ... br
---
Brian Richardson, Senior Technical Marketing Engineer, Intel Software
brian.richardson@intel.com -- @intel_Brian (Twitter & WeChat)
https://software.intel.com/en-us/meet-the-developers/evangelists/team/brian-richardson 

-----Original Message-----
From: edk2-devel [mailto:edk2-devel-bounces@lists.01.org] On Behalf Of Rebecca Cran
Sent: Wednesday, February 22, 2017 12:48 AM
To: Gao, Liming <liming.gao@intel.com>; edk2-devel@lists.01.org
Cc: Lu, ShifeiX A <shifeix.a.lu@intel.com>; Zimmer, Vincent <vincent.zimmer@intel.com>; Andrew Fish <afish@apple.com>; Gao, Liming <liming.gao@intel.com>; Wei, David <david.wei@intel.com>
Subject: Re: [edk2] Improvements to build system etc. for edk2-platforms devel-MinnowBoard3?

On 2/21/2017 9:50 PM, Gao, Liming wrote:

>    Could you introduce what change will be done for build improvement? I am also interested in this topic.

The first change I'd make is to set 'buildthreads' in BuildBIOS to %NUMBER_OF_PROCESSORS% - that by itself reduces the build time from around 7 minutes to 2.5 on my system. I also have some changes to the ReadMe.MD file to convert it to be a MD file instead of plain text, which massively increases readability on GitHub.  I've also been wondering about allowing more parameters to be passed through to 'build' 
such as reducing the verbosity to make compiler warnings more apparent. 
Finally, and this is likely more controversial - I've been wondering if the build scripts like BuildBIOS should be in python instead of Windows .cmd/.bat since python is already used elsewhere and is more flexible.

--
Rebecca
_______________________________________________
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel


^ permalink raw reply	[flat|nested] 12+ messages in thread

* Re: Improvements to build system etc. for edk2-platforms devel-MinnowBoard3?
  2017-02-22 16:34     ` Richardson, Brian
@ 2017-02-22 16:52       ` Rebecca Cran
  2017-02-22 19:10         ` Richardson, Brian
  0 siblings, 1 reply; 12+ messages in thread
From: Rebecca Cran @ 2017-02-22 16:52 UTC (permalink / raw)
  To: Richardson, Brian, Gao, Liming, edk2-devel@lists.01.org
  Cc: Lu, ShifeiX A, Zimmer, Vincent, Andrew Fish, Wei, David

On 2/22/2017 9:34 AM, Richardson, Brian wrote:
> Thanks for the input. For future reference, you can use the TianoCore Bugzilla to report issues on any EDK II feature/platform. https://github.com/tianocore/tianocore.github.io/wiki/Reporting-Issues
>
> I agree the readme.md should be present, and use markup instead of plain text to work better in github. You can open an issue on this in Bugzilla.
>
> Normally, we ask folks to change the number of processor threads based on their system configuration. We don't add a larger thread number by default, but it might be good to set it '5' by default (assuming a dual core processor with hyperthreading) instead of '1' (assuming a single core system w/o threading). I don't know if this will cause any compatibility issues on older systems, but it's worth a check.
>
> At this time, MinnowBoard 3 build is only validated in Windows. That's why there is no equivalent .sh file for BuildBIOS yet, but it will be added once Linux build is verified and checked in.

I'm more about _fixing_ issues I find rather than reporting them! Are 
you saying that patches wouldn't be welcome just now?  Is there a reason 
why you don't want to make full use of the CPU while building? And I 
understand that MinnowBoard 3 only builds under Windows at the moment, 
but if more of it built using python (and python is already listed as a 
prerequisite in the ReadMe file) the porting might be simpler.

-- 
Rebecca


^ permalink raw reply	[flat|nested] 12+ messages in thread

* Re: Improvements to build system etc. for edk2-platforms devel-MinnowBoard3?
  2017-02-22 16:52       ` Rebecca Cran
@ 2017-02-22 19:10         ` Richardson, Brian
  2017-02-23  2:49           ` Wei, David
  0 siblings, 1 reply; 12+ messages in thread
From: Richardson, Brian @ 2017-02-22 19:10 UTC (permalink / raw)
  To: Rebecca Cran, Gao, Liming, edk2-devel@lists.01.org
  Cc: Lu, ShifeiX A, Zimmer, Vincent, Andrew Fish, Wei, David

We're ok with help fixing issues (yay open source), so thanks for the help. Patches are welcome at this time. But we do track them in Bugzilla, so opening an issue there is the first step to a solution. 

For the processor thread setting, note that we have historically disabled processor threading by default because we don't know the build system configuration. While we at Intel prefer everyone own an Intel(R) Core(TM) i7 or Intel(R) Xeon processor, we know that's not the case ... so keeping it disabled has been seen as a safe option. We should definitely do a better job of documenting that setting change, but I think we need to consider generically changing that setting by default in target.txt (which requires a Bugzilla entry).

The .bat/.sh files are required to trigger post-build tools, which are OS dependent. Even if the build was triggered by a Python script, we still need to do some verification to make sure there are no functional differences when we build in a Windows environment versus Linux (work in progress).

Thanks ... br
---
Brian Richardson, Senior Technical Marketing Engineer, Intel Software
brian.richardson@intel.com -- @intel_Brian (Twitter & WeChat)
https://software.intel.com/en-us/meet-the-developers/evangelists/team/brian-richardson 

-----Original Message-----
From: edk2-devel [mailto:edk2-devel-bounces@lists.01.org] On Behalf Of Rebecca Cran
Sent: Wednesday, February 22, 2017 11:53 AM
To: Richardson, Brian <brian.richardson@intel.com>; Gao, Liming <liming.gao@intel.com>; edk2-devel@lists.01.org
Cc: Lu, ShifeiX A <shifeix.a.lu@intel.com>; Zimmer, Vincent <vincent.zimmer@intel.com>; Andrew Fish <afish@apple.com>; Wei, David <david.wei@intel.com>
Subject: Re: [edk2] Improvements to build system etc. for edk2-platforms devel-MinnowBoard3?

On 2/22/2017 9:34 AM, Richardson, Brian wrote:
> Thanks for the input. For future reference, you can use the TianoCore 
> Bugzilla to report issues on any EDK II feature/platform. 
> https://github.com/tianocore/tianocore.github.io/wiki/Reporting-Issues
>
> I agree the readme.md should be present, and use markup instead of plain text to work better in github. You can open an issue on this in Bugzilla.
>
> Normally, we ask folks to change the number of processor threads based on their system configuration. We don't add a larger thread number by default, but it might be good to set it '5' by default (assuming a dual core processor with hyperthreading) instead of '1' (assuming a single core system w/o threading). I don't know if this will cause any compatibility issues on older systems, but it's worth a check.
>
> At this time, MinnowBoard 3 build is only validated in Windows. That's why there is no equivalent .sh file for BuildBIOS yet, but it will be added once Linux build is verified and checked in.

I'm more about _fixing_ issues I find rather than reporting them! Are you saying that patches wouldn't be welcome just now?  Is there a reason why you don't want to make full use of the CPU while building? And I understand that MinnowBoard 3 only builds under Windows at the moment, but if more of it built using python (and python is already listed as a prerequisite in the ReadMe file) the porting might be simpler.

--
Rebecca
_______________________________________________
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel


^ permalink raw reply	[flat|nested] 12+ messages in thread

* Re: Improvements to build system etc. for edk2-platforms devel-MinnowBoard3?
  2017-02-22 19:10         ` Richardson, Brian
@ 2017-02-23  2:49           ` Wei, David
  2017-02-23 18:53             ` Brian J. Johnson
  0 siblings, 1 reply; 12+ messages in thread
From: Wei, David @ 2017-02-23  2:49 UTC (permalink / raw)
  To: Richardson, Brian, Rebecca Cran, Gao, Liming,
	edk2-devel@lists.01.org
  Cc: Lu, ShifeiX A, Zimmer, Vincent, Andrew Fish, Wei, David

Yes, as Brian said, at this stage,  patches are welcomed, including the python script.  You can also file bugs on https://bugzilla.tianocore.org/ for issue track and discussion.  Please remember to CC stakeholders. 

Specifically for the multi-thread building , I think maybe  NUMBER_OF_PROCESSORS Windows environment variable could be used as the proper thread number. 

Thanks,
David  Wei                                 


-----Original Message-----
From: Richardson, Brian 
Sent: Thursday, February 23, 2017 3:10 AM
To: Rebecca Cran <rebecca@bluestop.org>; Gao, Liming <liming.gao@intel.com>; edk2-devel@lists.01.org
Cc: Lu, ShifeiX A <shifeix.a.lu@intel.com>; Zimmer, Vincent <vincent.zimmer@intel.com>; Andrew Fish <afish@apple.com>; Wei, David <david.wei@intel.com>
Subject: RE: [edk2] Improvements to build system etc. for edk2-platforms devel-MinnowBoard3?

We're ok with help fixing issues (yay open source), so thanks for the help. Patches are welcome at this time. But we do track them in Bugzilla, so opening an issue there is the first step to a solution. 

For the processor thread setting, note that we have historically disabled processor threading by default because we don't know the build system configuration. While we at Intel prefer everyone own an Intel(R) Core(TM) i7 or Intel(R) Xeon processor, we know that's not the case ... so keeping it disabled has been seen as a safe option. We should definitely do a better job of documenting that setting change, but I think we need to consider generically changing that setting by default in target.txt (which requires a Bugzilla entry).

The .bat/.sh files are required to trigger post-build tools, which are OS dependent. Even if the build was triggered by a Python script, we still need to do some verification to make sure there are no functional differences when we build in a Windows environment versus Linux (work in progress).

Thanks ... br
---
Brian Richardson, Senior Technical Marketing Engineer, Intel Software
brian.richardson@intel.com -- @intel_Brian (Twitter & WeChat)
https://software.intel.com/en-us/meet-the-developers/evangelists/team/brian-richardson 

-----Original Message-----
From: edk2-devel [mailto:edk2-devel-bounces@lists.01.org] On Behalf Of Rebecca Cran
Sent: Wednesday, February 22, 2017 11:53 AM
To: Richardson, Brian <brian.richardson@intel.com>; Gao, Liming <liming.gao@intel.com>; edk2-devel@lists.01.org
Cc: Lu, ShifeiX A <shifeix.a.lu@intel.com>; Zimmer, Vincent <vincent.zimmer@intel.com>; Andrew Fish <afish@apple.com>; Wei, David <david.wei@intel.com>
Subject: Re: [edk2] Improvements to build system etc. for edk2-platforms devel-MinnowBoard3?

On 2/22/2017 9:34 AM, Richardson, Brian wrote:
> Thanks for the input. For future reference, you can use the TianoCore 
> Bugzilla to report issues on any EDK II feature/platform. 
> https://github.com/tianocore/tianocore.github.io/wiki/Reporting-Issues
>
> I agree the readme.md should be present, and use markup instead of plain text to work better in github. You can open an issue on this in Bugzilla.
>
> Normally, we ask folks to change the number of processor threads based on their system configuration. We don't add a larger thread number by default, but it might be good to set it '5' by default (assuming a dual core processor with hyperthreading) instead of '1' (assuming a single core system w/o threading). I don't know if this will cause any compatibility issues on older systems, but it's worth a check.
>
> At this time, MinnowBoard 3 build is only validated in Windows. That's why there is no equivalent .sh file for BuildBIOS yet, but it will be added once Linux build is verified and checked in.

I'm more about _fixing_ issues I find rather than reporting them! Are you saying that patches wouldn't be welcome just now?  Is there a reason why you don't want to make full use of the CPU while building? And I understand that MinnowBoard 3 only builds under Windows at the moment, but if more of it built using python (and python is already listed as a prerequisite in the ReadMe file) the porting might be simpler.

--
Rebecca
_______________________________________________
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel


^ permalink raw reply	[flat|nested] 12+ messages in thread

* Re: Improvements to build system etc. for edk2-platforms devel-MinnowBoard3?
  2017-02-23  2:49           ` Wei, David
@ 2017-02-23 18:53             ` Brian J. Johnson
  2017-02-23 19:10               ` Rebecca Cran
  0 siblings, 1 reply; 12+ messages in thread
From: Brian J. Johnson @ 2017-02-23 18:53 UTC (permalink / raw)
  To: Wei, David, Richardson, Brian, Rebecca Cran, Gao, Liming,
	edk2-devel@lists.01.org
  Cc: Lu, ShifeiX A, Zimmer, Vincent, Andrew Fish

Sorry if I'm bikeshedding...  NUMBER_OF_PROCESSORS isn't a good default 
for those of us who build on servers with hundreds of threads available. 
  The OS, disks, and build.exe/build.py become bottlenecks.  Maybe we 
could put a cap (say, 20) on the default thread limit, so it uses 
NUMBER_OF_PROCESSORS or 20 threads, whichever is less.

Or just set a small, fixed number of threads by default and document 
better how to change it, as others suggested.

Brian

On 02/22/2017 08:49 PM, Wei, David wrote:
> Yes, as Brian said, at this stage,  patches are welcomed, including the python script.  You can also file bugs on https://bugzilla.tianocore.org/ for issue track and discussion.  Please remember to CC stakeholders.
>
> Specifically for the multi-thread building , I think maybe  NUMBER_OF_PROCESSORS Windows environment variable could be used as the proper thread number.
>
> Thanks,
> David  Wei
>
>
> -----Original Message-----
> From: Richardson, Brian
> Sent: Thursday, February 23, 2017 3:10 AM
> To: Rebecca Cran <rebecca@bluestop.org>; Gao, Liming <liming.gao@intel.com>; edk2-devel@lists.01.org
> Cc: Lu, ShifeiX A <shifeix.a.lu@intel.com>; Zimmer, Vincent <vincent.zimmer@intel.com>; Andrew Fish <afish@apple.com>; Wei, David <david.wei@intel.com>
> Subject: RE: [edk2] Improvements to build system etc. for edk2-platforms devel-MinnowBoard3?
>
> We're ok with help fixing issues (yay open source), so thanks for the help. Patches are welcome at this time. But we do track them in Bugzilla, so opening an issue there is the first step to a solution.
>
> For the processor thread setting, note that we have historically disabled processor threading by default because we don't know the build system configuration. While we at Intel prefer everyone own an Intel(R) Core(TM) i7 or Intel(R) Xeon processor, we know that's not the case ... so keeping it disabled has been seen as a safe option. We should definitely do a better job of documenting that setting change, but I think we need to consider generically changing that setting by default in target.txt (which requires a Bugzilla entry).
>
> The .bat/.sh files are required to trigger post-build tools, which are OS dependent. Even if the build was triggered by a Python script, we still need to do some verification to make sure there are no functional differences when we build in a Windows environment versus Linux (work in progress).
>
> Thanks ... br
> ---
> Brian Richardson, Senior Technical Marketing Engineer, Intel Software
> brian.richardson@intel.com -- @intel_Brian (Twitter & WeChat)
> https://software.intel.com/en-us/meet-the-developers/evangelists/team/brian-richardson
>
> -----Original Message-----
> From: edk2-devel [mailto:edk2-devel-bounces@lists.01.org] On Behalf Of Rebecca Cran
> Sent: Wednesday, February 22, 2017 11:53 AM
> To: Richardson, Brian <brian.richardson@intel.com>; Gao, Liming <liming.gao@intel.com>; edk2-devel@lists.01.org
> Cc: Lu, ShifeiX A <shifeix.a.lu@intel.com>; Zimmer, Vincent <vincent.zimmer@intel.com>; Andrew Fish <afish@apple.com>; Wei, David <david.wei@intel.com>
> Subject: Re: [edk2] Improvements to build system etc. for edk2-platforms devel-MinnowBoard3?
>
> On 2/22/2017 9:34 AM, Richardson, Brian wrote:
>> Thanks for the input. For future reference, you can use the TianoCore
>> Bugzilla to report issues on any EDK II feature/platform.
>> https://github.com/tianocore/tianocore.github.io/wiki/Reporting-Issues
>>
>> I agree the readme.md should be present, and use markup instead of plain text to work better in github. You can open an issue on this in Bugzilla.
>>
>> Normally, we ask folks to change the number of processor threads based on their system configuration. We don't add a larger thread number by default, but it might be good to set it '5' by default (assuming a dual core processor with hyperthreading) instead of '1' (assuming a single core system w/o threading). I don't know if this will cause any compatibility issues on older systems, but it's worth a check.
>>
>> At this time, MinnowBoard 3 build is only validated in Windows. That's why there is no equivalent .sh file for BuildBIOS yet, but it will be added once Linux build is verified and checked in.
>
> I'm more about _fixing_ issues I find rather than reporting them! Are you saying that patches wouldn't be welcome just now?  Is there a reason why you don't want to make full use of the CPU while building? And I understand that MinnowBoard 3 only builds under Windows at the moment, but if more of it built using python (and python is already listed as a prerequisite in the ReadMe file) the porting might be simpler.
>
> --
> Rebecca
> _______________________________________________
> edk2-devel mailing list
> edk2-devel@lists.01.org
> https://lists.01.org/mailman/listinfo/edk2-devel
> _______________________________________________
> edk2-devel mailing list
> edk2-devel@lists.01.org
> https://lists.01.org/mailman/listinfo/edk2-devel
>


-- 
Brian J. Johnson
Enterprise X86 Lab

Hewlett Packard Enterprise

brian.johnson@hpe.com
+1 651 683 7521  Office

Eagan, MN
hpe.com


^ permalink raw reply	[flat|nested] 12+ messages in thread

* Re: Improvements to build system etc. for edk2-platforms devel-MinnowBoard3?
  2017-02-23 18:53             ` Brian J. Johnson
@ 2017-02-23 19:10               ` Rebecca Cran
  2017-02-23 23:11                 ` Richardson, Brian
  0 siblings, 1 reply; 12+ messages in thread
From: Rebecca Cran @ 2017-02-23 19:10 UTC (permalink / raw)
  To: Brian J. Johnson, Wei, David, Richardson, Brian, Gao, Liming,
	edk2-devel@lists.01.org
  Cc: Lu, ShifeiX A, Zimmer, Vincent, Andrew Fish

On 2/23/2017 11:53 AM, Brian J. Johnson wrote:

> Sorry if I'm bikeshedding...  NUMBER_OF_PROCESSORS isn't a good 
> default for those of us who build on servers with hundreds of threads 
> available.  The OS, disks, and build.exe/build.py become bottlenecks.  
> Maybe we could put a cap (say, 20) on the default thread limit, so it 
> uses NUMBER_OF_PROCESSORS or 20 threads, whichever is less.
>
> Or just set a small, fixed number of threads by default and document 
> better how to change it, as others suggested.

Good point. I think the best approach will be to have a --setup 
parameter which configures defaults in Conf/target.txt, after which 
users can change MAX_CONCURRENT_THREAD_NUMBER - instead of the current 
method which involves figuring out which of files ends up running 
build.exe with '-n <threads>'.

-- 
Rebecca


^ permalink raw reply	[flat|nested] 12+ messages in thread

* Re: Improvements to build system etc. for edk2-platforms devel-MinnowBoard3?
  2017-02-23 19:10               ` Rebecca Cran
@ 2017-02-23 23:11                 ` Richardson, Brian
  0 siblings, 0 replies; 12+ messages in thread
From: Richardson, Brian @ 2017-02-23 23:11 UTC (permalink / raw)
  To: Rebecca Cran, Brian J. Johnson, Wei, David, Gao, Liming,
	edk2-devel@lists.01.org
  Cc: Lu, ShifeiX A, Zimmer, Vincent, Andrew Fish

One thing we can try is automating the right setting for MAX_CONCURRENT_THREAD_NUMBER. In Windows, there's a command line utility to return the number of logical processors.

	C:\Users\sample>wmic cpu get numberoflogicalprocessors
	NumberOfLogicalProcessors
	4 

This could be used to set MAX_CONCURRENT_THREAD_NUMBER (NumberOfLogicalProcessors+1, so 5 in this case) automagically in target.txt (via edksetup.bat) or as a command line parameter (in BuildBios.bat). I'm sure there's an equivalent shell command in Linux that can be co-opted for our purposes.

Thoughts?

Thanks ... br
---
Brian Richardson, Senior Technical Marketing Engineer, Intel Software
brian.richardson@intel.com -- @intel_Brian (Twitter & WeChat)
https://software.intel.com/en-us/meet-the-developers/evangelists/team/brian-richardson 

-----Original Message-----
From: Rebecca Cran [mailto:rebecca@bluestop.org] 
Sent: Thursday, February 23, 2017 2:10 PM
To: Brian J. Johnson <brian.johnson@hpe.com>; Wei, David <david.wei@intel.com>; Richardson, Brian <brian.richardson@intel.com>; Gao, Liming <liming.gao@intel.com>; edk2-devel@lists.01.org
Cc: Lu, ShifeiX A <shifeix.a.lu@intel.com>; Zimmer, Vincent <vincent.zimmer@intel.com>; Andrew Fish <afish@apple.com>
Subject: Re: [edk2] Improvements to build system etc. for edk2-platforms devel-MinnowBoard3?

On 2/23/2017 11:53 AM, Brian J. Johnson wrote:

> Sorry if I'm bikeshedding...  NUMBER_OF_PROCESSORS isn't a good 
> default for those of us who build on servers with hundreds of threads 
> available.  The OS, disks, and build.exe/build.py become bottlenecks.
> Maybe we could put a cap (say, 20) on the default thread limit, so it 
> uses NUMBER_OF_PROCESSORS or 20 threads, whichever is less.
>
> Or just set a small, fixed number of threads by default and document 
> better how to change it, as others suggested.

Good point. I think the best approach will be to have a --setup parameter which configures defaults in Conf/target.txt, after which users can change MAX_CONCURRENT_THREAD_NUMBER - instead of the current method which involves figuring out which of files ends up running build.exe with '-n <threads>'.

-- 
Rebecca


^ permalink raw reply	[flat|nested] 12+ messages in thread

end of thread, other threads:[~2017-02-23 23:11 UTC | newest]

Thread overview: 12+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2017-02-22  1:20 Improvements to build system etc. for edk2-platforms devel-MinnowBoard3? Rebecca Cran
2017-02-22  4:50 ` Gao, Liming
2017-02-22  5:11   ` Andrew Fish
2017-02-22  5:48   ` Rebecca Cran
2017-02-22  7:35     ` Gao, Liming
2017-02-22 16:34     ` Richardson, Brian
2017-02-22 16:52       ` Rebecca Cran
2017-02-22 19:10         ` Richardson, Brian
2017-02-23  2:49           ` Wei, David
2017-02-23 18:53             ` Brian J. Johnson
2017-02-23 19:10               ` Rebecca Cran
2017-02-23 23:11                 ` Richardson, Brian

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox