[poky] build performance: bb-matrix on 4-core (BB_NUMBER_THREADS and PARALLEL_MAKE optimization)

[Chris Larson]

On Sat, Jul 9, 2011 at 1:36 AM, Darren Hart <dvhart at linux.intel.com> wrote:
> On 07/08/2011 01:44 PM, Robert Berger wrote:
>> Darren/Richard,
>>
>> Maybe we could instead of hacking hard coded default values (or nothing)
>> into the config file default to something like this:
>>
>> somehow get the number of CPUs:
>>
>> CPUS=$(grep ^processor /proc/cpuinfo | wc -l)
>> echo CPUS=${CPUS}
>>
>> or
>>
>> CPUS=`getconf _NPROCESSORS_ONLN`
>> echo CPUS_UBUNTU=${CPUS_UBUNTU}
>>
>> (don't know if the second one will also work with other distros than Ubuntu)
>>
>> Do some calculation which magic number for BB_NUMBER_THREADS and
>> PARALLEL_MAKE to use:
>>
>> e.g. what was suggested: BB=2*NR_CORES PM=1.5*NR_CORES
>
> My concern with this is that on larger machines I'm seeing very
> different optimal multipliers. On my 12 core with a RAID 0 build array,
> the ideal setting seems to be BB=12 PM=12.
>
> Until we can better characterize the ideal settings, I think we are
> better off documenting what works for specific systems. Now perhaps we
> need to do something that caps the number, but that is sure to be wrong
> in short order as well.
>
> As your signature suggests, the solution to this isn't likely to be
> simple ;-)

This may be rather specific to my personal setup, but I use
https://gist.github.com/776390 -- you'll note that you can adjust the
scaling factors via variables.
-- 
Christopher Larson
clarson at kergoth dot com
Founder - BitBake, OpenEmbedded, OpenZaurus
Maintainer - Tslib
Senior Software Engineer, Mentor Graphics