[poky] What's taking so long??

[Koen Kooi]

Op 20 jan 2011, om 16:58 heeft Joshua Lock het volgende geschreven:

> On Thu, 2011-01-20 at 08:11 -0700, Gary Thomas wrote:
>> My target platform/distro uses IPK and I have a local kernel
>> recipe which is based on the pre linux-yocto recipe that was
>> used by Poky.  When I build & package the kernel, I have
>> terrible performance - this is new, it used to be just fine.
>> 
>> Here's what I mean.  Building for my kernel and timing the
>> various steps, e.g.
>>   % time bitbake virtual/kernel -c compile
>> 
>> My kernel:
>>   compile            1m49s
>>   deploy             0m16s
>>   populate_sysroot   0m38s
>>   package           23m20s
>> 
>> Just to compare, using OTS Poky for BeagleBoard:
>>   compile            5m07s
>>   deploy             0m36s
>>   populate_sysroot   0m29s
>>   package            5m02s
>> 
>> That's a five-fold difference in the time to build & package
>> the kernel.
> 
> This is a significant performance impact. Thanks for taking the time to
> do some profiling.
> 
> I've filed a bug http://bugzilla.pokylinux.org/show_bug.cgi?id=666 and
> we'll get someone looking into that as a matter of priority.
> 
>> 
>> What am I doing [wrong?] that makes this so?
>> Would it be better if I switched to RPM like the BB is using?
>> 
> 
> It looks like the slow paths are in package format independent code, so
> probably not.

From a chat I had with RP 2 weeks ago:

Koen: there's still something slow in bitbake log output
Koen: do_package takes eons (> 10mins) with almost zero IO and CPU
Koen: ctrl-c'ing it shows it's in the installed-but-not-packaged output
Richard Purdie: poky's bitbake or upstream?
Koen: I only tried poky bitbake so far
Koen: upstream is slow as well, but I don't recall being so slow
Koen: we talked about it at ELC
Richard Purdie: My memory isn't remembering that 
Richard Purdie: This is do_package for the kernel or something large?
Koen: kernel
Koen: but I think anything would trigger it
Koen: it feels like bitbake is only doing N lines of output per second
Koen: effectively throttling it
Koen: and with you have a 4k lines out output....
Richard Purdie: The message handling means thread context switching
Richard Purdie: in runqueue.py there is a number 4096 and a 0 on the line below
Richard Purdie: try changing the 0 for something like 100000
Richard Purdie: You won't get nice console output but it might run faster
Richard Purdie: Was poky running "faster" until recently?
Koen: I think so
Koen: I haven't measured it

Does doing:

Richard Purdie: in runqueue.py there is a number 4096 and a 0 on the line below
Richard Purdie: try changing the 0 for something like 100000

help?

regards,

Koen