[poky] What's taking so long??

[Richard Purdie]

On Thu, 2011-01-20 at 08:11 -0700, Gary Thomas wrote:
> My target platform/distro uses IPK and I have a local kernel
> recipe which is based on the pre linux-yocto recipe that was
> used by Poky.  When I build & package the kernel, I have
> terrible performance - this is new, it used to be just fine.
> 
> Here's what I mean.  Building for my kernel and timing the
> various steps, e.g.
>    % time bitbake virtual/kernel -c compile
> 
> My kernel:
>    compile            1m49s
>    deploy             0m16s
>    populate_sysroot   0m38s
>    package           23m20s
> 
> Just to compare, using OTS Poky for BeagleBoard:
>    compile            5m07s
>    deploy             0m36s
>    populate_sysroot   0m29s
>    package            5m02s
> 
> That's a five-fold difference in the time to build & package
> the kernel.

This is a serious and worrying number. I just tried a test where I ran:

bitbake virtual/kernel -c package | ../predate.sh 

where predate.sh contains:

#!/bin/bash
while read line ; do
    echo "$(date): ${line}"
done

and I added a bb.error to the package functions interation loop in
package.bbclass.

I saw:

Fri Jan 21 13:58:00 GMT 2011: NOTE: Running task 667 of 667 (ID: 14, /media/build2/builds/rptest/meta/recipes-kernel/linux/linux-yocto_git.bb, do_package)
Fri Jan 21 13:58:00 GMT 2011: NOTE: package linux-yocto-2.6.37+git0+e3895db37bd65ea97e59b70ea936c84b398d3f3e_0+a9d833fda90e2f1257888a97e092135610b5f259-r15: task do_package: Started
Fri Jan 21 13:58:00 GMT 2011: ERROR: Running perform_packagecopy
Fri Jan 21 13:58:02 GMT 2011: ERROR: Running package_do_split_locales
Fri Jan 21 13:58:02 GMT 2011: ERROR: Running populate_packages
Fri Jan 21 13:58:02 GMT 2011: WARNING: <string>:5: RuntimeWarning: tmpnam is a potential security risk to your program
Fri Jan 21 13:58:02 GMT 2011: 
Fri Jan 21 13:58:18 GMT 2011: ERROR: Running package_do_filedeps
Fri Jan 21 13:59:08 GMT 2011: ERROR: Running package_do_shlibs
Fri Jan 21 13:59:10 GMT 2011: ERROR: Running package_do_pkgconfig
Fri Jan 21 13:59:10 GMT 2011: ERROR: Running read_shlibdeps
Fri Jan 21 13:59:11 GMT 2011: ERROR: Running package_depchains
Fri Jan 21 13:59:11 GMT 2011: ERROR: Running emit_pkgdata
Fri Jan 21 13:59:11 GMT 2011: ERROR: Running do_package_qa
Fri Jan 21 14:00:42 GMT 2011: NOTE: package linux-yocto-2.6.37+git0+e3895db37bd65ea97e59b70ea936c84b398d3f3e_0+a9d833fda90e2f1257888a97e092135610b5f259-r15: task do_package: Succeeded
Fri Jan 21 14:00:42 GMT 2011: NOTE: Tasks Summary: Attempted 667 tasks of which 615 didn't need to be rerun and 0 failed.

so it did this in 2.6 minutes which doesn't seem unreasonable.

Are you seeing the slow package step in current master and when you made
the timings above, were they the only task running or were there others
active too?

Cheers,

Richard