[poky] Rough timing of rpm vs opkg rootfs builds

[Richard Purdie]

On Thu, 2010-11-11 at 14:30 -0800, Saul Wold wrote:
> Mark, Qing, Dongxiao:
> 
> Richard and I were talking the other day and I started a little 
> experiment with checking the timing of RPM rootfs build vs OPKG. For the 
> minimal and SDK images:
> 
> Minimal - OPKG
> real	1m39.456s
> user	1m18.693s
> sys	0m4.188s
> 
> SDK - OPKG
> real	10m50.784s
> user	6m8.059s
> sys	0m51.523s
> 
> Minimal - RPM
> real	4m25.166s
> user	6m14.503s
> sys	0m27.534s
> 
> SDK - RPM
> real	24m40.979s
> user	7m29.856s
> sys	4m25.561s
> 
> Clearly there is some work we can do with RPM, which takes more than 
> double the time, yes, I know it's doing more work, but maybe there are 
> some optimizations that can be done to improve the speed.

Actually, I'm not sure RPM should be doing much more work here.

> For the autobuilder, we build 7 SDK images, which is about 3 hours using 
> RPM vs about 75 minutes for OPKG, the Sato and LSB images are not much 
> faster so this is another place we should be looking to help our build time.
> 
> This is just another place for us to open dialog and figure out what's 
> going on.

This is good, thanks for the info.

What would be more interesting again is a breakdown of this to see how
much time is spent indexing the packages and how much is actual rootfs
generation with rpm.

I've worked on optimising this time for opkg before. The trick was to
make the package indexing incremental rather than reparsing every
package, every time which is slow. I therefore taught the package
indexer just to look at the timestamps and size of the packages, if they
change it will reindex them, if they are the same, the previous result
will be used. You need to be careful to notice when packages are removed
or added.

I suspect there is an optimisation that can be added for the rpm
indexing to make this incremental updating possible.

We might also want to also not do the package indexing under pseudo as
another idea to get some speed.

Cheers,

Richard