[poky] SRC checksum checking broken

[Richard Purdie]

On Tue, 2012-04-17 at 16:14 -0600, Gary Thomas wrote:
> On 2012-04-17 15:20, Richard Purdie wrote:
> > On Tue, 2012-04-17 at 10:59 -0600, Gary Thomas wrote:
> >> On 2012-04-17 10:19, Richard Purdie wrote:
> >> Well, it seems that the checksums are encouraged, if not required, these days.
> >> I was just making sure that they actually work&  in my opinion, it shouldn't
> >> matter how you got the file, the checksums should still be checked and match.
> >
> > You can argue this both ways. I'm leaning that direction although it has
> > its risks.
> >
> >> Sadly, this is much worse.  It even [somehow] reached "through" my
> >> mirror setup and deleted the file on the backing store (i.e. the
> >> file /work/misc/Poky/sources/busybox-1.19.4.tar.bz2 gets deleted
> >> when the checksum fails even though it's part of the own-mirror)
> >
> > Was that the only issue? I can see how it managed to do that, I agree it
> > shouldn't :/.
> 
> It sort of worked - at least it didn't pass silently - but the messages
> don't say why it failed.  Here's what I get:
> 
> NOTE: package busybox-1.19.4-r2: task do_fetch: Started
> WARNING: Failed to fetch URL http://www.busybox.net/downloads/busybox-1.19.4.tar.bz2;name=tarball
> ERROR: Fetcher failure: Fetch command export HOME="/home/gthomas"; export GIT_CONFIG="/home/local/poky_test/tmp/sysroots/i686-linux/etc/gitconfig"; export 
> PATH="/home/local/poky_test/tmp/sysroots/i686-linux/usr/bin/armv7a-vfp-neon-poky-linux-gnueabi:/home/local/poky_test/tmp/sysroots/beagleboard/usr/bin/crossscripts:/home/local/poky_test/tmp/sysroots/i686-linux/usr/sbin:/home/local/poky_test/tmp/sysroots/i686-linux/usr/bin:/home/local/poky_test/tmp/sysroots/i686-linux/sbin:/home/local/poky_test/tmp/sysroots/i686-linux//bin:/home/local/poky-master/scripts:/home/local/poky-master/bitbake/bin/:/opt/amltd/bin:/usr/java/jdk1.6.0_10/bin:/home/gthomas/Android/android-sdk-linux_x86-1.1_r1/tools:/home/gthomas/bin:/usr/lib/qt-3.3/bin:/usr/kerberos/sbin:/usr/kerberos/bin:/usr/lib/ccache:/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin:/home/local/poky-master/scripts"; 
> /usr/bin/env wget -t 5 -nv --passive-ftp --no-check-certificate -P /home/local/poky_test/downloads 'http://www.busybox.net/downloads/busybox-1.19.4.tar.bz2' could not be run:
> None
> ERROR: Function failed: Fetcher failure for URL: 'http://www.busybox.net/downloads/busybox-1.19.4.tar.bz2;name=tarball'. Unable to fetch URL from any source.
> ERROR: Logfile of failure stored in: /home/local/poky_test/tmp/work/armv7a-vfp-neon-poky-linux-gnueabi/busybox-1.19.4-r2/temp/log.do_fetch.7907
> NOTE: package busybox-1.19.4-r2: task do_fetch: Failed
> ERROR: Task 2 (/home/local/poky-master/meta/recipes-core/busybox/busybox_1.19.4.bb, do_fetch) failed with exit code '1'
> NOTE: Tasks Summary: Attempted 8 tasks of which 7 didn't need to be rerun and 1 failed.
> 
> No indication of checksum failures anywhere.

Having gone over this several times, I think the original patch is
correct but it highlights some other issues. The nastiest of those is
the deletion of files. I think I've found a way of fixing that by
correctly using fetcher clean functions (which are null operations for
the local fetcher).

The remaining issue is then one of better output from the fetcher as
there wasn't enough info in the above logs but that is really a separate
problem.

I've sent out several fetcher patches to the bitbake list which will
hopefully address some of these issues. I'm travelling at the moment and
these were written without internet access so my testing is limited to
the local file:// urls but I'm confident they move things in the right
direction.

Cheers,

Richard