[poky] do_image_wic cross-recipe task dependency affects recipe-sysroot

[Jean-Francois Dagenais]

Hi all,

I have a image recipe (myimage.bb with Wic FSTYPE) which will need to bundle a file built by another recipe (let's say myrecipe.bb which inherits swupdate.bbclass) in the boot partition.

The boot partition is made by bootimg-partition.py through the wks file.

I list my file (myrecipe.swu) in IMAGE_BOOT_FILES so bootimg-partition will add it.

If I add "myrecipe" in EXTRA_IMAGEDEPENDS like one would think this was made for, a race occurs where there is no guarantee that myrecipe.swu will be ready in DEPLOYDIR when myimage:do_image_wic runs since EXTRA_IMAGEDEPENDS attaches myrecipe:do_populate_sysroot to myimage.do_image_complete. Thing is, myimage:do_image_complete depends on both do_image_wic and myrecipe:do_populate_sysroot equally, hence the race.

This can be easily demonstrated by requesting "bitbake myimage -c do_image_wic" which, if myrecipy.swu is not already present in DEPLOYDIR, will fail 100% of the time.

So here's my workaround which has worked well for quite a while:

In myimage.bb:

do_image_wic[depends] += "myrecipe:do_image_swu"

This works 100% of the time in that myrecipe.swu is always present before do_image_wic starts... GREAT!

Recently we've started having sporadic build issues. I am trying to understand what is going on:


NOTE: recipe myimage-1.0-r0: task do_image_wic: Started
ERROR: myimage-1.0-r0 do_image_wic: Error executing a python function in exec_python_func() autogenerated:

The stack trace of python calls that resulted in this exception/failure was:
File: 'exec_python_func() autogenerated', lineno: 2, function: <module>
     0001:
 *** 0002:extend_recipe_sysroot(d)
     0003:
File: '/builds/soft/yocto-mymachine/poky/meta/classes/staging.bbclass', lineno: 553, function: extend_recipe_sysroot
     0549:                    dest = newmanifest[l]
     0550:                    if l.endswith("/"):
     0551:                        staging_copydir(l, targetdir, dest, seendirs)
     0552:                        continue
 *** 0553:                    staging_copyfile(l, targetdir, dest, postinsts, seendirs)
     0554:
     0555:    bb.note("Installed into sysroot: %s" % str(msg_adding))
     0556:    bb.note("Skipping as already exists in sysroot: %s" % str(msg_exists))
     0557:
File: '/builds/soft/yocto-mymachine/poky/meta/classes/staging.bbclass', lineno: 151, function: staging_copyfile
     0147:        os.symlink(linkto, dest)
     0148:        #bb.warn(c)
     0149:    else:
     0150:        try:
 *** 0151:            os.link(c, dest)
     0152:        except OSError as err:
     0153:            if err.errno == errno.EXDEV:
     0154:                bb.utils.copyfile(c, dest)
     0155:            else:
Exception: FileExistsError: [Errno 17] File exists: '/cache/build/mymachine/tmp/sysroots-components/aarch64/libgcc/usr/lib/aarch64-mydistro-linux/8.2.0/crtbeginT.o' -> '/cache/build/mymachine/tmp/work/mymachine-mydistro-linux/myimage/1.0-r0/recipe-sysroot/usr/lib/aarch64-mydistro-linux/8.2.0/crtbeginT.o'




Examining what is being installed by extend_recipe_sysroot, I realised that by doing do_image_wic[depends] += "myrecipe:do_image_swu", the recipe-sysroot directory of myimage becomes huge. In fact, it seems that all the recursive target dependencies of myrecipe.bb (and the rootfs it bundles) are installed in myimage.bb's workdir!! Probably ultimately causing the conflicts I see above.

These artifacts installed in myimage's recipe-sysroot, are completely useless to the work that myimage has to do (do_rootfs, do_image_wic, etc.)

So back to my title: Is there a way to add a simple execution dependency between a task in one recipe and a task in another recipe, without affecting the dependent's recipe-sysroot?

Thanks for the help!

P.S.: Oh, and I tried changing myrecipe.bb (swu) for another, simpler, less deep recipe which also has a do_deploy step and outputs a file in deploy dir. I do indeed get much less useless things added to myimage's recipe-sysroot, but I still get the other recipe's dependencies.