[poky] summary about recent do_populate_sdk failures

[Richard Purdie]

On Mon, 2011-01-31 at 15:18 +0800, Tian, Kevin wrote:
> Lianhao and I are looking into recent master instability, more specifically starting
> from reported do_populate_sdk failures. There have been 3 problems revealed:
> 
> 1) unexpected do_package/do_package_write* rebuild
> 2) change PACKAGE_CLASSES causes more do_package rebuilds
> 3) do_populate_sdk finally exits due to eglibc not matching expected version
> 
> We've root caused 1), and the patch has been sent out:
> http://git.pokylinux.org/cgit/cgit.cgi/poky-contrib/commit/?h=tk/master1&id=68ad8ead1a83161afb8c2a65a28dfc205181d80e
> 
> We're not sure whether 3) is caused by too many unexpected do_package rebuilds,
> and is now testing whether 3) will disappear with the fix for 1). We'll also look into
> original failure to see whether real cause of 3) may be hidden with the fix.
> 
> Now the remaining open is about 2). We got the reason, but would like to welcome
> Comments on an elegant solution.
> 
> The phenomenon is that do_package checksum changes with switching the order
> In PACKAGE_CLASSES, e.g:
> 
> From
> PACKAGE_CLASSES = "package_rpm package_ipk"
> To
> PACKAGE_CLASSES = "package_ipk package_rpm"
> 
> This is undesired since the order only matters for final rootfs generation. The actual
> cause is related to how do_package is generated, which depends on when 
> package.bbclass is first brought in.
> 
> Take package_rpm.bbclass for example, which inherit package.bbclass. Bitbake
> generates below wrappers implicitly:
> 
> Do_package () {
> 	bb.build.exec_func('package_rpm_do_package', d)
> }
> 
> Package_rpm_do_package() {
> 	bb.build.exec_func('package_do_package', d)
> }
> 
> Above is implicitly created when bitbake parses class files based on inheritance tree,
> and once a class file is parsed it's then saved to the cache. When later package_ipk is
> parsed, bitbake will simply read from the cache for package.bbclass and then
> do_package remains tying to package_rpm_do_package.
> 
> If package_ipk is the 1st in PACKAGE_CLASSES, then do_package becomes:
> 
> Do_package () {
> 	bb.build.exec_func('package_ipk_do_package', d)
> }
> 
> This sure generates different checksum to break sstate installation, but in reality it's
> just name difference, no actual functional change. We considered to use vardeps and 
> varexcldeps which don't help though since it's the do_package itself changed.
> 
> One possible option is to sort the INHERIT list, which ensures we always parse
> Inherited class files in same order if no content change. We'll test whether this works,
> and also hear your comments.

For 2), I suggest we drop the EXPORT_FUNCTIONS for do_package since we
never actually use that functionality, there are other ways to customise
packaging. The patch would be:

diff --git a/meta/classes/package.bbclass b/meta/classes/package.bbclass
index 7e1f3f0..856858c 100644
--- a/meta/classes/package.bbclass
+++ b/meta/classes/package.bbclass
@@ -1065,7 +1065,7 @@ PACKAGEFUNCS ?= "perform_packagecopy \
                package_depchains \
                emit_pkgdata"
 
-python package_do_package () {
+python do_package () {
        packages = (bb.data.getVar('PACKAGES', d, True) or "").split()
        if len(packages) < 1:
                bb.debug(1, "No packages to build, skipping do_package")
@@ -1110,8 +1110,6 @@ do_package_write[noexec] = "1"
 do_build[recrdeptask] += "do_package_write"
 addtask package_write before do_build after do_package
 
-EXPORT_FUNCTIONS do_package do_package_write
-
 #
 # Helper functions for the package writing classes
 #