[Toaster] [PATCH] toaster.bbclass: improve package information collection

[Smith, Elliot]

Looks good to me. Tested, and the package counts match for the cases I
tried.

Added to toaster-next and sent to OpenEmbedded mailing list (
openembedded-core at lists.openembedded.org).

Thanks.
Elliot

On 12 April 2016 at 09:56, Joshua Lock <joshua.g.lock at intel.com> wrote:

> The PACKAGES variable doesn't include all packages potentially
> generated by a recipe, not least of all because it doesn't include
> dynamic packages created via do_split_packages() or similar.
>
> Instead of trying to guess which packages were generated, walk
> ${PKGDESTWORK}/runtime to find all package information that was
> written to disk. This allows us to read all information for the
> complete list of packages generated by the recipe.
>
> For example before this patch we get SinglePackageInfo events for
> 9 packages from ncurses. With the patch applied we get 20 events
> for all of the packages created during an ncurses build.
>
> As a bonus we also switch to using the same postfuncs for both
> do_packagedata and do_packagedata_setscene as they each result
> in a PKGDESTWORK with the relevant directories and files
> created.
>
> [YOCTO #9115]
>
> Signed-off-by: Joshua Lock <joshua.g.lock at intel.com>
> ---
>  meta/classes/toaster.bbclass | 50
> +++++++++++++-------------------------------
>  1 file changed, 15 insertions(+), 35 deletions(-)
>
> diff --git a/meta/classes/toaster.bbclass b/meta/classes/toaster.bbclass
> index 004e068..1a70f14 100644
> --- a/meta/classes/toaster.bbclass
> +++ b/meta/classes/toaster.bbclass
> @@ -112,45 +112,25 @@ def _toaster_load_pkgdatafile(dirpath, filepath):
>                  pass    # ignore lines without valid key: value pairs
>      return pkgdata
>
> -python toaster_package_dumpdata_setscene() {
> -    """
> -    Dumps the data created by package_setscene
> -    """
> -    # replicate variables from the package.bbclass
> -    packages = d.getVar('PACKAGES', True)
> -    pkgdatadir = d.getVar('PKGDATA_DIR', True)
> -    # scan and send data for each package
> -    lpkgdata = {}
> -    for pkg in packages.split():
> -        try:
> -          lpkgdata = _toaster_load_pkgdatafile(pkgdatadir + "/runtime/",
> pkg)
> -        except:
> -          # these are typically foo-locale which actually point into
> foo-locale-<language> in runtime-rprovides
> -          bb.note("toaster_package_dumpdata_setscene: failed to load pkg
> information for: %s:%s"%(pkg,sys.exc_info()[0]))
> -        # Fire an event containing the pkg data
> -        bb.event.fire(bb.event.MetadataEvent("SinglePackageInfo",
> lpkgdata), d)
> -
> -}
> -
> -
>  python toaster_package_dumpdata() {
>      """
> -    Dumps the data created by emit_pkgdata
> +    Dumps the data about the packages created by a recipe
>      """
> -    # replicate variables from the package.bbclass
> -
> -    packages = d.getVar('PACKAGES', True)
> -    pkgdatadir = d.getVar('PKGDESTWORK', True)
>
> -    # scan and send data for each package
> +    # No need to try and dumpdata if the recipe isn't generating packages
> +    if not d.getVar('PACKAGES', True):
> +        return
>
> +    pkgdatadir = d.getVar('PKGDESTWORK', True)
>      lpkgdata = {}
> -    for pkg in packages.split():
> -
> -        lpkgdata = _toaster_load_pkgdatafile(pkgdatadir + "/runtime/",
> pkg)
> -
> -        # Fire an event containing the pkg data
> -        bb.event.fire(bb.event.MetadataEvent("SinglePackageInfo",
> lpkgdata), d)
> +    datadir = os.path.join(pkgdatadir, 'runtime')
> +
> +    # scan and send data for each generated package
> +    for datafile in os.listdir(datadir):
> +        if not datafile.endswith('.packaged'):
> +            lpkgdata = _toaster_load_pkgdatafile(datadir, datafile)
> +            # Fire an event containing the pkg data
> +            bb.event.fire(bb.event.MetadataEvent("SinglePackageInfo",
> lpkgdata), d)
>  }
>
>  # 2. Dump output image files information
> @@ -401,8 +381,8 @@ toaster_collect_task_stats[eventmask] =
> "bb.event.BuildCompleted bb.build.TaskSu
>  addhandler toaster_buildhistory_dump
>  toaster_buildhistory_dump[eventmask] = "bb.event.BuildCompleted"
>
> -do_packagedata_setscene[postfuncs] += "toaster_package_dumpdata_setscene "
> -do_packagedata_setscene[vardepsexclude] +=
> "toaster_package_dumpdata_setscene "
> +do_packagedata_setscene[postfuncs] += "toaster_package_dumpdata "
> +do_packagedata_setscene[vardepsexclude] += "toaster_package_dumpdata "
>
>  do_package[postfuncs] += "toaster_package_dumpdata "
>  do_package[vardepsexclude] += "toaster_package_dumpdata "
> --
> 2.5.5
>
> --
> _______________________________________________
> toaster mailing list
> toaster at yoctoproject.org
> https://lists.yoctoproject.org/listinfo/toaster
>



-- 
Elliot Smith
Software Engineer
Intel Open Source Technology Centre
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.yoctoproject.org/pipermail/toaster/attachments/20160412/b9ea82b3/attachment-0001.html>