6 Tasks

Tasks are units of execution for BitBake. Recipes (.bb files) use tasks to complete configuring, compiling, and packaging software. This chapter provides a reference of the tasks defined in the OpenEmbedded build system.

6.1 Normal Recipe Build Tasks

The following sections describe normal tasks associated with building a recipe. For more information on tasks and dependencies, see the “Tasks” and “Dependencies” sections in the BitBake User Manual.

6.1.1 do_build

The default task for all recipes. This task depends on all other normal tasks required to build a recipe.

6.1.2 do_compile

Compiles the source code. This task runs with the current working directory set to ${B}.

The default behavior of this task is to run the oe_runmake function if a makefile (Makefile, makefile, or GNUmakefile) is found. If no such file is found, the do_compile task does nothing.

6.1.3 do_compile_ptest_base

Compiles the runtime test suite included in the software being built.

6.1.4 do_configure

Configures the source by enabling and disabling any build-time and configuration options for the software being built. The task runs with the current working directory set to ${B}.

The default behavior of this task is to run oe_runmake clean if a makefile (Makefile, makefile, or GNUmakefile) is found and CLEANBROKEN is not set to “1”. If no such file is found or the CLEANBROKEN variable is set to “1”, the do_configure task does nothing.

6.1.5 do_configure_ptest_base

Configures the runtime test suite included in the software being built.

6.1.6 do_deploy

Writes output files that are to be deployed to ${DEPLOY_DIR_IMAGE}. The task runs with the current working directory set to ${B}.

Recipes implementing this task should inherit the deploy class and should write the output to ${DEPLOYDIR}, which is not to be confused with ${DEPLOY_DIR}. The deploy class sets up do_deploy as a shared state (sstate) task that can be accelerated through sstate use. The sstate mechanism takes care of copying the output from ${DEPLOYDIR} to ${DEPLOY_DIR_IMAGE}.

Note

Do not write the output directly to ${DEPLOY_DIR_IMAGE}, as this causes the sstate mechanism to malfunction.

The do_deploy task is not added as a task by default and consequently needs to be added manually. If you want the task to run after do_compile, you can add it by doing the following:

addtask deploy after do_compile

Adding do_deploy after other tasks works the same way.

Note

You do not need to add before do_build to the addtask command (though it is harmless), because the base class contains the following:

do_build[recrdeptask] += "do_deploy"

See the “Dependencies” section in the BitBake User Manual for more information.

If the do_deploy task re-executes, any previous output is removed (i.e. “cleaned”).

6.1.7 do_fetch

Fetches the source code. This task uses the SRC_URI variable and the argument’s prefix to determine the correct fetcher module.

6.1.8 do_image

Starts the image generation process. The do_image task runs after the OpenEmbedded build system has run the do_rootfs task during which packages are identified for installation into the image and the root filesystem is created, complete with post-processing.

The do_image task performs pre-processing on the image through the IMAGE_PREPROCESS_COMMAND and dynamically generates supporting do_image_* tasks as needed.

For more information on image creation, see the “Image Generation” section in the Yocto Project Overview and Concepts Manual.

6.1.9 do_image_complete

Completes the image generation process. The do_image_complete task runs after the OpenEmbedded build system has run the do_image task during which image pre-processing occurs and through dynamically generated do_image_* tasks the image is constructed.

The do_image_complete task performs post-processing on the image through the IMAGE_POSTPROCESS_COMMAND.

For more information on image creation, see the “Image Generation” section in the Yocto Project Overview and Concepts Manual.

6.1.10 do_install

Copies files that are to be packaged into the holding area ${D}. This task runs with the current working directory set to ${B}, which is the compilation directory. The do_install task, as well as other tasks that either directly or indirectly depend on the installed files (e.g. do_package, do_package_write_*, and do_rootfs), run under fakeroot.

Note

When installing files, be careful not to set the owner and group IDs of the installed files to unintended values. Some methods of copying files, notably when using the recursive cp command, can preserve the UID and/or GID of the original file, which is usually not what you want. The host-user-contaminated QA check checks for files that probably have the wrong ownership.

Safe methods for installing files include the following:

  • The install utility. This utility is the preferred method.

  • The cp command with the --no-preserve=ownership option.

  • The tar command with the --no-same-owner option. See the bin_package.bbclass file in the meta/classes-recipe subdirectory of the Source Directory for an example.

6.1.11 do_install_ptest_base

Copies the runtime test suite files from the compilation directory to a holding area.

6.1.12 do_package

Analyzes the content of the holding area ${D} and splits the content into subsets based on available packages and files. This task makes use of the PACKAGES and FILES variables.

The do_package task, in conjunction with the do_packagedata task, also saves some important package metadata. For additional information, see the PKGDESTWORK variable and the “Automatically Added Runtime Dependencies” section in the Yocto Project Overview and Concepts Manual.

6.1.13 do_package_qa

Runs QA checks on packaged files. For more information on these checks, see the insane class.

6.1.14 do_package_write_deb

Creates Debian packages (i.e. *.deb files) and places them in the ${DEPLOY_DIR_DEB} directory in the package feeds area. For more information, see the “Package Feeds” section in the Yocto Project Overview and Concepts Manual.

6.1.15 do_package_write_ipk

Creates IPK packages (i.e. *.ipk files) and places them in the ${DEPLOY_DIR_IPK} directory in the package feeds area. For more information, see the “Package Feeds” section in the Yocto Project Overview and Concepts Manual.

6.1.16 do_package_write_rpm

Creates RPM packages (i.e. *.rpm files) and places them in the ${DEPLOY_DIR_RPM} directory in the package feeds area. For more information, see the “Package Feeds” section in the Yocto Project Overview and Concepts Manual.

6.1.17 do_packagedata

Saves package metadata generated by the do_package task in PKGDATA_DIR to make it available globally.

6.1.18 do_patch

Locates patch files and applies them to the source code.

After fetching and unpacking source files, the build system uses the recipe’s SRC_URI statements to locate and apply patch files to the source code.

Note

The build system uses the FILESPATH variable to determine the default set of directories when searching for patches.

Patch files, by default, are *.patch and *.diff files created and kept in a subdirectory of the directory holding the recipe file. For example, consider the bluez5 recipe from the OE-Core layer (i.e. poky/meta):

poky/meta/recipes-connectivity/bluez5

This recipe has two patch files located here:

poky/meta/recipes-connectivity/bluez5/bluez5

In the bluez5 recipe, the SRC_URI statements point to the source and patch files needed to build the package.

Note

In the case for the bluez5_5.48.bb recipe, the SRC_URI statements are from an include file bluez5.inc.

As mentioned earlier, the build system treats files whose file types are .patch and .diff as patch files. However, you can use the “apply=yes” parameter with the SRC_URI statement to indicate any file as a patch file:

SRC_URI = " \
    git://path_to_repo/some_package \
    file://file;apply=yes \
    "

Conversely, if you have a file whose file type is .patch or .diff and you want to exclude it so that the do_patch task does not apply it during the patch phase, you can use the “apply=no” parameter with the SRC_URI statement:

SRC_URI = " \
    git://path_to_repo/some_package \
    file://file1.patch \
    file://file2.patch;apply=no \
    "

In the previous example file1.patch would be applied as a patch by default while file2.patch would not be applied.

You can find out more about the patching process in the “Patching” section in the Yocto Project Overview and Concepts Manual and the “Patching Code” section in the Yocto Project Development Tasks Manual.

6.1.19 do_populate_lic

Writes license information for the recipe that is collected later when the image is constructed.

6.1.20 do_populate_sdk

Creates the file and directory structure for an installable SDK. See the “SDK Generation” section in the Yocto Project Overview and Concepts Manual for more information.

6.1.21 do_populate_sdk_ext

Creates the file and directory structure for an installable extensible SDK (eSDK). See the “SDK Generation” section in the Yocto Project Overview and Concepts Manual for more information.

6.1.22 do_populate_sysroot

Stages (copies) a subset of the files installed by the do_install task into the appropriate sysroot. For information on how to access these files from other recipes, see the STAGING_DIR* variables. Directories that would typically not be needed by other recipes at build time (e.g. /etc) are not copied by default.

For information on what directories are copied by default, see the SYSROOT_DIRS* variables. You can change these variables inside your recipe if you need to make additional (or fewer) directories available to other recipes at build time.

The do_populate_sysroot task is a shared state (sstate) task, which means that the task can be accelerated through sstate use. Realize also that if the task is re-executed, any previous output is removed (i.e. “cleaned”).

6.1.23 do_prepare_recipe_sysroot

Installs the files into the individual recipe specific sysroots (i.e. recipe-sysroot and recipe-sysroot-native under ${WORKDIR} based upon the dependencies specified by DEPENDS). See the “staging” class for more information.

6.1.24 do_rm_work

Removes work files after the OpenEmbedded build system has finished with them. You can learn more by looking at the “rm_work” section.

6.1.25 do_unpack

Unpacks the source code into a working directory pointed to by ${WORKDIR}. The S variable also plays a role in where unpacked source files ultimately reside. For more information on how source files are unpacked, see the “Source Fetching” section in the Yocto Project Overview and Concepts Manual and also see the WORKDIR and S variable descriptions.

6.2 Manually Called Tasks

These tasks are typically manually triggered (e.g. by using the bitbake -c command-line option):

6.2.1 do_checkuri

Validates the SRC_URI value.

6.2.2 do_clean

Removes all output files for a target from the do_unpack task forward (i.e. do_unpack, do_configure, do_compile, do_install, and do_package).

You can run this task using BitBake as follows:

$ bitbake -c clean recipe

Running this task does not remove the sstate cache files. Consequently, if no changes have been made and the recipe is rebuilt after cleaning, output files are simply restored from the sstate cache. If you want to remove the sstate cache files for the recipe, you need to use the do_cleansstate task instead (i.e. bitbake -c cleansstate recipe).

6.2.3 do_cleanall

Removes all output files, shared state (sstate) cache, and downloaded source files for a target (i.e. the contents of DL_DIR). Essentially, the do_cleanall task is identical to the do_cleansstate task with the added removal of downloaded source files.

You can run this task using BitBake as follows:

$ bitbake -c cleanall recipe

You should never use the do_cleanall task in a normal scenario. If you want to start fresh with the do_fetch task, use instead:

$ bitbake -f -c fetch recipe

Note

The reason to prefer bitbake -f -c fetch is that the do_cleanall task would break in some cases, such as:

$ bitbake -c fetch    recipe
$ bitbake -c cleanall recipe-native
$ bitbake -c unpack   recipe

because after step 1 there is a stamp file for the do_fetch task of recipe, and it won’t be removed at step 2 because step 2 uses a different work directory. So the unpack task at step 3 will try to extract the downloaded archive and fail as it has been deleted in step 2.

Note that this also applies to BitBake from concurrent processes when a shared download directory (DL_DIR) is setup.

6.2.4 do_cleansstate

Removes all output files and shared state (sstate) cache for a target. Essentially, the do_cleansstate task is identical to the do_clean task with the added removal of shared state (sstate) cache.

You can run this task using BitBake as follows:

$ bitbake -c cleansstate recipe

When you run the do_cleansstate task, the OpenEmbedded build system no longer uses any sstate. Consequently, building the recipe from scratch is guaranteed.

Note

Using do_cleansstate with a shared SSTATE_DIR is not recommended because it could trigger an error during the build of a separate BitBake instance. This is because the builds check sstate “up front” but download the files later, so it if is deleted in the meantime, it will cause an error but not a total failure as it will rebuild it.

The reliable and preferred way to force a new build is to use bitbake -f instead.

Note

The do_cleansstate task cannot remove sstate from a remote sstate mirror. If you need to build a target from scratch using remote mirrors, use the “-f” option as follows:

$ bitbake -f -c do_cleansstate target

6.2.5 do_pydevshell

Starts a shell in which an interactive Python interpreter allows you to interact with the BitBake build environment. From within this shell, you can directly examine and set bits from the data store and execute functions as if within the BitBake environment. See the “Using a Python Development Shell” section in the Yocto Project Development Tasks Manual for more information about using pydevshell.

6.2.6 do_devshell

Starts a shell whose environment is set up for development, debugging, or both. See the “Using a Development Shell” section in the Yocto Project Development Tasks Manual for more information about using devshell.

6.2.7 do_listtasks

Lists all defined tasks for a target.

6.2.8 do_package_index

Creates or updates the index in the Package Feeds area.

Note

This task is not triggered with the bitbake -c command-line option as are the other tasks in this section. Because this task is specifically for the package-index recipe, you run it using bitbake package-index.