The first thing you need to do is set up the Yocto Project build environment by sourcing the environment setup script as follows:

     $ source oe-init-build-env [build_dir]
            

The build_dir is optional and specifies the directory Yocto Project uses for the build. If you do not specify a build directory it defaults to build in your current working directory. A common practice is to use a different build directory for different targets. For example, ~/build/x86 for a qemux86 target, and ~/build/arm for a qemuarm target. See oe-init-build-env for more information on this script.

Once the Yocto Project build environment is set up, you can build a target using:

     $ bitbake <target>
            

The target is the name of the recipe you want to build. Common targets are the images in meta/recipes-core/images, /meta/recipes-sato/images, etc. all found in the Yocto Project files. Or, the target can be the name of a recipe for a specific piece of software such as busybox. For more details about the images the Yocto Project supports, see the "Reference: Images" appendix.

When building an image using GPL components, you need to maintain your original settings and not switch back and forth applying different versions of the GNU Public License. If you rebuild using different versions of GPL, dependency errors might occur due to some components not being rebuilt.

The log file for shell tasks is available in ${WORKDIR}/temp/log.do_taskname.pid. For example, the compile task for the QEMU minimal image for the x86 machine (qemux86) might be tmp/work/qemux86-poky-linux/core-image-minimal-1.0-r0/temp/log.do_compile.20830. To see what BitBake runs to generate that log, look at the corresponding run.do_taskname.pid file located in the same directory.

Presently, the output from Python tasks is sent directly to the console.

Any given package consists of a set of tasks. The standard BitBake behavior in most cases is: fetch, unpack, patch, configure, compile, install, package, package_write, and build. The default task is build and any tasks on which it depends build first. Some tasks exist, such as devshell, that are not part of the default build chain. If you wish to run a task that is not part of the default build chain, you can use the -c option in BitBake as follows:

     $ bitbake matchbox-desktop -c devshell
            

If you wish to rerun a task, use the -f force option. For example, the following sequence forces recompilation after changing files in the working directory.

     $ bitbake matchbox-desktop
               .
               .
        [make some changes to the source code in the working directory]
               .
               .
     $ bitbake matchbox-desktop -c compile -f
     $ bitbake matchbox-desktop
            

This sequence first builds matchbox-desktop and then recompiles it. The last command reruns all tasks (basically the packaging tasks) after the compile. BitBake recognizes that the compile task was rerun and therefore understands that the other tasks also need to be run again.

You can view a list of tasks in a given package by running the listtasks task as follows:

     $ bitbake matchbox-desktop -c listtasks
            

The results are in the file ${WORKDIR}/temp/log.do_listtasks.

Sometimes it can be hard to see why BitBake wants to build some other packages before a given package you have specified. The bitbake -g targetname command creates the depends.dot and task-depends.dot files in the current directory. These files show the package and task dependencies and are useful for debugging problems. You can use the bitbake -g -u depexp targetname command to display the results in a more human-readable form.

You can see debug output from BitBake by using the -D option. The debug output gives more information about what BitBake is doing and the reason behind it. Each -D option you use increases the logging level. The most common usage is -DDD.

The output from bitbake -DDD -v targetname can reveal why BitBake chose a certain version of a package or why BitBake picked a certain provider. This command could also help you in a situation where you think BitBake did something unexpected.

If you really want to build a specific .bb file, you can use the command form bitbake -b <somepath/somefile.bb>. This command form does not check for dependencies so you should use it only when you know its dependencies already exist. You can also specify fragments of the filename. In this case, BitBake checks for a unique match.

The -e option dumps the resulting environment for either the configuration (no package specified) or for a specific package when specified; or -b recipename to show the environment from parsing a single recipe file only.

Best practices exist while writing recipes that both log build progress and act on build conditions such as warnings and errors. Both Python and Bash language bindings exist for the logging mechanism:

For guidance on how logging is handled in both Python and Bash recipes, see the logging.bbclass file in the meta/classes directory of the Yocto Project files.

     python do_listtasks() {
         bb.debug(2, "Starting to figure out the task list")
         if noteworthy_condition:
             bb.note("There are 47 tasks to run")
         bb.debug(2, "Got to point xyz")
         if warning_trigger:
             bb.warn("Detected warning_trigger, this might be a problem later.")
         if recoverable_error:
             bb.error("Hit recoverable_error, you really need to fix this!")
         if fatal_error:
             bb.fatal("fatal_error detected, unable to print the task list")
         bb.plain("The tasks present are abc")
         bb.debug(2, "Finished figuring out the tasklist")
     }
                

     do_my_function() {
         bbdebug 2 "Running do_my_function"
         if [ exceptional_condition ]; then
             bbnote "Hit exceptional_condition"
         fi
         bbdebug 2  "Got to point xyz"
         if [ warning_trigger ]; then
             bbwarn "Detected warning_trigger, this might cause a problem later."
         fi
         if [ recoverable_error ]; then
             bberror "Hit recoverable_error, correcting"
         fi
         if [ fatal_error ]; then
             bbfatal "fatal_error detected"
         fi
         bbdebug 2 "Completed do_my_function"
     }
                

Here are some other tips that you might find useful:

Building an application from a single file that is stored locally (e.g. under files/) requires a recipe that has the file listed in the SRC_URI variable. Additionally, you need to manually write the do_compile and do_install tasks. The S variable defines the directory containing the source code, which is set to WORKDIR in this case - the directory BitBake uses for the build.

     DESCRIPTION = "Simple helloworld application"
     SECTION = "examples"
     LICENSE = "MIT"
     LIC_FILES_CHKSUM = "file://${COMMON_LICENSE_DIR}/MIT;md5=0835ade698e0bcf8506ecda2f7b4f302"
     PR = "r0"

     SRC_URI = "file://helloworld.c"

     S = "${WORKDIR}"

     do_compile() {
     	${CC} helloworld.c -o helloworld
     }

     do_install() {
     	install -d ${D}${bindir}
     	install -m 0755 helloworld ${D}${bindir}
     }
                

By default, the helloworld, helloworld-dbg, and helloworld-dev packages are built. For information on how to customize the packaging process, see the "Splitting an Application into Multiple Packages" section.

Applications that use Autotools such as autoconf and automake require a recipe that has a source archive listed in SRC_URI and also inherits Autotools, which instructs BitBake to use the autotools.bbclass file, which contains the definitions of all the steps needed to build an Autotool-based application. The result of the build is automatically packaged. And, if the application uses NLS for localization, packages with local information are generated (one package per language). Following is one example: (hello_2.3.bb)

     DESCRIPTION = "GNU Helloworld application"
     SECTION = "examples"
     LICENSE = "GPLv2+"
     LIC_FILES_CHKSUM = "file://COPYING;md5=751419260aa954499f7abaabaa882bbe"
     PR = "r0"

     SRC_URI = "${GNU_MIRROR}/hello/hello-${PV}.tar.gz"

     inherit autotools gettext
                 

The variable LIC_FILES_CHKSUM is used to track source license changes as described in the "Track License Change" section. You can quickly create Autotool-based recipes in a manner similar to the previous example.

Applications that use GNU make also require a recipe that has the source archive listed in SRC_URI. You do not need to add a do_compile step since by default BitBake starts the make command to compile the application. If you need additional make options you should store them in the EXTRA_OEMAKE variable. BitBake passes these options into the make GNU invocation. Note that a do_install task is still required. Otherwise BitBake runs an empty do_install task by default.

Some applications might require extra parameters to be passed to the compiler. For example, the application might need an additional header path. You can accomplish this by adding to the CFLAGS variable. The following example shows this:

     CFLAGS_prepend = "-I ${S}/include "
                

In the following example, mtd-utils is a makefile-based package:

     DESCRIPTION = "Tools for managing memory technology devices."
     SECTION = "base"
     DEPENDS = "zlib lzo e2fsprogs util-linux"
     HOMEPAGE = "http://www.linux-mtd.infradead.org/"
     LICENSE = "GPLv2"
     LIC_FILES_CHKSUM = "file://COPYING;md5=0636e73ff0215e8d672dc4c32c317bb3 \
                         file://include/common.h;beginline=1;endline=17;md5=ba05b07912a44ea2bf81ce409380049c"

     SRC_URI = "git://git.infradead.org/mtd-utils.git;protocol=git;tag=v${PV}"

     S = "${WORKDIR}/git/"

     EXTRA_OEMAKE = "'CC=${CC}' 'CFLAGS=${CFLAGS} -I${S}/include -DWITHOUT_XATTR' \
                     'BUILDDIR=${S}'"

     do_install () {
             oe_runmake install DESTDIR=${D} SBINDIR=${sbindir} MANDIR=${mandir} \
                                INCLUDEDIR=${includedir}
             install -d ${D}${includedir}/mtd/
             for f in ${S}/include/mtd/*.h; do
                     install -m 0644 $f ${D}${includedir}/mtd/
             done
     }
                

You can use the variables PACKAGES and FILES to split an application into multiple packages.

Following is an example that uses the libXpm recipe. By default, this recipe generates a single package that contains the library along with a few binaries. You can modify the recipe to split the binaries into separate packages:

     require xorg-lib-common.inc

     DESCRIPTION = "X11 Pixmap library"
     LICENSE = "X-BSD"
     LIC_FILES_CHKSUM = "file://COPYING;md5=3e07763d16963c3af12db271a31abaa5"
     DEPENDS += "libxext libsm libxt"
     PR = "r3"
     PE = "1"

     XORG_PN = "libXpm"

     PACKAGES =+ "sxpm cxpm"
     FILES_cxpm = "${bindir}/cxpm"
     FILES_sxpm = "${bindir}/sxpm"
                

In the previous example, we want to ship the sxpm and cxpm binaries in separate packages. Since bindir would be packaged into the main PN package by default, we prepend the PACKAGES variable so additional package names are added to the start of list. This results in the extra FILES_* variables then containing information that define which files and directories go into which packages. Files included by earlier packages are skipped by latter packages. Thus, the main PN package does not include the above listed files.

If you are building a library and the library offers static linking, you can control which static library files (*.a files) get included in the built library.

The PACKAGES and FILES_* variables in the meta/conf/bitbake.conf configuration file define how files installed by the do_install task are packaged. By default, the PACKAGES variable contains ${PN}-staticdev, which includes all static library files.

Following, is part of the BitBake configuration file. You can see where the static library files are defined:

     PACKAGES = "${PN}-dbg ${PN} ${PN}-doc ${PN}-dev ${PN}-staticdev ${PN}-locale"
     PACKAGES_DYNAMIC = "${PN}-locale-*"
     FILES = ""

     FILES_${PN} = "${bindir}/* ${sbindir}/* ${libexecdir}/* ${libdir}/lib*${SOLIBS} \
                 ${sysconfdir} ${sharedstatedir} ${localstatedir} \
                 ${base_bindir}/* ${base_sbindir}/* \
                 ${base_libdir}/*${SOLIBS} \
                 ${datadir}/${BPN} ${libdir}/${BPN}/* \
                 ${datadir}/pixmaps ${datadir}/applications \
                 ${datadir}/idl ${datadir}/omf ${datadir}/sounds \
                 ${libdir}/bonobo/servers"

     FILES_${PN}-doc = "${docdir} ${mandir} ${infodir} ${datadir}/gtk-doc \
                 ${datadir}/gnome/help"
     SECTION_${PN}-doc = "doc"

     FILES_${PN}-dev = "${includedir} ${libdir}/lib*${SOLIBSDEV} ${libdir}/*.la \
                     ${libdir}/*.o ${libdir}/pkgconfig ${datadir}/pkgconfig \
                     ${datadir}/aclocal ${base_libdir}/*.o"
     SECTION_${PN}-dev = "devel"
     ALLOW_EMPTY_${PN}-dev = "1"
     RDEPENDS_${PN}-dev = "${PN} (= ${EXTENDPKGV})"

     FILES_${PN}-staticdev = "${libdir}/*.a ${base_libdir}/*.a"
     SECTION_${PN}-staticdev = "devel"
     RDEPENDS_${PN}-staticdev = "${PN}-dev (= ${EXTENDPKGV})"
                

To add a post-installation script to a package, add a pkg_postinst_PACKAGENAME() function to the .bb file and use PACKAGENAME as the name of the package you want to attach to the postinst script. Normally PN can be used, which automatically expands to PACKAGENAME. A post-installation function has the following structure:

     pkg_postinst_PACKAGENAME () {
     #!/bin/sh -e
     # Commands to carry out
     }
                

The script defined in the post-installation function is called when the root filesystem is created. If the script succeeds, the package is marked as installed. If the script fails, the package is marked as unpacked and the script is executed when the image boots again.

Sometimes it is necessary for the execution of a post-installation script to be delayed until the first boot. For example, the script might need to be executed on the device itself. To delay script execution until boot time, use the following structure in the post-installation script:

     pkg_postinst_PACKAGENAME () {
     #!/bin/sh -e
     if [ x"$D" = "x" ]; then
          # Actions to carry out on the device go here
     else
          exit 1
     fi
     }
                

The previous example delays execution until the image boots again because the D variable points to the directory containing the image when the root filesystem is created at build time but is unset when executed on the first boot.

One way to get additional software into an image is to create a custom image. The following example shows the form for the two lines you need:

     IMAGE_INSTALL = "task-core-x11-base package1 package2"

     inherit core-image
                

By creating a custom image, a developer has total control over the contents of the image. It is important to use the correct names of packages in the IMAGE_INSTALL variable. You must use the OpenEmbedded notation and not the Debian notation for the names (e.g. eglibc-dev instead of libc6-dev).

The other method for creating a custom image is to modify an existing image. For example, if a developer wants to add strace into the core-image-sato image, they can use the following recipe:

     require core-image-sato.bb

     IMAGE_INSTALL += "strace"
                

For complex custom images, the best approach is to create a custom task package that is used to build the image or images. A good example of a tasks package is meta/recipes-sato/tasks/task-poky.bb. The PACKAGES variable lists the task packages to build along with the complementary -dbg and -dev packages. For each package added, you can use RDEPENDS and RRECOMMENDS entries to provide a list of packages the parent task package should contain. Following is an example:

     DESCRIPTION = "My Custom Tasks"

     PACKAGES = "\
         task-custom-apps \
         task-custom-apps-dbg \
         task-custom-apps-dev \
         task-custom-tools \
         task-custom-tools-dbg \
         task-custom-tools-dev \
         "

     RDEPENDS_task-custom-apps = "\
         dropbear \
         portmap \
         psplash"

     RDEPENDS_task-custom-tools = "\
         oprofile \
         oprofileui-server \
         lttng-control \
         lttng-viewer"

     RRECOMMENDS_task-custom-tools = "\
         kernel-module-oprofile"
                

In the previous example, two task packages are created with their dependencies and their recommended package dependencies listed: task-custom-apps, and task-custom-tools. To build an image using these task packages, you need to add task-custom-apps and/or task-custom-tools to IMAGE_INSTALL. For other forms of image dependencies see the other areas of this section.

Ultimately users might want to add extra image features to the set used by Yocto Project with the IMAGE_FEATURES variable. To create these features, the best reference is meta/classes/core-image.bbclass, which shows how the Yocto Project achieves this. In summary, the file looks at the contents of the IMAGE_FEATURES variable and then maps that into a set of tasks or packages. Based on this information the IMAGE_INSTALL variable is generated automatically. Users can add extra features by extending the class or creating a custom class for use with specialized image .bb files. You can also add more features by configuring the EXTRA_IMAGE_FEATURES variable in the local.conf file found in the Yocto Project files located in the build directory.

The Yocto Project ships with two SSH servers you can use in your images: Dropbear and OpenSSH. Dropbear is a minimal SSH server appropriate for resource-constrained environments, while OpenSSH is a well-known standard SSH server implementation. By default, the core-image-sato image is configured to use Dropbear. The core-image-basic and core-image-lsb images both include OpenSSH. To change these defaults, edit the IMAGE_FEATURES variable so that it sets the image you are working with to include ssh-server-dropbear or ssh-server-openssh.

It is possible to customize image contents by using variables used by distribution maintainers in the local.conf found in the Yocto Project build directory. This method only allows the addition of packages and is not recommended.

For example, to add the strace package into the image, you would add this package to the local.conf file:

     DISTRO_EXTRA_RDEPENDS += "strace"
                

However, since the DISTRO_EXTRA_RDEPENDS variable is for distribution maintainers, adding packages using this method is not as simple as adding them using a custom .bb file. Using the local.conf file method could result in some packages needing to be recreated. For example, if packages were previously created and the image was rebuilt, then the packages would need to be recreated.

Cleaning task-* packages are required because they use the DISTRO_EXTRA_RDEPENDS variable. You do not have to build them by hand because Yocto Project images depend on the packages they contain. This means dependencies are automatically built when the image builds. For this reason we do not use the rebuild task. In this case the rebuild task does not care about dependencies - it only rebuilds the specified package.

     $ bitbake -c clean task-boot task-base task-poky
     $ bitbake core-image-sato
                

To add a machine configuration you need to add a .conf file with details of the device being added to the conf/machine/ file. The name of the file determines the name the Yocto Project uses to reference the new machine.

The most important variables to set in this file are as follows:

You might also need these variables:

You can find full details on these variables in the reference section. You can leverage many existing machine .conf files from meta/conf/machine/.

The Yocto Project needs to be able to build a kernel for the machine. You need to either create a new kernel recipe for this machine, or extend an existing recipe. You can find several kernel examples in the Yocto Project file's meta/recipes-kernel/linux directory that you can use as references.

If you are creating a new recipe, normal recipe-writing rules apply for setting up a SRC_URI. Thus, you need to specify any necessary patches and set S to point at the source code. You need to create a configure task that configures the unpacked kernel with a defconfig. You can do this by using a make defconfig command or, more commonly, by copying in a suitable defconfig file and and then running make oldconfig. By making use of inherit kernel and potentially some of the linux-*.inc files, most other functionality is centralized and the the defaults of the class normally work well.

If you are extending an existing kernel, it is usually a matter of adding a suitable defconfig file. The file needs to be added into a location similar to defconfig files used for other machines in a given kernel. A possible way to do this is by listing the file in the SRC_URI and adding the machine to the expression in COMPATIBLE_MACHINE:

     COMPATIBLE_MACHINE = '(qemux86|qemumips)'
                

A formfactor configuration file provides information about the target hardware for which the Yocto Project is building and information that the Yocto Project cannot obtain from other sources such as the kernel. Some examples of information contained in a formfactor configuration file include framebuffer orientation, whether or not the system has a keyboard, the positioning of the keyboard in relation to the screen, and the screen resolution.

The Yocto Project uses reasonable defaults in most cases, but if customization is necessary you need to create a machconfig file in the Yocto Project file's meta/recipes-bsp/formfactor/files directory. This directory contains directories for specific machines such as qemuarm and qemux86. For information about the settings available and the defaults, see the meta/recipes-bsp/formfactor/files/config file found in the same area. Following is an example for qemuarm:

     HAVE_TOUCHSCREEN=1
     HAVE_KEYBOARD=1

     DISPLAY_CAN_ROTATE=0
     DISPLAY_ORIENTATION=0
     #DISPLAY_WIDTH_PIXELS=640
     #DISPLAY_HEIGHT_PIXELS=480
     #DISPLAY_BPP=16
     DISPLAY_DPI=150
     DISPLAY_SUBPIXEL_ORDER=vrgb
                

User-specific requirements drive the Multilib feature, Consequently, there is no one "out-of-the-box" configuration that likely exists to meet your needs.

In order to enable Multilib, you first need to ensure your recipe is extended to support multiple libraries. Many standard recipes are already extended and support multiple libraries. You can check in the meta/conf/multilib.conf configuration file in the Yocto Project files directory to see how this is done using the BBCLASSEXTEND variable. Eventually, all recipes will be covered and this list will be unneeded.

For the most part, the Multilib class extension works automatically to extend the package name from ${PN} to ${MLPREFIX}${PN}, where MLPREFIX is the particular multilib (e.g. "lib32-" or "lib64-"). Standard variables such as DEPENDS, RDEPENDS, RPROVIDES, RRECOMMENDS, PACKAGES, and PACKAGES_DYNAMIC are automatically extended by the system. If you are extending any manual code in the recipe, you can use the ${MLPREFIX} variable to ensure those names are extended correctly. This automatic extension code resides in multilib.bbclass.

After you have set up the recipes, you need to define the actual combination of multiple libraries you want to build. You accomplish this through your local.conf configuration file in the Yocto Project build directory. An example configuration would be as follows:

     MACHINE = "qemux86-64"
     require conf/multilib.conf
     MULTILIBS = "multilib:lib32"
     DEFAULTTUNE_virtclass-multilib-lib32 = "x86"
     MULTILIB_IMAGE_INSTALL = "lib32-connman"
                

This example enables an additional library named lib32 alongside the normal target packages. When combining these "lib32" alternatives, the example uses "x86" for tuning. For information on this particular tuning, see meta/conf/machine/include/ia32/arch-ia32.inc.

The example then includes lib32-connman in all the images, which illustrates one method of including a multiple library dependency. You can use a normal image build to include this dependency, for example:

     $ bitbake core-image-sato
                

You can also build Multilib packages specifically with a command like this:

     $  bitbake lib32-connman
                

Different packaging systems have different levels of native Multilib support. For the RPM Package Management System, the following implementation details exist:

For the IPK Package Management System, the following implementation details exist:

The LIC_FILES_CHKSUM variable contains checksums of the license text in the source code for the recipe. Following is an example of how to specify LIC_FILES_CHKSUM:

     LIC_FILES_CHKSUM = "file://COPYING;md5=xxxx \
                         file://licfile1.txt;beginline=5;endline=29;md5=yyyy \
                         file://licfile2.txt;endline=50;md5=zzzz \
                         ..."
                

The Yocto Project uses the S variable as the default directory used when searching files listed in LIC_FILES_CHKSUM. The previous example employs the default directory.

You can also use relative paths as shown in the following example:

     LIC_FILES_CHKSUM = "file://src/ls.c;beginline=5;endline=16;\
                                         md5=bb14ed3c4cda583abc85401304b5cd4e"
     LIC_FILES_CHKSUM = "file://../license.html;md5=5c94767cedb5d6987c902ac850ded2c6"
                

In this example, the first line locates a file in S/src/ls.c. The second line refers to a file in WORKDIR, which is the parent of S.

Note that this variable is mandatory for all recipes, unless the LICENSE variable is set to "CLOSED".

As mentioned in the previous section, the LIC_FILES_CHKSUM variable lists all the important files that contain the license text for the source code. It is possible to specify a checksum for an entire file, or a specific section of a file (specified by beginning and ending line numbers with the "beginline" and "endline" parameters, respectively). The latter is useful for source files with a license notice header, README documents, and so forth. If you do not use the "beginline" parameter, then it is assumed that the text begins on the first line of the file. Similarly, if you do not use the "endline" parameter, it is assumed that the license text ends with the last line of the file.

The "md5" parameter stores the md5 checksum of the license text. If the license text changes in any way as compared to this parameter then a mismatch occurs. This mismatch triggers a build failure and notifies the developer. Notification allows the developer to review and address the license text changes. Also note that if a mismatch occurs during the build, the correct md5 checksum is placed in the build log and can be easily copied to the recipe.

There is no limit to how many files you can specify using the LIC_FILES_CHKSUM variable. Generally, however, every project requires a few specifications for license tracking. Many projects have a "COPYING" file that stores the license information for all the source code files. This practice allows you to just track the "COPYING" file as long as it is kept up to date.

Often, developers want to extend the Yocto Project either by adding packages or by overriding files contained within the Yocto Project to add their own functionality. BitBake has a powerful mechanism called "layers", which provides a way to handle this extension in a fully supported and non-invasive fashion.

The Yocto Project files include several additional layers such as meta-rt and meta-yocto that demonstrate this functionality. The meta-rt layer is not enabled by default. However, the meta-yocto layer is.

To enable a layer, you simply add the layer's path to the BBLAYERS variable in your bblayers.conf file, which is found in the Yocto Project file's build directory. The following example shows how to enable the meta-rt:

     LCONF_VERSION = "1"

      BBFILES ?= ""
      BBLAYERS = " \
       /path/to/poky/meta \
       /path/to/poky/meta-yocto \
       /path/to/poky/meta-rt \
       "
               

BitBake parses each conf/layer.conf file for each layer in BBLAYERS and adds the recipes, classes and configurations contained within the layer to the Yocto Project. To create your own layer, independent of the Yocto Project files, simply create a directory with a conf/layer.conf file and add the directory to your bblayers.conf file.

The meta-yocto/conf/layer.conf file demonstrates the required syntax:

     # We have a conf and classes directory, add to BBPATH
     BBPATH := "${BBPATH}:${LAYERDIR}"

     # We have a packages directory, add to BBFILES
     BBFILES := "${BBFILES} ${LAYERDIR}/recipes-*/*/*.bb \
                 ${LAYERDIR}/recipes-*/*/*.bbappend"

     BBFILE_COLLECTIONS += "yocto"
     BBFILE_PATTERN_yocto := "^${LAYERDIR}/"
     BBFILE_PRIORITY_yocto = "5"
               

In the previous example, the recipes for the layers are added to BBFILES. The BBFILE_COLLECTIONS variable is then appended with the layer name. The BBFILE_PATTERN variable immediately expands with a regular expression used to match files from BBFILES into a particular layer, in this case by using the base pathname. The BBFILE_PRIORITY variable then assigns different priorities to the files in different layers. Applying priorities is useful in situations where the same package might appear in multiple layers and allows you to choose what layer should take precedence.

Note the use of the LAYERDIR variable with the immediate expansion operator. The LAYERDIR variable expands to the directory of the current layer and requires the immediate expansion operator so that BitBake does not wait to expand the variable when it's parsing a different directory.

BitBake can locate where other .bbclass and configuration files are applied through the BBPATH environment variable. For these cases, BitBake uses the first file with the matching name found in BBPATH. This is similar to the way the PATH variable is used for binaries. We recommend, therefore, that you use unique .bbclass and configuration file names in your custom layer.

We also recommend the following:

Following these recommendations keeps your Yocto Project files area and its configuration entirely inside the Yocto Project's core base.

Modifications to the Yocto Project are often managed under some kind of source revision control system. Because some simple practices can significantly improve usability, policy for committing changes is important. It helps to use a consistent documentation style when committing changes. The Yocto Project development team has found the following practices work well:

Following is an example commit:

    bitbake/data.py: Add emit_func() and generate_dependencies() functions

    These functions allow generation of dependency data between functions and
    variables allowing moves to be made towards generating checksums and allowing
    use of the dependency information in other parts of BitBake.

    Signed-off-by: Richard Purdie richard.purdie@linuxfoundation.org
                

All commits should be self-contained such that they leave the metadata in a consistent state that builds both before and after the commit is made. Besides being a good practice to follow, it helps ensure autobuilder test results are valid.

If a committed change results in changing the package output, then the value of the PR variable needs to be increased (or "bumped") as part of that commit. This means that for new recipes you must be sure to add the PR variable and set its initial value equal to "r0". Failing to define PR makes it easy to miss when you bump a package. Note that you can only use integer values following the "r" in the PR variable.

If you are sharing a common .inc file with multiple recipes, you can also use the INC_PR variable to ensure that the recipes sharing the .inc file are rebuilt when the .inc file itself is changed. The .inc file must set INC_PR (initially to "r0"), and all recipes referring to it should set PR to "$(INC_PR).0" initially, incrementing the last number when the recipe is changed. If the .inc file is changed then its INC_PR should be incremented.

When upgrading the version of a package, assuming the PV changes, the PR variable should be reset to "r0" (or "$(INC_PR).0" if you are using INC_PR).

Usually, version increases occur only to packages. However, if for some reason PV changes but does not increase, you can increase the PE variable (Package Epoch). The PE variable defaults to "0".

Version numbering strives to follow the Debian Version Field Policy Guidelines. These guidelines define how versions are compared and what "increasing" a version means.

There are two reasons for following the previously mentioned guidelines. First, to ensure that when a developer updates and rebuilds, they get all the changes to the repository and do not have to remember to rebuild any sections. Second, to ensure that target users are able to upgrade their devices using package manager commands such as opkg upgrade (or similar commands for dpkg/apt or rpm-based systems).

The goal is to ensure the Yocto Project has packages that can be upgraded in all cases.

It might not be immediately clear how you can use the Yocto Project in a team environment, or scale it for a large team of developers. The specifics of any situation determine the best solution. Granted that the Yocto Project offers immense flexibility regarding this, practices do exist that experience has shown work well.

The core component of any development effort with the Yocto Project is often an automated build and testing framework along with an image generation process. You can use these core components to check that the metadata can be built, highlight when commits break the build, and provide up-to-date images that allow developers to test the end result and use it as a base platform for further development. Experience shows that buildbot is a good fit for this role. What works well is to configure buildbot to make two types of builds: incremental and full (from scratch). See the buildbot for the Yocto Project for an example implementation that uses buildbot.

You can tie incremental builds to a commit hook that triggers the build each time a commit is made to the metadata. This practice results in useful acid tests that determine whether a given commit breaks the build in some serious way. Associating a build to a commit can catch a lot of simple errors. Furthermore, the tests are fast so developers can get quick feedback on changes.

Full builds build and test everything from the ground up. These types of builds usually happen at predetermined times like during the night when the machine load is low.

Most teams have many pieces of software undergoing active development at any given time. You can derive large benefits by putting these pieces under the control of a source control system that is compatible with the Yocto Project (i.e. Git or Subversion (SVN). You can then set the autobuilder to pull the latest revisions of the packages and test the latest commits by the builds. This practice quickly highlights issues. The Yocto Project easily supports testing configurations that use both a stable known good revision and a floating revision. The Yocto Project can also take just the changes from specific source control branches. This capability allows you to track and test specific changes.

Perhaps the hardest part of setting this up is defining the software project or the Yocto Project metadata policies that surround the different source control systems. Of course circumstances will be different in each case. However, this situation reveals one of the Yocto Project's advantages - the system itself does not force any particular policy on users, unlike a lot of build systems. The system allows the best policies to be chosen for the given circumstances.

Often, rather than re-flashing a new image, you might wish to install updated packages into an existing running system. You can do this by first sharing the tmp/deploy/ipk/ directory through a web server and then by changing /etc/opkg/base-feeds.conf to point at the shared server. Following is an example:

     $ src/gz all http://www.mysite.com/somedir/deploy/ipk/all
     $ src/gz armv7a http://www.mysite.com/somedir/deploy/ipk/armv7a
     $ src/gz beagleboard http://www.mysite.com/somedir/deploy/ipk/beagleboard
                

BitBake is the tool at the heart of the Yocto Project and is responsible for parsing the metadata, generating a list of tasks from it, and then executing those tasks. To see a list of the options BitBake supports, use the following help command:

     $ bitbake --help
            

The most common usage for BitBake is bitbake <packagename>, where packagename is the name of the package you want to build (referred to as the "target" in this manual). The target often equates to the first part of a .bb filename. So, to run the matchbox-desktop_1.2.3.bb file, you might type the following:

     $ bitbake matchbox-desktop
            

Several different versions of matchbox-desktop might exist. BitBake chooses the one selected by the distribution configuration. You can get more details about how BitBake chooses between different target versions and providers in the "Preferences and Providers" section.

BitBake also tries to execute any dependent tasks first. So for example, before building matchbox-desktop, BitBake would build a cross compiler and eglibc if they had not already been built.

A useful BitBake option to consider is the -k or --continue option. This option instructs BitBake to try and continue processing the job as much as possible even after encountering an error. When an error occurs, the target that failed and those that depend on it cannot be remade. However, when you use this option other dependencies can still be processed.

The .bb files are usually referred to as "recipes." In general, a recipe contains information about a single piece of software. The information includes the location from which to download the source patches (if any are needed), which special configuration options to apply, how to compile the source files, and how to package the compiled output.

The term "package" can also be used to describe recipes. However, since the same word is used for the packaged output from the Yocto Project (i.e. .ipk or .deb files), this document avoids using the term "package" when referring to recipes.

Class files (.bbclass) contain information that is useful to share between metadata files. An example is the Autotools class, which contains common settings for any application that Autotools uses. The "Reference: Classes" appendix provides details about common classes and how to use them.

The configuration files (.conf) define various configuration variables that govern the Yocto Project build process. These files fall into several areas that define machine configuration options, distribution configuration options, compiler tuning options, general common configuration options and user configuration options (local.conf, which is found in the Yocto Project files build directory).

When determining what parts of the system need to be built, BitBake uses a per-task basis and does not use a per-recipe basis. You might wonder why using a per-task basis is preferred over a per-recipe basis. To help explain, consider having the IPK packaging backend enabled and then switching to DEB. In this case, do_install and do_package output are still valid. However, with a per-recipe approach, the build would not include the .deb files. Consequently, you would have to invalidate the whole build and rerun it. Rerunning everything is not the best situation. Also in this case, the core must be "taught" much about specific tasks. This methodology does not scale well and does not allow users to easily add new tasks in layers or as external recipes without touching the packaged-staging core.

The shared state code uses a checksum, which is a unique signature of a task's inputs, to determine if a task needs to be run again. Because it is a change in a task's inputs that triggers a rerun, the process needs to detect all the inputs to a given task. For shell tasks, this turns out to be fairly easy because the build process generates a "run" shell script for each task and it is possible to create a checksum that gives you a good idea of when the task's data changes.

To complicate the problem, there are things that should not be included in the checksum. First, there is the actual specific build path of a given task - the WORKDIR. It does not matter if the working directory changes because it should not affect the output for target packages. Also, the build process has the objective of making native/cross packages relocatable. The checksum therefore needs to exclude WORKDIR. The simplistic approach for excluding the working directory is to set WORKDIR to some fixed value and create the checksum for the "run" script.

Another problem results from the "run" scripts containing functions that might or might not get called. The incremental build solution contains code that figures out dependencies between shell functions. This code is used to prune the "run" scripts down to the minimum set, thereby alleviating this problem and making the "run" scripts much more readable as a bonus.

So far we have solutions for shell scripts. What about python tasks? The same approach applies even though these tasks are more difficult. The process needs to figure out what variables a python function accesses and what functions it calls. Again, the incremental build solution contains code that first figures out the variable and function dependencies, and then creates a checksum for the data used as the input to the task.

Like the WORKDIR case, situations exist where dependencies should be ignored. For these cases, you can instruct the build process to ignore a dependency by using a line like the following:

     PACKAGE_ARCHS[vardepsexclude] = "MACHINE"
            

This example ensures that the PACKAGE_ARCHS variable does not depend on the value of MACHINE, even if it does reference it.

Equally, there are cases where we need to add dependencies BitBake is not able to find. You can accomplish this by using a line like the following:

      PACKAGE_ARCHS[vardeps] = "MACHINE"
            

This example explicitly adds the MACHINE variable as a dependency for PACKAGE_ARCHS.

Consider a case with inline python, for example, where BitBake is not able to figure out dependencies. When running in debug mode (i.e. using -DDD), BitBake produces output when it discovers something for which it cannot figure out dependencies. The Yocto Project team has currently not managed to cover those dependencies in detail and is aware of the need to fix this situation.

Thus far, this section has limited discussion to the direct inputs into a task. Information based on direct inputs is referred to as the "basehash" in the code. However, there is still the question of a task's indirect inputs - the things that were already built and present in the build directory. The checksum (or signature) for a particular task needs to add the hashes of all the tasks on which the particular task depends. Choosing which dependencies to add is a policy decision. However, the effect is to generate a master checksum that combines the basehash and the hashes of the task's dependencies.

At the code level, there are a variety of ways both the basehash and the dependent task hashes can be influenced. Within the BitBake configuration file, we can give BitBake some extra information to help it construct the basehash. The following statements effectively result in a list of global variable dependency excludes - variables never included in any checksum:

  BB_HASHBASE_WHITELIST ?= "TMPDIR FILE PATH PWD BB_TASKHASH BBPATH"
  BB_HASHBASE_WHITELIST += "DL_DIR SSTATE_DIR THISDIR FILESEXTRAPATHS"
  BB_HASHBASE_WHITELIST += "FILE_DIRNAME HOME LOGNAME SHELL TERM USER"
  BB_HASHBASE_WHITELIST += "FILESPATH USERNAME STAGING_DIR_HOST STAGING_DIR_TARGET"
            

The previous example actually excludes WORKDIR since it is actually constructed as a path within TMPDIR, which is on the whitelist.

The rules for deciding which hashes of dependent tasks to include through dependency chains are more complex and are generally accomplished with a python function. The code in meta/lib/oe/sstatesig.py shows two examples of this and also illustrates how you can insert your own policy into the system if so desired. This file defines the two basic signature generators OE-Core uses: "OEBasic" and "OEBasicHash". By default, there is a dummy "noop" signature handler enabled in BitBake. This means that behavior is unchanged from previous versions. OE-Core uses the "OEBasic" signature handler by default through this setting in the bitbake.conf file:

  BB_SIGNATURE_HANDLER ?= "OEBasic"
            

The "OEBasicHash" BB_SIGNATURE_HANDLER is the same as the "OEBasic" version but adds the task hash to the stamp files. This results in any metadata change that changes the task hash, automatically causing the task to be run again. This removes the need to bump PR values and changes to metadata automatically ripple across the build. Currently, this behavior is not the default behavior for OE-Core but is the default in poky.

It is also worth noting that the end result of these signature generators is to make some dependency and hash information available to the build. This information includes:

  BB_BASEHASH_task-<taskname> - the base hashes for each task in the recipe
  BB_BASEHASH_<filename:taskname> - the base hashes for each dependent task
  BBHASHDEPS_<filename:taskname> - The task dependencies for each task
  BB_TASKHASH - the hash of the currently running task
            

Checksums and dependencies, as discussed in the previous section, solve half the problem. The other part of the problem is being able to use checksum information during the build and being able to reuse or rebuild specific components.

The shared state class (sstate.bbclass) is a relatively generic implementation of how to "capture" a snapshot of a given task. The idea is that the build process does not care about the source of a task's output. Output could be freshly built or it could be downloaded and unpacked from somewhere - the build process doesn't need to worry about its source.

There are two types of output, one is just about creating a directory in WORKDIR. A good example is the output of either do_install or do_package. The other type of output occurs when a set of data is merged into a shared directory tree such as the sysroot.

The Yocto Project team has tried to keep the details of the implementation hidden in sstate.bbclass. From a user's perspective, adding shared state wrapping to a task is as simple as this do_deploy example taken from do_deploy.bbclass:

     DEPLOYDIR = "${WORKDIR}/deploy-${PN}"
     SSTATETASKS += "do_deploy"
     do_deploy[sstate-name] = "deploy"
     do_deploy[sstate-inputdirs] = "${DEPLOYDIR}"
     do_deploy[sstate-outputdirs] = "${DEPLOY_DIR_IMAGE}"

     python do_deploy_setscene () {
         sstate_setscene(d)
     }
     addtask do_deploy_setscene
            

In the example, we add some extra flags to the task, a name field ("deploy"), an input directory where the task sends data, and the output directory where the data from the task should eventually be copied. We also add a _setscene variant of the task and add the task name to the SSTATETASKS list.

If you have a directory whose contents you need to preserve, you can do this with a line like the following:

     do_package[sstate-plaindirs] = "${PKGD} ${PKGDEST}"
            

This method, as well as the following example, also works for multiple directories.

     do_package[sstate-inputdirs] = "${PKGDESTWORK} ${SHLIBSWORKDIR}"
     do_package[sstate-outputdirs] = "${PKGDATA_DIR} ${SHLIBSDIR}"
     do_package[sstate-lockfile] = "${PACKAGELOCK}"
            

These methods also include the ability to take a lockfile when manipulating shared state directory structures since some cases are sensitive to file additions or removals.

Behind the scenes, the shared state code works by looking in SSTATE_DIR and SSTATE_MIRRORS for shared state files. Here is an example:

     SSTATE_MIRRORS ?= "\
     file://.* http://someserver.tld/share/sstate/ \n \
     file://.* file:///some/local/dir/sstate/"
            

The shared state package validity can be detected just by looking at the filename since the filename contains the task checksum (or signature) as described earlier in this section. If a valid shared state package is found, the build process downloads it and uses it to accelerate the task.

The build processes uses the *_setscene tasks for the task acceleration phase. BitBake goes through this phase before the main execution code and tries to accelerate any tasks for which it can find shared state packages. If a shared state package for a task is available, the shared state package is used. This means the task and any tasks on which it is dependent are not executed.

As a real world example, the aim is when building an IPK-based image, only the do_package_write_ipk tasks would have their shared state packages fetched and extracted. Since the sysroot is not used, it would never get extracted. This is another reason why a task-based approach is preferred over a recipe-based approach, which would have to install the output from every task.

The code in the Yocto Project that supports incremental builds is not simple code. This section presents some tips and tricks that help you work around issues related to shared state code.

You can find these files in the BSP Layer at:

     meta-<bsp_name>/<bsp_license_file>
                

These optional files satisfy licensing requirements for the BSP. The type or types of files here can vary depending on the licensing requirements. For example, in the Crown Bay BSP all licensing requirements are handled with the COPYING.MIT file.

Licensing files can be MIT, BSD, GPLv*, and so forth. These files are recommended for the BSP but are optional and totally up to the BSP developer.

You can find this file in the BSP Layer at:

     meta-<bsp_name>/README
                

This file provides information on how to boot the live images that are optionally included in the binary/ directory. The README file also provides special information needed for building the image.

At a minimum, the README file must contain a list of dependencies, such as the names of any other layers on which the BSP depends and the name of the BSP maintainer with his or her contact information.

You can find this file in the BSP Layer at:

     meta-<bsp_name>/README.sources
                

This file provides information on where to locate the BSP source files. For example, information provides where to find the sources that comprise the images shipped with the BSP. Information is also included to help you find the metadata used to generate the images that ship with the BSP.

You can find these files in the BSP Layer at:

     meta-<bsp_name>/binary/<bootable_images>
                

This optional area contains useful pre-built kernels and user-space filesystem images appropriate to the target system. This directory typically contains graphical (e.g. sato) and minimal live images when the BSP tarball has been created and made available in the Yocto Project website. You can use these kernels and images to get a system running and quickly get started on development tasks.

The exact types of binaries present are highly hardware-dependent. However, a README file should be present in the BSP Layer that explains how to use the kernels and images with the target hardware. If pre-built binaries are present, source code to meet licensing requirements must also exist in some form.

You can find this file in the BSP Layer at:

     meta-<bsp_name>/conf/layer.conf
                

The conf/layer.conf file identifies the file structure as a Yocto Project layer, identifies the contents of the layer, and contains information about how Yocto Project should use it. Generally, a standard boilerplate file such as the following works. In the following example, you would replace "bsp" and "_bsp" with the actual name of the BSP (i.e. <bsp_name> from the example template).

     # We have a conf and classes directory, add to BBPATH
     BBPATH := "${BBPATH}:${LAYERDIR}"

     # We have a recipes directory containing .bb and .bbappend files, add to BBFILES
     BBFILES := "${BBFILES} ${LAYERDIR}/recipes/*/*.bb \ 
                 ${LAYERDIR}/recipes/*/*.bbappend"

     BBFILE_COLLECTIONS += "bsp"
     BBFILE_PATTERN_bsp := "^${LAYERDIR}/"
     BBFILE_PRIORITY_bsp = "5"
                

To illustrate the string substitutions, here are the last three statements from the Crown Bay conf/layer.conf file:

     BBFILE_COLLECTIONS += "crownbay"
     BBFILE_PATTERN_crownbay := "^${LAYERDIR}/"
     BBFILE_PRIORITY_crownbay = "6"
                

This file simply makes BitBake aware of the recipes and configuration directories. The file must exist so that the Yocto Project build system can recognize the BSP.

You can find these files in the BSP Layer at:

     meta-<bsp_name>/conf/machine/*.conf
                

The machine files bind together all the information contained elsewhere in the BSP into a format that the Yocto Project build system can understand. If the BSP supports multiple machines, multiple machine configuration files can be present. These filenames correspond to the values to which users have set the MACHINE variable.

These files define things such as the kernel package to use (PREFERRED_PROVIDER of virtual/kernel), the hardware drivers to include in different types of images, any special software components that are needed, any bootloader information, and also any special image format requirements.

Each BSP Layer requires at least one machine file. However, you can supply more than one file. For example, in the Crown Bay BSP shown earlier in this section, the conf/machine directory contains two configuration files: crownbay.conf and crownbay-noemgd.conf. The crownbay.conf file is used for the Crown Bay BSP that supports the Intel® Embedded Media and Graphics Driver (Intel® EMGD), while the crownbay-noemgd.conf file is used for the Crown Bay BSP that does not support the Intel® EMGD.

This crownbay.conf file could also include a hardware "tuning" file that is commonly used to define the package architecture and specify optimization flags, which are carefully chosen to give best performance on a given processor.

Tuning files are found in the meta/conf/machine/include directory. To use them, you simply include them in the machine configuration file. For example, the Crown Bay BSP crownbay.conf has the following statement:

     include conf/machine/include/tune-atom.inc
                

You can find these files in the BSP Layer at:

     meta-<bsp_name>/recipes-bsp/*
                

This optional directory contains miscellaneous recipe files for the BSP. Most notably would be the formfactor files. For example, in the Crown Bay BSP there is the formfactor_0.0.bbappend file, which is an append file used to augment the recipe that starts the build. Furthermore, there are machine-specific settings used during the build that are defined by the machconfig files. In the Crown Bay example, two machconfig files exist: one that supports the Intel® Embedded Media and Graphics Driver (Intel® EMGD) and one that does not:

     meta-crownbay/recipes-bsp/formfactor/formfactor/crownbay/machconfig
     meta-crownbay/recipes-bsp/formfactor/formfactor/crownbay-noemgd/machconfig
     meta-crownbay/recipes-bsp/formfactor/formfactor_0.0.bbappend
                

You can find these files in the BSP Layer at:

     meta-<bsp_name>/recipes-core/*
                

This directory contains recipe files that are almost always necessary to build a useful, working Linux image. Thus, the term "core" is used to group these recipes. For example, in the Crown Bay BSP there is the task-core-tools.bbappend file, which is an append file used to recommend that the SystemTap package be included as a package when the image is built.

You can find these files in the BSP Layer at:

     meta-<bsp_name>/recipes-graphics/*            
                

This optional directory contains recipes for the BSP if it has special requirements for graphics support. All files that are needed for the BSP to support a display are kept here. For example, the Crown Bay BSP contains two versions of the xorg.conf file. The version in crownbay builds a BSP that supports the Intel® Embedded Media Graphics Driver (EMGD), while the version in crownbay-noemgd builds a BSP that supports Video Electronics Standards Association (VESA) graphics only:

     meta-crownbay/recipes-graphics/xorg-xserver/xserver-xf86-config_0.1.bbappend
     meta-crownbay/recipes-graphics/xorg-xserver/xserver-xf86-config/crownbay/xorg.conf
     meta-crownbay/recipes-graphics/xorg-xserver/xserver-xf86-config/crownbay-noemgd/xorg.conf
                

You can find these files in the BSP Layer at:

     meta-<bsp_name>/recipes-kernel/linux/linux-yocto_*.bbappend
                

These files append your specific changes to the kernel you are using.

For your BSP, you typically want to use an existing Yocto Project kernel found in the Yocto Project repository at meta/recipes-kernel/linux. You can append your specific changes to the kernel recipe by using a similarly named append file, which is located in the BSP Layer (e.g. the meta-<bsp_name>/recipes-kernel/linux directory).

Suppose the BSP uses the linux-yocto_3.0.bb kernel, which is the preferred kernel to use for developing a new BSP using the Yocto Project. In other words, you have selected the kernel in your <bsp_name>.conf file by adding the following statements:

     PREFERRED_PROVIDER_virtual/kernel ?= "linux-yocto"
     PREFERRED_VERSION_linux-yocto = "3.0%"
                

You would use the linux-yocto_3.0.bbappend file to append specific BSP settings to the kernel, thus configuring the kernel for your particular BSP.

As an example, look at the existing Crown Bay BSP. The append file used is:

     meta-crownbay/recipes-kernel/linux/linux-yocto_3.0.bbappend
                

The following listing shows the file. Be aware that the actual commit ID strings in this example listing might be different than the actual strings in the file from the meta-intel Git source repository.

     FILESEXTRAPATHS_prepend := "${THISDIR}/${PN}:"

     COMPATIBLE_MACHINE_crownbay = "crownbay"
     KMACHINE_crownbay  = "yocto/standard/crownbay"
     KERNEL_FEATURES_append_crownbay += " cfg/smp.scc"

     COMPATIBLE_MACHINE_crownbay-noemgd = "crownbay-noemgd"
     KMACHINE_crownbay-noemgd  = "yocto/standard/crownbay"
     KERNEL_FEATURES_append_crownbay-noemgd += " cfg/smp.scc"

     SRCREV_machine_pn-linux-yocto_crownbay ?= "2247da9131ea7e46ed4766a69bb1353dba22f873"
     SRCREV_meta_pn-linux-yocto_crownbay ?= "d05450e4aef02c1b7137398ab3a9f8f96da74f52"

     SRCREV_machine_pn-linux-yocto_crownbay-noemgd ?= "2247da9131ea7e46ed4766a69bb1353dba22f873"
     SRCREV_meta_pn-linux-yocto_crownbay-noemgd ?= "d05450e4aef02c1b7137398ab3a9f8f96da74f52"
                

This append file contains statements used to support the Crown Bay BSP for both Intel® EMGD and the VESA graphics. The build process, in this case, recognizes and uses only the statements that apply to the defined machine name - crownbay in this case. So, the applicable statements in the linux-yocto_3.0.bbappend file are follows:

     FILESEXTRAPATHS_prepend := "${THISDIR}/${PN}:"

     COMPATIBLE_MACHINE_crownbay = "crownbay"
     KMACHINE_crownbay  = "yocto/standard/crownbay"
     KERNEL_FEATURES_append_crownbay += " cfg/smp.scc"

     SRCREV_machine_pn-linux-yocto_crownbay ?= "2247da9131ea7e46ed4766a69bb1353dba22f873"
     SRCREV_meta_pn-linux-yocto_crownbay ?= "d05450e4aef02c1b7137398ab3a9f8f96da74f52"
                

The append file defines crownbay as the compatible machine, defines the KMACHINE, points to some configuration fragments to use by setting the KERNEL_FEATURES variable, and then points to the specific commits in the Yocto Project files Git repository and the meta Git repository branches to identify the exact kernel needed to build the Crown Bay BSP.

One thing missing in this particular BSP, which you will typically need when developing a BSP, is the kernel configuration file (.config) for your BSP. When developing a BSP, you probably have a kernel configuration file or a set of kernel configuration files that, when taken together, define the kernel configuration for your BSP. You can accomplish this definition by putting the configurations in a file or a set of files inside a directory located at the same level as your append file and having the same name as the kernel. With all these conditions met simply reference those files in a SRC_URI statement in the append file.

For example, suppose you had a set of configuration options in a file called myconfig. If you put that file inside a directory named /linux-yocto and then added a SRC_URI statement such as the following to the append file, those configuration options will be picked up and applied when the kernel is built.

     SRC_URI += "file://myconfig"
                

As mentioned earlier, you can group related configurations into multiple files and name them all in the SRC_URI statement as well. For example, you could group separate configurations specifically for Ethernet and graphics into their own files and add those by using a SRC_URI statement like the following in your append file:

     SRC_URI += "file://myconfig \
            file://eth.cfg \
            file://gfx.cfg"
                

The FILESEXTRAPATHS variable is in boilerplate form in the previous example in order to make it easy to do that. This variable must be in your layer or BitBake will not find the patches or configurations even if you have them in your SRC_URI. The FILESEXTRAPATHS variable enables the build process to find those configuration files.

Before looking at BSP requirements, you should consider the following:

Following are the requirements for a released BSP that conforms to the Yocto Project:

Following are recommendations for a released BSP that conforms to the Yocto Project:

Designed to have a command interface somewhat like Git, each tool is structured as a set of sub-commands under a top-level command. The top-level command (yocto-bsp or yocto-kernel) itself does nothing but invoke or provide help on the sub-commands it supports.

Both tools reside in the scripts/ subdirectory of the Yocto Project Files. Consequently, to use the scripts, you must source the environment just as you would when invoking a build:

     $ source oe-init-build-env [build_dir]
                    

The most immediately useful function is to get help on both tools. The built-in help system makes it easy to drill down at any time and view the syntax required for any specific command. Simply enter the name of the command, or the command along with help to display a list of the available sub-commands. Here is an example:

     $ yocto-bsp
     $ yocto-bsp help

     Usage:

     Create a customized Yocto BSP layer.

     usage: yocto-bsp [--version] [--help] COMMAND [ARGS]

     The most commonly used 'yocto-bsp' commands are:
     create            Create a new Yocto BSP
     list              List available values for options and BSP properties

     See 'yocto-bsp help COMMAND' for more information on a specific command.


     Options:
     --version    show program's version number and exit
     -h, --help   show this help message and exit
     -D, --debug  output debug information
                    

Similarly, entering just the name of a sub-command shows the detailed usage for that sub-command:

     $ yocto-bsp create

     Usage:

     Create a new Yocto BSP
     usage: yocto-bsp create <bsp-name> <karch> [-o <DIRNAME> | --outdir <DIRNAME>]
             [-i <JSON PROPERTY FILE> | --infile <JSON PROPERTY_FILE>]

                 This command creates a Yocto BSP based on the specified parameters.
                 The new BSP will be a new Yocto BSP layer contained by default within
                 the top-level directory specified as 'meta-bsp-name'.  The -o option
                 can be used to place the BSP layer in a directory with a different
                 name and location.

                 ...
                    

For any sub-command, you can also use the word 'help' just before the sub-command to get more extensive documentation:

     $ yocto-bsp help create

     NAME
     yocto-bsp create - Create a new Yocto BSP

     SYNOPSIS
     yocto-bsp create <bsp-name> <karch> [-o <DIRNAME> | --outdir <DIRNAME>]
             [-i <JSON PROPERTY FILE> | --infile <JSON PROPERTY_FILE>]

     DESCRIPTION
     This command creates a Yocto BSP based on the specified
     parameters.  The new BSP will be a new Yocto BSP layer contained
     by default within the top-level directory specified as
     'meta-bsp-name'.  The -o option can be used to place the BSP layer
     in a directory with a different name and location.
			    
     The value of the 'karch' parameter determines the set of files
     that will be generated for the BSP, along with the specific set of
     'properties' that will be used to fill out the BSP-specific
     portions of the BSP.
			    
     ...
			    
     NOTE: Once created, you should add your new layer to your
     bblayers.conf file in order for it to be subsequently seen and
     modified by the yocto-kernel tool.
			    
     NOTE for x86- and x86_64-based BSPs: The generated BSP assumes the
     presence of the of the meta-intel layer, so you should also have a
     meta-intel layer present and added to your bblayers.conf as well.
                    

Now that you know where these two commands reside and how to access information on them, you should find it relatively straightforward to discover the commands necessary to create a BSP and perform basic kernel maintenance on that BSP using the tools. The next sections provide a concrete starting point to expand on a few points that might not be immediately obvious or that could use further explanation.

The yocto-bsp script creates a new BSP layer for any architecture supported by the Yocto Project, as well as QEMU versions of the same. The default mode of the script's operation is to prompt you for information needed to generate the BSP layer. For the current set of BSPs, the script prompts you for various important parameters such as:

You use the yocto-bsp create sub-command to create a new BSP layer. This command requires you to specify a particular architecture on which to base the BSP. Assuming you have sourced the environment, you can use the yocto-bsp list karch sub-command to list the architectures available for BSP creation as follows:

     $ yocto-bsp list karch
     Architectures available:
         arm
         powerpc
         i386
         mips
         x86_64
         qemu
                    

The remainder of this section presents an example that uses myarm as the machine name and qemu as the machine architecture. Of the available architectures, qemu is the only architecture that causes the script to prompt you further for an actual architecture. In every other way, this architecture is representative of how creating a BSP for a 'real' machine would work. The reason the example uses this architecture is because it is an emulated architecture and can easily be followed without requiring actual hardware.

As the yocto-bsp create command runs, default values for the prompts appear in brackets. Pressing enter without supplying anything on the command line or pressing enter and providing an invalid response causes the script to accept the default value.

Following is the complete example:

     $ yocto-bsp create myarm qemu
     Which qemu architecture would you like to use? [default: x86]
             1) common 32-bit x86
             2) common 64-bit x86
             3) common 32-bit ARM
             4) common 32-bit PowerPC
             5) common 32-bit MIPS
     3
     Would you like to use the default (3.2) kernel? (Y/n)
     Do you need a new machine branch for this BSP (the alternative is to re-use an existing branch)? [Y/n]
     Getting branches from remote repo git://git.yoctoproject.org/linux-yocto-3.2...
     Please choose a machine branch to base this BSP on => [default: standard/default/common-pc]
             1) base
             2) standard/base
             3) standard/default/arm-versatile-926ejs
             4) standard/default/base
             5) standard/default/beagleboard
             6) standard/default/cedartrailbsp (copy).xml
             7) standard/default/common-pc-64/base
             8) standard/default/common-pc-64/jasperforest
             9) standard/default/common-pc-64/romley
             10) standard/default/common-pc-64/sugarbay
             11) standard/default/common-pc/atom-pc
             12) standard/default/common-pc/base
             13) standard/default/crownbay
             14) standard/default/emenlow
             15) standard/default/fishriver
             16) standard/default/fri2
             17) standard/default/fsl-mpc8315e-rdb
             18) standard/default/mti-malta32-be
             19) standard/default/mti-malta32-le
             20) standard/default/preempt-rt
             21) standard/default/qemu-ppc32
             22) standard/default/routerstationpro
             23) standard/preempt-rt/base
             24) standard/preempt-rt/qemu-ppc32
             25) standard/preempt-rt/routerstationpro
             26) standard/tiny
     3
     Do you need SMP support? (Y/n)
     Does your BSP have a touchscreen? (y/N)
     Does your BSP have a keyboard? (Y/n)
     New qemu BSP created in meta-myarm
                    

Let's take a closer look at the example now:

Once the BSP Layer is created, you must add it to your bblayers.conf file. Here is an example:

     BBLAYERS = " \
        /usr/local/src/yocto/meta \
        /usr/local/src/yocto/meta-yocto \
        /usr/local/src/yocto/meta-myarm \
        "
                    

Adding the layer to this file allows the build system to build the BSP and the yocto-kernel tool to be able to find the layer and other metadata it needs on which to operate.

Assuming you have created a Yocto Project BSP Layer using yocto-bsp and you added it to your BBLAYERS variable in the bblayers.conf file, you can now use the yocto-kernel script to add patches and configuration items to the BSP's kernel.

The yocto-kernel script allows you to add, remove, and list patches and kernel config settings to a Yocto Project BSP's kernel .bbappend file. All you need to do is use the appropriate sub-command. Recall that the easiest way to see exactly what sub-commands are available is to use the yocto-kernel built-in help as follows:

     $ yocto-kernel
     Usage:

     Modify and list Yocto BSP kernel config items and patches.

     usage: yocto-kernel [--version] [--help] COMMAND [ARGS]

     The most commonly used 'yocto-kernel' commands are:
     config list       List the modifiable set of bare kernel config options for a BSP
     config add        Add or modify bare kernel config options for a BSP 
     config rm         Remove bare kernel config options from a BSP
     patch list        List the patches associated with a BSP
     patch add         Patch the Yocto kernel for a BSP
     patch rm          Remove patches from a BSP

     See 'yocto-kernel help COMMAND' for more information on a specific command.
                    

The yocto-kernel patch add sub-command allows you to add a patch to a BSP. The following example adds two patches to the myarm BSP:

     $ yocto-kernel patch add myarm ~/test.patch
     Added patches:
             test.patch

     $ yocto-kernel patch add myarm ~/yocto-testmod.patch
     Added patches:
             yocto-testmod.patch
                    

You can verify patches have been added by using the yocto-kernel patch list sub-command. Here is an example:

     $ yocto-kernel patch list myarm
     The current set of machine-specific patches for myarm is:
             1) test.patch
             2) yocto-testmod.patch
                    

You can also use the yocto-kernel script to remove a patch using the yocto-kernel patch rm sub-command. Here is an example:

     $ yocto-kernel patch rm myarm
     Specify the patches to remove:
             1) test.patch
             2) yocto-testmod.patch
     1
     Removed patches:
             test.patch
                    

Again, using the yocto-kernel patch list sub-command, you can verify that the patch was in fact removed:

     $ yocto-kernel patch list myarm
     The current set of machine-specific patches for myarm is:
             1) yocto-testmod.patch
                    

In a completely similar way, you can use the yocto-kernel config add sub-command to add one or more kernel config item settings to a BSP. The following commands add a couple of config items to the myarm BSP:

     $ yocto-kernel config add myarm CONFIG_MISC_DEVICES=y
     Added items:
             CONFIG_MISC_DEVICES=y

     $ yocto-kernel config add myarm KCONFIG_YOCTO_TESTMOD=y
     Added items:
             CONFIG_YOCTO_TESTMOD=y
                    

You can list the config items now associated with the BSP. Doing so shows you the config items you added as well as others associated with the BSP:

     $ yocto-kernel config list myarm
     The current set of machine-specific kernel config items for myarm is:
             1) CONFIG_MISC_DEVICES=y
             2) CONFIG_YOCTO_TESTMOD=y
                    

Finally, you can remove one or more config items using the yocto-kernel config rm sub-command in a manner completely analogous to yocto-kernel patch rm.

The Yocto Project provides toolchains that allow you to develop your application outside of the Yocto Project build system for specific hardware. These toolchains (called meta-toolchains) contain cross-development tools such as compilers, linkers, and debuggers that build your application for your target device. The Yocto Project also provides images that have toolchains for supported architectures included within the image. This allows you to compile, debug, or profile applications directly on the target device. See Appendix D, Reference: Images for a listing of the image types that Yocto Project supports.

Using the BitBake tool you can build a meta-toolchain or meta-toolchain-sdk target, which generates a tarball. Unpacking this tarball into the /opt/poky directory on your host produces a setup script (e.g. /opt/poky/environment-setup-i586-poky-linux) that you can source to initialize your build environment. Sourcing this script adds the compiler, QEMU scripts, QEMU binary, a special version of pkgconfig and other useful utilities to the PATH variable used by the Yocto Project build environment. Variables to assist pkgconfig and Autotools are also defined so that, for example, configure can find pre-generated test results for tests that need target hardware on which to run.

Using the toolchain with Autotool-enabled packages is straightforward - just pass the appropriate host option to configure. Following is an example:

     $ ./configure --host=arm-poky-linux-gnueabi
            

For projects that are not Autotool-enabled, it is usually just a case of ensuring you point to and use the cross-toolchain. For example, the following two lines of code in a Makefile that builds your application specify to use the cross-compiler arm-poky-linux-gnueabi-gcc and linker arm-poky-linux-gnueabi-ld, which are part of the meta-toolchain you would have previously established:

     CC=arm-poky-linux-gnueabi-gcc;
     LD=arm-poky-linux-gnueabi-ld;
            

The current release of the Yocto Project supports the Eclipse IDE plug-in to make developing software easier for the application developer. The plug-in provides capability extensions to the graphical IDE to allow for cross compilation, deployment and execution of the application within a QEMU emulation session. Support of the Eclipse plug-in also allows for cross debugging and profiling. Additionally, the Eclipse plug-in provides a suite of tools that allows the developer to perform remote profiling, tracing, collection of power consumption data, collection of latency data and collection of performance data.

To use the Eclipse plug-in you need the Eclipse Framework (Helios 3.6.1) along with other plug-ins installed into the Eclipse IDE. Once you have your environment setup you need to configure the Eclipse plug-in. For information on how to install and configure the Eclipse plug-in, see the "Working Within Eclipse" chapter in the Yocto Project Application Development Toolkit (ADT) User's Guide.

Running Poky QEMU images is covered in the "A Quick Test Run" section of the Yocto Project Quick Start.

The QEMU images shipped with the Yocto Project contain complete toolchains native to their target architectures. This support allows you to develop applications within QEMU similar to the way you would using a normal host development system.

Speed can be an issue depending on the target and host architecture mix. For example, using the qemux86 image in the emulator on an Intel-based 32-bit (x86) host machine is fast because the target and host architectures match. On the other hand, using the qemuarm image on the same Intel-based host can be slower. But, you still achieve faithful emulation of ARM-specific issues.

To speed things up, the QEMU images support using distcc to call a cross-compiler outside the emulated system. If you used runqemu to start QEMU, and distccd is present on the host system, any BitBake cross-compiling toolchain available from the build system is automatically used from within QEMU simply by calling distcc. You can accomplish this by defining the cross-compiler variable (e.g. export CC="distcc"). Alternatively, if a suitable SDK/toolchain is present in /opt/poky the toolchain is also automatically used.

Several mechanisms exist that let you connect to the system running on the QEMU emulator:

Working directly with the Yocto Project is a fast and effective development technique. The idea is that you can directly edit files in a working directory (WORKDIR) or the source directory (S) and then force specific tasks to rerun in order to test the changes. An example session working on the matchbox-desktop package might look like this:

     $ bitbake matchbox-desktop
     $ sh
     $ cd tmp/work/armv5te-poky-linux-gnueabi/matchbox-desktop-2.0+svnr1708-r0/
     $ cd matchbox-desktop-2
     $ vi src/main.c
           .
           .
      [Make your changes]
           .
           .
     $ exit
     $ bitbake matchbox-desktop -c compile -f
     $ bitbake matchbox-desktop
            

This example builds the matchbox-desktop package, creates a new terminal, changes into the work directory for the package, changes a file, exits out of the terminal, and then recompiles the package. Instead of using sh, you can also use two different terminals. However, the risk of using two terminals is that a command like unpack could destroy your changes in the work directory. Consequently, you need to work carefully.

It is useful when making changes directly to the work directory files to do so using the Quilt tool as detailed in the "Modifying Package Source Code with Quilt" section. Using Quilt, you can copy patches into the recipe directory and use the patches directly through use of the SRC_URI variable.

For a review of the skills used in this section, see the "BitBake" and "Running Specific Tasks" sections in this manual.

When debugging certain commands or even when just editing packages, devshell can be a useful tool. Using devshell differs from the example shown in the previous section in that when you invoke devshell source files are extracted into your working directory and patches are applied. Then, a new terminal is opened and you are placed in the working directory. In the new terminal all the Yocto Project build-related environment variables are still defined so you can use commands such as configure and make. The commands execute just as if the Yocto Project build system were executing them. Consequently, working this way can be helpful when debugging a build or preparing software to be used with the Yocto Project build system.

Following is an example that uses devshell on a target named matchbox-desktop:

     $ bitbake matchbox-desktop -c devshell
            

This command opens a terminal with a shell prompt within the Poky environment. The following occurs:

Within this environment, you can run configure or compile commands as if they were being run by the Yocto Project build system itself. As noted earlier, the working directory also automatically changes to the source directory (S).

When you are finished, you just exit the shell or close the terminal window.

The default shell used by devshell is xterm. You can use other terminal forms by setting the TERMCMD and TERMCMDRUN variables in the Yocto Project's local.conf file found in the build directory. For examples of the other options available, see the "UI/Interaction Configuration" section of the meta/conf/bitbake.conf configuration file in the Yocto Project files.

Because an external shell is launched rather than opening directly into the original terminal window, it allows easier interaction with BitBake's multiple threads as well as accomodates a future client/server split.

If you're working on a recipe that pulls from an external Source Code Manager (SCM), it is possible to have the Yocto Project build system notice new changes added to the SCM and then build the package that depends on them using the latest version. This only works for SCMs from which it is possible to get a sensible revision number for changes. Currently, you can do this with Apache Subversion (SVN), Git, and Bazaar (BZR) repositories.

To enable this behavior, simply add the following to the local.conf configuration file in the build directory of the Yocto Project files:

     SRCREV_pn-<PN> = "${AUTOREV}"
            

where PN is the name of the package for which you want to enable automatic source revision updating.

First, make sure Gdbserver is installed on the target. If it is not, install the package gdbserver, which needs the libthread-db1 package.

As an example, to launch Gdbserver on the target and make it ready to "debug" a program located at /path/to/inferior, connect to the target and launch:

     $ gdbserver localhost:2345 /path/to/inferior
            

Gdbserver should now be listening on port 2345 for debugging commands coming from a remote GDB process that is running on the host computer. Communication between Gdbserver and the host GDB are done using TCP. To use other communication protocols, please refer to the Gdbserver documentation.

Running GDB on the host computer takes a number of stages. This section describes those stages.

 
     /usr/local/poky/eabi-glibc/arm/bin/arm-poky-linux-gnueabi-gdb
                

     $ bitbake gdb-cross
                

     tmp/sysroots/<host-arch>/usr/bin/<target-abi>-gdb 
                

     $ <target-abi>-gdb rootfs/usr/bin/foo
                

     $ set solib-absolute-prefix /path/to/tmp/rootfs
                

     $ target remote remote-target-ip-address:2345
                

     (gdb) break main
     (gdb) continue
                

Using OProfile you can perform all the profiling work on the target device. A simple OProfile session might look like the following:

     # opcontrol --reset
     # opcontrol --start --separate=lib --no-vmlinux -c 5
              .
              .
        [do whatever is being profiled]
              .
              .
     # opcontrol --stop
     $ opreport -cl
            

In this example, the reset command clears any previously profiled data. The next command starts OProfile. The options used when starting the profiler separate dynamic library data within applications, disable kernel profiling, and enable callgraphing up to five levels deep.

After you perform your profiling tasks, the next command stops the profiler. After that, you can view results with the opreport command with options to see the separate library symbols and callgraph information.

Callgraphing logs information about time spent in functions and about a function's calling function (parent) and called functions (children). The higher the callgraphing depth, the more accurate the results. However, higher depths also increase the logging overhead. Consequently, you should take care when setting the callgraphing depth.

For more information on using OProfile, see the OProfile online documentation at http://oprofile.sourceforge.net/docs/.

A graphical user interface for OProfile is also available. You can download and build this interface from the Yocto Project at http://git.yoctoproject.org/cgit.cgi/oprofileui/. If the "tools-profile" image feature is selected, all necessary binaries are installed onto the target device for OProfileUI interaction.

Even though the Yocto Project usually includes all needed patches on the target device, you might find you need other OProfile patches for recent OProfileUI features. If so, see the OProfileUI README for the most recent information.

     # opcontrol --reset
     # opcontrol --start --separate=lib --no-vmlinux -c 5
            .
            .
     [do whatever is being profiled]
            .
            .
     # opcontrol --stop
     # oparchive -o my_archive