4 Source Directory Structure

The Source Directory consists of numerous files, directories and subdirectories; understanding their locations and contents is key to using the Yocto Project effectively. This chapter describes the Source Directory and gives information about those files and directories.

For information on how to establish a local Source Directory on your development system, see the “Locating Yocto Project Source Files” section in the Yocto Project Development Tasks Manual.

Note

The OpenEmbedded build system does not support file or directory names that contain spaces. Be sure that the Source Directory you use does not contain these types of names.

4.1 Top-Level Core Components

This section describes the top-level components of the Source Directory.

4.1.1 bitbake/

This directory includes a copy of BitBake for ease of use. The copy usually matches the current stable BitBake release from the BitBake project. BitBake, a Metadata interpreter, reads the Yocto Project Metadata and runs the tasks defined by that data. Failures are usually caused by errors in your Metadata and not from BitBake itself.

When you run the bitbake command, the main BitBake executable (which resides in the bitbake/bin/ directory) starts. Sourcing the environment setup script (i.e. oe-init-build-env) places the scripts/ and bitbake/bin/ directories (in that order) into the shell’s PATH environment variable.

For more information on BitBake, see the BitBake User Manual.

4.1.2 build/

This directory contains user configuration files and the output generated by the OpenEmbedded build system in its standard configuration where the source tree is combined with the output. The Build Directory is created initially when you source the OpenEmbedded build environment setup script (i.e. oe-init-build-env).

It is also possible to place output and configuration files in a directory separate from the Source Directory by providing a directory name when you source the setup script. For information on separating output from your local Source Directory files (commonly described as an “out of tree” build), see the “oe-init-build-env” section.

See the “The Build Directory — build/” section for details about the contents of the Build Directory.

4.1.3 documentation/

This directory holds the source for the Yocto Project documentation as well as templates and tools that allow you to generate PDF and HTML versions of the manuals. Each manual is contained in its own sub-folder; for example, the files for this reference manual reside in the ref-manual/ directory.

4.1.4 meta/

This directory contains the minimal, underlying OpenEmbedded-Core metadata. The directory holds recipes, common classes, and machine configuration for strictly emulated targets (qemux86, qemuarm, and so forth.)

4.1.5 meta-poky/

Designed above the meta/ content, this directory adds just enough metadata to define the Poky reference distribution.

4.1.6 meta-yocto-bsp/

This directory contains the Yocto Project reference hardware Board Support Packages (BSPs). For more information on BSPs, see the Yocto Project Board Support Package Developer’s Guide.

4.1.7 meta-selftest/

This directory adds additional recipes and append files used by the OpenEmbedded selftests to verify the behavior of the build system. You do not have to add this layer to your bblayers.conf file unless you want to run the selftests.

4.1.8 meta-skeleton/

This directory contains template recipes for BSP and kernel development.

4.1.9 scripts/

This directory contains various integration scripts that implement extra functionality in the Yocto Project environment (e.g. QEMU scripts). The oe-init-build-env script prepends this directory to the shell’s PATH environment variable.

The scripts directory has useful scripts that assist in contributing back to the Yocto Project, such as create-pull-request and send-pull-request.

4.1.10 oe-init-build-env

This script sets up the OpenEmbedded build environment. Running this script with the source command in a shell makes changes to PATH and sets other core BitBake variables based on the current working directory. You need to run an environment setup script before running BitBake commands. The script uses other scripts within the scripts directory to do the bulk of the work.

When you run this script, your Yocto Project environment is set up, a Build Directory is created, your working directory becomes the Build Directory, and you are presented with some simple suggestions as to what to do next, including a list of some possible targets to build. Here is an example:

$ source oe-init-build-env

### Shell environment set up for builds. ###

You can now run 'bitbake <target>'

Common targets are:
    core-image-minimal
    core-image-sato
    meta-toolchain
    meta-ide-support

You can also run generated QEMU images with a command like 'runqemu qemux86-64'

The default output of the oe-init-build-env script is from the conf-summary.txt and conf-notes.txt files, which are found in the meta-poky directory within the Source Directory. If you design a custom distribution, you can include your own versions of these configuration files where you can provide a brief summary and detailed usage notes, such as a list of the targets defined by your distribution. See the “Creating a Custom Template Configuration Directory” section in the Yocto Project Development Tasks Manual for more information.

By default, running this script without a Build Directory argument creates the build/ directory in your current working directory. If you provide a Build Directory argument when you source the script, you direct the OpenEmbedded build system to create a Build Directory of your choice. For example, the following command creates a Build Directory named mybuilds/ that is outside of the Source Directory:

$ source oe-init-build-env ~/mybuilds

The OpenEmbedded build system uses the template configuration files, which are found by default in the meta-poky/conf/templates/default directory in the Source Directory. See the “Creating a Custom Template Configuration Directory” section in the Yocto Project Development Tasks Manual for more information.

Note

The OpenEmbedded build system does not support file or directory names that contain spaces. If you attempt to run the oe-init-build-env script from a Source Directory that contains spaces in either the filenames or directory names, the script returns an error indicating no such file or directory. Be sure to use a Source Directory free of names containing spaces.

4.1.11 LICENSE, README, and README.hardware

These files are standard top-level files.

4.2 The Build Directory — build/

The OpenEmbedded build system creates the Build Directory when you run the build environment setup script oe-init-build-env. If you do not give the Build Directory a specific name when you run the setup script, the name defaults to build/.

For subsequent parsing and processing, the name of the Build directory is available via the TOPDIR variable.

4.2.1 build/buildhistory/

The OpenEmbedded build system creates this directory when you enable build history via the buildhistory class file. The directory organizes build information into image, packages, and SDK subdirectories. For information on the build history feature, see the “Maintaining Build Output Quality” section in the Yocto Project Development Tasks Manual.

4.2.2 build/cache/

This directory contains several internal files used by the OpenEmbedded build system.

It also contains sanity_info, a text file keeping track of important build information such as the values of TMPDIR, SSTATE_DIR, as well as the name and version of the host distribution.

4.2.3 build/conf/local.conf

This configuration file contains all the local user configurations for your build environment. The local.conf file contains documentation on the various configuration options. Any variable set here overrides any variable set elsewhere within the environment unless that variable is hard-coded within a file (e.g. by using ‘=’ instead of ‘?=’). Some variables are hard-coded for various reasons but such variables are relatively rare.

At a minimum, you would normally edit this file to select the target MACHINE, which package types you wish to use (PACKAGE_CLASSES), and the location from which you want to access downloaded files (DL_DIR).

If local.conf is not present when you start the build, the OpenEmbedded build system creates it from local.conf.sample when you source the top-level build environment setup script oe-init-build-env.

The source local.conf.sample file used depends on the TEMPLATECONF script variable, which defaults to meta-poky/conf/templates/default when you are building from the Yocto Project development environment, and to meta/conf/templates/default when you are building from the OpenEmbedded-Core environment. Because the script variable points to the source of the local.conf.sample file, this implies that you can configure your build environment from any layer by setting the variable in the top-level build environment setup script as follows:

TEMPLATECONF=your_layer/conf/templates/your_template_name

Once the build process gets the sample file, it uses sed to substitute final ${OEROOT} values for all ##OEROOT## values.

Note

You can see how the TEMPLATECONF variable is used by looking at the scripts/oe-setup-builddir script in the Source Directory. You can find the Yocto Project version of the local.conf.sample file in the meta-poky/conf/templates/default directory.

4.2.4 build/conf/bblayers.conf

This configuration file defines layers, which are directory trees, traversed (or walked) by BitBake. The bblayers.conf file uses the BBLAYERS variable to list the layers BitBake tries to find.

If bblayers.conf is not present when you start the build, the OpenEmbedded build system creates it from bblayers.conf.sample when you source the top-level build environment setup script (i.e. oe-init-build-env).

As with the local.conf file, the source bblayers.conf.sample file used depends on the TEMPLATECONF script variable, which defaults to meta-poky/conf/templates/default when you are building from the Yocto Project development environment, and to meta/conf/templates/default when you are building from the OpenEmbedded-Core environment. Because the script variable points to the source of the bblayers.conf.sample file, this implies that you can base your build from any layer by setting the variable in the top-level build environment setup script as follows:

TEMPLATECONF=your_layer/conf/templates/your_template_name

Once the build process gets the sample file, it uses sed to substitute final ${OEROOT} values for all ##OEROOT## values.

Note

You can see how the TEMPLATECONF variable is defined by the scripts/oe-setup-builddir script in the Source Directory. You can find the Yocto Project version of the bblayers.conf.sample file in the meta-poky/conf/templates/default directory.

4.2.5 build/conf/bblock.conf

This configuration file is generated by bblock and contains the signatures locked by bblock. By default, it does not exist and will be created upon the first invocation of bblock.

4.2.6 build/downloads/

This directory contains downloaded upstream source tarballs. You can reuse the directory for multiple builds or move the directory to another location. You can control the location of this directory through the DL_DIR variable.

4.2.7 build/sstate-cache/

This directory contains the shared state cache. You can reuse the directory for multiple builds or move the directory to another location. You can control the location of this directory through the SSTATE_DIR variable.

4.2.8 build/tmp/

The OpenEmbedded build system creates and uses this directory for all the build system’s output. The TMPDIR variable points to this directory.

BitBake creates this directory if it does not exist. As a last resort, to clean up a build and start it from scratch (other than the downloads), you can remove everything in the tmp directory or get rid of the directory completely. If you do, you should also completely remove the build/sstate-cache directory.

4.2.8.1 build/tmp/buildstats/

This directory stores the build statistics as generated by the buildstats class.

4.2.8.2 build/tmp/cache/

When BitBake parses the metadata (recipes and configuration files), it caches the results in build/tmp/cache/ to speed up future builds. The results are stored on a per-machine basis.

During subsequent builds, BitBake checks each recipe (together with, for example, any files included or appended to it) to see if they have been modified. Changes can be detected, for example, through file modification time (mtime) changes and hashing of file contents. If no changes to the file are detected, then the parsed result stored in the cache is reused. If the file has changed, it is reparsed.

4.2.8.3 build/tmp/deploy/

This directory contains any “end result” output from the OpenEmbedded build process. The DEPLOY_DIR variable points to this directory. For more detail on the contents of the deploy directory, see the “Images” and “Application Development SDK” sections in the Yocto Project Overview and Concepts Manual.

4.2.8.3.1 build/tmp/deploy/deb/

This directory receives any .deb packages produced by the build process. The packages are sorted into feeds for different architecture types.

4.2.8.3.2 build/tmp/deploy/rpm/

This directory receives any .rpm packages produced by the build process. The packages are sorted into feeds for different architecture types.

4.2.8.3.3 build/tmp/deploy/ipk/

This directory receives .ipk packages produced by the build process.

4.2.8.3.4 build/tmp/deploy/licenses/

This directory receives package licensing information. For example, the directory contains sub-directories for bash, busybox, and glibc (among others) that in turn contain appropriate COPYING license files with other licensing information. For information on licensing, see the “Maintaining Open Source License Compliance During Your Product’s Lifecycle” section in the Yocto Project Development Tasks Manual.

4.2.8.3.5 build/tmp/deploy/images/

This directory is populated with the basic output objects of the build (think of them as the “generated artifacts” of the build process), including things like the boot loader image, kernel, root filesystem and more. If you want to flash the resulting image from a build onto a device, look here for the necessary components.

Be careful when deleting files in this directory. You can safely delete old images from this directory (e.g. core-image-*). However, the kernel (*zImage*, *uImage*, etc.), bootloader and other supplementary files might be deployed here prior to building an image. Because these files are not directly produced from the image, if you delete them they will not be automatically re-created when you build the image again.

If you do accidentally delete files here, you will need to force them to be re-created. In order to do that, you will need to know the target that produced them. For example, these commands rebuild and re-create the kernel files:

$ bitbake -c clean virtual/kernel
$ bitbake virtual/kernel
4.2.8.3.6 build/tmp/deploy/sdk/

The OpenEmbedded build system creates this directory to hold toolchain installer scripts which, when executed, install the sysroot that matches your target hardware. You can find out more about these installers in the “Building an SDK Installer” section in the Yocto Project Application Development and the Extensible Software Development Kit (eSDK) manual.

4.2.8.4 build/tmp/hosttools/

The OpenEmbedded build system uses this directory to create symbolic links to some of the host components that are allowed to be called within tasks. These are basic components listed in the Required Packages for the Build Host section. These components are also listed in the HOSTTOOLS variable and are limited to this list to prevent host contamination.

4.2.8.5 build/tmp/pkgdata/

The OpenEmbedded build system uses this directory to store package metadata generated during the do_packagedata task. The files stored in this directory contain information about each output package produced by the OpenEmbedded build system, and are used in different ways by the build system such as “Viewing Package Information with oe-pkgdata-util”.

4.2.8.6 build/tmp/sstate-control/

The OpenEmbedded build system uses this directory for the shared state manifest files. The shared state code uses these files to record the files installed by each sstate task so that the files can be removed when cleaning the recipe or when a newer version is about to be installed. The build system also uses the manifests to detect and produce a warning when files from one task are overwriting those from another.

4.2.8.7 build/tmp/sysroots-components/

This directory is the location of the sysroot contents that the task do_prepare_recipe_sysroot links or copies into the recipe-specific sysroot for each recipe listed in DEPENDS. Population of this directory is handled through shared state, while the path is specified by the COMPONENTS_DIR variable. Apart from a few unusual circumstances, handling of the sysroots-components directory should be automatic, and recipes should not directly reference build/tmp/sysroots-components.

4.2.8.8 build/tmp/sysroots/

Previous versions of the OpenEmbedded build system used to create a global shared sysroot per machine along with a native sysroot. Since the 2.3 version of the Yocto Project, there are sysroots in recipe-specific WORKDIR directories. Thus, the build/tmp/sysroots/ directory is unused.

Note

The build/tmp/sysroots/ directory can still be populated using the bitbake build-sysroots command and can be used for compatibility in some cases. However, in general it is not recommended to populate this directory. Individual recipe-specific sysroots should be used.

4.2.8.9 build/tmp/stamps/

This directory holds information that BitBake uses for accounting purposes to track what tasks have run and when they have run. The directory is sub-divided by architecture, package name, and version. Here is an example:

stamps/all-poky-linux/distcc-config/1.0-r0.do_build-2fdd....2do

Although the files in the directory are empty of data, BitBake uses the filenames and timestamps for tracking purposes.

For information on how BitBake uses stamp files to determine if a task should be rerun, see the “Stamp Files and the Rerunning of Tasks” section in the Yocto Project Overview and Concepts Manual.

4.2.8.10 build/tmp/log/

This directory contains general logs that are not otherwise placed using the package’s WORKDIR. Examples of logs are the output from the do_check_pkg or do_distro_check tasks. Running a build does not necessarily mean this directory is created.

4.2.8.11 build/tmp/work/

This directory contains architecture-specific work sub-directories for packages built by BitBake. All tasks execute from the appropriate work directory. For example, the source for a particular package is unpacked, patched, configured and compiled all within its own work directory. Within the work directory, organization is based on the package group and version for which the source is being compiled as defined by the WORKDIR.

It is worth considering the structure of a typical work directory. As an example, consider linux-yocto-kernel-3.0 on the machine qemux86 built within the Yocto Project. For this package, a work directory of tmp/work/qemux86-poky-linux/linux-yocto/3.0+git1+<.....>, referred to as the WORKDIR, is created. Within this directory, the source is unpacked to linux-qemux86-standard-build and then patched by Quilt. (See the “Using Quilt in Your Workflow” section in the Yocto Project Development Tasks Manual for more information.) Within the linux-qemux86-standard-build directory, standard Quilt directories linux-3.0/patches and linux-3.0/.pc are created, and standard Quilt commands can be used.

There are other directories generated within WORKDIR. The most important directory is WORKDIR/temp/, which has log files for each task (log.do_*.pid) and contains the scripts BitBake runs for each task (run.do_*.pid). The WORKDIR/image/ directory is where “make install” places its output that is then split into sub-packages within WORKDIR/packages-split/.

4.2.8.11.1 build/tmp/work/tunearch/recipename/version/

The recipe work directory — ${WORKDIR}.

As described earlier in the “build/tmp/sysroots/” section, beginning with the 2.3 release of the Yocto Project, the OpenEmbedded build system builds each recipe in its own work directory (i.e. WORKDIR). The path to the work directory is constructed using the architecture of the given build (e.g. TUNE_PKGARCH, MACHINE_ARCH, or “allarch”), the recipe name, and the version of the recipe (i.e. PE:PV-PR).

Here are key subdirectories within each recipe work directory:

  • ${WORKDIR}/temp: Contains the log files of each task executed for this recipe, the “run” files for each executed task, which contain the code run, and a log.task_order file, which lists the order in which tasks were executed.

  • ${WORKDIR}/image: Contains the output of the do_install task, which corresponds to the ${D} variable in that task.

  • ${WORKDIR}/pseudo: Contains the pseudo database and log for any tasks executed under pseudo for the recipe.

  • ${WORKDIR}/sysroot-destdir: Contains the output of the do_populate_sysroot task.

  • ${WORKDIR}/package: Contains the output of the do_package task before the output is split into individual packages.

  • ${WORKDIR}/packages-split: Contains the output of the do_package task after the output has been split into individual packages. There are subdirectories for each individual package created by the recipe.

  • ${WORKDIR}/recipe-sysroot: A directory populated with the target dependencies of the recipe. This directory looks like the target filesystem and contains libraries that the recipe might need to link against (e.g. the C library).

  • ${WORKDIR}/recipe-sysroot-native: A directory populated with the native dependencies of the recipe. This directory contains the tools the recipe needs to build (e.g. the compiler, Autoconf, libtool, and so forth).

  • ${WORKDIR}/build: This subdirectory applies only to recipes that support builds where the source is separate from the build artifacts. The OpenEmbedded build system uses this directory as a separate build directory (i.e. ${B}).

4.2.8.12 build/tmp/work-shared/

For efficiency, the OpenEmbedded build system creates and uses this directory to hold recipes that share a work directory with other recipes. This is for example used for gcc and its variants (e.g. gcc-cross, libgcc, gcc-runtime, and so forth), or by the kernel class to make the kernel source code and kernel build artifacts available to out-of-tree kernel modules or other kernel-dependent recipes.

In practice, only a few recipes make use of the work-shared directory. This directory is especially useful for recipes that would induce a lot of storage space if they were to be shared with the standard Sysroot mechanism.

4.3 The Metadata — meta/

As mentioned previously, Metadata is the core of the Yocto Project. Metadata has several important subdivisions:

4.3.1 meta/classes*/

These directories contain the *.bbclass files. Class files are used to abstract common code so it can be reused by multiple packages. Every package inherits the base file. Examples of other important classes are autotools*, which in theory allows any Autotool-enabled package to work with the Yocto Project with minimal effort. Another example is kernel that contains common code and functions for working with the Linux kernel. Functions like image generation or packaging also have their specific class files such as image, rootfs* and package*.bbclass.

For reference information on classes, see the “Classes” chapter.

4.3.2 meta/conf/

This directory contains the core set of configuration files that start from bitbake.conf and from which all other configuration files are included. See the include statements at the end of the bitbake.conf file and you will note that even local.conf is loaded from there. While bitbake.conf sets up the defaults, you can often override these by using the (local.conf) file, machine file or the distribution configuration file.

4.3.2.1 meta/conf/machine/

This directory contains all the machine configuration files. If you set MACHINE = "qemux86", the OpenEmbedded build system looks for a qemux86.conf file in this directory. The include directory contains various data common to multiple machines. If you want to add support for a new machine to the Yocto Project, look in this directory.

4.3.2.2 meta/conf/distro/

The contents of this directory controls any distribution-specific configurations. For the Yocto Project, the defaultsetup.conf is the main file here. This directory includes the versions and the SRCDATE definitions for applications that are configured here. An example of an alternative configuration might be poky-bleeding.conf. Although this file mainly inherits its configuration from Poky.

4.3.2.3 meta/conf/machine-sdk/

The OpenEmbedded build system searches this directory for configuration files that correspond to the value of SDKMACHINE. By default, 32-bit and 64-bit x86 files ship with the Yocto Project that support some SDK hosts. However, it is possible to extend that support to other SDK hosts by adding additional configuration files in this subdirectory within another layer.

4.3.3 meta/files/

This directory contains common license files and several text files used by the build system. The text files contain minimal device information and lists of files and directories with known permissions.

4.3.4 meta/lib/

This directory contains OpenEmbedded Python library code used during the build process. It is enabled via the addpylib directive in meta/conf/local.conf. For more information, see Extending Python Library Code.

4.3.5 meta/recipes-bsp/

This directory contains anything linking to specific hardware or hardware configuration information such as “u-boot” and “grub”.

4.3.6 meta/recipes-connectivity/

This directory contains libraries and applications related to communication with other devices.

4.3.7 meta/recipes-core/

This directory contains what is needed to build a basic working Linux image including commonly used dependencies.

4.3.8 meta/recipes-devtools/

This directory contains tools that are primarily used by the build system. The tools, however, can also be used on targets.

4.3.9 meta/recipes-extended/

This directory contains non-essential applications that add features compared to the alternatives in core. You might need this directory for full tool functionality.

4.3.10 meta/recipes-gnome/

This directory contains all things related to the GTK+ application framework.

4.3.11 meta/recipes-graphics/

This directory contains X and other graphically related system libraries.

4.3.12 meta/recipes-kernel/

This directory contains the kernel and generic applications and libraries that have strong kernel dependencies.

4.3.13 meta/recipes-multimedia/

This directory contains codecs and support utilities for audio, images and video.

4.3.14 meta/recipes-rt/

This directory contains package and image recipes for using and testing the PREEMPT_RT kernel.

4.3.15 meta/recipes-sato/

This directory contains the Sato demo/reference UI/UX and its associated applications and configuration data.

4.3.16 meta/recipes-support/

This directory contains recipes used by other recipes, but that are not directly included in images (i.e. dependencies of other recipes).

4.3.17 meta/site/

This directory contains a list of cached results for various architectures. Because certain “autoconf” test results cannot be determined when cross-compiling due to the tests not able to run on a live system, the information in this directory is passed to “autoconf” for the various architectures.

4.3.18 meta/recipes.txt

This file is a description of the contents of recipes-*.