4 Source Directory Structure
The Source Directory consists of numerous files, directories and subdirectories; understanding their locations and contents is key to using the Yocto Project effectively. This chapter describes the Source Directory and gives information about those files and directories.
For information on how to establish a local Source Directory on your development system, see the “Locating Yocto Project Source Files” section in the Yocto Project Development Tasks Manual.
Note
The OpenEmbedded build system does not support file or directory names that contain spaces. Be sure that the Source Directory you use does not contain these types of names.
4.1 Top-Level Core Components
This section describes the top-level components of the Source Directory.
4.1.1 bitbake/
This directory includes a copy of BitBake for ease of use. The copy usually matches the current stable BitBake release from the BitBake project. BitBake, a Metadata interpreter, reads the Yocto Project Metadata and runs the tasks defined by that data. Failures are usually caused by errors in your Metadata and not from BitBake itself.
When you run the bitbake command, the main BitBake executable (which
resides in the bitbake/bin/ directory) starts. Sourcing the
environment setup script (i.e. oe-init-build-env) places
the scripts/ and bitbake/bin/ directories (in that order) into
the shell’s PATH environment variable.
For more information on BitBake, see the BitBake User Manual.
4.1.2 build/
This directory contains user configuration files and the output
generated by the OpenEmbedded build system in its standard configuration
where the source tree is combined with the output. The Build Directory
is created initially when you source
the OpenEmbedded build environment setup script (i.e.
oe-init-build-env).
It is also possible to place output and configuration files in a
directory separate from the Source Directory by
providing a directory name when you source the setup script. For
information on separating output from your local Source Directory files
(commonly described as an “out of tree” build), see the
“oe-init-build-env” section.
4.1.3 documentation/
This directory holds the source for the Yocto Project documentation as
well as templates and tools that allow you to generate PDF and HTML
versions of the manuals. Each manual is contained in its own sub-folder;
for example, the files for this reference manual reside in the
ref-manual/ directory.
4.1.4 meta/
This directory contains the minimal, underlying OpenEmbedded-Core
metadata. The directory holds recipes, common classes, and machine
configuration for strictly emulated targets (qemux86, qemuarm,
and so forth.)
4.1.5 meta-poky/
Designed above the meta/ content, this directory adds just enough
metadata to define the Poky reference distribution.
4.1.6 meta-yocto-bsp/
This directory contains the Yocto Project reference hardware Board Support Packages (BSPs). For more information on BSPs, see the Yocto Project Board Support Package Developer’s Guide.
4.1.7 meta-selftest/
This directory adds additional recipes and append files used by the
OpenEmbedded selftests to verify the behavior of the build system. You
do not have to add this layer to your bblayers.conf file unless you
want to run the selftests.
4.1.8 meta-skeleton/
This directory contains template recipes for BSP and kernel development.
4.1.9 scripts/
This directory contains various integration scripts that implement extra
functionality in the Yocto Project environment (e.g. QEMU scripts). The
oe-init-build-env script prepends this directory to the
shell’s PATH environment variable.
The scripts directory has useful scripts that assist in contributing
back to the Yocto Project, such as create-pull-request and
send-pull-request.
4.1.10 oe-init-build-env
This script sets up the OpenEmbedded build environment. Running this
script with the source command in a shell makes changes to PATH
and sets other core BitBake variables based on the current working
directory. You need to run an environment setup script before running
BitBake commands. The script uses other scripts within the scripts
directory to do the bulk of the work.
When you run this script, your Yocto Project environment is set up, a Build Directory is created, your working directory becomes the Build Directory, and you are presented with some simple suggestions as to what to do next, including a list of some possible targets to build. Here is an example:
$ source oe-init-build-env
### Shell environment set up for builds. ###
You can now run 'bitbake <target>'
Common targets are:
core-image-minimal
core-image-sato
meta-toolchain
meta-ide-support
You can also run generated QEMU images with a command like 'runqemu qemux86-64'
The default output of the oe-init-build-env script is from the
conf-notes.txt file, which is found in the meta-poky directory
within the Source Directory. If you design a
custom distribution, you can include your own version of this
configuration file to mention the targets defined by your distribution.
See the
“Creating a Custom Template Configuration Directory”
section in the Yocto Project Development Tasks Manual for more
information.
By default, running this script without a Build Directory argument
creates the build/ directory in your current working directory. If
you provide a Build Directory argument when you source the script,
you direct the OpenEmbedded build system to create a Build Directory of
your choice. For example, the following command creates a Build
Directory named mybuilds/ that is outside of the Source Directory:
$ source oe-init-build-env ~/mybuilds
The OpenEmbedded build system uses the template configuration files, which
are found by default in the meta-poky/conf/ directory in the Source
Directory. See the
“Creating a Custom Template Configuration Directory”
section in the Yocto Project Development Tasks Manual for more
information.
Note
The OpenEmbedded build system does not support file or directory
names that contain spaces. If you attempt to run the oe-init-build-env
script from a Source Directory that contains spaces in either the
filenames or directory names, the script returns an error indicating
no such file or directory. Be sure to use a Source Directory free of
names containing spaces.
4.1.11 LICENSE, README, and README.hardware
These files are standard top-level files.
4.2 The Build Directory — build/
The OpenEmbedded build system creates the Build Directory
when you run the build environment setup
script oe-init-build-env. If you do not give the Build
Directory a specific name when you run the setup script, the name
defaults to build/.
For subsequent parsing and processing, the name of the Build directory is available via the TOPDIR variable.
4.2.1 build/buildhistory/
The OpenEmbedded build system creates this directory when you enable build history via the buildhistory class file. The directory organizes build information into image, packages, and SDK subdirectories. For information on the build history feature, see the “Maintaining Build Output Quality” section in the Yocto Project Development Tasks Manual.
4.2.2 build/conf/local.conf
This configuration file contains all the local user configurations for
your build environment. The local.conf file contains documentation
on the various configuration options. Any variable set here overrides
any variable set elsewhere within the environment unless that variable
is hard-coded within a file (e.g. by using ‘=’ instead of ‘?=’). Some
variables are hard-coded for various reasons but such variables are
relatively rare.
At a minimum, you would normally edit this file to select the target MACHINE, which package types you wish to use (PACKAGE_CLASSES), and the location from which you want to access downloaded files (DL_DIR).
If local.conf is not present when you start the build, the
OpenEmbedded build system creates it from local.conf.sample when you
source the top-level build environment setup script
oe-init-build-env.
The source local.conf.sample file used depends on the
$TEMPLATECONF script variable, which defaults to meta-poky/conf/
when you are building from the Yocto Project development environment,
and to meta/conf/ when you are building from the OpenEmbedded-Core
environment. Because the script variable points to the source of the
local.conf.sample file, this implies that you can configure your
build environment from any layer by setting the variable in the
top-level build environment setup script as follows:
TEMPLATECONF=your_layer/conf
Once the build process gets the sample
file, it uses sed to substitute final
${OEROOT} values for all
##OEROOT## values.
Note
You can see how the TEMPLATECONF variable is used by looking at the
scripts/oe-setup-builddir script in the Source Directory.
You can find the Yocto Project version of the local.conf.sample file in
the meta-poky/conf directory.
4.2.3 build/conf/bblayers.conf
This configuration file defines
layers,
which are directory trees, traversed (or walked) by BitBake. The
bblayers.conf file uses the BBLAYERS
variable to list the layers BitBake tries to find.
If bblayers.conf is not present when you start the build, the
OpenEmbedded build system creates it from bblayers.conf.sample when
you source the top-level build environment setup script (i.e.
oe-init-build-env).
As with the local.conf file, the source bblayers.conf.sample
file used depends on the $TEMPLATECONF script variable, which
defaults to meta-poky/conf/ when you are building from the Yocto
Project development environment, and to meta/conf/ when you are
building from the OpenEmbedded-Core environment. Because the script
variable points to the source of the bblayers.conf.sample file, this
implies that you can base your build from any layer by setting the
variable in the top-level build environment setup script as follows:
TEMPLATECONF=your_layer/conf
Once the build process gets the sample file, it uses sed to substitute final
${OEROOT} values for all ##OEROOT## values.
Note
You can see how the TEMPLATECONF variable scripts/oe-setup-builddir
script in the Source Directory. You can find the Yocto Project
version of the bblayers.conf.sample file in the meta-poky/conf/
directory.
4.2.4 build/cache/sanity_info
This file indicates the state of the sanity checks and is created during the build.
4.2.5 build/downloads/
This directory contains downloaded upstream source tarballs. You can reuse the directory for multiple builds or move the directory to another location. You can control the location of this directory through the DL_DIR variable.
4.2.6 build/sstate-cache/
This directory contains the shared state cache. You can reuse the directory for multiple builds or move the directory to another location. You can control the location of this directory through the SSTATE_DIR variable.
4.2.7 build/tmp/
The OpenEmbedded build system creates and uses this directory for all the build system’s output. The TMPDIR variable points to this directory.
BitBake creates this directory if it does not exist. As a last resort,
to clean up a build and start it from scratch (other than the
downloads), you can remove everything in the tmp directory or get
rid of the directory completely. If you do, you should also completely
remove the build/sstate-cache directory.
4.2.8 build/tmp/buildstats/
This directory stores the build statistics.
4.2.9 build/tmp/cache/
When BitBake parses the metadata (recipes and configuration files), it
caches the results in build/tmp/cache/ to speed up future builds.
The results are stored on a per-machine basis.
During subsequent builds, BitBake checks each recipe (together with, for example, any files included or appended to it) to see if they have been modified. Changes can be detected, for example, through file modification time (mtime) changes and hashing of file contents. If no changes to the file are detected, then the parsed result stored in the cache is reused. If the file has changed, it is reparsed.
4.2.10 build/tmp/deploy/
This directory contains any “end result” output from the OpenEmbedded
build process. The DEPLOY_DIR variable points
to this directory. For more detail on the contents of the deploy
directory, see the
“Images” and
“Application Development SDK” sections in the Yocto
Project Overview and Concepts Manual.
4.2.11 build/tmp/deploy/deb/
This directory receives any .deb packages produced by the build
process. The packages are sorted into feeds for different architecture
types.
4.2.12 build/tmp/deploy/rpm/
This directory receives any .rpm packages produced by the build
process. The packages are sorted into feeds for different architecture
types.
4.2.13 build/tmp/deploy/ipk/
This directory receives .ipk packages produced by the build process.
4.2.14 build/tmp/deploy/licenses/
This directory receives package licensing information. For example, the
directory contains sub-directories for bash, busybox, and
glibc (among others) that in turn contain appropriate COPYING
license files with other licensing information. For information on
licensing, see the
“Maintaining Open Source License Compliance During Your Product’s Lifecycle”
section in the Yocto Project Development Tasks Manual.
4.2.15 build/tmp/deploy/images/
This directory is populated with the basic output objects of the build (think of them as the “generated artifacts” of the build process), including things like the boot loader image, kernel, root filesystem and more. If you want to flash the resulting image from a build onto a device, look here for the necessary components.
Be careful when deleting files in this directory. You can safely delete
old images from this directory (e.g. core-image-*). However, the
kernel (*zImage*, *uImage*, etc.), bootloader and other
supplementary files might be deployed here prior to building an image.
Because these files are not directly produced from the image, if you
delete them they will not be automatically re-created when you build the
image again.
If you do accidentally delete files here, you will need to force them to be re-created. In order to do that, you will need to know the target that produced them. For example, these commands rebuild and re-create the kernel files:
$ bitbake -c clean virtual/kernel
$ bitbake virtual/kernel
4.2.16 build/tmp/deploy/sdk/
The OpenEmbedded build system creates this directory to hold toolchain installer scripts which, when executed, install the sysroot that matches your target hardware. You can find out more about these installers in the “Building an SDK Installer” section in the Yocto Project Application Development and the Extensible Software Development Kit (eSDK) manual.
4.2.16.1 build/tmp/hosttools/
The OpenEmbedded build system uses this directory to create symbolic links to some of the host components that are allowed to be called within tasks. These are basic components listed in the Required Packages for the Build Host section. These components are also listed in the HOSTTOOLS variable and are limited to this list to prevent host contamination.
4.2.16.2 build/tmp/pkgdata/
The OpenEmbedded build system uses this directory to store package metadata generated during the do_packagedata task. The files stored in this directory contain information about each output package produced by the OpenEmbedded build system, and are used in different ways by the build system such as “Viewing Package Information with oe-pkgdata-util”.
4.2.17 build/tmp/sstate-control/
The OpenEmbedded build system uses this directory for the shared state manifest files. The shared state code uses these files to record the files installed by each sstate task so that the files can be removed when cleaning the recipe or when a newer version is about to be installed. The build system also uses the manifests to detect and produce a warning when files from one task are overwriting those from another.
4.2.18 build/tmp/sysroots-components/
This directory is the location of the sysroot contents that the task
do_prepare_recipe_sysroot
links or copies into the recipe-specific sysroot for each recipe listed
in DEPENDS. Population of this directory is
handled through shared state, while the path is specified by the
COMPONENTS_DIR variable. Apart from a few
unusual circumstances, handling of the sysroots-components directory
should be automatic, and recipes should not directly reference
build/tmp/sysroots-components.
4.2.19 build/tmp/sysroots/
Previous versions of the OpenEmbedded build system used to create a
global shared sysroot per machine along with a native sysroot. Since
the 2.3 version of the Yocto Project, there are sysroots in
recipe-specific WORKDIR directories. Thus, the
build/tmp/sysroots/ directory is unused.
Note
The build/tmp/sysroots/ directory can still be populated using the
bitbake build-sysroots command and can be used for compatibility in some
cases. However, in general it is not recommended to populate this directory.
Individual recipe-specific sysroots should be used.
4.2.20 build/tmp/stamps/
This directory holds information that BitBake uses for accounting purposes to track what tasks have run and when they have run. The directory is sub-divided by architecture, package name, and version. Here is an example:
stamps/all-poky-linux/distcc-config/1.0-r0.do_build-2fdd....2do
Although the files in the directory are empty of data, BitBake uses the filenames and timestamps for tracking purposes.
For information on how BitBake uses stamp files to determine if a task should be rerun, see the “Stamp Files and the Rerunning of Tasks” section in the Yocto Project Overview and Concepts Manual.
4.2.21 build/tmp/log/
This directory contains general logs that are not otherwise placed using
the package’s WORKDIR. Examples of logs are the output from the
do_check_pkg or do_distro_check tasks. Running a build does not
necessarily mean this directory is created.
4.2.22 build/tmp/work/
This directory contains architecture-specific work sub-directories for packages built by BitBake. All tasks execute from the appropriate work directory. For example, the source for a particular package is unpacked, patched, configured and compiled all within its own work directory. Within the work directory, organization is based on the package group and version for which the source is being compiled as defined by the WORKDIR.
It is worth considering the structure of a typical work directory. As an
example, consider linux-yocto-kernel-3.0 on the machine qemux86
built within the Yocto Project. For this package, a work directory of
tmp/work/qemux86-poky-linux/linux-yocto/3.0+git1+<.....>, referred
to as the WORKDIR, is created. Within this directory, the source is
unpacked to linux-qemux86-standard-build and then patched by Quilt.
(See the “Using Quilt in Your Workflow” section in
the Yocto Project Development Tasks Manual for more information.) Within
the linux-qemux86-standard-build directory, standard Quilt
directories linux-3.0/patches and linux-3.0/.pc are created, and
standard Quilt commands can be used.
There are other directories generated within WORKDIR. The most
important directory is WORKDIR/temp/, which has log files for each
task (log.do_*.pid) and contains the scripts BitBake runs for each
task (run.do_*.pid). The WORKDIR/image/ directory is where “make
install” places its output that is then split into sub-packages within
WORKDIR/packages-split/.
4.2.23 build/tmp/work/tunearch/recipename/version/
The recipe work directory — ${WORKDIR}.
As described earlier in the
“build/tmp/sysroots/” section,
beginning with the 2.3 release of the Yocto Project, the OpenEmbedded
build system builds each recipe in its own work directory (i.e.
WORKDIR). The path to the work directory is
constructed using the architecture of the given build (e.g.
TUNE_PKGARCH, MACHINE_ARCH, or “allarch”), the recipe
name, and the version of the recipe (i.e.
PE:PV-PR).
Here are key subdirectories within each recipe work directory:
${WORKDIR}/temp: Contains the log files of each task executed for this recipe, the “run” files for each executed task, which contain the code run, and alog.task_orderfile, which lists the order in which tasks were executed.${WORKDIR}/image: Contains the output of the do_install task, which corresponds to the${D}variable in that task.${WORKDIR}/pseudo: Contains the pseudo database and log for any tasks executed under pseudo for the recipe.${WORKDIR}/sysroot-destdir: Contains the output of the do_populate_sysroot task.${WORKDIR}/package: Contains the output of the do_package task before the output is split into individual packages.${WORKDIR}/packages-split: Contains the output of thedo_packagetask after the output has been split into individual packages. There are subdirectories for each individual package created by the recipe.${WORKDIR}/recipe-sysroot: A directory populated with the target dependencies of the recipe. This directory looks like the target filesystem and contains libraries that the recipe might need to link against (e.g. the C library).${WORKDIR}/recipe-sysroot-native: A directory populated with the native dependencies of the recipe. This directory contains the tools the recipe needs to build (e.g. the compiler, Autoconf, libtool, and so forth).${WORKDIR}/build: This subdirectory applies only to recipes that support builds where the source is separate from the build artifacts. The OpenEmbedded build system uses this directory as a separate build directory (i.e.${B}).
4.3 The Metadata — meta/
As mentioned previously, Metadata is the core of the Yocto Project. Metadata has several important subdivisions:
4.3.1 meta/classes/
This directory contains the *.bbclass files. Class files are used to
abstract common code so it can be reused by multiple packages. Every
package inherits the base file. Examples of other important
classes are autotools*, which in theory allows any
Autotool-enabled package to work with the Yocto Project with minimal
effort. Another example is kernel that contains common code
and functions for working with the Linux kernel. Functions like image
generation or packaging also have their specific class files such as
image, rootfs* and
package*.bbclass.
For reference information on classes, see the “Classes” chapter.
4.3.2 meta/conf/
This directory contains the core set of configuration files that start
from bitbake.conf and from which all other configuration files are
included. See the include statements at the end of the bitbake.conf
file and you will note that even local.conf is loaded from there.
While bitbake.conf sets up the defaults, you can often override
these by using the (local.conf) file, machine file or the
distribution configuration file.
4.3.3 meta/conf/machine/
This directory contains all the machine configuration files. If you set
MACHINE = "qemux86", the OpenEmbedded build system looks for a
qemux86.conf file in this directory. The include directory
contains various data common to multiple machines. If you want to add
support for a new machine to the Yocto Project, look in this directory.
4.3.4 meta/conf/distro/
The contents of this directory controls any distribution-specific
configurations. For the Yocto Project, the defaultsetup.conf is the
main file here. This directory includes the versions and the SRCDATE
definitions for applications that are configured here. An example of an
alternative configuration might be poky-bleeding.conf. Although this
file mainly inherits its configuration from Poky.
4.3.5 meta/conf/machine-sdk/
The OpenEmbedded build system searches this directory for configuration files that correspond to the value of SDKMACHINE. By default, 32-bit and 64-bit x86 files ship with the Yocto Project that support some SDK hosts. However, it is possible to extend that support to other SDK hosts by adding additional configuration files in this subdirectory within another layer.
4.3.6 meta/files/
This directory contains common license files and several text files used by the build system. The text files contain minimal device information and lists of files and directories with known permissions.
4.3.7 meta/lib/
This directory contains OpenEmbedded Python library code used during the build process.
4.3.8 meta/recipes-bsp/
This directory contains anything linking to specific hardware or hardware configuration information such as “u-boot” and “grub”.
4.3.9 meta/recipes-connectivity/
This directory contains libraries and applications related to communication with other devices.
4.3.10 meta/recipes-core/
This directory contains what is needed to build a basic working Linux image including commonly used dependencies.
4.3.11 meta/recipes-devtools/
This directory contains tools that are primarily used by the build system. The tools, however, can also be used on targets.
4.3.12 meta/recipes-extended/
This directory contains non-essential applications that add features compared to the alternatives in core. You might need this directory for full tool functionality.
4.3.13 meta/recipes-gnome/
This directory contains all things related to the GTK+ application framework.
4.3.14 meta/recipes-graphics/
This directory contains X and other graphically related system libraries.
4.3.15 meta/recipes-kernel/
This directory contains the kernel and generic applications and libraries that have strong kernel dependencies.
4.3.16 meta/recipes-multimedia/
This directory contains codecs and support utilities for audio, images and video.
4.3.17 meta/recipes-rt/
This directory contains package and image recipes for using and testing
the PREEMPT_RT kernel.
4.3.18 meta/recipes-sato/
This directory contains the Sato demo/reference UI/UX and its associated applications and configuration data.
4.3.19 meta/recipes-support/
This directory contains recipes used by other recipes, but that are not directly included in images (i.e. dependencies of other recipes).
4.3.20 meta/site/
This directory contains a list of cached results for various architectures. Because certain “autoconf” test results cannot be determined when cross-compiling due to the tests not able to run on a live system, the information in this directory is passed to “autoconf” for the various architectures.
4.3.21 meta/recipes.txt
This file is a description of the contents of recipes-*.