Yocto Project Application Development and the Extensible Software Development Kit (eSDK)

Welcome to the Yocto Project Application Development and the Extensible Software Development Kit (eSDK) manual. This manual provides information that explains how to use both the Yocto Project extensible and standard SDKs to develop applications and images.

All SDKs consist of the following:

Additionally, an extensible SDK has tools that allow you to easily add new applications and libraries to an image, modify the source of an existing component, test changes on the target hardware, and easily integrate an application into the OpenEmbedded build system.

You can use an SDK to independently develop and test code that is destined to run on some target machine. SDKs are completely self-contained. The binaries are linked against their own copy of libc, which results in no dependencies on the target system. To achieve this, the pointer to the dynamic loader is configured at install time since that path cannot be dynamically altered. This is the reason for a wrapper around the populate_sdk and populate_sdk_ext archives.

Another feature for the SDKs is that only one set of cross-compiler toolchain binaries are produced for any given architecture. This feature takes advantage of the fact that the target hardware can be passed to gcc as a set of compiler options. Those options are set up by the environment script and contained in variables such as CC and LD. This reduces the space needed for the tools. Understand, however, that every target still needs a sysroot because those binaries are target-specific.

The SDK development environment consists of the following:

In summary, the extensible and standard SDK share many features. However, the extensible SDK has powerful development tools to help you more quickly develop applications. Following is a table that summarizes the primary differences between the standard and extensible SDK types when considering which to build:

     * Extensible SDK contains the toolchain and debugger if SDK_EXT_TYPE is "full" or SDK_INCLUDE_TOOLCHAIN is "1", which is the default.

     ** Sysroot is managed through the use of devtool.  Thus, it is less likely that you will corrupt your SDK sysroot when you try to add additional libraries.

     *** You can add runtime package management to the standard SDK but it is not supported by default.

     **** You must build and make the shared state available to extensible SDK users for "packages" you want to enable users to install.
        

Fundamentally, the SDK fits into the development process as follows:

The SDK is installed on any machine and can be used to develop applications, images, and kernels. An SDK can even be used by a QA Engineer or Release Engineer. The fundamental concept is that the machine that has the SDK installed does not have to be associated with the machine that has the Yocto Project installed. A developer can independently compile and test an object on their machine and then, when the object is ready for integration into an image, they can simply make it available to the machine that has the Yocto Project. Once the object is available, the image can be rebuilt using the Yocto Project to produce the modified image.

You just need to follow these general steps:

The remainder of this manual describes how to use the extensible and standard SDKs. Information also exists in appendix form that describes how you can build, install, and modify an SDK.

The extensible SDK provides a cross-development toolchain and libraries tailored to the contents of a specific image. You would use the Extensible SDK if you want a toolchain experience supplemented with the powerful set of devtool commands tailored for the Yocto Project environment.

The installed extensible SDK consists of several files and directories. Basically, it contains an SDK environment setup script, some configuration files, an internal build system, and the devtool functionality.

The first thing you need to do is install the SDK on your Build Host by running the *.sh installation script.

You can download a tarball installer, which includes the pre-built toolchain, the runqemu script, the internal build system, devtool, and support files from the appropriate toolchain directory within the Index of Releases. Toolchains are available for several 32-bit and 64-bit architectures with the x86_64 directories, respectively. The toolchains the Yocto Project provides are based off the core-image-sato and core-image-minimal images and contain libraries appropriate for developing against that image.

The names of the tarball installer scripts are such that a string representing the host system appears first in the filename and then is immediately followed by a string representing the target architecture. An extensible SDK has the string "-ext" as part of the name. Following is the general form:

     poky-glibc-host_system-image_type-arch-toolchain-ext-release_version.sh

     Where:
         host_system is a string representing your development system:

                    i686 or x86_64.

         image_type is the image for which the SDK was built:

                    core-image-sato or core-image-minimal

         arch is a string representing the tuned target architecture:

                    aarch64, armv5e, core2-64, i586, mips32r2, mips64, ppc7400, or cortexa8hf-neon

         release_version is a string representing the release number of the Yocto Project:

                    3.1.1, 3.1.1+snapshot
            

For example, the following SDK installer is for a 64-bit development host system and a i586-tuned target architecture based off the SDK for core-image-sato and using the current 3.1.1 snapshot:

     poky-glibc-x86_64-core-image-sato-i586-toolchain-ext-3.1.1.sh
            

The SDK and toolchains are self-contained and by default are installed into the poky_sdk folder in your home directory. You can choose to install the extensible SDK in any location when you run the installer. However, because files need to be written under that directory during the normal course of operation, the location you choose for installation must be writable for whichever users need to use the SDK.

The following command shows how to run the installer given a toolchain tarball for a 64-bit x86 development host system and a 64-bit x86 target architecture. The example assumes the SDK installer is located in ~/Downloads/ and has execution rights.

     $ ./Downloads/poky-glibc-x86_64-core-image-minimal-core2-64-toolchain-ext-2.5.sh
     Poky (Yocto Project Reference Distro) Extensible SDK installer version 2.5
     ==========================================================================
     Enter target directory for SDK (default: ~/poky_sdk):
     You are about to install the SDK to "/home/scottrif/poky_sdk". Proceed [Y/n]? Y
     Extracting SDK..............done
     Setting it up...
     Extracting buildtools...
     Preparing build system...
     Parsing recipes: 100% |##################################################################| Time: 0:00:52
     Initialising tasks: 100% |###############################################################| Time: 0:00:00
     Checking sstate mirror object availability: 100% |#######################################| Time: 0:00:00
     Loading cache: 100% |####################################################################| Time: 0:00:00
     Initialising tasks: 100% |###############################################################| Time: 0:00:00
     done
     SDK has been successfully set up and is ready to be used.
     Each time you wish to use the SDK in a new shell session, you need to source the environment setup script e.g.
      $ . /home/scottrif/poky_sdk/environment-setup-core2-64-poky-linux

            

Once you have the SDK installed, you must run the SDK environment setup script before you can actually use the SDK. This setup script resides in the directory you chose when you installed the SDK, which is either the default poky_sdk directory or the directory you chose during installation.

Before running the script, be sure it is the one that matches the architecture for which you are developing. Environment setup scripts begin with the string "environment-setup" and include as part of their name the tuned target architecture. As an example, the following commands set the working directory to where the SDK was installed and then source the environment setup script. In this example, the setup script is for an IA-based target machine using i586 tuning:

     $ cd /home/scottrif/poky_sdk
     $ source environment-setup-core2-64-poky-linux
     SDK environment now set up; additionally you may now run devtool to perform development tasks.
     Run devtool --help for further details.
            

Running the setup script defines many environment variables needed in order to use the SDK (e.g. PATH, CC, LD, and so forth). If you want to see all the environment variables the script exports, examine the installation file itself.

The cornerstone of the extensible SDK is a command-line tool called devtool. This tool provides a number of features that help you build, test and package software within the extensible SDK, and optionally integrate it into an image built by the OpenEmbedded build system.

The devtool command line is organized similarly to Git in that it has a number of sub-commands for each function. You can run devtool --help to see all the commands.

Three devtool subcommands exist that provide entry-points into development:

As with the build system, "recipes" represent software packages within devtool. When you use devtool add, a recipe is automatically created. When you use devtool modify, the specified existing recipe is used in order to determine where to get the source code and how to patch it. In both cases, an environment is set up so that when you build the recipe a source tree that is under your control is used in order to allow you to make changes to the source as desired. By default, new recipes and the source go into a "workspace" directory under the SDK.

The remainder of this section presents the devtool add, devtool modify, and devtool upgrade workflows.

The devtool add command automatically creates a recipe based on the source tree you provide with the command. Currently, the command has support for the following:

Apart from binary packages, the determination of how a source tree should be treated is automatic based on the files present within that source tree. For example, if a CMakeLists.txt file is found, then the source tree is assumed to be using CMake and is treated accordingly.

The remainder of this section covers specifics regarding how parts of the recipe are generated.

     $ devtool reset -n recipename
                

     RDEPENDS_${PN} += "dependency1 dependency2 ..."
                

     $ devtool add "npm://registry.npmjs.org;name=forever;version=0.15.1"
                

     $ devtool add https://github.com/diversario/node-ssdp
                

When building a recipe using the devtool build command, the typical build progresses as follows:

For recipes in the workspace, fetching and unpacking is disabled as the source tree has already been prepared and is persistent. Each of these build steps is defined as a function (task), usually with a "do_" prefix (e.g. do_fetch, do_unpack, and so forth). These functions are typically shell scripts but can instead be written in Python.

If you look at the contents of a recipe, you will see that the recipe does not include complete instructions for building the software. Instead, common functionality is encapsulated in classes inherited with the inherit directive. This technique leaves the recipe to describe just the things that are specific to the software being built. A base class exists that is implicitly inherited by all recipes and provides the functionality that most recipes typically need.

The remainder of this section presents information useful when working with recipes.

If you use the devtool deploy-target command to write a recipe's build output to the target, and you are working on an existing component of the system, then you might find yourself in a situation where you need to restore the original files that existed prior to running the devtool deploy-target command. Because the devtool deploy-target command backs up any files it overwrites, you can use the devtool undeploy-target command to restore those files and remove any other files the recipe deployed. Consider the following example:

     $ devtool undeploy-target lighttpd root@192.168.7.2
            

If you have deployed multiple applications, you can remove them all using the "-a" option thus restoring the target device to its original state:

     $ devtool undeploy-target -a root@192.168.7.2
            

Information about files deployed to the target as well as any backed up files are stored on the target itself. This storage, of course, requires some additional space on the target machine.

Out of the box the extensible SDK typically only comes with a small number of tools and libraries. A minimal SDK starts mostly empty and is populated on-demand. Sometimes you must explicitly install extra items into the SDK. If you need these extra items, you can first search for the items using the devtool search command. For example, suppose you need to link to libGL but you are not sure which recipe provides libGL. You can use the following command to find out:

     $ devtool search libGL
     mesa                  A free implementation of the OpenGL API
            

Once you know the recipe (i.e. mesa in this example), you can install it:

     $ devtool sdk-install mesa
            

By default, the devtool sdk-install command assumes the item is available in pre-built form from your SDK provider. If the item is not available and it is acceptable to build the item from source, you can add the "-s" option as follows:

     $ devtool sdk-install -s mesa
            

It is important to remember that building the item from source takes significantly longer than installing the pre-built artifact. Also, if no recipe exists for the item you want to add to the SDK, you must instead add the item using the devtool add command.

If you are working with an installed extensible SDK that gets occasionally updated (e.g. a third-party SDK), then you will need to manually "pull down" the updates into the installed SDK.

To update your installed SDK, use devtool as follows:

     $ devtool sdk-update
            

The previous command assumes your SDK provider has set the default update URL for you through the SDK_UPDATE_URL variable as described in the "Providing Updates to the Extensible SDK After Installation" section. If the SDK provider has not set that default URL, you need to specify it yourself in the command as follows:

     $ devtool sdk-update path_to_update_directory
            

You might need to produce an SDK that contains your own custom libraries. A good example would be if you were a vendor with customers that use your SDK to build their own platform-specific software and those customers need an SDK that has custom libraries. In such a case, you can produce a derivative SDK based on the currently installed SDK fairly easily by following these steps:

The previous steps take the recipes added to the workspace and construct a new SDK installer that contains those recipes and the resulting binary artifacts. The recipes go into their own separate layer in the constructed derivative SDK, which leaves the workspace clean and ready for users to add their own recipes.

The Standard SDK provides a cross-development toolchain and libraries tailored to the contents of a specific image. You would use the Standard SDK if you want a more traditional toolchain experience as compared to the extensible SDK, which provides an internal build system and the devtool functionality.

The installed Standard SDK consists of several files and directories. Basically, it contains an SDK environment setup script, some configuration files, and host and target root filesystems to support usage. You can see the directory structure in the "Installed Standard SDK Directory Structure" section.

The first thing you need to do is install the SDK on your Build Host by running the *.sh installation script.

You can download a tarball installer, which includes the pre-built toolchain, the runqemu script, and support files from the appropriate toolchain directory within the Index of Releases. Toolchains are available for several 32-bit and 64-bit architectures with the x86_64 directories, respectively. The toolchains the Yocto Project provides are based off the core-image-sato and core-image-minimal images and contain libraries appropriate for developing against that image.

The names of the tarball installer scripts are such that a string representing the host system appears first in the filename and then is immediately followed by a string representing the target architecture.

     poky-glibc-host_system-image_type-arch-toolchain-release_version.sh

     Where:
         host_system is a string representing your development system:

                    i686 or x86_64.

         image_type is the image for which the SDK was built:

                    core-image-minimal or core-image-sato.

         arch is a string representing the tuned target architecture:

                    aarch64, armv5e, core2-64, i586, mips32r2, mips64, ppc7400, or cortexa8hf-neon.

         release_version is a string representing the release number of the Yocto Project:

                    3.1.1, 3.1.1+snapshot
            

For example, the following SDK installer is for a 64-bit development host system and a i586-tuned target architecture based off the SDK for core-image-sato and using the current 3.1.1 snapshot:

     poky-glibc-x86_64-core-image-sato-i586-toolchain-3.1.1.sh
            

The SDK and toolchains are self-contained and by default are installed into the poky_sdk folder in your home directory. You can choose to install the extensible SDK in any location when you run the installer. However, because files need to be written under that directory during the normal course of operation, the location you choose for installation must be writable for whichever users need to use the SDK.

The following command shows how to run the installer given a toolchain tarball for a 64-bit x86 development host system and a 64-bit x86 target architecture. The example assumes the SDK installer is located in ~/Downloads/ and has execution rights.

     $ ./Downloads/poky-glibc-x86_64-core-image-sato-i586-toolchain-3.1.1.sh
     Poky (Yocto Project Reference Distro) SDK installer version 3.1.1
     ===============================================================
     Enter target directory for SDK (default: /opt/poky/3.1.1):
     You are about to install the SDK to "/opt/poky/3.1.1". Proceed [Y/n]? Y
     Extracting SDK........................................ ..............................done
     Setting it up...done
     SDK has been successfully set up and is ready to be used.
     Each time you wish to use the SDK in a new shell session, you need to source the environment setup script e.g.
      $ . /opt/poky/3.1.1/environment-setup-i586-poky-linux
            

Again, reference the "Installed Standard SDK Directory Structure" section for more details on the resulting directory structure of the installed SDK.

Once you have the SDK installed, you must run the SDK environment setup script before you can actually use the SDK. This setup script resides in the directory you chose when you installed the SDK, which is either the default /opt/poky/3.1.1 directory or the directory you chose during installation.

Before running the script, be sure it is the one that matches the architecture for which you are developing. Environment setup scripts begin with the string "environment-setup" and include as part of their name the tuned target architecture. As an example, the following commands set the working directory to where the SDK was installed and then source the environment setup script. In this example, the setup script is for an IA-based target machine using i586 tuning:

     $ source /opt/poky/3.1.1/environment-setup-i586-poky-linux
            

When you run the setup script, the same environment variables are defined as are when you run the setup script for an extensible SDK. See the "Running the Extensible SDK Environment Setup Script" section for more information.

Once you have a suitable cross-development toolchain installed, it is very easy to develop a project using the GNU Autotools-based workflow, which is outside of the OpenEmbedded build system.

The following figure presents a simple Autotools workflow.

Follow these steps to create a simple Autotools-based "Hello World" project:

Simple Makefile-based projects use and interact with the cross-toolchain environment variables established when you run the cross-toolchain environment setup script. The environment variables are subject to general make rules.

This section presents a simple Makefile development flow and provides an example that lets you see how you can use cross-toolchain environment variables and Makefile variables during development.

The main point of this section is to explain the following three cases regarding variable behavior:

     $ make -e target
                

The remainder of this section presents a simple Makefile example that demonstrates these variable behaviors.

In a new shell environment variables are not established for the SDK until you run the setup script. For example, the following commands show a null value for the compiler variable (i.e. CC).

     $ echo ${CC}

     $
            

Running the SDK setup script for a 64-bit build host and an i586-tuned target architecture for a core-image-sato image using the current 3.1.1 Yocto Project release and then echoing that variable shows the value established through the script:

     $ source /opt/poky/3.1.1/environment-setup-i586-poky-linux
     $ echo ${CC}
     i586-poky-linux-gcc -m32 -march=i586 --sysroot=/opt/poky/2.5/sysroots/i586-poky-linux
            

To illustrate variable use, work through this simple "Hello World!" example: