10 Upgrading Recipes

Over time, upstream developers publish new versions for software built by layer recipes. It is recommended to keep recipes up-to-date with upstream version releases.

While there are several methods to upgrade a recipe, you might consider checking on the upgrade status of a recipe first. You can do so using the devtool check-upgrade-status command. See the “Checking on the Upgrade Status of a Recipe” section in the Yocto Project Reference Manual for more information.

The remainder of this section describes three ways you can upgrade a recipe. You can use the Automated Upgrade Helper (AUH) to set up automatic version upgrades. Alternatively, you can use devtool upgrade to set up semi-automatic version upgrades. Finally, you can manually upgrade a recipe by editing the recipe itself.

10.1 Using the Auto Upgrade Helper (AUH)

The AUH utility works in conjunction with the OpenEmbedded build system in order to automatically generate upgrades for recipes based on new versions being published upstream. Use AUH when you want to create a service that performs the upgrades automatically and optionally sends you an email with the results.

AUH allows you to update several recipes with a single use. You can also optionally perform build and integration tests using images with the results saved to your hard drive and emails of results optionally sent to recipe maintainers. Finally, AUH creates Git commits with appropriate commit messages in the layer’s tree for the changes made to recipes.

Note

In some conditions, you should not use AUH to upgrade recipes and should instead use either devtool upgrade or upgrade your recipes manually:

When AUH cannot complete the upgrade sequence. This situation usually results because custom patches carried by the recipe cannot be automatically rebased to the new version. In this case, devtool upgrade allows you to manually resolve conflicts.
When for any reason you want fuller control over the upgrade process. For example, when you want special arrangements for testing.

The following steps describe how to set up the AUH utility:

Be Sure the Development Host is Set Up: You need to be sure that your development host is set up to use the Yocto Project. For information on how to set up your host, see the “Preparing the Build Host” section.
Make Sure Git is Configured: The AUH utility requires Git to be configured because AUH uses Git to save upgrades. Thus, you must have Git user and email configured. The following command shows your configurations:
```
$ git config --list
```
If you do not have the user and email configured, you can use the following commands to do so:
```
$ git config --global user.name some_name
$ git config --global user.email username@domain.com
```

Clone the AUH Repository: To use AUH, you must clone the repository onto your development host. The following command uses Git to create a local copy of the repository on your system:

$ git clone git://git.yoctoproject.org/auto-upgrade-helper
Cloning into 'auto-upgrade-helper'... remote: Counting objects: 768, done.
remote: Compressing objects: 100% (300/300), done.
remote: Total 768 (delta 499), reused 703 (delta 434)
Receiving objects: 100% (768/768), 191.47 KiB | 98.00 KiB/s, done.
Resolving deltas: 100% (499/499), done.
Checking connectivity... done.

AUH is not part of the OpenEmbedded-Core (OE-Core) or Poky repositories.

Create a Dedicated Build Directory: Run the oe-init-build-env script to create a fresh Build Directory that you use exclusively for running the AUH utility:
```
$ cd poky
$ source oe-init-build-env your_AUH_build_directory
```
Re-using an existing Build Directory and its configurations is not recommended as existing settings could cause AUH to fail or behave undesirably.
Make Configurations in Your Local Configuration File: Several settings are needed in the local.conf file in the build directory you just created for AUH. Make these following configurations:
- If you want to enable Build History, which is optional, you need the following lines in the conf/local.conf file:
```
INHERIT =+ "buildhistory"
BUILDHISTORY_COMMIT = "1"
```
  With this configuration and a successful upgrade, a build history “diff” file appears in the upgrade-helper/work/recipe/buildhistory-diff.txt file found in your Build Directory.
- If you want to enable testing through the testimage class, which is optional, you need to have the following set in your conf/local.conf file:
```
IMAGE_CLASSES += "testimage"
```
  Note
  
  If your distro does not enable by default ptest, which Poky does, you need the following in your local.conf file:
  DISTRO_FEATURES:append = " ptest"
Optionally Start a vncserver: If you are running in a server without an X11 session, you need to start a vncserver:
```
$ vncserver :1
$ export DISPLAY=:1
```
Create and Edit an AUH Configuration File: You need to have the upgrade-helper/upgrade-helper.conf configuration file in your Build Directory. You can find a sample configuration file in the AUH source repository.

Read through the sample file and make configurations as needed. For example, if you enabled build history in your local.conf as described earlier, you must enable it in upgrade-helper.conf.

Also, if you are using the default maintainers.inc file supplied with Poky and located in meta-yocto and you do not set a “maintainers_whitelist” or “global_maintainer_override” in the upgrade-helper.conf configuration, and you specify “-e all” on the AUH command-line, the utility automatically sends out emails to all the default maintainers. Please avoid this.

This next set of examples describes how to use the AUH:

Upgrading a Specific Recipe: To upgrade a specific recipe, use the following form:
```
$ upgrade-helper.py recipe_name
```
For example, this command upgrades the xmodmap recipe:
```
$ upgrade-helper.py xmodmap
```
Upgrading a Specific Recipe to a Particular Version: To upgrade a specific recipe to a particular version, use the following form:
```
$ upgrade-helper.py recipe_name -t version
```
For example, this command upgrades the xmodmap recipe to version 1.2.3:
```
$ upgrade-helper.py xmodmap -t 1.2.3
```
Upgrading all Recipes to the Latest Versions and Suppressing Email Notifications: To upgrade all recipes to their most recent versions and suppress the email notifications, use the following command:
```
$ upgrade-helper.py all
```
Upgrading all Recipes to the Latest Versions and Send Email Notifications: To upgrade all recipes to their most recent versions and send email messages to maintainers for each attempted recipe as well as a status email, use the following command:
```
$ upgrade-helper.py -e all
```

Once you have run the AUH utility, you can find the results in the AUH Build Directory:

${BUILDDIR}/upgrade-helper/timestamp

The AUH utility also creates recipe update commits from successful upgrade attempts in the layer tree.

You can easily set up to run the AUH utility on a regular basis by using a cron job. See the weeklyjob.sh file distributed with the utility for an example.

10.2 Using `devtool upgrade`

As mentioned earlier, an alternative method for upgrading recipes to newer versions is to use devtool upgrade. You can read about devtool upgrade in general in the “Use devtool upgrade to Create a Version of the Recipe that Supports a Newer Version of the Software” section in the Yocto Project Application Development and the Extensible Software Development Kit (eSDK) Manual.

To see all the command-line options available with devtool upgrade, use the following help command:

$ devtool upgrade -h

If you want to find out what version a recipe is currently at upstream without any attempt to upgrade your local version of the recipe, you can use the following command:

$ devtool latest-version recipe_name

As mentioned in the previous section describing AUH, devtool upgrade works in a less-automated manner than AUH. Specifically, devtool upgrade only works on a single recipe that you name on the command line, cannot perform build and integration testing using images, and does not automatically generate commits for changes in the source tree. Despite all these “limitations”, devtool upgrade updates the recipe file to the new upstream version and attempts to rebase custom patches contained by the recipe as needed.

Note

AUH uses much of devtool upgrade behind the scenes making AUH somewhat of a “wrapper” application for devtool upgrade.

A typical scenario involves having used Git to clone an upstream repository that you use during build operations. Because you have built the recipe in the past, the layer is likely added to your configuration already. If for some reason, the layer is not added, you could add it easily using the “bitbake-layers” script. For example, suppose you use the nano.bb recipe from the meta-oe layer in the meta-openembedded repository. For this example, assume that the layer has been cloned into following area:

/home/scottrif/meta-openembedded

The following command from your Build Directory adds the layer to your build configuration (i.e. ${BUILDDIR}/conf/bblayers.conf):

$ bitbake-layers add-layer /home/scottrif/meta-openembedded/meta-oe
NOTE: Starting bitbake server...
Parsing recipes: 100% |##########################################| Time: 0:00:55
Parsing of 1431 .bb files complete (0 cached, 1431 parsed). 2040 targets, 56 skipped, 0 masked, 0 errors.
Removing 12 recipes from the x86_64 sysroot: 100% |##############| Time: 0:00:00
Removing 1 recipes from the x86_64_i586 sysroot: 100% |##########| Time: 0:00:00
Removing 5 recipes from the i586 sysroot: 100% |#################| Time: 0:00:00
Removing 5 recipes from the qemux86 sysroot: 100% |##############| Time: 0:00:00

For this example, assume that the nano.bb recipe that is upstream has a 2.9.3 version number. However, the version in the local repository is 2.7.4. The following command from your build directory automatically upgrades the recipe for you:

$ devtool upgrade nano -V 2.9.3
NOTE: Starting bitbake server...
NOTE: Creating workspace layer in /home/scottrif/poky/build/workspace
Parsing recipes: 100% |##########################################| Time: 0:00:46
Parsing of 1431 .bb files complete (0 cached, 1431 parsed). 2040 targets, 56 skipped, 0 masked, 0 errors.
NOTE: Extracting current version source...
NOTE: Resolving any missing task queue dependencies
       .
       .
       .
NOTE: Executing SetScene Tasks
NOTE: Executing RunQueue Tasks
NOTE: Tasks Summary: Attempted 74 tasks of which 72 didn't need to be rerun and all succeeded.
Adding changed files: 100% |#####################################| Time: 0:00:00
NOTE: Upgraded source extracted to /home/scottrif/poky/build/workspace/sources/nano
NOTE: New recipe is /home/scottrif/poky/build/workspace/recipes/nano/nano_2.9.3.bb

Note

Using the -V option is not necessary. Omitting the version number causes devtool upgrade to upgrade the recipe to the most recent version.

Continuing with this example, you can use devtool build to build the newly upgraded recipe:

$ devtool build nano
NOTE: Starting bitbake server...
Loading cache: 100% |################################################################################################| Time: 0:00:01
Loaded 2040 entries from dependency cache.
Parsing recipes: 100% |##############################################################################################| Time: 0:00:00
Parsing of 1432 .bb files complete (1431 cached, 1 parsed). 2041 targets, 56 skipped, 0 masked, 0 errors.
NOTE: Resolving any missing task queue dependencies
       .
       .
       .
NOTE: Executing SetScene Tasks
NOTE: Executing RunQueue Tasks
NOTE: nano: compiling from external source tree /home/scottrif/poky/build/workspace/sources/nano
NOTE: Tasks Summary: Attempted 520 tasks of which 304 didn't need to be rerun and all succeeded.

Within the devtool upgrade workflow, you can deploy and test your rebuilt software. For this example, however, running devtool finish cleans up the workspace once the source in your workspace is clean. This usually means using Git to stage and submit commits for the changes generated by the upgrade process.

Once the tree is clean, you can clean things up in this example with the following command from the ${BUILDDIR}/workspace/sources/nano directory:

$ devtool finish nano meta-oe
NOTE: Starting bitbake server...
Loading cache: 100% |################################################################################################| Time: 0:00:00
Loaded 2040 entries from dependency cache.
Parsing recipes: 100% |##############################################################################################| Time: 0:00:01
Parsing of 1432 .bb files complete (1431 cached, 1 parsed). 2041 targets, 56 skipped, 0 masked, 0 errors.
NOTE: Adding new patch 0001-nano.bb-Stuff-I-changed-when-upgrading-nano.bb.patch
NOTE: Updating recipe nano_2.9.3.bb
NOTE: Removing file /home/scottrif/meta-openembedded/meta-oe/recipes-support/nano/nano_2.7.4.bb
NOTE: Moving recipe file to /home/scottrif/meta-openembedded/meta-oe/recipes-support/nano
NOTE: Leaving source tree /home/scottrif/poky/build/workspace/sources/nano as-is; if you no longer need it then please delete it manually

Using the devtool finish command cleans up the workspace and creates a patch file based on your commits. The tool puts all patch files back into the source directory in a sub-directory named nano in this case.

10.3 Manually Upgrading a Recipe

If for some reason you choose not to upgrade recipes using Using the Auto Upgrade Helper (AUH) or by Using devtool upgrade, you can manually edit the recipe files to upgrade the versions.

Note

Manually updating multiple recipes scales poorly and involves many steps. The recommendation to upgrade recipe versions is through AUH or devtool upgrade, both of which automate some steps and provide guidance for others needed for the manual process.

To manually upgrade recipe versions, follow these general steps:

Change the Version: Rename the recipe such that the version (i.e. the PV part of the recipe name) changes appropriately. If the version is not part of the recipe name, change the value as it is set for PV within the recipe itself.
Update SRCREV if Needed: If the source code your recipe builds is fetched from Git or some other version control system, update SRCREV to point to the commit hash that matches the new version.
Build the Software: Try to build the recipe using BitBake. Typical build failures include the following:
- License statements were updated for the new version. For this case, you need to review any changes to the license and update the values of LICENSE and LIC_FILES_CHKSUM as needed.
  
  Note
  
  License changes are often inconsequential. For example, the license text’s copyright year might have changed.
- Custom patches carried by the older version of the recipe might fail to apply to the new version. For these cases, you need to review the failures. Patches might not be necessary for the new version of the software if the upgraded version has fixed those issues. If a patch is necessary and failing, you need to rebase it into the new version.
Optionally Attempt to Build for Several Architectures: Once you successfully build the new software for a given architecture, you could test the build for other architectures by changing the MACHINE variable and rebuilding the software. This optional step is especially important if the recipe is to be released publicly.
Check the Upstream Change Log or Release Notes: Checking both these reveals if there are new features that could break backwards-compatibility. If so, you need to take steps to mitigate or eliminate that situation.
Optionally Create a Bootable Image and Test: If you want, you can test the new software by booting it onto actual hardware.
Create a Commit with the Change in the Layer Repository: After all builds work and any testing is successful, you can create commits for any changes in the layer holding your upgraded recipe.

10 Upgrading Recipes

10.1 Using the Auto Upgrade Helper (AUH)

10.2 Using devtool upgrade

10.3 Manually Upgrading a Recipe

10.2 Using `devtool upgrade`