7 Upgrading Recipes
Over time, upstream developers publish new versions for software built by layer recipes. It is recommended to keep recipes up-to-date with upstream version releases.
While there are several methods to upgrade a recipe, you might
consider checking on the upgrade status of a recipe first. You can do so
using the devtool check-upgrade-status
command. See the
“Checking on the Upgrade Status of a Recipe”
section in the Yocto Project Reference Manual for more information.
The remainder of this section describes three ways you can upgrade a
recipe. You can use the Automated Upgrade Helper (AUH) to set up
automatic version upgrades. Alternatively, you can use
devtool upgrade
to set up semi-automatic version upgrades. Finally,
you can manually upgrade a recipe by editing the recipe itself.
7.1 Using the Auto Upgrade Helper (AUH)
The AUH utility works in conjunction with the OpenEmbedded build system in order to automatically generate upgrades for recipes based on new versions being published upstream. Use AUH when you want to create a service that performs the upgrades automatically and optionally sends you an email with the results.
AUH allows you to update several recipes with a single use. You can also optionally perform build and integration tests using images with the results saved to your hard drive and emails of results optionally sent to recipe maintainers. Finally, AUH creates Git commits with appropriate commit messages in the layer’s tree for the changes made to recipes.
Note
In some conditions, you should not use AUH to upgrade recipes
and should instead use either devtool upgrade
or upgrade your
recipes manually:
When AUH cannot complete the upgrade sequence. This situation usually results because custom patches carried by the recipe cannot be automatically rebased to the new version. In this case,
devtool upgrade
allows you to manually resolve conflicts.When for any reason you want fuller control over the upgrade process. For example, when you want special arrangements for testing.
The following steps describe how to set up the AUH utility:
Be Sure the Development Host is Set Up: You need to be sure that your development host is set up to use the Yocto Project. For information on how to set up your host, see the “Preparing the Build Host” section.
Make Sure Git is Configured: The AUH utility requires Git to be configured because AUH uses Git to save upgrades. Thus, you must have Git user and email configured. The following command shows your configurations:
$ git config --list
If you do not have the user and email configured, you can use the following commands to do so:
$ git config --global user.name some_name $ git config --global user.email username@domain.com
Clone the AUH Repository: To use AUH, you must clone the repository onto your development host. The following command uses Git to create a local copy of the repository on your system:
$ git clone git://git.yoctoproject.org/auto-upgrade-helper Cloning into 'auto-upgrade-helper'... remote: Counting objects: 768, done. remote: Compressing objects: 100% (300/300), done. remote: Total 768 (delta 499), reused 703 (delta 434) Receiving objects: 100% (768/768), 191.47 KiB | 98.00 KiB/s, done. Resolving deltas: 100% (499/499), done. Checking connectivity... done.
AUH is not part of the OpenEmbedded-Core (OE-Core) or Poky repositories.
Create a Dedicated Build Directory: Run the oe-init-build-env script to create a fresh Build Directory that you use exclusively for running the AUH utility:
$ cd poky $ source oe-init-build-env your_AUH_build_directory
Re-using an existing Build Directory and its configurations is not recommended as existing settings could cause AUH to fail or behave undesirably.
Make Configurations in Your Local Configuration File: Several settings are needed in the
local.conf
file in the build directory you just created for AUH. Make these following configurations:If you want to enable Build History, which is optional, you need the following lines in the
conf/local.conf
file:INHERIT =+ "buildhistory" BUILDHISTORY_COMMIT = "1"
With this configuration and a successful upgrade, a build history “diff” file appears in the
upgrade-helper/work/recipe/buildhistory-diff.txt
file found in your Build Directory.If you want to enable testing through the testimage class, which is optional, you need to have the following set in your
conf/local.conf
file:IMAGE_CLASSES += "testimage"
Note
If your distro does not enable by default ptest, which Poky does, you need the following in your
local.conf
file:DISTRO_FEATURES:append = " ptest"
Optionally Start a vncserver: If you are running in a server without an X11 session, you need to start a vncserver:
$ vncserver :1 $ export DISPLAY=:1
Create and Edit an AUH Configuration File: You need to have the
upgrade-helper/upgrade-helper.conf
configuration file in your Build Directory. You can find a sample configuration file in the AUH source repository.Read through the sample file and make configurations as needed. For example, if you enabled build history in your
local.conf
as described earlier, you must enable it inupgrade-helper.conf
.Also, if you are using the default
maintainers.inc
file supplied with Poky and located inmeta-yocto
and you do not set a “maintainers_whitelist” or “global_maintainer_override” in theupgrade-helper.conf
configuration, and you specify “-e all” on the AUH command-line, the utility automatically sends out emails to all the default maintainers. Please avoid this.
This next set of examples describes how to use the AUH:
Upgrading a Specific Recipe: To upgrade a specific recipe, use the following form:
$ upgrade-helper.py recipe_name
For example, this command upgrades the
xmodmap
recipe:$ upgrade-helper.py xmodmap
Upgrading a Specific Recipe to a Particular Version: To upgrade a specific recipe to a particular version, use the following form:
$ upgrade-helper.py recipe_name -t version
For example, this command upgrades the
xmodmap
recipe to version 1.2.3:$ upgrade-helper.py xmodmap -t 1.2.3
Upgrading all Recipes to the Latest Versions and Suppressing Email Notifications: To upgrade all recipes to their most recent versions and suppress the email notifications, use the following command:
$ upgrade-helper.py all
Upgrading all Recipes to the Latest Versions and Send Email Notifications: To upgrade all recipes to their most recent versions and send email messages to maintainers for each attempted recipe as well as a status email, use the following command:
$ upgrade-helper.py -e all
Once you have run the AUH utility, you can find the results in the AUH Build Directory:
${BUILDDIR}/upgrade-helper/timestamp
The AUH utility also creates recipe update commits from successful upgrade attempts in the layer tree.
You can easily set up to run the AUH utility on a regular basis by using a cron job. See the weeklyjob.sh file distributed with the utility for an example.
7.2 Using devtool upgrade
As mentioned earlier, an alternative method for upgrading recipes to
newer versions is to use
devtool upgrade.
You can read about devtool upgrade
in general in the
“Use devtool upgrade to Create a Version of the Recipe that Supports a Newer Version of the Software”
section in the Yocto Project Application Development and the Extensible
Software Development Kit (eSDK) Manual.
To see all the command-line options available with devtool upgrade
,
use the following help command:
$ devtool upgrade -h
If you want to find out what version a recipe is currently at upstream without any attempt to upgrade your local version of the recipe, you can use the following command:
$ devtool latest-version recipe_name
As mentioned in the previous section describing AUH, devtool upgrade
works in a less-automated manner than AUH. Specifically,
devtool upgrade
only works on a single recipe that you name on the
command line, cannot perform build and integration testing using images,
and does not automatically generate commits for changes in the source
tree. Despite all these “limitations”, devtool upgrade
updates the
recipe file to the new upstream version and attempts to rebase custom
patches contained by the recipe as needed.
Note
AUH uses much of devtool upgrade
behind the scenes making AUH somewhat
of a “wrapper” application for devtool upgrade
.
A typical scenario involves having used Git to clone an upstream
repository that you use during build operations. Because you have built the
recipe in the past, the layer is likely added to your
configuration already. If for some reason, the layer is not added, you
could add it easily using the
“bitbake-layers”
script. For example, suppose you use the nano.bb
recipe from the
meta-oe
layer in the meta-openembedded
repository. For this
example, assume that the layer has been cloned into following area:
/home/scottrif/meta-openembedded
The following command from your Build Directory adds the layer to
your build configuration (i.e. ${BUILDDIR}/conf/bblayers.conf
):
$ bitbake-layers add-layer /home/scottrif/meta-openembedded/meta-oe
NOTE: Starting bitbake server...
Parsing recipes: 100% |##########################################| Time: 0:00:55
Parsing of 1431 .bb files complete (0 cached, 1431 parsed). 2040 targets, 56 skipped, 0 masked, 0 errors.
Removing 12 recipes from the x86_64 sysroot: 100% |##############| Time: 0:00:00
Removing 1 recipes from the x86_64_i586 sysroot: 100% |##########| Time: 0:00:00
Removing 5 recipes from the i586 sysroot: 100% |#################| Time: 0:00:00
Removing 5 recipes from the qemux86 sysroot: 100% |##############| Time: 0:00:00
For this example, assume that the nano.bb
recipe that
is upstream has a 2.9.3 version number. However, the version in the
local repository is 2.7.4. The following command from your build
directory automatically upgrades the recipe for you:
$ devtool upgrade nano -V 2.9.3
NOTE: Starting bitbake server...
NOTE: Creating workspace layer in /home/scottrif/poky/build/workspace
Parsing recipes: 100% |##########################################| Time: 0:00:46
Parsing of 1431 .bb files complete (0 cached, 1431 parsed). 2040 targets, 56 skipped, 0 masked, 0 errors.
NOTE: Extracting current version source...
NOTE: Resolving any missing task queue dependencies
.
.
.
NOTE: Executing SetScene Tasks
NOTE: Executing RunQueue Tasks
NOTE: Tasks Summary: Attempted 74 tasks of which 72 didn't need to be rerun and all succeeded.
Adding changed files: 100% |#####################################| Time: 0:00:00
NOTE: Upgraded source extracted to /home/scottrif/poky/build/workspace/sources/nano
NOTE: New recipe is /home/scottrif/poky/build/workspace/recipes/nano/nano_2.9.3.bb
Note
Using the -V
option is not necessary. Omitting the version number causes
devtool upgrade
to upgrade the recipe to the most recent version.
Continuing with this example, you can use devtool build
to build the
newly upgraded recipe:
$ devtool build nano
NOTE: Starting bitbake server...
Loading cache: 100% |################################################################################################| Time: 0:00:01
Loaded 2040 entries from dependency cache.
Parsing recipes: 100% |##############################################################################################| Time: 0:00:00
Parsing of 1432 .bb files complete (1431 cached, 1 parsed). 2041 targets, 56 skipped, 0 masked, 0 errors.
NOTE: Resolving any missing task queue dependencies
.
.
.
NOTE: Executing SetScene Tasks
NOTE: Executing RunQueue Tasks
NOTE: nano: compiling from external source tree /home/scottrif/poky/build/workspace/sources/nano
NOTE: Tasks Summary: Attempted 520 tasks of which 304 didn't need to be rerun and all succeeded.
Within the devtool upgrade
workflow, you can
deploy and test your rebuilt software. For this example,
however, running devtool finish
cleans up the workspace once the
source in your workspace is clean. This usually means using Git to stage
and submit commits for the changes generated by the upgrade process.
Once the tree is clean, you can clean things up in this example with the
following command from the ${BUILDDIR}/workspace/sources/nano
directory:
$ devtool finish nano meta-oe
NOTE: Starting bitbake server...
Loading cache: 100% |################################################################################################| Time: 0:00:00
Loaded 2040 entries from dependency cache.
Parsing recipes: 100% |##############################################################################################| Time: 0:00:01
Parsing of 1432 .bb files complete (1431 cached, 1 parsed). 2041 targets, 56 skipped, 0 masked, 0 errors.
NOTE: Adding new patch 0001-nano.bb-Stuff-I-changed-when-upgrading-nano.bb.patch
NOTE: Updating recipe nano_2.9.3.bb
NOTE: Removing file /home/scottrif/meta-openembedded/meta-oe/recipes-support/nano/nano_2.7.4.bb
NOTE: Moving recipe file to /home/scottrif/meta-openembedded/meta-oe/recipes-support/nano
NOTE: Leaving source tree /home/scottrif/poky/build/workspace/sources/nano as-is; if you no longer need it then please delete it manually
Using the devtool finish
command cleans up the workspace and creates a patch
file based on your commits. The tool puts all patch files back into the
source directory in a sub-directory named nano
in this case.
7.3 Manually Upgrading a Recipe
If for some reason you choose not to upgrade recipes using Using the Auto Upgrade Helper (AUH) or by Using devtool upgrade, you can manually edit the recipe files to upgrade the versions.
Note
Manually updating multiple recipes scales poorly and involves many
steps. The recommendation to upgrade recipe versions is through AUH
or devtool upgrade
, both of which automate some steps and provide
guidance for others needed for the manual process.
To manually upgrade recipe versions, follow these general steps:
Change the Version: Rename the recipe such that the version (i.e. the PV part of the recipe name) changes appropriately. If the version is not part of the recipe name, change the value as it is set for PV within the recipe itself.
Update SRCREV if Needed: If the source code your recipe builds is fetched from Git or some other version control system, update SRCREV to point to the commit hash that matches the new version.
Build the Software: Try to build the recipe using BitBake. Typical build failures include the following:
License statements were updated for the new version. For this case, you need to review any changes to the license and update the values of LICENSE and LIC_FILES_CHKSUM as needed.
Note
License changes are often inconsequential. For example, the license text’s copyright year might have changed.
Custom patches carried by the older version of the recipe might fail to apply to the new version. For these cases, you need to review the failures. Patches might not be necessary for the new version of the software if the upgraded version has fixed those issues. If a patch is necessary and failing, you need to rebase it into the new version.
Optionally Attempt to Build for Several Architectures: Once you successfully build the new software for a given architecture, you could test the build for other architectures by changing the MACHINE variable and rebuilding the software. This optional step is especially important if the recipe is to be released publicly.
Check the Upstream Change Log or Release Notes: Checking both these reveals if there are new features that could break backwards-compatibility. If so, you need to take steps to mitigate or eliminate that situation.
Optionally Create a Bootable Image and Test: If you want, you can test the new software by booting it onto actual hardware.
Create a Commit with the Change in the Layer Repository: After all builds work and any testing is successful, you can create commits for any changes in the layer holding your upgraded recipe.