Migration notes for 3.4 (honister)
This section provides migration information for moving to the Yocto Project 3.4 Release (codename “honister”) from the prior release.
Override syntax changes
In this release, the :
character replaces the use of _
to
refer to an override, most commonly when making a conditional assignment
of a variable. This means that an entry like:
SRC_URI_qemux86 = "file://somefile"
now becomes:
SRC_URI:qemux86 = "file://somefile"
since qemux86
is an override. This applies to any use of override
syntax, so the following:
SRC_URI_append = " file://somefile"
SRC_URI_append_qemux86 = " file://somefile2"
SRC_URI_remove_qemux86-64 = "file://somefile3"
SRC_URI_prepend_qemuarm = "file://somefile4 "
FILES_${PN}-ptest = "${bindir}/xyz"
IMAGE_CMD_tar = "tar"
BASE_LIB_tune-cortexa76 = "lib"
SRCREV_pn-bash = "abc"
BB_TASK_NICE_LEVEL_task-testimage = '0'
would now become:
SRC_URI:append = " file://somefile"
SRC_URI:append:qemux86 = " file://somefile2"
SRC_URI:remove:qemux86-64 = "file://somefile3"
SRC_URI:prepend:qemuarm = "file://somefile4 "
FILES:${PN}-ptest = "${bindir}/xyz"
IMAGE_CMD:tar = "tar"
BASE_LIB:tune-cortexa76 = "lib"
SRCREV:pn-bash = "abc"
BB_TASK_NICE_LEVEL:task-testimage = '0'
This also applies to
variable queries to the datastore,
for example using getVar
and similar so d.getVar("RDEPENDS_${PN}")
becomes d.getVar("RDEPENDS:${PN}")
.
Whilst some of these are fairly obvious such as MACHINE and DISTRO
overrides, some are less obvious, for example the packaging variables such as
RDEPENDS, FILES and so on taking package names (e.g. ${PN}
,
${PN}-ptest
) as overrides. These overrides are not always in
OVERRIDES but applied conditionally in specific contexts
such as packaging. task-<taskname>
is another context specific override, the
context being specific tasks in that case. Tune overrides are another special
case where some code does use them as overrides but some does not. We plan to try
and make the tune code use overrides more consistently in the future.
There are some variables which do not use override syntax which include the
suffix to variables in layer.conf
files such as BBFILE_PATTERN,
SRCREV_xxx
where xxx
is a name from SRC_URI and
PREFERRED_VERSION_xxx
. In particular, layer.conf
suffixes
may be the same as a DISTRO override causing some confusion. We do
plan to try and improve consistency as these issues are identified.
To help with migration of layers, a script has been provided in OE-Core. Once configured with the overrides used by a layer, this can be run as:
<oe-core>/scripts/contrib/convert-overrides.py <layerdir>
Note
Please read the notes in the script as it isn’t entirely automatic and it isn’t
expected to handle every case. In particular, it needs to be told which overrides
the layer uses (usually machine and distro names/overrides) and the result should
be carefully checked since it can be a little enthusiastic and will convert
references to _append
, _remove
and _prepend
in function and variable
names.
For reference, this conversion is important as it allows BitBake to more reliably
determine what is an override and what is not, as underscores are also used in
variable names without intending to be overrides. This should allow us to proceed
with other syntax improvements and simplifications for usability. It also means
BitBake no longer has to guess and maintain large lookup lists just in case
e.g. functionname
in my_functionname
is an override, and thus should improve
efficiency.
New host dependencies
The lz4c
, pzstd
and zstd
commands are now required to be
installed on the build host to support LZ4 and Zstandard compression
functionality. These are typically provided by lz4
and zstd
packages in most Linux distributions. Alternatively they are available
as part of buildtools tarball if your distribution does not provide
them. For more information see
Required Packages for the Build Host.
Removed recipes
The following recipes have been removed in this release:
assimp
: problematic from a licensing perspective and no longer needed by anything elseclutter-1.0
: legacy component moved to meta-gnomeclutter-gst-3.0
: legacy component moved to meta-gnomeclutter-gtk-1.0
: legacy component moved to meta-gnomecogl-1.0
: legacy component moved to meta-gnomecore-image-clutter
: removed along with clutterlinux-yocto
: removed version 5.4 recipes (5.14 and 5.10 still provided)mklibs-native
: not actively tested and upstream mklibs still requires Python 2mx-1.0
: obsolete (last release 2012) and isn’t used by anything in any known layerpackagegroup-core-clutter
: removed along with clutter
Removed classes
clutter
: moved to meta-gnome along with clutter itselfimage-mklibs
: not actively tested and upstream mklibs still requires Python 2meta
: no longer useful. Recipes that need to skip installing packages should inherit nopackages instead.
Prelinking disabled by default
Recent tests have shown that prelinking works only when PIE is not enabled (see here and here), and as PIE is both a desirable security feature, and the only configuration provided and tested by the Yocto Project, there is simply no sense in continuing to enable prelink.
There’s also a concern that no one is maintaining the code, and there are open bugs (including this serious one). Given that prelink does intricate address arithmetic and rewriting of binaries the best option is to disable the feature. It is recommended that you consider disabling this feature in your own configuration if it is currently enabled.
Virtual runtime provides
Recipes shouldn’t use the virtual/
string in RPROVIDES and
RDEPENDS — it is confusing because virtual/
has no special
meaning in RPROVIDES and RDEPENDS (unlike in the
corresponding build-time PROVIDES and DEPENDS).
Tune files moved to architecture-specific directories
The tune files found in conf/machine/include
have now been moved
into their respective architecture name directories under that same
location; e.g. x86 tune files have moved into an x86
subdirectory,
MIPS tune files have moved into a mips
subdirectory, etc.
The ARM tunes have an extra level (armv8a
, armv8m
, etc.) and
some have been renamed to make them uniform with the rest of the tunes.
See this commit
for reference.
If you have any references to tune files (e.g. in custom machine configuration files) they will need to be updated.
Extensible SDK host extension
For a normal SDK, some layers append to TOOLCHAIN_HOST_TASK unconditionally which is fine, until the eSDK tries to override the variable to its own values. Instead of installing packages specified in this variable it uses native recipes instead — a very different approach. This has led to confusing errors when binaries are added to the SDK but not relocated.
To avoid these issues, a new TOOLCHAIN_HOST_TASK_ESDK variable has been created. If you wish to extend what is installed in the host portion of the eSDK then you will now need to set this variable.
Package/recipe splitting
perl-cross
has been split out from the mainperl
recipe to its ownperlcross
recipe for maintenance reasons. If you have bbappends for the perl recipe then these may need extending.The
wayland
recipe now packages its binaries in awayland-tools
package rather than putting them intowayland-dev
.Xwayland has been split out of the xserver-xorg tree and thus is now in its own
xwayland
recipe. If you need Xwayland in your image then you may now need to add it explicitly.The
rpm
package no longer hasrpm-build
in its RRECOMMENDS; if by chance you still need rpm package building functionality in your image and you have not already done so then you should addrpm-build
to your image explicitly.The Python
statistics
standard module is now packaged in its ownpython3-statistics
package instead ofpython3-misc
as previously.
Image / SDK generation changes
Recursive dependencies on the do_build task are now disabled when building SDKs. These are generally not needed; in the unlikely event that you do encounter problems then it will probably be as a result of missing explicit dependencies that need to be added.
Errors during “complementary” package installation (e.g. for
*-dbg
and*-dev
packages) during image construction are no longer ignored. Historically some of these packages had installation problems, that is no longer the case. In the unlikely event that you see errors as a result, you will need to fix the installation/packaging issues.When building an image, only packages that will be used in building the image (i.e. the first entry in PACKAGE_CLASSES) will be produced if multiple package types are enabled (which is not a typical configuration). If in your CI system you need to have the original behaviour, use
bitbake --runall build <target>
.The
-lic
package is no longer automatically added to RRECOMMENDS for every other package when LICENSE_CREATE_PACKAGE is set to “1”. If you wish all license packages to be installed corresponding to packages in your image, then you should instead add the newlic-pkgs
feature to IMAGE_FEATURES.
Miscellaneous
Certificates are now properly checked when BitBake fetches sources over HTTPS. If you receive errors as a result for your custom recipes, you will need to use a mirror or address the issue with the operators of the server in question.
avahi
has had its GTK+ support disabled by default. If you wish to re-enable it, setAVAHI_GTK = "gtk3"
in a bbappend for theavahi
recipe or in your custom distro configuration file.Setting the
BUILD_REPRODUCIBLE_BINARIES
variable to “0” no longer uses a strangely old fallback date of April 2011, it instead disables building reproducible binaries as you would logically expect.Setting noexec/nostamp/fakeroot varflags to any value besides “1” will now trigger a warning. These should be either set to “1” to enable, or not set at all to disable.
The previously deprecated
COMPRESS_CMD
andCVE_CHECK_CVE_WHITELIST
variables have been removed. Use CONVERSION_CMD andCVE_CHECK_WHITELIST
(replaced by CVE_CHECK_IGNORE in version 3.5) respectively instead.The obsolete
oe_machinstall
function previously provided in the utils class has been removed. For machine-specific installation it is recommended that you use the built-in override support in the fetcher or overrides in general instead.The
-P
(--clear-password
) option can no longer be used withuseradd
andusermod
entries in EXTRA_USERS_PARAMS. It was being implemented using a custom patch to theshadow
recipe which clashed with a-P
option that was added upstream inshadow
version 4.9, and in any case is fundamentally insecure. Hardcoded passwords are still supported but they need to be hashed, see examples in EXTRA_USERS_PARAMS.