7 Library Functions

This chapter lists common library functions available under the lib/ directory in BitBake.

These functions can be used in recipes or configuration files with inline-Python or Python functions.

7.1 Logging utilities

Different logging utilities can be used from Python code in recipes or configuration files.

The strings passed below can be formatted with str.format(), for example:

bb.warn("Houston, we have a %s", "bit of a problem")

Formatted string can also be used directly:

bb.error("%s, we have a %s" % ("Houston", "big problem"))

Python f-strings may also be used:

h = "Houston"
bb.fatal(f"{h}, we have a critical problem")

bb.debug(lvl, *args)

Prints a debug message.

Arguments:

lvl: debug level. Higher value increases the debug level (determined by bitbake -D).
args: one or more strings to print.

bb.error(*args, **kwargs)

Prints an error message.

Arguments:

args: one or more strings to print.

bb.erroronce(*args)

Prints an error message like bb.error(), but only prints the message once.

Arguments:

args: one or more strings to print.

bb.fatal(*args, **kwargs)

Prints an error message and stops the BitBake execution.

Arguments:

args: one or more strings to print.

bb.note(*args)

Prints a message at “note” level.

Arguments:

args: one or more strings to print.

bb.plain(*args)

Prints a message at “plain” level (higher level than a bb.note()).

Arguments:

args: one or more strings to print.

bb.verbnote(*args)

A higher priority note which will show on the console but isn’t a warning.

Use in contexts when something is happening the user should be aware of but they probably did something to make it happen.

Arguments:

args: one or more strings to print.

bb.warn(*args)

Prints a warning message.

Arguments:

args: one or more strings to print.

bb.warnonce(*args)

Prints a warning message like bb.warn(), but only prints the message once.

Arguments:

args: one or more strings to print.

7.2 `bb.utils`

BitBake Utility Functions

bb.utils.approved_variables(): Determine and return the list of variables which are approved to remain in the environment.

bb.utils.break_hardlinks(src, sstat=None)

Ensures src is the only hardlink to this file. Other hardlinks, if any, are not affected (other than in their st_nlink value, of course).

Arguments:

src: source file path.
sstat: os.stat_result to use when checking if the file is a link.

Returns True on success and False on failure.

bb.utils.build_environment(d)

Build an environment from all exported variables.

Arguments:

d: the data store.

No return value.

bb.utils.check_system_locale()

Make sure the required system locale are available and configured.

No return value.

bb.utils.clean_environment()

Clean up any spurious environment variables. This will remove any variables the user hasn’t chosen to preserve.

No return value.

bb.utils.contains(variable, checkvalues, truevalue, falsevalue, d)

Check if a variable contains all the values specified.

Arguments:

variable: the variable name. This will be fetched and expanded (using d.getVar(variable)) and then split into a set().
checkvalues: if this is a string it is split on whitespace into a set(), otherwise coerced directly into a set().
truevalue: the value to return if checkvalues is a subset of variable.
falsevalue: the value to return if variable is empty or if checkvalues is not a subset of variable.
d: the data store.

Returns True if the variable contains the values specified, False otherwise.

bb.utils.contains_any(variable, checkvalues, truevalue, falsevalue, d)

Check if a variable contains any values specified.

Arguments:

variable: the variable name. This will be fetched and expanded (using d.getVar(variable)) and then split into a set().
checkvalues: if this is a string it is split on whitespace into a set(), otherwise coerced directly into a set().
truevalue: the value to return if checkvalues is a subset of variable.
falsevalue: the value to return if variable is empty or if checkvalues is not a subset of variable.
d: the data store.

Returns True if the variable contains any of the values specified, False otherwise.

bb.utils.copyfile(src, dest, newmtime=None, sstat=None)

Copies a file from src to dest, preserving all permissions and attributes; mtime will be preserved even when moving across filesystems.

Arguments:

src: Source file.
dest: Destination file.
newmtime: new mtime to be passed as float seconds since the epoch.
sstat: os.stat_result to use for the destination file.

Returns an os.stat_result of the destination file if the source file is a symbolic link or the sstat argument represents a symbolic link - in which case the destination file will also be created as a symbolic link.

Otherwise, returns newmtime on success and False on failure.

bb.utils.disable_network(uid=None, gid=None)

Disable networking in the current process if the kernel supports it, else just return after logging to debug. To do this we need to create a new user namespace, then map back to the original uid/gid.

Arguments:

uid: original user id.
gid: original user group id.

No return value.

bb.utils.edit_bblayers_conf(bblayers_conf, add, remove, edit_cb=None)

Edit bblayers.conf, adding and/or removing layers.

Arguments:

bblayers_conf: path to bblayers.conf file to edit
add: layer path (or list of layer paths) to add; None or empty list to add nothing
remove: layer path (or list of layer paths) to remove; None or empty list to remove nothing
edit_cb: optional callback function that will be called after processing adds/removes once per existing entry.

Returns a tuple:

notadded: list of layers specified to be added but weren’t (because they were already in the list)
notremoved: list of layers that were specified to be removed but weren’t (because they weren’t in the list)

bb.utils.edit_metadata(meta_lines, variables, varfunc, match_overrides=False)

Edit lines from a recipe or config file and modify one or more specified variable values set in the file using a specified callback function. Lines are expected to have trailing newlines.

Arguments:

meta_lines: lines from the file; can be a list or an iterable (e.g. file pointer)
variables: a list of variable names to look for. Functions may also be specified, but must be specified with () at the end of the name. Note that the function doesn’t have any intrinsic understanding of :append, :prepend, :remove, or overrides, so these are considered as part of the name. These values go into a regular expression, so regular expression syntax is allowed.
varfunc: callback function called for every variable matching one of the entries in the variables parameter.

The function should take four arguments:
- varname: name of variable matched
- origvalue: current value in file
- op: the operator (e.g. +=)
- newlines: list of lines up to this point. You can use this to prepend lines before this variable setting if you wish.
And should return a four-element tuple:
- newvalue: new value to substitute in, or None to drop the variable setting entirely. (If the removal results in two consecutive blank lines, one of the blank lines will also be dropped).
- newop: the operator to use - if you specify None here, the original operation will be used.
- indent: number of spaces to indent multi-line entries, or -1 to indent up to the level of the assignment and opening quote, or a string to use as the indent.
- minbreak: True to allow the first element of a multi-line value to continue on the same line as the assignment, False to indent before the first element.
To clarify, if you wish not to change the value, then you would return like this:
```
return origvalue, None, 0, True
```
match_overrides: True to match items with _overrides on the end, False otherwise

Returns a tuple:

updated: True if changes were made, False otherwise.
newlines: Lines after processing.

bb.utils.edit_metadata_file(meta_file, variables, varfunc)

Edit a recipe or configuration file and modify one or more specified variable values set in the file using a specified callback function. The file is only written to if the value(s) actually change. This is basically the file version of bb.utils.edit_metadata(), see that function’s description for parameter/usage information.

Returns True if the file was written to, False otherwise.

bb.utils.empty_environment()

Remove all variables from the environment.

No return value.

bb.utils.environment(**envvars)

Context manager to selectively update the environment with the specified mapping.

No return value.

bb.utils.exec_flat_python_func(func, *args, **kwargs)

Execute a flat python function (defined with def funcname(args): ...)

Returns the return value of the function.

bb.utils.explode_dep_versions(s)

Take an RDEPENDS style string of format:

DEPEND1 (optional version) DEPEND2 (optional version) ...

Skips null values and items appeared in dependency string multiple times.

Arguments:

s: input RDEPENDS style string

Returns a dictionary of dependencies and versions.

bb.utils.explode_dep_versions2(s, *, sort=True)

Takes an RDEPENDS style string of format:

DEPEND1 (optional version) DEPEND2 (optional version) ...

Arguments:

s: input RDEPENDS style string
*: Unused.
sort: whether to sort the output or not.

Returns a dictionary of dependencies and versions.

bb.utils.explode_deps(s)

Takes an RDEPENDS style string of format:

DEPEND1 (optional version) DEPEND2 (optional version) ...

Arguments:

s: input RDEPENDS style string

Returns a list of dependencies.

Version information is ignored.

bb.utils.fileslocked(files, *args, **kwargs)

Context manager for locking and unlocking file locks. Uses bb.utils.lockfile() and bb.utils.unlockfile() to lock and unlock files.

No return value.

bb.utils.filter(variable, checkvalues, d)

Return all words in the variable that are present in the checkvalues.

Arguments:

variable: the variable name. This will be fetched and expanded (using d.getVar(variable)) and then split into a set().
checkvalues: if this is a string it is split on whitespace into a set(), otherwise coerced directly into a set().
d: the data store.

Returns a list of string.

bb.utils.filter_environment(good_vars)

Create a pristine environment for bitbake. This will remove variables that are not known and may influence the build in a negative way.

Arguments:

good_vars: list of variable to exclude from the filtering.

No return value.

bb.utils.get_file_layer(filename, d, collection_res={})

Determine the collection (or layer name, as defined by a layer’s layer.conf file) containing the specified file.

Arguments:

filename: the filename to look for.
d: the data store.
collection_res: dictionary with the layer names as keys and file patterns to match as defined with the BBFILE_COLLECTIONS and BBFILE_PATTERN variables respectively. The return value of bb.utils.get_collection_res() is the default if this variable is not specified.

Returns the layer name containing the file. If multiple layers contain the file, the last matching layer name from collection_res is returned.

bb.utils.get_referenced_vars(start_expr, d)

Get the names of the variables referenced in a given expression.

Arguments:

start_expr: the expression where to look for variables references.

For example:
${VAR_A} string ${VAR_B}
Or:
${@d.getVar('VAR')}
If a variables makes references to other variables, the latter are also returned recursively.
d: the data store.

Returns the names of vars referenced in start_expr (recursively), in quasi-BFS order (variables within the same level are ordered arbitrarily).

bb.utils.goh1_file(filename)

Returns the hexadecimal string representation of the Go mod h1 checksum of the filename. The Go mod h1 checksum uses the Go dirhash package. The package defines hashes over directory trees and is used by go mod for mod files and zip archives.

Arguments:

filename: path to the file.

bb.utils.is_local_uid(uid='')

Check whether uid is a local one or not. Can’t use pwd module since it gets all UIDs, not local ones only.

Arguments:

uid: user id. If not specified the user id is determined from os.getuid().

Returns True is the user id is local, False otherwise.

bb.utils.is_semver(version)

Arguments:

version: the version string.

Returns True if the version string follow semantic versioning, False otherwise.

See https://semver.org/spec/v2.0.0.html.

bb.utils.join_deps(deps, commasep=True)

Take a result from bb.utils.explode_dep_versions() and generate a dependency string.

Arguments:

deps: dictionary of dependencies and versions.
commasep: makes the return value separated by commas if True, separated by spaces otherwise.

Returns a comma-separated (space-separated if comma-sep is False) string of dependencies and versions.

bb.utils.lockfile(name, shared=False, retry=True, block=False)

Use the specified file (with filename name) as a lock file, return when the lock has been acquired. Returns a variable to pass to unlockfile().

Arguments:

shared: sets the lock as a shared lock instead of an exclusive lock.
retry: True to re-try locking if it fails, False otherwise.
block: True to block until the lock succeeds, False otherwise.

The retry and block parameters are kind of equivalent unless you consider the possibility of sending a signal to the process to break out - at which point you want block=True rather than retry=True.

Returns the locked file descriptor in case of success, None otherwise.

bb.utils.md5_file(filename)

Arguments:

filename: path to the input file.

Returns the hexadecimal string representation of the MD5 checksum of filename.

bb.utils.mkdirhier(directory)

Create a directory like ‘mkdir -p’, but does not complain if directory already exists like os.makedirs().

Arguments:

directory: path to the directory.

No return value.

bb.utils.mkstemp(suffix=None, prefix=None, dir=None, text=False)

Generates a unique temporary file, independent of time.

mkstemp() in glibc (at least) generates unique file names based on the current system time. When combined with highly parallel builds, and operating over NFS (e.g. shared sstate/downloads) this can result in conflicts and race conditions.

This function adds additional entropy to the file name so that a collision is independent of time and thus extremely unlikely.

Arguments:

suffix: filename suffix.
prefix: filename prefix.
dir: directory where the file will be created.
text: if True, the file is opened in text mode.

Returns a tuple containing:

the file descriptor for the created file
the name of the file.

bb.utils.movefile(src, dest, newmtime=None, sstat=None)

Moves a file from src to dest, preserving all permissions and attributes; mtime will be preserved even when moving across filesystems. Returns True on success and False on failure. Move is atomic.

Arguments:

src – Source file.
dest – Destination file.
newmtime – new mtime to be passed as float seconds since the epoch.
sstat – os.stat_result to use for the destination file.

Returns an os.stat_result of the destination file if the source file is a symbolic link or the sstat argument represents a symbolic link - in which case the destination file will also be created as a symbolic link.

Otherwise, returns newmtime on success and False on failure.

bb.utils.path_is_descendant(descendant, ancestor)

Returns True if the path descendant is a descendant of ancestor (including being equivalent to ancestor itself). Otherwise returns False.

Correctly accounts for symlinks, bind mounts, etc. by using os.path.samestat() to compare paths.

May raise any exception that os.stat() raises.

Arguments:

descendant: path to check for being an ancestor.
ancestor: path to the ancestor descendant will be checked against.

bb.utils.preserved_envvars(): Returns the list of variables which are taken from the environment and placed in the metadata.

bb.utils.preserved_envvars_exported(): Returns the list of variables which are taken from the environment and placed in and exported from the metadata.

bb.utils.profile_function(profile, function, output_fn, process=True)

Common function to profile a code block and optionally process the output using or processing function.

Arguments:

profile: a boolean saying whether to enable profiling or not
function: the function call to profile/run
outputfn: where to write the profiling data
process: whether to process the profiling data and write a report

Returns the wrapped function return value

bb.utils.prune_suffix(var, suffixes, d)

Check if var ends with any of the suffixes listed in suffixes and remove it if found.

Arguments:

var: string to check for suffixes.
suffixes: list of strings representing suffixes to check for.

Returns the string var without the suffix.

bb.utils.prunedir(topdir, ionice=False)

Delete everything reachable from the directory named in topdir.

Arguments:

topdir: directory path.
ionice: prepends ionice -c 3 to the rm command. See man ionice.

No return value.

bb.utils.remove(path, recurse=False, ionice=False)

Equivalent to rm -f or rm -rf.

Arguments:

path: path to file/directory to remove.
recurse: deletes recursively if True.
ionice: prepends ionice -c 3 to the rm command. See man ionice.

No return value.

bb.utils.sha1_file(filename)

Returns the hexadecimal representation of the SHA1 checksum of the filename

Arguments:

filename: path to the file.

bb.utils.sha256_file(filename)

Returns the hexadecimal representation of the 256-bit SHA checksum of filename.

Arguments:

filename: path to the file.

bb.utils.sha384_file(filename)

Returns the hexadecimal representation of the SHA384 checksum of the filename

Arguments:

filename: path to the file.

bb.utils.sha512_file(filename)

Returns the hexadecimal representation of the SHA512 checksum of the filename

Arguments:

filename: path to the file.

bb.utils.signal_on_parent_exit(signame)

Trigger signame to be sent when the parent process dies.

Arguments:

signame: name of the signal. See man signal.

No return value.

bb.utils.split_version(s)

Split a version string into its constituent parts (PE, PV, PR).

Arguments:

s: version string. The format of the input string should be:
```
${PE}:${PV}-${PR}
```

Returns a tuple (pe, pv, pr).

bb.utils.to_boolean(string, default=None)

Check input string and return boolean value True/False/None depending upon the checks.

Arguments:

string: input string.
default: default return value if the input string is None, 0, False or an empty string.

Returns True if the string is one of “y”, “yes”, “1”, “true”, False if the string is one of “n”, “no”, “0”, or “false”. Return default if the input string is None, 0, False or an empty string.

bb.utils.to_filemode(input)

Take a bitbake variable contents defining a file mode and return the proper python representation of the number

Arguments:

input: a string or number to convert, e.g. a bitbake variable string, assumed to be an octal representation

Returns the python file mode as a number

bb.utils.umask(new_mask)

Context manager to set the umask to a specific mask, and restore it afterwards.

No return value.

bb.utils.unlockfile(lf)

Unlock a file locked using bb.utils.lockfile().

Arguments:

lf: the locked file descriptor.

No return value.

bb.utils.vercmp_string(a, b)

Split version strings using bb.utils.split_version() and compare them with bb.utils.vercmp().

Arguments:

a: left version string operand.
b: right version string operand.

Returns what bb.utils.vercmp() returns.

bb.utils.vercmp_string_op(a, b, op)

Takes the return value bb.utils.vercmp() and returns the operation defined by op between the return value and 0.

Arguments:

a: left version string operand.
b: right version string operand.
op: operator string. Can be one of =, ==, <=, >=, >, >>, <, << or !=.

bb.utils.which(path, item, direction=0, history=False, executable=False)

Locate item in the list of paths path (colon separated string like $PATH).

Arguments:

path: list of colon-separated paths.
item: string to search for.
direction: if non-zero then the list is reversed.
history: if True then the list of candidates also returned as result,history where history is the list of previous path checked.
executable: if True then the candidate defined by path has to be an executable file, otherwise if False the candidate simply has to exist.

Returns the item if found in the list of path, otherwise an empty string. If history is True, return the list of previous path checked in a tuple with the found (or not found) item as (item, history).

7 Library Functions

7.1 Logging utilities

7.2 bb.utils

7.2 `bb.utils`