[meta-xilinx] [PATCH] bootbin: fix race condition in create_bif

Wed May 16 09:36:06 PDT 2018

Hi Jean-Francois,

> -----Original Message-----
> From: Jean-Francois Dagenais [mailto:jeff.dagenais at gmail.com]
> Sent: Wednesday, May 16, 2018 8:57 AM
> To: meta-xilinx at yoctoproject.org; Manjukumar Harthikote Matha
> <MANJUKUM at xilinx.com>
> Cc: nathan at nathanrossi.com; Jean-Francois Dagenais <jeff.dagenais at gmail.com>
> Subject: [PATCH] bootbin: fix race condition in create_bif
> 
> This problem was introduced by a622db96dd962c9a5ceb999ccd99da7ad7a40faf
> 
> From documentation of DEPENDS variable:
> [...]This mechanism is implemented by having do_configure depend on the
> do_populate_sysroot task of each recipe listed in DEPENDS[...]
> 
> This is why the xilinx-bootbin:do_configure task can start as soon as
> (for example) bitstream-extraction:do_populate_sysroot succeeds.
> Consequently, bitstream-extraction:do_deploy may not have run yet when
> xilinx-bootbin:do_configure (and create_bif) runs. The result is a
> python stack trace pointing at line 'if os.stat(imagestr).st_size == 0:'
> with an error which looks like this:
> 
> Exception: FileNotFoundError: [Errno 2] No such file or directory:
> '/builds/yocto/mymachine/tmp/deploy/images/mymachine/download-
> mymachine.bit'
> 
> A re-attempt of the build will ultimately succeed as the do_deploy of
> each dependency is spawned in parallel and get to do their job.
> 
> Targeting the "do_deploy" steps specifically fixes the problem
> completely.
> 
> The ':do_deploy' is added to the BIF_PARTITION_DEPENDS, otherwise, it
> would force the files assigned to BIF_PARTITION_IMAGE to be in the
> DEPLOY_DIR.
> 
> Signed-off-by: Jean-Francois Dagenais <jeff.dagenais at gmail.com>
> ---
>  README.md                                     |  6 +++---
>  recipes-bsp/bootbin/machine-xilinx-zynq.inc   |  6 +++---
>  recipes-bsp/bootbin/machine-xilinx-zynqmp.inc | 10 +++++-----
>  recipes-bsp/bootbin/xilinx-bootbin_1.0.bb     |  2 +-
>  4 files changed, 12 insertions(+), 12 deletions(-)
> 
> diff --git a/README.md b/README.md
> index bd78938..f9f30c3 100644
> --- a/README.md
> +++ b/README.md
> @@ -79,7 +79,7 @@ IMAGE_CLASSES += " xilinx-bootbin"
>  BIF_PARTITION_ATTR= "fsbl u-boot"
> 
>  BIF_PARTITION_IMAGE[fsbl]="${DEPLOY_DIR_IMAGE}/fsbl-${MACHINE}.elf"
> -BIF_PARTITION_DEPENDS[fsbl]="virtual/fsbl"
> +BIF_PARTITION_DEPENDS[fsbl]="virtual/fsbl:do_deploy"
> 
>  BIF_PARTITION_IMAGE[u-boot]="${DEPLOY_DIR_IMAGE}/u-boot-${MACHINE}.elf"
> 
> @@ -95,11 +95,11 @@ BIF_PARTITION_ATTR= "fsbl pmu atf u-boot"
> 
>  BIF_PARTITION_ATTR[fsbl]="bootloader"
>  BIF_PARTITION_IMAGE[fsbl]="${DEPLOY_DIR_IMAGE}/fsbl-${MACHINE}.elf"
> -BIF_PARTITION_DEPENDS[fsbl]="virtual/fsbl"
> +BIF_PARTITION_DEPENDS[fsbl]="virtual/fsbl:do_deploy"
> 
>  BIF_PARTITION_ATTR[pmu]="destination_cpu=pmu"
>  BIF_PARTITION_IMAGE[pmu]="${DEPLOY_DIR_IMAGE}/pmu-${MACHINE}.elf"
> -BIF_PARTITION_DEPENDS[pmu]="virtual/pmufw"
> +BIF_PARTITION_DEPENDS[pmu]="virtual/pmufw:do_deploy"
> 
>  BIF_PARTITION_ATTR[atf]="destination_cpu=a53-0,exception_level=el-
> 3,trustzone"
>  BIF_PARTITION_IMAGE[atf]="${DEPLOY_DIR_IMAGE}/arm-trusted-firmware-
> ${TUNE_PKGARCH}.elf"
> diff --git a/recipes-bsp/bootbin/machine-xilinx-zynq.inc b/recipes-
> bsp/bootbin/machine-xilinx-zynq.inc
> index aa537d6..b8d75c4 100644
> --- a/recipes-bsp/bootbin/machine-xilinx-zynq.inc
> +++ b/recipes-bsp/bootbin/machine-xilinx-zynq.inc
> @@ -5,13 +5,13 @@ BIF_PARTITION_ATTR ?= "fsbl bitstream u-boot"
>  #bootloader is FSBL. Location where FSBL binary is present and dependency to
> build FSBL
>  BIF_PARTITION_ATTR[fsbl] ?= "bootloader"
>  BIF_PARTITION_IMAGE[fsbl] ?= "${DEPLOY_DIR_IMAGE}/fsbl-${MACHINE}.elf"
> -BIF_PARTITION_DEPENDS[fsbl] ?= "virtual/fsbl"
> +BIF_PARTITION_DEPENDS[fsbl] ?= "virtual/fsbl:do_deploy"
> 
>  #specify BIF partition attributes for u-boot
>  #Location where u-boot binary is present
>  BIF_PARTITION_IMAGE[u-boot] ?= "${DEPLOY_DIR_IMAGE}/u-boot-
> ${MACHINE}.elf"
> -BIF_PARTITION_DEPENDS[u-boot] ?= "virtual/bootloader"
> +BIF_PARTITION_DEPENDS[u-boot] ?= "virtual/bootloader:do_deploy"
> 
>  # enable bitstream-Note this is not enabled by default (missing in
> BIF_PARTITION_ATTR)
>  BIF_PARTITION_IMAGE[bitstream] ?= "${DEPLOY_DIR_IMAGE}/download-
> ${MACHINE}.bit"
> -BIF_PARTITION_DEPENDS[bitstream] ?= "virtual/bitstream"
> +BIF_PARTITION_DEPENDS[bitstream] ?= "virtual/bitstream:do_deploy"
> diff --git a/recipes-bsp/bootbin/machine-xilinx-zynqmp.inc b/recipes-
> bsp/bootbin/machine-xilinx-zynqmp.inc
> index fe1b36c..3734880 100644
> --- a/recipes-bsp/bootbin/machine-xilinx-zynqmp.inc
> +++ b/recipes-bsp/bootbin/machine-xilinx-zynqmp.inc
> @@ -8,27 +8,27 @@ BIF_PARTITION_ATTR ?= "fsbl bitstream pmu atf u-boot"
>  # bootloader is FSBL. Location where FSBL binary is present and dependency to
> build FSBL
>  BIF_PARTITION_ATTR[fsbl] ?= "bootloader, destination_cpu=a53-0"
>  BIF_PARTITION_IMAGE[fsbl] ?= "${DEPLOY_DIR_IMAGE}/fsbl-${MACHINE}.elf"
> -BIF_PARTITION_DEPENDS[fsbl] ?= "virtual/fsbl"
> +BIF_PARTITION_DEPENDS[fsbl] ?= "virtual/fsbl:do_deploy"
> 
>  # specify BIF partition attributes for PMU Firmware
>  # destination cpu for PMU. Location where PMU binary is present and
> dependency to build PMU Firmware
>  BIF_PARTITION_ATTR[pmu] ?= "destination_cpu=pmu"
>  BIF_PARTITION_IMAGE[pmu] ?= "${DEPLOY_DIR_IMAGE}/pmu-firmware-
> ${MACHINE}.elf"
> -BIF_PARTITION_DEPENDS[pmu] ?= "virtual/pmu-firmware"
> +BIF_PARTITION_DEPENDS[pmu] ?= "virtual/pmu-firmware:do_deploy"
> 
>  # specify BIF partition attributes for ATF
>  # destination cpu for ATF, security levels. Location where ATF binary is present
> (dependency is not required as ATF is always built for ZU+, see zcu102-
> zynqmp.conf)
>  BIF_PARTITION_ATTR[atf] ?= "destination_cpu=a53-0,exception_level=el-
> 3,trustzone"
>  BIF_PARTITION_IMAGE[atf] ?= "${DEPLOY_DIR_IMAGE}/arm-trusted-firmware.elf"
> -BIF_PARTITION_DEPENDS[atf] ?= "arm-trusted-firmware"
> +BIF_PARTITION_DEPENDS[atf] ?= "arm-trusted-firmware:do_deploy"
> 
>  # specify BIF partition attributes for u-boot
>  # destination cpu for u-boot, security levels. Location where u-boot binary is
> present (dependency is not required as u-boot is always built for ZU+, see zcu102-
> zynqmp.conf)
>  BIF_PARTITION_ATTR[u-boot] ?= "destination_cpu=a53-0,exception_level=el-2"
>  BIF_PARTITION_IMAGE[u-boot] ?= "${DEPLOY_DIR_IMAGE}/u-boot-
> ${MACHINE}.elf"
> -BIF_PARTITION_DEPENDS[u-boot] ?= "virtual/bootloader"
> +BIF_PARTITION_DEPENDS[u-boot] ?= "virtual/bootloader:do_deploy"
> 
>  # enable bitstream-Note this is not enabled by default (missing in
> BIF_PARTITION_ATTR)
>  BIF_PARTITION_ATTR[bitstream] ?= "destination_device=pl"
>  BIF_PARTITION_IMAGE[bitstream] ?= "${DEPLOY_DIR_IMAGE}/download-
> ${MACHINE}.bit"
> -BIF_PARTITION_DEPENDS[bitstream] ?= "virtual/bitstream"
> +BIF_PARTITION_DEPENDS[bitstream] ?= "virtual/bitstream:do_deploy"
> diff --git a/recipes-bsp/bootbin/xilinx-bootbin_1.0.bb b/recipes-
> bsp/bootbin/xilinx-bootbin_1.0.bb
> index ecaee00..a01aa7a 100644
> --- a/recipes-bsp/bootbin/xilinx-bootbin_1.0.bb
> +++ b/recipes-bsp/bootbin/xilinx-bootbin_1.0.bb
> @@ -11,7 +11,7 @@ inherit xsct-tc deploy
> 
>  PROVIDES = "virtual/boot-bin"
> 
> -DEPENDS += "${@get_bootbin_depends(d)}"
> +do_configure[depends] += "${@get_bootbin_depends(d)}"
> 
>  PACKAGE_ARCH = "${MACHINE_ARCH}"

ACK,

Will change python code to add  
bootbindeps = bootbindeps + " " + attrdepends[cfg] + ":do_deploy" 
instead of do_deploy in BIF_PARTITION_DEPENDS and apply it.

This will be applied to master and rel-v2018.2 branch.

Thanks,
Manju