[meta-xilinx] Xilinx, why are you breaking the meta-xilinx-bsp layer?

[Martin Lund]

Hi Manju,

Thank you for your answers - please see my responses below.

>> We are trying to put together a system built on top of the meta-xilinx-bsp layer
>> and we have done so successfully up until the v2017.4 release with a few helpful
>> patches from this excellent community. That is, we have been able to create our
>> system without using the meta-xilinx-tools and meta-petalinux layers. Meaning
>> meta-xilinx-bsp has offered a fairly independent and functional BSP layer, in our
>> case, for the zcu102 dev board.
>
>I am not sure how you achieved to build pmu firmware in OSS/OSL flow, using a rel-v2017.4 branch.
>https://github.com/Xilinx/meta-xilinx/tree/rel-v2017.4/recipes-bsp<https://github.com/Xilinx/meta-xilinx/tree/rel-v2017.4/recipes-bsp>
>There was no recipe available to build one nor support to build SPL.

I stand corrected. We actually did use the rocko branch of meta-xilinx. I meant to write v2017.3 because we see a lot of v2017.3 footprints in the rocko branch.


>> However, with the introduction of Xilinx release v2018.1 it seems some basic BSP
>> things are broken or split out and moved to the meta-xilinx-tools or meta-
>> petalinux for no apparent good reason - basically breaking the independence of
>> the meta-xilinx-bsp layer. Up until this point we are still struggling to get meta-
>> xilinx-bsp booting on our zcu102 board.
>>
>
>Release branches in github is very specific for Xilinx tool release. Meaning all the rel-v2018.x are supposed to work with the layer stack (meta-petalinux, meta-xilinx-tools, meta-xilinx-bsp and meta-xilinx-contrib). This is the same practice which has >been done since 2017. If you see older releases like rel-v2017.1, SPL support, certain board support code is not present in release branches.
>
>Please see
>https://lists.yoctoproject.org/pipermail/meta-xilinx/2016-October/002174.html

Thank you for clarifying - we didn't realize the substantial difference between Xilinx release branches and Yocto release branches in meta-xilinx.

It explains why we are running into trouble trying to use rel-v2018.1 with YP rocko.

That being said, I'm not sure I understand why Xilinx insists on maintaining it's own branch for what seems an effort to enforce the Xilinx toolchain. I mean, it seems clear to me that not many developers appreciate the way the Xilinx toolchain is put together. In fact most professionals I talk to seem to prefer the open source standard tools offered by e.g. the YP toolchain. I understand there is a lot history to Xilinx using its own tools and maybe a hint of NIH but perhaps it would be easier for everyone if Xilinx adapted the OSS tools when operating in the domain of modern Linux BSP solutions. Also, wouldn't it be in the interest of Xilinx to have one less branch to maintain?

I realize I might be stepping on some toes stating the above but I'm just relaying what many developers here are thinking.


>Saying that, 2018.2 release branch will be fixed to include u-boot-mkimage, and fw-utils.
>This will be pushed mid-june.

Ok.


>One more thing to add, Xilinx release branches works on top of a previous version of the available Yocto release.
>Meaning rel-v2018.1 (and rest of rel-v2018.x ) is based on rocko branch of meta-xilinx-bsp.
>rel-v2019.1 will be based on thud (v2.6) branch of meta-xilinx-bsp etc
>
>We don't change any of the OSS/OSL flow related recipes in the Xilinx tool related releases.

Hopefully we don't have to wait for 2019 for the next Xilinx YP branch update of meta-xilinx-bsp.


>> Another more important example is the omission of an updated pmu-firmware in
>> meta-xilinx-bsp for the v2018.1 release. A simple and straightforward way of
>> building the pmu-firmware used to reside in meta-xilinx-bsp (see
>> https://github.com/Xilinx/meta-xilinx/tree/rel-v2018.1/meta-xilinx-bsp/recipes-
>> bsp/pmu-firmware <https://github.com/Xilinx/meta-xilinx/tree/rel-v2018.1/meta-
>> xilinx-bsp/recipes-bsp/pmu-firmware> ) but now a new way of building the pmu-
>> firmware has been introduced/forced in meta-xilinx-tools (see
>> https://github.com/Xilinx/meta-xilinx-tools/tree/rel-v2018.1/recipes-bsp/pmu-
>> firmware).
>
>Using xsct to build pmu-firmware is not new, this layer and recipes has been introduced probably in 2016 timeframe.
>

Ok - is there a particular good reason for using xsct to build the PMU firmware?

I mean, isn't the microblaze gcc/newlib toolchain good enough for Xilinx?


>> As mentioned in https://lists.yoctoproject.org/pipermail/meta-
>> xilinx/2018-May/003809.html, this new approach uses all the spokes and wheels
>> of the proprietary xilinx toolchain involving all sorts of cumbersome tools and
>> components (xsdk, X/gtk3, xsct, tcl, yaml, etc.) which makes it quite troublesome
>> to deploy on common build servers. I can't understand why Xilinx thinks it is a
>> good idea to go down this route. They take something simple and make it
>> complicated, much slower, and storage demanding. Additionally, what makes
>> matters even worse is that the build definition for is pmu-firmware is actually
>> placed in a bbclass (see https://github.com/Xilinx/meta-xilinx-tools/blob/rel-
>> v2018.1/classes/xsctapp.bbclass <https://github.com/Xilinx/meta-xilinx-
>> tools/blob/rel-v2018.1/classes/xsctapp.bbclass> ) but it really belongs in the pmu-
>> firmware_*.bb file. Either way, the end result is that if one is using only the
>> v2018.1 meta-xilinx-bsp layer then the old v2017.3 pmu-firmware is built which
>> does not seem to be compatible with v2018.1 atf, u-boot, kernel etc.
>
>
>Same as the above, recipes are tied together with the above mentioned layers for release branches.
>

Understood. The Xilinx release branches ties the four layers together and the effect is various inter-dependencies between the layers - in other words, the are tightly coupled - use em all or nothing.


>>
>> Seen from the outside, it all seems like a bit of a mess.
>
>Xilinx tools release cannot be independent, the workflow is different.
>If you continue to use rel-v2018 (release branches), then the recommendation is to use all layers.

Got it.

Though, seen from the outside it is not very clear what the responsibility / feature split is between the xilinx layers but I guess that does not matter as long as Xilinx can keep track.
Either way, my main concern is the meta-xilinx-bsp layer and how to use it the OSS YP way independently from the other xilinx layers.


>If you use yocto based release branches (rocko, pyro, etc), meta-xilinx-bsp should fairly work.
>
>Look out for patches that are being posted for sumo branch.

Ok. We see a handful of commits on the meta-xilinx master since the rocko release. I assume master will eventually be branched out into a sumo branch?
Can we expect the current Xilinx rocko branch in meta-xilinx to be updated to use updated (v2018.1?) versions of linux-xlnx, u-boot-xlnx, arm-trusted-firmware, and pmu-firmware?
Or do we have to wait for a sumo branch? If so, when would such a branch be available for a test run?
It would be nice if Xilinx would maintain a wiki documenting the schedule for Xilinx YP branches. Xilinx wiki is good stuff!

While we now realize trying to bring up Xilinx rel-v2018.1 with YP rocko is a mistake we do feel we are very close to having a bootable zcu102 system with our effort - it's just that we are running into a hard showstopper issue with the arm-trusted-firmware/pmu-firmware combination and we are not sure what the issue is except something goes wrong with the communication between ATF and PMU when the ATF requests the chip ID from the PMU during boot. See my findings here: https://lists.yoctoproject.org/pipermail/meta-xilinx/2018-May/003809.html

I don't know how far Xilinx is preparing for the next YP rocko/sumo update but I assume you will have to fix the same issue - perhaps it would possible to get some support from Xilinx to resolve this issue now?
I mean, if Xilinx helps fix the issue now it can go directly into the next rocko/sumo update for everyone to benefit now and when next release is due. I suspect this is an issue which can be quickly resolved with the help of Xilinx ATF/PMU experts.
Also, we have backtracked through the mailing list archive but couldn't find any patches that seems related to the ATF/PMU issue we see on the zcu102.


> We have been sending patches to mailing list to update the meta-xilinx-bsp master for Sumo release, this will include newer kernel etc

We are perhaps a front runner here but the very reason we tried to bring up the rel-v2018.1 with YP rocko is that we wanted to use the new v2018.1 components to benefit from the various bug fixes and improvements. In particular we wanted to try out the much never kernel (4.9 vs 4.14) because we see some instability issues with the Xilinx qspi-nor driver when running some UBI stress tests. First we tried back porting most of the qspi-nor changes from 4.14 to 4.9 but that didn't fix the instability. There are some deeper infrastructure changes in the qspi-nor driver (e.g. change from mmio to ioctl API calls) that we coulnd't easily backport so that put us on the path of trying to upgrade everything to the new rel-v2081.1 to get to the new kernel.


>> I understand that Xilinx only tests meta-petalinux in combination with the rest but
>> I really want to make a plea for Xilinx to also start testing meta-xilinx-bsp in the
>> aim to keep it independent and functional.
>
>Xilinx tools release cannot be independent, the workflow is different and needs all the layers.
>If you continue to use rel-v2018 (release branches), then the recommendation is to use all layers.
>If you use yocto based release branches, meta-xilinx-bsp should fairly work ( except master as changes to OE-Core master might cause breakages)

Understood.


>The plan to collapse all layers under meta-xilinx is proposed, and the first step of splitting has happened.
>We will continue to make all the necessary changes to have all layers under meta-xilinx later this year.
>There are certain dependencies which needs to be addressed before having meta-petalinux in meta-xilinx umbrella.
>We are working with OE-core to resolve some of these depedencies
>https://lists.yoctoproject.org/pipermail/meta-xilinx/2017-November/003309.html

Great - as long as Xilinx is committed to continue putting out Xilinx YP branch releases so that we can use the clean cut YP / meta-xilinx-bsp combination without any involvement of meta-xilinx-tools and meta-xilinx-petalinux, then I think both the community and many customers will be happy.

Best regards, Martin
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.yoctoproject.org/pipermail/meta-xilinx/attachments/20180516/faf89420/attachment-0001.html>