[linux-yocto] Locating kernel metadata

[Bruce Ashfield]

On 13-09-01 8:32 AM, Peter A. Bigot wrote:
> Thanks for the additional information. My entire expectation of how
> linux-yocto metadata worked turns out to be wrong, but you've managed to
> clear it up for me.
>
> What I really wanted from linux-yocto was a way to isolate documented
> kernel configuration fragments to share configuration data among a set
> of related BSPs, and keep particular features (such as OMAP ISP) out
> unless a particular build desired that feature.  The linux-mainline

Before I comment more below, I have to say that what you have described
is definitely one of the primary drivers of linux-yocto. Sharing of
fragments between many different BSPs and even projects. So I hope that
our deep dive hasn't made it sound like that isn't a goal.

> approach of a monolithic defconfig with no comments explaining why
> particular Kconfig options are set is itself unmaintainable, especially
> when updating to new kernels where names change.

And this is another reason why things are they way they are. I lived
through many kernel update cycles with defconfigs, things break, you
don't know why, there's no way to couple configs to patches and
finally you have to edit a multitude of defconfigs sprayed around
directories to have a consistent software/hardware policy between
boards.

>
> That linux-yocto is designed to support building kernels for multiple
> BSPs simultaneously from a shared metadata hierarchy is an architectural
> driver that was not clearly communicated. Perhaps because that need is
> completely irrelevant to my goals so I didn't notice it mentioned
> before.  It also explains why I failed to communicate my discomfort with
> the way recipe-space metadata is handled differently from in-tree metadata.
>
> For your case in-tree metadata (by which I mean scc and cfg files in
> particular) needs to be untouched because it's shared.

.. or I'd say "global" as well as "shared". Just like the mainline
Kconfig files are considered a common base, and if you want to
change them, you get a change merged to the mainline kernel or
carry a patch temporarily. We do the same thing here.

>
> For me it has pieces that are wrong and that should be overridden in a
> distinct step so I can review in-place the metadata that the kernel
> tools are actually going to use to construct the config file.  I thought
> SRC_URI provided a way for me to override those files prior to their

And what you are describing is to be expected (at least I expect it), since
the meta data you see is the result of 100-150 different BSPs over
several years.

The data set is not etched in stone, and as we get more, different and
better use cases. I expect things to change, both in the tools and
in the data .. which is why this is such a useful conversation.

> being interpreted, rather like providing a default config file to be
> copied into the source tree.  In fact linux-yocto is much more active
> than I expected when it unconditionally processes the scc files in
> SRC_URI during fetch, rather than only if they're referenced from the
> root BSP description.

Heh. What you described as you expectation .. is actually how the first
implementation behaved! You didn't specify anything on the SRC_URI
but instead simply used the in-tree data + some special variables to
extend it. But that makes the process so significantly different than
any other recipe/patch/config process in the system that it becomes
unusable for most.

So what we have now is a compromise. At the most basic level, you add
something to the SRC_URI, it can be a patch, and config, a .scc or
a directory that points to .scc and .cfgs. Those are processed and
can extend, modify or override the base configuration. At the simplest
case, it looks like any other recipe with just "foo.patch" and foo.patch
is applied (or attempted to be applied).  Same with .cfg's, etc.

But the system is really flexible. If you don't want to use it like that ..
don't. The goal was that the simple interface gets someone up and running
with no specific linux-yocto learning curve. When you realize that you
have competing features, many BSPs, branches and merges you want to
do .. questions are asked, and then the rest of the system can be used
to meet those goals.

Pretty much any configuration or management scheme you want, can likely
be supported. That's why I was describing the ability to override, branch,
merge .. or change whatever you want. Just not quite in the way you
described, but the end result is the same.

>
> At this point I need to go try to see how to do what I want to do within
> the current framework.  Prior to this my attempts failed because I
> couldn't figure out what the tools were doing with my metadata; now I
> have more clues and may be able to make progress.
>
> On 09/01/2013 12:30 AM, Bruce Ashfield wrote:
>> On 13-08-31 12:07 PM, Peter A. Bigot wrote:
>>> Let's start by defining terms that I think don't exist or aren't used
>>> consistently.  In what follows:
>>
>> Some of what you may be missing is in the "kernel architecture" manuals,
>> which I wrote ages ago .. and which we subsequently moved to a harder
>> to find area of the Yocto site .. since they contained too much detailed
>> information that wasn't generally useful.
>
> I think a pointer to this kernel architecture documentation belongs in
> the YPLKDM "Other Resources" section.  I've added bug 5093 to address
> this for 1.5.  In the meantime, perhaps you could provide a link here?

I couldn't find the one I wanted on the yocto site .. only this older
version: 
http://www.yoctoproject.org/docs/1.2/kernel-manual/kernel-manual.html#kernel-architecture

But when I'm back to work on Tuesday, I can hunt up something better.

Bruce

>
> Thanks again.
>
> Peter