[Toaster] [RFC] Configuration variables in Toaster

Reyna, David david.reyna at windriver.com
Fri Jul 11 00:16:23 PDT 2014


Hi Alex,

> In this respect, I propose that we blacklist from Toaster interface any variable in category b), at least for 1.7 release.

While I am reluctant, I can accept this if we are just trying to solve 1.7, and come back to this in 1.8.

I still stand by my point that the buildbot (or other infrastructure) will _always_ have to dynamically compute and "sanitize" the (b) variables to do its job, so I am still unclear how you are saving work for the buildbot with your proposal. Perhaps you are hoping to use buildbot unmodified and/or without an intermediary agent, but I do not see how that will work.

I would also like to understand your OOB scenario for Toaster users, since you are counter-proposing my suggestions on that point.

For example:
  * Will Toaster create a default admin setup on-the-fly for first time users, just as today it will create a default database on-the-fly?
  * Will Toaster aggressively detect and reject if the user should attempt to manually add the (b) variables in the custom configure var page?

I will mention that my immediate issue is that I am positioning Toaster to replace the Wind River Linux 7.0 project configuration tools, and for that I _do_ have to consider the local case. I can accept some extra work on my team above and beyond your Toaster-1.7 design, as long as the Toaster-1.7 does not prevent what I need to do.

Thanks,
David

From: Damian, Alexandru [mailto:alexandru.damian at intel.com]
Sent: Thursday, July 10, 2014 8:48 AM
To: Reyna, David
Cc: BARROS PENA, BELEN; toaster at yoctoproject.org; Wymore, Farrell; Paul Eggleton; Richard Purdie
Subject: Re: [Toaster] [RFC] Configuration variables in Toaster

Hello,
I've been reviewing this proposal, and I think it's good to have this discussion now, and the points raised are excellent.

I'm seeing a different angle to the variable story. CCing Paul and Richard so they can track this discussion.
I think that we can separate the bitbake variables in two different categories, even if bitbake itself makes no such separation, -
a). variables that influence WHAT will be built - e.g EXTRA_IMAGE_FEATURES, or IMAGE_INSTALL_append

b). variables that influence HOW it will be build - e.g. PARALLEL_MAKE, or SSTATE_MIRRORS for that matter

When you configure bitbake, you have to put two different hats, a developer hat for a). and a sysadmin hat for b).
I think that, at the current point, Toaster should focus at being only a developer tool, and it should concern itself only with the a). hat.
In this respect, I propose that we blacklist from Toaster interface any variable in category b), at least for 1.7 release.

In this way, we reduce the chance that Toaster settings will screw something up on the infrastructure setup, and we'll also solve the problem of having the buildbot (or other infrastructure) "sanitize" the variables set by Toaster.

In respect to the a). variables, I think it's a good idea to provide a set of commonly-used variables with sane defaults, and offer help for the user when he adds other variables, e.g. link to manual description or help searching variable name.
What do you think ?

Cheers,
Alex

On Wed, Jul 9, 2014 at 11:28 PM, Reyna, David <david.reyna at windriver.com<mailto:david.reyna at windriver.com>> wrote:
Hi Belén,

You write: "In a Toaster instance connected to a build farm, a user of Toaster should not be able to override any variables that impact how hardware runs builds and where
artifacts are stored."

I would think of this in the opposite manner.

Proposed wording: "In a Toaster instance connected to a build farm, the remote build instance will _always_ override any variables that impact how hardware runs builds and where artifacts are stored, regardless of any default bitbake settings or explicit user settings."

Here is my reasoning:

  * The remote build instance's manager/configuration will need to dynamically set these values regardless and in all cases, based on its current resources and environment, whether they are previously defined or not.

  * We cannot prevent the user from manually setting the values no matter what we do, as per your mention of the free form "add variables" option.

  * We can think of this class of variables as "preferences". If it is a local build then these values will be used (not overridden) and user will get what they expect. If it is a remote build then the values will be expected to be overridden by design and again the user will get what they expect.

  * With this model there are no surprises, both workflows are easily covered, and everyone is happy.

- David



> -----Original Message-----
> From: Barros Pena, Belen [mailto:belen.barros.pena at intel.com<mailto:belen.barros.pena at intel.com>]
> Sent: Wednesday, July 09, 2014 9:42 AM
> To: toaster at yoctoproject.org<mailto:toaster at yoctoproject.org>
> Cc: Wymore, Farrell; Reyna, David; Lerner, Dave
> Subject: [RFC] Configuration variables in Toaster
>
> Some discussion happened today about the list of variables that Toaster
> should expose (see below from previous thread). It was observed that, for
> cases when you are in control of the build hardware, it would be handy to
> be able to set variables like PARALLEL_MAKE, BB_NUMBER_THREADS and
> SSTATE_DIR. I took out those variables from the list because they impact
> how hardware runs builds and where build artifacts are stored, which are
> things that a user of a Toaster instance connected to a build farm should
> probably not touch.
>
> It was suggested that maybe the list of variables could change if Toaster
> is running on the same machine that runs the builds. There is also the
> "add variables" option, which is free form and that will allow you to type
> any variable and any value you want. So that option would in theory allow
> you to set PARALLEL_MAKE or anything else you might want to set.
>
> Now: that brings up a question about variable handling. In a Toaster
> instance connected to a build farm, a user of Toaster should not be able
> to override any variables that impact how hardware runs builds and where
> artifacts are stored. If the user adds PARALLEL_MAKE = "8" to a Toaster
> project configuration, that should have no effect on the PARALLEL_MAKE
> value that a build server in a farm is supposed to use, and that should
> have been set by whomever administers that farm. Same goes for SSTATE_DIR,
> SSTATE_MIRRORS, DL_DIR, etc. Maybe I am just not looking at this in the
> right way, or there is something I don't fully understand, but it seems to
> me that Toaster could become a liability otherwise.
>
> Any thoughts on how this should be handled would be much appreciated.
>
> Thanks!
>
> Belén
>
> On 08/07/2014 15:40, "Barros Pena, Belen" <belen.barros.pena at intel.com<mailto:belen.barros.pena at intel.com>>
> wrote:
>
> >That's the Toaster page that allows you to edit the project local.conf
> >file. I am trying to come up with a final list of the variables that will
> >be listed in that page.
> >
> >Looking at the default local.conf file, and excluding stuff that should be
> >defined by the Toaster instance (things like SSTATE_MIRRORS or DL_DIR),
> >this is what I'm left with:
> >
> >PACKAGE_CLASSES
> >SDKMACHINE
> >EXTRA_IMAGE_FEATURES
> >USER_CLASSES
> >TEST_IMAGE
> >OE_TERMINAL
> >PATCHRESOLVE
> >PACKAGECONFIG_pn-qemu-native (for Qemu configuration)
> >ASSUME_PROVIDED (for Qemu configuration)
> >CONF_VERSION
> >
> >Of the above, I am not sure we should expose the Qemu configuration stuff
> >or CONF_VERSION, but I could be wrong.
> >
> >
> >Hob also allows you to set:
> >
> >INCOMPATIBLE_LICENSE
> >IMAGE_FSTYPES
> >IMAGE_EXTRA_SPACE
> >TOOLCHAIN_BUILD
> >LINGUAS_INSTALL
> >plus proxy configuration variables (http_proxy, https_proxy, ftp_proxy,
> >all_proxy, CVS_PROXY_HOST, CVS_PROXY_PORT)
> >
> >Should we list this Hob stuff as well (except the proxy variables, I
> >guess)?
> >
> >Apart from the above, there will be a way to set IMAGE_INSTALL_append, and
> >also an option to add any other variable (free text kind of thing).
> >
> >Any thoughts on this list?
> >
> >Thanks!
> >
> >Belén
> >
> >--
> >_______________________________________________
> >toaster mailing list
> >toaster at yoctoproject.org<mailto:toaster at yoctoproject.org>
> >https://lists.yoctoproject.org/listinfo/toaster
>

--
_______________________________________________
toaster mailing list
toaster at yoctoproject.org<mailto:toaster at yoctoproject.org>
https://lists.yoctoproject.org/listinfo/toaster



--
Alex Damian
Yocto Project
SSG / OTC
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.yoctoproject.org/pipermail/toaster/attachments/20140711/bf8096b8/attachment-0001.html>


More information about the toaster mailing list