[Toaster] [RFC] Standard setup for a customer-oriented Toaster instance

[Barros Pena, Belen]

Hi all,

I wanted to start a discussion about establishing a standard process to
set up a secure, remote Toaster instance for "customers", where
"customers" could be people using your commercial distro, or people you
are doing consulting for, or a team of web developers who need to build
their own images, or ... [insert your own here].

This is I think one of the big gaps we still have, and I believe should be
plugged in the Toaster manual for the 2.2 release. With the introduction
of Docker containers, Django fixtures, non-git layers and the ability to
delete projects from the Toaster UI, this should now be possible (I hope).
 


>From what I hear, such a set up normally requires to provide a limited set
of layers locked to a specific version (a certain commit), being built
with a certain bitbake commit as well.

In my head, such set up would involve solving 3 problems:

1. How to lock the BitBake version
2. How to populate Toaster with the correct layer information
3. How to provide access control

I'll get through each.

1) How to lock the BitBake version:

For this I hope we could use the local release. In the Toaster instance,
that would be the only release available to users, and will be used for
all projects. When creating projects, users will only have to enter the
project name: no release selection menu will be available to them. The
Toaster administrator will need to checkout the version of BitBake she
wants users to build with. Hopefully this will correspond to one of the
stable releases (for example, 2.2), and so will simply require to clone a
stable poky repo or a stable bitbake.

2) How to populate the layer information: this is more fun ;)

2.1 You are of course unlikely to want all the layers from the OE Layer
Index in your Toaster, so we need to provide a way to remove the layer
index from the Toaster configuration, or a way of not running the
lsupdates command, or something like that. This might already be in place,
but I am not sure.

2.2 The next problem is how to generate the recipe and machine information
for the layers you want to expose to your customers, since this is what
makes Toaster useful. I can see a couple of options for this:

2.2.1 Set up your own instance of the layer index, and get it to parse
your layers. The pros of this approach: the layer index provides machine
data, and we can use lsupdates to populate the toaster database with the
layer information. The cons: I suspect setting up a local instance of the
layer index to parse your layers might not be straightforward.


2.2.2 Use a Django fixture. For this, you would probably need to

1) import your layers manually into an empty Toaster instance (or load the
basic layer information, i.e name, source code location and dependencies,
via a Django fixture)

2) build all the layers

3) dump the layer database into a fixture and load it into the customer
Toaster instance (or delete the project from the customer Toaster instance
if that's what you used for the initial builds).

The pros of this approach: you get package data too, which will make image
customisation a breeze for customers. The cons: machine information will
be missing, since we don't have a way to get machine information from
builds, so you will need to add machines manually into the Django fixture
if you want the machines to appear in Toaster. Another con is that this
method is untested: it should work in theory, but ...

3) Access control: 

Since Toaster still does not have the concept of users or permissions, we
will need to provide access control in some other way. I know Michael Wood
has used Apache in the past for this, so maybe that's what we recommend.

The above comes from my somehow limited understanding of the Toaster
internals, and my limited views on how people interact with their
customers. So I would need Toaster contributors, and people thinking of
Toaster as a tool for their customers, to highlight what's wrong,
impossible or missing from the above. So please, pick on it :)

Thanks

Belén