[meta-virtualization] Docker: include docker images to the Yocto image

[Stefan Agner]

On 2019-07-25 16:10, Bruce Ashfield wrote:
> On Thu, Jul 25, 2019 at 3:50 AM Валентин <imort.kz at gmail.com> wrote:
>>
>> Hello everyone,
>>
>> I wonder if it's possible to pull some docker images from the repository in the end of the Yocto image build so they'll be available without the need to load them at the first boot?
>>
> 
> I haven't heard of anyone doing this (and I know that it isn't
> something I'm doing, or have considered), but hopefully if someone is
> doing something like this, they can offer some advice.

I know that balena is doing this in their docker-disk recipe:
https://github.com/balena-os/meta-balena/tree/master/meta-balena-common/recipes-containers/docker-disk

However, they do access the host Docker to make this happen, which seems
not particular nice.

It really seems to me that the Docker image storage is well guarded by
the Docker daemon, and it is not meant that somebody accesses the
storage directly (and as far as I am aware there are no tools to do
this).

> 
> What I can say, is that with an approach like this I see a few issues
> off the top of my head. You'd either need a docker-native (not easy to
> do, I've tried) or one of the other container manipulation projects
> (check the layer index for skopeo, etc) (and you'd also need them in
> -native .. also very hard to do .. I've tried) to fetch/pull the
> container image and put it into a place for the build to pickup and
> get into the image. If you didn't have those -native tools, you'd have
> a binding to the host.
> 
> And it gets worse, if you are building and booting for x86-64, you
> might get away with things. But as a general solution, you'd likely
> need to run docker in qemu-user or otherwise specify your target
> architecture to even get a container of the right arch.
> 
> On the non-technical side, if you were commercially shipping an image
> assembled like this, you'd have licensing and other issues to deal
> with for the image you pulled from the container registry.
> 
> I can say that many of us, are building containers and installing them
> into images so they'll be available on first boot and can simply be
> run. The image-oci class that is part of meta-virt takes care of
> taking an OE built image and putting it into OCI format so it can be
> directly executed by runc (I'd recommend runc versus docker if you
> just want to execute container images). Those OCI images are
> compatible with docker, so you also should be able to install them to
> an image and run them directly. I just haven't worked through the
> details of that myself.
> 
> Other alternatives could be  if you manually pull the containers,
> export them and create a recipe/bbclass that takes the exported
> container image and installs them to the image. You'd have the same
> licensing issues that I mentioned above, but from the point of view
> getting them into the image, it is just like any of the other "binary
> installs" that some people have done over the years.
> 
> Another option could be to have a local to the target docker registry
> running, and when you boot, you'd pull them from there and not have to
> go to the wider network. That gives you some interesting upgrade
> options, etc, but has some unfinished technical challenges (like
> running and populating the registry as part of the build).

Probably an easier version of this is to store the image using `docker
save`, put it somewhere into the image and have a one-time service which
does `docker load` and cleanup.


What I am currently looking into is using podman. It is designed as a
replacement, and the storage format is more open. skopeo can write
podman images directly, so then its only a matter of invoking skopeo
with the correct settings.


Best regards,
Stefan

> 
> Cheers,
> 
> Bruce
> 
>> I'm ok both with `docker pull` or `docker load` options.
>>
>> Thank you
>>
>>
>> --
>> Best regards,
>> Valentin Kim
>>
>> Skype: imortkz
>> Mobile & WhatsApp: +7 777 314 82 19
>>
>> --
>> _______________________________________________
>> meta-virtualization mailing list
>> meta-virtualization at yoctoproject.org
>> https://lists.yoctoproject.org/listinfo/meta-virtualization
> 
> 
> 
> --
> - Thou shalt not follow the NULL pointer, for chaos and madness await
> thee at its end
> - "Use the force Harry" - Gandalf, Star Trek II