[Toaster] Toaster Not Respecting Environment Variables

[Gillespie, Bryan]

David,


Thanks for the reply. I was able to get local toaster builds working using the "defect"/workaround + .gitconfig find/replace (to switch all git:// requests to be https:// requests... not sure if there is a more standard way of telling yocto to use https git urls).


Eventually we probably will have a remote build VM, so I think this was only a temporary setback.


Thanks again!

Bryan


=========================
Lab: x3214, Office: x5956
________________________________
From: Reyna, David <david.reyna at windriver.com>
Sent: Friday, January 20, 2017 4:40:44 PM
To: Gillespie, Bryan; toaster at yoctoproject.org
Subject: RE: Toaster Not Respecting Environment Variables

Hi Bryan,

Thank you for your question.

  1. Git is not hard coded to use the git port in Toaster. It just uses standard "git clone" commands (in toaster/bldcontrol/localhostbecontroller.py" specifically). But see the answer to #2.

  2. The local environment is indeed purged/ignored by Toaster. The reason was that for the remote build aspect of Toaster we wanted the person who was the "build master" to control the environment over remote users, so that the builds were safe. I understand your point that this is a problem for local usage of Toaster, where you are your own build master.

There is actually a defect around this topic which may be of interest:
  Bug 10281<https://bugzilla.yoctoproject.org/show_bug.cgi?id=10281> - Configuration variables: I can set a value for standard shell environment variable http_proxy

What is interesting about this defect is that this shows you how you can actually get around the limitation, and specifically for "http_proxy", by setting it as a Toaster/bitbake environment variable. Obviously before we "fix" this defect (to close this loophole) we should have a general solution for supporting local builders like yourself.

Also, by explicitly setting this variable in Toaster builds, remote builds across the web will still get the proper value set, as per the goal that Toaster it is not dependent on (or vulnerable to) your local environment.

- David

From: toaster-bounces at yoctoproject.org [mailto:toaster-bounces at yoctoproject.org] On Behalf Of Gillespie, Bryan
Sent: Tuesday, January 17, 2017 8:40 AM
To: toaster at yoctoproject.org
Subject: [Toaster] Toaster Not Respecting Environment Variables


All,



I am trying to use Toaster to configure and build project, however, I am working behind a corporate firewall that requires a proxy. When I try to build core-image-minimal with Toaster, it fails immediately with:


2017-01-17 11:22:44,942 ERROR runbuilds: Error launching build command: git fetch --all && git reset --hard "origin/krogoth"
fatal: Unable to look up git.yoctoproject.org (port 9418) (Temporary failure in name resolution)
error: Could not fetch origin

Traceback (most recent call last):
  File "/home/user/poky/bitbake/lib/toaster/bldcontrol/management/commands/runbuilds.py", line 67, in schedule
    "%d:%d" % (br.pk, bec.be.pk))
  File "/home/user/poky/bitbake/lib/toaster/bldcontrol/localhostbecontroller.py", line 255, in triggerBuild
    layers = self.setLayers(bitbake, layers, targets)
  File "/home/user/poky/bitbake/lib/toaster/bldcontrol/localhostbecontroller.py", line 161, in setLayers
    self._shellcmd('git fetch --all && git reset --hard "%s"' % ref, localdirname)
  File "/home/user/poky/bitbake/lib/toaster/bldcontrol/localhostbecontroller.py", line 70, in _shellcmd
    raise ShellCmdException(err)
ShellCmdException: command: git fetch --all && git reset --hard "origin/krogoth"
fatal: Unable to look up git.yoctoproject.org (port 9418) (Temporary failure in name resolution)
error: Could not fetch origin




Two questions:

1. Why is it using git protocol (port 9418)? My origin remote for the project is setup using https:


(venv) user at mantis:~/poky/build$ git remote -v
origin https://git.yoctoproject.org/git/poky (fetch)
origin https://git.yoctoproject.org/git/poky (push)


Is it hardcoded? I prefer https because it is much faster in our network setup.



2. That command definitely works when the proxy environment variables are set. I added the following snippet to  bitbake/lib/toaster/bldcontrol/localhostbecontroller.py:


logger.debug("ENV::",os.environ)



The output of that log indicates to me that Toaster is purging all environment variables, hence why the git fetch fails.



Should I file a bug report or am I just missing something?



Thanks,

Bryan


=========================
Lab: x3214, Office: x5956
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.yoctoproject.org/pipermail/toaster/attachments/20170121/e7b21b93/attachment-0001.html>