[Toaster] [review-request][PATCH] toaster: Fix import of custom layers from toasterjson.conf
Barros Pena, Belen
belen.barros.pena at intel.com
Tue Sep 15 06:35:44 PDT 2015
On 11/09/2015 07:16, "toaster-bounces at yoctoproject.org on behalf of sujith
h" <toaster-bounces at yoctoproject.org on behalf of sujith.h at gmail.com>
wrote:
>Hi Brian,
>
>
>I tried building your branch and I am still not able to reproduce the
>failure. The branch I have used is:
>poky-contrib/bavery/submit/sujith/20150827_custom_layer_import
Sujith asked me to try and reproduce the problems Brian was seeing with
his custom toaster configuration. I was able to load Sujith's
configuration into Toaster without problems, although we discovered that,
as it was, it required the existence of a poky directory (if I cloned the
poky repo into directory 'xyz' using 'git clone
git://git.yoctoproject.org/poky xyz' the builds failed). I believe Sujith
has a patch to fix that.
After loading the custom configuration into Toaster and cloning the needed
layers locally I was able to build core-image-sato with the qtbase package
no problems. The build completed, the rootfs that was created seems fine,
and the list of packages in the rootfs.manifest file seems correct.
However, the information about the build collected by Toaster is
incomplete.
I then reproduced the build getting Toaster to clone the layers instead of
using local layers in my builds machine: the build completed without
problems and the information came into Toaster correctly.
So it looks like Sujith's patches might be ok, and what we have is some
kind of data collection issue whenever you build a local layer that is not
one of the YP core layers (openembedded-core, meta-yocto and
meta-yocto-bsp). What I've been able to verify is:
1. That the information about the additional local layers is not picked up
by Toaster
2. That the list of packages installed is incorrect
I am not sure how we move forward from here, but this is what seems
logical to me:
* Sujith, I guess we need a new version of your patch with the 'poky'
hardcoding removed
* I guess we need to open an issue in Bugzilla for the local layers data
collection issue
Is there anything else that would need to be done?
Thanks!
Belén
>
>Here is th output:
>
>sujith at dockers:~/MEL/toaster_test$ ls build/tmp/deploy/images/qemux86/ -lt
>total 1308448
>lrwxrwxrwx 1 sujith sujith 50 Sep 11 00:27
>core-image-sato-qemux86.ext3 ->
>core-image-sato-qemux86-20150910101208.rootfs.ext3
>lrwxrwxrwx 1 sujith sujith 50 Sep 11 00:27
>core-image-sato-qemux86.ext4 ->
>core-image-sato-qemux86-20150910101208.rootfs.ext4
>lrwxrwxrwx 1 sujith sujith 51 Sep 11 00:27
>core-image-sato-qemux86.jffs2 ->
>core-image-sato-qemux86-20150910101208.rootfs.jffs2
>lrwxrwxrwx 1 sujith sujith 54 Sep 11 00:27
>core-image-sato-qemux86.manifest ->
>core-image-sato-qemux86-20150910101208.rootfs.manifest
>lrwxrwxrwx 1 sujith sujith 53 Sep 11 00:27
>core-image-sato-qemux86.tar.bz2 ->
>core-image-sato-qemux86-20150910101208.rootfs.tar.bz2
>-rw-r--r-- 1 sujith sujith 538497024 Sep 11 00:27
>core-image-sato-qemux86-20150910101208.rootfs.ext3
>-rw-r--r-- 1 sujith sujith 154014569 Sep 11 00:27
>core-image-sato-qemux86-20150910101208.rootfs.tar.bz2
>-rw-r--r-- 1 sujith sujith 538497024 Sep 11 00:26
>core-image-sato-qemux86-20150910101208.rootfs.ext4
>-rw-r--r-- 1 sujith sujith 196345856 Sep 11 00:26
>core-image-sato-qemux86-20150910101208.rootfs.jffs2
>-rw-r--r-- 1 sujith sujith 21170 Sep 11 00:24
>core-image-sato-qemux86-20150910101208.rootfs.manifest
>-rw-r--r-- 2 sujith sujith 294 Sep 11 00:21
>README_-_DO_NOT_DELETE_FILES_IN_THIS_DIRECTORY.txt
>lrwxrwxrwx 1 sujith sujith 71 Sep 10 22:34 bzImage ->
>bzImage--4.1.2+git0+45393dd54f_4e30e64c44-r0-qemux86-20150910101208.bin
>lrwxrwxrwx 1 sujith sujith 71 Sep 10 22:34 bzImage-qemux86.bin ->
>bzImage--4.1.2+git0+45393dd54f_4e30e64c44-r0-qemux86-20150910101208.bin
>lrwxrwxrwx 1 sujith sujith 71 Sep 10 22:34 modules-qemux86.tgz ->
>modules--4.1.2+git0+45393dd54f_4e30e64c44-r0-qemux86-20150910101208.tgz
>-rw-rw-r-- 2 sujith sujith 82529712 Sep 10 22:34
>modules--4.1.2+git0+45393dd54f_4e30e64c44-r0-qemux86-20150910101208.tgz
>-rw-r--r-- 2 sujith sujith 6924944 Sep 10 22:34
>bzImage--4.1.2+git0+45393dd54f_4e30e64c44-r0-qemux86-20150910101208.bin
>sujith at dockers:~/MEL/toaster_test$
>
>Also I have noticed that qtbase got built successfully. And the rootfs
>has the qtbase installed. In the IMAGE_INSTALL_append I have added only
>qtbase.
>
>
>The poky is cloned under: /home/sujith/MEL/toaster_test
>
>sujith at dockers:~/MEL/toaster_test$ ls -lt
>total 13744
>-rw-rw-r-- 1 sujith sujith 5357 Sep 11 11:39 build_config.tar.bz2
>-rw-r--r-- 1 sujith sujith 14029824 Sep 11 11:25 toaster.sqlite
>drwxr-xr-x 8 sujith sujith 4096 Sep 11 00:27 build
>-rw-rw-r-- 1 sujith sujith 282 Sep 10 15:37 toaster_server.log
>drwxrwxr-x 2 sujith sujith 4096 Sep 10 15:36 toaster_build_artifacts
>-rw-rw-r-- 1 sujith sujith 4703 Sep 10 15:30 toasterconf.json
>drwxrwxr-x 6 sujith sujith 4096 Sep 10 15:06 venv
>drwxrwxr-x 11 sujith sujith 4096 Sep 8 14:35 poky
>sujith at dockers:~/MEL/toaster_test$
>
>
>And the cloned_layers directory resides in /home/sujith/MEL:
>sujith at dockers:~/MEL$ ls -lt cloned_layers/
>total 8
>drwxrwxr-x 8 sujith sujith 4096 Sep 10 15:04 meta-qt5
>drwxrwxr-x 19 sujith sujith 4096 Sep 10 15:03 meta-openembedded
>sujith at dockers:~/MEL$
>
>
>
>
>I am also attaching the build config tar ball with the mail.
>
>
>Unfortunately even the size of toaster_ui.log is very less. Attaching
>that along with the mail.
>
>
>Let me know if you still face the same issue.
>
>
>Thanks,
>
>Sujith H
>
>
>On Thu, Sep 10, 2015 at 8:52 PM, sujith h
><sujith.h at gmail.com> wrote:
>
>Hi Brian,
>
>
>I am testing on your poky-contrib branch ( i.e,
>bavery/submit/sujith/20150827_custom_layer_import) with core-image-sato
>and qtbase added in IMAGE_INSTALL_append.The build is still in progress
>some 39% task completed. So I would be able to deliver the configuration
> for the same build tomorrow only :( Sorry for the inconvenience.
>
>
>
>Thanks,
>
>Sujith H
>
>On Wed, Sep 9, 2015 at 9:16 PM, sujith h
><sujith.h at gmail.com> wrote:
>
>Hi Brian,
>
>
>I am attaching the tar file which contains the conf folder from the build
>directory and the toasterconf.json file just outside poky folder.
>
>
>Thanks,
>
>Sujith H
>
>
>On Wed, Sep 2, 2015 at 2:26 PM, sujith h
><sujith.h at gmail.com> wrote:
>
>Hi Brian,
>
>
>Was there an instance of bitbake running in your build folder before you
>triggered the build from toaster?
>
>
>Thanks,
>
>Sujith H
>
>
>
>On Wed, Sep 2, 2015 at 2:12 PM, sujith h
><sujith.h at gmail.com> wrote:
>
>
>
>On Tue, Sep 1, 2015 at 10:39 PM, Brian Avery
><avery.brian at gmail.com> wrote:
>
>Hi,
>
>I'm on the correct patch afaik. I put the toasterconf.json into the
>directory above poky and loaded it with:
>poky/bitbake/lib/toaster/manage.py loadconf
>/home/bavery/src/intel/myBugs2/sujith/toasterconf.json
>
>
>
>
>This is not my latest patch which I worked on :) It was with subject:
>[PATCH] toaster: Import external layers outside poky from toasterconf.json
>
>
>
>
>I also used option 0 so that toaster did not attempt to load the
>layers from openembedded.
>
>My branch for this test is:
>http://git.yoctoproject.org/cgit/cgit.cgi/poky-contrib/?h=bavery%2Fsubmit%
>2Fsujith%2F20150827_custom_layer_import
>
>I ran a build of core-image-sato with qtbase and iw set in
>IMAGE_INSTALL_append for the project.
>
>The results:
>The good-
>I got a core-image-sato as well as the qt5 and iw packages; so the build
>worked.
>
>The not so good -
>1) My build progress bar did not update.
>2) my recipe list after the build looks like:
>https://www.dropbox.com/s/qs7hulsbmcgg4ik/Screenshot%202015-09-01%2010.02.
>29.png?dl=0
>3) in the build directory I set when starting up toaster (where the
>tmp/deploy ends up among other things) the toaster_ui.log file was
>filled with errors (it ended up being 17G by the end of the build).
>I'm attaching the first 120 lines of it so you can see what it looked
>like.
>
>
>
>I would like to share my poky and cloned_layers directory and the ui log
>file attached:
>
>------------
>sujith at kdekidd0:~/MEL/test_poky$ pwd
>/home/sujith/MEL/test_poky
>sujith at kdekidd0:~/MEL/test_poky$ ls -lt
>total 16488
>-rw-r--r-- 1 sujith sujith 16855040 Sep 2 13:46 toaster.sqlite
>drwxrwxr-x 3 sujith sujith 4096 Sep 2 13:45 toaster_build_artifacts
>drwxr-xr-x 8 sujith sujith 4096 Sep 2 13:43 build
>drwxrwxr-x 11 sujith sujith 4096 Sep 2 10:01 poky
>-rw-rw-r-- 1 sujith sujith 4632 Aug 28 12:21 toasterconf.json
>drwxrwxr-x 6 sujith sujith 4096 Aug 28 11:53 venv
>sujith at kdekidd0:~/MEL/test_poky$ ls ../cloned_layers/ -l
>total 8
>drwxrwxr-x 19 sujith sujith 4096 Aug 28 11:51 meta-oe
>drwxrwxr-x 8 sujith sujith 4096 Aug 28 12:10 meta-qt5
>sujith at kdekidd0:~/MEL/test_poky$
>------------
>
>
>Now the step done to load configuration:
>
>------------
>sujith at kdekidd0:~/MEL/test_poky$ source venv/bin/activate
>(venv)sujith at kdekidd0:~/MEL/test_poky$ ./poky/bitbake/bin/toaster
>Syncing...
>Creating tables ...
>Creating table auth_permission
>Creating table auth_group_permissions
>Creating table auth_group
>Creating table auth_user_groups
>Creating table auth_user_user_permissions
>Creating table auth_user
>Creating table django_content_type
>Creating table django_session
>Creating table django_admin_log
>Creating table south_migrationhistory
>
>You just installed Django's auth system, which means you don't have any
>superusers defined.
>Would you like to create one now? (yes/no): yes
>Username (leave blank to use 'sujith'):
>Email address:
>Password:
>Password (again):
>Superuser created successfully.
>Installing custom SQL ...
>Installing indexes ...
>Installed 0 object(s) from 0 fixture(s)
>
>Synced:
> > django.contrib.auth
> > django.contrib.contenttypes
> > django.contrib.messages
> > django.contrib.sessions
> > django.contrib.admin
> > django.contrib.staticfiles
> > django.contrib.humanize
> > south
>
>Not synced (use migrations):
> - bldcontrol
> - orm
>(use ./manage.py migrate to migrate these)
>Running migrations for orm:
> - Migrating forwards to 0024_auto__add_field_recipe_is_image.
> > orm:0001_initial
> > orm:0002_auto__add_field_build_timespent
> > orm:0003_timespent
> - Migration 'orm:0003_timespent' is marked for no-dry-run.
> > orm:0004_auto__add_field_package_installed_name
> >
>orm:0005_auto__add_target_image_file__add_target_file__add_field_variableh
>istor
> >
>orm:0006_auto__add_field_target_image_size__add_field_target_license_manif
>est_p
> > orm:0007_auto__add_helptext
> >
>orm:0008_auto__chg_field_variablehistory_operation__chg_field_recipe_descr
>iptio
> >
>orm:0009_auto__add_projectvariable__add_projectlayer__add_projecttarget__a
>dd_pr
> >
>orm:0010_auto__add_field_project_branch__add_field_project_short_descripti
>on__a
> > orm:0011_auto__add_field_projectlayer_dirpath
> >
>orm:0012_auto__add_field_projectlayer_optional__add_field_projecttarget_ta
>sk
> >
>orm:0013_auto__add_release__add_layerversiondependency__add_unique_layerve
>rsion
> >
>orm:0014_auto__chg_field_package_summary__chg_field_layer_summary__chg_fie
>ld_re
> >
>orm:0015_auto__add_field_layer_vcs_web_url__add_field_layer_vcs_web_tree_b
>ase_u
> >
>orm:0016_auto__add_field_release_helptext__chg_field_release_branch__add_i
>ndex_
> >
>orm:0017_auto__del_toastersettingdefaultlayer__add_releaselayersourceprior
>ity__
> > orm:0018_auto__add_field_layer_version_project
> > orm:0019_auto__add_buildartifact
> >
>orm:0020_auto__add_field_layer_version_local_path__add_field_recipe_pathfl
>ags__
> >
>orm:0021_auto__chg_field_build_project__chg_field_project_bitbake_version_
>_chg_
> >
>orm:0022_auto__add_field_target_task__add_field_layer_version_local_path__
>del_f
> >
>orm:0023_auto__del_field_build_warnings_no__del_field_build_errors_no__del
>_fiel
> > orm:0024_auto__add_field_recipe_is_image
> - Loading initial data for orm.
>Installed 0 object(s) from 0 fixture(s)
>Running migrations for bldcontrol:
> - Migrating forwards to 0008_brarchive.
> > bldcontrol:0001_initial
> >
>bldcontrol:0002_auto__add_field_buildenvironment_sourcedir__add_field_buil
>denvironment
> > bldcontrol:0003_auto__add_field_brlayer_dirpath
> > bldcontrol:0004_loadinitialdata
> - Migration 'bldcontrol:0004_loadinitialdata' is marked for no-dry-run.
> > bldcontrol:0005_auto__add_brerror
> > bldcontrol:0006_auto__add_brbitbake
> >
>bldcontrol:0007_auto__add_field_buildrequest_environment__chg_field_buildr
>equest_build
> > bldcontrol:0008_brarchive
> - Migration 'bldcontrol:0008_brarchive' is marked for no-dry-run.
> - Loading initial data for bldcontrol.
>Installed 0 object(s) from 0 fixture(s)
>Toaster needs to know in which directory it can download build log files
>and other artifacts.
> Toaster suggests "/home/sujith/MEL/test_poky/toaster_build_artifacts/".
> Press Enter to select
>"/home/sujith/MEL/test_poky/toaster_build_artifacts/" or type the full
>path to a different directory:
>
>Verifying the Build Environment. If the local Build Environment is not
>properly configured, you will be asked to configure it.
>
> -- Validation: The checkout directory must be set.
>Toaster needs to know in which directory it should check out the layers
>that will be needed for your builds.
> Toaster suggests "/home/sujith". If you select this directory, a layer
>like "meta-intel" will end up in "/home/sujith/meta-intel".
> Press Enter to select "/home/sujith" or type the full path to a
>different directory (must be a parent of current checkout directory):
>
>Verifying the Build Environment. If the local Build Environment is not
>properly configured, you will be asked to configure it.
>
> -- Validation: The build directory must be set.
>Toaster needs to know where is your build directory.
> The build directory is where all the artifacts created by your builds
>will be stored. Type the full path to the directory (for example: "
>/home/sujith/build")/home/sujith/MEL/test_poky/build
>Build configuration saved
>Verifying the Build Environment. If the local Build Environment is not
>properly configured, you will be asked to configure it.
>
>Toaster can use a SINGLE predefined configuration file to set up default
>project settings and layer information sources.
>
> Toaster will list now the configuration files that it found. Select the
>number to use the desired configuration file.
> [1] -
>/home/sujith/MEL/toaster_work/poky/meta-yocto/conf/toasterconf.json
> [2] - /home/sujith/MEL/cedar/poky/meta-yocto/conf/toasterconf.json
> [3] -
>/home/sujith/MEL/toaster_MEL_final/poky/meta-yocto/conf/toasterconf.json
> [4] - /home/sujith/MEL/test_poky/poky/meta-yocto/conf/toasterconf.json
> [5] -
>/home/sujith/MEL/upstream_poky_work/poky-contrib/meta-yocto/conf/toasterco
>nf.json
> [6] - /home/sujith/gitroot/poky/meta-yocto/conf/toasterconf.json
>
> [0] - Exit without importing any file
>
> Enter your option: 0
>Starting webserver...
>Webserver address: http://0.0.0.0:8000/
>Starting browser...
>Toaster is now running. You can stop it with Ctrl-C
>^Z
>[1]+ Stopped ./poky/bitbake/bin/toaster
>(venv)sujith at kdekidd0:~/MEL/test_poky$
>./poky/bitbake/lib/toaster/manage.py loadconf toasterconf.json
>
>(venv)sujith at kdekidd0:~/MEL/test_poky$ fg
>./poky/bitbake/bin/toaster
>2015-09-02 10:11:54,415 DEBUG localhostbecontroller, our git repos are
>{(u'', u'HEAD'): [('bitbake', u'bitbake')],
> (u'git://git.yoctoproject.org/poky <http://git.yoctoproject.org/poky>',
>u'HEAD'): [(u'openembedded-core',
> u'meta'),
> (u'meta-yocto',
> u'meta-yocto'),
> (u'meta-yocto-bsp',
> u'meta-yocto-bsp')],
> (u'git://github.com/meta-qt5/meta-qt5.git
><http://github.com/meta-qt5/meta-qt5.git>', u'HEAD'): [(u'meta-qt5',
> u'meta-qt5')],
> (u'git://github.com/openembedded/meta-oe.git
><http://github.com/openembedded/meta-oe.git>', u'HEAD'): [(u'meta-python',
>
>u'meta-python'),
>
>(u'meta-multimedia',
>
>u'meta-multimedia'),
>
>(u'meta-filesystems',
>
>u'meta-filesystems'),
>
>(u'meta-networking',
>
>u'meta-networking'),
> (u'meta-oe',
> u'meta-oe')]}
>2015-09-02 10:11:54,416 DEBUG lbc_shellcmmd: (/home/sujith/Documents) git
>remote -v
>2015-09-02 10:11:54,419 WARNING localhostbecontroller: shellcmd error
>command: git remote -v
>
>fatal: Not a git repository (or any of the parent directories): .git
>
>2015-09-02 10:11:54,419 DEBUG lbc_shellcmmd: (/home/sujith/.thumbnails)
>git remote -v
>2015-09-02 10:11:54,422 WARNING localhostbecontroller: shellcmd error
>command: git remote -v
>
>fatal: Not a git repository (or any of the parent directories): .git
>
>2015-09-02 10:11:54,422 DEBUG lbc_shellcmmd: (/home/sujith/.emacs.d) git
>remote -v
>2015-09-02 10:11:54,425 WARNING localhostbecontroller: shellcmd error
>command: git remote -v
>
>fatal: Not a git repository (or any of the parent directories): .git
>
>2015-09-02 10:11:54,425 DEBUG lbc_shellcmmd: (/home/sujith/Templates) git
>remote -v
>2015-09-02 10:11:54,429 WARNING localhostbecontroller: shellcmd error
>command: git remote -v
>
>fatal: Not a git repository (or any of the parent directories): .git
>
>2015-09-02 10:11:54,429 DEBUG lbc_shellcmmd: (/home/sujith/.pylint.d) git
>remote -v
>2015-09-02 10:11:54,432 WARNING localhostbecontroller: shellcmd error
>command: git remote -v
>
>fatal: Not a git repository (or any of the parent directories): .git
>
>2015-09-02 10:11:54,432 DEBUG lbc_shellcmmd: (/home/sujith/.cscache) git
>remote -v
>2015-09-02 10:11:54,439 WARNING localhostbecontroller: shellcmd error
>command: git remote -v
>
>fatal: Not a git repository (or any of the parent directories): .git
>
>2015-09-02 10:11:54,439 DEBUG lbc_shellcmmd: (/home/sujith/.config) git
>remote -v
>2015-09-02 10:11:54,443 WARNING localhostbecontroller: shellcmd error
>command: git remote -v
>
>fatal: Not a git repository (or any of the parent directories): .git
>
>2015-09-02 10:11:54,443 DEBUG lbc_shellcmmd: (/home/sujith/bin) git
>remote -v
>2015-09-02 10:11:54,448 WARNING localhostbecontroller: shellcmd error
>command: git remote -v
>
>fatal: Not a git repository (or any of the parent directories): .git
>
>2015-09-02 10:11:54,448 DEBUG lbc_shellcmmd: (/home/sujith/MEL) git
>remote -v
>2015-09-02 10:11:54,452 WARNING localhostbecontroller: shellcmd error
>command: git remote -v
>
>fatal: Not a git repository (or any of the parent directories): .git
>
>2015-09-02 10:11:54,452 DEBUG lbc_shellcmmd: (/home/sujith/Pictures) git
>remote -v
>2015-09-02 10:11:54,455 WARNING localhostbecontroller: shellcmd error
>command: git remote -v
>
>fatal: Not a git repository (or any of the parent directories): .git
>
>2015-09-02 10:11:54,455 DEBUG lbc_shellcmmd: (/home/sujith/.pip) git
>remote -v
>2015-09-02 10:11:54,459 WARNING localhostbecontroller: shellcmd error
>command: git remote -v
>
>fatal: Not a git repository (or any of the parent directories): .git
>
>2015-09-02 10:11:54,459 DEBUG lbc_shellcmmd: (/home/sujith/Music) git
>remote -v
>2015-09-02 10:11:54,463 WARNING localhostbecontroller: shellcmd error
>command: git remote -v
>
>fatal: Not a git repository (or any of the parent directories): .git
>
>2015-09-02 10:11:54,463 DEBUG lbc_shellcmmd: (/home/sujith/debian) git
>remote -v
>2015-09-02 10:11:54,466 WARNING localhostbecontroller: shellcmd error
>command: git remote -v
>
>fatal: Not a git repository (or any of the parent directories): .git
>
>2015-09-02 10:11:54,466 DEBUG lbc_shellcmmd: (/home/sujith/.distlib) git
>remote -v
>2015-09-02 10:11:54,470 WARNING localhostbecontroller: shellcmd error
>command: git remote -v
>
>fatal: Not a git repository (or any of the parent directories): .git
>
>2015-09-02 10:11:54,470 DEBUG lbc_shellcmmd: (/home/sujith/.mozilla) git
>remote -v
>2015-09-02 10:11:54,473 WARNING localhostbecontroller: shellcmd error
>command: git remote -v
>
>fatal: Not a git repository (or any of the parent directories): .git
>
>2015-09-02 10:11:54,474 DEBUG lbc_shellcmmd: (/home/sujith/Django_study)
>git remote -v
>2015-09-02 10:11:54,478 WARNING localhostbecontroller: shellcmd error
>command: git remote -v
>
>fatal: Not a git repository (or any of the parent directories): .git
>
>2015-09-02 10:11:54,478 DEBUG lbc_shellcmmd: (/home/sujith/.java) git
>remote -v
>2015-09-02 10:11:54,481 WARNING localhostbecontroller: shellcmd error
>command: git remote -v
>
>fatal: Not a git repository (or any of the parent directories): .git
>
>2015-09-02 10:11:54,482 DEBUG lbc_shellcmmd:
>(/home/sujith/patches_to_push) git remote -v
>2015-09-02 10:11:54,485 WARNING localhostbecontroller: shellcmd error
>command: git remote -v
>
>fatal: Not a git repository (or any of the parent directories): .git
>
>2015-09-02 10:11:54,485 DEBUG lbc_shellcmmd: (/home/sujith/.ssh) git
>remote -v
>2015-09-02 10:11:54,488 WARNING localhostbecontroller: shellcmd error
>command: git remote -v
>
>fatal: Not a git repository (or any of the parent directories): .git
>
>2015-09-02 10:11:54,488 DEBUG lbc_shellcmmd: (/home/sujith/Downloads) git
>remote -v
>2015-09-02 10:11:54,492 WARNING localhostbecontroller: shellcmd error
>command: git remote -v
>
>fatal: Not a git repository (or any of the parent directories): .git
>
>2015-09-02 10:11:54,492 DEBUG lbc_shellcmmd: (/home/sujith/.pki) git
>remote -v
>2015-09-02 10:11:54,495 WARNING localhostbecontroller: shellcmd error
>command: git remote -v
>
>fatal: Not a git repository (or any of the parent directories): .git
>
>2015-09-02 10:11:54,496 DEBUG lbc_shellcmmd: (/home/sujith/Public) git
>remote -v
>2015-09-02 10:11:54,499 WARNING localhostbecontroller: shellcmd error
>command: git remote -v
>
>fatal: Not a git repository (or any of the parent directories): .git
>
>2015-09-02 10:11:54,499 DEBUG lbc_shellcmmd: (/home/sujith/.repoconfig)
>git remote -v
>2015-09-02 10:11:54,502 WARNING localhostbecontroller: shellcmd error
>command: git remote -v
>
>fatal: Not a git repository (or any of the parent directories): .git
>
>2015-09-02 10:11:54,502 DEBUG lbc_shellcmmd: (/home/sujith/.pc) git
>remote -v
>2015-09-02 10:11:54,505 WARNING localhostbecontroller: shellcmd error
>command: git remote -v
>
>fatal: Not a git repository (or any of the parent directories): .git
>
>2015-09-02 10:11:54,505 DEBUG lbc_shellcmmd: (/home/sujith/wsgiref) git
>remote -v
>2015-09-02 10:11:54,510 WARNING localhostbecontroller: shellcmd error
>command: git remote -v
>
>fatal: Not a git repository (or any of the parent directories): .git
>
>2015-09-02 10:11:54,510 DEBUG lbc_shellcmmd: (/home/sujith/.mentor) git
>remote -v
>2015-09-02 10:11:54,513 WARNING localhostbecontroller: shellcmd error
>command: git remote -v
>
>fatal: Not a git repository (or any of the parent directories): .git
>
>2015-09-02 10:11:54,515 DEBUG lbc_shellcmmd:
>(/home/sujith/MENTOR_TOOLCHAIN) git remote -v
>2015-09-02 10:11:54,517 WARNING localhostbecontroller: shellcmd error
>command: git remote -v
>
>fatal: Not a git repository (or any of the parent directories): .git
>
>2015-09-02 10:11:54,518 DEBUG lbc_shellcmmd: (/home/sujith/Desktop) git
>remote -v
>2015-09-02 10:11:54,521 WARNING localhostbecontroller: shellcmd error
>command: git remote -v
>
>fatal: Not a git repository (or any of the parent directories): .git
>
>2015-09-02 10:11:54,521 DEBUG lbc_shellcmmd: (/home/sujith/.local) git
>remote -v
>2015-09-02 10:11:54,525 WARNING localhostbecontroller: shellcmd error
>command: git remote -v
>
>fatal: Not a git repository (or any of the parent directories): .git
>
>2015-09-02 10:11:54,525 DEBUG lbc_shellcmmd: (/home/sujith/.dbus) git
>remote -v
>2015-09-02 10:11:54,529 WARNING localhostbecontroller: shellcmd error
>command: git remote -v
>
>fatal: Not a git repository (or any of the parent directories): .git
>
>2015-09-02 10:11:54,529 DEBUG lbc_shellcmmd: (/home/sujith/workspace) git
>remote -v
>2015-09-02 10:11:54,532 WARNING localhostbecontroller: shellcmd error
>command: git remote -v
>
>fatal: Not a git repository (or any of the parent directories): .git
>
>2015-09-02 10:11:54,533 DEBUG lbc_shellcmmd:
>(/home/sujith/.gstreamer-0.10) git remote -v
>2015-09-02 10:11:54,536 WARNING localhostbecontroller: shellcmd error
>command: git remote -v
>
>fatal: Not a git repository (or any of the parent directories): .git
>
>2015-09-02 10:11:54,536 DEBUG lbc_shellcmmd: (/home/sujith/.gnome) git
>remote -v
>2015-09-02 10:11:54,540 WARNING localhostbecontroller: shellcmd error
>command: git remote -v
>
>fatal: Not a git repository (or any of the parent directories): .git
>
>2015-09-02 10:11:54,540 DEBUG lbc_shellcmmd: (/home/sujith/.cache) git
>remote -v
>2015-09-02 10:11:54,544 WARNING localhostbecontroller: shellcmd error
>command: git remote -v
>
>fatal: Not a git repository (or any of the parent directories): .git
>
>2015-09-02 10:11:54,544 DEBUG lbc_shellcmmd:
>(/home/sujith/codebench-linux-install-2014.11-96-arm) git remote -v
>2015-09-02 10:11:54,547 WARNING localhostbecontroller: shellcmd error
>command: git remote -v
>
>fatal: Not a git repository (or any of the parent directories): .git
>
>2015-09-02 10:11:54,547 DEBUG lbc_shellcmmd: (/home/sujith/Videos) git
>remote -v
>2015-09-02 10:11:54,550 WARNING localhostbecontroller: shellcmd error
>command: git remote -v
>
>fatal: Not a git repository (or any of the parent directories): .git
>
>2015-09-02 10:11:54,550 DEBUG lbc_shellcmmd: (/home/sujith/.gconf) git
>remote -v
>2015-09-02 10:11:54,554 WARNING localhostbecontroller: shellcmd error
>command: git remote -v
>
>fatal: Not a git repository (or any of the parent directories): .git
>
>2015-09-02 10:11:54,555 DEBUG lbc_shellcmmd: (/home/sujith/.subversion)
>git remote -v
>2015-09-02 10:11:54,558 WARNING localhostbecontroller: shellcmd error
>command: git remote -v
>
>fatal: Not a git repository (or any of the parent directories): .git
>
>2015-09-02 10:11:54,558 DEBUG lbc_shellcmmd: (/home/sujith/.thunderbird)
>git remote -v
>2015-09-02 10:11:54,561 WARNING localhostbecontroller: shellcmd error
>command: git remote -v
>
>fatal: Not a git repository (or any of the parent directories): .git
>
>2015-09-02 10:11:54,561 DEBUG lbc_shellcmmd: (/home/sujith/gitroot) git
>remote -v
>2015-09-02 10:11:54,564 WARNING localhostbecontroller: shellcmd error
>command: git remote -v
>
>fatal: Not a git repository (or any of the parent directories): .git
>
>2015-09-02 10:11:54,564 DEBUG lbc_shellcmmd: (/home/sujith/.oe) git
>remote -v
>2015-09-02 10:11:54,568 WARNING localhostbecontroller: shellcmd error
>command: git remote -v
>
>fatal: Not a git repository (or any of the parent directories): .git
>
>2015-09-02 10:11:54,570 DEBUG localhostbecontroller: current layer list
>[u'/home/sujith/MEL/test_poky/poky/meta',
> u'/home/sujith/MEL/test_poky/poky/meta-yocto',
> u'/home/sujith/MEL/test_poky/poky/meta-yocto-bsp',
> u'/home/sujith/MEL/cloned_layers/meta-oe/meta-python',
> u'/home/sujith/MEL/cloned_layers/meta-oe/meta-multimedia',
> u'/home/sujith/MEL/cloned_layers/meta-oe/meta-filesystems',
> u'/home/sujith/MEL/cloned_layers/meta-oe/meta-networking',
> u'/home/sujith/MEL/cloned_layers/meta-oe/meta-oe',
> u'/home/sujith/MEL/cloned_layers/meta-qt5']
>2015-09-02 10:11:54,571 DEBUG lbc_shellcmmd: (/home/sujith) bash -c
>"source /home/sujith/MEL/test_poky/poky/oe-init-build-env
>/home/sujith/MEL/test_poky/build"
>2015-09-02 10:11:54,684 DEBUG localhostbecontroller: shellcmd success
>2015-09-02 10:11:54,691 DEBUG localhostbecontroller: running the listener
>at /home/sujith/MEL/test_poky/poky/bitbake/bin/bitbake
>2015-09-02 10:11:54,691 DEBUG localhostbecontroller: starting builder
>bash -c "source /home/sujith/MEL/test_poky/poky/oe-init-build-env
>/home/sujith/MEL/test_poky/build 2>&1 >toaster_server.log && bitbake
>--read /home/sujith/MEL/test_poky/build/conf/toaster-pre.conf --postread
>/home/sujith/MEL/test_poky/build/conf/toaster.conf
> --server-only -t xmlrpc -B 0.0.0.0:0 <http://0.0.0.0:0> 2>&1
>>>toaster_server.log "
>
>2015-09-02 10:11:54,691 DEBUG lbc_shellcmmd: (/home/sujith) bash -c
>"source /home/sujith/MEL/test_poky/poky/oe-init-build-env
>/home/sujith/MEL/test_poky/build 2>&1 >toaster_server.log && bitbake
>--read /home/sujith/MEL/test_poky/build/conf/toaster-pre.conf
> --postread /home/sujith/MEL/test_poky/build/conf/toaster.conf
>--server-only -t xmlrpc -B
>0.0.0.0:0 <http://0.0.0.0:0> 2>&1 >>toaster_server.log "
>2015-09-02 10:11:56,198 DEBUG localhostbecontroller: shellcmd success
>2015-09-02 10:11:56,199 DEBUG localhostbecontroller: Found bitbake server
>port 33000
>
>2015-09-02 10:11:56,200 DEBUG localhostbecontroller: Waiting bitbake
>server to start
>2015-09-02 10:11:56,701 DEBUG localhostbecontroller: Waiting bitbake
>server to start
>2015-09-02 10:11:57,202 DEBUG localhostbecontroller: Waiting bitbake
>server to start
>2015-09-02 10:11:57,704 DEBUG localhostbecontroller: Waiting bitbake
>server to start
>2015-09-02 10:11:58,205 DEBUG localhostbecontroller: Started bitbake
>server
>2015-09-02 10:11:58,236 DEBUG localhostbecontroller: Build launched,
>exiting. Follow build logs at
>/home/sujith/MEL/test_poky/build/toaster_ui.log
>------------
>
>
>
>The path to tmp/deploy/image and the content inside the directory:
>
>------------
>sujith at kdekidd0:~/MEL/test_poky/build$ ls tmp/deploy/images/qemux86/ -lt
>total 1244112
>lrwxrwxrwx 1 sujith sujith 53 Sep 2 13:43
>core-image-sato-qemux86.tar.bz2 ->
>core-image-sato-qemux86-20150902044346.rootfs.tar.bz2
>lrwxrwxrwx 1 sujith sujith 50 Sep 2 13:43
>core-image-sato-qemux86.ext3 ->
>core-image-sato-qemux86-20150902044346.rootfs.ext3
>lrwxrwxrwx 1 sujith sujith 50 Sep 2 13:43
>core-image-sato-qemux86.ext4 ->
>core-image-sato-qemux86-20150902044346.rootfs.ext4
>lrwxrwxrwx 1 sujith sujith 54 Sep 2 13:43
>core-image-sato-qemux86.manifest ->
>core-image-sato-qemux86-20150902044346.rootfs.manifest
>lrwxrwxrwx 1 sujith sujith 51 Sep 2 13:43
>core-image-sato-qemux86.jffs2 ->
>core-image-sato-qemux86-20150902044346.rootfs.jffs2
>-rw-r--r-- 1 sujith sujith 196083712 Sep 2 13:43
>core-image-sato-qemux86-20150902044346.rootfs.jffs2
>-rw-r--r-- 1 sujith sujith 535930880 Sep 2 13:42
>core-image-sato-qemux86-20150902044346.rootfs.ext4
>-rw-r--r-- 1 sujith sujith 535930880 Sep 2 13:42
>core-image-sato-qemux86-20150902044346.rootfs.ext3
>-rw-r--r-- 1 sujith sujith 154285168 Sep 2 13:42
>core-image-sato-qemux86-20150902044346.rootfs.tar.bz2
>-rw-r--r-- 1 sujith sujith 21112 Sep 2 13:41
>core-image-sato-qemux86-20150902044346.rootfs.manifest
>-rw-r--r-- 2 sujith sujith 294 Sep 2 13:40
>README_-_DO_NOT_DELETE_FILES_IN_THIS_DIRECTORY.txt
>lrwxrwxrwx 1 sujith sujith 71 Sep 2 12:53 bzImage ->
>bzImage--4.1.6+git0+429f9e2ff0_59b8c4f5e8-r0-qemux86-20150902044346.bin
>lrwxrwxrwx 1 sujith sujith 71 Sep 2 12:53 bzImage-qemux86.bin ->
>bzImage--4.1.6+git0+429f9e2ff0_59b8c4f5e8-r0-qemux86-20150902044346.bin
>lrwxrwxrwx 1 sujith sujith 71 Sep 2 12:53 modules-qemux86.tgz ->
>modules--4.1.6+git0+429f9e2ff0_59b8c4f5e8-r0-qemux86-20150902044346.tgz
>-rw-rw-r-- 2 sujith sujith 83308568 Sep 2 12:53
>modules--4.1.6+git0+429f9e2ff0_59b8c4f5e8-r0-qemux86-20150902044346.tgz
>-rw-r--r-- 2 sujith sujith 6957488 Sep 2 12:53
>bzImage--4.1.6+git0+429f9e2ff0_59b8c4f5e8-r0-qemux86-20150902044346.bin
>sujith at kdekidd0:~/MEL/test_poky/build$
>------------
>
>
>The screenshot of image built successfully:
>http://i.imgur.com/V3qtXFh.png <http://i.imgur.com/V3qtXFh.png>
>
>
>---
>So, you may want to look at/try my poky-contrib branch and look
>closely at my toasterconf.json file to see if you can tell why you do
>not see these issues.
>
>
>
>
> The toaster-pre.conf of mine looks as:
>sujith at kdekidd0:~/MEL/test_poky/build/conf$ cat toaster-pre.conf
>IMAGE_INSTALL_append=" qtbase iw"
>PACKAGE_CLASSES="package_ipk"
>MACHINE="qemux86"
>SDKMACHINE="x86_64"
>DISTRO="poky"
>IMAGE_FSTYPES="ext3 jffs2 tar.bz2"
>TOASTER_BRBE="1:1"
>sujith at kdekidd0:~/MEL/test_poky/build/conf$
>
>
>Somehow I am not able to figure out what is going on wrong at your end :(
>
>
>
>-brian
>
>On Sun, Aug 30, 2015 at 11:47 PM, sujith h <sujith.h at gmail.com> wrote:
>> Hi Brian,
>>
>> On Sat, Aug 29, 2015 at 1:13 AM, Brian Avery <avery.brian at gmail.com>
>>wrote:
>>>
>>> Hi Sujith,
>>>
>>> The new toasterconf does make it easier to test, thanks! I've
>>> attached mine so you can see what I did in case that caused any of the
>>> issues below.
>>> I changed:
>>> 1) directory paths
>>> 2) you had "branches": ["mel", "yocto", "HEAD"], which i
>>> changed to "branches": ["HEAD", "master", "fido",
>>> "dizzy"],
>>>
>>>
>>>
>>> So here is my directory layout
>>>
>>> /big/src/intel/myBugs2
>>> cloned_layers/
>>> meta-oe/
>>> meta-qt5/
>>> sujith/
>>> poky
>>
>>
>> Here is my directory layout:
>> /home/sujith/MEL/test_poky
>> poky
>> /home/sujith/MEL/cloned_layers
>> meta-oe
>> meta-qt5
>>
>>>
>>>
>>> I put the attached toasterconf.json into the sujith/meta-yocto/conf/
>>> directory (overwriting the default one) and toaster found it on
>>> startup.
>>
>>
>> I have added my configuration into:
>> /home/sujith/MEL/test_poky/toasterconf.json
>>
>> I never tried over writing toasterconf.json file in meta-yocto/conf
>>
>> I use option [0] while running poky/bitbake/bin/toaster i.e, exit
>>without
>> importing any layers. Then I manually try to import it using command:
>> poky/bitbake/lib/toaster/manage.py loadconf toasterconf.json
>>
>>> Question:
>>> Where are you putting your toasterconf.json file? I haven't gone
>>> through the search logic yet but toaster seemed to have trouble
>>> finding it in other directories.
>>>
>>> My directory settings for starting toaster are:
>>> build log -> /home/bavery/toaster_build_artifacts
>>> layer checkout -> /big/src/intel/myBugs2/sujith/
>>> build -> /home/bavery/toaster_build_artifacts
>>>
>>>
>>> I found the following issues:
>>> 1)On startup toaster complained about not being able to load from
>>> openembedded.layers.org <http://openembedded.layers.org> and once
>>>toaster was up, there were no layers
>>> available except the local ones.
>>>
>>>
>>>(https://www.dropbox.com/s/jdkayha71l8i9fe/Screenshot%202015-08-28%2012.
>>>29.06.png?dl=0)
>>> 2) I started a build of core-image-sato with MAGE_INSTALL_append =
>>> "qt5 iws"
>>>
>>>(https://www.dropbox.com/s/m3l6uyxq6ptr5vv/Screenshot%202015-08-28%2012.
>>>32.16.png?dl=0)
>>> and the build no longer shows progress on the progress bar. top
>>> running shows that I am well into the build but that is not reflected
>>> in the UI
>>>
>>>(https://www.dropbox.com/s/x3cl2whp2twt8xm/Screenshot%202015-08-28%2012.
>>>32.44.png?dl=0)
>>> 3) the toaster_ui.log in the build directory is huge and is filled
>>> with things like :Cannot determine the vcs_reference for layer version
>>> {'build_id': 1, '_layer_cache': <Layer: __FIXME__unidentified_layer /
>>> None >, 'layer_id': 204, '_state': <django.db.models.base.ModelSt...."
>>>
>>>
>>>
>>> Do you see any of these?
>>
>>
>> Since I haven't tried overwriting toasters default configuration file, I
>> couldn't hit with any of these errors. Would you mind to try with my
>>way, if
>> you don't mind?
>> Let me know if you face any issue with the procedure I followed.
>>
>>
>> Thanks,
>> Sujith H
>>>
>>>
>>> -b
>>>
>>> On Fri, Aug 28, 2015 at 12:06 AM, sujith h <sujith.h at gmail.com> wrote:
>>> >
>>> >
>>> > On Thu, Aug 27, 2015 at 9:56 PM, sujith h <sujith.h at gmail.com> wrote:
>>> >>
>>> >>
>>> >>
>>> >> On Thu, Aug 27, 2015 at 9:36 PM, Brian Avery <avery.brian at gmail.com>
>>> >> wrote:
>>> >>>
>>> >>> Hi Sujith,
>>> >>>
>>> >>> I'd like to test this out. Is it possible to get a config file
>>>that
>>> >>> exercises the new features?
>>> >>
>>> >>
>>> >> I have attached the toaster configuration file which I used for
>>> >> testing.
>>> >> What you can also do is remove meta-mentor,meta-sourcery and
>>>meta-mx6q
>>> >> layers from the
>>> >> configuration file. And try with the layers we have. That might help
>>> >> you
>>> >> in testing. Also you
>>> >> may have to remove some additional settings I have used in the
>>> >> configuration, like Distro as "mel",
>>> >> toolchain path etc
>>> >>
>>> >> Or the simplest testing would be by adding poky and meta-oe layer.
>>>Then
>>> >> try to build
>>> >> some recipe from meta-oe. That would be the simplest test.
>>> >>
>>> >> Let me know if you need more information or face any issues. I would
>>> >> help
>>> >> you with that.
>>> >
>>> >
>>> > Hi Brian,
>>> >
>>> > I have attached a new configuration file which I am testing right
>>>away.
>>> > The
>>> > build is in progress.
>>> > In the IMAGE_INSTALL_append ( from UI ), I have added qtbase and iw
>>> > packages. The 2 repos I have
>>> > cloned is:
>>> > git clone git://github.com/meta-qt5/meta-qt5.git
>>><http://github.com/meta-qt5/meta-qt5.git>
>>> > and
>>> > git clone git://github.com/openembedded/meta-oe.git
>>><http://github.com/openembedded/meta-oe.git>
>>> >
>>> > Accordingly adjust your paths mentioned in the configuration file.
>>> >
>>> > I hope this configuration file should help you proceed with your
>>>test.
>>> > Let
>>> > me know if you face any issues.
>>> >
>>> > Thanks,
>>> > Sujith H
>>> >>
>>> >>
>>> >> Thanks,
>>> >> Sujith H
>>> >>>
>>> >>>
>>> >>> -b
>>> >>>
>>> >>>
>>> >>> On Mon, Aug 24, 2015 at 7:16 PM, sujith h <sujith.h at gmail.com>
>>>wrote:
>>> >>> > On 24 Aug 2015 6:49 pm, "sujith h" <sujith.h at gmail.com> wrote:
>>> >>> >>
>>> >>> >>
>>> >>> >>
>>> >>> >> On Fri, Aug 21, 2015 at 8:07 PM, Damian, Alexandru
>>> >>> >> <alexandru.damian at intel.com> wrote:
>>> >>> >>>
>>> >>> >>>
>>> >>> >>>
>>> >>> >>> On Fri, Aug 21, 2015 at 1:52 PM, sujith h <sujith.h at gmail.com>
>>> >>> >>> wrote:
>>> >>> >>>>
>>> >>> >>>>
>>> >>> >>>>
>>> >>> >>>> On Fri, Aug 21, 2015 at 4:45 PM, Damian, Alexandru
>>> >>> >>>> <alexandru.damian at intel.com> wrote:
>>> >>> >>>>>
>>> >>> >>>>> Hi,
>>> >>> >>>>>
>>> >>> >>>>> I think we need a better patch description. If I understand
>>> >>> >>>>> correctly,
>>> >>> >>>>> the issue you were facing is:
>>> >>> >>>>> - even if we have layers at HEAD that are not under the poky/
>>> >>> >>>>> tree,
>>> >>> >>>>> the
>>> >>> >>>>> code would always return the
>>> >>> >>>>> checkout path in the poky/ tree; I recognize that this is a
>>> >>> >>>>> valid
>>> >>> >>>>> issue.
>>> >>> >>>>>
>>> >>> >>>>> I feel that this issue can be handled a bit differently if we
>>> >>> >>>>> frame
>>> >>> >>>>> it
>>> >>> >>>>> as such:
>>> >>> >>>>> - if we have a HEAD branch specified for a layer, we need to
>>> >>> >>>>> find
>>> >>> >>>>> out
>>> >>> >>>>> the current checkout directory for that layer, since no
>>>checkout
>>> >>> >>>>> would be
>>> >>> >>>>> performed.
>>> >>> >>>>>
>>> >>> >>>>> As such, when we import a toasterconf.json file (they only
>>>way
>>> >>> >>>>> to
>>> >>> >>>>> get
>>> >>> >>>>> HEAD branches for a layer) we need to store the actual layer
>>> >>> >>>>> checkout path
>>> >>> >>>>> in the database (in the localpath, perhaps?), and reuse that
>>> >>> >>>>> value,
>>> >>> >>>>> instead
>>> >>> >>>>> of employing special logic to search for it. What I mean, the
>>> >>> >>>>> core
>>> >>> >>>>> of the
>>> >>> >>>>> problem is solved by replacing the whole gitrepos checkout
>>>logic
>>> >>> >>>>> with a
>>> >>> >>>>> database value for all "HEAD" layers, and eliminate the
>>>magic in
>>> >>> >>>>> getGitCloneDirectory().
>>> >>> >>>>>
>>> >>> >>>>> I have a couple more comments on the patch below.
>>> >>> >>>>>
>>> >>> >>>>> Cheers,
>>> >>> >>>>> Alex
>>> >>> >>>>>
>>> >>> >>>>> On Fri, Aug 21, 2015 at 9:41 AM, Sujith H
>>><sujith.h at gmail.com>
>>> >>> >>>>> wrote:
>>> >>> >>>>>>
>>> >>> >>>>>> This patch fixes import of custom layers from
>>>toasterjson.conf
>>> >>> >>>>>> file. For example it was verified using layers like
>>> >>> >>>>>> meta-mentor,
>>> >>> >>>>>> meta-fsl-arm, meta-sourcery etc.
>>> >>> >>>>>>
>>> >>> >>>>>> Signed-off-by: Sujith Haridasan
>>><Sujith_Haridasan at mentor.com>
>>> >>> >>>>>> ---
>>> >>> >>>>>> .../toaster/bldcontrol/localhostbecontroller.py | 49
>>> >>> >>>>>> ++++++++++++++++++++--
>>> >>> >>>>>> .../bldcontrol/management/commands/loadconf.py | 12
>>>++++--
>>> >>> >>>>>> 2 files changed, 54 insertions(+), 7 deletions(-)
>>> >>> >>>>>>
>>> >>> >>>>>> diff --git
>>> >>> >>>>>> a/bitbake/lib/toaster/bldcontrol/localhostbecontroller.py
>>> >>> >>>>>> b/bitbake/lib/toaster/bldcontrol/localhostbecontroller.py
>>> >>> >>>>>> index 231a7d3..c43f33f 100644
>>> >>> >>>>>> ---
>>>a/bitbake/lib/toaster/bldcontrol/localhostbecontroller.py
>>> >>> >>>>>> +++
>>>b/bitbake/lib/toaster/bldcontrol/localhostbecontroller.py
>>> >>> >>>>>> @@ -178,7 +178,7 @@ class
>>> >>> >>>>>> LocalhostBEController(BuildEnvironmentController):
>>> >>> >>>>>> self.be.save()
>>> >>> >>>>>> logger.debug("localhostbecontroller: Stopped
>>>bitbake
>>> >>> >>>>>> server")
>>> >>> >>>>>>
>>> >>> >>>>>> - def getGitCloneDirectory(self, url, branch):
>>> >>> >>>>>> + def
>>> >>> >>>>>>
>>> >>> >>>>>> getGitCloneDirectory(self, url, branch, cached_layers=None):
>>> >>> >>>>>> """ Utility that returns the last component of a
>>>git
>>> >>> >>>>>> path
>>> >>> >>>>>> as
>>> >>> >>>>>> directory
>>> >>> >>>>>> """
>>> >>> >>>>>> import re
>>> >>> >>>>>> @@ -191,6 +191,17 @@ class
>>> >>> >>>>>> LocalhostBEController(BuildEnvironmentController):
>>> >>> >>>>>>
>>> >>> >>>>>> # word of attention; this is a localhost-specific
>>> >>> >>>>>> issue;
>>> >>> >>>>>> only
>>> >>> >>>>>> on the localhost we expect to have "HEAD" releases
>>> >>> >>>>>> # which _ALWAYS_ means the current poky checkout
>>> >>> >>>>>> + if cached_layers:
>>> >>> >>>>>
>>> >>> >>>>>
>>> >>> >>>>> Using cached_layers here works only by coincidence - if the
>>> >>> >>>>> desired
>>> >>> >>>>> layer is not manually checked out under be.sourcedir, it will
>>> >>> >>>>> not
>>> >>> >>>>> be found
>>> >>> >>>>> by cached_layers code, and will not appear here.
>>> >>> >>>>
>>> >>> >>>>
>>> >>> >>>> Ah, so would it be good to read from the database? Because I
>>>was
>>> >>> >>>> giving
>>> >>> >>>> full path of layers in the toasterconf.json file. I assume
>>>that
>>> >>> >>>> layer
>>> >>> >>>> information from the toasterconf.json file would be updated in
>>> >>> >>>> the
>>> >>> >>>> database?
>>> >>> >>>
>>> >>> >>>
>>> >>> >>> Would be inserted into the database as the file is read.
>>> >>> >>>
>>> >>> >>>>>
>>> >>> >>>>>
>>> >>> >>>>> I think a completely new piece of code is needed, which will
>>> >>> >>>>> track
>>> >>> >>>>> where any layer that has a HEAD branch is actually checked
>>>out.
>>> >>> >>>>> This path
>>> >>> >>>>> would be discovered and stored by importing a
>>>toasterconf.json
>>> >>> >>>>> file
>>> >>> >>>>> (the
>>> >>> >>>>> only place that can generate a HEAD branch), and verified to
>>> >>> >>>>> actually exist
>>> >>> >>>>> at build time in this function, getGitCloneDirectory(). This
>>> >>> >>>>> patch
>>> >>> >>>>> touches
>>> >>> >>>>> loadconf, but I think we might be able to get a better
>>>handling
>>> >>> >>>>> of
>>> >>> >>>>> that.
>>> >>> >>>>>
>>> >>> >>>>>
>>> >>> >>>>>> + if not url.endswith('.git'):
>>> >>> >>>>>
>>> >>> >>>>>
>>> >>> >>>>> I am not sure why the actual GIT url format is relevant for
>>> >>> >>>>> "HEAD"
>>> >>> >>>>> branches; we don't need to figure out where the layer would
>>>get
>>> >>> >>>>> checked out,
>>> >>> >>>>> but where it actually exists.
>>> >>> >>>>>
>>> >>> >>>>>
>>> >>> >>>>>> + from os.path import dirname
>>> >>> >>>>>> + extractDir = dirname(url)
>>> >>> >>>>>> + for key, val in cached_layers.iteritems():
>>> >>> >>>>>> + if val == extractDir:
>>> >>> >>>>>> + local_checkout_path = key
>>> >>> >>>>>> + return cached_layers[local_checkout_path]
>>> >>> >>>>>> + else:
>>> >>> >>>>>> + local_checkout_path = cached_layers[url]
>>> >>> >>>>>> + return local_checkout_path
>>> >>> >>>>>> from os.path import dirname as DN
>>> >>> >>>>>> local_checkout_path =
>>> >>> >>>>>> DN(DN(DN(DN(DN(os.path.abspath(__file__))))))
>>> >>> >>>>>> #logger.debug("localhostbecontroller: using HEAD
>>> >>> >>>>>> checkout
>>> >>> >>>>>> in
>>> >>> >>>>>> %s" % local_checkout_path)
>>> >>> >>>>>> @@ -245,7 +256,14 @@ class
>>> >>> >>>>>> LocalhostBEController(BuildEnvironmentController):
>>> >>> >>>>>>
>>> >>> >>>>>> # 3. checkout the repositories
>>> >>> >>>>>> for giturl, commit in gitrepos.keys():
>>> >>> >>>>>> - localdirname = os.path.join(self.be.sourcedir,
>>> >>> >>>>>> self.getGitCloneDirectory(giturl, commit))
>>> >>> >>>>>> + if not giturl.endswith('.git'):
>>> >>> >>>>>> + for key, val in cached_layers.iteritems():
>>> >>> >>>>>> + if os.path.dirname(giturl) == val:
>>> >>> >>>>>> + tmpKey = key
>>> >>> >>>>>> + tmpVal = gitrepos[(giturl, commit)]
>>> >>> >>>>>> + gitrepos[(tmpKey, commit)] += tmpVal
>>> >>> >>>>>> + giturl = tmpKey
>>> >>> >>>>>> + localdirname = os.path.join(self.be.sourcedir,
>>> >>> >>>>>> self.getGitCloneDirectory(giturl, commit, cached_layers))
>>> >>> >>>>>> logger.debug("localhostbecontroller: giturl
>>>%s:%s
>>> >>> >>>>>> checking out in current directory %s" % (giturl, commit,
>>> >>> >>>>>> localdirname))
>>> >>> >>>>>>
>>> >>> >>>>>> # make sure our directory is a git repository
>>> >>> >>>>>> @@ -280,13 +298,36 @@ class
>>> >>> >>>>>> LocalhostBEController(BuildEnvironmentController):
>>> >>> >>>>>>
>>> >>> >>>>>> # verify our repositories
>>> >>> >>>>>> for name, dirpath in gitrepos[(giturl,
>>>commit)]:
>>> >>> >>>>>> - localdirpath = os.path.join(localdirname,
>>> >>> >>>>>> dirpath)
>>> >>> >>>>>
>>> >>> >>>>>
>>> >>> >>>>> I am not sure what this change tries to do; dirpath is
>>>actually
>>> >>> >>>>> a
>>> >>> >>>>> path
>>> >>> >>>>> in the git repo where the layer lives; it is not immediate
>>>that
>>> >>> >>>>> the
>>> >>> >>>>> all the
>>> >>> >>>>> layers live in the top directory of a git repo. This change
>>> >>> >>>>> invalidates the
>>> >>> >>>>> check below, where we verify that the layer path actually
>>>exists
>>> >>> >>>>> after
>>> >>> >>>>> checkout.
>>> >>> >>>>>
>>> >>> >>>>>>
>>> >>> >>>>>> + localdirpath = localdirname
>>> >>> >>>>>>
>>> >>> >>>>>> logger.debug("localhostbecontroller:
>>> >>> >>>>>> localdirpath
>>> >>> >>>>>> expected '%s'" % localdirpath)
>>> >>> >>>>>> if not os.path.exists(localdirpath):
>>> >>> >>>>>> raise BuildSetupException("Cannot find
>>> >>> >>>>>> layer
>>> >>> >>>>>> git
>>> >>> >>>>>> path '%s' in checked out repository '%s:%s'. Aborting." %
>>> >>> >>>>>> (localdirpath,
>>> >>> >>>>>> giturl, commit))
>>> >>> >>>>>>
>>> >>> >>>>>> + layerlisturl, layerlistdir = "", ""
>>> >>> >>>>>> + layerlisturl = [localurl for localurl,
>>> >>> >>>>>> localpath
>>> >>> >>>>>> in
>>> >>> >>>>>> cached_layers.iteritems() if localpath == localdirpath]
>>> >>> >>>>>
>>> >>> >>>>>
>>> >>> >>>>> I suspect this logic will go away if we're not going to use
>>> >>> >>>>> cached_layers to discover the local checkout directories for
>>> >>> >>>>> HEAD
>>> >>> >>>>> urls.
>>> >>> >>>>>
>>> >>> >>>>>
>>> >>> >>>>>>
>>> >>> >>>>>> + if len(layerlisturl) != 0:
>>> >>> >>>>>> + layerlisturl = layerlisturl[0]
>>> >>> >>>>>> + if len(layerlisturl) == 0:
>>> >>> >>>>>> + if
>>> >>> >>>>>> os.path.basename(localdirpath).startswith("_"):
>>> >>> >>>>>> + layerlisturl = localdirpath
>>> >>> >>>>>> if name != "bitbake":
>>> >>> >>>>>> -
>>>layerlist.append(localdirpath.rstrip("/"))
>>> >>> >>>>>> + for gitlocal, localpath in
>>> >>> >>>>>> gitrepos.iteritems():
>>> >>> >>>>>> + if layerlisturl in gitlocal:
>>> >>> >>>>>> + if len(localpath) > 2:
>>> >>> >>>>>> + for paths in localpath:
>>> >>> >>>>>> + if "bitbake" not in
>>> >>> >>>>>> paths[1]:
>>> >>> >>>>>> + layerlistdir =
>>> >>> >>>>>> localdirpath
>>> >>> >>>>>> +"/" + str(paths[1])
>>> >>> >>>>>> + if layerlistdir not in
>>> >>> >>>>>> layerlist:
>>> >>> >>>>>> +
>>> >>> >>>>>> layerlist.append(layerlistdir.rstrip("/"))
>>> >>> >>>>>> + else:
>>> >>> >>>>>> +
>>> >>> >>>>>> layerlist.append(localdirpath.rstrip("/"))
>>> >>> >>>>>> + break
>>> >>> >>>>>> + if len(layerlist) == 0:
>>> >>> >>>>>> + for gitlocal, localpath in
>>> >>> >>>>>> gitrepos.iteritems():
>>> >>> >>>>>> + for paths in localpath:
>>> >>> >>>>>> + if "bitbake" not in paths:
>>> >>> >>>>>> +
>>> >>> >>>>>> layerlist.append(layerlisturl +
>>> >>> >>>>>> "/"
>>> >>> >>>>>> + str(paths[1]))
>>> >>> >>>>>> +
>>> >>> >>>>>>
>>> >>> >>>>>> logger.debug("localhostbecontroller: current layer
>>> >>> >>>>>> list
>>> >>> >>>>>> %s "
>>> >>> >>>>>> % pformat(layerlist))
>>> >>> >>>>>>
>>> >>> >>>>>> diff --git a/
>>> >>> >>>>>>
>>> >>> >>>>>>
>>>bitbake/lib/toaster/bldcontrol/management/commands/loadconf.py
>>> >>> >>>>>>
>>> >>> >>>>>>
>>>b/bitbake/lib/toaster/bldcontrol/management/commands/loadconf.py
>>> >>> >>>>>> index 5022b59..a22f744 100644
>>> >>> >>>>>> ---
>>> >>> >>>>>>
>>> >>> >>>>>>
>>>a/bitbake/lib/toaster/bldcontrol/management/commands/loadconf.py
>>> >>> >>>>>> +++
>>> >>> >>>>>>
>>> >>> >>>>>>
>>>b/bitbake/lib/toaster/bldcontrol/management/commands/loadconf.py
>>> >>> >>>>>> @@ -45,12 +45,14 @@ class Command(BaseCommand):
>>> >>> >>>>>> for i in ['bitbake', 'releases', 'defaultrelease',
>>> >>> >>>>>> 'config',
>>> >>> >>>>>> 'layersources']:
>>> >>> >>>>>> assert i in data
>>> >>> >>>>>>
>>> >>> >>>>>> - def _read_git_url_from_local_repository(address):
>>> >>> >>>>>> + def _read_git_url_from_local_repository(address,
>>>path
>>> >>> >>>>>> =
>>> >>> >>>>>> None):
>>> >>> >>>>>
>>> >>> >>>>>
>>> >>> >>>>> If you want to pass in the path, please move the whole
>>>function
>>> >>> >>>>> outside
>>> >>> >>>>> _import_layer_config, as it's no longer a closure.
>>> >>> >>>>>
>>> >>> >>>>>
>>> >>> >>>>>>
>>> >>> >>>>>> url = None
>>> >>> >>>>>> + if not path:
>>> >>> >>>>>> + path = os.path.dirname(filepath)
>>> >>> >>>>>> # we detect the remote name at runtime
>>> >>> >>>>>> import subprocess
>>> >>> >>>>>> (remote, remote_name) = address.split(":", 1)
>>> >>> >>>>>> - cmd = subprocess.Popen("git remote -v",
>>> >>> >>>>>> shell=True,
>>> >>> >>>>>> cwd =
>>> >>> >>>>>> os.path.dirname(filepath), stdout=subprocess.PIPE, stderr =
>>> >>> >>>>>> subprocess.PIPE)
>>> >>> >>>>>> + cmd = subprocess.Popen("git remote -v",
>>> >>> >>>>>> shell=True,
>>> >>> >>>>>> cwd =
>>> >>> >>>>>> path, stdout=subprocess.PIPE, stderr = subprocess.PIPE)
>>> >>> >>>>>> (out,err) = cmd.communicate()
>>> >>> >>>>>> if cmd.returncode != 0:
>>> >>> >>>>>> logging.warning("Error while importing
>>>layer
>>> >>> >>>>>> vcs_url:
>>> >>> >>>>>> git error: %s" % err)
>>> >>> >>>>>> @@ -121,7 +123,11 @@ class Command(BaseCommand):
>>> >>> >>>>>> logger.error("Local layer path %s
>>>must
>>> >>> >>>>>> exists. Are you trying to import a layer that does not
>>>exist ?
>>> >>> >>>>>> Check your
>>> >>> >>>>>> local toasterconf.json" % lo.local_path)
>>> >>> >>>>>>
>>> >>> >>>>>> if
>>> >>> >>>>>> layerinfo['vcs_url'].startswith("remote:"):
>>> >>> >>>>>> - lo.vcs_url =
>>> >>> >>>>>> _read_git_url_from_local_repository(layerinfo['vcs_url'])
>>> >>> >>>>>> + if layerinfo['local_path'] not in
>>> >>> >>>>>> ["meta",
>>> >>> >>>>>> "meta-yocto", "meta-yocto-bsp"]:
>>> >>> >>>>>
>>> >>> >>>>>
>>> >>> >>>>> Please don't use hardcoded layer names anywhere in the code.
>>> >>> >>>>>
>>> >>> >>>>> We do not want special handling for some layers, all layers
>>>need
>>> >>> >>>>> to
>>> >>> >>>>> be
>>> >>> >>>>> treated in the same, uniform, way.
>>> >>> >>>>>
>>> >>> >>>>>
>>> >>> >>>>>> + repo_path =
>>> >>> >>>>>> layerinfo['local_path']
>>> >>> >>>>>> + else:
>>> >>> >>>>>> + repo_path = None
>>> >>> >>>>>> + lo.vcs_url =
>>> >>> >>>>>> _read_git_url_from_local_repository(layerinfo['vcs_url'],
>>> >>> >>>>>> repo_path)
>>> >>> >>>>>
>>> >>> >>>>>
>>> >>> >>>>> I think this code needs
>>> >>> >>>>>
>>> >>> >>>>>
>>> >>> >>>>>>
>>> >>> >>>>>> if lo.vcs_url is None:
>>> >>> >>>>>> logger.error("The toaster
>>>config
>>> >>> >>>>>> file
>>> >>> >>>>>> references the local git repo, but Toaster cannot detect
>>> >>> >>>>>> it.\nYour
>>> >>> >>>>>> local
>>> >>> >>>>>> configuration for layer %s is invalid. Make sure that the
>>> >>> >>>>>> toasterconf.json
>>> >>> >>>>>> file is correct." % layerinfo['name'])
>>> >>> >>>>
>>> >>> >>>>
>>> >>> >>>> A bit more explanation would help me move further to improvise
>>> >>> >>>> the
>>> >>> >>>> patch. If you don't mind please? So if I understand correctly
>>>a
>>> >>> >>>> separate
>>> >>> >>>> function should be there to handle HEAD branches provided by
>>> >>> >>>> toasterconf.json file, right?
>>> >>> >>>
>>> >>> >>>
>>> >>> >>>
>>> >>> >>> Yep, exactly my point.
>>> >>> >>>
>>> >>> >>> When inserting the Layer_Version objects for the "HEAD" branch,
>>> >>> >>> the
>>> >>> >>> Layer_Version.local_path should
>>> >>> >>>
>>> >>> >>> be set to the layer path.
>>> >>> >>>
>>> >>> >>> When checking out the layers in localhost bbcontroller, the
>>> >>> >>> checkout
>>> >>> >>> should be skipped at all times for "HEAD" branches; instead,
>>>the
>>> >>> >>> value in
>>> >>> >>> Layer_Version.local_path should be used.
>>> >>> >>
>>> >>> >>
>>> >>> >>
>>> >>> >> Is there a way to access local_path of toasterconf.json file
>>>from
>>> >>> >> localhostbecontroller.py file?
>>> >>> >
>>> >>> > I got the solution. I have modified local_path from "/" to the
>>>value
>>> >>> > in
>>> >>> > the
>>> >>> > configuration file in the database.
>>> >>> >
>>> >>> >
>>> >>> >>
>>> >>> >>
>>> >>> >>>
>>> >>> >>>
>>> >>> >>>
>>> >>> >>>>>>
>>> >>> >>>>>>
>>> >>> >>>>>> --
>>> >>> >>>>>> 1.9.1
>>> >>> >>>>>>
>>> >>> >>>>>> --
>>> >>> >>>>>> _______________________________________________
>>> >>> >>>>>> toaster mailing list
>>> >>> >>>>>> toaster at yoctoproject.org
>>> >>> >>>>>>
>https://lists.yoctoproject.org/listinfo/toaster
><https://lists.yoctoproject.org/listinfo/toaster>
>>> >>> >>>>>
>>> >>> >>>>>
>>> >>> >>>>>
>>> >>> >>>>>
>>> >>> >>>>> --
>>> >>> >>>>> Alex Damian
>>> >>> >>>>> Yocto Project
>>> >>> >>>>> SSG / OTC
>>> >>> >>>>
>>> >>> >>>>
>>> >>> >>>>
>>> >>> >>>>
>>> >>> >>>> --
>>> >>> >>>> സുജിത് ഹരിദാസന്
>>> >>> >>>> Bangalore
>>> >>> >>>> <Project>Contributor to KDE project
>>> >>> >>>>
>http://fci.wikia.com/wiki/Anti-DRM-Campaign
><http://fci.wikia.com/wiki/Anti-DRM-Campaign>
>>> >>> >>>> <Blog>
>http://sujithh.info <http://sujithh.info>
>>> >>> >>>
>>> >>> >>>
>>> >>> >>>
>>> >>> >>>
>>> >>> >>> --
>>> >>> >>> Alex Damian
>>> >>> >>> Yocto Project
>>> >>> >>> SSG / OTC
>>> >>> >>
>>> >>> >>
>>> >>> >>
>>> >>> >>
>>> >>> >> --
>>> >>> >> സുജിത് ഹരിദാസന്
>>> >>> >> Bangalore
>>> >>> >> <Project>Contributor to KDE project
>>> >>> >>
>http://fci.wikia.com/wiki/Anti-DRM-Campaign
><http://fci.wikia.com/wiki/Anti-DRM-Campaign>
>>> >>> >> <Blog> http://sujithh.info
>>> >>> >
>>> >>> >
>>> >>> > --
>>> >>> > _______________________________________________
>>> >>> > toaster mailing list
>>> >>> > toaster at yoctoproject.org
>>> >>> >
>https://lists.yoctoproject.org/listinfo/toaster
><https://lists.yoctoproject.org/listinfo/toaster>
>>> >>> >
>>> >>
>>> >>
>>> >>
>>> >>
>>> >> --
>>> >> സുജിത് ഹരിദാസന്
>>> >> Bangalore
>>> >> <Project>Contributor to KDE project
>>> >>
>http://fci.wikia.com/wiki/Anti-DRM-Campaign
><http://fci.wikia.com/wiki/Anti-DRM-Campaign>
>>> >> <Blog> http://sujithh.info
>>> >
>>> >
>>> >
>>> >
>>> > --
>>> > സുജിത് ഹരിദാസന്
>>> > Bangalore
>>> > <Project>Contributor to KDE project
>>> >
>http://fci.wikia.com/wiki/Anti-DRM-Campaign
><http://fci.wikia.com/wiki/Anti-DRM-Campaign>
>>> > <Blog> http://sujithh.info
>>
>>
>>
>>
>> --
>> സുജിത് ഹരിദാസന്
>> Bangalore
>> <Project>Contributor to KDE project
>>
>http://fci.wikia.com/wiki/Anti-DRM-Campaign
><http://fci.wikia.com/wiki/Anti-DRM-Campaign>
>> <Blog> http://sujithh.info
>
>
>
>
>
>
>
>
>
>
>--
>സുജിത് ഹരിദാസന്
>Bangalore
><Project>Contributor to KDE project
>http://fci.wikia.com/wiki/Anti-DRM-Campaign
><Blog> http://sujithh.info
>
>
>
>
>
>
>
>
>
>
>
>--
>സുജിത് ഹരിദാസന്
>Bangalore
><Project>Contributor to KDE project
>http://fci.wikia.com/wiki/Anti-DRM-Campaign
><Blog> http://sujithh.info
>
>
>
>
>
>
>
>
>
>
>--
>സുജിത് ഹരിദാസന്
>Bangalore
><Project>Contributor to KDE project
>http://fci.wikia.com/wiki/Anti-DRM-Campaign
><Blog> http://sujithh.info
>
>
>
>
>
>
>
>
>
>
>--
>സുജിത് ഹരിദാസന്
>Bangalore
><Project>Contributor to KDE project
>http://fci.wikia.com/wiki/Anti-DRM-Campaign
><Blog> http://sujithh.info
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>--
>സുജിത് ഹരിദാസന്
>Bangalore
><Project>Contributor to KDE project
>http://fci.wikia.com/wiki/Anti-DRM-Campaign
><Blog> http://sujithh.info
>
>
>
>
>
>
More information about the toaster
mailing list