[Toaster] any tricks when modifying "buildinfohelper.py"?

[Reyna, David]

Hi Alex,

I was able to get a process for "buildinfohelper.py" to work and resolve #6050. Here are my notes in case that may be useful to anyone else who may start working with the backend, plus I am sure you have additional techniques and tricks of the trade.

I would be curious to learn how one would use pydev to debug the backend. I know that you passed some information to Dave (which I quote below) but those instructions appears to be for custom modules and not the mainline code base. No hurry, but it would be very helpful for Toaster-1.7 development.

Here was my devel/debug cycle:

  1) Stop Toaster
  2) Edit the backend files like "buildinfohelper.py" and "models.py".
  3) Restart Toaster
  4) Build a trivial package to execute the new code (e.g. "bitbake base-files")
  5) Test the changes in your browser

Here are the error states and workflow issues that I encountered:

  * Unlike the GUI files when you can make changes midstream with Toaster still running, you have to stop and restart Toaster in order for your backend changes to become active (I think that when Toaster starts is when the updated code gets compiled into the ".pyc" runtime code). Some files (like "models.py") are in between (where schema changes need a restart but method changes may not), but it is easy to get into an intermediate state and have unexpected failures doing this on the fly.

  * If there is a syntax error in your backend code, the Toaster will start and then very quickly halt. You will most likely see many messages of the type "ERROR: Could not connect to server at localhost:8200 ([Errno 111] Connection refused)", and a similar error when you try to connect with the browser. You will also know it failed if the "*.pyc" file did not get updated.

My resolution was to stop the Toaster to insure that the full state was cleaned up, re-examine the code, and then re-start Toaster. When I was lucky the actual error was captured in the "toaster_ui.log" file. I found that some of the background tasks took some 5 to 15 seconds before they fully stopped, so I had to either retry a few times or just be patient.

  * If there was a runtime error, it would most likely also be captured in "toaster_ui.log". One may not notice the failure state while using the browser until you get unexpected page errors, like indexes out of range (for when the respective data object never got populated because your code failed).

  * If you put "print" statements into your code for simple tracing purposes, that output will again appear in "toaster_ui.log". (I myself typically insert and use a "_log" helper function to append the messages to a log file under "/tmp/toaster.log" for easy persistence and management so that I can track and study it at length).

Thanks,
David

vvvvvvvvvvvvvvvvvvvvvvvvvvv

Dave Lerner wrote: "I found the best way to debug python with the database is (with help from Alex)"

1) To get into python and have access to the database class models, execute this script which will prompt you to get to the right spot, then prompt you as to how to setup for the models.

+ if [ ! -e manage.py ] ; then
+    echo "Please run from the lib/toaster directory"
+    exit 1
+ fi
+ echo
+ echo do...
+ echo from orm.models import Build, Package, Target
+ echo
+ MANAGE=$PWD
+ python $MANAGE/manage.py shell

Import any python model that you want from orm.models. Then you can do things like (>>> is the prompt)
>>> target_name = Target.objects.get(pk=3).target


2) To run code in a module that you put into toaster/toastergui/mytest.py

* import toastergui.mytest
* This will run any code that isn't inside a class or def.
* In another session, if you want to change the module, edit it, then
  >>>  reload(toastergui.mytest)
  and it will rerun that module

3) to debug or use a function in that module

* edit functions in your module, and rename the functions with a '_' or '__' prefix (at first) so that a simple import works without hiding it during import which is what the prefixes do
- def _my_ func(build_id, target_id)
+ def my_func(build_id, target_id)
...

* insert 'import pdb ; pdb.set_trace()' where you want debugging to start
        import toastergui.mytest
* invoke the full path to the function
        toastergui.mytest.my_func(1, 2)
  and the code will break where you put your breakpoint
* iterate development by ...
*** editing the module
*** exiting debug mode with
        q
*** reload the module with
        reload(toastergui.mytest)
*** rerun the function with
        toastergui.mytest.my_func(1, 2)

^^^^^^^^^^^^^^^^^^^^^^^^
      _____________________________________________
      From: Reyna, David
      Sent: Friday, March 28, 2014 11:20 AM
      To: belen.barros.pena at intel.com; Damian, Alexandru (alexandru.damian at intel.com)
      Subject: any tricks when modifying "buildinfohelper.py"?


      Hi,

      I am trying to implement adding the task descriptions to the Variable table.

      It appears that the changes I need to make are in "def save_build_variables" in the "buildinfohelper.py" file, but my changes do not appear to be making any affect, including writing to external log files if/when this procedure is called.

      Is there a trick I need to do to have file changes registered, like restarting Toaster or removing the "buildinfohelper.pyc"?

      - David


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.yoctoproject.org/pipermail/toaster/attachments/20140330/7e0cf10f/attachment.html>