[Toaster] [review-request] adamian/20140429-toastertesting

[Damian, Alexandru]

Hello all,

In a bid to get more help from automating sanity testing, I've written a
Toaster command that will trigger some testing on a submitted review
request. In short - if you've got a branch ready for submission, this test
will make sure that the new code will not break anything already working.


How this works:
 - the command should be run in the root poky source directory, i.e. where
you git clone'd the poky tree;
- run *./bitbake/lib/toaster/manage.py selftest branch-to-test*
- it takes a branch as parameter, and performs a suite of tests on that
branch before cleaning up.

*!IMPORTANT!*
- It doesn't need a "build" directory or running "source oe-init-build-env".
- It supports applying a custom database configuration to allow usage of a
speedier backend than sqlite. The custom configuration should come in the
form of a GIT commit id - see the source.


Basically what this command does is:

- verifies that the submitted branch can be cleanly rebased on top of
origin/master
- verifies that there are no python syntax errors in the new branch
- *runs any defined Django tests for Toaster*
- verifies that the system starts and stops correctly, using a temporary
build environment
- crawls the website and verifies that there are no links that get a 404,
and there are no broken pages (i.e. 500)

The results are printed to standard output.



Planned development include:
- include running the *oe-selftest* tests for Toaster
- *automated integration* with the *toaster mailing list* to automatically
test and report back the test result by email for each and all
 [review-request] pull requests



The proper way to extend this command to run any tests you want to add is
to either:
- add a standard Django test if you're developing Django front-end code
- add a *oe-selftest* Toaster test if you're developing Toaster back-end
code outside of the direct scope of the Django project.


Any new automated test for existing or planned features will be greatly
appreciated !


I'm waiting your comments on the initiative and the patch.

Cheers,
Alex

-- 
Alex Damian
Yocto Project
SSG / OTC
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.yoctoproject.org/pipermail/toaster/attachments/20140429/5da6856b/attachment.html>