[Toaster] [review-request] V2: 6137 excessive load time for All Recipes page

[Damian, Alexandru]

Hello,

I have a small modification to David's patch that improves the loading time
about 5-fold compared to his latest version, from 5 seconds to less than 1
second on my laptop.

The trick used is to bring the whole list of dependencies for the entire
recipe list in a single call, and do the match manually, instead of getting
a call per recipe.

This approach allows the code to perform constant lookup times independent
of number of rows.

Since I can't push to contrib on my laptop (something funky about keys),
I'm attaching the patch for review.

Please let me know how it goes.

Cheers,
Alex


On Fri, Apr 11, 2014 at 5:26 PM, Reyna, David <david.reyna at windriver.com>wrote:

> Hi Belén,
>
> > page still seems to be performing much worse than all other pages. Should
> > we be trying to fix the root cause of the problem, ie. the "100*(2+2)
> > foreign key lookups and filters/count"?
>
> If we stay with the current design of the page, then I think the solution
> is along the lines:
>
>   1. Add two fields to the Recipe class to hold the respective forward and
> reverse counts to support instant lookup. These values are fixed when the
> build completes so they can be consider constant for our purposes, plus
> they would add very little overhead to the database (if you can get this
> proposal past Alex). An alternate implementation to keep the database
> pristine is to add the ability to cache information like this in some
> runtime table, so that the calculation time is only spent once.
>
>   2. Add a dynamic lookup of a given forward or reverse dependency list
> when one of those buttons are clicked. In this manner we only spend the
> list computation time when explicitly asked, not for every recipe whether
> they want it or not.
>
> - David
>
> > -----Original Message-----
> > From: Barros Pena, Belen [mailto:belen.barros.pena at intel.com]
> > Sent: Friday, April 11, 2014 5:41 AM
> > To: Reyna, David; Lerner, Dave
> > Cc: toaster at yoctoproject.org
> > Subject: Re: [review-request] V2: 6137 excessive load time for All
> Recipes
> > page
> >
> > On 11/04/2014 01:44, "Reyna, David" <david.reyna at windriver.com> wrote:
> >
> > >Here are the timing results on my slow host for the rendering time:
> > >
> > >  (a) Original: 13 seconds
> > >  (b) V1      :  7 seconds
> > >  (c) V2      :  4 seconds
> >
> > This is obviously a huge improvement. What bothers me is that the Recipes
> > page still seems to be performing much worse than all other pages. Should
> > we be trying to fix the root cause of the problem, ie. the "100*(2+2)
> > foreign key lookups and filters/count"?
> >
> > Thanks!
> >
> > Belén
> >
>
> --
> _______________________________________________
> toaster mailing list
> toaster at yoctoproject.org
> https://lists.yoctoproject.org/listinfo/toaster
>



-- 
Alex Damian
Yocto Project
SSG / OTC
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.yoctoproject.org/pipermail/toaster/attachments/20140411/4d483867/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 0001-bitbake-toaster-reduce-redundant-foreign-key-lookups.patch
Type: text/x-patch
Size: 4551 bytes
Desc: not available
URL: <http://lists.yoctoproject.org/pipermail/toaster/attachments/20140411/4d483867/attachment-0001.bin>