[Toaster] Errors with composite/multiple date filters (6040)

[Damian, Alexandru]

Thank you for this detailed analysis.

I'm focusing now on debugging _get_queryset.

Alex


On Tue, Apr 8, 2014 at 6:57 AM, Reyna, David <david.reyna at windriver.com>wrote:

>  Hi Alex,
>
> I have looked at the date filtering problem up close, and here is my
> results and recommendations. This issue is blocking 6040.
>
> 1) The “__lte” does not do the equal, it just does “__lt”. That is why for
> “yesterday” you have to do “started_on__gte;Bstarted_on__lt” of
> “<day>;<day+1>”.
>
> RECOMMENDATION: I can just use “__lt” for now and adjust the end date
> accordingly.
>
> 2) The multiple filter function of “_get_filtering_query” does not work
> correctly, whereas if you do the filters as separate operations you get the
> expected results.
>
> Here is a an example of what I found. I ran a set of date range URLs with
> the existing mutiple filters and then with a routine that applies the
> filters separately.
>
> My "Started on" dates are these: 4/4/14, 6/4/14, 7/4/14
>
> Here is what happens, where the invalid results have an ‘x’:
>
>   filter=started_on__gte%3Bstarted_on__lt%3A2014-04-04%3B2014-04-05
>   Filter Together: 4/4/14, x6/4/14, x7/4/14
>   Filter Separate: 4/4/14
>
>   filter=started_on__gte%3Bstarted_on__lt%3A2014-04-05%3B2014-04-06
>   Filter Together: x6/4/14, x7/4/14
>   Filter Separate: <none>
>
>   filter=started_on__gte%3Bstarted_on__lt%3A2014-04-06%3B2014-04-07
>   Filter Together: 6/4/14, x7/4/14
>   Filter Separate: 6/4/14
>
>   filter=started_on__gte%3Bstarted_on__lt%3A2014-04-07%3B2014-04-08
>   Filter Together: 7/4/14
>   Filter Separate: 7/4/14
>
>   filter=started_on__gte%3Bstarted_on__lt%3A2014-04-04%3B2014-04-07
>   Filter Together: 4/4/14, 6/4/14, x7/4/14
>   Filter Separate: 4/4/14, 6/4/14
>
>   filter=started_on__gte%3Bstarted_on__lt%3A2014-04-04%3B2014-04-08
>   Filter Together: 4/4/14, 6/4/14, 7/4/14
>   Filter Separate: 4/4/14, 6/4/14, 7/4/14
>
> In all cases the separate filter application gave the expected answer,
> where the multiple filter often did not.
>
> RECOMMENDATION: Until we can debug and fix “_get_filtering_query”, change
> it for now to execute the filter queries separately, and I can rebase on
> that. Currently only the data range filters use the multiple filter syntax,
> so the impact is limited. If you do not want to change this routine, then I
> have my work around listed below.
>
> Here is my sample routine for executing the separate filtering:
>
> *def date_filtering_query(queryset, filter_string):*
> *    search_terms = filter_string.split(FIELD_SEPARATOR)*
> *    keys = search_terms[0].split(VALUE_SEPARATOR)*
> *    values = search_terms[1].split(VALUE_SEPARATOR)*
> *    for i in range(len(keys)):*
>                 *filter_query =
> _get_filtering_query(keys[i]+":"+values[i])*
>                 *queryset = queryset.filter(filter_query)*
>                 *_log("FILTER_COUNT="+str(queryset.count()))*
> *    return queryset*
>
> I called it this way in my ‘build’ view class.
>
> *    …*
> *    if **True and **(filter_string.startswith('started_on') or
> filter_string.startswith('completed_on')):*
> *        queryset = _get_queryset(Build, queryset_all, '', search_term,
> ordering_string, '-completed_on')*
> *        queryset = date_filtering_query(queryset,filter_string)*
> *    else:*
> *        queryset = _get_queryset(Build, queryset_all, filter_string,
> search_term, ordering_string, '-completed_on')*
> *    …*
>
> - David
>
>
>



-- 
Alex Damian
Yocto Project
SSG / OTC
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.yoctoproject.org/pipermail/toaster/attachments/20140408/b0f7b62e/attachment-0001.html>