[Toaster] Errors with composite/multiple date filters (6040)

[Damian, Alexandru]

Hi David,

Can you please check if the patch at the top of:
*adamian/20140408-getfilteringquery* will fix this issue for you ?

I managed to reproduce what you're seeing, and this patch works for me.

If it's fine, can you please rebase your bug fix on top of this patch ?

Thank you,
Alex


On Tue, Apr 8, 2014 at 2:33 PM, Damian, Alexandru <
alexandru.damian at intel.com> wrote:

> Thank you for this detailed analysis.
>
> I'm focusing now on debugging _get_queryset.
>
> Alex
>
>
> On Tue, Apr 8, 2014 at 6:57 AM, Reyna, David <david.reyna at windriver.com>wrote:
>
>>  Hi Alex,
>>
>> I have looked at the date filtering problem up close, and here is my
>> results and recommendations. This issue is blocking 6040.
>>
>> 1) The “__lte” does not do the equal, it just does “__lt”. That is why
>> for “yesterday” you have to do “started_on__gte;Bstarted_on__lt” of
>> “<day>;<day+1>”.
>>
>> RECOMMENDATION: I can just use “__lt” for now and adjust the end date
>> accordingly.
>>
>> 2) The multiple filter function of “_get_filtering_query” does not work
>> correctly, whereas if you do the filters as separate operations you get the
>> expected results.
>>
>> Here is a an example of what I found. I ran a set of date range URLs with
>> the existing mutiple filters and then with a routine that applies the
>> filters separately.
>>
>> My "Started on" dates are these: 4/4/14, 6/4/14, 7/4/14
>>
>> Here is what happens, where the invalid results have an ‘x’:
>>
>>   filter=started_on__gte%3Bstarted_on__lt%3A2014-04-04%3B2014-04-05
>>   Filter Together: 4/4/14, x6/4/14, x7/4/14
>>   Filter Separate: 4/4/14
>>
>>   filter=started_on__gte%3Bstarted_on__lt%3A2014-04-05%3B2014-04-06
>>   Filter Together: x6/4/14, x7/4/14
>>   Filter Separate: <none>
>>
>>   filter=started_on__gte%3Bstarted_on__lt%3A2014-04-06%3B2014-04-07
>>   Filter Together: 6/4/14, x7/4/14
>>   Filter Separate: 6/4/14
>>
>>   filter=started_on__gte%3Bstarted_on__lt%3A2014-04-07%3B2014-04-08
>>   Filter Together: 7/4/14
>>   Filter Separate: 7/4/14
>>
>>   filter=started_on__gte%3Bstarted_on__lt%3A2014-04-04%3B2014-04-07
>>   Filter Together: 4/4/14, 6/4/14, x7/4/14
>>   Filter Separate: 4/4/14, 6/4/14
>>
>>   filter=started_on__gte%3Bstarted_on__lt%3A2014-04-04%3B2014-04-08
>>   Filter Together: 4/4/14, 6/4/14, 7/4/14
>>   Filter Separate: 4/4/14, 6/4/14, 7/4/14
>>
>> In all cases the separate filter application gave the expected answer,
>> where the multiple filter often did not.
>>
>> RECOMMENDATION: Until we can debug and fix “_get_filtering_query”, change
>> it for now to execute the filter queries separately, and I can rebase on
>> that. Currently only the data range filters use the multiple filter syntax,
>> so the impact is limited. If you do not want to change this routine, then I
>> have my work around listed below.
>>
>> Here is my sample routine for executing the separate filtering:
>>
>> *def date_filtering_query(queryset, filter_string):*
>> *    search_terms = filter_string.split(FIELD_SEPARATOR)*
>> *    keys = search_terms[0].split(VALUE_SEPARATOR)*
>> *    values = search_terms[1].split(VALUE_SEPARATOR)*
>> *    for i in range(len(keys)):*
>>                 *filter_query =
>> _get_filtering_query(keys[i]+":"+values[i])*
>>                 *queryset = queryset.filter(filter_query)*
>>                 *_log("FILTER_COUNT="+str(queryset.count()))*
>> *    return queryset*
>>
>> I called it this way in my ‘build’ view class.
>>
>> *    …*
>> *    if **True and **(filter_string.startswith('started_on') or
>> filter_string.startswith('completed_on')):*
>> *        queryset = _get_queryset(Build, queryset_all, '', search_term,
>> ordering_string, '-completed_on')*
>> *        queryset = date_filtering_query(queryset,filter_string)*
>> *    else:*
>> *        queryset = _get_queryset(Build, queryset_all, filter_string,
>> search_term, ordering_string, '-completed_on')*
>> *    …*
>>
>> - David
>>
>>
>>
>
>
>
> --
> Alex Damian
> Yocto Project
> SSG / OTC
>



-- 
Alex Damian
Yocto Project
SSG / OTC
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.yoctoproject.org/pipermail/toaster/attachments/20140408/20c820ba/attachment.html>