Inconsistent ordering between pages with set_sort_by_value_then_relevance

[Olly Betts]

Quoting from ​http://thread.gmane.org/gmane.comp.search.xapian.general/4045/focus=4056 :

​http://5nord.org/~mrks/xapian_sortorder1.pl

This program populates a fresh database with 10 documents, 5 with term "test" (termcount:1) and values 0-4, and 5 with term "test" (termcount:2) and also values 0-4. Then it performs some searches on this index, all with the same query ("test"). The first search requests 10 results, starting from 0. The following searches produce the same result, but requests first 1 result starting from 0, 1 result starting from 1, and 8 results starting from 2. Both methods should give the same output. If you comment out the set_sort_by_value_then_relevance() in line 34, this is actually the case, but with sorting by value first, the results differ. This is the output of my program:

Without sorting by value (line 34 commented out):

(unpaged) ID 6 99% ID 7 99% ID 8 99% ID 9 99% ID 10 99% ID 1 86% ID 2 86% ID 3 86% ID 4 86% ID 5 86% (paged) ID 6 99% ID 7 99% ID 8 99% ID 9 99% ID 10 99% ID 1 86% ID 2 86% ID 3 86% ID 4 86% ID 5 86%

And with sorting by value (line 34 untouched):

(paged) ID 10 99% ID 5 86% ID 9 99% ID 4 86% ID 8 99% ID 3 86% ID 7 99% ID 2 86% ID 6 99% ID 1 86% (paged) ID 5 100% <-- this is where it becomes strange ID 5 86% <-- the same doc_id twice, with different relevances ID 9 99% ID 4 86% ID 8 99% ID 3 86% ID 7 99% ID 2 86% ID 6 99% ID 1 86%

[Olly Betts]

perl script which demonstrates the problem

[Olly Betts]

"I am using Xapian 0.9.9 (the ubuntu package 0.9.9-1.99dapper), and Search-Xapian-0.9.9.1."

[Olly Betts]

Reproducible with SVN HEAD using C++ (so not a bug in the Perl bindings).

[Olly Betts]

Fixed in SVN HEAD. I'll try to backport the fix to 0.9.9.

[Olly Betts]

Patch to fix this for 0.9.9

[Olly Betts]

This will be fixed in 0.9.10 (coming soon).