When defaultOp is AND, QueryParser produces <matchNothing> for requests containing a wildcard term and at least two stopwords

[Daniel Ménard]

One of my users reported that our search engine was not always removing stopwords from the user query.

By investigating, I found that the query parser was returning a <matchNothing> Query for requests containing a wildcard and at least two stopwords, but only when defaultOp is AND (everyhing is fine if defaultOp is OR).

Attached is a small PHP script which reproduce the problem.

On my machine (Windows XP, Xapian php-bindings 1.0.16 binaries from flax.co.uk, php 5.2.9-2), this script produces the following output:

C:\php>php b:\test.php
PHP version 5.2.9-2, Xapian version 1.0.16

Creating a new db containing a document with terms : test, tester, testable, user,

-------------- DefaultOp=OR--------------
query: this is a test
result: Xapian::Query(test:(pos=4))

query: test*
result: Xapian::Query((test:(pos=1) OR testable:(pos=1) OR tester:(pos=1)))

query: a test*
result: Xapian::Query((test:(pos=2) OR testable:(pos=2) OR tester:(pos=2)))

query: is a test*
result: Xapian::Query((test:(pos=3) OR testable:(pos=3) OR tester:(pos=3)))

query: this is a test*
result: Xapian::Query((test:(pos=4) OR testable:(pos=4) OR tester:(pos=4)))

query: this is a us* test*
result: Xapian::Query((user:(pos=4) OR test:(pos=5) OR testable:(pos=5) OR tester:(pos=5)))

query: this is a user test*
result: Xapian::Query((user:(pos=4) OR test:(pos=5) OR testable:(pos=5) OR tester:(pos=5)))

-------------- DefaultOp=AND--------------
query: this is a test
result: Xapian::Query(test:(pos=4))

query: test*
result: Xapian::Query((test:(pos=1) OR testable:(pos=1) OR tester:(pos=1)))

query: a test*
result: Xapian::Query((test:(pos=2) OR testable:(pos=2) OR tester:(pos=2)))

query: is a test*
result: Xapian::Query()
FAILS.

query: this is a test*
result: Xapian::Query()
FAILS.

query: this is a us* test*
result: Xapian::Query()
FAILS.

query: this is a user test*
result: Xapian::Query((user:(pos=4) AND (test:(pos=5) OR testable:(pos=5) OR tester:(pos=5))))

The problem is very specific: the query must contain a wildcard term, at least two stop words (1 wildcard + 1 stopword works well!) and no other term.

Also, the problem is not new: I've tested with xapian 1.0.11 and 1.0.12 which produce the same output.

[Daniel Ménard]

PHP script which reproduce the problem

[Olly Betts]

Reproduces for me with trunk.

Marking for 1.1.3, at least for now.

[Olly Betts]

Fixed in trunk r13583 (one line fix!)

Marking for backport for 1.0.17.

[Olly Betts]

Backported for 1.0.17 in r13596.

[Daniel Ménard]

Great!

Thanks a lot, Olly.