Review ValueWeightPostingSource, possibly replace with a query operator

[Richard Boulton]

External PostingSources have at least two annoying limitations (don't work with remote databases, don't work well with multi databases).

The newly added ValueWeightPostingSource simply reads a value slot, returns documents with a non empty value, and returns the weight obtained by applying sortable_unserialise to the slot. Therefore, it could be implemented instead by a query operator, which would be similar to the existing OP_VALUE_... operators. This would make the feature available with remote databases and multi databases.

There may be a cleaner alternative which we haven't thought of yet, too.

Marking this for 1.1.0, since ValueWeightPostingSource isn't yet in the API for any release, and we should remove it before making a release if we're going to remove it at all.

[Richard Boulton]

Note that #295 has a patch attached which would allow PostingSources (and ValueWeightPostingSource in particular) to work with multi databases. I might shortly be extending this patch to allow it to work with remote databases, too.

It might still be cleaner to have a new OP_VALUE_WEIGHT operator instead of ValueWeightPostingSource, anyway.

[Olly Betts]

I'm not sure I see why OP_VALUE_WEIGHT is cleaner (if remote and multi databases aren't an issue).

Hmm, and how does OP_VALUE_WEIGHT help with multi databases?

[Olly Betts]

This wouldn't be an incompatible change, so bumping to 1.1.1.

[Olly Betts]

Oops, wrong ticket. Undoing...

[Richard Boulton]

I'm not sure whether it's particularly cleaner: the main way in which it might be cleaner is that it avoids any need to keep track of the PostingSource instance, and delete it after the match (and not before the match). Because they're not reference counted, this can be awkward.

Implementing OP_VALUE_WEIGHT the obvious way (ie, by making a new PostList for it) would work with multi databases without special effort. ValueWeightPostingSource currently doesn't work with multi databases. As long as the operator was added to the serialisation format for queries, it would work for remote databases, too.

I think I favour fixing ValueSources to work with remote and multi databases, instead, though. It's more flexible, and easy enough to use.

[Richard Boulton]

I think it would be okay to resolve this ticket as "WONTFIX". Possibly we should wait until #295 is resolved first, though.

[Richard Boulton]

Erk - I didn't mean to close the ticket when I made that last comment!

[Richard Boulton]

#295 is now resolved, so as discussed above marking this as WONTFIX.

[Olly Betts]

Removing milestone!