Release Overview for 1.4.26

This page contains a high level description of the most notable changes in this release. For full details of user-visible changes, see the NEWS files in each module:

• ​xapian-core
• ​omega
• ​xapian-bindings

See also the full list of bug reports marked as fixed in this release.

API

• Weight: Document that Weight statistics DOC_LENGTH_MIN, DOC_LENGTH_MAX and WDF_MAX are for the shard rather than the whole database. Usually this is what we want as with a sharded database it gives tighter bounds and so better match optimisation, but it does make them unsuitable for uses such as calculating a suitable offset to add to every get_sumextra() to allow implementing a weighting formula which can give a negative term independent weight contribution. This case will be addressed in the next release series which also provides bounds such as DB_DOC_LENGTH_MIN which are for the whole database.

• LMWeight: This class was meant to implement the "Language Model" Weighting scheme, but we've discovered the implementation was incorrect and fixing it requires ABI-incompatible changes. For 1.4.x we need to leave it in place so as not to break existing code, but it's now deprecated and we recommend avoiding using it. It will be removed in the next release series and replaced with new separate classes implementing Language Model weighting with each smoothing.

• PL2PlusWeight: Fix bug in implementation of formula. Our variable mean is 1/lambda_t from the PL2+ paper, so we need to check mean>1 for lambda_t<1 but we were actually checking mean<1 instead. The result of this is that PL2+ actually returned a zero weight unless the term occurred frequently enough in the collection.

• Enquire::set_expansion_scheme(): Add prob as new preferred name for probabilistic query expansion, with the previous trad still being accepted for now.

• QueryParser::set_prefix() and set_boolean_prefix(): Allow an optional trailing : on the field name. This makes the API here more consistent with ranges, where you need to include the : if you want one. See #720.

portability

• Fix to compile as C++20 and C++23.

• Resolve SIGPIPE issues on NetBSD, which were causing testcase keepalive1 to fail. These seem to be due to SO_NOSIGPIPE not working correctly there so we now use MSG_NOSIGNAL instead for NetBSD.

Omega

• Support "$set{weighting,bm25+} and $set{weighting,pl2+}.

• Deprecate $set{weighting,lm}. This was meant to implement the "Language Model" Weighting scheme, but we've discovered the implementation was incorrect and fixing it requires ABI-incompatible changes in xapian-core. For 1.4.x we need to leave it in place so as not to break existing code, but we recommended avoiding using it. It will be removed in the next release series and replaced with new separate classes implementing Language Model weighting with each smoothing.

• Add prob as new preferred name for probabilistic query expansion in $set{expansion,prob}, with the previous trad still being accepted for now.