term generation for some French elisions produces imperfect results.

[Paul Rudin]

Using the xapian.TermGenerator with the standard French stemmer text containing, for example, "l'Etat" gives terms "l'etat" and "Zl'etat". The problem is that if you then search for "etat" you won't get a match but in most cases this is probably what users want.

I suppose that the correct thing would be to stem to etat?

[Olly Betts]

I guess we need to decide if it is the TermGenerator's job to handle the apostrophe in cases like this, or the stemmer's job to cope with the apostrophe appropriately.

Currently TermGenerator treats apostrophe as a word character, and the English stemmer understands "'s" suffixes, but I don't think any other stemmers do anything special with apostrophes.

And QueryParser needs to match TermGenerator in this regard.

[Kelson]

Probably kind of obvious, but this is not only causing a problem for "l'", but as well for "d'" which is really common as well. See this Kiwix ticket ​https://github.com/openzim/libzim/issues/592 for an other concrete example of the problem.

[Olly Betts]

I think ideally we'd deal with this in Snowball so I've opened an issue there: ​https://github.com/snowballstem/snowball/issues/187