$highlight{} doesn't handle accented characters correctly

[arjen]

I was just notified of the behaviour that words with é (and probably other characters aswell) are split into multiple words, as far as I know that shouldn't happen? Working around is of course to do a phrase search, but afaik xapian should either replace the é with an e or treat is as a normal character.

[olly]

Bug is in omindex.cc / scriptindex.cc. Similar code should be pulled out into a shared file and fixed to handle accented characters. Probably transliterate most to unaccented characters to normalise accent representation...

[james]

Presumably this will also affect query construction? Or does that already transliterate characters? Whatever, $highlight{} will also be affected, as it will need to do the transliteration while it's looking for words to highlight. I have a feeling I didn't pull out code when writing $highlight{} - it's probably duplicating code in query.cc or similar, and so will need to be fixed twice.

[olly]

Now fixed up everywhere apart from $highlight{} in query.cc which should share code with indextext.cc.

[olly]

Fixed in CVS

[olly]

Fixed in 0.8.2