Dynamic summaries / snippets

[olly]

Xapian should include features to allow dynamic summaries to be generated from snippets of text around where the query terms occur in a matching document.

This has been asked about several times on the mailing list, for example:

http://thread.gmane.org/gmane.comp.search.xapian.general/5097

[richard]

Some python code which implements this is available at: http://xappy.googlecode.com/svn/trunk/xappy/highlight.py

However, the approach taken by this has some shortcomings - in particular, it doesn't have any handling for phrases, so terms which are only present in the query as phrases can be highlighted individually.

Also, of course, it's implemented in python, so isn't accessible to users of other languages.

[olly]

I wonder if we could have some sort of positionlist->postlist adaptor class and rerun the query on the positionlists for a matching document.

Problem is where to put the document boundaries - using certain punctuation (as xappy appears to) makes sense, but we'd need to generate those positions. That could be done at index time and then compressed with interpolative coding I guess...