Unknown Exception from scriptindex when indexing a very long document

[arjen]

When testing something else I noticed that scriptindex didn't index a specific batch of documents, outputted the message "Unknown Exception" and stopped that batch.

I had a suspicion about a certain document in that set and seperated it from the batch. When I tried to create a new database with only that document, I got the same exception. I'm not sure whether its the documents length (its the longest we have, about 5.5MB in plain text) or its content.

Since the document is not entirely free for me to attach along, I've e-mailed a bzip2ed-version of it to Olly.

[olly]

The problem is trying to "unhtml" text which contains "</body>".

Fixed in CVS HEAD.

[olly]

Fixed in 0.8.5.