When changesets are being generated, old changesets aren't cleaned up

[richard]

Currently, changesets are generated when the "XAPIAN_MAX_CHANGESETS" environment variable is set to a non-empty value. However, they are never removed. Whenever a changeset is generated, the number of changesets around should be checked, and old changesets should be removed if too many old changesets exist.

Alternatively (or as well), a different criteria might be useful for the changesets: it might be useful to be able to set an absolute limit on the total size of the changesets, or perhaps, a limit on the total size of the changesets as a proportion of the total database size.

[richard]

(Setting component to Backend-Chert, though it also affects flint.)

[olly]

Fixed to not generate a changeset for the first revision in trunk r11069 (description updated).

In order to efficiently implement keeping a maximum number of changesets and/or maximum size of changesets, I think we want to store the lowest changeset number somewhere. Then we can calculate current_revision - lowest_changeset_number and compare with maximum_number to implement that. And if we also store the total size (as number of blocks would make most sense probably), we can then delete the lowest numbered changeset files and decrease this total appropriately until it is within the specified limit. We can also age by date by checking the timestamps of the lowest numbered changeset files.

(We could establish the lowest changeset number in O(log(current_revision)) by binary chop, but that seems unnecessary work...)

[olly]

(In #227) This is done except for the two blockers, and a bug just to track two others is just clutter so unmarking the blockage and closing.