Opening a writable database should retry if database is locked

[Richard Boulton]

Writable databases currently fail to open with a DatabaseLockedError exception if the database is already opened by a different process. In situations where multiple processes (or threads) want to open a database for writing, this forces users to implement a retry mechanism to handle these errors. It might be helpful if databases had an option which would cause them to retry (until a timeout was reached) if the database was locked, to avoid users having to implement retry mechanisms in every application wanting this feature.

Alternatively, it might be better to continue with our current approach of suggesting that users structure applications such that database writes are marshalled into a single location and performed on a single WritableDatabase object.

[Olly Betts]

Fixed wikimarkup in description.

The "single writer" design works well, but it isn't going to work for everyone, and it's impossible to achieve the efficiency of a blocking attempt to lock the lockfile by polling the API, so I think this is a change worth considering.

For a remote database, we would ideally retry with a protocol level blocking-open message. Probably with a retry if the link drops (e.g. timed out).

That should be doable as a minor protocol version change, and the API part should be backward compatible, as should the ABI, so marking for 1.2.x.

[Olly Betts]

Now we have flags passed to the WritableDatabase constructor, this could be implemented as something like:

    Xapian::WritableDatabase db("/path/to/db", Xapian::DB_BLOCK);

Not sure if that's the best name - perhaps DB_RETRY is clearer.

[Olly Betts]

Implemented in master [eec6d9f] for 1.3.3. I named the new flag DB_RETRY_LOCK.

Includes remote support without needing to bump the protocol version - a 1.3.2 server just ignores where a 1.3.3 client passes the new flag, and so will throw DatabaseLockError, and a 1.3.2 client passes an empty string for the flags which will cause a 1.3.3 server to behave as a 1.3.2 server would.