remote connection should pass 'writable' flag

[Mark Hammond]

When a client application uses (say) xapian.remote_open() to connect to a server running in 'writable' mode, the server still opens the database for the connection in 'writable' mode, even though this was not requested by the caller.

This limitation means that an application might need to use 2 servers - one for writable and one for read-only - as usage of the writable server will lock out all other read-only requests, which would be unacceptable in some environments.

The fix is not trivial as the protocol doesn't provide a way of providing connection-specific options. A solution would be to have the client send a MSG_KNOCK (?) message at connection with options (just this flag in the first instance) and the server could respond with its REPLY_GREETING if all is well. I understand this isn't going to make 1.0 though (well, unless you are really keen and would accept a patch if I could make one :)

[Olly Betts]

It's possible we could consider this for 1.0.X, but not for 1.0.0.

But if we can avoid it, I'd prefer not to add an extra message to the start of every connection.

I think we can achieve this using something like:

Client connects, but remote server is in use, and instead of the usual REPLY_GREETING gets:

<- REPLY_BUSY <PROTOCOL VERSION>

And then the client can either disconnect or send:

-> MSG_WAITING

MSG_WAITING avoids a race-like condition where the server closes the current writer, and the opens a new WritableDatabase for a client which is just disconnecting anyway. This is mostly harmless, but seems like wasted effort.

If that seems workable, I think we can sensibly add code to 1.0.0 for REPLY_BUSY just to handle it in the client by disconnecting (since in normal use that code path won't be seen), so then we won't need to bump the remote protocol version to add this later.

[Mark Hammond]

The problem is still that not all connections want to block or queue - only ones that intend to write. What I'm trying to come up with is a scheme where a single server can be used for *both* readonly and writable connections, with the writable semantics only kicking in if that is really what the client needs.

[Olly Betts]

I'm reluctant to force the extra latency of communicating the access mode upon every connection.

It seems a reasonable plan to require use of two ports in this situation (i.e. one for read-only connections, one for writable). This readonly/writable split also allows access to be firewalled differently which will be useful in some setups. For example, you don't ever want your front end search boxes to be able to update your index even if they're compromised, but your indexing boxes do need to be able to.

It would be pretty easy to allow xapian-tcpsrv to listen on two ports for connections so you'd only need one master server process running, if you're just trying to avoid having to run two servers here.

Otherwise, there are certainly other options, for example:

• The server could open the database read-only and then accept an "I want to

write" message which tries to reopen it for writing instead.

• A writable server could send a different greeting message. The client would

then have to specify if it wanted to read or write, so the extra message would only be required for servers which allow writing.

Hmm, thinking about this I suspect there's currently a buglet here in that you can probably open a WritableDatabase onto a readonly server, it will just throw an exception if you try to write...

[Mark Hammond]

I understand your concerns.

Hmm, thinking about this I suspect there's currently a buglet here in that you can probably open a WritableDatabase onto a readonly server, it will just throw an exception if you try to write...

There is indeed. A different greeting would almost work - a read-only connection can (basically) ignore the greeting, while a writable connection would check the greeting and throw an exception if it doesn't match. On the other hand, this doesn't solve the more general problem this bug is trying to solve - it would still force you to have 2 servers. IOW, it seems that this solution wouldn't actually solve anything other than to raise an exception slightly earlier than it does now.

The idea of a new request to start a write would seem to work, but may be intrusive - it would mean the server would need to close its writable connection and re-open as read-only. If that was not a problem, it might just work - the client bindings would always try to send the new command when writable, allowing us to detect a mismatch immediately after the handshake. A slight bit of extra latency for writable connections probably isn't too bad, given that connection will be doing lots of work anyway.

Finally, please note that this isn't holding us up in the short-term - 2 servers is a suitable workaround.

[Richard Boulton]

(In #120) Remove the unfixed dependencies so we can close this bug - they're all marked for the 1.1.0 milestone.

[Olly Betts]

Fixed in trunk r12065.