Opened 9 years ago

Closed 9 years ago

Last modified 9 years ago

#259 closed defect (fixed)

DatabaseError when interleaving reads and writes

Reported by: richard Owned by: richard
Priority: high Milestone: 1.0.7
Component: Backend-Flint Version: SVN trunk
Severity: normal Keywords:
Cc: Blocked By:
Blocking: Operating System: All


Alessandro Pasotti reported a problem to xapian-discuss on 23 April 2008, where some PHP code, using the clustering branch of xapian, was performing an indexing run and then a "cluster" operation on the same WritableDatabase? object without an intermediate flush. The cluster operation returned DatabaseError?: inflate failed (invalid distance too far back). He later sent me a testcase by private email which I was able to use to track down the problem, and work out a smaller testcase.

I was quickly able to reproduce the problem on SVN HEAD, and have now worked out a minimal testcase which reproduces the same problem, in a different way.

I have narrowed down the problem to the "next_for_sequential" method in FlintTable. This is called (via next()) by FlintTable.read_tag(), when reading the subsequent components of a multipiece tag, and should never return false. (We should probably have a check in FlintTable.read_tag() that next() doesn't return false, which raises DatabaseCorruptError? if it does).

However, when the next piece of a tag is in a later block than the current block, and that new block has been newly added (since the last flush, or the database was opened), next_for_sequential incorrectly returns false. If I comment out the call to next_for_sequential in next(), so that next_default() is always used instead, the code works correctly.

I believe the problem is that next_for_sequential checks whether a block is in the database by comparing its number with base.get_last_block(). However, the last_block member in base is only updated when a flush is called, so doesn't reflect newly added blocks. Thush, next_for_sequential returns false incorrectly.

This bug affects both Flint and Chert databases.

Attachments (1)

patch (612 bytes) - added by richard 9 years ago.
Testcase which reproduces the problem

Download all attachments as: .zip

Change History (3)

Changed 9 years ago by richard

Testcase which reproduces the problem

comment:1 Changed 9 years ago by richard

  • Resolution set to fixed
  • Status changed from new to closed

Fixed in SVN HEAD in revision [10401]. I'm applying a backport to the 1.0 branch, and will update the clustering branch with recent changes in HEAD shortly to check that the fix works for the original testcase, too.

comment:2 Changed 9 years ago by richard

Applied to branch1.0 now.

Note: See TracTickets for help on using tickets.