Opened 12 years ago

Closed 6 years ago

#426 closed defect (fixed)

Chert should write all base files, then flush all base files

Reported by: Olly Betts Owned by: Olly Betts
Priority: normal Milestone: 1.2.19
Component: Backend-Chert Version: SVN trunk
Severity: normal Keywords:
Cc: oliver.g.charles@… Blocked By:
Blocking: Operating System: Mac OS X

Description

OS X supports the F_FULLSYNC ioctl, other systems don't (currently, at least AFAIK). Xapian uses F_FULLSYNC if it is available to ensure that changes have actually made it to disk:

http://trac.xapian.org/ticket/288

On other systems, it uses fdatasync() or fsync() which ensure that the data has at least left the OS, but it may only be in the controller or drive cache for potentially several seconds more.

Currently Xapian writes the base file for the table, then syncs it, then does the next table. I bet it would be more efficient to write them all and then sync them all, especially with F_FULLSYNC (hence I'm marking this as "Mac OS X").

I'll take a look at doing that. It shouldn't be hard.

Change History (9)

comment:1 by Richard Boulton, 12 years ago

Currently, Xapian iterates through the tables, calling a sync method first on the .DB file, then on the base file.

We should probably be syncing at two points per transaction: once after writing all the changes to the DB files and writing all the new base files except for the record table, and once after writing the record table base file.

comment:2 by Richard Boulton, 12 years ago

I don't think this is a critical problem, or affects the API / ABI, so could probably be punted to 1.2.x.

comment:3 by Olly Betts, 12 years ago

Milestone: 1.1.41.2.x

Indeed.

comment:4 by acid2, 12 years ago

Hi, just like to add a little bit of information about my system here. I use notmuchmail.org with svn HEAD (at time of writing) of xapian-core. By disabling F_FULLSYNC with this regex:

perl -pi -e 's/^#ifdef F_FULLFSYNC/#if 0/' backends/*/*_io.h

The time to complete a basic operation drops by an order of magnitude from 0.7s to 0.06s. If there is any actual useful information I can give you, please let me know - I'm happy to provide anything I can, or be a test case for patches.

Last edited 8 years ago by Olly Betts (previous) (diff)

comment:5 by acid2, 12 years ago

Cc: oliver.g.charles@… added

comment:6 by Olly Betts, 12 years ago

Note that you shouldn't extrapolate from the overhead on a "basic operation".

The drive's writeback buffer is a fixed size, so the time to write out all the data from it to the disk platters is bounded, so the delay incurred by each commit by using F_FULLFSYNC is likely to be at most a few seconds even if you're adding a million documents to the database in that commit.

So the overhead of the F_FULLFSYNC is going to be proportionally much worse for a "basic operation" (changing a single tag and committing it) than when changes are batched up. And batching up changes is recommended for other reasons if you want good indexing performance.

comment:7 by Olly Betts, 12 years ago

Component: Backend-BrassBackend-Chert
Status: newassigned
Summary: Backends should write all base files, then flush all base filesChert and flint should write all base files, then flush all base files

Brass now only has a single base file for all tables, and syncs all the table and then the base file after writing them all (at least on the brass-btree branch, as of r14086), so marking this ticket for chert.

comment:8 by Olly Betts, 7 years ago

Summary: Chert and flint should write all base files, then flush all base filesChert should write all base files, then flush all base files

A few relevant updates:

As of 1.2.19 (and 1.3.2), we only full sync once per commit, which should still give the same guarantees as full syncing all the files (unless you move the tables elsewhere and symlink them in, or similar odd cases).

1.3.2 defaults to not using F_FULLSYNC, and provides a Xapian::DB_FULL_SYNC flag which can be passed to get full-sync where available (which seems to be only OS X currently).

And this is only a performance issue, so I don't think it makes sense to change flint at this point - 1.2.x defaults to chert, and supports flint mainly to allow people to easily migrate.

I would be interested to know how 1.2.19 or later performs on OS X in practice.

comment:9 by Olly Betts, 6 years ago

Milestone: 1.2.x1.2.19
Resolution: fixed
Status: assignedclosed

Close to 8 months on and no explicit feedback sadly.

But I've also not heard any complaints about commit performance, so I'm going to assume the changes in 1.2.19 helped, and I'm going to mark this as fixed (especially as it has definitely been addressed for the glass backend in 1.3.x). If anyone is still seeing issues, please comment with details and we can reopen.

Note: See TracTickets for help on using tickets.