Ticket #222 (assigned enhancement)

Opened 13 months ago

Last modified 3 weeks ago

omindex should make use of O_NOATIME where available

Reported by: olly Owned by: olly
Priority: normal Milestone: 1.1.1
Component: Omega Version: SVN trunk
Severity: minor Keywords:
Cc: Blocked By:
Operating System: All Blocking:

Description (last modified by olly) (diff)

On Linux >= 2.6.8, open() accepts a O_NOATIME flag which is intended for use by "indexing or backup programs". That means us!

I have a patch for this, which I'll attach shortly.

There's a wrinkle though - in some cases O_NOATIME will cause open to fail with EPERM and you need to retry the open call without O_NOATIME:

EPERM The O_NOATIME flag was specified, but the effective user ID of

the caller did not match the owner of the file and the caller was not privileged (CAP_FOWNER).

So for example, if we're indexing /usr/share/doc as a non-root user, we incur an extra syscall for each file - in this case it would be more efficient not to use O_NOATIME at all.

We need to quantify this overhead, and (if it's an issue) look at how to reduce it. One thought I had was, on a per-directory basis, to give up on using O_NOATIME if we failed to open a file using it. Then we only incur one syscall per directory for a read-only tree. Various tweaks to this are possible - e.g. give up for this directory and all subdirectories.

Attachments

omindex-noatime.patch (4.7 kB) - added by olly 13 months ago.
Patch which adds O_NOATIME support

Change History

Changed 13 months ago by olly

Patch which adds O_NOATIME support

Changed 13 months ago by olly

  • owner changed from newbugs to olly

Changed 13 months ago by olly

  • status changed from new to assigned

Changed 13 months ago by trac

  • platform set to All

Changed 3 months ago by olly

  • description modified (diff)

For Unix, we can look at st_dev in struct stat to determine if this is the same FS - O_NOATIME is likely to apply at the FS level.

Changed 3 months ago by olly

  • milestone set to 1.1.0

We should at least consider this for 1.1.0.

Changed 3 weeks ago by olly

  • milestone changed from 1.1.0 to 1.1.1

The patch isn't ready to apply and this isn't an incompatible change so postponing to 1.1.1.

Note: See TracTickets for help on using tickets.