Opened 17 years ago
Closed 19 months ago
#195 closed defect (duplicate)
Writable databases should take a parameter indicating flush threshold.
Reported by: | Richard Boulton | Owned by: | Olly Betts |
---|---|---|---|
Priority: | normal | Milestone: | 2.0.0 |
Component: | Backend-Glass | Version: | git master |
Severity: | normal | Keywords: | |
Cc: | Blocked By: | ||
Blocking: | Operating System: | All |
Description (last modified by )
Possibly, this should be a global parameter (ie, applies to all databases), or maybe it should be a database specific parameter (ie, set as a parameter to the "open" method for chert writable databases).
In any case, the current way of setting a flush threshold (ie, setting an environment variable) is unsatisfactory, due to being difficult to set in some circumstances (or on some OSes), and it being easy for users to forget to export the variable, resulting in bogus bug reports. A parameter to a Xapian function would be a cleaner API for this. However, we intend to improve the handling of automatic flushes in future, such that the count of added documents won't be the crucial factor; instead, amount of memory used will be. We need to ensure we don't add a parameter to the API which will shortly become meaningless.
Change History (14)
comment:1 by , 17 years ago
comment:2 by , 17 years ago
Owner: | changed from | to
---|
There's one issue with this that has just struck me - it's likely to make some bugs harder to reproduce if when we flush depends on how much memory is in use by other processes at the time. I'm not saying we should reject the idea, but perhaps we should try to design in some behavioural stability - e.g. if we check free memory we could round the result to the nearest N megabytes.
comment:3 by , 17 years ago
That's a very good point; rounding will help, though maybe not very much. I suppose we could also introduce an environment variable to specify how much memory to use, to get repeatable test runs.
comment:4 by , 17 years ago
Operating System: | → All |
---|---|
Status: | new → assigned |
comment:6 by , 16 years ago
Component: | Backend-Flint → Backend-Chert |
---|---|
Description: | modified (diff) |
Summary: | Flint writable databases should take a parameter indicating flush threshold. → Chert writable databases should take a parameter indicating flush threshold. |
Changed this bug to be about Chert - we want to put new features like this into Chert rather than Flint now.
comment:7 by , 12 years ago
Component: | Backend-Chert → Backend-Brass |
---|---|
Summary: | Chert writable databases should take a parameter indicating flush threshold. → Writable databases should take a parameter indicating flush threshold. |
Updating to be about brass, and marking to consider for 1.3.x.
comment:8 by , 12 years ago
Milestone: | → 1.3.x |
---|
Oops, I failed to actually set the milestone with my previous comment.
comment:9 by , 11 years ago
Milestone: | 1.3.x → 1.3.3 |
---|
comment:11 by , 10 years ago
Component: | Backend-Brass → Backend-Glass |
---|
comment:12 by , 10 years ago
Milestone: | 1.3.3 → 1.3.4 |
---|
comment:13 by , 9 years ago
Milestone: | 1.3.4 → 1.4.x |
---|
I don't have good answers to the issues here, so postponing.
comment:14 by , 20 months ago
Milestone: | 1.4.x → 2.0.0 |
---|---|
Version: | SVN trunk → git master |
comment:15 by , 19 months ago
Resolution: | → duplicate |
---|---|
Status: | assigned → closed |
I just spotted we also have #266 which covers this and more, so closing this ticket as a duplicate.
More thoughts:
Perhaps the parameter should be modifiable for an existing WritableDatabase object - sometimes usage patterns change for an existing object and it would be annoying to have to close and reopen just to change the parameter.
And ideally we probably want a "how aggressive to be" measure - e.g. from 0 for "minimise resource usage" to 1 (or 10 or 100 or 1000 or whatever) for "this machine only exists to service your every whim".