Opened 17 years ago

Closed 16 years ago

Last modified 16 years ago

#111 closed defect (released)

omindex should impose resource limits on filter programs

Reported by: Olly Betts Owned by: Olly Betts
Priority: normal Milestone:
Component: Omega Version: SVN trunk
Severity: normal Keywords:
Cc: Blocked By:
Blocking: #120 Operating System: All

Description

It would be a good idea to have some mechanism to limit the CPU and memory that a filter program can use - it's better to skip a problematic file than have the indexer hang or worse render the server unusable if a filter program should enter an infinite loop or consume excessive amounts of memory.

setrlimit() allows limiting CPU and memory usage, but somehow we need to determine a sane memory limit. sysconf(_SC_AVPHYS_PAGES) seems suitable on Linux at least.

Attachments (2)

omindex-limit-filter-resource-usage.patch (3.8 KB ) - added by Olly Betts 17 years ago.
First attempt at a patch
omindex-limit-ram-usage.patch (6.2 KB ) - added by Olly Betts 17 years ago.
Patch to limit RAM usage

Download all attachments as: .zip

Change History (7)

comment:1 by Olly Betts, 17 years ago

Status: newassigned

Solaris has sysconf(_SC_AVPHYS_PAGES) too:

http://bama.ua.edu/cgi-bin/man-cgi?sysconf+3C

We need to multiply it by sysconf(_SC_PAGESIZE) to get bytes, but being careful that the value doesn't overflow.

This wiki page may be useful:

http://www.net-snmp.org/wiki/index.php/Memory_HAL

Perhaps we should use something like:

max(X% of total physical memory, free physical memory)

comment:2 by Olly Betts, 17 years ago

Blocking: 120 added

Seems suitable for 1.0.x.

comment:3 by Olly Betts, 17 years ago

Partly committed to SVN - filters are now limited to 5 minutes of CPU time.

by Olly Betts, 17 years ago

Patch to limit RAM usage

comment:4 by Olly Betts, 17 years ago

attachments.isobsolete: 01

comment:5 by Olly Betts, 16 years ago

Resolution: fixed
Status: assignedclosed

I've committed this patch (with an extra doxygen comment). It's not growing implementations for other platforms just sitting as an attachment here...

comment:6 by Olly Betts, 16 years ago

Operating System: All
Resolution: fixedreleased
Note: See TracTickets for help on using tickets.