Couldn't open /recoll-index/position.baseA: Too many open files

[Percent01]

Hello everybody!

I use Recoll 1.19.14 with XapianDB 1.2.17

I start recoll-webui this command: "nohup python ./webui-standalone.py >/dev/null 2>&1&" I get this error message after !!202!! searching (I test many times and I always get error message after 202. webui search):

---

Sorry, the requested URL '​http://X.X.X.X/results?query=asdf&dir=%3Call%3E&after=&before=&sort=relevancyrating&ascending=0&page=1' caused an error:

Internal Server Error

Exception:

EnvironmentError("Can't open index",)

Traceback:

Traceback (most recent call last): File "/opt/recoll/recoll-webui-master/bottle.py", line 744, in _handle return route.call(args) File "/opt/recoll/recoll-webui-master/bottle.py", line 1479, in wrapper rv = callback(*a, ka) File "/opt/recoll/recoll-webui-master/bottle.py", line 2850, in wrapper result = func(*args, kwargs) File "/opt/recoll/recoll-webui-master/webui.py", line 241, in results res, nres, timer = recoll_search(query) File "/opt/recoll/recoll-webui-master/webui.py", line 180, in recoll_search query = recoll_initsearch(q) File "/opt/recoll/recoll-webui-master/webui.py", line 157, in recoll_initsearch db = recoll.connect(configconfdir) EnvironmentError: Can't open index

---

I get this error message on the console: :2:../rcldb/rcldb.cpp:830:Db::open: exception while opening recoll-index: Error opening table `/recoll-index/position.': Couldn't open /recoll-index/position.baseA: Too many open files Couldn't open /recoll-index/position.baseB: Too many open files

:2:pyrecoll.cpp:1531:Db_init: db open error Traceback (most recent call last): File "/opt/recoll/recoll-webui-master/bottle.py", line 744, in _handle return route.call(args) File "/opt/recoll/recoll-webui-master/bottle.py", line 1479, in wrapper rv = callback(*a, ka) File "/opt/recoll/recoll-webui-master/bottle.py", line 2850, in wrapper result = func(*args, kwargs) File "/opt/recoll/recoll-webui-master/webui.py", line 241, in results res, nres, timer = recoll_search(query) File "/opt/recoll/recoll-webui-master/webui.py", line 180, in recoll_search query = recoll_initsearch(q) File "/opt/recoll/recoll-webui-master/webui.py", line 157, in recoll_initsearch db = recoll.connect(configconfdir) EnvironmentError: Can't open index

---

[Olly Betts]

The issue is what it says - there are too many open files. We need some free file descriptors to open the index, but there are a finite number available - e.g. on modern Linux the limit is 65536. So it sounds like recoll isn't closing files in some case, as it's unlikely it really needs so many files open at once.

This is a recoll issue, not a Xapian one. There's nothing we can really do if there aren't file handles available, short of randomly closing some which wouldn't be helpful overall. You should report this to the recoll author (Jean Francois Dockes).

Closing as notabug (in Xapian).

[Percent01]

There is a similar issue in 2009: ​http://lists.xapian.org/pipermail/xapian-discuss/2009-January/006351.html[]

[Olly Betts]

Please read the whole of that thread - that issue was in C# code, and due to garbage collection delaying the destruction of database objects. And that wasn't a bug in Xapian either - the fix was to call WritableDatabase::close() rather than rely on the object being destroyed in a timely fashion.

I'm fairly sure Recoll is written in C++ not C#, so it's unlikely to be related anyway.