xapian-bindings for php / dl(): Unable to load dynamic library

[Felix Ostmann]

I have a strange problem with the xapian-bindings for php with a debian wheezy.

When i build as a user with

xapian-core: ./configure --prefix=/opt/myproject/xapian-core

xapian-bindings: ./configure --with-php --prefix=/opt/myproject/xapian-bindings-php PHP_EXTENSION_DIR=/opt/myproject/php-extensions XAPIAN_CONFIG=/opt/myproject/xapian-core/bin/xapian-config

all is fine and a make check is fine for xapian-bindings.

$ make check
Making check in php
make[1]: Entering directory `/home/user/myproject/build/xapian-bindings-1.2.19/php'
make  check-am
make[2]: Entering directory `/home/user/myproject/build/xapian-bindings-1.2.19/php'
make  check-TESTS
make[3]: Entering directory `/home/user/myproject/build/xapian-bindings-1.2.19/php'
PASS: smoketest.php
=============
1 test passed
=============
make[3]: Leaving directory `/home/user/myproject/build/xapian-bindings-1.2.19/php'
make[2]: Leaving directory `/home/user/myproject/build/xapian-bindings-1.2.19/php'
make[1]: Leaving directory `/home/user/myproject/build/xapian-bindings-1.2.19/php'
make[1]: Entering directory `/home/user/myproject/build/xapian-bindings-1.2.19'
make[1]: Leaving directory `/home/user/myproject/build/xapian-bindings-1.2.19'

The ldd looks fine for this:

$ ldd /home/user/myproject/build/xapian-bindings-1.2.19/php/.libs/xapian.so 
	linux-vdso.so.1 =>  (0x00007fff32bff000)
	libxapian.so.22 => /opt/myproject/xapian-core/lib/libxapian.so.22 (0x00007ffc95cfc000)
	libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x00007ffc95ade000)
	libuuid.so.1 => /lib/x86_64-linux-gnu/libuuid.so.1 (0x00007ffc958d8000)
	libstdc++.so.6 => /usr/lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007ffc955d1000)
	libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007ffc9534f000)
	libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007ffc94fc2000)
	libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007ffc94dac000)
	/lib64/ld-linux-x86-64.so.2 (0x00007ffc9636f000)

$ ls -l /opt/myproject/xapian-core/lib/libxapian.so.22*
lrwxrwxrwx 1 user user       19 Oct 28 16:53 /opt/myproject/xapian-core/lib/libxapian.so.22 -> libxapian.so.22.6.6
-rwxr-xr-x 1 user user 26646198 Oct 28 16:53 /opt/myproject/xapian-core/lib/libxapian.so.22.6.6

But when i want it build as root without special prefix (defaults to /usr/local), i get the following error:

Making check in php
make[1]: Entering directory `/root/build/xapian-bindings-1.2.19/php'
make  check-am
make[2]: Entering directory `/root/build/xapian-bindings-1.2.19/php'
make  check-TESTS
make[3]: Entering directory `/root/build/xapian-bindings-1.2.19/php'
/root/build/xapian-bindings-1.2.19/php/php5/xapian.php:22: dl(): Unable to load dynamic library '/root/build/xapian-bindings-1.2.19/php/.libs/xapian.so' - /root/build/xapian-bindings-1.2.19/php/.libs/xapian.so: undefined symbol: _ZN6Xapian13TermGenerator19set_max_word_lengthEj
FAIL: smoketest.php
=======================================
1 of 1 test failed
Please report to http://xapian.org/bugs
=======================================
make[3]: *** [check-TESTS] Error 1
make[3]: Leaving directory `/root/build/xapian-bindings-1.2.19/php'
make[2]: *** [check-am] Error 2
make[2]: Leaving directory `/root/build/xapian-bindings-1.2.19/php'
make[1]: *** [check] Error 2
make[1]: Leaving directory `/root/build/xapian-bindings-1.2.19/php'
make: *** [check-recursive] Error 1

I tried with providing XAPIAN_CONFIG=/usr/local/bin/xapian-config, building as user and only make install as root, using 1.2.18, removing the system xapian-config or many other things. Nothing worked.

Here are some infos (like requested in #71):

# ldd /root/build/xapian-bindings-1.2.19/php/.libs/xapian.so 
	linux-vdso.so.1 =>  (0x00007fff50bff000)
	libxapian.so.22 => /usr/lib/libxapian.so.22 (0x00007f9cad87a000)
	libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x00007f9cad663000)
	libuuid.so.1 => /lib/x86_64-linux-gnu/libuuid.so.1 (0x00007f9cad45d000)
	libstdc++.so.6 => /usr/lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007f9cad156000)
	libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f9caced4000)
	libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f9cacb47000)
	libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007f9cac931000)
	/lib64/ld-linux-x86-64.so.2 (0x00007f9cadef6000)

# ls -l /usr/lib/libxapian.so*
lrwxrwxrwx 1 root root      19 Dec 11  2012 /usr/lib/libxapian.so.22 -> libxapian.so.22.5.0
-rw-r--r-- 1 root root 1984552 Dec 11  2012 /usr/lib/libxapian.so.22.5.0

And after some searching i find the correct libxapian.so, but they are ignored.

# ls -l /usr/local/lib/libxapian.so*
lrwxrwxrwx 1 root staff       19 Oct 28 16:35 /usr/local/lib/libxapian.so -> libxapian.so.22.6.6
lrwxrwxrwx 1 root staff       19 Oct 28 16:35 /usr/local/lib/libxapian.so.22 -> libxapian.so.22.6.6
-rwxr-xr-x 1 root staff 26646166 Oct 28 16:35 /usr/local/lib/libxapian.so.22.6.6

xapian-config tell the correct location:

# xapian-config --libs
-L/usr/local/lib -lxapian

When i delete the /usr/lib/libxapian.so* files and retry configure & make, ldd tells me, that he can't find libxapian.so (not found). I also tried with LIBS=/usr/local/lib, still not found.

# ldd php/.libs/xapian.so | grep xapian
	libxapian.so.22 => not found

[Felix Ostmann]

The only way i get this to work is to copy the libxapian.so* files from /usr/local/lib to /usr/lib. But there are already the system libxapian.so* (different version).

[Olly Betts]

When i delete the /usr/lib/libxapian.so* files and retry configure & make, ldd tells me, that he can't find libxapian.so (not found).

You'll probably need to run ldconfig (as root) to regenerate the run time linker's cache. This usually gets done automatically by libtool when you install, but you'll need to do it by hand if you delete installed shared libraries by hand.

I have a wheezy VM to hand, so I'll try to reproduce this.

[Felix Ostmann]

OK, thanks for the hint. I tried again with calling ldconfig after make install in xapian-core and all works fine.

Sorry for not finding this problem by myself. At least i was on the right way.

There was a hint about ld & -LLIBDIR (etc) while running make install, but i don't use ldconfig often. Perhaps a better hint will help? Additional there was a call to ldconfig:

---

libtool: finish: PATH="/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/sbin" ldconfig -n /usr/local/lib

---

----------------------------------------------------------------------
Libraries have been installed in:
   /usr/local/lib

If you ever happen to want to link against installed libraries
in a given directory, LIBDIR, you must either use libtool, and
specify the full pathname of the library, or use the `-LLIBDIR'
flag during linking and do at least one of the following:
   - add LIBDIR to the `LD_LIBRARY_PATH' environment variable
     during execution
   - add LIBDIR to the `LD_RUN_PATH' environment variable
     during linking
   - use the `-Wl,-rpath -Wl,LIBDIR' linker flag
   - have your system administrator add LIBDIR to `/etc/ld.so.conf'

See any operating system documentation about shared libraries for
more information, such as the ld(1) and ld.so(8) manual pages.
----------------------------------------------------------------------

[Olly Betts]

The issue isn't that you need to run ldconfig after make install, but that you need to run it after manually changing what libraries are installed where, as you did when you deleted /usr/lib/libxapian.so* earlier. In fact, make install runs ldconfig for you, but restricted to the directory which the new library gets installed in - that's /usr/local/lib in this case, so it wouldn't fix up the cache for your manual changes in /usr/lib.

The advice you paste comes from libtool rather than our own code, so it's not easy for us to change, but it looks fine to me as-is. Since xapian-bindings builds with libtool, it's not really relevant for this situation anyway.

As for the original error, on my Debian wheezy VM, I build xapian-core 1.2.19 from source as a non-root user, then make install as root, and then build xapian-bindings 1.2.19 from source, configuring, building and running the tests like so:

./configure --with-php XAPIAN_CONFIG=/usr/local/bin/xapian-config
make
make check

Then I don't see the failure you describe - the tests pass. Perhaps I'm missing some step, or perhaps in your experimenting you got things set up oddly (like the situation manually running ldconfig fixed).

If you're able to supply a series of steps to reproduce the failure, I'm happy to investigate further.

[Felix Ostmann]

Perhaps the problem is our system. It's a squeeze install with wheezy upgrade. libxapian22 is installed (dependency for aptitude i guess). Additional there is a xapian-config in /usr/bin from an old non-debian installation under squeeze.

# dpkg -l | grep xapian
ii  libxapian22                          1.2.12-2                      amd64        Search engine library

I tried to reproduce this on another debian-wheezy-system without success. It simple build and works like a charm.

Sorry for wasting your time.

I close this ticket as invalid.

[Olly Betts]

Yeah, the libxapian22 package gets installed by default (it's on my system too).

The old xapian-config in /usr/bin shouldn't be an issue (since you specified the right one via XAPIAN_CONFIG), but perhaps other remnants of that install are causing issues.

I'm not sure "invalid" is exactly right, as ideally this should always just work. But without a way to reproduce it, it's going to be hard to work out what's going on.