Packaging Xapian

If you're wanting to produce binary packages of Xapian, here are some tips.

• 1.4.x is the current "stable" version. 1.2.x is no longer supported as of 2017-09-26 and 1.0.x is no longer supported as of 2011-01-14. If packaging for a new platform, pick 1.4.x, and if you are still packaging 1.2.x (or 1.0.x), please update to 1.4.x.

• If you package a development release (middle number odd, e.g. 1.3.x), please be aware that between releases there are very likely to be incompatible ABI changes, and possibly also incompatible API, database file format, and network protocol changes. We encourage users to try these versions, but they should be aware that they are doing so and what the drawbacks are. So just asking the package manager to "install xapian" shouldn't install a development release by default.

• xapian-core's testsuite contains some timed tests of scalability which can occasionally failed under uneven load. If you run the testsuite as part an automatic package build, we recommend setting environment variable AUTOMATED_TESTING to 1 which will cause such tests to be skipped.

• Some of the examples (quest and copydatabase) are useful tools in their own right, and probably worth packaging. The "simple" examples are intended to be instructive rather than useful tools (so you probably don't want to package these in /usr/bin or the equivalent on your platform).

• If you're wondering which bindings to package, it seems that Python is the most popular, followed by PHP (but see below), Ruby, and Perl, though relative popularity probably varies by platform.

• Binary packages of the PHP bindings are probably not redistributable (due to the PHP licence being incompatible with the GPL). Our understanding is that source packages are probably OK to redistribute.

• The library ABI and file formats are not affected by the sizes of types such as off_t and time_t (and we will avoid introducing any such dependencies in future), so you can safely enable largefile support (though this should get enabled by default) or switch your packages from 32-bit to 64-bit time_t without affecting dependent packages.

• Xapian >= 1.2.1 (and Xapian 1.0.x for x >= 21) default to using SSE2 maths on x86. This improves performance a little, but the main motivation is correctness - the excess precision in the results of 387 FP instructions can lead to the same calculations in different pieces of code giving slightly different results.

If you need to build a package which still supports x86 processors without SSE2 (for Intel that means prior to Pentium 4; for AMD prior to Athlon 64) then you can configure with --disable-sse, but because this is mostly a correctness issue this should be avoided if at all possible.

If you can rely on at least SSE1, then use --enable-sse=sse. Otherwise if possible it's better to build two versions of the library and install them such that the appropriate version for the processor in use gets loaded (at least on Linux, the dynamic loader has such functionality). The Debian package of Xapian does this like so:

mkdir -p build
case $DEB_HOST_ARCH in
i386|*-i386)
  cd build && ../configure ${configure_flags} --disable-sse
  mkdir -p build-sse2
  cd build-sse2 && ../configure ${configure_flags} --disable-static
  ;;
*)
  cd build && ../configure ${configure_flags}
  ;;
esac
staging=`pwd`/debian
make -C build DESTDIR="$staging/tmp" install
case $DEB_HOST_ARCH in
i386|*-i386)
  make -C build-sse2 DESTDIR="$staging/tmp-sse2" install
  mkdir "$staging/tmp/usr/lib/sse2"
  mv "$staging"/tmp-sse2/usr/lib/libxapian*.so.* "$staging/tmp/usr/lib/sse2"
  rm -rf "$staging/tmp-sse2"
  ;;
esac

And then put the directory tree under $staging/tmp in your package. The dynamic loader will look in /usr/lib/sse2 before /usr/lib if the CPU is SSE2 capable.

• If you find you need to patch Xapian during packaging, please feed the patches back to us. Even if you think the patch isn't the right fix, it's useful to know that there are issues.

• If you're making packages publicly available, feel free to announce them on xapian-discuss. We can add a link from the download page too.