xapian-omega can't find non-system iconv installations

[Ryan Carsten Schmidt]

Hello, I'm investigating the possibility of ​adding xapian-omega to MacPorts. I used the --with-iconv configure flag, which resulted in the error message:

checking for library containing iconv... no
configure: error: iconv not found, but --with-iconv specified

The config.log says:

configure:17416: checking for library containing iconv
configure:17447: ccache /usr/bin/clang++ -o conftest -pipe -Os -stdlib=libc++ -arch x86_64 -arch i386 -std=gnu++11 -I/opt/local/include -I/opt/local/include -L/opt/local/lib -Wl,-headerpad_max_install_names -arch x86_64 -arch i386 conftest.cpp  >&5
Undefined symbols for architecture x86_64:
  "_iconv", referenced from:
      _main in conftest-5ad1ad.o
ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)

and then:

configure:17447: ccache /usr/bin/clang++ -o conftest -pipe -Os -stdlib=libc++ -arch x86_64 -arch i386 -std=gnu++11 -I/opt/local/include -I/opt/local/include -L/opt/local/lib -Wl,-headerpad_max_install_names -arch x86_64 -arch i386 conftest.cpp -liconv   >&5
Undefined symbols for architecture x86_64:
  "_iconv", referenced from:
      _main in conftest-519ed3.o
ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)

So it tried to find iconv both with and without -liconv and failed both times. libiconv is installed with MacPorts so it should have succeeded when using -liconv.

The reason it failed is that it is searching only for a symbol called iconv. Only a primary libiconv installation, such as that provided by an operating system vendor (and configured with LIBICONV_PLUG set), will have the symbol iconv. But in any secondary copies of libiconv, such as the one provided by MacPorts, Fink, Homebrew, or manually compiled by the user, the symbol will instead be called libiconv. Other symbols in secondary libiconv installations are similarly mangled. This symbol mangling is intentional, so that a user-supplied copy of libiconv does not interfere with a system-provided possibly older version of libiconv or even a non-libiconv iconv implementation.

As long as you #include <iconv.h> this symbol renaming is transparent to you. So you probably only need to modify the configure script to include that header in that test. I'm not an autotools expert and don't know the best way to accomplish that. There is an iconv.m4 file provided by gettext which you might be able to look at for ideas.

[Olly Betts]

Thanks for the iconv.m4 tip - it looks like we may just be able to use that. My only reservation is it apparently isn't well compartmentalised and seems to require dragging in a load of gettext-specific stuff we really don't need.

[Olly Betts]

With a bit of hackery I've managed to reduce the bloat from AM_ICONV, so let's try going with that.

The only thing we were doing which AM_ICONV doesn't have something equivalent to is adding -L/sw/lib to ICONV_LDFLAGS if Fink is detected, but it sounds like that probably doesn't actually work nowadays anyway (presumably it did back in 2006 when that special case was added).

So this should be fixed on git master by 043d0a514ef261d8ab897ce6a4ac78aa381c5b86. I'm not sure if the macOS CI build needs adjusting to benefit from this change (that uses homebrew, and it looks like it currently selects the system iconv).

Marking for backporting to 1.4.x once we've checked portability.

[Ryan Carsten Schmidt]

Replying to olly:

With a bit of hackery I've managed to reduce the bloat from AM_ICONV, so let's try going with that.

Thanks, I'll give it a try.

The only thing we were doing which AM_ICONV doesn't have something equivalent to is adding -L/sw/lib to ICONV_LDFLAGS if Fink is detected, but it sounds like that probably doesn't actually work nowadays anyway (presumably it did back in 2006 when that special case was added).

Thanks for removing the /sw stuff. I was going to report that separately, since of course when building in MacPorts we don't want it to go looking for things in Fink.

[Olly Betts]

Ah yes, we should probably allow for having Fink and another package system installed on the same machine.

I've made the travis macOS build use the homebrew libiconv by installing libiconv from homebrew and passing --with-libiconv-prefix=/usr/local/opt/libiconv to configure, which seems to work:

​https://travis-ci.org/xapian/xapian/jobs/488703766#L2916

Does this also work for you on macports?

BTW, if you want to add a macports-based build to the travis config please do - that would help catch regressions sooner.

[Olly Betts]

I'd really like confirmation that this actually works for you, but I don't want to delay 1.4.10 any further and having checked these changes work on Linux and macOS with homebrew I think this is worth including in 1.4.10 as-is, so I've backported in d715130f7d777e2ac4bef812fde3e33094745c03 and subsequent commits.