remotetcp tests all fail on MacOS X

[Ivo Jansch]

Hi,

as per the request at the end of the 'make check' script I hereby report my results of a build on mac osx leopard:

xapian-core-1.0.12/tests/.libs/apitest backend remotetcp: 0 tests passed, 152 failed, 3 skipped. xapian-core-1.0.12/tests/.libs/apitest total: 772 tests passed, 152 failed, 26 skipped.

It's not in the list of succesful builds in the INSTALL doc, but it seems to work quite a bit; the only thing that fails tests is the remote backend, which many people won't use anyway.

If you need more details please let me know. I suggest adding osx to the list of succesful builds with the note that the remote backend might fail tests.

[Olly Betts]

The remote backend certainly used to work on OS X, and there's not a good reason why it shouldn't any more.

Please can you run this:

make check > make-check.log

And then attach make-check.log to this ticket.

[Ivo Jansch]

Output of make check on osx

[Ivo Jansch]

Done. From the looks of it, it's the same exception that occurs in all the failed tests.

[James Aylett]

I told Richard about this a while ago. ISTR that we had this problem last time I tried to test, some months back on 10.4; it was either a sequencing issue or something similar, and I never got to the bottom of it.

[Olly Betts]

James: was that with trunk or 1.0?

Anyone: what does the following do:

cd tests
./apitest -bflint termstats
cd ..
echo 'remote 127.0.0.1:1239' > tmp.stub
bin/xapian-tcpsrv --one-shot --interface 127.0.0.1 --port 1239 tests/.flint/db=apitest_simpledata &
examples/quest -d tmp.stub document

[Ivo Jansch]

it fails on the command that starts xapian-tcpsrv:

Starting server on host 127.0.0.1, port 1239 NetworkError: bind failed (Can't assign requested address)

I checked if the localhost interface for some reason is broken but it's fine:

oeliimac:xapian-core-1.0.12 ivo$ ping 127.0.0.1 PING 127.0.0.1 (127.0.0.1): 56 data bytes 64 bytes from 127.0.0.1: icmp_seq=0 ttl=64 time=0.045 ms

Tried it as root to see if it's a privileges issue, same result.

[Olly Betts]

OK, we're getting somewhere at least - there's a problem is binding to the port on localhost for some reason.

Do any of these work any better:

bin/xapian-tcpsrv --one-shot --interface localhost --port 1239 tests/.flint/db=apitest_simpledata

bin/xapian-tcpsrv --one-shot --interface ::1 --port 1239 tests/.flint/db=apitest_simpledata

bin/xapian-tcpsrv --one-shot --port 1239 tests/.flint/db=apitest_simpledata

If not, try them with a different, higher port I guess - say port 54321.

[James Aylett]

The last one, only, works for me with a fairly recent trunk. Now we've got some more details, I remember looking into this and not being able to understand why the Xapian code wasn't looking things up correctly; but also that there's something slightly odd about sockaddr_in and similar on MacOS when I tried to dig deeper.

I'm guessing the problem is that memcpy() isn't doing the right thing in the case of MacOS. I confess I don't really understand why struct hostent works in the way it does, so I may be confusing things here.

[Olly Betts]

Patch which might help

[Olly Betts]

Does the attached patch help?

[James Aylett]

Doesn't seem to; I may be being confused because the dependency tracking seems broken on MacOS, or possibly just on my checkout.

--interface ::1 doesn't resolve, incidentally; I suspect it needs building with a different switch to pull in the ipv6 libraries or something (actually, looks like gethostbyname2() should be used; this seems to be the same on linux using GNU libraries, but no idea what you're supposed to do on other platforms).

I'll try to investigate further tonight.

[James Aylett]

(I take part of that back; dependency tracking is fine, I was just confused.)

[James Aylett]

(And it's not gethostbyname2, it's the RES_USE_INET6 option, according to Stevens.)

[James Aylett]

Okay, solved this. BSDs have a length field in struct sockaddr_in and similar which "currently" should be set to 0. (I'm guessing that on other BSDs it is ignored?) So memset(&addr, 0, sizeof(addr)) on creation and all is well. The changes in your patch aren't needed, because gethostbyname() will not return ip6 addresses unless it's configured using RES_USE_INET6 (Stevens UNPv1 seems clear about this; man pages aren't).

[Olly Betts]

Another patch to test, which also updates tcpclient.cc similarly.

[Olly Betts]

The ::1 was just a "something to try while we're blindly probing for what's going on" guess, but this does raise the point that we don't currently support IPv6 so I've opened #374 for that issue.

[Olly Betts]

jbergstroem on #xapian confirmed attachment:net-tcpserver-and-client.patch​ works on 10.5.7 (and the same failure is present without it) so applied to trunk in r12769 and backported to branches/1.0 in r12770.