Migrate from XS Search::Xapian to SWIG version

[Olly Betts]

This ticket is to track the issues remaining to solve before the SWIG Perl bindings are a complete replacement for the XS ones.

• ~"$it" doesn't work in SWIG, while in XS it dereferences (for TermIterator and ValueIterator at least). I did some digging last year and it seemed there was some problematic interaction with eq/ne which I didn't get to the bottom of.~
• ~"Stringify" is also missing for subclasses of Error (which in the XS version I'm using in 1.2.4 to give backwards compatibility for cases which were still throwing string exceptions in 1.2.3).~
• ~XS version has simple anon-sub wrappers for MatchDecider and ExpandDecider. SWIG doesn't support directors for Perl in current releases (though there's work on a branch). But perhaps we can support these simple wrappers without needing directors.~
• ~The SWIG-generated version is fussier about what sort of scalars you pass - for example, if an integer is expected, passing a string containing an integer or floating point number doesn't work, at least in some cases.~ Now documented at least.

One option is to deprecate the stringify stuff and make the switch at the start of the next release series.

[Olly Betts]

Doing this for the next release series seems to make most sense. Now we've made a release branch for 1.2.x, I've removed the XS Search::Xapian from trunk.

[Olly Betts]

It would be nice to wrap iterators in a more natural way for Perl - this article gives a good approach which would probably work:

​http://www.perl.com/pub/2005/06/16/iterators.html

[Olly Betts]

Backported r16227

[Olly Betts]

I talked through these issues at a Wellington PM meeting a while ago and have notes from that, but I think this is going to take a while to work through properly.

I'm pondering if we should release 1.4.0 with the Perl bindings clearly labelled as "still experimental", and then resolve this in early 1.4.x.

Moving to 1.3.x for now.

[Olly Betts]

SWIG now supports directors for Perl, though taking a code ref (which would typically be an anonymous subroutine) would be a nicer Perl API for the simple functors.

[Olly Betts]

Postpone until 1.4.1

[Olly Betts]

Search::Xapian 1.2.24.0 will be compatible with xapian-core 1.4.x so this is a little less urgent.

[Olly Betts]

Ticket retargeted after milestone closed

[Olly Betts]

Ticket retargeted after milestone closed

[Olly Betts]

Ticket retargeted after milestone closed

[Eric Wong]

Xapian::PostingIterator lacks "->inc" sub from Search::Xapian::PostingIterator; so I need to rely on "++" which I find confusing (overloading confuses me :x)

Also, I could not find Xapian::ENQ_ASCENDING, since I rely on Search::Xapian::ENQ_ASCENDING.

[Olly Betts]

Xapian::PostingIterator lacks "->inc" sub from Search::Xapian::PostingIterator; so I need to rely on "++" which I find confusing (overloading confuses me :x)

Thanks for reporting. I've pushed fde36e74d72b36b19dea6eb83b70ee1311423c40 to fix that, and will backport for 1.4.16.

[Olly Betts]

I could not find Xapian::ENQ_ASCENDING, since I rely on Search::Xapian::ENQ_ASCENDING.

This is wrapped as $Xapian::Enquire::ASCENDING (which better matches the C++ API).

But we try to provide compatibility with Search::Xapian so I've pushed d7776b1206fb6fd33a715a38b87678e14ecadd3f to add aliases for these.

[Olly Betts]

I debugged what happens when stringify overloads are added back in.

The problem is that SWIG internally adds objects to a hash using the object as the hash key. Perl stringifies the hash key to get a string to hash, and that doesn't work out well if the stringify overloads aren't a unique identifier for the C++ object (which the Search::Xapian overloads aren't).

It'd be better if SWIG didn't rely on stringifying objects like this, but I'm not sure the Search::Xapian stringify overloads are a good idea either.

[Olly Betts]

The convert to integer overload on PositionIterator also seems to cause problems, though I haven't tried to dig into why.

I think these magic implicit method calls on conversions to string and integer are too confusing, and it's better not to support them. This is incompatible with Search::Xapian, but we can document this and it's easy to write code which works with both versions - just make the method call explicit.

[Olly Betts]

We should also document a good way to load whichever of Search::Xapian or Xapian is available, rather than everybody having to invent their own way. Perhaps:

BEGIN {  
    eval {  
        require Xapian; Xapian->import(':all');  
    };  
    if ($@) {  
        require Search::Xapian; Search::Xapian->import(':all');  
    }  
}

[Olly Betts]

I've improved the documentation of the newer bindings and added a compatibility section which shows the require code to load either, and at least document the increased fussiness about string vs integer scalars.

I think the main thing left is wrapping MatchDecider and ExpandDecider to call Perl functions.

[Olly Betts]

I've added support for passing a Perl sub for simple functor classes in d5d26eb54bfb68e1674b8f0e314de29e81e16b1d.

SWIG now supports directors for Perl so we should be able to support subclassing, but accepting a Perl sub as an alternative still seems useful for simple cases (it'd be nice to support for C++ too).

I think the coding work for this ticket is now done (unless further incompatibilities are found), but we should make sure the documentation reflects the updated status of the SWIG-generated Perl bindings.

[Olly Betts]

[b2c5a064bb28088f024ac1526c429e5e0876dee3] updated some wording about the status of these bindings.

I'm working through putting the docs for these on the website too, though probably that won't be completed until the next 1.4.x release as the docs there are built from release tarballs.

I'll also update the getting started guide.