Opened 13 years ago

Closed 8 years ago

#566 closed enhancement (fixed)

Merge learning to rank branch

Reported by: Richard Boulton Owned by: Olly Betts
Priority: normal Milestone: 1.5.0
Component: Library API Version:
Severity: normal Keywords:
Cc: parth Blocked By:
Blocking: Operating System: All

Description (last modified by Olly Betts)

We should work on merging the letor branch to trunk, to ensure it doesn't bitrot and get lost.

To get an overview of changes on the letor branch, see https://github.com/xapian/xapian/compare/master...svn%2Fgsoc2011-parth

Several things need to be done. Initially, by parth:

  • Done: add the letor/dir_contents file, so the build system works.
  • Done: remove letor/svm.h; the installed svm.h should be used instead.
  • Done: add the formulae to the docs/letor.rst file (in latex form, we can worry about making them display nicely later).
  • Done: add a description of which versions of libsvm should work to the docs/letor.rst file.
  • Done: remove the include of letor.h from xapian.h, and add a comment to the top of letor.h to indicate that it is not part of the official stable API.

At this point, parth can submit his code to google, and the gsoc project is finished.

Then, by olly or me:

  • Done: merge recent changes in trunk to the branch
  • Done: merge recent changes in trunk to the branch
  • Done: add detection of libsvm.h to configure.ac
  • not relevant now xapian-letor is a separate module: make the build skip letor if svm.h isn't available
  • make the code conform to the coding conventions we use in the rest of the code (largely done)
  • add automated tests (parth will try to do this)
  • Done: anything else required to merge the branch to trunk
  • check that at least one buildbot is building the letor code.

I've marked this as milestone 1.3.0 so it doesn't get forgotten, but consider that a soft milestone.

Change History (15)

comment:1 by James Aylett, 13 years ago

What's the minimum version of libsvm required? 2.91 is in Ubuntu 11.04, but it's not the latest by any means. (Even assuming it's the same libsvm.) 2.85 is in Debian oldstable, and I've put it on atreus, but I suspect that's way out of date…

in reply to:  1 comment:2 by parth, 13 years ago

Replying to james:

What's the minimum version of libsvm required? 2.91 is in Ubuntu 11.04, but it's not the latest by any means. (Even assuming it's the same libsvm.) 2.85 is in Debian oldstable, and I've put it on atreus, but I suspect that's way out of date…

libsvm-3.1 was used during the development of the Letor Project. By the way, my system works fine with system installed libsvm-3.0-2 too.

Parth.

comment:3 by Richard Boulton, 13 years ago

Description: modified (diff)

The version of libsvm to use should be noted in the docs/letor.rst file.

comment:4 by Olly Betts, 12 years ago

Milestone: 1.3.01.3.x

Currently the 1.3.0 milestone is really "stuff to address before 1.4.0" so I've created 1.3.x for that role leaving 1.3.0 for stuff to really do for 1.3.0.

comment:5 by Olly Betts, 12 years ago

Milestone: 1.3.x1.3.2

Pencilling in for 1.3.2.

comment:6 by Olly Betts, 12 years ago

Component: OtherLibrary API
Description: modified (diff)

I've updated the description to mark what has been done, and to note a few more things to do.

I wonder if it would make more sense to make this a separate top-level module (e.g. xapian-letor), rather than an optionally built part of xapian-core?

comment:7 by Olly Betts, 12 years ago

Parth was in favour of a separate xapian-letor module (either on IRC or comments in melange).

It also avoids having to make libsvm a requirement for all users, or having an --enable-letor configure option and forcing packagers choose whether to enable it or not.

comment:8 by Olly Betts, 12 years ago

Description: modified (diff)

I've now split off a xapian-letor top-level module.

comment:9 by Olly Betts, 12 years ago

Cc: parth added
Milestone: 1.3.21.3.1
Owner: changed from parth to Olly Betts
Status: newassigned

Merged the xapian-letor module to trunk in r16580, and working on avoiding needing to patch omindex.

Claiming this ticket, as I'm actively working on it.

comment:10 by Olly Betts, 12 years ago

Description: modified (diff)

I've added detection of libsvm to configure, and also cleaned up a lot of the deviations from the coding conventions now.

comment:11 by Olly Betts, 11 years ago

Milestone: 1.3.11.3.2

I'm not at all clear what the current state of things is with the letor branch, but it's time we got 1.3.1 out.

comment:12 by Olly Betts, 10 years ago

Milestone: 1.3.21.3.3

comment:13 by Olly Betts, 9 years ago

Milestone: 1.3.31.3.4

We're not in GSoC 2015, so this isn't a blocker for more letor projects this year.

comment:14 by Olly Betts, 8 years ago

Milestone: 1.3.41.4.x

It's time to get brutal for 1.4.0 - as a new module this could be introduced in a stable point release without problems.

comment:15 by Olly Betts, 8 years ago

Milestone: 1.4.x1.5.0
Resolution: fixed
Status: assignedclosed

Thanks to this year's GSoC project the letor code is now on git master so "Merge learning to rank branch" is done, so we can close this ticket.

I think at this stage we don't try to backport this to 1.4.x, but instead focus on getting it into great shape for the next stable release series, which we're aiming to release much sooner after 1.4.0 than 1.4.0 was after 1.2.0.

Note: See TracTickets for help on using tickets.