Ticket #481: geospatial_clean1.patch

xapian-core/docs/geospatial.rst

@@ +1 @@
+.. Copyright (C) 2008 Lemur Consulting Ltd
+
+================================
+Geospatial searching with Xapian
+================================
+
+.. contents:: Table of contents
+
+Introduction
+============
+
+This document describes a set of features present in Xapian which are designed
+to allow geospatial searches to be supported.  Currently, the geospatial
+support allows sets of locations to be stored associated with each document, as
+latitude/longitude coordinates, and allows searches to be restricted or
+reordered on the basis of distance from a second set of locations.
+
+Three types of geospatial searches are supported:
+
+ - Returning a list of documents in order of distance from a query location.
+   This may be used in conjunction with any Xapian query.
+
+ - Returning a list of documents within a given distance of a query location.
+   This may be used in conjunction with any other Xapian query, and with any
+   Xapian sort order.
+
+ - Returning a set of documents in a combined order based on distance from a
+   query location, and relevance.
+
+Locations are stored in value slots, allowing multiple independent locations to
+be used for a single document.  It is also possible to store multiple
+coordinates in a single value slot, in which case the closest coordinate will
+be used for distance calculations.
+
+Metrics
+=======
+
+A metric is a function which calculates the distance between two points.
+
+Calculating the exact distance between two geographical points is an involved
+subject.  In fact, even defining the meaning of a geographical point is very
+hard to do precisely.  Xapian leaves the system used to define latitude and
+longitude up to the user: the important thing, as far as Xapian is concerned,
+is that all the points it's given use the same system.
+
+Since there are lots of ways of calculating distances between two points, using
+different assumptions about the approximations which are valid, Xapian allows
+user-implemented metrics.  These are subclasses of the Xapian::LatLongMetric
+class; see the API documentation for details on how to implement the various
+required methods.
+
+There is currently only one built-in metric - the GreatCircleMetric.  As the
+name suggests, this calculates the distance between a latitude and longitude
+based on the assumption that the world is a perfect sphere.  The radius of the
+world can be specified as a constructor parameter, but defaults to a reasonable
+approximation of the radius of the Earth.  The calculation uses the Haversine
+formula, which is accurate for points which are close together, but can have
+significant error for coordinates which are on opposite sides of the sphere: on
+the other hand, such points are likely to be at the end of a ranked list of
+search results, so this probably doesn't matter.
+
+Indexing
+========
+
+To index a set of documents with location, you need to store serialised
+latitude-longitude coordinates in a value slot in your documents.  To do this,
+use the LatLongCoord class.  For example, this is how you might store a
+latitude and longitude corresponding to "London" in value slot 0::
+
+  Xapian::Document doc;
+  doc.add_value(0, Xapian::LatLongCoord(51.53, 0.08).serialise());
+
+Of course, often a location is a bit more complicated than a single point - for
+example, postcode regions in the UK can cover a fairly wide area.  If a search
+were to treat such a location as a single point, the distances returned could
+be incorrect by as much as a couple of miles.  Xapian therefore allows you to
+store a set of points in a single slot - the distance calculation will return
+the distance to the closest of these points.  This is often a good enough work
+around for this problem - if you require greater accuracy, you will need to
+filter the results after they are returned from Xapian.
+
+To store multiple coordinates in a single slot, use the LatLongCoords class::
+
+  Xapian::Document doc;
+  Xapian::LatLongCoords coords;
+  coords.insert(Xapian::LatLongCoord(51.53, 0.08));
+  coords.insert(Xapian::LatLongCoord(51.51, 0.07));
+  coords.insert(Xapian::LatLongCoord(51.52, 0.09));
+  doc.add_value(0, coords.serialise());
+
+(Note that the serialised form of a LatLongCoords object containing a single
+coordinate is exactly the same as the serialised form of the corresponding
+LatLongCoord object.)
+
+Searching
+=========
+
+Sorting results by distance
+---------------------------
+
+If you simply want your results to be returned in order of distance, you can
+use the LatLongDistanceKeyMaker class to calculate sort keys.  For example, to
+return results in order of distance from the coordinate (51.00, 0.50), based on
+the values stored in slot 0, and using the great-circle distance::
+
+  Xapian::Database db("my_database");
+  Xapian::Enquire enq(db);
+  enq.set_query(Xapian::Query("my_query"));
+  GreatCircleMetric metric;
+  LatLongCoord centre(51.00, 0.50);
+  Xapian::LatLongDistanceKeyMaker keymaker(0, centre, metric);
+  enq.set_sort_by_key(keymaker, False);
+
+Filtering results by distance
+-----------------------------
+
+To return only those results within a given distance, you can use the
+LatLongDistancePostingSource.  For example, to return only those results within
+5 miles of coordinate (51.00, 0.50), based on the values stored in slot 0, and
+using the great-circle distance::
+
+  Xapian::Database db("my_database");
+  Xapian::Enquire enq(db);
+  Xapian::Query q("my_query");
+  GreatCircleMetric metric;
+  LatLongCoord centre(51.00, 0.50);
+  double max_range = Xapian::miles_to_metres(5);
+  Xapian::LatLongDistancePostingSource ps(0, centre, metric, max_range)
+  q = Xapian::Query(Xapian::Query::OP_FILTER, q, Xapian::Query(ps));
+  enq.set_query(q);
+
+Ranking results on a combination of distance and relevance
+----------------------------------------------------------
+
+To return results ranked by a combination of their relevance and their
+distance, you can also use the LatLongDistancePostingSource.  Beware that
+getting the right balance of weights is tricky: there is little solid
+theoretical basis for this, so the best approach is often to try various
+different parameters, evalutate the results, and settle on the best.  The
+LatLongDistancePostingSource returns a weight of 1.0 for a document which is at
+the specified location, and a lower, but always positive, weight for points
+further away. It has two parameters, k1 and k2, which control how fast the
+weight decays, which can be specified to the constructor (but aren't in this
+example) - see the API documentation for details of these parameters.::
+
+  Xapian::Database db("my_database");
+  Xapian::Enquire enq(db);
+  Xapian::Query q("my_query");
+  GreatCircleMetric metric;
+  LatLongCoord centre(51.00, 0.50);
+  double max_range = Xapian::miles_to_metres(5);
+  Xapian::LatLongDistancePostingSource ps(0, centre, metric, max_range)
+  q = Xapian::Query(Xapian::Query::AND, q, Xapian::Query(ps));
+  enq.set_query(q);
+
+
+Performance
+===========
+
+The location information associated with each document is stored in a document
+value.  This allows it to be looked up quickly at search time, so that the
+exact distance from the query location can be calculated.  However, this method
+requires that the distance of each potential match is checked, which can be
+expensive.
+
+Some experimental code exists to produce terms corresponding to a hierarchical
+index of locations (using the O-QTM algorithm - see references below), which
+can be used to narrow down the search so that only a small number of potential
+matches need to be checked.  Contact the Xapian developers (on email or IRC) if
+you would like to help finish and test this code.
+
+It is entirely possible that a more efficient implementation could be performed
+using "R trees" or "KD trees" (or one of the many other tree structures used
+for geospatial indexing - see http://en.wikipedia.org/wiki/Spatial_index for a
+list of some of these).  However, using the QTM approach will require minimal
+effort and make use of the existing, and well tested, Xapian database.
+Additionally, by simply generating special terms to restrict the search, the
+existing optimisations of the Xapian query parser are taken advantage of.
+
+References
+==========
+
+The O-QTM algorithm is described in "Dutton, G. (1996). Encoding and handling
+geospatial data with hierarchical triangular meshes. In Kraak, M.J. and
+Molenaar, M. (eds.)  Advances in GIS Research II. London: Taylor & Francis,
+505-518." , a copy of which is available from
+http://www.spatial-effects.com/papers/conf/GDutton_SDH96.pdf
+
+Some of the geometry needed to calculate the correct set of QTM IDs to cover a
+particular region is detailed in
+ftp://ftp.research.microsoft.com/pub/tr/tr-2005-123.pdf
+
+Also, see:
+http://www.sdss.jhu.edu/htm/doc/c++/htmInterface.html

xapian-core/geospatial/Makefile.mk

@@ +1 @@
+EXTRA_DIST += \
+        geospatial/dir_contents \
+        geospatial/Makefile
+
+lib_src += \
+        geospatial/latlongcoord.cc \
+        geospatial/latlong_distance_keymaker.cc \
+        geospatial/latlong_metrics.cc \
+        geospatial/latlong_posting_source.cc

xapian-core/geospatial/latlong_distance_keymaker.cc

@@ +1 @@
+/** \file latlong_distance_keymaker.cc
+ * \brief LatLongDistanceKeyMaker implementation.
+ */
+/* Copyright 2008 Lemur Consulting Ltd
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License as
+ * published by the Free Software Foundation; either version 2 of the
+ * License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write to the Free Software
+ * Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA  02110-1301
+ * USA
+ */
+
+#include <config.h>
+
+#include "xapian/geospatial.h"
+#include "xapian/document.h"
+#include "xapian/queryparser.h" // For sortable_serialise.
+
+using namespace Xapian;
+
+std::string
+LatLongDistanceKeyMaker::operator()(const Document &doc) const
+{
+    std::string val(doc.get_value(valno));
+    LatLongCoords doccoords = LatLongCoords::unserialise(val);
+    if (doccoords.empty()) {
+        return defkey;
+    }
+    double distance = (*metric)(centre, doccoords);
+    return sortable_serialise(distance);
+}
+
+LatLongDistanceKeyMaker::~LatLongDistanceKeyMaker()
+{
+    delete metric;
+}

xapian-core/geospatial/latlong_posting_source.cc

@@ +1 @@
+/** @file latlong_posting_source.cc
+ * @brief LatLongPostingSource implementation.
+ */
+/* Copyright 2008 Lemur Consulting Ltd
+ * Copyright 2010 Richard Boulton
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License as
+ * published by the Free Software Foundation; either version 2 of the
+ * License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write to the Free Software
+ * Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA  02110-1301
+ * USA
+ */
+
+#include <config.h>
+
+#include "xapian/geospatial.h"
+
+#include "xapian/document.h"
+#include "xapian/error.h"
+#include "xapian/registry.h"
+
+#include "serialise.h"
+#include "serialise-double.h"
+#include "utils.h"
+
+#include <math.h>
+
+namespace Xapian {
+
+static double
+weight_from_distance(double dist, double k1, double k2)
+{
+    return k1 * pow(dist + k1, -k2);
+}
+
+void
+LatLongDistancePostingSource::calc_distance()
+{
+    std::string val(*value_it);
+    LatLongCoords coords = LatLongCoords::unserialise(val);
+    dist = (*metric)(centre, coords);
+}
+
+LatLongDistancePostingSource::LatLongDistancePostingSource(
+        valueno slot_,
+        const LatLongCoords & centre_,
+        const LatLongMetric * metric_,
+        double max_range_,
+        double k1_,
+        double k2_)
+        : ValuePostingSource(slot_),
+          centre(centre_),
+          metric(metric_),
+          max_range(max_range_),
+          k1(k1_),
+          k2(k2_)
+{
+    if (k1 <= 0)
+        throw InvalidArgumentError(
+            "k1 parameter to LatLongDistancePostingSource must be greater "
+            "than 0; was " + om_tostring(k1));
+    if (k2 <= 0)
+        throw InvalidArgumentError(
+            "k2 parameter to LatLongDistancePostingSource must be greater "
+            "than 0; was " + om_tostring(k2));
+    set_maxweight(weight_from_distance(0, k1, k2));
+}
+
+LatLongDistancePostingSource::LatLongDistancePostingSource(
+        valueno slot_,
+        const LatLongCoords & centre_,
+        const LatLongMetric & metric_,
+        double max_range_,
+        double k1_,
+        double k2_)
+        : ValuePostingSource(slot_),
+          centre(centre_),
+          metric(metric_.clone()),
+          max_range(max_range_),
+          k1(k1_),
+          k2(k2_)
+{
+    if (k1 <= 0)
+        throw InvalidArgumentError(
+            "k1 parameter to LatLongDistancePostingSource must be greater "
+            "than 0; was " + om_tostring(k1));
+    if (k2 <= 0)
+        throw InvalidArgumentError(
+            "k2 parameter to LatLongDistancePostingSource must be greater "
+            "than 0; was " + om_tostring(k2));
+    set_maxweight(weight_from_distance(0, k1, k2));
+}
+
+LatLongDistancePostingSource::~LatLongDistancePostingSource()
+{
+    delete metric;
+}
+
+void
+LatLongDistancePostingSource::next(weight min_wt)
+{
+    ValuePostingSource::next(min_wt);
+
+    while (value_it != db.valuestream_end(slot)) {
+        calc_distance();
+        if (max_range == 0 || dist <= max_range)
+            break;
+        ++value_it;
+    }
+}
+
+void
+LatLongDistancePostingSource::skip_to(docid min_docid,
+                                      weight min_wt)
+{
+    ValuePostingSource::skip_to(min_docid, min_wt);
+
+    while (value_it != db.valuestream_end(slot)) {
+        calc_distance();
+        if (max_range == 0 || dist <= max_range)
+            break;
+        ++value_it;
+    }
+}
+
+bool
+LatLongDistancePostingSource::check(docid min_docid,
+                                    weight min_wt)
+{
+    if (!ValuePostingSource::check(min_docid, min_wt)) {
+        // check returned false, so we know the document is not in the source.
+        return false;
+    }
+    if (value_it == db.valuestream_end(slot)) {
+        // return true, since we're definitely at the end of the list.
+        return true;
+    }
+
+    calc_distance();
+    if (max_range > 0 && dist > max_range) {
+        return false;
+    }
+    return true;
+}
+
+weight
+LatLongDistancePostingSource::get_weight() const
+{
+    return weight_from_distance(dist, k1, k2);
+}
+
+LatLongDistancePostingSource *
+LatLongDistancePostingSource::clone() const
+{
+    return new LatLongDistancePostingSource(slot, centre,
+                                            metric->clone(),
+                                            max_range, k1, k2);
+}
+
+std::string
+LatLongDistancePostingSource::name() const
+{
+    return std::string("Xapian::LatLongDistancePostingSource");
+}
+
+std::string
+LatLongDistancePostingSource::serialise() const
+{
+    std::string serialised_centre = centre.serialise();
+    std::string metric_name = metric->name();
+    std::string serialised_metric = metric->serialise();
+
+    std::string result = encode_length(slot);
+    result += encode_length(serialised_centre.size());
+    result += serialised_centre;
+    result += encode_length(metric_name.size());
+    result += metric_name;
+    result += encode_length(serialised_metric.size());
+    result += serialised_metric;
+    result += serialise_double(max_range);
+    result += serialise_double(k1);
+    result += serialise_double(k2);
+    return result;
+}
+
+LatLongDistancePostingSource *
+LatLongDistancePostingSource::unserialise_with_registry(const std::string &s,
+                                             const Registry & registry) const
+{
+    const char * p = s.data();
+    const char * end = p + s.size();
+
+    valueno new_slot = decode_length(&p, end, false);
+    size_t len = decode_length(&p, end, true);
+    std::string new_serialised_centre(p, len);
+    p += len;
+    len = decode_length(&p, end, true);
+    std::string new_metric_name(p, len);
+    p += len;
+    len = decode_length(&p, end, true);
+    std::string new_serialised_metric(p, len);
+    p += len;
+    double new_max_range = unserialise_double(&p, end);
+    double new_k1 = unserialise_double(&p, end);
+    double new_k2 = unserialise_double(&p, end);
+    if (p != end) {
+        throw NetworkError("Bad serialised LatLongDistancePostingSource - junk at end");
+    }
+
+    LatLongCoords new_centre = 
+            LatLongCoords::unserialise(new_serialised_centre);
+
+    const Xapian::LatLongMetric * metric_type =
+            registry.get_lat_long_metric(new_metric_name);
+    if (metric_type == NULL) {
+        throw InvalidArgumentError("LatLongMetric " + new_metric_name +
+                                   " not registered");
+    }
+    LatLongMetric * new_metric =
+            metric_type->unserialise(new_serialised_metric);
+
+    return new LatLongDistancePostingSource(new_slot, new_centre,
+                                            new_metric,
+                                            new_max_range, new_k1, new_k2);
+}
+
+void
+LatLongDistancePostingSource::init(const Database & db_)
+{
+    ValuePostingSource::init(db_);
+    if (max_range > 0.0) {
+        // Possible that no documents are in range.
+        termfreq_min = 0;
+        // Note - would be good to improve termfreq_est here, too, but
+        // I can't think of anything we can do with the information
+        // available.
+    }
+}
+
+std::string
+LatLongDistancePostingSource::get_description() const
+{
+    return "Xapian::LatLongDistancePostingSource(slot=" + om_tostring(slot) + ")";
+}
+
+}

xapian-core/geospatial/latlong_metrics.cc

@@ +1 @@
+/** \file latlong_metrics.cc
+ * \brief Geospatial distance metrics.
+ */
+/* Copyright 2008 Lemur Consulting Ltd
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License as
+ * published by the Free Software Foundation; either version 2 of the
+ * License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write to the Free Software
+ * Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA  02110-1301
+ * USA
+ */
+
+#include <config.h>
+
+#include "xapian/geospatial.h"
+#include "xapian/error.h"
+#include "serialise-double.h"
+
+#include <math.h>
+
+using namespace Xapian;
+
+/** Quadratic mean radius of the earth in metres.
+ */
+#define QUAD_EARTH_RADIUS_METRES 6372797.6
+
+/** Set M_PI if it's not already set.
+ */
+#ifndef M_PI
+#define M_PI 3.14159265358979323846
+#endif
+
+LatLongMetric::~LatLongMetric()
+{
+}
+
+double
+LatLongMetric::operator()(const LatLongCoords & a, const LatLongCoords &b) const
+{
+    if (a.empty() || b.empty()) {
+        throw InvalidArgumentError("Empty coordinate list supplied to LatLongMetric::operator()().");
+    }
+    double min_dist = 0.0;
+    bool have_min = false;
+    for (std::set<LatLongCoord>::const_iterator a_iter = a.begin();
+         a_iter != a.end();
+         ++a_iter)
+    {
+        for (std::set<LatLongCoord>::const_iterator b_iter = b.begin();
+             b_iter != b.end();
+             ++b_iter)
+        {
+            double dist = operator()(*a_iter, *b_iter);
+            if (!have_min) {
+                min_dist = dist;
+                have_min = true;
+            } else if (dist < min_dist) {
+                min_dist = dist;
+            }
+        }
+    }
+    return min_dist;
+}
+
+
+GreatCircleMetric::GreatCircleMetric()
+        : radius(QUAD_EARTH_RADIUS_METRES)
+{}
+
+GreatCircleMetric::GreatCircleMetric(double radius_)
+        : radius(radius_)
+{}
+
+double
+GreatCircleMetric::operator()(const LatLongCoord & a,
+                              const LatLongCoord & b) const
+{
+    double lata = a.latitude * (M_PI / 180.0);
+    double latb = b.latitude * (M_PI / 180.0);
+
+    double latdiff = lata - latb;
+    double longdiff = (a.longitude - b.longitude) * (M_PI / 180.0);
+
+    double sin_half_lat = sin(latdiff / 2);
+    double sin_half_long = sin(longdiff / 2);
+    double h = sin_half_lat * sin_half_lat +
+            sin_half_long * sin_half_long * cos(lata) * cos(latb);
+    double sqrt_h = sqrt(h);
+    if (sqrt_h > 1.0) sqrt_h = 1.0;
+    return 2 * radius * asin(sqrt_h);
+}
+
+LatLongMetric *
+GreatCircleMetric::clone() const
+{
+    return new GreatCircleMetric(radius);
+}
+
+std::string 
+GreatCircleMetric::name() const
+{
+    return "Xapian::GreatCircleMetric";
+}
+
+std::string 
+GreatCircleMetric::serialise() const
+{
+    return serialise_double(radius);
+}
+
+LatLongMetric * 
+GreatCircleMetric::unserialise(const std::string & s) const
+{
+    const char * p = s.data();
+    const char * end = p + s.size();
+
+    double new_radius = unserialise_double(&p, end);
+    if (p != end) {
+        throw Xapian::NetworkError("Bad serialised GreatCircleMetric - junk at end");
+    }
+
+    return new GreatCircleMetric(new_radius);
+}

xapian-core/geospatial/dir_contents

@@ +1 @@
+<Directory>geospatial</Directory>
+
+<Description>
+Support for geospatial matching, and parsing of locations.
+</Description>

xapian-core/geospatial/latlongcoord.cc

@@ +1 @@
+/** \file latlong.cc
+ * \brief Latitude and longitude representations.
+ */
+/* Copyright 2008 Lemur Consulting Ltd
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License as
+ * published by the Free Software Foundation; either version 2 of the
+ * License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write to the Free Software
+ * Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA  02110-1301
+ * USA
+ */
+
+#include <config.h>
+
+#include "xapian/geospatial.h"
+#include "xapian/error.h"
+
+#include "serialise.h"
+#include "serialise-double.h"
+#include "str.h"
+
+#include <math.h>
+
+using namespace Xapian;
+
+LatLongCoord::LatLongCoord(double latitude_, double longitude_)
+        : latitude(latitude_),
+          longitude(longitude_)
+{
+    if (latitude < -90.0 || latitude > 90.0)
+        throw InvalidArgumentError("Latitude out-of-range");
+    longitude = fmod(longitude_, 360);
+    if (longitude <= -180) longitude += 360;
+    if (longitude > 180) longitude -= 360;
+    if (longitude == -0.0) longitude = 0.0;
+}
+
+LatLongCoord
+LatLongCoord::unserialise(const std::string & serialised)
+{
+    const char * ptr = serialised.data();
+    const char * end = ptr + serialised.size();
+    LatLongCoord result = unserialise(&ptr, end);
+    if (ptr != end)
+        throw InvalidArgumentError(
+                "Junk found at end of serialised LatLongCoord");
+    return result;
+}
+
+LatLongCoord
+LatLongCoord::unserialise(const char ** ptr, const char * end)
+{
+    try {
+        // This will raise NetworkError for invalid serialisations.
+        double latitude = unserialise_double(ptr, end);
+        double longitude = unserialise_double(ptr, end);
+        return LatLongCoord(latitude, longitude);
+    } catch (const NetworkError & e) {
+        // FIXME - modify unserialise_double somehow so we don't have to catch
+        // and rethrow the exceptions it raises.
+        throw InvalidArgumentError(e.get_msg());
+    }
+}
+
+std::string
+LatLongCoord::serialise() const
+{
+    std::string result(serialise_double(latitude));
+    result += serialise_double(longitude);
+    return result;
+}
+
+std::string
+LatLongCoord::get_description() const
+{
+    std::string res("Xapian::LatLongCoord(");
+    res += str(latitude);
+    res += ", ";
+    res += str(longitude);
+    res += ")";
+    return res;
+}
+
+LatLongCoords
+LatLongCoords::unserialise(const std::string & serialised)
+{
+    const char * ptr = serialised.data();
+    const char * end = ptr + serialised.size();
+    LatLongCoords coords = unserialise(ptr, end);
+    return coords;
+}
+
+LatLongCoords
+LatLongCoords::unserialise(const char * ptr, const char * end)
+{
+    LatLongCoords result;
+    try {
+        while (ptr != end) {
+            // This will raise NetworkError for invalid serialisations (so we
+            // catch and re-throw it).
+            result.coords.insert(LatLongCoord::unserialise(&ptr, end));
+        }
+    } catch (const NetworkError & e) {
+        // FIXME - modify unserialise_double somehow so we don't have to catch
+        // and rethrow the exceptions it raises.
+        throw InvalidArgumentError(e.get_msg());
+    }
+    if (ptr != end)
+        throw InvalidArgumentError(
+                "Junk found at end of serialised LatLongCoords");
+    return result;
+}
+
+std::string
+LatLongCoords::serialise() const
+{
+    std::string result;
+    std::set<LatLongCoord>::const_iterator coord;
+    for (coord = coords.begin(); coord != coords.end(); ++coord)
+    {
+        result += serialise_double(coord->latitude);
+        result += serialise_double(coord->longitude);
+    }
+    return result;
+}
+
+std::string
+LatLongCoords::get_description() const
+{
+    std::string res("Xapian::LatLongCoords(");
+    std::set<LatLongCoord>::const_iterator coord;
+    for (coord = coords.begin(); coord != coords.end(); ++coord) {
+        if (coord != coords.begin()) {
+            res += ", ";
+        }
+        res += "(";
+        res += str(coord->latitude);
+        res += ", ";
+        res += str(coord->longitude);
+        res += ")";
+    }
+    res += ")";
+    return res;
+}

xapian-core/geospatial/Makefile

@@ +1 @@
+# Makefile for use in directories built by non-recursive make.
+
+SHELL = /bin/sh
+
+all check:
+        cd .. && $(MAKE) $@
+
+clean:
+        rm -f *.o *.obj *.lo

xapian-core/tests/api_geospatial.cc

@@ +1 @@
+/** @file api_geospatial.cc
+ * @brief Tests of geospatial functionality.
+ */
+/* Copyright 2008 Lemur Consulting Ltd
+ * Copyright 2010 Richard Boulton
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License as
+ * published by the Free Software Foundation; either version 2 of the
+ * License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write to the Free Software
+ * Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA  02110-1301
+ * USA
+ */
+
+#include <config.h>
+#include "api_geospatial.h"
+#include <xapian/geospatial.h>
+#include <xapian/error.h>
+
+#include "apitest.h"
+#include "testsuite.h"
+#include "testutils.h"
+#include <iomanip>
+
+using namespace std;
+using namespace Xapian;
+
+// #######################################################################
+// # Tests start here
+
+static void
+builddb_coords1(Xapian::WritableDatabase &db, const string &)
+{
+    Xapian::LatLongCoord coord1(10, 10);
+    Xapian::LatLongCoord coord2(20, 10);
+    Xapian::LatLongCoord coord3(30, 10);
+
+    Xapian::Document doc;
+    doc.add_value(0, coord1.serialise());
+    db.add_document(doc);
+
+    doc = Xapian::Document();
+    doc.add_value(0, coord2.serialise());
+    db.add_document(doc);
+
+    doc = Xapian::Document();
+    doc.add_value(0, coord3.serialise());
+    db.add_document(doc);
+}
+
+/// Test behaviour of the LatLongPostingSource
+DEFINE_TESTCASE(latlongpostingsource1, backend && writable && !remote && !inmemory) {
+    Xapian::Database db = get_database("coords1", builddb_coords1, "");
+    Xapian::LatLongCoord coord1(10, 10);
+    Xapian::LatLongCoord coord2(20, 10);
+    Xapian::LatLongCoord coord3(30, 10);
+
+    // Chert doesn't currently support opening a value iterator for a writable database.
+    SKIP_TEST_FOR_BACKEND("chert");
+
+    Xapian::GreatCircleMetric metric;
+    Xapian::LatLongCoords centre;
+    centre.insert(coord1);
+    double coorddist = metric(coord1, coord2);
+    TEST_EQUAL_DOUBLE(coorddist, metric(coord2, coord3));
+
+    // Test a search with no range restriction.
+    {
+        Xapian::LatLongDistancePostingSource ps(0, centre, metric);
+        ps.init(db);
+
+        ps.next(0.0);
+        TEST_EQUAL(ps.at_end(), false);
+        TEST_EQUAL_DOUBLE(ps.get_weight(), 1.0);
+        TEST_EQUAL(ps.get_docid(), 1);
+
+        ps.next(0.0);
+        TEST_EQUAL(ps.at_end(), false);
+        TEST_EQUAL_DOUBLE(ps.get_weight(), 1000.0 / (1000.0 + coorddist));
+        TEST_EQUAL(ps.get_docid(), 2);
+
+        ps.next(0.0);
+        TEST_EQUAL(ps.at_end(), false);
+        TEST_EQUAL_DOUBLE(ps.get_weight(), 1000.0 / (1000.0 + coorddist * 2));
+        TEST_EQUAL(ps.get_docid(), 3);
+
+        ps.next(0.0);
+        TEST_EQUAL(ps.at_end(), true);
+    }
+
+    // Test a search with a tight range restriction
+    {
+        Xapian::LatLongDistancePostingSource ps(0, centre, metric, coorddist * 0.5);
+        ps.init(db);
+
+        ps.next(0.0);
+        TEST_EQUAL(ps.at_end(), false);
+        TEST_EQUAL_DOUBLE(ps.get_weight(), 1.0);
+
+        ps.next(0.0);
+        TEST_EQUAL(ps.at_end(), true);
+    }
+
+    // Test a search with a looser range restriction
+    {
+        Xapian::LatLongDistancePostingSource ps(0, centre, metric, coorddist);
+        ps.init(db);
+
+        ps.next(0.0);
+        TEST_EQUAL(ps.at_end(), false);
+        TEST_EQUAL_DOUBLE(ps.get_weight(), 1.0);
+
+        ps.next(0.0);
+        TEST_EQUAL(ps.at_end(), false);
+        TEST_EQUAL_DOUBLE(ps.get_weight(), 1000.0 / (1000.0 + coorddist));
+        TEST_EQUAL(ps.get_docid(), 2);
+
+        ps.next(0.0);
+        TEST_EQUAL(ps.at_end(), true);
+    }
+
+    // Test a search with a looser range restriction, but not enough to return
+    // the next document.
+    {
+        Xapian::LatLongDistancePostingSource ps(0, centre, metric, coorddist * 1.5);
+        ps.init(db);
+
+        ps.next(0.0);
+        TEST_EQUAL(ps.at_end(), false);
+        TEST_EQUAL_DOUBLE(ps.get_weight(), 1.0);
+
+        ps.next(0.0);
+        TEST_EQUAL(ps.at_end(), false);
+        TEST_EQUAL_DOUBLE(ps.get_weight(), 1000.0 / (1000.0 + coorddist));
+        TEST_EQUAL(ps.get_docid(), 2);
+
+        ps.next(0.0);
+        TEST_EQUAL(ps.at_end(), true);
+    }
+
+    // Test a search with a loose enough range restriction that all docs should
+    // be returned.
+    {
+        Xapian::LatLongDistancePostingSource ps(0, centre, metric, coorddist * 2.5);
+        ps.init(db);
+
+        ps.next(0.0);
+        TEST_EQUAL(ps.at_end(), false);
+        TEST_EQUAL_DOUBLE(ps.get_weight(), 1.0);
+
+        ps.next(0.0);
+        TEST_EQUAL(ps.at_end(), false);
+        TEST_EQUAL_DOUBLE(ps.get_weight(), 1000.0 / (1000.0 + coorddist));
+        TEST_EQUAL(ps.get_docid(), 2);
+
+        ps.next(0.0);
+        TEST_EQUAL(ps.at_end(), false);
+        TEST_EQUAL_DOUBLE(ps.get_weight(), 1000.0 / (1000.0 + coorddist * 2));
+        TEST_EQUAL(ps.get_docid(), 3);
+
+        ps.next(0.0);
+        TEST_EQUAL(ps.at_end(), true);
+    }
+
+    return true;
+}
+
+// Test various methods of LatLongCoord and LatLongCoords
+DEFINE_TESTCASE(latlongcoords1, !backend) {
+    LatLongCoord c1(0, 0);
+    LatLongCoord c2(1, 0);
+    LatLongCoord c3(1, 0);
+
+    // Test comparison
+    TEST_NOT_EQUAL(c1.get_description(), c2.get_description());
+    TEST(c1 < c2 || c2 < c1);
+    TEST_EQUAL(c2.get_description(), c3.get_description());
+    TEST(!(c2 < c3) && !(c3 < c2));
+
+    // Test serialisation
+    std::string s1 = c1.serialise();
+    LatLongCoord c4 = LatLongCoord::unserialise(s1);
+    TEST(!(c1 < c4 || c4 < c1));
+    const char * ptr = s1.data();
+    const char * end = ptr + s1.size();
+    c4 = LatLongCoord::unserialise(&ptr, end);
+    TEST_EQUAL(c1.get_description(), c4.get_description());
+    TEST_EQUAL(c1.get_description(), "Xapian::LatLongCoord(0, 0)");
+    TEST_EQUAL(ptr, end);
+
+    // Test building a set of LatLongCoords
+    LatLongCoords g1(c1);
+    TEST(!g1.empty());
+    TEST_EQUAL(g1.size(), 1);
+    TEST_EQUAL(g1.get_description(), "Xapian::LatLongCoords((0, 0))");
+    g1.insert(c2);
+    TEST_EQUAL(g1.size(), 2);
+    TEST_EQUAL(g1.get_description(), "Xapian::LatLongCoords((0, 0), (1, 0))");
+    // c3 == c2, so already in the set, so no change if we add c3
+    g1.insert(c3);
+    TEST_EQUAL(g1.size(), 2);
+    TEST_EQUAL(g1.get_description(), "Xapian::LatLongCoords((0, 0), (1, 0))");
+    TEST_EQUAL(g1.get_description(), "Xapian::LatLongCoords((0, 0), (1, 0))");
+    g1.erase(c3);
+    TEST_EQUAL(g1.get_description(), "Xapian::LatLongCoords((0, 0))");
+
+    // Test building an empty LatLongCoords
+    LatLongCoords g2;
+    TEST(g2.empty());
+    TEST_EQUAL(g2.size(), 0);
+    TEST_EQUAL(g2.get_description(), "Xapian::LatLongCoords()");
+
+    return true;
+}
+
+// Test various methods of LatLongMetric
+DEFINE_TESTCASE(latlongmetric1, !backend) {
+    LatLongCoord c1(0, 0);
+    LatLongCoord c2(1, 0);
+    Xapian::GreatCircleMetric m1;
+    double d1 = m1(c1, c2);
+    TEST_REL(d1, >, 111226.0);
+    TEST_REL(d1, <, 111227.0);
+
+    // Let's make another metric, this time using the radius of mars, so
+    // distances should be quite a bit smaller.
+    Xapian::GreatCircleMetric m2(3310000);
+    double d2 = m2(c1, c2);
+    TEST_REL(d2, >, 57770.0);
+    TEST_REL(d2, <, 57771.0);
+
+    // Check serialise and unserialise.
+    Xapian::Registry registry;
+    std::string s1 = m2.serialise();
+    const Xapian::LatLongMetric * m3;
+    m3 = registry.get_lat_long_metric(m2.name());
+    TEST(m3 != NULL);
+    m3 = m3->unserialise(s1);
+    double d3 = (*m3)(c1, c2);
+    TEST_EQUAL_DOUBLE(d2, d3);
+
+    delete m3;
+
+    return true;
+}
+
+// Test a LatLongDistanceKeyMaker directly.
+DEFINE_TESTCASE(latlongkeymaker1, !backend) {
+    Xapian::GreatCircleMetric m1(3310000);
+    LatLongCoord c1(0, 0);
+    LatLongCoord c2(1, 0);
+    LatLongCoord c3(2, 0);
+    LatLongCoord c4(3, 0);
+
+    LatLongCoords g1(c1);
+    g1.insert(c2);
+
+    LatLongDistanceKeyMaker keymaker(0, g1, m1);
+    Xapian::Document doc1;
+    doc1.add_value(0, g1.serialise());
+    Xapian::Document doc2;
+    doc2.add_value(0, c3.serialise());
+    Xapian::Document doc3;
+    doc3.add_value(0, c4.serialise());
+    Xapian::Document doc4;
+
+    std::string k1 = keymaker(doc1);
+    std::string k2 = keymaker(doc2);
+    std::string k3 = keymaker(doc3);
+    std::string k4 = keymaker(doc4);
+    TEST_REL(k1, <, k2);
+    TEST_REL(k2, <, k3);
+    TEST_REL(k3, <, k4);
+
+    LatLongDistanceKeyMaker keymaker2(0, g1, m1, 0);
+    std::string k3b = keymaker2(doc3);
+    std::string k4b = keymaker2(doc4);
+    TEST_EQUAL(k3, k3b);
+    TEST_REL(k3b, >, k4b);
+
+    return true;
+}

xapian-core/tests/Makefile.am

@@ -155 +155 @@
  api_unicode.cc \
  api_valuestats.cc \
  api_valuestream.cc \
+ api_geospatial.cc \
  api_wrdb.cc
 
 apitest_SOURCES = apitest.cc dbcheck.cc $(collated_apitest_sources) \

xapian-core/include/xapian/postingsource.h

@@ -31 +31 @@
 
 namespace Xapian {
 
+class Registry;
+
 /** Base class which provides an "external" source of postings.
  *
  *  Warning: the PostingSource interface is currently experimental, and is
@@ -285 +287 @@
      */
     virtual PostingSource * unserialise(const std::string &s) const;
 
+    /** Create object given string serialisation returned by serialise().
+     *
+     *  Note that the returned object will be deallocated by Xapian after use
+     *  with "delete".  It must therefore have been allocated with "new".
+     *
+     *  This method is supplied with a Registry object, which can be used when
+     *  unserialising objects contained within the posting source.  The default
+     *  implementation simply calls unserialise() which doesn't take the
+     *  Registry object, so you do not need to implement this method unless you
+     *  want to take advantage of the Registry object when unserialising.
+     *
+     *  @param s A serialised instance of this PostingSource subclass.
+     */
+    virtual PostingSource * unserialise_with_registry(const std::string &s,
+                                      const Registry & registry) const;
+
     /** Set this PostingSource to the start of the list of postings.
      *
      *  This is called automatically by the matcher prior to each query being

xapian-core/include/xapian/registry.h

@@ -30 +30 @@
 namespace Xapian {
 
 // Forward declarations.
+class LatLongMetric;
 class MatchSpy;
 class PostingSource;
 class Weight;
@@ -105 +106 @@
      */
     const Xapian::MatchSpy *
             get_match_spy(const std::string & name) const;
+
+    /// Register a user-defined lat-long metric class.
+    void register_lat_long_metric(const Xapian::LatLongMetric &metric);
+
+    /** Get a lat-long metric given a name.
+     *
+     *  The returned metric is owned by the registry object.
+     *
+     *  Returns NULL if the metric could not be found.
+     */
+    const Xapian::LatLongMetric *
+            get_lat_long_metric(const std::string & name) const;
+
 };
 
 }

xapian-core/include/xapian/geospatial.h

@@ +1 @@
+/** @file geospatial.h
+ * @brief Geospatial search support routines.
+ */
+/* Copyright 2008,2009 Lemur Consulting Ltd
+ * Copyright 2010 Richard Boulton
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License as
+ * published by the Free Software Foundation; either version 2 of the
+ * License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write to the Free Software
+ * Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA  02110-1301
+ * USA
+ */
+
+#ifndef XAPIAN_INCLUDED_GEOSPATIAL_H
+#define XAPIAN_INCLUDED_GEOSPATIAL_H
+
+#include <xapian/enquire.h>
+#include <xapian/postingsource.h>
+#include <xapian/queryparser.h> // For sortable_serialise
+#include <xapian/keymaker.h>
+#include <xapian/visibility.h>
+#include <string>
+#include <set>
+
+namespace Xapian {
+
+class Registry;
+
+/** Convert from miles to metres.
+ */
+inline XAPIAN_VISIBILITY_DEFAULT double
+miles_to_metres(double miles)
+{
+    return 1609.344 * miles;
+}
+
+/** Convert from metres to miles.
+ */
+inline XAPIAN_VISIBILITY_DEFAULT double
+metres_to_miles(double metres)
+{
+    return metres * (1.0 / 1609.344);
+}
+
+/** Convert from feet to metres.
+ */
+inline XAPIAN_VISIBILITY_DEFAULT double
+feet_to_metres(double feet)
+{
+    return 0.3048 * feet;
+}
+
+/** Convert from metres to feet.
+ */
+inline XAPIAN_VISIBILITY_DEFAULT double
+metres_to_feet(double metres)
+{
+    return metres * (1.0 / 0.3048);
+}
+
+/** Convert from nautical miles to metres.
+ */
+inline XAPIAN_VISIBILITY_DEFAULT double
+nautical_miles_to_metres(double nautical_miles)
+{
+    return 1852.0 * nautical_miles;
+}
+
+/** Convert from metres to nautical miles.
+ */
+inline XAPIAN_VISIBILITY_DEFAULT double
+metres_to_nautical_miles(double metres)
+{
+    return metres * (1.0 / 1852.0);
+}
+
+/** A latitude-longitude coordinate.
+ *
+ *  Note that latitude-longitude coordinates are only precisely meaningful if
+ *  the datum used to define them is specified.  This class ignores this
+ *  issue - it is up to the caller to ensure that the datum used for each
+ *  coordinate in a system is consistent.
+ */
+struct XAPIAN_VISIBILITY_DEFAULT LatLongCoord {
+  public:
+    /** A latitude, as decimal degrees.
+     *
+     *  Should be in the range -90 <= longitude <= 90
+     *
+     *  Postive latitudes represent the northern hemisphere.
+     */
+    double latitude;
+
+    /** A longitude, as decimal degrees.
+     *
+     *  Should be in the range -180 < latitude <= 180
+     *
+     *  Positive longitudes represent the eastern hemisphere.
+     */
+    double longitude;
+
+    /** Construct a coordinate.
+     *
+     *  If the supplied longitude is out of range, an exception will be raised.
+     *
+     *  If the supplied latitude is out of range, it will be normalised to the
+     *  appropriate range.
+     */
+    LatLongCoord(double latitude_, double longitude_);
+
+    /** Construct a coordinate by unserialising a string.
+     *
+     *  @param serialised the string to unserialise the coordinate from.
+     *
+     *  @exception Xapian::InvalidArgumentError if the string does not contain
+     *  a valid serialised latitude-longitude pair, or contains extra data at
+     *  the end of it.
+     */
+    static LatLongCoord unserialise(const std::string & serialised);
+
+    /** Construct a coordinate by unserialising a string.
+     *
+     *  The string may contain further data after that for the coordinate.
+     *
+     *  @param ptr A pointer to the start of the string.  This will be updated
+     *  to point to the end of the data representing the coordinate.
+     *  @param end A pointer to the end of the string.
+     *
+     *  @exception Xapian::InvalidArgumentError if the string does not contain
+     *  a valid serialised latitude-longitude pair.
+     */
+    static LatLongCoord unserialise(const char ** ptr, const char * end);
+
+    /** Return a serialised representation of the coordinate.
+     */
+    std::string serialise() const;
+
+    /** Compare with another LatLongCoord.
+     */
+    bool operator<(const LatLongCoord & other) const
+    {
+        if (latitude < other.latitude) return true;
+        return (longitude < other.longitude);
+    }
+
+    /// Return a string describing this object.
+    std::string get_description() const;
+};
+
+/** A set of latitude-longitude coordinate.
+ */
+class XAPIAN_VISIBILITY_DEFAULT LatLongCoords {
+    /// The coordinates.
+    std::set<LatLongCoord> coords;
+
+  public:
+    std::set<LatLongCoord>::const_iterator begin() const
+    {
+        return coords.begin();
+    }
+
+    std::set<LatLongCoord>::const_iterator end() const
+    {
+        return coords.end();
+    }
+
+    size_t size() const
+    {
+        return coords.size();
+    }
+
+    size_t empty() const
+    {
+        return coords.empty();
+    }
+
+    void insert(const LatLongCoord & coord)
+    {
+        coords.insert(coord);
+    }
+
+    void erase(const LatLongCoord & coord)
+    {
+        coords.erase(coord);
+    }
+
+    /// Construct an empty set of coordinates.
+    LatLongCoords() : coords() {}
+
+    /// Construct a set of coordinates containing one coordinate.
+    LatLongCoords(const LatLongCoord & coord) : coords()
+    {
+        coords.insert(coord);
+    }
+
+    /** Construct a set of coordinates by unserialising a string.
+     *
+     *  @param serialised the string to unserialise the coordinates from.
+     *
+     *  @exception Xapian::InvalidArgumentError if the string does not contain
+     *  a valid serialised latitude-longitude pair, or contains junk at the end
+     *  of it.
+     */
+    static LatLongCoords unserialise(const std::string & serialised);
+
+    /** Construct a set of coordinates by unserialising a string.
+     *
+     *  The string may NOT contain further data after the coordinates (the
+     *  representation of the list of coordinates is not self-terminating).
+     *
+     *  @param ptr A pointer to the start of the string.
+     *  @param end A pointer to the end of the string.
+     *
+     *  @exception Xapian::InvalidArgumentError if the string does not contain
+     *  a valid serialised latitude-longitude pair, or contains junk at the end
+     *  of it.
+     */
+    static LatLongCoords unserialise(const char * ptr, const char * end);
+
+    /** Return a serialised form of the coordinate list.
+     */
+    std::string serialise() const;
+
+    /// Return a string describing this object.
+    std::string get_description() const;
+};
+
+/** Base class for calculating distances between two lat/long coordinates.
+ */
+class XAPIAN_VISIBILITY_DEFAULT LatLongMetric {
+  public:
+    /// Destructor.
+    virtual ~LatLongMetric();
+
+    /** Return the distance between two coordinates, in metres.
+     */
+    virtual double operator()(const LatLongCoord & a, const LatLongCoord &b) const = 0;
+
+    /** Return the distance between two coordinate lists, in metres.
+     *
+     *  The distance between the coordinate lists is defined to the be minimum
+     *  pairwise distance between coordinates in the lists.
+     *
+     *  If either of the lists is empty, an InvalidArgumentError will be raised.
+     */
+    double operator()(const LatLongCoords & a, const LatLongCoords &b) const;
+
+    /** Clone the metric. */
+    virtual LatLongMetric * clone() const = 0;
+
+    /** Return the full name of the metric.
+     *
+     *  This is used when serialising and unserialising metrics; for example,
+     *  for performing remote searches.
+     *
+     *  If the subclass is in a C++ namespace, the namespace should be included
+     *  in the name, using "::" as a separator.  For example, for a
+     *  LatLongMetric subclass called "FooLatLongMetric" in the "Xapian"
+     *  namespace the result of this call should be "Xapian::FooLatLongMetric".
+     */
+    virtual std::string name() const = 0;
+
+    /** Serialise object parameters into a string.
+     *
+     *  The serialised parameters should represent the configuration of the
+     *  metric.
+     */
+    virtual std::string serialise() const = 0;
+
+    /** Create object given string serialisation returned by serialise().
+     *
+     *  @param s A serialised instance of this LatLongMetric subclass.
+     */
+    virtual LatLongMetric * unserialise(const std::string & s) const = 0;
+};
+
+/** Calculate the great-circle distance between two coordinates on a sphere.
+ *
+ *  This uses the haversine formula to calculate the distance.  Note that this
+ *  formula is subject to inaccuracy due to numerical errors for coordinates on
+ *  the opposite side of the sphere.
+ *
+ *  See http://en.wikipedia.org/wiki/Haversine_formula
+ */
+class XAPIAN_VISIBILITY_DEFAULT GreatCircleMetric : public LatLongMetric {
+    /** The radius of the sphere in metres.
+     */
+    double radius;
+
+  public:
+    /** Construct a GreatCircleMetric.
+     *
+     *  The (quadratic mean) radius of the earth will be used by this
+     *  calculator.
+     */
+    GreatCircleMetric();
+
+    /** Construct a GreatCircleMetric using a specified radius.
+     *
+     *  @param radius_ The radius of to use, in metres.
+     */
+    GreatCircleMetric(double radius_);
+
+    /** Return the great-circle distance between points on the sphere.
+     */
+    double operator()(const LatLongCoord & a, const LatLongCoord &b) const;
+
+    LatLongMetric * clone() const;
+    std::string name() const;
+    std::string serialise() const;
+    LatLongMetric * unserialise(const std::string & s) const;
+};
+
+/** Posting source which returns a weight based on geospatial distance.
+ *
+ *  Results are weighted by the distance from a fixed point, or list of points,
+ *  calculated according to the metric supplied.  If multiple points are
+ *  supplied (either in the constructor, or in the coordinates stored in a
+ *  document) , the closest pointwise distance is returned.
+ *
+ *  Documents further away than a specified maximum range (or with no location
+ *  stored in the specified slot) will not be returned.
+ *
+ *  The weight returned will be computed from the distance using the formula:
+ *  k1 * (distance + k1) ** (- k2)
+ *
+ *  (Where k1 and k2 are (strictly) positive, floating point, constants, and
+ *  default to 1000 and 1, respectively.  Distance is measured in metres, so
+ *  this means that something at the centre gets a weight of 1.0, something 1km
+ *  away gets a weight of 0.5, and something 3km away gets a weight of 0.25,
+ *  etc)
+ */
+class XAPIAN_VISIBILITY_DEFAULT LatLongDistancePostingSource : public ValuePostingSource
+{
+    /// Current distance from centre.
+    double dist;
+
+    /// Centre, to compute distance from.
+    LatLongCoords centre;
+
+    /// Metric to compute the distance with.
+    const LatLongMetric * metric;
+
+    /// Maximum range to allow.  If set to 0, there is no maximum range.
+    double max_range;
+
+    /// Constant used in weighting function.
+    double k1;
+
+    /// Constant used in weighting function.
+    double k2;
+
+    /** Calculate the distance for the current document.
+     *
+     *  Returns true if the distance was calculated ok, or false if the
+     *  document didn't contain a valid serialised set of coordinates in the
+     *  appropriate value slot.
+     */
+    void calc_distance();
+
+    /// Internal constructor; used by clone() and serialise().
+    LatLongDistancePostingSource(Xapian::valueno slot_,
+                                 const LatLongCoords & centre_,
+                                 const LatLongMetric * metric_,
+                                 double max_range_,
+                                 double k1_,
+                                 double k2_);
+
+  public:
+    /** Construct a new match decider which returns only documents within
+     *  range of one of the central coordinates.
+     *
+     *  @param db_ The database to read values from.
+     *  @param slot_ The value slot to read values from.
+     *  @param centre_ The centre point to use for distance calculations.
+     *  @param metric_ The metric to use for distance calculations.
+     *  @param max_range_ The maximum distance for documents which are returned.
+     *  @param k1_ The k1 constant to use in the weighting function.
+     *  @param k2_ The k2 constant to use in the weighting function.
+     */
+    LatLongDistancePostingSource(Xapian::valueno slot_,
+                                 const LatLongCoords & centre_,
+                                 const LatLongMetric & metric_,
+                                 double max_range_ = 0.0,
+                                 double k1_ = 1000.0,
+                                 double k2_ = 1.0);
+    ~LatLongDistancePostingSource();
+
+    void next(Xapian::weight min_wt);
+    void skip_to(Xapian::docid min_docid, Xapian::weight min_wt);
+    bool check(Xapian::docid min_docid, Xapian::weight min_wt);
+
+    Xapian::weight get_weight() const;
+    LatLongDistancePostingSource * clone() const;
+    std::string name() const;
+    std::string serialise() const;
+    LatLongDistancePostingSource *
+            unserialise_with_registry(const std::string &s,
+                                      const Registry & registry) const;
+    void init(const Database & db_);
+
+    std::string get_description() const;
+};
+
+/** KeyMaker subclass which sorts by distance from a latitude/longitude.
+ *
+ *  Results are ordered by the distance from a fixed point, or list of points,
+ *  calculated according to the metric supplied.  If multiple points are
+ *  supplied (either in the constructor, or in the coordinates stored in a
+ *  document), the closest pointwise distance is returned.
+ *
+ *  If a document contains no 
+ */
+class XAPIAN_VISIBILITY_DEFAULT LatLongDistanceKeyMaker : public KeyMaker {
+
+    /// The value slot to read.
+    Xapian::valueno valno;
+
+    /// The centre point (or points) for distance calculation.
+    LatLongCoords centre;
+
+    /// The metric to use when calculating distances.
+    const LatLongMetric * metric;
+
+    /// The default key to return, for documents with no value stored.
+    std::string defkey;
+
+  public:
+    LatLongDistanceKeyMaker(Xapian::valueno valno_,
+                            const LatLongCoords & centre_,
+                            const LatLongMetric & metric_,
+                            double defdistance = 10E10)
+            : valno(valno_),
+              centre(centre_),
+              metric(metric_.clone()),
+              defkey(sortable_serialise(defdistance))
+    {}
+
+    LatLongDistanceKeyMaker(Xapian::valueno valno_,
+                            const LatLongCoord & centre_,
+                            const LatLongMetric & metric_,
+                            double defdistance = 10E10)
+            : valno(valno_),
+              centre(),
+              metric(metric_.clone()),
+              defkey(sortable_serialise(defdistance))
+    {
+        centre.insert(centre_);
+    }
+
+    ~LatLongDistanceKeyMaker();
+
+    std::string operator()(const Xapian::Document & doc) const;
+};
+
+}
+
+#endif /* XAPIAN_INCLUDED_GEOSPATIAL_H */

xapian-core/include/Makefile.mk

@@ -34 +34 @@
         include/xapian/unicode.h\
         include/xapian/valueiterator.h\
         include/xapian/valuesetmatchdecider.h\
+        include/xapian/geospatial.h\
         include/xapian/visibility.h\
         include/xapian/weight.h
 

xapian-core/include/xapian.h

@@ -62 +62 @@
 // Unicode support
 #include <xapian/unicode.h>
 
+// Geospatial
+#include <xapian/geospatial.h>
+
 // ELF visibility annotations for GCC.
 #include <xapian/visibility.h>
 

xapian-core/common/registryinternal.h

@@ -32 +32 @@
     class Weight;
     class PostingSource;
     class MatchSpy;
+    class LatLongMetric;
 }
 
 class Xapian::Registry::Internal : public Xapian::Internal::RefCntBase {
@@ -46 +47 @@
     /// Registered match spies.
     std::map<std::string, Xapian::MatchSpy *> matchspies;
 
+    /// Registered lat-long metrics.
+    std::map<std::string, Xapian::LatLongMetric *> lat_long_metrics;
+
     /// Add the standard subclasses provided in the API.
     void add_defaults();
 
@@ -58 +62 @@
     /// Clear all registered match spies.
     void clear_match_spies();
 
+    /// Clear all registered lat-long metrics.
+    void clear_lat_long_metrics();
+
   public:
     Internal();
     ~Internal();

xapian-core/common/output.h

@@ -6 +6 @@
  * Copyright 2002 Ananova Ltd
  * Copyright 2002,2003,2004,2007,2009 Olly Betts
  * Copyright 2007 Lemur Consulting Ltd
+ * Copyright 2010 Richard Boulton
  *
  * This program is free software; you can redistribute it and/or
  * modify it under the terms of the GNU General Public License as
@@ -62 +63 @@
 XAPIAN_OUTPUT_FUNCTION(Xapian::ESet)
 XAPIAN_OUTPUT_FUNCTION(Xapian::Enquire)
 
+#include <xapian/geospatial.h>
+XAPIAN_OUTPUT_FUNCTION(Xapian::LatLongCoord)
+XAPIAN_OUTPUT_FUNCTION(Xapian::LatLongCoords)
+
 #include <xapian/stem.h>
 XAPIAN_OUTPUT_FUNCTION(Xapian::Stem)
 

xapian-core/api/postingsource.cc

@@ -102 +102 @@
     throw Xapian::UnimplementedError("unserialise() not supported for this PostingSource");
 }
 
+PostingSource *
+PostingSource::unserialise_with_registry(const std::string &s,
+                                         const Registry &) const
+{
+    return unserialise(s);
+}
+
 string
 PostingSource::get_description() const
 {

xapian-core/api/registry.cc

@@ -24 +24 @@
 #include "xapian/registry.h"
 
 #include "xapian/error.h"
+#include "xapian/geospatial.h"
 #include "xapian/matchspy.h"
 #include "xapian/postingsource.h"
 #include "xapian/weight.h"
@@ -154 +155 @@
     RETURN(lookup_object(internal->matchspies, name));
 }
 
+void
+Registry::register_lat_long_metric(const Xapian::LatLongMetric &metric)
+{
+    LOGCALL_VOID(API, "Xapian::Registry::register_lat_long_metric", metric.name());
+    register_object(internal->lat_long_metrics, metric);
+}
 
+const Xapian::LatLongMetric *
+Registry::get_lat_long_metric(const string & name) const
+{
+    LOGCALL(API, const Xapian::MatchSpy *, "Xapian::Registry::get_lat_long_metric", name);
+    RETURN(lookup_object(internal->lat_long_metrics, name));
+}
+
 Registry::Internal::Internal()
         : Xapian::Internal::RefCntBase(),
           wtschemes(),
-          postingsources()
+          postingsources(),
+          lat_long_metrics()
 {
     add_defaults();
 }
@@ -168 +183 @@
     clear_weighting_schemes();
     clear_posting_sources();
     clear_match_spies();
+    clear_lat_long_metrics();
 }
 
 void
@@ -190 +206 @@
     postingsources[source->name()] = source;
     source = new Xapian::FixedWeightPostingSource(0.0);
     postingsources[source->name()] = source;
+    source = new Xapian::LatLongDistancePostingSource(0,
+        Xapian::LatLongCoords(),
+        Xapian::GreatCircleMetric());
+    postingsources[source->name()] = source;
 
     Xapian::MatchSpy * spy;
     spy = new Xapian::ValueCountMatchSpy();
     matchspies[spy->name()] = spy;
+
+    Xapian::LatLongMetric * metric;
+    metric = new Xapian::GreatCircleMetric();
+    lat_long_metrics[metric->name()] = metric;
 }
 
 void
@@ -223 +247 @@
     }
 }
 
+void
+Registry::Internal::clear_lat_long_metrics()
+{
+    map<string, Xapian::LatLongMetric *>::const_iterator i;
+    for (i = lat_long_metrics.begin(); i != lat_long_metrics.end(); ++i) {
+        delete i->second;
+    }
 }
+
+}

xapian-core/Makefile.am

@@ -140 +140 @@
 include common/Makefile.mk
 include examples/Makefile.mk
 include expand/Makefile.mk
+include geospatial/Makefile.mk
 include include/Makefile.mk
 include languages/Makefile.mk
 include matcher/Makefile.mk

xapian-bindings/xapian.i

@@ -790 +790 @@
 
 %include <xapian/valuesetmatchdecider.h>
 
+%ignore Xapian::LatLongCoord::operator< const;
+%include <xapian/geospatial.h>
+
 namespace Xapian {
 
 #if defined SWIGPYTHON