Modify build system to use non-recursive make

[Richard Boulton]

Currently, if you are in the xapian-core directory, have modified one of the source files (say, api/omdatabase.cc), and run "make libxapian.la", libxapian.la will not be rebuilt. Similarly, if you are in the xapian-core/tests directory, and run "make apitest", apitest will not be rebuilt.

This is due to the current build system using a recursive make; as a result of this, dependency information is not shared between the make system in each subdirectory. Thus, the top level build system does not know that libxapian.la depends on api/omdatabase.cc, and the "test" directory build system does not know that apitest ultimately depends on api/omdatabase.cc, too.

This can be fixed by moving the build system to a recursive make, which has been possible to support with automake for a while (using the subdir-objects option, amongst other things).

Such a modification of the build system would result in a single Makefile.am located in the top level directory, which would include "submake.mak" files in each subdirectory. These submake.mak files would contain the lists of source files currently held in the Makefile.am subdirectories, and any subdirectory specific rules.

In the build tree, this would result in a single Makefile, in the top level directory. Issues such as conditional compilation of subdirectories can still be dealt with by use of automake conditional statements. However, we would lose the ability to simply run "make" in a subdirectory to compile all standard compilation items in that directory. This could be offset by providing simple, manually written, stub makefiles in those subdirectories for which this would be useful (in particular, in tests/ and docs/) which provide the most useful targets, and implement them by calling the top-level makefile appropriately.

[Olly Betts]

I've actually already made a start on this...

[Olly Betts]

Reassigning, as I'm actively working on this.

So far, I've converted api, common, languages, and tests/harness.

I'm not sure we necessarily want to convert everything incidentally - the docs/ subdirectory at least is perhaps best left alone, since most of the benefits of non-recursive make don't apply there.

[Olly Betts]

OK, everything in libxapian is now converted, which is where the major wins are (in particular, a built tree will now require a lot less disk space, and parallel make will be more effective).

I'll let the dust settle and see if this all works as intended, then we can consider the remaining directories.

[Olly Betts]

All directories now converted except docs and tests, so everything which "make all" does in a source tree unpacked from a tarball should now be non-recursive.

I think the docs Makefile.am is too complex to be worth converting, as it's inherently a more separate set of tasks to building the code.

The tests Makefile.am is fairly complex too, but perhaps worth considering converting in the future, since the dependencies matter more there. The example in the original report has an easy workaround at least:

make -C .. && make apitest && ./runtest ./apitest

Instead of:

./runtest ./apitest

So I'm closing this bug now.

[Olly Betts]

Fixed in 1.0.0 release.