BuildBot

The current status of our buildbot can be seen in its web interface.

Setting up a new build slave

Buildbot consists of a central "build master", and a series of build slaves. The master farms out builds to the slaves, at frequencies, and with configuration options, specified in the central configuration. Build slaves are relatively lightweight, but must have all the software installed on them which is necessary to perform a build of xapian.

In addition, at present, the buildbot client software must run on the slave. We are working on making buildbot able to run all commands over ssh (or perhaps even over telnet), to avoid the need to run a buildbot server on the slave, but until this is done, build slaves will require a buildbot slave server to be running.

Setting up a windows Cygwin build slave

To set up a new windows Cygwin build slave, take the following steps (assuming a fresh installation of windows) (Note that we have experienced problems on machines which have old copies of cygwin1.dll lying around - check c:\WINDOWS for this):

  • Download and run the Cygwin setup.exe (from http://www.cygwin.com/setup.exe). Install a minimal set of packages. If you understand cygwin, check that the directory /etc/alternatives exists after the installation - I've observed some problems with /etc/alternatives not getting created correctly. If it doesn't exist, try using a different mirror to get a newer version of the alternatives package.
  • Run setup.exe a second time. This time select and install the following essential packages (these will pull in other packages as dependencies; it's fine to install other stuff too if you want it):
    • autoconf2.5 (or later)
    • automake1.10 (or later)
    • bash
    • bison
    • bzip2
    • ccache
    • coreutils
    • doxygen
    • gcc-g++
    • gcc-mingw-g++
    • help2man
    • libtool
    • make
    • pcre-devel
    • perl
    • python
    • subversion
    • tar
    • zlib
  • Enable the compiler cache (ccache). This will use more disk space, but speed up rebuilds; you can skip it if this doesn't sound helpful. There are several ways to enable it, but the following worked well for me:
    • create a file named /usr/local/bin/gcc containing 
      #!/bin/sh
ccache /usr/bin/gcc "$@"
    • create a second file named /usr/local/bin/g++, with similar contents, but with gcc replaced by g++.
  • Install easy_install. This can be done by going to http://pypi.python.org/pypi/setuptools and downloading the latest setuptools egg (currently http://pypi.python.org/packages/2.5/s/setuptools/setuptools-0.6c8-py2.5.egg). Then, open a cygwin command prompt, and run the egg using sh. For example, if the egg was saved in "C:\", you would run the command "sh /cygdrive/c/setuptools-0.6c8-py2.5.egg".
  • Install twisted and buildbot: from the cygwin shell, simply run "easy_install twisted" and then "easy_install buildbot".
  • Get a password and name for your slave from the xapian buildbot maintainers (contact us on IRC or via the xapian-devel mailing list).
  • Create your build slave, using the following command (in a newly opened cygwin shell), where "<name>" is the name for your build slave, and "<password>" is the password:
    • "buildbot create-slave buildbot_slave buildbot.xapian.org:9989 <name> <password>"
  • Edit the information files for the build slave:
    • from the cygwin prompt run "notepad.exe buildbot_slave/info/admin", and adjust the file to contain your contact details.
    • from the cygwin prompt run "notepad.exe buildbot_slave/info/host", and adjust the file to contain a brief description of the host (eg, rough specifications, what OS it is running, whether it is permanently connected).
  • Start your build slave:
    • from the cygwin prompt run "buildbot start buildbot_slave". You should see, after a few other lines of text pop up, "The buildslave appears to have (re)started correctly."

Ideally, you will need to ensure that the buildbot restarts automatically when the build slave machine is restarted. To do this, add a new String value to the Windows Registry at HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\Run - name it 'Buildbot' and give it a path to a batch file that will start a Cygwin shell and run "buildbot start buildbot_slave". Such a batch file follows: 

@echo off
C:
chdir C:\cygwin\bin
bash --login -c "/usr/bin/buildbot start /home/Administrator/buildbot_slave"