Migrate to git

[Olly Betts]

Still to sort out:

• xapian-maintainer-tools/make-xapian-svn-snapshot-tarballs
• xapian-maintainer-tools/svn-tag-release
• Trac - enable git integration
• Website
  • /bleeding
  • links to files in SVN via trac.xapian.org/browser
• Buildbot
• Make SVN repo read-only
• Enable pushing to master git repo
• Dismantle SVN to git mirroring machinery

Once we've moved, we can drop handling for SVN from:

• bootstrap (needs pushing)
• xapian-core/HACKING (git-svn requirement, etc)
• website (/bleeding)

Also to update:

• debian packaging - e.g. Vcs-Browser and Vcs-svn in control files

Miscellaneous:

• Trac - currently we're patching 1.0.2 - we should sort out feeding these upstream, or doing them via a plugin (if that's possible):
  • Allowing r12345 and [12345] to keep working for SVN, while allowing [103357b] to work for git
  • Allow 7 hex digits for a git commit hash (as that's what git's abbreviated commit hash is by default) - trac requires 8 by default: ​http://trac.edgewall.org/ticket/11992
• Rename svn-ci to xapian-commit
• Consider a switch to gitolite or Simon Tatham's stuff
• Migrate website from CVS (!) to git
• Hook up git commits to xapian-commits list? Is this still useful? Nobody seems to think so~~

[Olly Betts]

git-tag-release written based on svn-tag-release (but not yet tested).

git equivalent of make-xapian-svn-snapshot-tarballs close to working.

[Olly Betts]

Trac git support enabled

[Olly Betts]

Migrated buildbot in r18372.

[Olly Betts]

The next step which needs doing involves adding users to a group on atreus, so reassigning to james for that.

[James Aylett]

Pushing to git for atreus users via the xapian-git group should now work; reassigning to Olly.

[Olly Betts]

Works after a chmod -R g+w of the relevant files.

Longer term we may want to move to using gitolite or Simon's stuff to provide a bit more control.

[Olly Betts]

Pushed the patch for 7 character abbreviated git commit hashes upstream.

[Olly Betts]

The website is now in git, hosted on git.xapian.org and mirrored to github. I noted down what I had to do to achieve this in README in the xapian-git directory on atreus, so it'll be easier to repeat.

[Olly Betts]

xapian-commits is no longer getting traffic. Not sure if it actually still useful, but we should at least email it to (a) canvas the ~20 subscribers, and (b) so the archives show why it's no longer active if we let it lie. And if it is to die, we should disable posting so spammers can't get to it (though it is fairly well protected).

[Olly Betts]

Emailed xapian-commits and xapian-discus about the list. Given we've a couple of useful steps forward, I'm bumping the rest to 1.3.5.

[Olly Betts]

We concluded the commits list wasn't worth ressurecting.

This is not a blocker for releasing 1.4.0.

[Olly Betts]

So with the server switch, we're now on trac 1.0.13 but without the local patch and without the SVN revision id links working (as there's no copy of the SVN repo). The lack of the SVN repo means that the revision number/commit hash disambiguation is no longer an issue for us.

The notable upshot is that you can't abbreviate git commit hashes to 7 characters now (only 8+), which is irritating as the default in git is 7 (when 7 is unique). 7 is short enough that large repos tend to have collisions, so it's probably wise to increase this as a local setting, but that means people need to actually do it. I don't think there's a way to configure this in the repo such that you get it just by cloning, but I may be wrong.

Perhaps this isn't worth carrying a patch for - I opened a bug against trac which seems like it may eventually get dealt with.

I think I'm also happy with how git access is set up now - once migration is complete we'll have anon access via https (secure, more likely to work through firewalls) or the git protocol (which seems to have few advantages these days), and write access via ssh, so we can probably cross off "Consider a switch to gitolite or Simon Tatham's stuff".

[James Aylett]

At least at this point I agree we don't need to think about things like gitolite. Simon's restricted git shell probably isn't appropriate for us, as it maps closely onto how they merge things, and just using the normal restricted git-shell for the git write account (which we now do) should be fine.

We're using a xapian-specific install of trac, in the same way as the previous machine, so we can carry a local patch if we feel it's valuable. However (via ​an article linked in the trac ticket) I note that 7 character is no longer unique across the whole of Xapian:

$ git rev-list --all --abbrev=0 --abbrev-commit |
>   awk '{ a[length] += 1 } END { for (len in a) print len, a[len] }'
4 1894
5 16105
6 2708
7 193
8 8

So we may be better off advising people to set core.abbrev in git clone to 8. We could have bootstrap warn if it's in a git clone with the default (git config core.abbrev will return the empty string). I'm unaware of any way of "pushing" configuration on clone.

I'm not convinced from the trac ticket that they'll get to it any time soon unless someone chips in.

[Olly Betts]

I've added a command to find the git commit corresponding to a given SVN revision number to https://xapian.org/bleeding :

git log --grep '^git-svn-id: .*@12345 ' `git branch --list -r 'origin/svn/*'`

So we may be better off advising people to set core.abbrev in git clone to 8. We could have bootstrap warn if it's in a git clone with the default (git config core.abbrev will return the empty string). I'm unaware of any way of "pushing" configuration on clone.

I've just written a simple check for the value and then if it's < 8 and not set on the repo itself, to set it to 8 on the repo. But if we already have collisions at length 8, that's clearly too short. I think we should go with 12 - that's not so long as to be clumsy, but long enough that we are unlikely to see collisions for a significant time. Using the full SHA1 is preferable in places (e.g. commit messages and the backporting git notes I've recently started adding).

I'm not convinced from the trac ticket that they'll get to it any time soon unless someone chips in.

I'm happy I've at least led that horse to water.

[Olly Betts]

066bfa03586d3b50682c8a6cf9284b2b2ae65b56 implements encouraging use of core.abbrev=12 as described above.

With that, we can at last close this ticket - there's nothing else that we still want to do here.

I intend to backport the core.abbrev bootstrap commit to both the release branches, so setting milestone appropriately.