Automatically mirroring SVN repositories to git and github

Updated 21 February 2013: the scripts below have now been adapted to run conveniently from cron and mirror multiple repositories. You can find full scripts and documentation on github

There are a number of scripts around, including an earlier blog post of my own, for making a once-off conversion of an SVN repository to a Git repository.

However, what if the project team is not yet ready for a full move to git? You may want to start benefiting from git (e.g. using the project as a submodule of another git project, or using Travis-CI.org for automated builds) while development officially continues to take place in SVN.

If you don't need to make commits into git just now, you can set up a mirror and refresh it periodically from SVN.

This process can be used with various SVN repositories, including Sourceforge and Google Code SVN repositories.

Process overview

• An authors.txt file is created manually before the first run - see my other blog entry for convenient scripts to help generate authors.txt
• A git-svn clone of the SVN repository is created. This clone needs to be retained between replication runs.
• On each run, a temporary git repository is created. The svn-clone is rebased and pushed to the fresh git repository.
• Fixes (e.g. branch renames and tagging) are made in the temporary git repository.
• The final results are then pushed from the temporary repository to the real git mirror that is being maintained.
• Finally, the temporary git repository is deleted, but the svn-clone is kept.

Here is a diagram of the workflow:

Config file

Here is a sample config file for the script, it mirrors the rather complicated reSIProcate repository with 10,000 commits:

PROJECT_ROOT=/home/daniel/svn-mirror/resiprocate
AUTHORS_FILE=${PROJECT_ROOT}/authors.txt
SVN_REPO=https://svn.resiprocate.org/rep/resiprocate
SVN_LAYOUT="--trunk=main --branches=branches --tags=tags"
GIT_REPO=git@github.com:resiprocate/resiprocate.git

Of particular note here:

• The $PROJECT_ROOT is where the svn-clone and the temporary repositories will be created
• $GIT_REPO is the final destination repository. For testing, it could just be a local directory initialised with git init --bare .
• In this repository, our trunk is actually called main, hence we have the layout argument --trunk=main. Many projects would just use --trunk=trunk

The script

The script itself is below. I'd suggest running it manually a few times first and observing how it behaves, disk space requirements, etc.

#!/bin/bash

set -e

if [ ! $# -eq 1 ];
then
  echo "Usage:"
  echo "$0 "
  exit 1
fi

CONFIG_FILE="$1"

. "${CONFIG_FILE}" || exit 1

SVN_CLONE="${PROJECT_ROOT}/svn-clone"
GIT_BARE="${PROJECT_ROOT}/git-bare-tmp"

if [ ! -d "${PROJECT_ROOT}" ];
then
  mkdir -p "${PROJECT_ROOT}"
fi

cd "${PROJECT_ROOT}"

if [ ! -d "${SVN_CLONE}" ];
then
  git svn clone \
    "${SVN_REPO}" \
    -A "${AUTHORS_FILE}" \
    ${SVN_LAYOUT} \
    "${SVN_CLONE}"
  cd "${SVN_CLONE}"
else
  cd "${SVN_CLONE}"
  git remote rm bare || echo "failed to delete remote:bare, proceeding anyway"
  git svn rebase \
    --fetch-all \
    -A "${AUTHORS_FILE}"
fi

git remote add bare "${GIT_BARE}"
git config remote.bare.push 'refs/remotes/*:refs/heads/*'

if [ -d "${GIT_BARE}" ];
then
  rm -rf "${GIT_BARE}"
fi

mkdir -p "${GIT_BARE}"
cd "${GIT_BARE}"
git init --bare .
git symbolic-ref HEAD refs/heads/trunk

cd "${SVN_CLONE}"
git push bare

cd "${GIT_BARE}"
git branch -m trunk master
git for-each-ref --format='%(refname)' refs/heads/tags | \
   cut -d / -f 4 | \
   while read ref;
   do
      git tag "$ref" "refs/heads/tags/$ref"
      git branch -D "tags/$ref"
   done
git remote add origin "${GIT_REPO}"
git config branch.master.remote origin
git config branch.master.merge refs/heads/master
git push --tags origin master
git push --all

rm -rf "${GIT_BARE}"