Tuesday, February 15, 2011

Git, Gerrit, Redmine, gitflow: An ideal software development and release management setup

I've been using Subversion for more than 5 years and have been following Git's development, adoption and maturation during that entire time. At work, each time we would create a new repo for a project, I cringed at the thought and questioned, "but, we could be doing this in Git..." However, given the development environment as well as business context (not a software shop), it just wasn't suitable to immediately jump into Git. So what did it take to make the jump? Some rather significant dissonant (and concurrent) development tasks, branch management issues and several deploys that cost more time (and thus, money) than they could have.

Our primary requirements for the SCM migration included the following:
  1. 1. Central repo to act as an on-site, internally maintained, redundant source authority with replication capabilities
  2. 2. Integration into Redmine
  3. 3. A well defined methodology for maintaining several projects through multiple concurrent phases
  4. 4. Mature CLI toolset
  5. 5. Hudson/Jenkins support
  6. 6. IntelliJ integration
  7. 7. SCM should be open source with a vibrant community
I knew most of these requirements could be met with Git, but didn't want to make a blind choice and simply choose Git as the new internal de-facto standard. I've had some simple experience with Mercurial in the past and even spent some time doing some projects in Darcs several years ago. I have a good friend whose company (software shop) moved to Mercurial and he offered their arguments in support of choosing Mercurial. Frankly, I liked a lot about what I saw in Mercurial and I didn't ever have any negative experience when toying around with it. Though, truth be told, the projects I conjured up for trying it out were very simplistic and never moved outside of my local development environment. During my investigation I found stackoverflow.com to be immensely helpful in identifying specific differences and perceived strengths and weaknesses.


In the end, after all the reading, playing, comparing--I found that Git just jived with me and it satisfied our requirements (4,5,6,7). Darcs just didn't fit, and Mercurial looks great, so I have nothing negative against those projects. Plus, they don't have Gerrit. Kudos to the Java Posse for a recent podcast in which Gerrit was discussed, timing happened to be critical as it was in the middle of my research, and it sounded like just what we needed to address a recent issue of lack of code review. Even though I was already heavily leaning toward Git at this point, Gerrit clearly brought additional benefit to the migration and methodology changes we were considering.


Gerrit is wonderfully simple to get setup and running. I very much appreciate that it's quite self-contained and offers OpenID support to avoid the grief of maintaining local user accounts. In fact, Gerrit will not only help facilitate code review and branch maintenance, it can act as our centralized repo while abstracting OS-level duties (specifically, user management) for utilizing SSH as our transport protocol. Gerrit's permissions model is more than adequate for our needs on a per-project basis, but is simple enough to setup and get going. Thus, Gerrit easily satisfied requirement #1 and gave us the extra bang-for-the-buck with its inherent review functionality.


Redmine (1.0.1) was easy to modify for our situation and it made the most sense to replicate the central repo as a read-only repository locally available to the Redmine instance. Once the clone completed, the only remaining task was setting up a periodic cronjob for updating (git fetch --all) the repo.


gitflow was the answer to requirement #3. One of the beauties of Git (and DVCS in general) is the fundamental capability of determining your own release cycle/phase/management process. In some cases when dealing with a DVCS setup it means there's a lot of rope to hang yourself. Our previous release practice was suffering, from time to time, with 100% consistency. I stumbled on gitflow only after deciding on Gerrit, and it made a lot of sense to embrace it not only for what it provides out of the box (helpful bash scripts) but also for the well defined development convention that it helps to enforce. Actually, it's not that gitflow just helps to enforce process, rather, it eases process implementation. Turns out that its process definition matches, and enhances, what we've already (mostly) been doing. Vincent deserves a heap of credit and has our gratitude.


Hudson/Jenkins (Oracle is being a hoser about this, IMO) has two plugins options:
Gerrit: http://wiki.jenkins-ci.org/display/JENKINS/Gerrit+Plugin
Git: http://wiki.jenkins-ci.org/display/JENKINS/Git+Plugin

Really like the idea of pushing changes to Gerrit fires off CI tasks in Jenkins, then verifies/fails the changeset depending on the results. However, integrating that plugin during the initial migration turned out to be a lower priority. At the very least we simply swapped out the current setup of using Subversion with just the normal Git plugin.


While I'm in a shell nearly 100% of the time, on occasion it's convenient to have some SCM support in Intellij 10. However, I did run into some issues with merging in IntelliJ and spent some time looking into various merge tools. My emacs blood revolted when I chose the Perforce merge tool over emerge (which I liked a lot better than opendiff, Meld or diffmerge). Thanks to Andy Mcintosh for his tips.


After walking through multiple discussions with several team members, it's been very clear to them the benefits and strengths this setup provides over our current process with Subversion. So as of this post, the first major project (72k LOC) has been migrated. This setup feels right, and it's good to see additional corroboration in the community (thanks, AlBlue).


vinay said...


I just ran into your blog post. Do you have any tips or a guide on how to get Gerrit and Remdine working together?

RR said...

@vinay Don't have a guide, but we're essentially following the mirroring recommendation outlined here:

We clone the repos we care to track onto the same Redmine machine and have a cronjob that invoking a script like:
cd /pub/git/whatever_repo1.git;git fetch --all;
cd /pub/git/whatever_repo2.git;git fetch --all;
cd /pub/git/whatever_repo3.git;git fetch --all;

This way Redmine has visibility to the entire repository and all the branch work. Be aware that this may clutter up some tickets when using ref/fixes annotation in the commit messages, but we think that's a negligible side-effect.

Hope that helps!

Jeremy Kitchen said...

Which gitflow branches do you push into gerrit, or, more specifically, how are you integrating gitflow and gerrit?

I'm setting this exact same thing up right now, and I'm thinking have people push unobstructed (no review) into the development branch and release and production branches will be signed-off by team members. Eventually working to have everything committed into the development branch be code reviewed, but that's probably not going to happen at this stage :)

RR said...

@Jeremy, we are actually pushing everything through Gerrit now. As much as we wanted to enforce review at _some_ stage, we just haven't had the time/need yet due to our team's small size. We'll still proceed with at that reviewing step, but it's been delayed and probably won't happen until later in 2012 or 2013.

Georg Grütter said...

We're trying to set up a Gerrit/git-flow environment at our shop, too. While reviewing commits going from a feature branch (which is local and never merged to by Gerrit) works fine, I wasn't able to to the same with release branches, which I'd like to publish to other team members. Did this work for you? What happens if you push something for review with e. g.

git push origin HEAD:refs/for/release/1.1

In my Gerrit, this creates a branch named


rather than


Any ideas?

RR said...

@Georg, haven't tried that approach so I don't have any help to offer :(