Branching in source control can be a rather controversial topic. We are unlikely to settle the matter today, but we want to share our philosophy, which is based on some basic principles and some hard-learned lessons.
Why branch?
The motivation behind creating a branch is the desire to isolate a copy of the code from changes being made by other developers. In that sense, the most basic branch is an individual developer’s working copy (we certainly don’t want to work on the same set of shared files). In fact, in distributed version control systems, you must create a “proper” branch (by cloning the repository) before you can create a working copy. From this point, we will exclude “working copy” branches from our discussion, as I think everyone agrees that they are a Good Thing.
Developers create branches for two basic reasons:
- Isolating code that is/will be released to production (a release branch)
- Isolating code for the purposes of long-term feature development (a feature branch)
In the context of a small to mid-size enterprise, we believe that development teams should regularly make release branches, and never (or perhaps very rarely) create feature branches. We believe in trunk-based development and the branch-for-release pattern.
Why release branches are good
Release branches ensure that you have easy access to the code that you are currently using in production. They allow you to make isolated bug fixes and rapidly roll out “patches” to production without the risk of a fresh rollout from trunk. They are your most important bulwark against “losing” code that your business depends on. As we have discussed before, we strongly encourage following a regular schedule for deployments to ensure that you are getting your newest code in front of your users, gathering feedback, and keeping the agile cycle turning. “Cutting” a fresh release branch from trunk should be a simple procedure (not necessarily an automated one, though — if you find that the manual process is too cumbersome, consider simplifying the process before automating), and you should make it part of your weekly (or bi-weekly, or daily) release routine.
To make a regular release procedure work, consider adopting the following practices:
- Make sure the entire team knows when the branch will be cut. This is all the more reason to cut on a regular schedule, so that the team internalizes the “rhythm” of the release process.
- Avoid re-using the same release branch and “merging” changes up to it. This is a recipe for disaster if the merge gets messed up. It is much less risky to “cut” a fresh branch every time, because you are guaranteed a branch that looks just like trunk did at the time of the cut.
- Adopt a standard naming convention for release branches. Our preference is to use branches that include the application name and the date in YYYY-MM-DD format. In subversion, you would create a directory like MyApp/releases/2013-05-17. A standard convention will help your team know where to go when they need to find a particular release branch.
- Create a CI build for each new release branch. In the same way that we prefer a fresh branch for each release, a fresh build is a good idea as well. You should keep your old release builds around until software built from them is no longer in production, then retire them.
- Only release software to production if it was built off a release branch. This may seem obvious, but is the linchpin to a successful branch-for-release workflow. Resist the temptation to throw trunk software out there “just this once”, because doing so will make life harder on everyone who needs to maintain that release.
Why feature branches are bad
Feature branches differ from release branches in one very important way — a feature branch is intended to be merged back into trunk at some point, while a release branch will never be merged (it will diverge from trunk until it is no longer needed, at which point it can simply be abandoned or deleted). This means that the basic purpose of a feature branch is to delay integration of the work being done there. This is the fundamental problem with feature branches. They represent a development pattern that is the complete opposite of what we are trying to achieve with continuous integration. Remember, CI is more than just having an automated build, it is the principle that all work should be integrated into a shared mainline as soon as possible.
Let’s review some of the benefits of following CI:
- The smaller each change to the mainline is, the easier it is to merge changes. Ideally, merging is rarely required, and is trivial when it is.
- There is always a single “latest” version of the software that all developers are working on. This version should always work and ideally be releasable to production.
- It encourages incremental change and progress. To continuously integrate, developers must find ways to split large features into small chunks that can be committed to the trunk. This is a good forcing mechanism for the creation of a well-factored code base.
- Code checked in to trunk will trigger a continuous build and a run of the automated test suite, ensuring that the trunk is always compiling and unit-tested.
- If you follow branch-for-release, it ensures that work gets deployed to production quickly and the feedback loop from your users is delayed as little as possible.
Now, consider how feature branching subverts those benefits:
- When the feature branch is complete, there will be a “big merge” at the end, which could lead to a painful merge process and the risk that the merge is not done correctly.
- There are now multiple “latest” versions of the software. Your team is no longer sharing a single code base.
- It discourages incremental change. In fact, it encourages broad, sweeping changes across the whole code base to get a feature done, as opposed to the incremental approach which would require the creation of well-factored interfaces and abstractions.
- You do not get the benefit of the continuous build unless you create a build for every feature branch.
- Since it delays integration and release, it delays your ability to get feedback from your users.
A common objection to the anti-feature-branch stance is that “you just feel that way because branching is hard in subversion — come over here and see the light of git!” I grant that branching and merging are much cheaper and easier under git than they are under subversion. However, just because something is cheap and easy doesn’t make it right. Feature branching fundamentally breaks the continuous integration workflow, no matter how easy the “big merge” is at the end.
We hope you’ll consider our point-of-view on this and think about religiously branching for release and avoiding feature branches like the plague. Care to disagree? Let us know in the comments.
Why have a new branch for each release? If you have release/master branch and just one more dev branch you must really mess things up to get a merge conflict. It could make sense for a software program, but I don’t think it does for a service like a web site. The reason to this is that for a service you must likely want the changes that went into the release branch also in next version, that means merging them back to dev. This will make the risk for conflicts when going from dev to release/master minimal, at least if you are using a decent cvs as git. Also, I think having just one master branch will simplify your process.
Hi Tomas,
My whole point here is that you shouldn’t be making “dev branches” — there should only be a single “dev branch” — the trunk. You then create a branch from trunk for each release. I feel that this process minimizes merging risk while maximizing the stability of released code. I certainly don’t like the idea of releasing to production directly from the “master/trunk” branch. You may be correct that this process isn’t ideal for web development — that’s not really our specialty here, as our focus is on enterprise software (primarily desktop, console, and service apps). However, I have used this process for intranet-based web applications and it has worked well for me. I would be interested in hearing more about how your web deployment/branching process works.
Hi Stuart,
I agree basically with the whole article but I believe sometimes feature branch could be necessary. The case of “I’m building a feature, the iteration/release is over and we are going to production but I couldn’t finish it”… how would you manage these cases when a switch for the new functionality is not possible?
I’d appreciate any feedback
thanks
Sebas
Hi Sebas,
Thanks for the question! I had an answer all ready until I read “a switch for the new functionality is not possible”. That is typically my argument – that a new feature or major change be hidden behind a configurable switch. I am curious about the situation that is making it impossible to switch off the new feature. If it is a major cross-cutting change (with a lot of refactoring), I recommend breaking it down into small chunks that can be checked in and tested independently without breaking existing functionality. The principles in play behind my “no feature branching” policy are (1) that integration of new development should not be delayed and that (2) the integrated trunk should always be in a releasable state. Keeping these principles from conflicting is our (sometimes challenging) task, but if we can do it, we gain the power to deliver working software quickly and consistently while minimizing the risk of changes. Thanks again for reding!
Hi Stuart,
That is the response I always hear. Just break it down smaller.
We are building a new case management system from Scratch. The old system is 18 years old and we will be using none of it.
An opportunity very few people have. We aren’t simply adding features to a system.
In addition, the clerks don’t work for us so we can’t just drop stuff in weekly without approval from each county.
We will be doing parallel development of all functions and using a central “Party” database in a high traffic web app.
Implementing small changes over 77 counties that function differently is not a solid model because we forget the major issue here… Training of the new features. The customer has to know how to use it. No one ever talks about teaching the new customer how to use the new features with every release.
It’s always … break it down small so you can implement it daily or weekly. The problem here is … if you build small pieces of a new system and try to bolt them on to the old system, you consume a great deal of time building in old functionality to your new system that will require major refactoring. This kind of work increases the time frame of the end product. In addition, the benefit of frequent implementation is offset by the frustration of the users with constant untrained change in their environment. Lets get real here… people don’t like change.
What bothers me most about this or any other “savior of business” is it’s always a cookie cutter approach. “It will work for everyone”. You just can’t say that when you don’t know everyone’s business model. Since most of us would agree, it’s not about MIS Processes or Technology but rather about customer satisfaction. The theory is similar to many others.
In addition, isn’t this continuous delivery model built on the premise that the Agile process is working and accepted by the customer? What if the customer doesn’t want that?
I love shooting for high goals as it always makes us better. But the idea of forcing Developers and Customers to jump through hoops simply to follow a process doesn’t sit well with our developers or our customers. Unhappy developers make for poor work force and unhappy customers get you fired.
I like the ideas though and I see the merits in a high maintenance shop.