These days, distributed version control systems like Git have "won the war" of version control. One of the arguments I used to hear when DVCSs were gaining traction was around how easy it is to branch and merge with a VCS like Git. However, I'm a big fan of Trunk-Based Development (TBD), and I want to tell you why.
With trunk-based development, all developers work on a single branch (e.g. 'main'). You might have read or heard Martin Fowler or Dave Farley talking about it. It was when I was working with Dave (around about the time that Git was rapidly becoming the "go-to" version control system) that I really saw the benefits of trunk-based development for the team, particularly in an environment that was pioneering Continuous Delivery - Dave was writing the book with Jez Humble while I worked with him.
In contrast, the branching model encourages developers to create separate branches for every feature, bug fix, or enhancement. Although branching may seem like a logical approach to isolate changes and reduce risk, several factors make me more comfortable with trunk-based development.
1. Speed and Efficiency
In trunk-based development, the entire team works on a single branch. This model allows for quicker integrations and fewer merge conflicts. This is literally "Continuous Integration (CI)", as originally suggested by the practices of Extreme Programming. While these days we tend to mean "run your build and tests on a team server every time you commit" when we say CI, what CI really meant was actually integrate your code regularly. Code living on separate branches is, by definition, not integrated. And the longer these branches live for, the more challenging it is to merge them back into the main codebase. It might seem fast to develop your fixes and improvements on a separate branch that isn't impacted by other developers' changes, but you still have to pay that cost at some point. Integrating small changes regularly into your code is usually less painful than a big merge at the end of a longer period of time.
2. Greater Code Stability
Trunk-based development encourages frequent commits, which leads to smaller and more manageable changes. How and why? For the same reason that we don't want big merges from long-lived branches - the longer we leave it to commit our changes, the higher the chances our commit will clash with someone else's changes. By frequently pulling in the other developers' changes, and frequently pushing small changes of working code, we know the codebase is stable and working. Of course, this assumption of "stable and working" is easier to check if we have a CI server that's running the build and tests for each of these commits. We also have to stop making commits if the build breaks at any time and focus on fixing that build. Continuously pushing small, frequent commits when the build is already broken isn't going to do anyone any favours.
In the branching model, large and infrequent merges can introduce bugs that are hard to identify and resolve due to the sheer size of the changes. Have you ever merged the trunk into your branch after someone else has merged their own big piece of work and found your code no longer works? It can take a LOT of time to track down why your tests are failing or the application isn't working the way you expect when you've made a whole bunch of changes and someone else has made a whole bunch of different, or overlapping, changes. And that's assuming you actually have reliable test coverage that can tell you there's a problem.
3. Enhanced Team Collaboration
My favourite way of sharing knowledge between team members is pair programming. I know not everyone is a fan or is in a position to do it (especially now more people are working remotely, but if so, check out JetBrains' Code With Me). If you're not pairing, then at least you want to be working on the same code, right? If you're all working on your own branches, you are not collaborating. You are competing. To see who can get their code in fastest. To avoid being stomped on by someone else's code changes.
If you're all working on the same branch, you tend to have a better awareness of the changes being made. This approach fosters greater team collaboration and knowledge sharing. In contrast, branching can create a siloed work environment where you're all working independently, leading to knowledge gaps within the team.
4. Improved Continuous Integration and Delivery (CI/CD) Practices
Dave Farley's book, "Continuous Delivery", and his blog posts and videos, argue something along the lines of "trunk-based development is inherently compatible with Continuous Integration and Continuous Delivery (CI/CD) practices".
In a trunk-based model, continuous integration becomes more straightforward because your code is committed frequently to trunk, and that's the branch your CI environment is running the build and tests on. Any failures there are seen and addressed promptly, reducing the risk of nasty failures. It's usually easy to track down which changes caused the problem. If the issue can't be fixed immediately, you can back the specific changes that caused it.
By now we should know the importance of a quick feedback loop - when you find problems faster you can locate the cause faster, and you can fix it faster. This improves your software's quality.
Continuous delivery also thrives in a trunk-based development environment. Successful continuous delivery hinges on the ability to have a codebase that is always in a deployable state. The trunk-based development approach ensures this by promoting frequent commits, frequent integrations, and tests on all of these integrations. The small number of changes being introduced at any one time makes the software easier to deploy and test.
In contrast, implementing effective CI/CD can be more complex and time-consuming with the branching model. While it's tempting to think "Well, I run my build and all my tests on my branch", you're not actually integrating every time you commit. It's at merge (or rebase) time that you start to see any integration issues. All those tests you were running on your branch "in CI" were not testing any kind of integration at all.
Merging and testing code from different branches can introduce delays and potential errors, which takes away some of the benefits of having a build pipeline in the first place.
5. Reduced Technical Debt
Long-lived branches often lead to 'merge hell', where the differences between one branch (like 'main') and another (for example your feature branch) are so great that merging becomes a nightmare. This can result in technical debt as you may resort to quick fixes to resolve merge conflicts or accept suggestions from your IDE that resolve the merge but that you don't fully understand. With trunk-based development, frequent merges and smaller changes make it easier to manage and reduce the build-up of technical debt.
In conclusion
I, personally, think trunk-based development has clear advantages, and I have experienced them first-hand working in teams that have adopted this approach. However, it requires a mindset, a culture, within the development team. You need to frequently merge in other developers' changes into your own code. You need to commit small changes, frequently, which requires you to only change small sections of the code and make incremental changes, something which can be a difficult habit to get used to. Pair programming, comprehensive automated testing, and maybe code reviews are key practices to helping all the team to adopt the same approach and culture.
Trunk-based development, done in a disciplined way, streamlines the development process, enhances team collaboration, improves code stability, supports efficient CI/CD practices, and may result in less technical debt. While it may be challenging to adapt to this approach if you've been working with a branch-based model, the long-term benefits are worthwhile. If you want to change the way your team works to move to a trunk-based development model, you may also want to read Dave's article addressing barriers to trunk-based development.
It Actually exposes the skill weakness of the team faster..that makes them level up faster too..
Is that because you can see the code other dev’s are writing? I like that, it’s a great angle. I encourage a WIP branch to be immediately posted and hopefully the dev will ask for sanity checks.
Yes exactly! You’re seeing changes other devs have made pretty much as they make them. It means you’re not only up-to-date with what’s happening in the code base, the whole team is effectively code reviewing all the time.
As always great post, the ones that make you think regardless you agree or not at first, referencing brilliant people.
But what about code reviews? Trying to piece together 10 commits for a single fix/ticket/issue is nearly impossible – anything complex isn’t going to be done in a day.
I use a different style of code review for this development style, which I’ve talked about here. If you’re interested, I outline a member of different approaches to code review in a series of blogs over on that site.
The important thing to have clear up front is WHY are you doing code reviews? Then you can focus on the how and what you’re supposed to be reviewing. I’ve talked about this too: https://youtu.be/jXi8h44cbQA
Interesting, I never understood TBD to mean “no branching” I always took it to mean “single source of truth” the issue of models like gitflow are these long lived branches you work on and have interweave commits onto.
We’ve been following TBD at scale for years now and it strikes the best balance I feel between active repo’s (we have around 15 engineers pushing changes in our team) and controlled flow https://trunkbaseddevelopment.com/#scaled-trunk-based-development
Wonder what you think about this style of development and whether for you it is no longer TBD?
Idk about this, seems like all your problems with branching have to do with poor repo hygeine and cause the same problems in a trunk-flow, except the latter makes it harder to remediate, since _everything_ gets disrupted until the conflicts can be sorted.
Keep PRs small and focused on a single bug/feat/enhancement/chore, and you’ll still be ‘continuously integrating’, and you magically get all the benefits you ascribe to a trunk-flow _and_ without any of the awkward gymanstics
You could equally argue:
“seems like all your problems with trunk-based development have to do with poor commit hygeine and cause the same problems in a branch-flow, except the latter makes it harder to remediate, since you only find problems at the _end_ of the feature and then you have to spend ages sorting out the conflicts.
Keep commits small and focused, and you’ll be ‘continuously integrating’, and you magically get all the benefits you ascribe to a branch-flow _and_ without any of the awkward merge conflicts or painful code reviews”
It all comes down to discipline, and the team. For me, I prefer to see IMMEDIATELY if there are any problems caused by any developer’s commit. I prefer people not to commit if the trunk is red. I prefer people to fix problems AS SOON as they occur, and if they cannot be fixed inside 5 minutes (say), back out that commit and fix it locally. I prefer to work on the same codebase as everyone else, and not some isolated branch, even if that branch is only a day old.
“trunk based development” is really more about “short-lived branches” than anything else. Or at least all the advantages come from that.
Because even if you only ever work on main you absolutely do have a local branch that is separate from the remote branch and all the problems you enumerate could just as easily happen there.
The only difference is whether you merge origin/main into your local main or a differently named branch. If you don’t merge your code will run apart in both situations.
Now with short-lived branches this isn’t as big of a problem, because.. well the article does a good job of listing all the advantages.
Using separate short-lived branches gives you all the advantages listed here, but also allows you to easily have a quality gate that stops you from breaking the build and impeding others.
Because nobody likes to merge main and find out that they can no longer build or that half the tests are red. And even fewer people like the stress that comes from just having broken the build for 50 other people and having to scramble to fix the problem. There’s just no reason to not have a good quality gate and automated tests.
“trunk based development” is really more about “short-lived branches” than anything else. Or at least all the advantages come from that.
Because even if you only ever work on main you absolutely do have a local branch that is separate from the remote branch and all the problems you enumerate could just as easily happen there.
The only difference is whether you merge origin/main into your local main or a differently named branch. If you don’t merge your code will run apart in both situations.
Now with short-lived branches this isn’t as big of a problem, because.. well the article does a good job of listing all the advantages.
Using separate short-lived branches gives you all the advantages listed here, but also allows you to easily have a quality gate that stops you from breaking the build and impeding others.
Because nobody likes to merge main and find out that they can no longer build or that half the tests are red. And even fewer people like the stress that comes from just having broken the build for 50 other people and having to scramble to fix the problem. There’s just no reason to not have a good quality gate and automated tests.
PS: Apparently we’re back in the good ol’ days where you need two crlf for one visible newline -someone can delete the other post 🙂
This is a great write-up. I think we absolutely need to question trends in development, particularly when it comes to process methodologies. I do entirely disagree with you though!
This sort of process would be untenable for a huge number of enterprise teams imo. The amount of enmity and stress it would produce for the entire team to be constantly beholden to its weakest links would not be sustainable. Have you ever heard of a “blanket party” in military culture? It seems to me that you may prefer it essentially because it forces softskills solutions to problems.
The weaknesses of a distributed vcs strategy can be mitigated with small simple convention and process improvements, iteratively and progressively, over varying time scales. The weaknesses of trunk-based strategies demand immediate and intensive commitment of your human resources, cannot be mitigated except through solutions that invalidate the strategy (more than one trunk), and introduce a much higher risk of team and company instability.
The bottom line is that the strengths of both strategies don’t adequately address the real issues in development and that is something you have to have a high level of vigilance team-wide to address no matter your VCS methodology. Trunk-based strategies just make you deal with them more immediately, that’s the only value I see, but that value is also a liability in a lot of professional environments.
I prefer short lived branches with force-rebase-before-merge workflow, with allowance for small commits (that don’t require review, like typo fixes, doc improvements…) directly pushed on the trunk.
The maintainer of the branch feature has to keep up to date with master, regularly rebasing his branch on the trunk.
Same here! CI can still be a thing if I rebase my branch regularly. The bright side of short-lived branches with force-rebase-before-merge is code isolation, which trunk-based development doesn’t deliver.
Is it not much more “sizing”?
Trunk based is much better for small things (and most things we deal with are small). “You have four hours on this feature and are pushing it”.
When those things become larger (in a coherent unit of work sense, think this might be a month or two of work) then branch based starts to make a lot more sense.
I think not integrating your code (either from trunk or to trunk) for a whole month is exactly what leads to the anti-patterns of feature-branch development. If you’re working on a huge piece of work, sure it makes sense to make sure this continues to contain changes from other developers? And surely there’s a way to break up this piece of work into smaller bits (even/especially if they don’t “add value” to the user yet) and commit them back to the trunk? It seems like a recipe for disaster to have code living on its own for so long.
I concur with other comments that trunk-based development isn’t about only pushing commits to main. I was surprised to see that this was the way this article defined this. I’ve always understood it to mean branches simply must be off of main and merged quickly back into main. Which is the practice most people follow with regard to feature branches.
I disagree with point 5 “Reduced Technical Debt”. To the contrary, you could make the argument that if you don’t use feature branches, “dirty” code can slip in much more easily.
In addition, if you do code reviews (which you should), this is of course way easier with branches.
Having said that, yes, of course there can be situations were trunk-based developments has its advantages. Personally, my impression is that trunk-based development is fitted for new software (or a complete new set of features in an existing application), and feature-based development is for when that initial implementation is done.
I agree 100% with the author on this one. Also, the DevOps Research and Assessment (DORA) research program, which interviews thousands of high-performing teams every year, found the same thing. Teams achieve higher levels of software delivery and operational performance if they use trunk-based development. Here are their recommendations for how to implement it: https://cloud.google.com/architecture/devops/devops-tech-trunk-based-development
The best team I worked in followed straight to main tbd practices.
It required discipline around the use of feature flags and small, incremental, working commits.
I’ve only worked in the one team that did it & I’ve yet to see a team run that well.
It does require discipline, and a shared understanding how the team expects things to be done (e.g. feature flags), it’s true.
“Integrating small changes regularly into your code is usually less painful than a big merge at the end of a longer period of time.”
I would love to see some research on this point, because it is the opposite of my experience.
It’s also the opposite of what we see in other fields. You don’t hire your plumber, electrician, cabinetmaker, and painter to all work in your kitchen at the same time.
Sure, but you can hire a plumber, electrician, cabinetmaker, and painter to work in your house at the same time. Regardless of whether you’re working on separate branches or working on the same branch, it’s probably not a good idea for multiple developers to be working on the same section of code at the same time. In fact, if you’re working on multiple branches, you are going to see a big nasty merge at the end. If you’re on the same branch, you’ll effectively be regularly integrating all of those changes as you go.
I refuse to accept “if you’re working on multiple branches, you are going to see a big nasty merge at the end” as a necessary truth. That’s only the case if the feature branches are never rebased into main/trunk—effectively bringing recent changes into their branch, just like trunk-based development. You state that as a necessary characteristic of branch-based development, but it’s just not. It’s a possibility, but not a requirement.
We rebase our features branches regularly and don’t see “big nasty merges” as stated above. And at the same time, we break the main/trunk much less often than if we practiced trunk development.
I assume then your team only works on one release at a time? Teams i work on generally are working at least 2 or 3 releases at the same time. How does that work?
Yes that’s an excellent point. Branches are pretty much mandatory for multiple releases. Of course it all depends on the type of application you’re writing. Library code almost always needs different versions that are “live” at the same time, and some types of services too.
We easily solved same problem and still have benefits of branches by having feature branches off a single master branch that auto rebase. CI forces branch to rebase on master and test that combined form so your changes are being integrated consistently even when not yet merged. If it can’t auto rebase CI fails and developer must fix their branch before they can continue development. Even if a branch is worked on for months its still always up to date with what everyone else is doing.
This is a nice compromise!