Notes on CI / CD
Last edit: 2021-12-17
Some good starting places
Are you interested in continuous integration, continuous delivery, and/or continuous deployment? The following are, in my opinion, the best places to start:
- Accelerate by Forsgren, Humble, and Kim
- The DevOps Handbook by Kim, Humble, DeBois, and Willis
- Continous Delivery by Farley and Humble
I have a simple test I use to determine if teams are really practicing CI...In my experience, less than 20% of teams that think they are doing CI can actually pass the test.
The test you can take consists of the following:
- Engineers push commits to mainline (or trunk) every day, not feature branches
- Every commit triggers an automated run of a suite of unit test
- When the build breaks, it is almost always fixed within 10 minutes
Dave Farley has a more detailed explanation why feature branches are disfavored in a video he has posted.
Commit to mainline every day
My general rule of thumb is that every developer should commit to the repository every day. In practice it's often useful if developers commit more frequently than that. The more frequently you commit, the less places you have to look for conflict errors, and the more rapidly you fix conflicts.
It's really important to CI that commits are made to mainline every day. To achieve this end, CI practitioners favor trunk-based development.
This alone rules out most software teams that think they are doing CI. Today, it's far more common that only a pull / merge request gets opened daily. Notice that this does not meet the definition above, which is that every developer on the team commits to trunk at least once per day, and even more than that, if possible.
More importantly, what would a software team actually get out of CI / CD?
The book Accelerate has a valuable summary of a few points, excerpted here:
Our research also found that developing off trunk/master rather than on long-lived feature branches was correlated with higher delivery performance. Teams that did well had fewer than three active branches at any time, their branches had very short lifetimes (less than a day) before being merged into trunk and never had “code freeze” or stabilization periods. It’s worth re-emphasizing that these results are independent of team size, organization size, or industry.
Even after finding that trunk-based development practices contribute to better software delivery performance, some developers who are used to the “GitHub Flow” workflow remain skeptical. This workflow relies heavily on developing with branches and only periodically merging to trunk. We have heard, for example, that branching strategies are effective if development teams don’t maintain branches for too long—and we agree that working on short-lived branches that are merged into trunk at least daily is consistent with commonly accepted continuous integration practices.
We conducted additional research and found that teams using branches that live a short amount of time (integration times less than a day) combined with short merging and integration periods (less than a day) do better in terms of software delivery performance than teams using longer-lived branches. Anecdotally, and based on our own experience, we hypothesize that this is because having multiple long-lived branches discourages both refactoring and intrateam communication. We should note, however, that GitHub Flow is suitable for open source projects whose contributors are not working on a project full time. In that situation, it makes sense for branches that part-time contributors are working on to live for longer periods of time without being merged.
The rest of Accelerate shows even more evidence for practicing CI / CD. But, Fowler's previously-referenced article also offers this great point:
By doing this frequently, developers quickly find out if there’s a conflict between two developers. The key to fixing problems quickly is finding them quickly. With developers committing every few hours a conflict can be detected within a few hours of it occurring, at that point not much has happened and it’s easy to resolve. Conflicts that stay undetected for weeks can be very hard to resolve.