Estimate project
← All posts

What I learnt from Continuous Delivery Pipelines

Several takeaways from Dave Farley's book "Continuous Delivery Pipelines".

9 min read
DevOpsBook ReviewKnowledge Sharing

Someone wise once told me that life is too short for good books – you should spend it on the outstanding ones. “Continuous Delivery Pipelines” by Dave Farley definitely falls into that group.

Dave Farley has roughly 40 years of experience, 30 of them spent on large scale distributed systems, which makes him one of the best-known people in the DevOps community. Beyond his remarkable expertise, he regularly speaks at conferences and – probably most notably – co-wrote the highly regarded book “Continuous Delivery”.

This man has been working for longer than I’ve been alive! He’s also one of the engineers I look up to. Setting personal admiration aside, though, I think a résumé this impressive speaks for itself: Dave is someone you should listen to.

With that in mind, I’d like to pass on a few lessons I took away from his book.

A change of paradigm

The first thing that stood out to me was how a deployment pipeline is defined. I must admit – for some people, even quite experienced engineers, this may be a real paradigm shift. As Farley himself puts it:

The Deployment Pipeline is NOT:

  • merely an automated workflow that builds, tests and deploys
  • a set of independent Pipelines for building, testing and deploying
  • simply a bundle of tools and processes
  • a way of proving that new software is good1

And there I am. Nearly 10 years in the job, building pipelines that build, test and deploy software with an entire arsenal of tools and processes. And a 5-point bullet list wipes out my whole career. Cheers, Dave…

What it actually is

So if a deployment pipeline isn’t any of that, then what is it? Two definitions come into play here – the what and the how.

The what is Continuous Delivery – a holistic approach covering every aspect of software development, from the idea all the way to the finished product in users’ hands. It isn’t purely a technical discipline, since it also calls for the right organizational structure, performance and culture. All of these are crucial for encouraging collaboration, teamwork, and giving teams the power to decide about their code and share responsibility for it. It also means working in iterations and optimizing for learning through fast, frequent feedback.

Above all, Continuous Delivery is an Engineering discipline that applies the scientific principle of falsifiability, testing code with the aim of rejecting it – should even one test fail, the Release Candidate gets thrown away.

Consider for a moment what the previous paragraph means for quality. “This test fails all the time, no idea why, let’s just ignore it.” Not anymore! Either you write proper software and proper tests (preferably with TDD), or your code won’t get released. End of story.

The how is the Deployment Pipeline – a platform that helps development teams deliver high quality software. Its design should make it possible to test ideas and change things safely, by gathering test results and generating data on stability, lead-time and throughput, so that decisions can be based on evidence.

It should also be arranged so that fast, technical tests run first, and be scoped to a unit that can be deployed independently (i.e module or microservice).

Most importantly, though, the Deployment Pipeline determines releasability and is the sole path to production. As a result, it includes every step required to reach releasability – unit tests, acceptance tests, validation, integration, etc. As Dave puts it: “What we’re after is that once the Pipeline finishes, we’re comfortable deploying with no extra work required”2

This way of working pays off enormously – when everything that reaches production must pass through the pipeline, no manual change can slip by unrecorded. Let’s be honest – quick and dirty manual hot-fixes are the main cause of infrastructure configuration drift.

Going lean

The definitions above may look intimidating, but keep in mind that Continuous Delivery takes a lean approach. Its focus is on cutting lead times by getting rid of waste – duplication, delays, convoluted organizational structures, etc.

It follows that the Deployment Pipeline should include only the steps strictly required to release the software to production, and execute each of them just once. Anything that departs from this principle is waste and should go, both to keep the process simple and to shorten lead time.

The opposite of this is every kind of gate-keeping that managers are so fond of – change management boards, manual sign-offs and a risk-averse attitude to change (“If it works, don’t touch it!”). According to the DORA report, these processes correlate negatively with lead-time and deployment frequency, and show no correlation at all with change failure rate. They are wasteful and should be eliminated or cut down to the bare minimum.

A brief look at agility

Unsurprisingly, Continuous Delivery has deep roots in agile. In fact, the name itself comes from the Agile Manifesto: “What matters most to us is keeping the customer satisfied by delivering valuable software early and continuously”. Accordingly, the book offers some ideas for putting agile practices into action.

The most evident one is working in small iterations – ideally so small that new code is committed straight to Trunk/Main every 10 - 15 minutes, and at least once a day. The goal is for the pipeline to evaluate the code continuously (not deploy it!). Yes, you read that right – Farley’s ideal branching strategy is “Don’t branch!”3

Another insight concerns acceptance testing – every acceptance criterion defined in a user story should be covered by at least one acceptance test.

A comforting anecdote

The book explains how to build a Continuous Delivery pipeline from the ground up, recommending an iterative approach: begin with the bare minimum (picture a few deployment scripts and a hard drive to store artifacts) and grow its capabilities along with the project.

It goes on to say that getting this minimal Continuous Delivery Pipeline in place may take a few weeks. Personally, I find that comforting.

Once (at a previous employer – this company is awesome!) I was asked to estimate how long it would take to build a pipeline completely from scratch, and my own manager mocked the 20 man day figure I came up with. It’s nice to know I wasn’t wrong, and to have that confirmed by someone far more senior than me.

Just an implementation detail

Everyone wants to use the newest, shiniest tools, right? Wrong! Technology alone won’t get you anywhere. In fact, in Continuous Delivery, tooling is nothing more than an implementation detail.

Near the end of the book, Dave describes the case of LMAX (London Multi-Asset Exchange) – one of the world’s highest performance financial exchanges. In his words: “At one point or another, we migrated most, if not all, of the tools and parts of the system. We picked whatever technology looked most suitable at that moment, and we changed and evolved implementations and decisions as the system grew and shifted, and as we learned more about what was needed and what worked.”4

Now that’s agile, architecture and paying down technical debt! None of it would have been possible without design, approach, patterns and techniques guiding the project from the very start.

Conversely – had the LMAX team chosen a poor architecture and approach, they would have been stuck with the tools they picked at the outset, and the entire project would have ended in disaster.

Side note – the book does describe the recommended architecture, but I won’t give everything away. Buy the book – it’s well worth it :)

A word about test environments

In the end, you always test in prod. Before you flay me for such heresy, let me explain. Real deployments, and the disasters that come with them, happen in production. That’s undeniable. Whatever testing we do, it can’t guarantee we’ll be spared problems in production – regardless of how many “pre-prod” environments we run.

What we can do is reduce the chance of issues by making sure every single environment is deployed and configured with exactly the same tooling and scripts. Then each pre-prod deployment also serves as a test of the process itself.

A complete, all-round engineer

Dave Farley undoubtedly has a one-of-a-kind perspective. What struck me hard, though, was understanding where it comes from.

Dave is, above all, a software engineer. He spent years writing millions of lines of code long before any DevOps tools existed. He knows the development process thoroughly, and that lets him come up with efficient, scalable automation solutions.

Every DevOps Engineer should consider it a matter of honor to reach a comparable depth of understanding, because it opens up an entirely new view of our roles and the work we do.

Looking in the mirror

Across its 145 pages, the book is packed with checklists, bullet points and rules of thumb. For instance:

  • Commit stage tests finish in less than 5 minutes
  • Give yourself 10 minutes to commit or revert a change
  • Run the whole pipeline in less than an hour

I encourage (or challenge) you to examine all these guidelines in the context of your own work and think about what you could improve. To be honest, my most frequent response was a loud, sorrowful “Ouch!”.

Closing thoughts

Continuous Delivery provides a consistent approach to building and deploying software, built around technical practices (test driven development, pair programming, trunk-based development), along with modular architecture and shared responsibility.

These are exactly the things we tend to dismiss as unfit for the “real world”, because “We need to ship features quickly and this holds us back”. So the obvious question is: “Does this approach scale?”. The book offers some answers.

To start with, it cites the DORA report on how Continuous Delivery affects software development:

  • 44% more time devoted to features
  • 50% higher market cap growth across 3 years
  • 8000x shorter deployment lead time
  • 50% less time spent on fixing security defects
  • 50% lower change-failure rate
  • 21% less time spent on unplanned work and rework5

And what about LMAX – the project Dave takes such pride in? Well… let’s let him speak.

“Together with the software, the Deployment Pipeline grew to cope with nearly 100,000 tests, process 1.3x Twitter’s daily data-volume, routinely handle £100 billion worth of asset transactions per day, and archive more than 1TB of data each week.

The pipeline was integrated with 20 to 30 external third-party systems of many different kinds and purposes: from exposing APIs for external trading, to intricate integrations with clearing houses and customer verification systems.

Any change to any part of the system was automatically compliant with regulations, since producing all the FCA documentation and meeting audit requirements were baked into the Deployment Pipeline”6

That’s the thought I’d like to leave you with. If you’ve read this far – congratulations, and thank you. I hope this text was useful and gave you something to think about. And one more time – get the book. It’s worth it :)


  1. Dave Farley, Continuous Delivery Pipelines: How To Build Better Software Faster (Great Britain: Amazon, 2021), 13 ↩
  2. Dave Farley, Continuous Delivery Pipelines: How To Build Better Software Faster (Great Britain: Amazon, 2021), 62 ↩
  3. Dave Farley, Continuous Delivery Pipelines: How To Build Better Software Faster (Great Britain: Amazon, 2021), 44 ↩
  4. Dave Farley, Continuous Delivery Pipelines: How To Build Better Software Faster (Great Britain: Amazon, 2021), 127 ↩
  5. Dave Farley, Continuous Delivery Pipelines: How To Build Better Software Faster (Great Britain: Amazon, 2021), 4 ↩
  6. Dave Farley, Continuous Delivery Pipelines: How To Build Better Software Faster (Great Britain: Amazon, 2021), 126 ↩