DevOps Research and Performance, now known as DORA, popularised four metrics intended to answer a deceptively simple question: is this engineering organisation actually getting better at delivering software?
They are the four below. The rest of this article is about the ways they get misused, because that is where most of the confusion comes from.
The four metrics
| Metric | What it measures | Direction |
|---|---|---|
| Deployment frequency | How often production deploys happen | Higher is better |
| Lead time for changes | Time from commit to live in production | Lower is better |
| Change failure rate | Share of deploys causing a failure needing remediation | Lower is better |
| Failed deployment recovery time | Time to restore service after a bad deploy | Lower is better |
Note what is not on that list. There is no metric for number of tests, code coverage, lines of code, ticket throughput, or developer productivity. DORA deliberately measures outcomes, and the omission is the most important part of the model.
Why these four work together
Individually, each metric is easy to game. Together they form something much harder to fake, because improving one almost always degrades another unless you are genuinely getting better.
The obvious example: a team that merges to production forty times a day. Deployment frequency looks excellent. If twenty-five of those deploys cause an incident and each takes three hours to resolve, the team has not improved anything — it has automated the delivery of bad changes more quickly.
Move fast and stay safe. A team that is only fast has not improved. A team that is only safe has not either. DORA describes the quadrant where both move together.
Misuse one: deployment frequency as a target
Once a metric becomes a target, it stops being a measurement. Teams get pressure to hit a number, and the cheapest way to hit a deployment-frequency number is to ship trivial, reversible changes constantly — a comment fix, a config tweak, a no-op release.
That behaviour is easy to detect and pointless to reward. Deployment frequency is a signal of how comfortable the team is with change, not a KPI to be maximised. If someone can improve the number without improving the delivery system, the number was the wrong thing to look at.
Misuse two: lead time measured from the wrong starting point
This is the most common measurement error we see. Lead time for changes is defined as commit to production. Teams frequently measure from when a ticket is created, or from when a pull request is first opened, or from when a change is merged to main and the deploy pipeline starts.
Each of those is a different and much more flattering number. Time waiting in a review queue is precisely the time worth measuring, and it is excluded by all three.
# Where the time actually goes, commit to production
git log --format='%h %cI %s' --since='30 days ago'For a more honest breakdown, you want at least three separate numbers: time from first commit to first review, review wait time, and time from merge to production. The middle one is usually the largest and the most fixable.
Misuse three: change failure rate defined by the wrong team
A deployment causes a failure if it required unplanned remediation. That is the definition, and the words unplanned are doing a lot of work.
- Fixing a bug you shipped yesterday is a failure. Count it.
- A feature flag turned off during release is planned. Not a failure.
- Scheduled maintenance to patch a CVE is not a failure.
- Rolling back a release is a planned remediation. Not a failure.
Teams that quietly reclassify incidents to protect the ratio are not gaming the metric maliciously. They have usually been shown the metric without the definitions, and optimised the definition instead of the thing.
Misuse four: comparing across incomparable systems
Average DORA figures published across thousands of companies are aggregated from wildly different contexts: a two-person startup shipping a static site, and a regulated bank deploying to a mainframe estate. Averaging those produces a number that describes nobody.
Compare your team against itself over time. That is the only comparison that is reliably meaningful, and it is the one most performance reviews get wrong.
Pick a fixed quarter, publish your own four numbers, and compare them to your own four numbers from the previous quarter. A trend of your own is worth far more than a benchmark against an industry average.
Misuse five: treating recovery time as purely technical
Mean time to restore is the metric most sensitive to organisational factors, and the least responsive to tooling alone.
A well-run service with clear runbooks, known decision authority, and a rehearsed rollback will beat a sophisticated platform nobody understands during an incident. The fastest recoveries we have seen were almost never the result of a clever automated rollback — they were the result of someone who had done it before and did not have to think.
What we actually look at first
When we run an assessment, the four numbers are the starting point, not the finding. The useful questions are usually:
- Is the lead time dominated by queueing? If changes sit waiting rather than moving slowly, the fix is process, not tooling.
- What is the rollback story? If the honest answer is a database restore, recovery time is measured in hours no matter what the dashboard says.
- How many deploys in the last quarter were boring? This is the real signal. Uneventful is the goal.
- Who can deploy, and is that person available? A pipeline only one person can operate is a single point of failure wearing a CI badge.
The metric that is not on the list
Unprompted, we always ask how long it takes to undo the last deploy. If nobody can answer without checking, the recovery-time figure is theoretical, whatever the tooling claims.
Deploy less often if you need to. Deploy far more often once you can undo one in under five minutes. That single number is worth more than the other three combined.