October marked the launch of Google Cloud’s highly-anticipated 2024 DevOps Research and Assessment (DORA) Report. If you haven’t read it yet, download it here. It’s a great read for all those involved in the software delivery process.
At Shipyard, DORA is at the core of what we do. We help teams ship faster and with more confidence. Once you’re able to do that, good DORA metrics will follow.
This year’s DORA report surveyed over 3,000 practitioners, and picked up on some new areas that are relevant to software delivery – it introduced Platform and DevEx to DORA’s scope.
Platform’s relationship with DORA
2023’s DORA report concluded that Platform teams are most effective when they treat developers as users. Studying developer workflows is essential to improving an IDP — it’ll help identify which stages undergo the most friction and blockers.
The 2024 report reiterates the importance of user-centeredness to Platform, and it goes further by discussing an IDP’s impact on software delivery. They found a few important takeaways:
- 89% of those surveyed were using an IDP of some form
- Teams using IDPs had 6% better organizational performance (combined score of four key DORA metrics)
- There’s a correlation between those who use IDPs and those reporting a higher change failure rate
- Teams using IDPs are slightly slower when it comes to deployment velocity (by 8%)
Platform Engineering
As we learned in last year’s report, Platform engineering initiatives benefit from treating Platform development as one might treat application development. This means Platform might prioritize collecting “user feedback” from developers, database admins, and security.
Platform engineering, at best, can be a net positive to developer workflows. However, when not done right, it ends up being quite detrimental. Increased complexity leads to burnout. DORA isn’t sure of the exact cause of burnout, but they have a few theories.
Internal Developer Platforms
DORA’s research concluded that IDPs are most beneficial when made of separate components that can be decoupled. Developer independence and self-service shouldn’t be overlooked in Platform design, but their tradeoff is higher Platform architecture complexity.
Another interesting insight was the suggestion to use SRE-type practices to monitor Platform stability, including SLOs and SLAs. Doing so will show how much developers can “trust” IDPs in higher-stakes use cases. Interestingly, developers who use IDPs tend to be more ambitious — sometimes justifiably so — and take more risks, assuming that the Platform acts as a safeguard against instability.
Impact of developer experience
Developer experience (DevEx) isn’t featured in 2023’s report. This might be because DevEx and pure DevOps are rarely analyzed together (try a Google search — surprising, right?); DevEx is generally reserved for the inner development loop. DORA recognizes why DevEx is relevant in the context of delivery.
2024’s DORA report focuses heavily on Platform, and it would be remiss to talk about Platform without looking at it through the lens of developer experience.
Like its findings on Platform, DORA recommends that teams focus on user-centric application design. This has implications for both developer burnout/happiness and product performance. Developers who are more involved in user research tend to feel overall more clarity when it comes to their own day-to-day. When developers take a user-centric approach, they can understand which features are most critical and prioritize them on the roadmap.
Internal documentation quality is closely linked to overall product performance. Additionally (like Platform), documentation should be written with an audience in mind. In this case, your colleagues and your future self are your doc’s audience. DORA also recommends incorporating user signals into internal documentation — this is important context around a feature and its expected behavior. And finally, documentation written for the sake of well, documentation, defeats its primary purpose: helping your team understand code with reduced cognitive load.
DORA’s 10 year retrospective
As the 10th annual State of DevOps report, the DORA team reflected on their past decade of research. When the State of DevOps first launched, DevOps itself was a young discipline (the portmanteau was coined in 2009).
If you’re interested in learning more about DORA’s ten year journey, Nathen Harvey’s presentation A Decade with DORA shares a timeline/narrative of the team’s history and findings.
DORA has become a cornerstone in the DevOps sphere. This report celebrated its evolution, and tied together years worth of research that has shaped DevOps processes everywhere.
Some of DORA’s most critical core findings have been:
- Organizations can have high throughput and high stability; these aren’t mutually exclusive
- High DORA scores result from efforts in software delivery and operational performance
- A healthy culture is tied directly to organizational performance
Bringing DORA into your org
One of the best parts of DORA is that it works well as an internal benchmarking system. DORA is designed to promote continuous improvement within your org. There isn’t an “across the board standard” for DORA (although Elite/High/etc performers show DORA trends) — the best metrics for your org aren’t necessarily the best for someone else’s.
DORA is most useful when you use it to assess, guide your improvements, and assess again. If your process/tooling/culture changes are working, you should see gradual, month-to-month DORA progress.
DORA metrics are simple to measure manually, and you can automate measurement by linking webhooks to the relevant deployment events. DORA even offers a “quick check” which can help you assess your DORA metrics, and see your scores relative to those of other orgs.
And as always, if you’re looking for an effective way to improve your software delivery performance, check out Shipyard. Using ephemeral environments has a positive impact on all four key metrics. Self-service environments cut out staging wait times, allow developers to test features more often, and can help expedite patches/hotfixes to production. Try it today.
