SAFe 4.5 DevOps Metrics – Scale Continuous Delivery with the Scaled Agile Framework

July 3, 2017 CollabNet VersionOne

The latest release of the Scaled Agile Framework®, SAFe® 4.5 includes focus on DevOps and the Continuous Delivery Pipeline. Our AgileLIVE webinar, SAFe DevOps Metrics, featured Scaled Agile Inc.’s insights on what is helping large enterprise organizations succeed at Continuous Delivery, DevOps and SAFe.

SAFe’s scalable DevOps approach

SAFe DevOps CALMR approachDevOps supports the entire flow of value. The CALMR approach speaks to Culture, Automation, Lean-flow, Measurement and Recovery. Culture is a shared responsibility across the organization from development to operations to security – everybody who’s building anything needs to be part of this. Automation is very important – making sure you automate the entire delivery pipeline. You can’t do just Agile or just DevOps. You have to merge the concepts together so lean-flow is an important part of the CALMR approach. “If you are building things in large batches and have very advanced continuous deployment automation, but are still developing large batches, it doesn’t help you.” said Inbar Oren, SAFe Fellow, Principal Consultant at Scaled Agile, Inc. If you want to release more often, your need to be able to recover and monitor risk.

Hypothesis-driven Continuous Delivery Pipeline

SAFe hypothesis driven continuous delivery pipeline A central change in SAFe 4.5 is the concept of a hypothesis driven continuous delivery pipeline. When it comes to DevOps a lot of people just focus on delivery pipelines but Scaled Agile feels that continuous delivery starts much earlier if the software development lifecycle.

Inbar Oren shared that you need a holistic view of from exploring new ideas and what customers want to integrate those ideas into larger and larger things. This is bigger than just automation or having a continuous build process. We need to look at it as a full loop of hypothesized build, measure and learn.

Continuously Explore Customer Needs

Continuously Explore Customer Needs Measuring continuous exploration is the first of four elements of the continuous delivery pipeline. Product managers need to talk to internal people, system architects, business owners, and customers as they do market research. Scaled Agile puts a lot of emphasis on the lean startup cycle and Lean UX cycle at this phase. This concept of fostering innovation with the lean startup cycle is crucial to the understanding of SAFe 4.5.

Foster innovation with the Lean Startup Cycle

Foster Innovation with the Lean Startup CycleYou need to enable decision makers at the portfolio program level to take their efforts, break them into minimal viable products, build them, integrate them, deploy them, release them, and measure it to see if it is actually providing results from adding features.

Increase experimentation over time

Increase experimentation over timeSAFe 4.5 now refers to the epic value statement as a hypothesis statement. The hypothesis statement describes the purpose of an epic and how it is going to be measured. The larger and more complex your organization is, the harder it is to have an idea, develop a hypothesis, get it to market and test the hypothesis. Inbar Oren explained that they see a lot of companies who have a hypothesis but don’t acknowledge it or only measure them once a year and that they want to change this and see more hypotheses tested over time.

Continuous integration of components, systems, & solutions

continuous integrationInbar Oren shared that from Scaled Agile’s prospective, continuous integration is bigger than just integrating on a dev machine or at the team level, its integrating at all the different levels of the organization. Far too often there are changes or commits made to the body of code that can’t be tied back to specific business value (backlog item, epic or feature). As we move faster we need to get better at audit and compliance and being able to be sure about what changes have been made in software. If we don’t connect every single commit back to business value, it becomes really difficult to measure the flow of business value once we’ve converted commits into binary objects or into some compiled object.

Continuous integration metrics

Traditional Continuous Integration metrics apply

  • Build length
  • Build fails
  • Testing reliability
  • Number of commits per developer per timebox
  • Rogue commits

More Continuous Integration metrics apply at scale

  • Feature cycle time from backlog to approval on staging
  • Cycle time from commit to System Demo
  • Cycle time from System Demo to Solution Demo
  • Cadence of demos

The focus of SAFe DevOps metrics is to measure how long it takes to develop an idea and deliver it to your customers and the cycle time at different stages of the software development lifecycle.

Release on-demand

release on-demandScaled Agile suggests focusing on releasing on-demand as opposed to releasing anytime. There may be business reasons that you can’t release but you can always deploy. Start looking at your solution to see what you can decouple into smaller chunks of value that can be delivered at different times or subsystems that can be released or security fixes that can be done. You will find you can actually do more deployments that you may think. Deploying more often ensures that your software is tested and ready to go.

Measure the entire value stream

Measure the Entire Value StreamHow do you measure an entire value stream? Oren suggested using a Program Kanban board to visualize the flow of work from idea to completion.

Identify bottlenecks with Cumulative Flow Diagrams

Identify bottlenecks with Cumulative Flow DiagramsWe’ve leveraged cumulative flow diagrams in the agile world for quite a while but we are now bringing the cumulative flow diagram all the way to the definition of done, or at least to delivered value, to identify bottlenecks and opportunities.

Map pipeline efficiency

map pipeline efficiencyIn DevOps, efficiency is a strong indicator of bottlenecks and where there are opportunities. It is a combination of touch time and wait time. If you know cycle time at any given phase, then to calculate efficiency you only need to understand either wait time or touch time and you can derive the others.


This is just a recap of our AgileLIVE webinar, SAFe DevOps Metrics. Check out the recording to hear more from Scaled Agile Inc. on what is helping large enterprise organizations succeed at Continuous Delivery, DevOps and SAFe.
SAFe 4.5 DevOps Metrics
Scaled Agile Framework and SAFe are registered trademarks of Scaled Agile, Inc.

The post SAFe 4.5 DevOps Metrics – Scale Continuous Delivery with the Scaled Agile Framework appeared first on VersionOne Blog.

Previous Article
Leading DevOps — Inspiring and Influencing Enterprise Agility
Leading DevOps — Inspiring and Influencing Enterprise Agility

I’ve had many conversations over the course of 2017 that I want to share with you. There are key components...

Next Article
Q&A with Read It Quik Editor Meghna Lal
Q&A with Read It Quik Editor Meghna Lal

I recently had the pleasure of being interviewed by editor Meghna Lal. We discussed several ...