Since entering the agile transformation facilitation business in 2004, we’ve fielded a huge range of questions and concerns from all types of organizations – from corporations to non-profits to government agencies. While every case has its particulars, they often share a common set of concerns – ones apparently not readily answered by the standard characterizations of agile and Scrum. The following are answers to a few of the most common questions regularly asked by those investigating agile transformations.
Q: With agile, how do we measure success?
We want to be more productive without quality suffering; What metrics are important and how do we measure productivity within an agile environment?
A: Agile trades the attempt (and usually the failure) to deliver everything we can think of at the outset of a project – regardless of business value – for the consistent delivery of the highest-value features first.
Success is measured by:
1. The increased ability to correct an errant course offered by frequent and regular inspect and adapt cycles: Is this what we wanted? Is this what our customers wanted? Should we adjust based on what we’ve discovered? If we’re failing, let’s know early on.
2. Consistently high quality as an inherent by-product of agile practice, not an afterthought: Whenever possible, for backlog items to be accepted as finished, they must be ‘3x Done’ – running, tested, and integrated (including documentation) – at the end of every iteration. If these criteria are not met, the item can’t truly be considered done, and must go back into the backlog.
3. The accumulated business value of all running tested features (3x Done) is the most important metric of progress, not a ‘phase complete’ designation or point on a timeline.
Q: How do we gain senior management support for expected short term productivity dip?
One barrier to implementing agile is the expected dip in productivity during change; how can this be quantified and balanced against improvement later on?
A1: Forego a ‘big bang’, top-down agile diktat for starting with a focused, well-protected pilot.
A well-planned, limited pilot effort localizes any dip in productivity to the pilot group. This is both less scary for management, and offers the economy of starting with smaller numbers for the requisite training and coaching. Secondary benefits are that smaller groups are easier to train and coach in unison; are easier to protect from outside influence; and therefore easier to control for proper organization and practice. Limited pilots are also more easily measured, with the benefits more easily quantified.
Once a well-protected pilot group begins to show success, the advantages of agile become apparent to the wider organization. The challenge then usually moves from the question of whether or not agile presents a benefit, to an investigation of how to scale the pilot’s successes.
A2: Real productivity before agile was probably abysmal, which is exactly why we’re ‘risking’ a new approach.
While it is true that teams new to agile and Scrum often show little or no velocity for the first few iterations as they adjust to the rigor of the framework, it should be remembered that this is likely no worse than the reality behind the picture painted by the numbers of the prior traditional approach. In fact, while their person-hours denominated estimates may lend the veneer of accuracy, traditionally-managed software projects’ actual productivity is rarely quantifiable – though what is quantifiable is their often spectacularly disappointing results.
In the beginning, numbers tracked in agile estimations and metrics are no more ‘real’ than those used in a traditional development process. But because of agile’s focus on managing the value of output rather than managing time-based inputs, the productivity of agile development teams becomes clearer over time, with the delivery of the highest-value features at frequent and regular intervals.
Q: How do we promise new features to customers?
How do we provide our customers with predictive release information?
A: You don’t; Instead of predictions that never bear out (because fixed features ≠ fixed scope), refocus on delivered value.
Fixing features and schedule virtually guarantees that scope & quality will slip as project realities emerge. The common conception of fixed features = fixed scope is a myth, because there’s always multiple ways to solve a software problem.
In Scrum, when a fixed schedule (at a consistent effort) is desired, customers must be involved at regular intervals to inspect and prioritize features based on unfolding understanding and discovered realities of the project. Over time, the rate of delivery of features emerges, allowing a projection of the delivery date of a given number of features based on this rate (Velocity). Instead of a blind promise to deliver an ever-expanding amount of work by a certain date, we promise to deliver consistent effort that is focused on only the highest-value features, and we build in transparency, constant customer input, and the ability to reprioritize at no extra cost.
Q: How is software testing affected by agile?
How do we integrate our testing functions and groups into an agile environment?
A: Continuous integration is a foundational practice of agile & lean development.
In Scrum, QA is integral to the definition of done – remember ‘3x Done’? Functionally, this means testers must be moved from their silos on QA groups and installed as full members of cross-functional feature teams. While this is often among the more difficult structural improvements organizations undertake, it is crucial to eliminating the bottlenecks and context-switching waste endemic to the practice of treating QA as an afterthought and outside service.
Once testers are fulltime participants of feature teams, the real work of continuous integration can begin in making both unit and integration tests standard practice for every iteration, with the ultimate goal of automating integration tests at every check-in. As the saying goes, daily builds are for wimps.
Q: How will our product managers budget for complete?
How does your solution identify a project as complete and therefore control budget spend?
A: The accumulated business value of your running tested features (to 3x Done) is your metric of progress, not a ‘phase complete’ or point on a timeline.
• When release dates are discretionary, a common approach is to release when the highest-value features have been completed and return on effort expended is decreasing. An ROI figure based on the ratio of business value to estimated effort can be used to signal this threshold.
• When release dates are fixed, features delivered are discretionary, focusing on those that represent the highest value to the customer/business.
Q: How do we measure employee performance?
How is individual performance of team members measured in an agile environment, and how are associated HR procedures for non-performance affected?
A: With agile, the focus on performance shifts from the individual to the team.
Consider the Xerox study: 152 teams of 3-9 members*
• ”The group reward condition resulted most consistently in high performance, although individual rewards worked just as well for the teams that were assigned solitary tasks.
• But when the task required teamwork, the group rewards resulted in the highest effectiveness….
• Interdependent teams benefit the most from group rewards.
• These are also the teams that enter group flow and generate maximum innovation.”
* Group Genius, Sawyer, 2007 (p. 72)
In addition to evidence that individual rewards for performance erode teamwork, there’s mounting evidence that extrinsic motivators such as ‘incentives’ and bonuses actually harm performance, possibly because they supplant or contravene intrinsic motivation to learn, succeed, and excel. Adopting agile practices may underscore these findings, suggesting a reconsideration of the utility of existing employee incentives and evaluations.
A start would be to not only reward teams instead of individuals, but also to empower teams to make their own membership decisions. Non-performing individuals will very quickly reveal themselves in a teamwork-intensive environment. Empowered, self-organizing teams may reduce the need for the ‘resource management’ activities of typical HR departments.