How to measure an Agile Transformation

Home » How to measure an Agile Transformation
How to measure an Agile Transformation
How to measure an Agile Transformation is paramount to succeeding in the long term. The true test of a successful Agile Transformation, will be continued (or sustained) success.

Contributors:
Jeffrey von Drehnen – Senior Consultant
Patrick Fitzgerald – Senior Consultant
Yi Jin – Associate
Terry Shea – Consultant
Rachna Verma – Senior Consultant

How to measure an Agile Transformation is paramount to succeeding in the long term. The true test of a successful Agile Transformation, will be continued (or sustained) success.

It’s critical to understand from the outset how the Agile Transformation will be measured. Otherwise, you will essentially be ‘following the rules’ of the game, without understanding how to win.  

The organisation’s Agile maturity will be determined by how seamless the measurement practices are in place.

The scenario is you are part of an organisation that is about to begin or starting an Agile Transformation.  There is an assumption that you understand what an Agile Transformation is, or at the very least – you understand the “why”.  

You may have been part of an pilot agile program that’s now being scaled to the rest of the organisation.  That is, Agile at Scale (aka scaling agile).

At ADAPTOVATE, we use the seven enablers required to move the needle in an Agile Transformation.   Examining these seven enablers allows us to understand what we need to focus on to improve.

The seven enablers are:

  1. Alignment on purpose
  2. Certainty of team and funding
  3. Autonomy and collaboration
  4. Empowered people with authority
  5. Growth Mindset Culture, with test and learn behaviours
  6. Sustainable Agile and lean practices
  7. Flexible architecture and integration

On top of that – we understand the ultimate success of an Agile Transformation can be simplified down to improved performance in three broader areas:

  • Culture and Behaviour
  • Productivity
  • Value to Business

Where to start to measure an Agile Transformation?

Readiness Assessment

A readiness assessment is a good tool to measure and report on an agile transformation, because it shows the teams’ sentiments at the beginning of the transformation, and then at a future interval, and results can be compared and analysed.

When looking to scale agile in an organisation a readiness assessment is used. The assessment is based on the 7 key enablers of agility discussed above, and through a series of questions, you can understand how the organisation manifests these.

The assessment is performed in 4 stages: Prepare, Assess, Analyse and Recommend.

The assessment findings will identify 3 things:

  1. What the organisation must fix immediately
  2. What the organisation can fix in 6 months
  3. What the organisation can fix after 12 months

To commit to these changes an initial roadmap is created. Repeat the assessment xx months in the future and compare the results. These two data points provide you with data to measure and report on the success of the agile transformation.

Continuous Improvement

We understand one of the core tenets of Agile methodology is continuous improvement.  The metrics you use to measure the success of an Agile Transformation will not be purely the numbers at the beginning and the numbers at the end as described in the readiness assessment.

Rather we need to be looking at ongoing behavioural metrics during the actual transformation. Using predictive metrics will allow measurement of behaviours aligned with agile methods, that in turn will lead to stronger positive outcomes for the organisation.

Predictive Metrics

Predictive metrics is best described as a way to measure the levers that drive success, rather than the actual outcome metrics. 

Consider this example of a cricket team.  A cricket team’s ultimate goal is to score more runs than the opposition and win more games than other clubs, thereby winning the championship.   If that’s the only metric they use to measure how they are going, their success rate will likely not last long.    

Magic will happen when coaches start tracking a player’s levers. Eg: The ability of speed to catch and release a ball when in field, or a batsman’s zero to top speed when running between wickets, or how accurately the ball is released when bowling. 

Being able to track the levers that lead to the most runs, and then improve those metrics in the athletes by certain prescribed training, will ultimately drive a far stronger win rate.

It’s the same when measuring an Agile Transformation.  We know that being Agile requires a change in behaviour.   Measuring behaviour, not results will allow the company to track certain key indicators. These key indicators will show that there is a fundamental change happening within the business.

Surveys, interviews and observations are all means by which we can assess the Agile maturity within an organisation. Perhaps one of the most telling means by which we can assess Agile maturity is by observing the level of autonomy with which teams operate. The flow of communication and degree of collaboration between teams serve as strong indicators by which we can determine Agile maturity at a glance.  With that in mind think about our three areas of improvement:

Culture and Behaviour (team health)

Are we empowering our agile teams more to deliver the work?
Are we creating career paths to specialisation for team members?
Are we encouraging test & learn culture and becoming more tolerant to trial & error?

Productivity

Are we getting more visibility and transparency on the work to be done?
Are we more aligned to the organisation’s goals?
Are we clear on the priorities?

Value to business

Are we getting a better NPS score?
Are we responding to customers’ wants & needs, or general feedback, faster?


If the organisation has paid attention to driving the seven enablers, the metrics should be showing ongoing improvement.

If it’s not, you can ladder back from the 3 broader areas and start to unpack which of the seven enablers may be holding the business back, and you may pay more attention to specific areas for a time.   

Measuring increased outputs and speed to market may be possible for some businesses. Eg: how many widgets are we producing and how quickly – but even this can hide a lot of success.

Other types of behavioural questions to consider:  Is there improved COLLABORATION? Are we understanding our CUSTOMERS better than before? More FLEXIBLE to change? Producing BETTER widgets? How does the TEAM feel about coming to work?

Measure all levels and act on your retrospectives

Moving the organisation to a new level will require measuring the transformation at all levels. While there’s always a variety of methods to measure — it’s always important to keep taking account of all teams’ direction via retrospectives.

Even something as simple as, “What do we stop doing? What do we keep doing? What do we start doing?” helps us self-steer the group to a more fitting set of group norms.

As with most things in Agile, it’s important not just to “do”, but learn and make changes.

That is true with all the findings from measuring. Keep things simplified where you can when measuring, while learning the nuances of putting learnings into action. Measuring the behaviour, learning from findings and acting with continuous improvement will result in a sustained, successful transformation.

Adaptovate - Business Agility specialists
Adaptovate - Business Agility specialists