An overview and brief introduction
This is the first in a series of posts that get to the heart of what a data-driven organisation is for a given organisation. The problem for an author is that each organisation will have its own goals and needs. So a one size fits all notion of a data-driven organisation is either going to be far too vague to be applicable (i.e. measure all the things, then apply mathematical reasoning to come up with decision making!) or simply not relevant. To get around this, we are going to focus in on an area that is vital, common to all organisations and will bootstrap the organisation into getting itself into a position where being ‘data-driven’ is just the de facto mode of being. It will provide tools, methods and examples that can be applied elsewhere.
This post is about why we should care about data-driven organisations, the specific problem that drives our inquiry into this area and forms the basis of the direction in how this large topic can be broken down. Before we can define a solution to a problem, we need to understand the problem it is attempting to solve.
Every organisation needs to make decisions - if you are reading this, hopefully you want to base these decisions on measuring the external world - IE data. This is the broadest and least useful understanding of a data-driven organisation.
Technique
Decisions can be made in many ways - after all, it is better to be lucky and technically wrong than unlucky and technically correct! However, in any field, bad technique will eventually limit your ability to progress. In organisations the technique we care about is decision-making ability, Superficially it appears that an organisation that makes decisions at all levels based upon data is a data-driven organisation.
What we are going to go into here are a set of key problems that need to be addressed as part of the core of developing such a strategy. The nub is that if companies knew what and how to measure, they would already be data-driven organisations. So a definition based around everyone measuring, analysing and having access to data to increase competitive advantage demonstrates a dangerous misunderstanding of the problem that is being addressed. If they follow this definition and start to measure things and analyse the data they are going to invest in a lot of extra work with very little understanding of how this will generate any return. It will be a return of the same practises that led to asking ‘how can we make better decisions’ in the first place.
The Problem (or the Why)
We know from looking at the history of whatever industry you are in that we are not very good at successful projects. What makes you think you are different? Do you have a way to answer that with confidence? How are you moving forward?
Dunning-Kruger
A man once robbed a bank without a mask, was caught and prosecuted. He proclaimed ‘But I wore the Juice’. Somehow he thought that if he wore lemon juice (because it is used in invisible ink) the camera wouldn’t be able to see him.
This is quite a spectacular example of a set of assumptions that could be good in a different world, if only the individual had tested them. As a result, there is a paper on this cognitive bias by Dunning and Kruger called ‘Unskilled and Unaware of It: How Difficulties in Recognizing One's Own Incompetence Lead to Inflated Self-Assessments’.
This is especially interesting as it is an example of a local optima. The problem with moving from one peak to another is that it is necessary to descend into a valley of chaos.
Why Should You Care?
- How do you know that you are not in the midst of a Dunning-Kruger scenario?
- Where does this confidence (or lack thereof) come from?
Question
Organisation-wise, we can restate the above questions:
- What happens if we collectively have found ourselves in a reinforcing system that keeps us all trapped in this state with regards to organisational skill and awareness?
- How can you tell?
How Does Your Organisation Learn?
The hypothesis is that we are trapped and therefore, are not learning because every industry has projects running late and over-budget - yet every industry has examples of organisations that buck this trend. Even when they are finished, things have been cut (and added). This is why we tragedies like the Grenfell Tower Disaster occur and we still have the same cladding on buildings a year later. Similarly, we get web projects that are not only a year late, but cost more than the system being replaced and provide fewer features.
My challenge to the reader is to prove the hypothesis wrong; what follows in these posts are a set of ideas and strategies to get into the heart of the problem.
That we are bad at software projects was true 20 years ago and is still true today - this suggests that we are bad at this and we have not gotten better. We are stuck in a Dunning Kruger scenario - despite some interest the status quo has not moved. But it also suggests that there is a lot of space from innovation here.
Dunning-Kruger is where an entity is unable to evaluate its ideas and therefore unable to move in a directed fashion - because its parts move independently and in conflicting directions.
The Danger of Fads
There have been many approaches to this over the years, however, many of these approaches are based upon received wisdom of management consultants - usually based on fads. A great example of fad-based advice that is easier to relate to is dietary advice.
With dietary advice, results of analysis have so much noise and variance it is very hard to know what is true and what is not. The result is that it is next to impossible to evaluate all the different and contradictory recommendations in an objective way - so most stay safe and restrict calories and take exercise.
The problem with dietary advice is that some will be correct and some will be wrong (Assuming that some are true!). We have no way of being able to tell and so we find a compromise - i.e. we reject both the correct and false to take a new position that must be less than optimal - it is not correct and has some false in it. As a result, it is more than likely that very few people have that ‘healthy’ diet we all strive for - but what we have is okay. Yet according to NHS Statistics, not only is obesity going up, admission due to it is going up but 97% of adults are active. Other stats show overweight as being fairly constant - yet obesity is increasing. Maybe diet advice is contributing to obesity? (But we will see correlation vs causation later). Maybe this compromise is far more damaging than our primitive brains realize!
Rather than compromise like this, we are going to look for situations where we can find false assumptions and try to break contradictions by finding a context in which we can meet both conditions ie find win-win scenarios. Here it is the desire to maintain a healthy body against 2 contradictory hypotheses - perhaps we can look at the context and determine that in some situations both are valid, yet in others one is false. Now we have testable hypotheses.
Given the complexities and feedback loops in the largely invisible world of biochemistry this compromise is understandable. Fortunately, companies are far simpler and can become largely observable.
Scientific Method
Scientific method consists of making a hypothesis from logic and then testing this through repeatable experiments. Understanding the effect of changing something requires isolating that thing and changing it on its own to measure the effect and graph the result - from this we can build up complex relationships and test them.
If we want to be data-driven, we need to not only react to data but also know what that data means. We do this by constantly building our understanding of the external and internal world of our organisation and running tests against it.
Therefore, we need to not only have the tools to carry out this analysis, but we also need to know what we want to measure. More importantly, we want a powerful logic that enables us to identify and change only one part of an organisation at a time - so that we can measure its effects upon the organisation as a whole.
Fortunately, someone else has already done a lot of this work, his name is Eliyahu Goldratt and he came up with the Theory of Constraints. Using this in conjunction with data-driven organisational goals and principles is a powerful focusing tool.
Hint at Resolution
The focus here will be the use of being data-driven to make an organisation far more capable of meeting its goals. This involves understanding how to define goals, how to align areas to them and break them down, how to help people represent their work in terms of the measures derived from organisational needs and so meet the needs of the whole. The goal is to avoid the idea of there being localised goals at all (For example a departmental goal that doesn’t directly connect back up).
Technical Debt as epitome of the problems to overcome
One example of this could be the concept of technical debt. It is a terrible phrase - it implies that there is something technically correct beyond that which serves the needs of the business. When people want to get technical debt out of a system there is push back. The organisation has goals it wants to reach and ‘technical debt’ isn’t in those terms.
Moreover, the ‘debt’ is there because a decision was made to not do something because it didn’t align with a larger organisational goal. It is a very good example of a local measure - a group of developers don’t like a global decision. The term seems to have value but really just serves to work against the real goal. When it comes to ‘fix’ this there will be a very simple business case - this work needs to be done now because we want to do x, in order to make x faster and safer to do we need to do work y. It all just folds naturally into a piece of work without any hoo har.
Technical debt exists in organisations where product and development teams are not talking the language of organisational needs. To have ‘technical debt sprints’ is the most organisational contradictory thing that I see and really it is hiding some other problem - but that is a topic for another time!
Optimise the Whole
So, the solution will be to look at organisations as a whole and run experiments that benefit the organisation rather than its individual parts. By reorienting around the whole, we look to use logic and data to hypothesis and then careful measurement with statistical methods to evaluate and gain new direction.
The resolution cannot be a control structure; we need an organisation to organise itself into being successful, otherwise we only have the expertise of the very few making those decisions.
During the process of doing this, a reorientation of everyone's decision and judgement techniques needs to be realigned towards a global direction rather than local. Nobody can be told to do this, it needs to be a behaviour that the system induces through its needs (Demming). Whether this happens organically if done well or everyone becomes aligned is moot, this is really governed by the structure of the system (we will see more of this later with Demmin and Sange whilst discussing learning organisations). Once we think we understand our system and start to focus on changing the system (ie communication patterns) to remove constraints in order to meet needs then - precisely because we meet people's needs - the behaviour will emerge.
What Next?
Thanks for reading! Next time we will look at what a data-driven organisation will look like, from the perspective of what and how it does. This will get into data vs information, correlation vs causation, behaviour, learning organisations and systems thinking.
