We’ve been blogging recently on Decision Optimization
- The Customer Journey to Decision Optimization
- Ongoing Decision Improvement Drives Decision Excellence
It is often hard to make the best possible decision. Those trying to improve and optimize their decisions report various challenges. One in particular is that they lack the data they really need – there are data gaps that make it hard to tell which approach will work best. For instance:
- There might be a lack of information about the effectiveness of approaches that have not been tried before.
- There is data about how existing customers respond to certain decision-making approaches but no data about those who were rejected would have responded had they been accepted.
- Some approaches have never been tried on certain segments – higher risk customers might never have been targeted with price reductions, say.
- There may be historical bias that limits the data available.
There are many possible limitations. Without this data, finding better decision-making approaches can be impossible.
The solution to these data challenges is to experiment. Instead of bemoaning the absence of data, run experiments to collect it. Deliberately try things (at a small, but significant, scale) you haven’t done before and see what happens. Try accepting people you might otherwise have rejected. Try counter-intuitive approaches on a small sample to see what they do. Pilot radical new approaches on a target segment. Experiment to fill your data gaps.
The most effective way to do this is to challenge your current approach with new ideas and new ways of deciding. Randomly select groups of customers and use the experimental approach on them, to prevent bias, and ensure a clean test Keep information on both groups – what you would normally do and what you experimented on – so you can compare the approaches later. Track how well each approach works. Gather the data you need to systematically improve your decision-making.
You don’t have to fill all your data gaps, just those with the most value You might target the most valuable groups or specific kinds of data that will enhance the data you already have. You can also use a mix of aggressive and conservative experiments to create a broader pool of data to inform your decisions.
Experimentation at the beginning of your journey is essential to make sure you understand where you are starting. But the most effective way to use experimentation is to integrate it with the improvement lifecycle so you are always running experiments, always gathering new data, always learning.
Gary Loveman, who left Harvard to become the CEO of Harrah’s Entertainment Inc., the largest gaming corporation in the world, famously said there were three ways to get fired from the hotel and casino company: theft, sexual harassment, and running an experiment without a control group. That’s a pretty hardcore attitude to experimentation but taking it seriously is definitely worth your while. Invest some time learning about experimental design and run your own.
This new paper puts experimentation into context as part of you journey to optimal decisions.