Friday, 28 February 2014

Intuition vs Analysis

Last week the acquisition of WhatsApp by Facebook was breaking news, a typical Mergers and Acquisitions decision. These types of decisions are one of a kind, need to be made in secrecy and at high speed as they can have huge impact on shareholder value. Many executives would argue that in these situations their intuition and experience is what makes the difference.  In their opinion, analysis would take too much time, moreover mathematical models can’t take into account the forces that either make or break the success of such a decision. They believe that intuition is indispensable when making business decisions. Just listen to Sir Richard Branson, Steve Jobs or Jack Welch and the compelling stories of their success in decision making. The counter side is that executive decisions are also subject to overconfidence. An executive might have a very strong feeling that a takeover, product or service will be successful, without considering the probability that a rival is already ahead in undertaken a similar action. Research from Daniel Kahneman shows that the amount of success it takes for us to become overconfident isn’t terribly large. Some executives therefore have achieved a reputation for great successes when in fact all they have done is take chances that reasonable people wouldn’t take.

The above quote is attributed to Albert Einstein. Whether he really has said it isn’t relevant, it strikingly describes the current shift towards more analytical decision making, which to my opinion is a good thing. We are surrounded by Apps, devices and sensors that gather data which is analysed to provide us with suggestions or offers that we are most likely to accept. It's a trend that is also entering the C-suite. However, the growing popularity of Big Data and “technically sophisticated, computationally intensive statistical approaches” has an unfortunate side effect: a “shut up and calculate the numbers” ethos, rather than one that promotes critical thinking and stimulates ideas about what the numbers actually mean. Question is where should the balance lie between intuition and analysis?

What is the best decision strategy, trusting your intuition of performing the analysis, is part of an ongoing debate. Two prominent actors in that discussion are Nobel Prize winner Daniel Kahneman and Gary Klein. Kahneman recently published a book Thinking Fast and Slow in which he analyses decision making and explains why we as humans are not very good at it, especially in situations with a high level of uncertainty. So maybe Intuition is not a very good basis for decision making. Gary Klein, writer of ThePower of Intuition, is a strong proponent on using your intuition in decision making.  He indicates that we need to take our gut feeling as an important data point, but conscious and deliberate evaluation is required to see if it makes sense in the context of your decision.  This suggests that intuition and analysis should go hand in hand.

The human brain, even the brain of executives, cannot oversee the vast amount of alternatives that usually need to be evaluated when making a complex decision. In my experience, with common sense and intuition a solution close to the best possible one (say close to optimal) can be achieved. But it would leave money on the table and might not take into account all relevant limitations, impacting the final result. In situations where margins are small, it could make the difference between a profit and a loss. When taking a more analytical approach, using techniques from Operation Research, new and sometimes counterintuitive solutions will be found that bring in the remaining value, taking into account all relevant limitations. There are many examples in which the use of Operations Research resulted in vast improvements that were not even considered possible.

One great example of a counterintuitive and successful solution from using Operations Research comes from Patrick Blackett, one of the founders of the Operations Research field of expertise. His analysis of the loss of ships crossing the Atlantic due to attacks by German submarines resulted in a breakthrough. The British and US Navy were convinced (by intuition) that single ships had a lower change of being discovered and attacked by German submarines than convoys. However Blackett showed that the use of convoys would improve the survival rate of ships crossing the Atlantic dramatically. It took some effort to convince the Navy, but when his advice was followed through the loss of ships dropped drastically.

What does it take to create a good math optimisation model or perform the right analysis you might ask? I would say intuition and experience. I think the process of finding the right approach or model is similar to the way in which experienced firefighters take split second decisions as Gary Klein mentions in Intuition at Work. He finds that firefighters are able to do a rapid and unconscious situation assessment and recognition from an array of stored templates followed by the taking of appropriate action when a fit is found. In modelling it is the same thing. After selecting a promising approach, validation and calibration of the model takes place. Sometimes this leads to a rejection of the model and a new way must be identified. Otherwise the model is used to do that analysis.

So, there is a balance in intuition and analysis even in creating math models. Therefore there is no analysis versus intuition but analysis and intuition. Use analysis to verify your intuition and apply intuition to find models. Even when the time frame is thigh, analysis can be used to at least verify intuition and bring new directions to consider or think about. Balancing intuition and analysis will boost the quality of decision making and train your intuitive mind.