We all know it, data doesn’t have any value whether it is big or small, structured or unstructured, available in real time or just sitting in your data warehouse. You need to process data to create insights and then act upon them. For example, being able to accurately forecast next year’s share price of a company doesn’t bring you any value, unless you decide to invest (or divest). Analysing data, creating predictions and determining the best possible action all require algorithms. Organisations are increasingly adopting algorithms to support them in decision making. Gartner expects that the use of algorithms will increase heavily in the next 5 years. Gartner SVP Peter Sondergaard envisions in his 2015 keynote that by 2020 there will be marketplaces similar to app stores where algorithms can be bought or sold. Algorithms can be bought to solve a specific problem or create new opportunities from the exponential growth of data and the Internet of things. Or organisations can monetise their algorithms by selling them to other organisations. The Algorithm Economy will bring the App Economy to analytics according to Sondergaard.
Algorithms are ill utilised
Algorithms are not new nor is interest in them caused by the growing amount of data or the Internet of Things. Algorithms have been around for a quite a while, some areover 3500 years old, and exist because we as humans had an interest in solving problems in an efficient and repeatable manner. Computers have sped up the development and use of algorithms allowing us to solve bigger and more complex problems faster and enables the analysis of vast amounts of data. Companies like Google, Facebook and Amazon use algorithms at the core of their business, it has been this capability for them to have become so big and influential. This is however not because they were the only ones with access to algorithms. Everyone can get access to state of art algorithms as many of them are taught in university’s maths or computer science classes. A well trained computer scientist or operation researcher can design and implement them for you. Some of them are even for free, open source statistical package R for example contains the latest and most advanced machine learning algorithms. What is striking is that, even though a lot of very advanced algorithms are easily accessible, not a lot of companies seem to be using them. In a Gartner survey from 2013 as little as 3% of the interviewed companies reported using prescriptive analytics, 16% used prescriptive analytics. So, even though we have the algorithms available, still a lot of companies are not using them. Why is that?
Algorithmic decision making requires high level of analytics maturity
The explanation is simple. Having the data and technology available simply is not enough, it’s a necessary but not a sufficient condition for success. For organisations to be successful with algorithms they need the technology, but also require the people that understand algorithms and decision makers that are willing to act upon the algorithm outcomes. Acquiring the right analytics talent requires finding people with the technical competency to design, build, assess and use algorithms, usually they have a background in operations research, mathematics or computer science. Next to that, the analysts must have the right business sense to understand the business problem and the right domain knowledge. Analysts need to have well developed communication skills so the right business requirements are identified, otherwise the right answer to the wrong question will be found. Besides the right people that understand algorithms, decision makers must be convinced that with algorithms they can make better decisions. For analytics to be more than just a one-of initiative, senior management needs to support the development of an analytical culture and facilitate algorithm supported decision making throughout the organisation, fully automated or in support of human decision making. They should show their trust in algorithms by using it in their own decision making, show the benefits and stimulate others to do so as well. The current low adoption of advanced analytics methods shows that currently the majority of organisations either are not mature enough or do not have the need for analytical methods.
Gartner expects that by 2018, more than half of the large organizations globally will compete using advanced analytics and proprietary algorithms, causing the disruption of entire industries. This is quite a bold prediction (even though it is not very precise). For that to happen, in my opinion, these organisations must first grow their analytics maturity. So, instead of investing only in technology companies should invest in getting the right talent and develop an analytical culture. This will benefit them on the short term as doing analytics right will bring immediate value. It will however take more than the projected 2 years for more than half of the large organizations globally to compete with algorithms in such a manner that it will disrupt entire industries.
edge comes from specific algorithms not the Algorithm Economy
From an economic perspective, I don’t think that there will be a huge market for algorithms. I expect the demand in the algorithms market place to be low as these algorithms will be general purpose algorithms like face recognition algorithms or SVM implementations. Nice building blocks, but not the differentiator you are looking for. For your organisation to gain a competitive edge, your algorithms need to be unique and specific to you organisation’s business. You therefore will need to design and build them yourself or hire people who can do that for you. That is what Google, Facebook and Amazon did and is also the case in other industries. Take for example pricing algorithms in the airline industry. All major airlines have their own algorithm to optimise their ticket prices even though there are general purpose pricing algorithms available. Reason they don’t use those is that they don’t expect to gain a competitive edge if they would use the technology that is available for everyone else. So, I expect the algorithm markets as envisioned by Sondergaard to be rather small only containing general purpose algorithms. I do think that algorithms will take the centre stage as they will be a key enabler for companies to become and stay competitive in the future. For that, companies not only need the technology, but should invest in the right talent and proceed with building their analytical competences and culture.