Prediction Markets

Our understanding of the market mechanism has progressed significantly over the past two decades, through practical use and more experimentation. It is still incomplete but several of the mysterious cogwheels responsible for the impressive performance of prediction markets have meanwhile been unraveled.

Behavioral scientists found that humans economize most decisions by using as little brain power as possible. This so-called System 1 of thinking follows emotions, intuitions and gut instincts. Only when we recognize that a decision may be more complex than meets the eye do our brains switch to System 2, a more careful and logical but also more strenuous mode of thinking (Kahneman, 2011).

There is increasing agreement - in particular by behavioral science-oriented market researchers - that traditional questions about future purchase intent are not a good method as they elicit only claimed behavior. However, there is disagreement on why this may be the case, and consequently how to get more reliable results.

Some researchers think that purchase intent questions trigger a System 2 response which they do not trust because the shopper in the actual purchase situation will decide by System 1, which is intuitive and emotional. For them, the path to better research is asking questions which can actually be answered by System 1, e.g., by asking about immediate emotions (Kearon, 2007).

With this explanation, not much needs to change. Respondents can continue to click questionnaire checkboxes, with or without much thought. Researchers can continue to ask direct questions, as long as they avoid questions about intent. Their role remains to translate respondents’ answers with some proprietary art into a prediction for the actual question at hand, so that clients must rely on a secret black box. Is this first explanation actually sufficient? Does the solution improve results’ reliability? The evidence presented so far points to an improvement in discrimination between product alternatives, while otherwise only maintaining the unsatisfactory reliability level of traditional research.

Next generation prediction markets start from the hypothesis of exactly the opposite: System 2 is very much needed, the problem is that purchase intent questions in a traditional questionnaire trigger the wrong type of a System 2 response. This theory recognizes that survey respondents are primarily motivated by a desire to earn the incentive. They seek to complete the questionnaire as efficiently as possible, using a minimum of brain power.

Respondents’ System 2 thinking is applied very creatively to this selfish purpose, even going so far as swindling. Panel providers fight a whole range of creatively-named response fraud types: speeders, flatliners, dupes, mental absentees, etc. The total financial damage from response fraud has not yet been estimated reliably but could be substantial (Hofkirchner, 2014). For an indication, the Association of National Advertisers estimates that U.S. businesses are losing US$6.3 billion a year to click fraud.

What else leads us to suspect a System 2 response? Think of a shopper in the actual situation who will in fact put most articles into the shopping cart intuitively, without much thinking. However, if something changes noticeably in the supermarket, be it a new product, new packaging, a changed price, or even just a different shelf position, our shopper will pause and take a mental step backwards. The noticed change from the normal triggers a System 2 response, a much more conscious, deliberative process: the shopper will start comparing prices, quantities, and qualities of the changed offering. Once the mind is thus made up, the shopper will act from this new basis using System 1, but only next time.

If we work with this second explanation we need a very different solution. Instead of simply asking questions, we must challenge respondents to bet on the future. We know from neuroscience that betting - and trading on the financial markets - triggers a vastly different brain reaction than just answering a question. Simply put: when asked to commit with a bet, respondents will think much harder about their true opinions and expectations; they will try to get it right to maximize their expected profit.

Next: The third party perspective