Prediction Markets

When Francis Galton observed farmers guessing the ox’s weight, there was an important element present which could be easily overlooked. The farmers did not just answer a question, they made a bet. Despite the seeming similarities, there are crucial differences between filling out a questionnaire and making a bet.

The obvious one is that there is no consequence for respondents whether they fill out a questionnaire correctly - to their best knowledge - or not.

Second, tasks with uncertain rewards activate people’s brains far more than do tasks with certain rewards (Zald, 2004). The farmers had to get it right to win the prize, so there was clearly uncertainty in their task. The potential of an uncertain reward being a strong motivator, the farmers would think significantly harder than if somebody just asked them for an opinion.

Third, the discrepancy between a person’s actual and predicted reward determines whether learning occurs. When answering prediction market questions, participants subsequently learn from their mistakes and improve in their forecasting ability.

This learning effect can be observed in a public prediction market that we ran for the Swiss national television network, SRF, and Swiss print media group, TAmedia. Over twelve years, or three election periods, a large number of participants stayed on and formed a community. Figure 4 shows a continual improvement of their market forecast, consistent with a learning effect.

In the first election project in 2007, the prediction market outperformed the traditional polls by 36%, its mean absolute error (MAE) was only 0.97% vs. 1.51% of the polls. (An MAE of 1% is generally accepted to be a “hit” in polling.)

In the second election in 2011, the prediction market’s accuracy improved to an MAE of 0.60% while the traditional poll’s average was still 1.42%, corresponding to an outperformance of 53%. In the most recent Swiss election in 2015, the prediction market improved even further to an MAE of 0.48% versus 0.59% for the average poll, an improvement of 19%. However, we note that in 2015 all forecasts came in very close and better than the 1% gold standard, which looks odd at first. Looking more closely we find that the traditional Swiss polling institutes started to use a new practice in 2015 which they call “combining”, whereby their polls are “integrated” with the results of other providers including the public prediction market (Longchamp, 2015).

Prediction markets are a very new method and they work according to very different new rules. Some cardinal principles of frequentist research do not apply: questions are indirect, sampling is opportunistic, incentives are not certain and, oddly, respondents can see the current result before they trade. Most importantly, it is not a researcher but the respondents themselves who translate their own minds into bets. And it works well. Once automated prediction market platforms gain broader acceptance, marketers and strategists may switch more and more to do-it-yourself research, saving time and money. The next section will cover practical applications for the prediction market method to inform important marketing decisions.

– – – End – – –