Prediction Markets

Up to the early 2000s, marketers were taught to act directly on what consumers say. Even today, most believe that for successful product development or communication measures they must recruit random consumers for questionnaires, ask them about their likings or purchase intent, and decide according to their claimed behavior.

These traditional research methods which rely on self-reported statements have persisted despite the contempt expressed by some of the most successful innovators and businessmen for quite some time. Henry Ford famously said: “If I had asked people what they wanted, they would have said faster horses”. Steve Jobs agreed: “People don't know what they want until you show it to them," and David Ogilvy coined the classic: "People don't think what they feel, don't say what they think and don't do what they say”.

A key difference between prediction markets and traditional research is the form of the question, which always avoids such direct self-reports. For example, an election prediction market asks indirectly: “How will the people vote on election day?” whereas traditional pollsters ask directly: “How would you vote if there were elections on Sunday?”

The improvement in accuracy when asking questions in the third person about others is striking. When a group of students in an experiment was asked whether they intended to tidy up the room after a meeting, fifty percent answered positively: they would. Then they were asked to predict how many students would clean up. The median answer to the question was quite different: 15%. This result is but a third of the claimed behavior.

When the researchers studied the number of students who did in fact clean up, the proportion was 13%. The forecast error of 233% with the traditional question has almost disappeared, just by changing to a third person question. When there is no cost to them, people will make claims that please the questioner or which are in line with generally accepted behavior (Puleston, Hofkirchner, and Wheatley, 2014). This is one of the key reasons why respondents always overstate their claimed purchase intent.

Next: Small biases, big errors