From Predictive Models to Prediction Markets

Ray Ryan is the president of Patten and Patten, an investment management firm, and a registered investment adviser in Chattanooga. Ray is a CFA charter holder, a member of the advisory board for UTC’s College of Business, and an adjunct professor of finance at UTC. He is a graduate of Princeton University, where he had the privilege of taking a course taught by former Federal Reserve Chairman Ben Bernanke.

 

 

Consider how frequently you check the weather app on your phone.

 

Weather apps present forecasts from meteorological models. Most forecasts are developed from models. A model is a simulation of reality using historical data.

Models reflect expert analysis of hard data such as economic indicators and soft data such as surveys and polls. Analysis usually involves technical, statistical, and quantitative extrapolation.

All models have limitations and flaws. Cognitive biases reduce model accuracy, and models are inherently limited by their assumptions. There are countless models of the stock market, and their shortcomings have been exposed by outlier events, such as stock market crashes.

Despite rigorous model construction, predictive value remains quite limited. Nevertheless, the public relies on many models for planning, policy, and general decision-making. For example, most people rely on weather forecasts from smartphone apps, despite varying perceptions of accuracy. Weather models update every six hours. According to the NOAA, weather models are approximately 80% accurate up to seven days and 50% accurate at 10 days.

The Survey of Professional Forecasters (SPF) has collected predictions of key economic indicators such as unemployment, inflation, and economic growth since 1968. SPF data informs key policy makers such as the Federal Reserve. A recent study using SPF data tested the accuracy of 16,559 forecasts from 396 different forecasters. Results of the study: “forecasters report 53% confidence in the accuracy of their forecasts, but are correct only 23% of the time.” In other words, economic and financial market forecasts “claim greater certainty than their accuracy justifies.”

The authors of one study stated: “traditional forecasting formats exacerbate the human tendency toward over-confidence.” Over-confidence is a cognitive bias that arguably has the largest impact on forecast error. There are three types: a) optimistic over-estimation of future performance; b) exaggerated belief in one’s forecast superiority to others; and c) excessive certainty in the precision or accuracy of one’s beliefs or forecasts.

Fortunately, technology has facilitated the adoption of market-based forecasts that could reduce dependence on models in the future. Market-based approaches offer fewer biases and structural limitations. Examples of market-based indicators include those that forecast inflation and interest rates.

 

 

A market is essentially an aggregation of information, and the crowd (i.e., the “market”) is usually wiser than the individual. Studies confirm the median estimate of a diverse group of people is more accurate than individual estimates.

Prediction markets are a form of crowdsourced market-based forecast. Theory asserts, “the prediction market price equals the mean belief among traders.” Prediction markets involve purchasing contracts (i.e., “betting”) on the likely winner of a contest or the outcome of an event. At the end of a contest, the winning contract is worth $1, and the losing contract is worth zero. During the contest, a contract price of $0.60 corresponds with 60% probability of success.

Election studies demonstrate that prediction markets in 1988-2004 were more accurate than polls 74% of the time. They also had a lower error rate – 1.6% – as compared with the Gallup poll error rate of 1.9%. Researchers claim, “in a truly efficient prediction market, the market price will be the best predictor of the event, and no combination of available polls or other information can be used to improve on the market-generated forecasts.”

Researchers commented further, “the power of prediction markets derives from the fact that they provide incentives for truthful revelation, they provide incentives for research and information discovery, and the market provides an algorithm for aggregating opinions … If traders are typically well-informed, prediction market prices will aggregate information into useful forecasts.”

Prediction markets are not immune to collective bias. For example, prediction markets are susceptible to anchoring – i.e., perceptions remain “anchored” to recent contract prices. This bias could create a self-reinforcing loop. Studies have also observed an “asymmetry bias” in that optimists tend to bet more aggressively than pessimists. In addition, research indicates the public consistently exhibits asymmetric interpretations of probability.

In prediction markets, a small number of highly active traders dominate total volume and number of transactions. Thus, traders with extreme beliefs often have a disproportionate impact on price. There have also been documented incidents of contract price manipulation. 

Source for quotes regarding prediction markets: Justin Wolfers and Eric Zitzewitz, “Prediction Markets in Theory and Practice,” National Bureau of Economic Research: Working Paper 12083, 2006.