New evidence on the relative accuracy of prediction markets

The paper Comparing face-to-face meetings, nominal groups, Delphi and prediction markets on an estimation task by Andreas Graefe and Scott Armstrong was accepted for publication with the International Journal of Forecasting.

The study presents results from a laboratory experiment, in which traditional face-to-face meetings were compared to three structured group methods (i.e., prediction markets, nominal groups, and the Delphi method) on a quantitative judgment task. Overall, Delphi performed best, followed by nominal groups, and prediction markets. As expected, unstructured interaction in meetings yielded the least accurate results. However, differences were small.

Furthermore, as one would expect, the three structured approaches yielded more accurate results than participants' prior individual estimates. However, only in the case of Delphi, the group results were more accurate than a simple average of the prior individual estimates. Prediction markets and nominal groups provided little additional value compared to such staticized groups in situations where people drew upon similar information

The authors also examined participants’ perceptions of the group and the group process. Participants rated personal communication more favorable than computer-mediated interaction. Group interaction in meetings and nominal groups was perceived as highly cooperative and effective. Prediction markets were rated least favorable. Prediction market participants were least satisfied with the group process and perceived their method as most difficult.