Be sceptical of the experts who are confident and cocksure.
Time’s Healthland Blog has an informative interview with Dan Gardner, the author of a new book, Future Babble: Why Expert Predictions Are Next to Worthless and You Can Do Better.
Now, there’s a confusingly named professor in a study of experts that you write about.
‘Myron Fox’ is whom every person should model themselves after if they want to be media superstars. The name was invented by the researcher who created this stereotype. ‘Dr. Fox’ is an erudite, confident academic who gave a lecture specifically designed for the experiment. The researcher hired an actor to play the role and wrote a lecture for Dr. Fox to give, which was complete gibberish but was brilliantly delivered.
[The original lecture was given to an audience of psychiatrists, psychologists and social workers, and was about "mathematical game theory as applied to physician education." It was full of contradictory statements and non-sequiturs.]
Did the audience recognize that he was talking gibberish? The answer is no. Even educated audiences who witnessed the performance were very impressed. It is one of most depressing pieces of social science research ever conducted.[Link]
The fascinating story of the Myron Fox experiment of 1973 can be read in this extended chapter from The Mad Science Book by Reto U. Schneider.
For full technical details of the Dr. Myron Fox experiment, you can check Naftulin, D. H., Ware J.E., Jr. and Donnelly, F.A. “The Doctor Fox Lecture: A paradigm of educational seduction,” Journal of Medical Education, 1973, No. 48. pp.630-35.
But the more important insights from the Gardner interview, particularly because of the public policy expert views we consume every day, lie here.
And Dr. Fox helps explain why. It’s people who tell simple, clear compelling stories who are perfectly confident that they are right who become media superstars. Reporters turn to them, audiences turn to them, corporations pay huge money to them to give lectures.
…One reason is that the media don’t check accuracy rates of experts, so the consequences for making bad predictions are: Heads, I win; tails, I forget that we had a bet.
There’s also a psychological aversion to uncertainty that drives demand for expert forecasts. When a reporter wants to answer questions for the reader like, What will happen with the economy? An economist would say, ‘Well, I think there are nine key factors, maybe 10. Some point one direction. Others point in a different direction. It may be possible that…’ By that point, the reporter is pulling his hair out; it isn’t a satisfying response.
A hedgehog would have one big idea, a simple story, a final answer, and that satisfies the psychological craving for certainty. Harry Truman once said that he wanted to hear from a one-armed economist [so that the guy wouldn't say], ‘On the other hand…’
…We’re also deeply susceptible to confidence. We find it compelling, and think that they must know the answer. We have to learn to distinguish between the type of expert who is worthy of serious consideration and the blowhard who is trying to bowl us over.[Link]
We have a surfeit of hedgehogs amidst us, experts hailed by the media who are likely to use words like certain and impossible. Unlike the foxes, who are dismissed as being ‘fence-sitters’, the hedgehogs’ confidence is seductive not only for the lay-reader but also for other fellow experts.
Perhaps it is time we followed Gardner’s tip:
Frankly, when I hear somebody making grand pronouncements with perfect certainty, I write them off.[Link]