Some of you will have heard of one of the founding fathers of statistics, Francis Galton (1822 – 1911). He is certainly someone worth reading about, he conducted a famous experiment where people were asked to guess the weight of an ox at a local Stock and Poultry Exhibition. From this experiment we learned a theory often referred to as the Wisdom of the Crowd; 787 people guessed the weight of the ox with an average guess of 1,197 pounds the actual weight was 1,198 pounds. Naturally on the surface this is amazing but it is worth digging a little deeper.
I want to caution that the theory doesn’t work all the time it needs certain conditions to be present and usually more than 1 attempt at prediction so perhaps a better name, Wisdom of the Diverse Crowd Most of the Time.
I don’t want to write a long letter so I am going to get into the heavy stuff quickly and introduce a mathematical identity as to the way this theory works. I am going to borrow from Scott E. Page’s example he used for his students at the University of Michigan.
The Crowd’s Prediction
Mark, David and Bruce make a forecast for the number of new clients they expect next month. Mark predicts 12, David predicts 6, and Bruce predicts 15. The crowd’s prediction is going to be an average of these three predictions. If we sum these up, we get 33; the crowd’s prediction is equal to 11, which is the average of the three people.
Let’s suppose that the actual number of new clients turns out to be 10. They’re a smart crowd but we want some way of measuring the accuracy of these individuals as well as the accuracy of the crowd. How do we do it?
The Accuracy of Individuals and Crowds
Typically it is done by taking the difference between the actual prediction and the true value and squaring that amount. They call the result the squared error. In the case of Mark, we take 12 − 10, which is the true amount, and square that to get a total of 4.
Let’s compute the errors for the other people as well. Let’s look at David. David predicts 6, the actual value is 10 so (6 − 10)2, is going to be 16. Bruce predicted 15, (15 − 10) is five, and if we square that we get 25. Mark is off by 4, David is off by 6, and Bruce is off by 15. If we sum these squared error up 45, we are going to get an average of 15. The individual squared error, in this case, is 15. Remember the crowd guessed 11, so 11 − 10 is 1. The crowd’s squared error is 1. What we get is the crowd is more accurate than the individuals are on average. Notice, the crowd is also more accurate than anybody in the crowd.
You should at this point be wondering what makes the crowd smarter. Instead of getting too deep into the statistics there is an important factor to distinguish within the crowd. We need to be able to measure the accuracy of the people. The standard approach Dr Page follows is to look at the difference between the individual predictions and the crowd predictions, typically known as variance. Variance in the context of our study is to mean Diversity. Stay with me:
Mark 12 – average prediction 11 = 1 squared = 1
David 6 – 11 = 5 squared = 25
Bruce 15 – 11 = 4 squared = 16
1+25+16 = 42 / 3 = 14 The Diversity of Predictions
Let us summarise the 3 numbers we have calculated:
Individual Squared Error = 15
Crowd Squared Error = 1
Diversity Error = 14
Ok this is what I have been building towards, phew!
The crowds error = individual error – diversity error. This a mathematical identity and is always true.
The point I am trying to make and will try and drive home is that what is equally important to having a smart crowd is to have a diverse crowd. That means if there is any diversity in the room in any way, then the crowd is going to be better than the average person in it because it is just a mathematical fact. The crowd’s error is the average individual error minus the diversity. If the crowd’s got positive diversity at all then the crowd’s error has to be smaller than the average individual error. Crowds are better than the people in them, at least on average.
Two Ways to Arrive at a Wise Crowd
If you want a wise crowd, this suggests we have two options. We could find brilliant people who all know the answer so then the individual error would be zero, crowd error would be zero, and diversity would be zero; or we can find a bunch of fairly smart people who have moderate errors, who happen to be diverse, so you also get moderate diversity. If you want to make more accurate predictions you need to bring diversity to your decision making process. If you shut out dissenting voices who have a different opinions to yours, you are robbing your forecasts the wisdom of the crowd.
I have been writing a lot about the poor decision making made by central bankers over the decades. I am reading the most amazing book The Lords of Easy Money about the inside workings at the Fed. If there was ever an institution built around shutting out diversity this is it. They continuously lobby their voters to put a united front to the markets. They would do well to reread some basic statistic books.