Wherever there is judgement, there is noise

I am reading a fascinating book by Nobel Prize winner Daniel Kahneman, Cass Sunstein & Olivier Sibony called, NOISE: A Flaw in Human Judgement.

There are lots of different teachings I will draw out from this book in coming letters. For today I want to focus on 2 key ideas, unwanted variability and the illusion of agreement.

The book gives many examples of how experts such as judges, doctors, insurance assessors….. give their expert judgement with a degree in variability far far greater than what people think. As you can imagine the consequences are deadly in some medical cases, unjust in many legal sentencing cases and extremely costly in misjudging insurance risks. The problem being that the variability, otherwise known as noise, does not cancel each other out. Noisy systems do not make multiple judgements of the same case. They make noisy judgements of different cases. For example when a patient with certain symptoms presents at a hospital the doctor chosen to diagnose the case is somewhat randomly chosen. Unless the patient goes for a second opinion that random choice will be very influential in their future wellbeing. When a rapist stands before the judge sentencing him, the random choice of the judge will have a major impact on the sentencing of his case.

The authors of this book were brought in as consultants to a prestigious asset management company. To illustrate the degree of variability present in the human condition across all areas of judgement they asked 42 analysts of the asset management company to value a company based on a 1 page description together with a simplified profit & loss and cash flow statement and balance sheet for just 3yrs of business. The median noise measured was 41% in a company with analysts trained to use the same valuation model. The owners of the company were naturally shocked.

This brings us to the illusion of agreement. Most people don’t enjoy conflict and most people don’t want to ruin or risk their careers by making a prediction that is too far away from their peers. What therefore tends to happen is people behave like school kids copying or being influenced by their buddies. The problem with this approach is if you copy or get influenced by your buddy and he or she doesn’t know their work then you are all going to get similarly wrong answers. Like the C and/or D target in the illustration above.

My advice to truth seekers, people who want to receive unbiased advice / market forecasts you need to try and get access to people who are not part of the scene. You want to follow mavericks who say it the way they see it and are fearless in saying it as it is no matter how extreme. If you want the average and are happy to play in the safe middle then by all means take note of the Bloomberg polls, etc….. but don’t expect to truly add value.

If you want to shatter the illusion of agreement you need to grow up. The more you do something the less supervision you need. When your kids are learning to walk you need to be there to hold them, watch them and then you move on and let them do their thing. When professionals have served their apprenticeship it is time to go out and be confident and make your mark in the world and not constantly look over your shoulder looking and listening at your colleagues. [please don’t read this as never discussing and assessing past and current performance]

You might also like

Leave a Reply

Your email address will not be published. Required fields are marked *