Daniel Kahneman, Nobel Prize winner, cognitive scientist and behavioural economist, is one of my favourite researchers. He has dedicated his career to understanding the mechanisms behind our choices and the errors of judgement of which we are victims. In particular, he popularised the notion of cognitive bias. His best-known work, Thinking Fast and Slow (see my summary), highlighted the weakness of our intuitions in the statistical field.
In Noise, Kahneman, together with O. Sibony and C. R. Sunstein, focuses on another vector of misjudgment: noise. R. Sunstein to another vector of misjudgment: noise. His findings apply directly to the HR world, where judgements are plentiful, from recruitment to performance reviews.
In this article discover :
- What is noise?
- How does it impact on HR decisions?
The notion of cognitive bias has been widely documented.
According to Jean-François Le Ny, a psychologist specialising in cognition: " A bias is a distortion (systematic deviation from a norm) that information undergoes when entering or leaving the cognitive system. In the first case, the subject selects the information, in the second, he selects the answers ".
Let's take an example: the halo effect. A positive first impression, such as a pleasing appearance, leads to a more positive judgement of other characteristics of an individual, such as competence. M. Clifford and E. Walster have shown that children were judged more intelligent than others by their teachers on the basis of their appearance*.
As you can see, with a bias, the judgment error is homogeneous and causally explained (beauty ➞ intelligence).
Now that we have clarified the notion of bias, what is noise? Noise is the unwanted and unexplained variance observable in human judgements.
For example, when five judges give completely different sentences for the same offence, we are faced with a noise phenomenon. Or when two doctors give different diagnoses to the same patient. These differences in judgment are undesirable and cannot be explained by a bias that would make them predictable.
But what is the basis for these differences in judgement between individuals ? It may be differences in perception, but also external factors that are irrelevant to the decision, such as our mood, the weather or the time since our last meal. A study has shown that in an Israeli court, the probability of a favourable decision decreases from 65% at the opening to almost 0% in the last judgement before the lunch break****.
Noise is also about unwarranted differences in assessment for the same individual. Researchers Grimstad and Jørgensen showed that when a developer was asked to estimate the time needed to complete a task several days apart, the projection varied by 71% on average*****.
Noise is therefore quite different from the concept of bias, although both refer to distortions in judgement. Let's take a simple illustration to understand the difference between the two concepts. At Acme Corp, during a recruitment process for a project manager position, three HR professionals systematically over-assess well-dressed or good-looking candidates. This is a bias.
Now let's look at another company. At Beam Corp, for the same candidate, evaluations vary widely, but there is no bias. This is called noise. Three qualified HR professionals from the same company should not have so much variation in their judgement. Moderate variations are normal and acceptable, but not extreme differences.
Another interesting illustration of the difference between bias and noise is provided in their book by the authors and transcribed in an article in the Harvard Business Review**. On this target, in the case of bias (Figure C), all shots are missed AND clustered. One could imagine that this is due to a malfunction on the rifle. Conversely, in the case of noise (figure B), the shots are missed but randomly scattered. We cannot risk to put forward any justification.
Most organisations (including yours?) think they are immune to this phenomenon. Yet, according to Kahneman, Sibony and Sunstein, "wherever there are judgements, there is noise, and much more than we imagine".
The authors give us some examples of sectors subject to high noise levels:
- Asylum adjudication. In the US, the probability of a refugee being granted asylum is a lottery. A study called "Refugee Roulette"*** showed that for the same panel of cases, some judges accepted 88% of applications, while others accepted only 5%. The probability of being admitted therefore depends mainly on the judge who examines your case.
- Medicine. When several doctors are asked to examine a single patient, they often give different diagnoses. This phenomenon is particularly strong in psychiatry, which leaves a lot of room for subjectivity. In an American study, several well-known psychiatrists were asked to diagnose the same patients without talking to each other. They arrived at the same diagnosis in between 4 and 15% of cases... ******. Even in a more objective field such as radiology, a study has shown that among different radiologists the observed false negative rate is between 0 and more than 50%.*******
As you can see, noise is omnipresent in organisations, as soon as human judgements are present. This phenomenon is characterised by its unwanted and inexplicable nature, unlike bias. The implications for the HR world, where evaluations are constant (recruitment, performance...), are enormous. In a second article to come, I will look at the HR actions most sensitive to noise and the methodologies proposed by Kahneman, Sibony and Sunstein to reduce it.
Did you like this article? Read my summary of Kahneman's previous book, Thinking Fast and Slow. If you want to receive our latest publications, follow us on Linkedin and subscribe to our newsletter.
*Clifford, M.M and Walster, E.H, 'The effect of physical attractiveness on teacher expectation', Sociology of education, vol. 46, 1973, pp. 248-258(read online[archive])
* * https://hbr.org/2016/10/noise
***https://www.stanfordlawreview.org/print/article/feature-refugee-roulette-disparities-in-asylum-adjudication/
**** Shai Danziger, Jonathan Levav, and Liora Avnaim-Pesso, "Extraneous factors in judicial decisions", PNAS April 26, 2011 108 (17) 6889-6892(link). The results of this study are now subject to debate - The irrational hungry judge effect revisited: Simulations reveal that the magnitude of the effect is overestimated - Andreas Glöckner - Judgment and Decision Making, Vol. 11, No. 6, November 2016, pp. 601-610(link)
***** Stein Grimstad, Magne Jørgensen, "Inconsistency of expert judgment-based estimates of software development effort" Journal of Systems and Software 80(link)
****** Liblich et al. e-5
*******Craig A. Variability in the interpretation of screening mammography by radiologists" 1996
