Learning from Leaders: A Conversation with Danny Kahneman

It’s not every day that one is able to spend their lunch hour hearing from a Nobel prize-winning psychologist. Joining Yale School of Management Professor Shane Frederick, professor emeritus of psychology and public affairs at Princeton University, Daniel Kahneman joined the September Learning from Leaders session to shed light on the intricacies of his newest publication, Noise, A Flaw in Human Judgement.

Joined by guests from over 21 countries, Professors Frederick and Kahneman delve into the differences between noise and bias, how this new research builds upon Kahneman’s existing body of work, and how good decision-hygiene can be implemented systematically and individually.

Frederick: What is the distinction between Noise and Bias?

Kahneman: The analytical discussion of judgement really builds on the theory of measurement. You can think of judgement as measurement, where the instrument is the human mind.

If you think of a physical measurement, for example, measuring the length of a line with a very fine ruler. If you take repeated measurements, you’ll find that the measurements are not all identical, that variability is noise. The average error, for this you’ll need a true score, the average error is the bias.  Noise is the standard deviation; bias is the average. You can assess noise without knowing the true answer but to assess bias, you need the true answer. Those are two different sources of error, because accuracy suffers when you have variability, even when there is no bias.

Bias and noise, although they’re very different, have equivalent roles in overall error. This also implies that if you reduce one of them by fifty percent, it’s just the same as if you reduce the other by the same percentage. When you think of errors you typically think of bias, so the book was attempting to address that imbalance between these two sources of error.

Frederick: You probably feel this too, that bias feels sexier than noise does. Why do you think that’s the case?

Kahneman: Well, I think, in Thinking Fast and Slow, I drew a distinction between two modes of thinking: statistical and causal. And causal thinking is about individual cases and it's very compelling. There is a sort of force associated with it. Statistical thinking is about ensembles and its sort of disembodied and doesn’t feel like anything. In the context of errors, you can look at an error and think, “This is the consequence of bias,” but you will never be able to say that a single error is caused by noise. It takes multiple errors to even define noise, so it's this natural fit between bias and causal stories or interpretations that I think makes for the sexiness of bias, relative to noise.

Frederick: Can you explain why you chose the term ‘decision-hygiene’ as the metaphor?

Kahneman: When you are focused on biases the way that most people are and you're trying to do something about biases, then basically you're trying to do something specific. In medical terms it's a medication or it's a vaccine which is specific to a particular disease. Washing your hands is qualitatively different.  You wash your hands to kill germs, but you have no idea what germs you're killing and if you're lucky, you will never know. And yet, washing your hands is a very good idea, so there is a whole class of things, of procedures, that we think of in that way.

We suggest that these procedures are noise- reducing and quality-improving in general, without being focused on biases and on errors. We chose the term decision-hygiene in part because it's an apt metaphor, and in part because it is slightly shocking and off-putting, and I like that.

Frederick: And what are some of the components of good decision-hygiene?

Kahneman: One central component is independence. Your observations should be independent of each other, your judgements should be independent. When you're looking at different aspects of the problem, evaluations of the different aspects should be independent. So that's one big principle. Breaking up problems into sub-problems that you can evaluate independently is definitely a good idea.

Another general principle of hygiene is delaying intuition. You can’t get rid of intuition and you don’t want to get rid of it, you want a final judgment that you hold with intuitive confidence But intuition gets in the way of information collection because it induces confirmation bias and that's what you want to avoid. So, independence is really a way to control confirmation bias. 

Those are two big principles. Another principle is trying to make judgments comparative rather than absolute because comparative judgments tend to be more accurate. We know this from a lot of psychological research. So, try to develop comparative scales where people compare the case that they're looking at to a scale of previous cases. There are several ways of making judgments comparative and that will, in general, decrease noise and bias.

Frederick: Recently we’ve talked about de-biasing techniques, at least at an individual level, being aware of something in your own behavior or decision-making to try not to act on that bias. Are there ways to reduce noise in the way we reduce bias?

Kahneman: We discussed this a lot in the book: what does it mean for a non-repeated decision to be noisy? Basically, the answer, which my colleague Olivier Sibony put forth, was that a singular event is a repeated event that happens only once.  It follows that if decision-hygiene is applicable to a repeated decision, there is no reason to assume that it is not applicable to single decisions. You should reserve hygiene for decisions that deserve the effort, but for certain important decisions it might not be a bad idea.

Frederick: Does noise originate more from system one or system two processes?

Kahneman: I think there is a lot of noise in system two processes. There is a lot of noise in important decisions, and in decisions that people don't make impulsively. Of course, intuition contributes to noise in system one thinking everywhere but I don't think that merely thinking seriously about something means there won’t be noise. There is a lot of noise in tasks that people take quite seriously, like professional tasks, where you cannot assume that physicians who disagree are doing so because of system one. I think the whole distinction between system one and system two is barely mentioned in Noise.

Catch the full interview with Professor Daniel Kahneman here.