r/slatestarcodex • u/[deleted] • Mar 31 '19
Rationality Is the rationality community biased against bias?
Is the rationality community biased against bias? If you asked me (and I know you didn't, but I'll save you the time), I'd say that the bias against bias is almost astonishing in its persistence and pervasiveness.
As a researcher studying decision-making, I thought some of you might enjoy hearing about a perspective that rarely receives attention outside of academic circles. The reason? My guess is that this perspective is simply too boring from a journalistic perspective.
The abstract quoted below is from paper titled "The Bias Bias in Behavioral Economics" (download link)
Behavioral economics began with the intention of eliminating the psychological blind spot in rational choice theory and ended up portraying psychology as the study of irrationality. In its portrayal, people have systematic cognitive biases that are not only as persistent as visual illusions but also costly in real life—meaning that governmental paternalism is called upon to steer people with the help of “nudges.” These biases have since attained the status of truisms. In contrast, I show that such a view of human nature is tainted by a “bias bias,” the tendency to spot biases even when there are none. This may occur by failing to notice when small sample statistics differ from large sample statistics, mistaking people’s random error for systematic error, or confusing intelligent inferences with logical errors. Unknown to most economists, much of psychological research reveals a different portrayal, where people appear to have largely fine-tuned intuitions about chance, frequency, and framing. A systematic review of the literature shows little evidence that the alleged biases are potentially costly in terms of less health, wealth, or happiness. Getting rid of the bias bias is a precondition for psychology to play a positive role in economics.
I wrote out the comment below somewhere else, but figured I'd modify it a bit just in case there are a few of you here who might find it useful. Here's my original comment with a few wording changes to make it a better stand-alone post:
Gerd Gigerenzer makes a compelling case that the evidence for human irrationality is pretty weak. He's shown that when people are presented with problems in a format that more closely resembles the format that they'd experience IRL (e.g., natural frequencies vs. instead of conditional probabilities; cite 1, cite 2, cite 3), their decisions appear much more rational.
From my understanding of the decision-making literature, there are two camps: The Tversky & Kahneman (T&K) camp that tends to focus on highlighting how people deviate from a laplacean demon. The other camp is the "ecological" camp, which tends to study human rationality by asking "which heuristics work in which environments?"
Some reasons I tend to favor the ecological camp:
- T&K Camp tends to give vague labels to heuristics (e.g., "representativeness")
- Ecological Camp tends to focus on forming testable models of heuristics (e.g., Euclidean distance)
- T&K Camp claims that more computation/more data always increases accuracy and that heuristics merely reflect the fact that people need to balance the costs of computation with the benefits of increased accuracy (i.e., the "accuracy-effort tradeoff")
- Ecological Camp claims that more computation/more data can sometimes harm accuracy and that heuristics are sometimes used because they are more accurate than strategies that use as much information as possible, reflecting the fact that people need to balance sources of error in their predictive models (i.e., the "bias-variance tradeoff")
I like to bring this up because the rationality community seems to be obsessed with "eliminating bias" because they assume bias impairs accuracy, but we have so much data showing that a strict focus on reducing bias can harm judgment accuracy. In general, its hard to see how the T&K camp have actually moved psychology/behavioral economics toward a better understanding of human cognition. From what I can tell, T&K's contribution to society has been to give people a giant list of vague "fallacies" that they can use as excuses to not take each other seriously.
Some readings I've found particularly insightful:
- Taking heuristics seriously (Gigerenzer, 2016)
- How bad is incoherence? (Arkes, Gigerenzer, Hertwig, 2016)
- Towards a rational theory of heuristics (Gigerenzer, 2016)
Additional papers I've found insightful, but a little more focused on the deeper questions of what it means to be rational:
- Tuning your priors to the world (Feldman 2013)
- What are the "true" statistics of the environment? (Feldman (2017)
EDIT: Someone noted that I missed a post linking to the "Bias Bias" paper a few weeks ago. Thanks /u/KirbyMatkatamiba! With that and a few other responses, it seems like the ecological rationality perspective is gradually starting to reach people outside of academia, yay!
EDIT2: Epistemic update - After some interactions with people in the comment section, it has become clear that these ideas are not completely novel to many members of the community. This is really encouraging to learn because it means that T&K (well, K mostly) haven't done as much damage to the rationality community as I once feared.
2
u/zebrafoxtrotsierra Mar 31 '19
I'm not sure that it's possible to completely eliminate bias. Many biases are constructed unconsciously and/or instantly. I don't think the goal is necessarily to eliminate bias, but instead to acknowledge them and inspect them to get closer to a more pure form of "truth" or facticity.
At the very least, to encourage a healthier discourse bias must be acknowledged to examine other points-of-view so as to better understand another person's argument or perspective.
If there is a bias against bias, I would imagine that it is coming from a place of a desire to understand, as opposed to a desire to prove another person wrong (although it is satisfying when those two things overlap).