“Think Like You Don’t Know:”
A Form of Belief Bias in
Judgments of Bayesian
Rationality
Richard Anderson
Department of Psychology
Bowling Green State University
Bowling Green, Ohio
1
INTRODUCTION
2
INTEGRATING DISPARATE PROBLEM DOMAINS
Belief Bias in Syllogistic
Reasoning:
Our belief that a particular
conclusion is true or false,
affects how valid the argument
seems.
"If it were true that all fish are
animals, and if it were true that
all animals have wings, then it
would also be true that all fish
have wings."
Bayesian Inference:
"If the presence or absence of
Disease A produces Symptom
S with the probability, p, or q,
respectively,
and if Disease A has a prior
probability, or Base Rate, R:
If the symptom is present,
what is the probability that
Disease A is also present?"
Signal Detection Modeling:
If Categories I and II partially overlap on Dimension E,
how does the pattern of correct and erroneous
classifications reveal the magnitude of category overlap
and the location criterion, on E, for classification?
Metacognition:
Broadly: Thinking about
thinking. More specifically:
thinking about Bayesian
rationality.
3
A Few Links in the Research Chain
1. Belief Bias in Syllogistic Reasoning: If a syllogism's conclusion is
believable (i.e., is apparently true), that makes it seem like the
conclusion is valid (i.e., makes it appear to follow logically from the
premises (Evans, Handley, & Harper, 2001).
...
4
A Few Links in the Research Chain
1. Belief Bias in Syllogistic Reasoning: If a syllogism's conclusion is
believable (i.e., is apparently true), that makes it seem like the
conclusion is valid (i.e., makes it appear to follow logically from the
premises (Evans, Handley, & Harper, 2001).
2. Belief Bias, not in Syllogistic Reasoning, and not in Bayesian
reasoning, but in Judgments of Sample Size Adequacy:
When a sample is used to make a statistical inference:
The sample size seems less adequate if the value of the sample
statistic happens to be unrepresentative of the known population
parameter (Anderson & Hartzler, 2013).
...
5
A Few Links in the Research Chain
1. Belief Bias in Syllogistic Reasoning: If a syllogism's conclusion is
believable (i.e., is apparently true), that makes it seem like the
conclusion is valid (i.e., makes it appear to follow logically from the
premises (Evans, Handley, & Harper, 2001).
2. Belief Bias, not in Syllogistic Reasoning, and not in Bayesian
reasoning, but in Judgments of Sample Size Adequacy: When a
sample is used to make a statistical inference, the sample size seems
less adequate when the decision-maker believes the inference to be
false (Anderson & Hartzler, 2013).
3. The Present Study:
Belief Bias in Judgments about Bayesian Rationality
Anderson, Leventhal, Fasko, Basehore, Billman, Zhang, Gamsby,
Branch, & Patrick (manuscript in preparation). A Form of Belief Bias in
Judgments of Bayesian Rationality.
6
METHOD
Trial 1 of 32. Note that the UNDERLINED text may CHANGE on each trial.
A DOCTOR KNOWS THAT . . .



95% of pregnant women are pregnant with ONLY ONE fetus, and
5% are pregnant with TWINS.
There is a TEST that indicates whether one or two fetuses are present.
 The test is accurate 70% of the time for women who are pregnant with only one fetus.
 The test is accurate 70% of the time for women who are pregnant with twins.
ACTUAL STATUS . . . A particular woman is actually pregnant with TWINS.
TEST RESULT . . . . . . Her test result indicates she is pregnant with TWINS.
CONCLUSION . . . . . . The doctor cannot have direct knowledge of the pregnancy's actual
status. But using only the percentages and test results described above, the doctor concludes
that the woman is probably pregnant with ONLY ONE FETUS.
Regardless of whether the doctor's
conclusion turned out to be correct or
incorrect, did the doctor draw the most
reasonable conclusion given the test result?
Yes.
It was the most
reasonable
conclusion.
No.
It wasn't the most
reasonable
conclusion.
How certain are you that the Yes or No answer
you just gave is the correct answer?
Not at
all
certain
Slightly
certain
Moderately
certain
Very
certain
(Reality consistency pertained to whether the diagnosis matched the actual status.)
8
RESULTS & DISCUSSION
Error bars indicate
95% confidence
intervals. N = 98.
9
RESULTS & DISCUSSION
Error bars indicate
95% confidence
intervals. N = 98.
10
11
RESULTS & DISCUSSION
12
A Conceptual Cognitive Model
13
INTEGRATING DISPARATE PROBLEM DOMAINS
Belief Bias in Syllogistic
Reasoning:
Our belief that a particular
conclusion is true or false,
affects how valid the argument
seems.
"If it were true that all fish are
animals, and if it were true that
all animals have wings, then it
would also be true that all fish
have wings."
Bayesian Inference:
"If the presence or absence of
Disease A produces Symptom
S with the probability, p, or q,
respectively,
and if Disease A has a prior
probability, or Base Rate, R:
If the symptom is present,
what is the probability that
Disease A is also present?"
Signal Detection Modeling:
If Categories I and II partially overlap on Dimension E,
how does the pattern of correct and erroneous
classifications reveal the magnitude of category overlap
and the location criterion, on E, for classification?
Metacognition:
Broadly: Thinking about
thinking. More specifically:
thinking about Bayesian
rationality.
14
A Form of Belief Bias in
Judgments of Bayesian
Rationality
Richard Anderson
Department of Psychology
Bowling Green State University
Bowling Green, Ohio
15