Design and Analysis of Sample Surveys
Andrew Gelman
Department of Statistics and Department of Political Science
Columbia University
Class 14a: Network sampling
Andrew Gelman
Design and Analysis of Sample Surveys
Ronald Reagan was a statistician
Andrew Gelman
Design and Analysis of Sample Surveys
“But the simplest way for each of you to judge recovery
lies in the plain facts of your own individual situation. Are
you better off than you were last year?”
Andrew Gelman
Design and Analysis of Sample Surveys
Who do you know?
I
How many people do you know named Nicole?
I
How many people do you know named Anthony?
I
How many lawyers do you know?
I
How many people do you know who were robbed in the past
year?
Andrew Gelman
Design and Analysis of Sample Surveys
Scale-up method
I
On average, you knew 0.6 Nicoles
I
0.13% of Americans are named Nicole
I
Assume 0.13% of your acquaintances are Nicoles
I
Estimate: on average, you know 0.6/0.0013 = 450 people
I
On average, you know 0.8 Anthonys
I
0.31% of Americans are named Anthony
I
Estimate: on average, you know 1.6/0.0031 = 260 people
I
Why do these differ?
Andrew Gelman
Design and Analysis of Sample Surveys
Question wording for “How many X’s” surveys
Andrew Gelman
Design and Analysis of Sample Surveys
Perception vs. reality: names
Andrew Gelman
Design and Analysis of Sample Surveys
Perception vs. reality: groups
Andrew Gelman
Design and Analysis of Sample Surveys
Social structure!
How many Jacquelines
do you know?
How many men in prison
do you know?
●
●
●
●
0
5
10 or more
Andrew Gelman
0
5
Design and Analysis of Sample Surveys
10 or more
The gay penumbra
0
1
2-5
6-10
4.0
3.5
3.0
2.5
2.0
10+
0
1
2-5
6-10
10+
Marriage Support: Republicans
Marriage Support: Independents
1
2-5
6-10
number of gay people known
Andrew Gelman
10+
3.5
3.0
2.5
4.0
3.5
3.0
2.5
2.0
0
4.0
number of gay people known
Mean response on marriage equality
number of gay people known
1.5
Mean response on marriage equality
Marriage Support: Democrats
Mean response on marriage equality
4.0
3.5
3.0
2.5
Mean response on marriage equality
Marriage Support: Everyone
0
1
2-5
6-10
number of gay people known
Design and Analysis of Sample Surveys
10+
Penumbra sampling: goals
I
Learning about small groups (getting to the “penumbra”)
I
Relating networks to attitudes and behavior (gay rights,
foreign policy, etc.)
I
Learning about people who are not in the sample
Andrew Gelman
Design and Analysis of Sample Surveys
Penumbra sampling: methods
I
Network sampling, snowball sampling, referral sampling, . . .
I
Hypernetwork sampling (the National Congregations Study)
I
Questions about social networks (to learn about the survey
respondent)
I
Questions about attributes of friends, neighbors, etc. (to learn
about others)
I
Learning about unreachable groups (children, prisoners, dead
people, surly people)
I
Identifying penumbras of small groups (homeschoolers, Jews,
...)
Andrew Gelman
Design and Analysis of Sample Surveys
Polarization during the 2012 election campaign
I
An opportunity
I
Some survey questions
I
Data analysis
Andrew Gelman
Design and Analysis of Sample Surveys
An opportunity
Andrew Gelman
Design and Analysis of Sample Surveys
Some survey questions
Andrew Gelman
Design and Analysis of Sample Surveys
Data!
Andrew Gelman
Design and Analysis of Sample Surveys
Polarization during the 2012 election campaign
Andrew Gelman
Design and Analysis of Sample Surveys
Our next penumbra project
I
What groups to study?
I
Descriptive findings
I
Causal identification
Andrew Gelman
Design and Analysis of Sample Surveys
Your friends tend to agree with you
Andrew Gelman
Design and Analysis of Sample Surveys
You think your friends really agree with you
Andrew Gelman
Design and Analysis of Sample Surveys
Polarization, and perception of polarization
I
Two models for the Friend Sense results
I
I
I
Projection
Stereotyping
Other ways to measure perceptions of polarization?
Andrew Gelman
Design and Analysis of Sample Surveys
Fractal sampling
I
Fractal sampling in time
I
Fractal sampling in space
I
Gathering fractal information by sampling or interviewing
I
Turning a bug into a feature
I
Practical challenges
Andrew Gelman
Design and Analysis of Sample Surveys