Research supported by TLRI
Randomisation: part 2
Randomisation S1
Study design and inference
Experiments & the Randomisation Test:
• The difference between two means
Watch out for:
•
•
The ‘chance is acting alone’ explanation
How we assess the plausibility of the ‘chance
alone’ explanation – (test for ‘chance alone’)
Randomisation S2
What does ‘chance alone’ look like?
iNZightVIT
Randomisation
Chapter 1
The Walking Babies Experiment
Does a special exercise programme lower
walking age?
Phillip R. Zelazo, Nancy Ann Zelazo, & Sarah Kolb, “Walking in the Newborn”
Science, Vol. 176 (1972), pp314-315
10 male infants (& parents) were randomly assigned to
one of two treatment groups.
First walked without support:
Treatment
Exercise
Control
9
13.25
Age (months)
9.5
9.75
10
11.5
12
13.5
11
11.5
Randomisation S4
The Walking Babies Experiment
9
10
11
12
Age (months)
13
14
Randomisation S5
Is chance alone
likely to generate
differences as big
as our difference?
Tail proportion:
roughly ____%
Re-randomisation distribution of differences
(under chance alone)
The Walking Babies
Experiment
Possible explanation:
One possible explanation for the observed difference
between these two groups:
Chance is acting alone (the exercise has no effect)
• We can rule out ‘chance is acting alone’ as a plausible
explanation for the difference between the two groups.
• We have evidence against ‘chance is acting alone’
• We have evidence that chance is not acting alone
Randomisation S7
The Walking Babies
Experiment
Possible explanation:
If chance
not acting
thendifference
what else
One
possibleis
explanation
for alone,
the observed
is also acting
togroups:
help produce the observed
between
these two
Chance
is acting alone (the exercise has no effect)
difference?
• We can rule out ‘chance is acting alone’ as a plausible
Remember:
explanation for the difference between the two groups.
Random assignment to 2 groups & each group receives
• different
We havetreatment.
evidence against ‘chance is acting alone’
• We have evidence that chance is not acting alone
Randomisation S8
The Walking Babies
Experiment
Conclusion:
Because the male infants (& parents) were
randomly assigned to the groups, we may claim
that the exercise was effective in lowering
the walking age.
Because these subjects in this experiment were
volunteers (not randomly selected), then we
would need to consider carefully as to which wider
group(s) this conclusion may apply.
Randomisation S9
Two types of Inference
There are two types of inference
1. Sample-to-population
eg x = 172cm so the population mean is about 172cm.
2. Experiment-to-causation
eg The treatment was effective
Randomisation S10
The Walking Babies Experiment
Does a special exercise programme lower
walking age?
Phillip R. Zelazo, Nancy Ann Zelazo, & Sarah Kolb, “Walking in the Newborn”
Science, Vol. 176 (1972), pp314-315
10 male infants (& parents) randomly assigned to
either the Exercise group or the Control exercise group.
First walked without support:
Treatment
Exercise
Control exercise
9
11
Age (months)
9.5
9.75
10
10
15
11.75
13
10.5
Randomisation S11
The Walking Babies
Experiment
9
10
11
12
13
14
15
Age (months)
Randomisation S12
The Walking Babies
Experiment
Possible explanation:
One possible explanation for the observed difference
between these two groups:
Chance is acting alone (the exercise has no effect)
Randomisation S13
The Walking Babies
Experiment
Possible explanation:
One possible explanation for the observed difference
between these two groups:
Chance is acting alone (the exercise has no effect)
Is the ‘Chance alone’ explanation simply not plausible?
• Would our observed difference be unlikely when chance
is acting alone?
• How do we determine whether a difference is
unlikely when chance is acting alone?
• See what’s likely and what’s unlikely when chance is
acting alone
Randomisation S14
Our observed difference
= 1.4 months
Is chance alone
likely to generate
differences as big
as our difference?
Re-randomisation distribution of differences
(under chance alone)
Tail proportion:
roughly ___/1000
(____%)
Do the actual exercises
lower the walking age?
Our tail proportion of ____/1000 = ___% means:
•
___ times out-of-a-1000 times we get a difference of
1.4 months or more, when chance is acting alone.
• Under chance alone, it’s not unusual to get a difference
bigger than or equal to our observed difference of 1.4
months.
• A difference of 1.4 months or greater is not
unusual when chance is acting alone, . . .
therefore chance could be acting alone.
Randomisation S16
The Walking Babies
Experiment
Possible explanation:
One possible explanation for the observed difference
between these two groups:
Chance is acting alone (the treatment has no effect)
• We can NOT rule out ‘chance is acting alone’ as a
plausible explanation for the observed difference
between the two groups.
• We have no evidence against ‘chance-is-acting-alone’
• Chance COULD be acting alone . . . BUT something else,
as well as chance, COULD ALSO be acting.
Randomisation S17
The Walking Babies
Experiment
Possible explanation:
One
possibleis
explanation
for alone,
the observed
If chance
not acting
thendifference
what else
between
thesebe
two
groups:to help produce the
could also
acting
Chance is acting alone (the treatment has no effect)
observed difference?
• We can NOT rule out ‘chance is acting alone’ as a
Remember:
plausible explanation for the observed difference
Random
assignment
to 2 groups & each group receives
between
the two groups.
different treatment.
• We have no evidence against ‘chance-is-acting-alone’
• Chance could be acting alone . . . BUT something else,
as well as chance, COULD ALSO be acting.
Randomisation S18
The Walking Babies
Experiment
Conclusion:
Because the male infants (& parents) were
randomly assigned to the groups, we conclude
that the observed difference is the result of
• EITHER chance acting alone
• OR an exercise effect together with
chance acting
– we do NOT have ENOUGH INFORMATION to
MAKE A CALL as to which one.
Randomisation S19
Did she brush her teeth (2)?
1. Formulate statement to test.
1. She has brushed her teeth.
2. Data (information at hand).
2. The toothbrush is wet.
3. Consider 1. and the data:
3. The-toothbrush-is-wet would
If 1. is true, then what are the
be likely if she had brushed
chances of getting data like
her teeth.
that in 2.?
4. Review the statement in 1. in 4. Therefore, she could have
light of 3. together with the
brushed her teeth.
data in 2.
We have no evidence that she
has not brushed her teeth.
I do not know
Randomisation S20
Is the actual exercise effective?
1. Formulate statement to test.
1. Chance is acting alone.
2. Data (information at hand).
2. Observed diff = 1.4 months
3. Consider 1. and the data:
3. A difference of 1.4 months or
If 1. is true, then what are the
greater is
chances of getting data like
that in 2.?
4. Review the statement in 1. in
light of 3. together with the
data in 2.
Randomisation S21
Is the actual exercise effective?
1. Formulate statement to test.
1. Chance is acting alone.
2. Data (information at hand).
2. Observed diff = 1.4 months
3. Consider 1. and the data:
3. A difference of 1.4 months or
If 1. is true, then what are the
greater is not unusual when
chances of getting data like
chance is acting alone.
that in 2.?
4. Therefore, chance could
4. Review the statement in 1. in
be acting alone OR
light of 3. together with the
something else could be
acting along with
data in 2.
chance.
we do NOT have ENOUGH INFORMATION
to MAKE A CALL as to which one.Randomisation S22
Guidelines for assessing
‘Chance alone
’
When the tail proportion is small (less than 10%):
• the observed difference would be unlikely when
chance is acting alone . . . therefore, it’s a fairly safe
bet chance is not acting alone.
• we have evidence against ‘chance-is-acting-alone’
• we have evidence that chance is not acting alone
Chapter 1
Guidelines for assessing
‘Chance alone’
When the tail proportion is large (10% or more ) then:
• the observed difference is not unusual when chance is
acting alone, therefore chance could be acting alone
• we have NO evidence against ‘chance is acting alone’
• EITHER chance could be acting alone OR something
else as well as ‘chance’ COULD also be acting.
-- we do NOT have ENOUGH INFORMATION to MAKE A
CALL as to which one.
Randomisation S24
Questions…
Chapter 1