Chapter 19: Two-Factor Studies with Equal
Sample Size
Lecture 13
April 3, 2007
Psych 791
Slide 1 of 36
Seen Previously
■
Overview
Last Thursday we talked about:
◆
How the 2-way ANOVA model is:
● Previous Class
● Today’s Class
■
Formulated.
■
Estimated.
■
How significant effects are determined.
Testing Effects
Strategy for Analysis
Estimating Means
Multiple Comparisons
Interaction
Data Example
Wrapping Up
Slide 2 of 36
Today’s Class
Overview
● Previous Class
● Today’s Class
■
Today, we are going to go one step further.
■
How do we determine where the differences are once we
have determined that there are significant effects.
Testing Effects
Strategy for Analysis
Estimating Means
Multiple Comparisons
Interaction
Data Example
Wrapping Up
Slide 3 of 36
Testing Effects
Slide 4 of 36
Testing for Main Effect of A
■
Null Hypothesis:
Overview
H0 : α1 = α2 = . . . = αa = 0
Testing Effects
● Testing for Main Effect of A
● Testing for Main Effect of B
● Testing for Interaction Effect
of A × B
Ha : not all αi = 0
● Family of Significance
Strategy for Analysis
■
or alternatively:
Estimating Means
Multiple Comparisons
H0 : µ1· = µ2· = . . . = µa·
Interaction
Data Example
Ha : not all µi· equal
Wrapping Up
■
To test:
M SA
F (a − 1, ab(n − 1)) =
M SE
Slide 5 of 36
Testing for Main Effect of B
■
Null Hypothesis:
Overview
H0 : β1 = β2 = . . . = βb = 0
Testing Effects
● Testing for Main Effect of A
● Testing for Main Effect of B
● Testing for Interaction Effect
of A × B
Ha : not all βj = 0
● Family of Significance
Strategy for Analysis
■
or alternatively:
Estimating Means
Multiple Comparisons
H0 : µ·1 = µ·2 = . . . = µ·b
Interaction
Data Example
Ha : not all µ·j equal
Wrapping Up
■
To test:
M SB
F (b − 1, ab(n − 1)) =
M SE
Slide 6 of 36
Testing for Interaction Effect of A × B
■
Null Hypothesis:
Overview
H0 : all (αβ)ij = 0
Testing Effects
● Testing for Main Effect of A
● Testing for Main Effect of B
● Testing for Interaction Effect
of A × B
Ha : not all (αβ)ij = 0
● Family of Significance
Strategy for Analysis
■
or alternatively:
Estimating Means
Multiple Comparisons
H0 : all µij equal
Interaction
Data Example
Ha : not all µij equal
Wrapping Up
■
To test:
F ((a − 1)(b − 1), ab(n − 1)) =
M SAB
M SE
Slide 7 of 36
Family of Significance
■
The Bonferroni inequality was a way of controlling our alpha level.
■
So, if we have three significance tests (both main effects and interaction),
each at an alpha level of 0.05, then our total α according to Bonferroni is:
α ≤ α1 + α2 + α3
.15 ≤ .05 + .05 + .05
■
A second way of looking at this is called the Kimball inequality.
■
This takes into account that the SSE is in each test statistic (common
denominator):
α ≤ 1 − (1 − α1 )(1 − α2 )(1 − α3 )
.143 ≤ 1 − (.95)(.95)(.95)
Slide 8 of 36
Strategy for Analysis
■
There are 6 steps that you should follow when analyzing two factor studies:
1. Examine whether the two factors interact.
2. If they do not interact, examine whether the main effects for both factors (A
and/or B) are important. If important, describe the nature of the effects.
3. If they do interact, examine if the interactions are important.
4. If they are unimportant, proceed to step 2.
5. If they are important, can they be made unimportant by a transformation?
If so, go to step 2.
6. For important interactions (irrespective of scale), examine the two factors
jointly (or the nature of the interaction.
Slide 9 of 36
Factor Level Means
µi· or µ·j
Overview
■
Estimate them by changing µ to Y
Testing Effects
µi· = Y¯i··
Strategy for Analysis
Estimating Means
● Factor Level Means
● Contrasts
µ·j = Y¯·j·
● Contrasts, the other way
● Linear Combinations
Multiple Comparisons
■
Interaction
Data Example
Variance now are not simply σ 2
2
σ
M SE
σ (Y¯i·· ) =
=
bn
bn
2
Wrapping Up
2
σ
M SE
=
σ (Y¯·j· ) =
an
an
2
■
Usual confidence intervals can be constructed: test statistic
is t with (n − 1)abdf .
Slide 10 of 36
Contrast in Factor Level Means
■
You can estimate a contrast, or difference in factor level
means for Factor A
Overview
Testing Effects
L=
Strategy for Analysis
X
ci µi· where
X
ci = 0
Estimating Means
● Factor Level Means
● Contrasts
■
Estimate it by substituting Y
● Contrasts, the other way
● Linear Combinations
L=
Multiple Comparisons
X
ci Y¯i··
Interaction
Data Example
■
The variance of this contrast is then:
Wrapping Up
2 X
σ
M SE X
2 b
σ (L) =
ci =
ci
bn
bn
Slide 11 of 36
Contrasts, the other way
■
Again, you can estimate a contrast, or difference in factor
level means, for Factor B
Overview
Testing Effects
L=
Strategy for Analysis
X
cj µ·j where
X
cj = 0
Estimating Means
● Factor Level Means
● Contrasts
■
Estimate it by substituting Y
● Contrasts, the other way
● Linear Combinations
L=
Multiple Comparisons
X
cj Y¯·j·
Interaction
Data Example
■
The variance of this contrast is then:
Wrapping Up
2 X
σ
M SE X
2 b
σ (L) =
cj =
cj
an
an
Slide 12 of 36
Linear Combinations
Overview
Testing Effects
■
This is a "contrast" where the c’s sum to 1 instead of 0.
■
Estimate these in the same way as the previous 2 slides,
same estimates, same variances.
Strategy for Analysis
Estimating Means
● Factor Level Means
● Contrasts
● Contrasts, the other way
● Linear Combinations
Multiple Comparisons
Interaction
Data Example
Wrapping Up
Slide 13 of 36
Multiple Comparison Procedures
■
Usually, you don’t want to look at one or two mean
differences, you want to look at all of them at the same time.
■
We can employ the same multiple comparison procedures
used in single factor studies (Tukey, Bonferroni, Scheffé),
with minor tweaking.
Overview
Testing Effects
Strategy for Analysis
Estimating Means
Multiple Comparisons
● Tukey
● Bonferroni
● Both A and B
● Multiple Contrasts - Scheffé
● Multiple Contrasts Bonferroni
● Both Factor A and B
Interaction
Data Example
Wrapping Up
Slide 14 of 36
Tukey
■
We have the same test for difference (for Factor A as an
example):
Overview
Testing Effects
D = µi· − µi′ ·
Strategy for Analysis
Estimating Means
■
Multiple Comparisons
We have the same confidence interval:
● Tukey
● Bonferroni
b ± t(1 − α/2; nT − r) ∗ s(D)
b
D
● Both A and B
● Multiple Contrasts - Scheffé
● Multiple Contrasts Bonferroni
● Both Factor A and B
Interaction
■
But two things change, the standard deviation and the test
statistic:
Data Example
Wrapping Up
1
T = √ q(1 − α; a, (n − 1)ab)
2
2M SE
b
s (D) =
bn
2
Slide 15 of 36
Bonferroni
■
Use the same formulas as above for difference and standard
deviation, but replace T with B:
Overview
Testing Effects
Strategy for Analysis
B = t(1 − α/2g; (n − 1)ab)
Estimating Means
Multiple Comparisons
● Tukey
● Bonferroni
● Both A and B
● Multiple Contrasts - Scheffé
● Multiple Contrasts Bonferroni
● Both Factor A and B
Interaction
Data Example
Wrapping Up
Slide 16 of 36
Both A and B
■
Both the Tukey and the Bonferroni method look at the effects
of a single factor.
■
If both are significant, we will probably want to perform
multiple comparison procedures on both of these effects.
■
You can do this in two ways:
Overview
Testing Effects
Strategy for Analysis
Estimating Means
Multiple Comparisons
● Tukey
● Bonferroni
● Both A and B
● Multiple Contrasts - Scheffé
● Multiple Contrasts -
1. Use the Bonferroni method, where g is ALL the
differences for both factors.
Bonferroni
● Both Factor A and B
Interaction
Data Example
Wrapping Up
2. Use both Bonferroni and Tukey: calculate a Bonferroni for
the number of factors you want to test, then run a Tukey
using that Bonferroni (basically if you estimate 2 effects,
use an α = 0.025 for the Tukey test).
Slide 17 of 36
Multiple Contrasts - Scheffé
b ± S ∗ s(L)
b
L
Overview
■
For analyzing multiple contrasts in two factor studies, we
define S as follows.
■
For the A factor:
Testing Effects
Strategy for Analysis
Estimating Means
p
S = (a − 1)F (1 − α; a − 1, (n − 1)ab)
Multiple Comparisons
● Tukey
● Bonferroni
● Both A and B
● Multiple Contrasts - Scheffé
● Multiple Contrasts Bonferroni
● Both Factor A and B
Interaction
Data Example
■
For the B factor:
p
S = (b − 1)F (1 − α; b − 1, (n − 1)ab)
Wrapping Up
Slide 18 of 36
Multiple Contrasts - Bonferroni
■
Insert a B for the S in the previous formula.
b ± B ∗ s(L)
b
L
Overview
Testing Effects
Strategy for Analysis
■
Estimating Means
Multiple Comparisons
● Tukey
● Bonferroni
● Both A and B
Define B as:
B = t(1 − α/2g; (n − 1)ab)
● Multiple Contrasts - Scheffé
● Multiple Contrasts Bonferroni
● Both Factor A and B
Interaction
Data Example
Wrapping Up
Slide 19 of 36
Both Factor A and B
■
When using multiple contrasts involving both factors at the
same time, we still want to control our total error.
■
There are three possibilities in this case:
Overview
Testing Effects
Strategy for Analysis
Estimating Means
Multiple Comparisons
1. Use the Bonferroni method, changing g to the total
number of contrasts.
● Tukey
● Bonferroni
● Both A and B
2. Use Bonferroni for the total α for more than one Scheffé.
● Multiple Contrasts - Scheffé
● Multiple Contrasts Bonferroni
● Both Factor A and B
Interaction
Data Example
3. Change the S in Scheffé to:
p
S = (a + b − 2)F (1 − α; a + b − 2, (n − 1)ab)
Wrapping Up
Slide 20 of 36
Important Interaction
Overview
■
Each of the above tests are for factor level means.
■
Let us go back to the idea of an important interaction.
■
Suppose we find a significant interaction that we deem
important, what do we do next?
■
It would be nice to have some way to test differences in the
treatment means.
Testing Effects
Strategy for Analysis
Estimating Means
Multiple Comparisons
Interaction
● Important Interaction
● Tukey
● Bonferroni
● Multiple Treatment Contrasts
● Scheffé
● Bonferroni
Data Example
Wrapping Up
Slide 21 of 36
Multiple Comparisons - Treatment
■
We are just going to extend this idea of multiple pairwise
comparisons to treatment means.
■
We are going to look at all possible pairwise µij or subsets of
the pairs.
Overview
Testing Effects
Strategy for Analysis
Estimating Means
D = µij − µi′ j ′
Multiple Comparisons
Interaction
● Important Interaction
● Tukey
● Bonferroni
● Multiple Treatment Contrasts
● Scheffé
■
Using this new difference estimate, we can now use either
Tukey or Bonferroni’s method on these treatment means.
● Bonferroni
Data Example
Wrapping Up
Slide 22 of 36
Tukey
b ± T ∗ s(D)
b
D
Overview
b = 2M SE
s2 (D)
n
Testing Effects
Strategy for Analysis
Estimating Means
■
Then we just need to define our T:
Multiple Comparisons
Interaction
● Important Interaction
● Tukey
● Bonferroni
1
√
q(1 − α; ab, (n − 1)ab)
T =
2
● Multiple Treatment Contrasts
● Scheffé
● Bonferroni
Data Example
Wrapping Up
Slide 23 of 36
Bonferroni
b ± B ∗ s(D)
b
D
Overview
b =
s(D)
Testing Effects
Strategy for Analysis
Estimating Means
■
2M SE
n
Then we just need to define our B:
Multiple Comparisons
Interaction
● Important Interaction
● Tukey
B = t(1 − α/2g; (n − 1)ab)
● Bonferroni
● Multiple Treatment Contrasts
● Scheffé
● Bonferroni
Data Example
Wrapping Up
Slide 24 of 36
Multiple Treatment Contrasts
Overview
■
We may want to look at contrasts for treatment effects.
■
These contrasts would then be of the form:
Testing Effects
L=
Strategy for Analysis
X
cij µij where
X
cij = 0
Estimating Means
Multiple Comparisons
■
Interaction
● Important Interaction
● Tukey
The same idea is employed, just with slightly different
methods.
● Bonferroni
● Multiple Treatment Contrasts
● Scheffé
● Bonferroni
Data Example
Wrapping Up
Slide 25 of 36
Scheffé
■
Multiple Contrasts can be estimated using Scheffé’s method:
b ± S ∗ s(L)
b
L
Overview
Testing Effects
Strategy for Analysis
XX
M
SE
b =
s (L)
c2ij
n
2
Estimating Means
Multiple Comparisons
Interaction
● Important Interaction
● Tukey
● Bonferroni
■
For analyzing multiple contrasts in two factor studies for the
interaction term, we define S as follows:
● Multiple Treatment Contrasts
● Scheffé
● Bonferroni
Data Example
p
S = (ab − 1)F (1 − α; ab − 1, (n − 1)ab)
Wrapping Up
Slide 26 of 36
Bonferroni
■
Overview
B = t(1 − α/2g; (n − 1)ab)
Testing Effects
Strategy for Analysis
■
Estimating Means
Multiple Comparisons
Same as on previous slide, but replace S with B:
Just as in single factor studies, Bonferroni is preferred when
the number of contrasts is small.
Interaction
● Important Interaction
● Tukey
● Bonferroni
● Multiple Treatment Contrasts
● Scheffé
● Bonferroni
Data Example
Wrapping Up
Slide 27 of 36
Putting It All Together
■
The primary objective of the Study on the Efficacy of
Nosocomial Infection Control (SENIC Project) was to
determine whether infection surveillance and control
programs have reduced the rates of nosocomial
(hospital-acquired) infection in the United States. 113
subjects were studied in this sample.
■
Research Question: The researchers are interested in
determining if medical school affiliation and region of the
country are related to infection risk.
Overview
Testing Effects
Strategy for Analysis
Estimating Means
Multiple Comparisons
Interaction
Data Example
● Putting It All Together
● SAS Code - Step 1
● Output - Step 1
● SAS Code - Step 2
● Output - Step 2
● SAS code - test differences
● Output - test differences
Wrapping Up
Slide 28 of 36
SAS Code - Step 1
Overview
Testing Effects
Strategy for Analysis
data senic;
infile ’path\senicex1.csv’ dlm=’,’;
input id infrisk
medaff region;
run;
Estimating Means
Multiple Comparisons
Interaction
Data Example
● Putting It All Together
proc glm data=senic;
class medaff region;
model infrisk = medaff|region;
run;
● SAS Code - Step 1
● Output - Step 1
● SAS Code - Step 2
● Output - Step 2
● SAS code - test differences
● Output - test differences
Wrapping Up
Slide 29 of 36
Output - Step 1
Dependent Variable: infrisk
DF
Sum of
Squares
Mean Square
F Value
Pr > F
Model
7
30.1153614
4.3021945
2.64
0.0150
Error
105
171.2644616
1.6310901
Corrected Total
112
201.3798230
Source
Overview
Testing Effects
Strategy for Analysis
Estimating Means
Multiple Comparisons
R-Square
Coeff Var
Root MSE
infrisk Mean
0.149545
29.32676
1.277141
4.354867
Interaction
Data Example
● Putting It All Together
● SAS Code - Step 1
● Output - Step 1
Source
DF
Type I SS
Mean Square
F Value
Pr > F
1
3
3
10.93551541
11.64824242
7.53160355
10.93551541
3.88274747
2.51053452
6.70
2.38
1.54
0.0110
0.0738
0.2087
DF
Type III SS
Mean Square
F Value
Pr > F
1
3
3
7.61386560
5.38396032
7.53160355
7.61386560
1.79465344
2.51053452
4.67
1.10
1.54
0.0330
0.3525
0.2087
● SAS Code - Step 2
● Output - Step 2
● SAS code - test differences
● Output - test differences
medaff
region
medaff*region
Wrapping Up
Source
medaff
region
medaff*region
Slide 30 of 36
SAS Code - Step 2
Overview
Testing Effects
proc glm data=senic;
class medaff region;
model infrisk = medaff region;
run;
Strategy for Analysis
Estimating Means
Multiple Comparisons
Interaction
Data Example
● Putting It All Together
● SAS Code - Step 1
● Output - Step 1
● SAS Code - Step 2
● Output - Step 2
● SAS code - test differences
● Output - test differences
Wrapping Up
Slide 31 of 36
Output - Step 2
Dependent Variable: infrisk
DF
Sum of
Squares
Mean Square
F Value
Pr > F
Model
4
22.5837578
5.6459395
3.41
0.0115
Error
108
178.7960652
1.6555191
Corrected Total
112
201.3798230
Source
Overview
Testing Effects
Strategy for Analysis
Estimating Means
Multiple Comparisons
R-Square
Coeff Var
Root MSE
infrisk Mean
0.112145
29.54556
1.286670
4.354867
Interaction
Data Example
● Putting It All Together
● SAS Code - Step 1
● Output - Step 1
Source
DF
Type I SS
Mean Square
F Value
Pr > F
medaff
region
1
3
10.93551541
11.64824242
10.93551541
3.88274747
6.61
2.35
0.0115
0.0769
Source
DF
Type III SS
Mean Square
F Value
Pr > F
medaff
region
1
3
8.58681851
11.64824242
8.58681851
3.88274747
5.19
2.35
0.0247
0.0769
● SAS Code - Step 2
● Output - Step 2
● SAS code - test differences
● Output - test differences
Wrapping Up
Slide 32 of 36
SAS code - test differences
Overview
Testing Effects
Strategy for Analysis
proc glm data=senic;
class medaff region;
model infrisk = medaff region;
lsmeans medaff/tdiff adjust=tukey;
run;
Estimating Means
Multiple Comparisons
Interaction
Data Example
● Putting It All Together
● SAS Code - Step 1
● Output - Step 1
● SAS Code - Step 2
● Output - Step 2
● SAS code - test differences
● Output - test differences
Wrapping Up
Slide 33 of 36
Output - test differences
The GLM Procedure
Least Squares Means
Adjustment for Multiple Comparisons: Tukey-Kramer
Overview
medaff
infrisk
LSMEAN
H0:LSMean1=LSMean2
t Value
Pr > |t|
Testing Effects
Strategy for Analysis
1
2
5.05375330
4.27289190
2.28
0.0247
Estimating Means
Multiple Comparisons
Interaction
Data Example
● Putting It All Together
● SAS Code - Step 1
● Output - Step 1
● SAS Code - Step 2
● Output - Step 2
● SAS code - test differences
● Output - test differences
Wrapping Up
Slide 34 of 36
Final Thought
■
Today we covered the
multiple comparisons side
of 2-way ANOVA.
■
Everything shown was
identical to what happened
in 1-way ANOVA.
■
With this we can put Chapter 19 to bed.
■
We will encounter interactions later in the book.
Overview
Testing Effects
Strategy for Analysis
Estimating Means
Multiple Comparisons
Interaction
Data Example
Wrapping Up
● Final Thought
● Next Class
Slide 35 of 36
Next Time
■
Chapter 21 (skip Chapter 20): Randomized Complete Block
Designs.
Overview
Testing Effects
Strategy for Analysis
Estimating Means
Multiple Comparisons
Interaction
Data Example
Wrapping Up
● Final Thought
● Next Class
Slide 36 of 36