Mixed, fixed, and random
effects models
Presented by:
Shannon Hodges
Jessica Raterman
Definitions
Fixed Effects Model: Assumes one true effect size which
underlies all studies in the analysis; differences due to random
error.
Random Effects Model: Considers within study variance (like
fixed-effects) and also between study variance (heterogeneity);
unobserved variables assumed to be uncorrelated with (or, more
strongly, statistically independent of) all the observed variables.
– RE modeling special case of the FE modeling
Mixed Model: Contains both fixed and random effects.
FEM vs. REM
• Fixed effects model -- individual specific effect is
correlated with the independent variables
– Dummies are considered part of the intercept
– Examines group differences in intercepts
– Assumes the same slopes and constant variance across
entities or subjects
FEM vs. REM
• Random effects model -- individual specific
effects are uncorrelated with the independent
variables
– Dummies act as an error term
– Data consist of a hierarchy of populations whose
differences relate to that hierarchy
– Estimates variance components for groups and error,
assuming the same intercept and slopes.
– The difference among groups (or time periods) lies in
their variance of the error term, not in their intercepts.
Fixed Effects Model
Examples of fixed effects factors:
– Sex: Male and Female
– Age: Minor and Adult
Random Effects Model
Yij   0   j   ij
Examples of random effects factors:
- group identity
- habitat identity
- parent identity
- genotype
Example: Fixed vs. Random
Intercepts
Slopes
When to choose multilevel?
• If dependence is a problem
– OLS regression suspect because the assumption
of independent residuals is invalid
• If dependence is interesting
– Disentangle variability at the various levels…
see the structure in more detail
– Give insight as to the directions where further
explanation may be sought
Choosing between fixed and random models
• 1. If groups are unique entities and inference should
focus on these groups, then FEM
– This often is the case with a small number of groups
• 2. If groups are regarded as sample (subset) from a
population and inference should focus on this
population, then REM
– This often is the case with a large number of groups.
• 3. If level-two (macro) effects are to be tested, then REM
Snijders (2011)
Choosing between fixed and random models
• 4. If group sizes are small and there are many groups,
and it is reasonable to assume exchangeability of
group-level residuals, then REM makes better use of
the data
• 5. If the researcher is interested only in within-group
effects, and is suspicious about the model for betweengroup differences, then FEM is more robust
• 6. If group effects are not nearly normally distributed,
REM is risky (or use more complicated multilevel
models).
Snijders (2011)
FEM vs. REM
Bell & Jones (2015): There are few occasions in which
FE modeling is preferable to RE modeling
• REM: greater flexibility and generalizability; more parsimonious
parameterization; and crucial ability to model context.
• “Assumptions made by RE models, including the exogeneity of
covariates and the Normality of residuals, are at least as
reasonable as those made by FE models when the model is
correctly specified.”
• “In controlling out context, FE models effectively cut out much of
what is going on, goings-on which are usually of interest to the
researcher, the reader, and the policy maker.”
FEM vs. REM
• REM omits the fixed effect and models the error
structure to overcome omitted variables bias
– REM is the way to estimate coefficients for variables
exhibiting multicollinearity
• Treating predictors in a model as a RE allows for more
general conclusions
• If the RE assumption holds, the REM is more efficient
than the FEM. If not, the REM is not consistent.
Mixed Effects Models
• Assumes that a linear
relationship exists
between independent
and dependent variables.
y  X   Z  
where
  fixed effects parameter estimates
X  fixed effects
Z  Random effects parameter estimates
  random effects
  errors
Variance of y  V  ZGZ '  R
G and R require covariance structure
fitting
Advantages of Mixed Effects Models
1.
2.
3.
4.
Can allow random effects to be properly specified
and computed, unlike the GLM.
It can allow correlation of errors, unlike the GLM. It
therefore has more flexibility in modeling the error
covariance structure.
It can allow the error terms to exhibit nonconstant
variability, unlike the GLM, allowing more flexibility
in modeling the dependent variable.
It can handle missing data, whereas the repeated
measures GLM cannot.
Mixed vs. Fixed
Figure 2. Correlation between the number
of offspring per decade of pair duration and
the total length a pair lasted.
References
•
Bell, A., & Jones, K. (2015). Explaining fixed effects: random effects modeling of timeseries cross-sectional and panel data. PSRM, 3(01), 133-153.
•
Crawley, M. J. (2007). “Mixed Effects models.” The R Book (First ed.) NJ: Wiley.
Fernandez-Duque, E. & Huck, M. (2013). Till Death (Or an Intruder) Do Us Part:
Intrasexual-Competition in a Monogamous Primate. PLOS One, 8(1): e53724.
•
Schofer, E. Multilevel Models. [Internet]. Available at:
http://www.socsci.uci.edu/~schofer/2007soc8811/publicfiles.htm
•
Snijders, T. A. (2011). Multilevel analysis (pp. 879-882). Springer Berlin Heidelberg.
pp. 627-659.
•
Wooldridge, Jeffrey M. (2013). "Fixed Effects Estimation.” Introductory Econometrics:
A Modern Approach (Fifth international ed.). Mason, OH: South-Western. pp. 466–474.
•
Yaffee, R.A. Mixed Analysis of Variance Models with SPSS. [Internet]. Available at:
http://google.ttu.edu/search?q=yaffee+&numgm=5&site=Campus&client=texas_tech&o
utput=xml_no_dtd&proxystylesheet=texas_tech&submit.x=0&submit.y=0