Multilevel Modeling Break
Multilevel Modeling 1. Overview 2. Application #1: Growth Modeling Break 3. Application # 2: 4. Questions? Individuals Nested Within Groups Overview 1. 2. 3. 4. 5. 6. 7. 8. What is multilevel modeling? Examples of multilevel data structures Brief history Current applications Why multilevel modeling? What types of studies use multilevel modeling? Computer Programs (HLM 6 SAS Mixed Resources Multilevel Question What effects do the following variables have on 3rd grade reading achievement? School Size Classroom Climate Student Gender What is Multilevel or Hierarchical Linear Modeling? Nested Data Structures Several Types of Nesting 1. Individuals Nested Within Groups Individuals Undivided Unit of Analysis = Individuals Individuals Nested Within Groups Unit of Analysis = Individuals + Classes … and Further Nested Unit of Analysis = Individuals + Classes + Schools Examples of Multilevel Data Structures Neighborhoods are nested within communities Families are nested within neighborhoods Children are nested within families Examples of Multilevel Data Structures Schools are nested within districts Classes are nested within schools Students are nested within classes Multilevel Data Structures Level 4 District (l) Level 3 School (k) Level 2 Class (j) Level 1 Student (i) 2nd Type of Nesting Repeated Measures Nested Within Individuals Focus = Change or Growth Time Points Nested Within Individuals Repeated Measures Nested Within Individuals Carlos Day Monday = 0 Tuesday = 1 Wednes. = 2 Thursday = 3 Friday =4 Energy Level 98 90 85 72 70 Repeated Measures Nested Within Individuals 100 90 80 ENERG Y 70 60 0 DAY 1 2 3 4 5 Repeated Measures Nested Within Individuals 100 90 80 ENERGY 70 60 Rsq = 0.9641 0 DAY 1 2 3 4 5 Changes for 5 Individuals Changes in Energy Level Over the Week 100.00 Energy Level 75.00 50.00 25.00 0 0 1.00 2.00 Time 3.00 4.00 3rd Type of Nesting (similar to the 2nd) Repeated Measures Nested Within Individuals Focus is not on change Focus in on relationships between variables within an individual Repeated Measures Nested Within Individuals Carlos Day Hours of Sleep Energy Level Monday 9 98 Tuesday 8 90 Wednesday 8 85 Thursday 6 72 Friday 7 70 Repeated Measures Nested Within Individuals (Not Change) 100 90 80 ENERGY 70 60 5.5 6.0 HOURS 6.5 7.0 7.5 8.0 8.5 9.0 9.5 Repeated Measures Nested Within Individuals (Not Change) 100 90 80 ENERGY 70 60 5.5 6.0 HOURS 6.5 7.0 7.5 8.0 8.5 9.0 9.5 Repeated Measures Nested Within Individuals Repeated Measures Nested Within Individuals (3 Individuals) 100.00 Energy Level 75.00 50.00 25.00 0 2.00 4.50 7.00 Hours of Sleep 9.50 12.00 Repeated Measures Within Persons Level 2 Student (i) Level 1 Repeated Measures Over Time (t) Nested Data Data nested within a group tend to be more alike than data from individuals selected at random. Nature of group dynamics will tend to exert an effect on individuals. Nested Data Intraclass correlation (ICC) provides a measure of the clustering and dependence of the data 0 (very independent) to 1.0 (very dependent) Details discussed later Brief History of Multilevel Modeling Robinson, W. S. (1950). Ecological correlations and the behavior of individuals. Sociological Review, 15, 351357. Burstein, Leigh (1976). The use of data from groups for inferences about individuals in educational research. Doctoral Dissertation, Stanford University. Table 1 Frequency of HLM application evidenced in Scholarly Journals Journal 1999 2000 2001 2002 2003 Total by journal American Educational Research Journal 3 5 4 3 ? ~15 Child Development 3 2 6 5 13 29 Cognition and Instruction 1 0 0 0 0 1 Contemporary Educational Psychology 0 0 0 0 0 0 Developmental Psychology 2 1 2 5 7 17 Educational Evaluation and Policy Analysis 2 1 5 2 2 12 Educational Technology, Research and Development 0 0 0 0 0 0 Journal of Applied Psychology 1 1 5 7 6 20 Journal of Counseling Psychology 0 2 1 0 0 3 Journal of Educational Computing Research 0 0 0 0 0 0 Journal of Educational Psychology 1 2 3 6 1 13 Journal of Educational Research 2 0 3 3 5 13 Journal of Experimental Child Psychology 0 0 0 0 0 0 Journal of Experimental Education 0 0 0 0 1 1 Journal of Personality and Social Psychology 4 4 6 5 13 32 Journal of Reading Behavior/Literacy Research 0 0 0 0 0 0 Journal of Research in Mathematics Education 0 0 0 0 0 0 Reading Research Quarterly 0 0 0 1 0 1 Sociology of Education 1 2 5 2 1 11 Total by Year 20 20 40 39 49 ~168 Multilevel Articles Frequency of Studies Employing HLM in Education or Related Journals 50 Total for 19 Journals Reviewed Journal of Personality and Social Psychology Child Development Frequency Journal of Educational Research 25 0 1999 2000 2001 Year 2002 2003 Some Current Applications of Multilevel Modeling Growth Curve Analysis Value Added Modeling of Teacher and School Effects Meta-Analysis Multilevel Modeling Seems New But…. Extension of General Linear Modeling Simple Linear Regression Multiple Linear Regression ANOVA ANCOVA Repeated Measures ANOVA Multilevel Modeling Our focus will be on observed variables (not Latent Variables as in Structural Equation Modeling) Why Multilevel Modeling vs. Traditional Approaches? Traditional Approaches – 1-Level 1. 2. Individual level analysis (ignore group) Group level analysis (aggregate data and ignore individuals) Problems with Traditional Approaches 1. Individual level analysis (ignore group) Violation of independence of data assumption leading to misestimated standard errors (standard errors are smaller than they should be). Problems with Traditional Approaches 1. Group level analysis (aggregate data and ignore individuals) Aggregation bias = the meaning of a variable at Level-1 (e.g., individual level SES) may not be the same as the meaning at Level-2 (e.g., school level SES) Multilevel Approach 2 or more levels can be considered simultaneously Can analyze within- and betweengroup variability What Types of Studies Use Multilevel Modeling? Quantitative Experimental *Nonexperimental (Survey, Observational) How Many Levels Are Usually Examined? 2 or 3 levels very common 15 students x 10 classes x 10 schools = 1,500 Types of Outcomes Continuous Scale (Achievement, Attitudes) Binary (pass/fail) Categorical with 3 + categories Software to do Multilevel Modeling SPSS Users 2 SAV Files: Level 1 Level 2 HLM 6 (Menu Driven) (Raudenbush, Bryk, Cheong, & Congdon, 2004) HLM 6 Software to do Multilevel Modeling SAS Users Proc Mixed Resources (Sample…see handouts for more complete list) Books Hierarchical Linear Models: Applications and Data Analysis Methods, 2nd ed. Raudenbush & Bryk, 2002. Introducing Multilevel Modeling. Kreft & DeLeeum, 1998. Journals Educational and Psychological Measurement Journal of Educational and Behavioral Sciences Multilevel Modeling Newsletter Resources (cont) (Sample…see handouts for more complete list) Software HLM6 SAS (NLMIXED and PROC MIXED) MLwiN Journal Articles See Handouts for various methodological and applied articles Data Sets NAEP Data NELS:88; High School and Beyond Self-Check 1 A teacher with 1 classroom of 24 students used weekly curriculumbased measurements to monitor reading over a 14 week period. The teacher was interested in individual students’ rates of change and differences in change by male and female students. Self-Check 1 How would you classify this situation? (a) not multilevel (b) 2-level (c) 3-level Self-Check 2 A researcher randomly selected 50 elementary schools and randomly selected 30 teachers within each school. The researcher was interested in the relationships between 2 predictors (school size and teachers’ years experience at their current school) and teachers’ job satisfaction. Self-Check 2 How would you classify this situation? (a) not multilevel (b) 2-level (c) 3-level Self-Check 3 60 undergraduates from the research participant pool volunteered for a study that used written vignettes to manipulate the interactional style (warm, not warm) of a professor interacting with a student. 30 randomly assigned students read the vignette depicting warmth and 30 randomly assigned students read the vignette depicting a lack of warmth. After reading the vignette students used a questionnaire to rate the likeability of the professor. Self-Check 3 How would you classify this situation? (Select ONLY one) (a) not multilevel (b) 2-level (c) 3-level Growth Curve Modeling Studying the growth in reading achievement over a two year period Studying changes in student attitudes over the middle school years Research Questions What is the form of change for an individual during the study? Research Questions What is an individual’s initial status on the outcome of interest? Research Questions How much does an individual change during the course of the study? Rise Run b Rise Run Research Questions What is the average initial status of the participants? Research Questions What is the average change of the participants? Research Questions To what extent do participants vary in their initial status? Research Questions To what extent do participants vary in their growth? Research Questions To what extent does initial status relate to growth? Research Questions To what extent is initial status related to predictors of interest? Research Questions To what extent is growth related to predictors of interest? Design Issues How many waves a data collection are needed? >2 Depends on complexity of growth curve Design Issues Can there be different numbers of observations for different participants? Examples Missing data Planned missingness Design Issues Can the time between observations vary from participant to participant? Example: Students observed 1, 3, 5, & 7 months 1, 2, 4, & 8 months 2, 4, 6, & 8 months Design Issues How many participants are needed? More is better Power analyses > 30 rule of thumb Design Issues How should participants be sampled? What you have learned about sampling still applies Design Issues What is the value of random assignment? What you have leaned about random assignment still applies Design Issues How should the outcome be measured? What you have learned about measurement still applies Example Context description A researcher was interested in changes in verbal fluency of 4th grade students, and differences in the changes between boys and girls. ID 1 2 3 4 5 6 7 8 9 Gender 0 0 0 0 0 1 1 1 1 Time______ t0 t4 20 40 45 50 42 45 39 46 44 30 44 40 55 48 52 55 58 49 t7 30 49 60 59 53 61 63 68 59 Example Level-1 model specification Yfluency 0 1 * (Time) error1 Example Level-2 model specification 0 G00 G01 * (Gender ) error2 1 G10 G11 * (Gender ) Example Combined Model Yfluency G00 G01 * (Gender ) G10 * (Time ) G11 * (Gender ) * (Time) error2 error1 Example SAS program proc mixed covtest; class gender; model score = time gender time*gender/s; random intercept / sub=student s; Example SAS output – variance estimates Covariance Parameter Estimates Cov Parm Subject Estimate Standard Error Intercept Residual Student 62.5125 14.1173 35.9682 4.9912 Z Value Pr Z 1.74 2.83 0.0411 0.0023 Example SAS output – fixed effects Solution for Fixed Effects Effect Gender Intercept time Gender Gender time*Gender time*Gender F M F M Estimate 39.8103 1.5077 5.7090 0 1.0692 0 Standard Error DF t Value 3.7975 0.3295 5.6962 . 0.4943 . 7 16 16 . 16 . 10.48 4.58 1.00 . 2.16 . Pr > |t| <.0001 0.0003 0.3311 . 0.0460 . Example Graph – fixed effects 100.00 GENDER = 0 GENDER = 1 SCORE 75.00 50.00 25.00 0 0 2.50 5.00 TIME 7.50 10.00 Example Conclusions Fourth grade girl’s verbal fluency is increasing at a faster rate than boy’s. Persons Nested in Contexts Studying attitudes of teachers who are nested in schools Studying achievement for students who are nested in classrooms that are nested in schools Research Questions How much variation occurs within and among groups? To what extent do teacher attitudes vary within schools? To what extent does the average teacher attitude vary among schools? Research Questions What is the relationship among selected within group factors and an outcome? To what extent do teacher attitudes vary within schools as function of years experience? To what extent does student achievement vary within schools as a function of SES? Research Questions What is the relationship among selected between group factors and an outcome? To what extent do teacher attitudes vary across schools as function of principal leadership style? To what extent does student math achievement vary across schools as a function of the school adopted curriculum? Research Questions To what extent is the relationship among selected within group factors and an outcome moderated by a between group factor? To what extent does the within schools relationship between student achievement and SES depend on the school adopted curriculum? Design Issues Consider a design where students are nested in schools How should schools should be sampled? How should students be sampled within schools? Design Issues Consider a design where students are nested in schools How many schools should be sampled? How many students should be sampled per school? Design Issues What kind of outcomes can be considered? Continuous Binary Count Ordinal Design Issues How will level-1 variables be conceptualized and measured? SES How will level-2 variables be conceptualized and measured? SES Terminology Individual growth trajectory – individual growth curve model A model describing the change process for an individual Intercept Predicted value of an individual’s status at some fixed point The intercept cold represent the status at the beginning of a study Slope The average amount of change in the outcome for every 1 unit change in time Intercept & Slope Illustration 25 20 Score 15 10 b Rise 5 Rise Run Run 0 0 intercept 1 2 3 4 5 Time 6 7 8 9 10 Curvature =Acceleration=Quadratic Component 35 30 Score 25 20 15 10 5 0 0 1 2 Time 3 4 HLM Hierarchical Linear Model The hierarchical or nested structure of the data For growth curve models, the repeated measures are nested within each individual Levels in Multilevel Models Level 1 = time-series data nested within an individual Y 0 1 *(Time) error Levels in Multilevel Models Level 2 = model that attempts to explain the variation in the level 1 parameters 0 G00 G01 * (Sessions) error 1 G10 G11 * (Sessions) error More terminology Fixed coefficient A regression coefficient that does not vary across individuals Random coefficient A regression coefficient that does vary across individuals More terminology Balanced design Unbalanced design Unequal number of observation per unit Unconditional model Equal number of observations per unit Simplest level 2 model; no predictors of the level 1 parameters (e.g., intercept and slope) Conditional model Level 2 model contains predictors of level 1 parameters Estimation Methods Empirical Bayes (EB) estimate “optimal composite of an estimate based on the data from that individual and an estimate based on data from other similar individuals” (Bryk, Raudenbush, & Condon, 1994, p.4) Estimation Methods Expectation-maximization (EM) algorithm An iterative numerical algorithm for producing maximum likelihood estimates of variance covariance components for unbalanced data.
© Copyright 2024