Designs With Nesting And Random Effects Dr. Jacqueline Asscher Copyright  2003 by Jacqueline Asscher. All rights reserved. Printed in Israel. No part of this publication may be reproduced, stored in a retrieval system, or transmitted in any form by any means, electronic, mechanical, photocopying, recording, or otherwise without the prior written permission of the author. No commercial use of this publication is permitted without the prior written permission of the author. Why Bother? The data structure dictates an analysis that takes into account whether effects are fixed or random, and whether they are crossed or nested. 2 Today We’ll Discuss Is an effect fixed or random? Is an effect crossed or nested? How do we define a model for a given experiment or for a sample from production data? How do we build the model using JMP? Notation of models Definition of several candidate models and choice of initial model/s to fit Prediction of observations in the data and of new data 3 Fixed Effects If the levels of an effect are determined by the experimenter, then this is a “fixed” effect E.g. temperature = 300°C, 350°C The factors in an experiment are usually fixed effects A future predicted response depends on the level of fixed effects e.g. expect machine1 to give high Y values 4 Random Effects If the levels of an effect are a random sample of values, then this is a “random” effect E.g. four batches are included in an experiment, or two operators participate People, animals (“subjects”), blocks and units of raw materials are random effects Interactions between random and fixed effects are random 5 Random Effects cont. Interactions between random and fixed effects are random A future predicted response based on the fitted model does not depend on the level of random effects, since the predicted level of a random effect is 0. E.g. we don’t expect the next batch to give low/high Y values (However, JMP predictions do depend on the level of random effects) 6 Components of Variance More than one random effect → estimates of “components of variance” (item-to-item, batch-to-batch) But: when a random effect has few levels → the estimate of that component of variance is poor So a typical analysis gives a mixture of poor and good estimates of components of variance 7 Example batch1 it1 it2 it3 batch2 it4 it1 it2 it3 batch3 it4 it1 it2 it3 it4 An experiment with four items from each of three batches has 3(4-1)=9 df for estimating item-toitem (within batch) variation, but only (3-1)=2 df for estimating batch-to-batch variation 8 Meaning of Significance for Fixed and Random Effects If two batches are included in an experiment and the batch effect is insignificant → these two batches were similar; a larger sample of batches could show differences But if a fixed effect is insignificant, → we conclude that there is no statistical evidence that the response depends on the level of the effect 9 Nested Effects Levels of B only occur within a single level of effect A → B is “nested” within A E.g. items are nested within batches, since each item occurs in only one batch The notation B[A] is used to show that B is nested within A We cannot evaluate the interaction between A and B if B is nested within A. 10 Crossed Effects Levels of an effect B occur for every level of an effect A → B is “crossed” with A E.g. locations are crossed with items, since each location occurs on every item We can evaluate the interaction between A and B if B is crossed with A 11 Are the Categories Fixed/random and Crossed/nested Related? Sort of! A nested effect is usually random A fixed effect is usually crossed (and should be if possible, if you are designing an experiment) 12 The Classification Is Not Always Clear Example 1: items are usually treated as random and nested within batches, since each item occurs in only one batch But the item effect could be a systematic effect (e.g. the items are baked in an oven) Which would be fixed and crossed with batches 13 Naming the levels Can Help Us Decide: Fixed or Random? 1) 2) Each item can be given its own name, say it9[batch1880], with nothing in common with it9[batch1882] → items are random and nested within batches Or: items it9[batch1880] and it9[batch1882] were both in shelf 9, which tends to give a high result → items are fixed and crossed with batches (each batch has each level of item) 14 Define the Aim 1. 2. 3. 4. Most models we fit are only approximate; tools for solving a problem rather than exact mathematical relationships There are different possible aims when fitting a model. For example: Check for effects e.g. differences in machines, effect of process change. Data can be from an experiment or observational Understanding the nature of a process, e.g. is variation random or systematic? Prediction Estimating components of variation, e.g. for a power analysis to determine sample size 15 Let’s Draw People… 1. drug1 1. 2. drug2 2. drug1 drug2 Drug: fixed or random? Subjects: fixed or random? Add hair and clothes to the people to represent a nested design (analysis?) Add hair and clothes to the people to represent a crossed design (analysis?) 16 Defining an Initial Model 1. 2. Look at the structure of the data Define one or more candidate statistical models (possible initial model/s to fit). Define the nesting structure, and whether effects are random or fixed A graphical display of the data and engineering knowledge re the process can contribute This is an initial, full model → include as many effects as possible 17 Defining an Initial Model 3. 4. Check degrees of freedom Often the process has a more complex (interesting) structure than can be modeled by the data available e.g. replicates are missing → can’t isolate components of variance; combinations of factors are missing → can’t estimate interactions 18 Oven Example Batches of six items are baked in an oven with three shelves: bottom, center and top Define a full initial model … (try!) 19 Oven Example 1. 2. Candidate full, initial models: Batch is random, location is fixed B, L, BL check df: B has (4-1)=3, L has (3-1)=2, LB has 3×2=6, pure error 12(2-1)=12 (→ B, L, BL or B, L or B *) Batch is random, location is random B, L[B] check df: B has (4-1)=3, L[B] has 4(3-1)=8, pure error 12(2-1)=12 (→ B, L[B] or B *) Strategy: fit Model 1 first; if the fixed location effect is insignificant, fit model 2. *why not consider removing B? 20 Oven Example - Models B, L B, L[B] B B, L, BL 21 Oven Example - Models 1. 2. Batch is random, location is fixed B, L, BL (→ B, L, BL or B, L or B) Yijk = µ + αi + βj + (αβ)ij + εijk (really εk(ij)) i=1…4, j=1…3, k=1,2 αi~N(0,σα2) batch, Σβj=0 location, (αβ)ij~N(0,σαβ2) Batch is random, location is random B, L[B] (→ B, L[B] or B) Yijk = µ + αi + βj(i) + εijk i=1…4, j=1…3, k=1,2 αi~N(0,σα2) batch, βj~N(0,σβ2) location 22 Oven Example - Output B, L[B] 23 Oven Example 2 – Define Models Use ovenexample2.jmp Define candidate full, initial models, and a strategy for fitting them Write the models using notation Yijk=… 24 Oven Example 2 – Fit Models 1. 2. 3. 1. 2. Use ovenexample2.jmp For Y5, Y6, Y7, Y8: Look at the Variability Chart Try to select a model using the graph alone Follow the strategy to select a model. Save the selected models and the residuals. Did you select the same model? For Y5, Y6, Y7, Y8: Look for a dependence of variability on a fixed factor (hint: repeat the Variability Charts, with residuals as response) Suggest a “quick and dirty” solution to this problem 25 Oven Example 2 26 Prediction of Observations in the Data The predicted value of an observation in the data (used for example to calculate residuals) depends on the level of the random effects for that observation This is the Conditional Pred Formula that we save using JMP We can display the model with a Variability Chart with the Conditional Pred Formula as the response 27 Prediction of New Data The predicted value of new data depends only on the levels of the fixed effects since the predicted value of all random effects is 0 This is the Prediction Formula that we save using JMP Note that in previous versions of JMP (5 and older), the Prediction Formula that we saved using JMP had to be adjusted for new data by replacing random effects with 0 28 Prediction of Observations in the Data - Example Note that the sum of the random batch effects is 0, when REML is used 29 Prediction of New Data Example 30 Oven Example2 – Prediction 1. 2. 1. 2. For Y5, Y6, Y7, Y8, predict observations in the data using the models you selected: What is the predicted value for items from batch2 that ran in the center of oven1? Display the model for prediction of observations in the data For Y5, Y6, Y7, Y8, predict new data using the models you selected: What is the predicted value of a new data point from the center of oven1? Display the model for prediction of new data 31 Statistical Questions What is REML? REstricted or REsidual Maximum Likelihood The model is fitted using a computingintensive, iterative algorithm The older, alternative method is EMS (Expected Mean Squares) 32 Statistical Questions What are “shrunken estimates”? When we have an estimate for the variance of a random effect, some estimates of a level of the effect may seem too big, and they will be adjusted towards zero – “shrunken”. For example, this will happen if the between batch variation is small, and one particular batch (with only a few items) has far higher results than the other batches. Why let the variance component estimates be negative? The unbiased estimates can be negative. This happens when the random effect has small variance and/or very few levels. There is an option in JMP to constrain the variance estimates to be nonnegative, but this leads to bias in the tests for fixed effects. 33