Manuscript under review for Psychological Science Challenges for the Estimation of an Upper-Bound on Relations between Accumulated Deliberate Practice and the Associated Performance of Novices and Experts: Comments on Macnemara, Hambrick, and Oswald’s (2014) Published Meta Analysis r Fo Journal: Manuscript ID: Date Submitted by the Author: Complete List of Authors: Commentary n/a Ericsson, Karl; Florida State University, Psychology vi Keywords: Draft Re Manuscript Type: Psychological Science Learning, Performance ew ly On Page 1 of 8 Challenges for the Estimation of an Upper-Bound on Relations between Accumulated Deliberate Practice and the Associated Performance of Novices and Experts: Comments on Macnemara, Hambrick, and Oswald’s (2014) Published Meta Analysis r Fo K. Anders Ericsson Re Send correspondence about this commentary to: vi Department of Psychology Florida State University Tallahassee, Florida 32306-4301 ew e-mail: [email protected] On Phone(Office): 850-644-9860 FAX: 850-644-7739 ly 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 Manuscript under review for Psychological Science Manuscript under review for Psychological Science Abstract Ericsson, Krampe, and Tesch- Römer, (1993) reviewed evidence on conditions for optimal learning and established criteria for deliberate practice, namely practice with effective training tasks (selected by a supervising teacher) with clear performance goals directed to mastery in the domain. In contrast, Macnemara, Hambrick, and Oswald’s (2014) collected studies that “referred to at least one publication on deliberate practice by Ericsson and his colleagues” (p. 1610, italics added). This selection criterion led to the inclusion of studies that r Fo did not even mention “deliberate practice” and most included studies violated our criteria for deliberate practice. Among the effect sizes that met the criteria for deliberate practice only one of them included a study that included both novices and experts in the domain of expertise. More Re fruitful directions are proposed for research on the limits of deliberate practice to improve performance in domains of expertise. ew Words=143 vi ly On 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 Page 2 of 8 Page 3 of 8 In our original paper Ericsson, Krampe, and Tesch- Römer, (1993) reviewed evidence on “conditions for optimal learning and improvement of performance” (p. 367) and cited Chase and Ericsson’s (1982, see also Ericsson, 2013) research demonstrating that a college student was able, with individual practice and immediate feedback, to improve his memory span from 7 to 82 digits (corresponding to an effect size exceeding d=50). Ericsson et al., (1993) found that effective practice activities with immediate feedback that led to desired objective performance in real-world domains of expertise. They identified some domains, however, where students are r Fo given individualized instruction and “the teacher designs practice activities that the individual can engage in between meetings with the teacher” (p. 368). They distinguished this type of practice by giving it its own name: Ericsson et al (1993) used the term “deliberate practice for Re the individualized training activities specially designed by a coach or teacher to improve specific vi aspects of an individual's performance through repetition and successive refinement.” (Ericsson & Lehmann, 1996, pp. 278-279). ew Macnemara, Hambrick, and Oswald’s (2014) meta-analysis took a different approach and avoided the need to specify how studies actually measured deliberate practice. They used an On unusual criterion for inclusion of studies, namely that “the study report referred to at least one ly 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 Manuscript under review for Psychological Science publication on deliberate practice by Ericsson and his colleagues” (p.1610) supplemented with a subjective judgment of relevance. Their criterion led to inclusion of studies that did not even mention “deliberate practice”. Studies of education included 45 (88%) effect sizes that came from studies that never used the term “deliberate practice” anywhere in the text of their articles (see supplementary materials). Their only mention of deliberate practice was in the title of Plant, Ericsson, Hill and Asberg’s (2005) article “Why study time does not predict grade point average: Implications of deliberate Manuscript under review for Psychological Science practice for academic performance” (GPA) (p. 96). This article did not study deliberate practice, but instead examined study behavior in college “in light of characteristics of deliberate practice” and found “important similarities as well as differences” (p. 114). The majority of the remaining studies in the meta-analysis used operational definitions of practice that violate our original definition of deliberate practice. Many studies measured practice where teachers did not evaluate the individuals’ performance nor assigned their trainees effective individual practice activities with performance goals. Even in studies where teachers r Fo and coaches are typically present during team and group practice, the associated practice time is not primarily spent on individualized practice activities that would be most beneficial for each individual trainee. Finally several included studies tested performance for a different skill than Re had been the assigned target of the deliberate practice. Instead of measuring the ability to vi perform music pieces that had been extensively rehearsed, investigators tested performance for ew playing scales or playing unrehearsed music (sight reading music). Applying all of these criteria led to the rejection of 137 (87%) of all 157 included effect sizes (see supplementary materials for details). On These alternative forms of practice are not excluded because they are ineffective, but ly 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 because they do not measure uncontaminated deliberate practice. One potentially confusing issue concerns findings that expert performers in some domains, such as chess and SCRABBLE, have been found to be able to design effective practice activities for themselves while they practice alone and that these practice activities are similar to teacher-assigned practice tasks (Ericsson, 2006, 2014). Teacher-assigned individualized practice (deliberate practice) thus differs from other forms of practice by containing a large proportion of learning activities meeting the criteria for deliberate practice (Ericsson et al., 1993). Other practice activities, such as studying alone in Page 4 of 8 Page 5 of 8 chess, is predicted to consist of some unknown amount of time spent in deliberate practice. The duration of deliberate practice may be correlated with the total duration of practice alone with a correlation ranging from 0.0 to almost 1.0 depending on age and skill level of performer and the particular domain of expertise. However, until studies have successfully measured these correlations it is not possible to estimate the proportion of deliberate practice from estimates of practice alone. The remaining studies do not support Macnemara et al.’s (2014, p. 1615) rejection of r Fo Ericsson and Moxley’s (2012, p. 145, italics added) statement that “the concept of deliberate practice can account for the large individual differences between experts and novices”. Only one remaining study analyzed both novices and experts, namely Ericsson et al.’s (1993) Study 2 (see Re supplementary materials). This study was estimated by Macnemara et al. (2014) to show that vi accumulated deliberate practice accounted for over 80% of the variance. Among the remaining ew studies it is possible to conduct an analysis of novices and experts. Ruthsatz, Detterman, Griscom, and Cirullo (2008) reported correlations and accumulated practice estimates for novices (high-school band members, N=178, M=1062.77, SD=553.03, r=0.34), and for experts On (members of a conservatory orchestra, N=64, M=10055.20, SD=5386.06, r=0.31). When these ly 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 Manuscript under review for Psychological Science two groups are combined in a single categorical ANOVA an uncorrected correlation of r=0.85 (based on values above), which after correction, would be estimated to explain all (100%) of the variance in attained performance. In sum, Macnemara et al.’s (2014) meta-analysis and its unusual method for selecting studies did not succeed in collecting uncontaminated estimates of deliberate practice or correlations with the differences between experts and novices. More generally, I have never claimed that deliberate practice can explain all reliable variance in attained performance (see, Manuscript under review for Psychological Science Ericsson, 2014, for an extended discussion). On the contrary I have acknowledged for decades that height and body size (Ericsson, 1998) cannot be changed by training, yet influence the attainment of elite performance in some domains of expertise. Since then I have searched for limits of training and deliberate practice in attaining expert levels of performance by identifying immutable attributes that invariably constrain healthy individuals from reaching an expert level of performance. Although evidence for such attributes is still limited today (Ericsson, 2014), it is entirely possible that future research will uncover additional immutable attributes, which will be r Fo acknowledged once the new replicated empirical evidence has passed scientific review. Words = 999 ew vi Re ly On 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 Page 6 of 8 Page 7 of 8 References Chase, W. G., & Ericsson, K. A. (1982). Skill and working memory. In G. H. Bower (Ed.), the psychology of learning and motivation, Vol. 16 (pp. 1-58). New York: Academic Press. Ericsson, K. A. (1990). Peak performance and age: An examination of peak performance in sports. P. B. Baltes and M. M. Baltes (Eds.), Successful aging: Perspectives from the behavioral sciences, pp. 164-195. New York: Cambridge University Press. Ericsson, K. A. (2006). The influence of experience and deliberate practice on the development r Fo of superior expert performance. In K. A. Ericsson, N. Charness, P. Feltovich, and R. R. Hoffman (Eds.). Cambridge handbook of expertise and expert performance (pp. 685706). Cambridge, UK: Cambridge University Press. Re Ericsson, K. A. (2013). Exceptional memory and expert performance: From Simon and Chase’s vi theory of expertise to skilled memory and beyond. In J. Staszewski (Ed.), Expertise and ew skills acquisition (pp. 201-228). Abington, Oxon, UK: Taylor & Francis. Ericsson, K. A. (2014). Why expert performance is special and cannot be extrapolated from studies of performance in the general population: A response to criticisms. Intelligence, 45, 81-103. doi: 10.1016/j.intell 2013.12.001 ly On 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 Manuscript under review for Psychological Science Ericsson, K. A., Krampe, R. T., & Tesch-Römer, C. (1993). The role of deliberate practice in the acquisition of expert performance. Psychological Review, 100, 363–406. doi:10.1037/0033-295X.100.3.363 Ericsson, K. A., & Lehmann, A. C. (1996). Expert and exceptional performance: Evidence of maximal adaptations to task constraints. Annual Review of Psychology, 47. 273-305 Manuscript under review for Psychological Science Ericsson, K. A., & Moxley, J. H. (2012). A critique of Howard's argument for innate limits of chess performance or why we need an account based on acquired skill and deliberate practice. Applied Cognitive Psychology, 26, 649–653. http://dx.doi.org/10.1002/acp.2841 Macnamara, B. N., Hambrick, D. Z., & Oswald, F. L. (2014). Deliberate practice and performance in music, games, sports, education, and professions: A meta-analysis. Psychological Science, 25, 1608-1618 doi: 10.1177/0956797614535810 Plant, E. A., Ericsson, K. A., Hill, L., & Asberg, K. (2005). Why study time does not predict r Fo grade point average across college students: Implications of deliberate practice for academic performance. Contemporary Educational Psychology, 30, 96–116. doi:10.1016/j.cedpsych.2004.06.001 Re Ruthsatz, J., Detterman, D., Griscom, W. S., & Cirullo, B. A. (2008). Becoming an expert in the vi musical domain: It takes more than just practice. Intelligence, 36, 330–338. doi:10.1016/j.intell.2007.08.003 ew ly On 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 Page 8 of 8
© Copyright 2024