Manuscript under review for Psychological Science
Challenges for the Estimation of an Upper-Bound on
Relations between Accumulated Deliberate Practice and the
Associated Performance of Novices and Experts:
Comments on Macnemara, Hambrick, and Oswald’s (2014)
Published Meta Analysis
r
Fo
Journal:
Manuscript ID:
Date Submitted by the Author:
Complete List of Authors:
Commentary
n/a
Ericsson, Karl; Florida State University, Psychology
vi
Keywords:
Draft
Re
Manuscript Type:
Psychological Science
Learning, Performance
ew
ly
On
Page 1 of 8
Challenges for the Estimation of an Upper-Bound on Relations between Accumulated Deliberate
Practice and the Associated Performance of Novices and Experts:
Comments on Macnemara, Hambrick, and Oswald’s (2014) Published Meta Analysis
r
Fo
K. Anders Ericsson
Re
Send correspondence about this commentary to:
vi
Department of Psychology
Florida State University
Tallahassee, Florida 32306-4301
ew
e-mail: [email protected]
On
Phone(Office): 850-644-9860
FAX: 850-644-7739
ly
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
Manuscript under review for Psychological Science
Manuscript under review for Psychological Science
Abstract
Ericsson, Krampe, and Tesch- Römer, (1993) reviewed evidence on conditions for
optimal learning and established criteria for deliberate practice, namely practice with effective
training tasks (selected by a supervising teacher) with clear performance goals directed to
mastery in the domain. In contrast, Macnemara, Hambrick, and Oswald’s (2014) collected
studies that “referred to at least one publication on deliberate practice by Ericsson and his
colleagues” (p. 1610, italics added). This selection criterion led to the inclusion of studies that
r
Fo
did not even mention “deliberate practice” and most included studies violated our criteria for
deliberate practice. Among the effect sizes that met the criteria for deliberate practice only one of
them included a study that included both novices and experts in the domain of expertise. More
Re
fruitful directions are proposed for research on the limits of deliberate practice to improve
performance in domains of expertise.
ew
Words=143
vi
ly
On
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
Page 2 of 8
Page 3 of 8
In our original paper Ericsson, Krampe, and Tesch- Römer, (1993) reviewed evidence on
“conditions for optimal learning and improvement of performance” (p. 367) and cited Chase and
Ericsson’s (1982, see also Ericsson, 2013) research demonstrating that a college student was
able, with individual practice and immediate feedback, to improve his memory span from 7 to 82
digits (corresponding to an effect size exceeding d=50). Ericsson et al., (1993) found that
effective practice activities with immediate feedback that led to desired objective performance in
real-world domains of expertise. They identified some domains, however, where students are
r
Fo
given individualized instruction and “the teacher designs practice activities that the individual
can engage in between meetings with the teacher” (p. 368). They distinguished this type of
practice by giving it its own name: Ericsson et al (1993) used the term “deliberate practice for
Re
the individualized training activities specially designed by a coach or teacher to improve specific
vi
aspects of an individual's performance through repetition and successive refinement.” (Ericsson
& Lehmann, 1996, pp. 278-279).
ew
Macnemara, Hambrick, and Oswald’s (2014) meta-analysis took a different approach and
avoided the need to specify how studies actually measured deliberate practice. They used an
On
unusual criterion for inclusion of studies, namely that “the study report referred to at least one
ly
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
Manuscript under review for Psychological Science
publication on deliberate practice by Ericsson and his colleagues” (p.1610) supplemented with a
subjective judgment of relevance.
Their criterion led to inclusion of studies that did not even mention “deliberate practice”.
Studies of education included 45 (88%) effect sizes that came from studies that never used the
term “deliberate practice” anywhere in the text of their articles (see supplementary materials).
Their only mention of deliberate practice was in the title of Plant, Ericsson, Hill and Asberg’s
(2005) article “Why study time does not predict grade point average: Implications of deliberate
Manuscript under review for Psychological Science
practice for academic performance” (GPA) (p. 96). This article did not study deliberate practice,
but instead examined study behavior in college “in light of characteristics of deliberate practice”
and found “important similarities as well as differences” (p. 114).
The majority of the remaining studies in the meta-analysis used operational definitions of
practice that violate our original definition of deliberate practice. Many studies measured
practice where teachers did not evaluate the individuals’ performance nor assigned their trainees
effective individual practice activities with performance goals. Even in studies where teachers
r
Fo
and coaches are typically present during team and group practice, the associated practice time is
not primarily spent on individualized practice activities that would be most beneficial for each
individual trainee. Finally several included studies tested performance for a different skill than
Re
had been the assigned target of the deliberate practice. Instead of measuring the ability to
vi
perform music pieces that had been extensively rehearsed, investigators tested performance for
ew
playing scales or playing unrehearsed music (sight reading music). Applying all of these criteria
led to the rejection of 137 (87%) of all 157 included effect sizes (see supplementary materials for
details).
On
These alternative forms of practice are not excluded because they are ineffective, but
ly
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
because they do not measure uncontaminated deliberate practice. One potentially confusing issue
concerns findings that expert performers in some domains, such as chess and SCRABBLE, have
been found to be able to design effective practice activities for themselves while they practice
alone and that these practice activities are similar to teacher-assigned practice tasks (Ericsson,
2006, 2014). Teacher-assigned individualized practice (deliberate practice) thus differs from
other forms of practice by containing a large proportion of learning activities meeting the criteria
for deliberate practice (Ericsson et al., 1993). Other practice activities, such as studying alone in
Page 4 of 8
Page 5 of 8
chess, is predicted to consist of some unknown amount of time spent in deliberate practice. The
duration of deliberate practice may be correlated with the total duration of practice alone with a
correlation ranging from 0.0 to almost 1.0 depending on age and skill level of performer and the
particular domain of expertise. However, until studies have successfully measured these
correlations it is not possible to estimate the proportion of deliberate practice from estimates of
practice alone.
The remaining studies do not support Macnemara et al.’s (2014, p. 1615) rejection of
r
Fo
Ericsson and Moxley’s (2012, p. 145, italics added) statement that “the concept of deliberate
practice can account for the large individual differences between experts and novices”. Only one
remaining study analyzed both novices and experts, namely Ericsson et al.’s (1993) Study 2 (see
Re
supplementary materials). This study was estimated by Macnemara et al. (2014) to show that
vi
accumulated deliberate practice accounted for over 80% of the variance. Among the remaining
ew
studies it is possible to conduct an analysis of novices and experts. Ruthsatz, Detterman,
Griscom, and Cirullo (2008) reported correlations and accumulated practice estimates for
novices (high-school band members, N=178, M=1062.77, SD=553.03, r=0.34), and for experts
On
(members of a conservatory orchestra, N=64, M=10055.20, SD=5386.06, r=0.31). When these
ly
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
Manuscript under review for Psychological Science
two groups are combined in a single categorical ANOVA an uncorrected correlation of r=0.85
(based on values above), which after correction, would be estimated to explain all (100%) of the
variance in attained performance.
In sum, Macnemara et al.’s (2014) meta-analysis and its unusual method for selecting
studies did not succeed in collecting uncontaminated estimates of deliberate practice or
correlations with the differences between experts and novices. More generally, I have never
claimed that deliberate practice can explain all reliable variance in attained performance (see,
Manuscript under review for Psychological Science
Ericsson, 2014, for an extended discussion). On the contrary I have acknowledged for decades
that height and body size (Ericsson, 1998) cannot be changed by training, yet influence the
attainment of elite performance in some domains of expertise. Since then I have searched for
limits of training and deliberate practice in attaining expert levels of performance by identifying
immutable attributes that invariably constrain healthy individuals from reaching an expert level
of performance. Although evidence for such attributes is still limited today (Ericsson, 2014), it is
entirely possible that future research will uncover additional immutable attributes, which will be
r
Fo
acknowledged once the new replicated empirical evidence has passed scientific review.
Words = 999
ew
vi
Re
ly
On
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
Page 6 of 8
Page 7 of 8
References
Chase, W. G., & Ericsson, K. A. (1982). Skill and working memory. In G. H. Bower (Ed.), the
psychology of learning and motivation, Vol. 16 (pp. 1-58). New York: Academic Press.
Ericsson, K. A. (1990). Peak performance and age: An examination of peak performance in
sports. P. B. Baltes and M. M. Baltes (Eds.), Successful aging: Perspectives from the
behavioral sciences, pp. 164-195. New York: Cambridge University Press.
Ericsson, K. A. (2006). The influence of experience and deliberate practice on the development
r
Fo
of superior expert performance. In K. A. Ericsson, N. Charness, P. Feltovich, and R. R.
Hoffman (Eds.). Cambridge handbook of expertise and expert performance (pp. 685706). Cambridge, UK: Cambridge University Press.
Re
Ericsson, K. A. (2013). Exceptional memory and expert performance: From Simon and Chase’s
vi
theory of expertise to skilled memory and beyond. In J. Staszewski (Ed.), Expertise and
ew
skills acquisition (pp. 201-228). Abington, Oxon, UK: Taylor & Francis.
Ericsson, K. A. (2014). Why expert performance is special and cannot be extrapolated from
studies of performance in the general population: A response to criticisms. Intelligence,
45, 81-103. doi: 10.1016/j.intell 2013.12.001
ly
On
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
Manuscript under review for Psychological Science
Ericsson, K. A., Krampe, R. T., & Tesch-Römer, C. (1993). The role of deliberate practice in the
acquisition of expert performance. Psychological Review, 100, 363–406.
doi:10.1037/0033-295X.100.3.363
Ericsson, K. A., & Lehmann, A. C. (1996). Expert and exceptional performance: Evidence of
maximal adaptations to task constraints. Annual Review of Psychology, 47. 273-305
Manuscript under review for Psychological Science
Ericsson, K. A., & Moxley, J. H. (2012). A critique of Howard's argument for innate limits of
chess performance or why we need an account based on acquired skill and deliberate
practice. Applied Cognitive Psychology, 26, 649–653. http://dx.doi.org/10.1002/acp.2841
Macnamara, B. N., Hambrick, D. Z., & Oswald, F. L. (2014). Deliberate practice and
performance in music, games, sports, education, and professions: A meta-analysis.
Psychological Science, 25, 1608-1618 doi: 10.1177/0956797614535810
Plant, E. A., Ericsson, K. A., Hill, L., & Asberg, K. (2005). Why study time does not predict
r
Fo
grade point average across college students: Implications of deliberate practice for
academic performance. Contemporary Educational Psychology, 30, 96–116.
doi:10.1016/j.cedpsych.2004.06.001
Re
Ruthsatz, J., Detterman, D., Griscom, W. S., & Cirullo, B. A. (2008). Becoming an expert in the
vi
musical domain: It takes more than just practice. Intelligence, 36, 330–338.
doi:10.1016/j.intell.2007.08.003
ew
ly
On
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
Page 8 of 8