1. health and fitness
2. disorders

Psychiarry Research, 46:175-199
Elsevier
175
Factors of the Wisconsin Card Sorting Test as Measures
of Frontal-Lobe Function in Schizophrenia and in
Chronic Alcoholism
Edith V. Sullivan, Daniel H. Mathalon, Robert 6. Zipursky,
Zoe Kersteen-Tucker,
Robert T. Knight, and Adolf Pfefferbaum
Received June 3, 1991; revised version received April 17, 1992; accepted June 13, 1992.
Abstract. The purpose of this study was to examine the factor structure of the
Wisconsin Card Sorting Test (WCST). The scores of 22 patients with schizophrenia, 20 patients with chronic alcoholism, and 16 normal control subjects were
entered into a principal components
analysis, which yielded three factors:
Perseveration,
Inefficient
Sorting, and Nonperseverative
Errors. WCST performance
of seven patients with lesions invading the dorsolateral
prefrontal
cortex,
available
from another
study, provided
criterion
validity for the
Perseveration factor and, less strongly, for the Inefficient Sorting factor. Two
patterns of performance characterized the three patient groups: the schizophrenic
group and frontal lobe group had the highest Perseveration factor scores, whereas
the alcoholic
group
had
the highest
Inefficient
Sorting
scores;
the
Nonperseverative
Errors factor showed no significant group differences. Construct validity of these factors involved assessing, in all but the frontal group, the
degree of overlap (convergent validity) and separation (discriminant
validity) of
each WCST factor with scores from tests of other cognitive functions. The convergent and discriminant
validity of the Perseveration
factor, but not the remaining
two factors, received support only within the group of schizophrenic
patients.
Key Words. Dorsolateral
ponents analysis.
prefrontal
cortex,
neuropsychology,
principal
com-
Classical studies of brain-behavior
relationships
pose questions about impaired and
spared cognitive functions in patients with known brain lesions. Clinical assessment,
by contrast,
typically
uses neuropsychological
tests to document
patterns
of
functional
sparing and loss in an effort to determine the presence and locus of brain
Edith V. Sullivan, Ph.D., is Health Science Specialist,
Psychiatry
Service, Palo Alto Department
of
Veterans
Affairs Medical Center, and Senior Research
Associate,
Department
of Psychiatry
and
Behavioral
Sciences, Stanford University School of Medicine. Daniel H. Mathalon,
Ph.D., is a medical
student, Stanford
University School of Medicine. Robert B. Zipursky,
M.D., was Staff Psychiatrist,
Psychiatry
Service, Palo Alto Department
of Veterans Affairs Medical-Center,
and Assistant Professor,
Department
of Psvchiatrv
and Behavioral
Sciences. Stanford
University
School of Medicine. Zoe
Kersteen-Tucker,
Ph.D., is a postdoctoral
associate; School of Education,
University of California,
Berkeley, and Department
of Neurology,
University of California,
Davis. Robert T. Knight, M.D., is
Professor of Neurolcgy,
University of California, Davis. Adolf Pfefferbaum,
M.D., is Chief of Psychiatry
Research, Psychiatry Service, Palo Alto Department
of Veterans Affairs Medical Center, and Professor,
Department
of Psychiatry
and Behavioral
Sciences, Stanford
University
School of Medicine.
Dr.
Zipursky is now at the Clarke Institute of Psychiatry in Toronto. (Reprint requests to Dr. E.V. Sullivan,
Psychiatry Service, 116A3, DVA Medical Center, 3801 Miranda Ave., Palo Alto, CA 94304, USA.)
0165-1781/93/$06.00
@ 1993 Elsevier Scientific
Publishers
Ireland
Ltd.
176
pathology.
Criterion
validity
is usually inferred from the sensitivity
of a neuropsychological
test to selective functional
impairments
in patients with known focal
lesions. Patients with focal lesions in a common
location, forming the criterion
group, are likely to have deficient test performance
for a common reason-their
shared pathology.
Deficient performance
on the same test by patients without
demonstrable
brain lesions, however, may be attributable
to the dyshnction
of brain
regions other than or in addition to the region implicated
in the criterion group.
Thus, it cannot be assumed that the criterion validity of a neuropsychological
test
will generalize to populations
of patients with unknown or diffuse brain pathology.
Outside of the original criterion population,
the use of a neuropsychological
test
for the assessment of a specific brain function (or dysfunction)
requires additional
evidence of validity in the particular
population
under study. The difficulty in
establishing
this additional validity is that no single validity criterion is available in
patients with unknown or diffuse brain pathology (see also Bornstein,
1986; Cur et
al., 19900, 1990h; Heinrichs,
1990; Matarazzo.
1990; Yeo et al., 1990). Given this
criterion problem, the only recourse is to demonstrate
construct \wlidit~~ (Cronbach
and Meehl. 1955) by showing
patterns
of convergence
or correlaiions
among
neuropsychological
tests that have established sensitivity to focal lesions (i.e., tests
for which similar criterion validity has been demonstrated).
We present here an
example of an approach that addresses both criterion and construct validity of a
commonly used neuropsychological
test, the Wisconsin Card Sorting Test (WCST),
in psychiatric
diseases in which the underlying
pathology
is global or poorly
localired
in this case, schizophrenia
(Zipursky et al., 1992) and chronic alcoholism
(Pfefferbaum
et al.. 1992).
Perhaps the earliest evidence for the dorsolateral
prefrontal
cortical (I)l.Pl-(‘)
specificitv of the WCST comes from Milner’s (1963) description of patients who had
undergone cortical excisions of epileptogenic
foci. Patients with frontal-lobe
lesions
invading the Dt.PFC were impaired in the number of categories sorted and committed a large number of perseverativc errors as compared with patients with trontallobe lesions sparing the DLPFC
and patients with temporal-lobe
excisions.
An
observation
of particular importance
and relevance to psychiatric patients, who do
not typically have large focal cortical abnormalities.
is Milner’s observation
that the
WCST performance
of patients whose epileptogenic
focus included the DLPFC was
impaired even before surgery, suggesting that such physiological
derangement
is
sufficient to impair function.
Schizophrenia
and chronic alcoholism are characterized
by global cortical pathology, involving (among other structures) the frontal lobes (Weinberger
et al., 1986;
Pakkenberg,
1987; Ron, 1987; Freund and Ballinger, 1988; Shimamura
et al.. 1988:
Harper and Kril, 1990; Lishman, 1990; Zipursky et al., 1992; Pfefferbaum et al., 1992).
The WCST is widely used in psychiatric studies as a quantitative
measure of function
of the frontal lobes, in particular,
the DLPFC (e.g., in schizophrenia:
Kolb and
Whishaw, 1983; Weinberger et al., 1986, 1988; Goldberg et al., 1987; Berman et al.,
1988; Yates et al., 1990; Braff et al.. 1991; e.g., in chronic alcoholism:
Tarter, 1973;
Malmo, 1974; for reviews, see Heaton, 1981; Ryan and Butters, 1983; Parsons et al.,
1987). Its construct validity in nonlesioned groups, however, is not yet fully established.
177
Furthermore,
while it is tempting to attribute poor performance
on the WCST by
schizophrenic
and alcoholic subjects to frontal-lobe
dysfunction,
a clear attribution
is obscured for at least two reasons: (1) Several studies of patients with circumscribed
lesions do not fully support the frontal specificity of the WCST (Eslinger and
Damasio, 1985; Heck and Bryer, 1986; Kersteen, 1989; Anderson et al., 1991). (2) The
WCST is a complex problem-solving
task that probably requires multiple cognitive
processes rather than a single function (e.g., Anderson
et al., 1991; Dehaene and
Changeux,
1991). An expanded scoring system developed by Heaton (198 I), referred
to here as “conventional
scores,” offers an opportunity
to differentiate
these
processes. Although
a high degree of intercorrelation
exists among the WCST
scores, the possibility remains that the scoring methods do not simply measure a
unitary function.
Rather, several dimensions
could underlie the various WCST
scores, not all of which would necessarily reflect the cognitive function associated
with DLPFC integrity. The multiple measures of the WCST (Heaton,
1981) have
never been submitted to factor analysis, which could reduce redundancies
of the
many
scoring
systems
and, further,
suggest dissociable
cognitive
processes
contributing
to performance.
Accordingly,
the purpose of this study was three-fold:
(1) to identify multiple
dimensions
of WCST performance
by principal components
analysis; (2) to examine
the criterion
validity of the resulting factors as reflected in their sensitivity
to
impairment
in patients with lesions invading the DLPFC;
and (3) to assess the
construct validity of each WCST factor against other measures of frontal-lobe
and
nonfrontal-lobe
functions in two psychiatric populations.
In connection
with construct
validity,
we used bivariate
and multivariate
approaches to examine both the convergent and discriminant
validity of the WCST
factors in schizophrenia
and alcoholism.
To this end, we used tests previously
demonstrated
to be sensitive to DLPFC function
(i.e., tests with demonstrated
criterion validity), other than the WCST, to seek evidence supporting the convergent
validity of the WCST factors as potentially valid measures of the executive function
of the frontal lobes. Although Goldberg et al. (1988) used four tests of frontal-lobe
function (WCST, Category Test, Trail Making B, and verbal fluency) in an attempt
to characterize the relationship
among them and to determine which was the best to
detect impairment
in schizophrenia,
only one test (WCST) was known from lesion
studies to be sensitive specifically to DLPFC function. The discriminant
validity, or
specificity,
of the WCST factors as measures of executive function
would be
supported
by showing that WCST performance
was not associated with tests of
nonexecutive
function. Thus, we used memory tests known to be dependent
upon
medial temporal-lobe
integrity and dissociable from frontal-lobe
integrity (Mimer,
1958, 1971; Milner et al., 1985, 1991).
The tests used here (i.e., self-ordered
pointing and temporal ordering tests) to
assess the construct validity of the WCST factors have been shown in studies of
patients with circumscribed
lesions to be specific in double dissociation
paradigms.
In particular,
studies by Petrides and Milner (1982; Milner et al., 1985) have
shown that patients with frontal-lobe
lesions (mostly chronic epileptic patients
treated with resection) were selectively impaired on self-ordered pointing tests (but
178
see Kersteen, 1989), but not on tests of fact memory. By contrast, patients with large
excisions of the temporal lobes could show impairment
on self-ordering
tests, but
patients with small excisions of the temporal lobes were not impaired; moreover,
both temporal-lobe
groups showed significant impairment
on tests of fact memory.
Clearly, if the alcoholic and the schizophrenic
patients of the present study have
medial temporal-lobe
pathology at all, it is more likely to fall in the category of small
rather than large lesions. Thus, it is unlikely that the contribution
of the potential
medial temporal-lobe
pathology
in producing
an ordering deficit would be significant. Furthermore,
an unqualified
double dissociation
has been shown for the
temporal ordering tests, whereby patients with frontal-lobe
lesions are impaired in
making temporal-order
judgments but not,in item recognition,
whereas patients with
temporal-lobe
lesions invading the hippocampal
complex show the opposite pattern
of sparing and loss (Mimer, 1971; Milner et al., 1991).
The selection of tests for demonstrations
of construct
validity is necessarily
imperfect.
No single study can provide the definitive
answer to the question of
construct
validity. Rather, the test relationships
reported in this article can only
contribute
to the accumulating
“nomological
network”
of associations
and
dissociations
on which the ultimate answer to the construct validity question must
rest (after Cronbach and Meehl, 1955).
Methods
Subjects. All schizophrenic
patients, alcoholic patients, and normal control subjects were
right-handed
male veterans of the United States armed forces. The schizophrenicpatients
met
DSM-III-R
criteria for the diagnosis of schizophrenia
(American
Psychiatric
Association,
1987) and were recruited from the inpatient psychiatric service of the Palo Alto Department
of
Veterans Affairs Medical Center (PADVAMC)
(Table 1). All but one of the subjects were
inpatient{ on the unlocked voluntary ward of the Mental Health Clinical Research Center
(MHCRC);
the remaining patient was housed on a locked ward at the time of the study but
had previously been a patient on the MHCRC ward. Informed consent for participation
in
this study was obtained for all subjects. Patients meeting criteria for current DSM-III-R
alcohol abuse or who had ever met criteria for DSM-III-R
alcohol dependence were excluded
from the study. Other reasons for exclusion were a history of significant medical illness, head
injury resulting in loss of consciousness
for more than 30 minutes, or current DSM-III-R
diagnosis of drug dependence. DSM-III-R
diagnoses were determined in clinical interview by
a psychiatrist,
clinical psychologist,
or psychiatric research fellow. The diagnoses for all but
one of the subjects were reached by consensus of two of the above clinicians; the remaining
patient was diagnosed using an earlier system for establishing
diagnoses at the MHCRC in
which the DSM-III-R
diagnosis was determined
by a single clinician. All schizophrenic
patients met DSM-III-R
criteria for chronic schizophrenia
and were subtyped as follows:
paranoid (n = 5), disorganized
(n = 4) undifferentiated
(n = IO), and residual (n = 3). In
all patients
were diagnosed
by Research
addition
to receiving DSM-III-R
diagnoses,
Diagnostic Criteria (RDC; Spitzer et al., 1975). The Schedule for Affective Disorders and
Schizophrenia-Lifetime
Version (SADS-L; Endicott and Spitzer, 1978) was administered
to
each patient by a trained research assistant or research nurse. RDC diagnoses were established
by a process%consensus
among the same clinicians involved in the DSM-III-R diagnosis along
with the SADS-L interviewer. Of the 22 patients inthis study, 19 met RDC for chronic schizophrenia of the following subtypes: paranoid (‘n = IO), disorganized
(n = 4) undifferentiated
(n = 2) and residual (n = 3). The three remaining patients met RDC for schizoaffective
disorder with the following subtypes: chronic, depressed type, mainly schizophrenic (n = 1); chronic, depressed type, other (n = I); and subchronic, manic type, mainly schizophrenic
(n = 1).
(n = 20)
(n = 22)
SD
Mean
SD
Mean
SD
Mean
9.4
61.92
5.1
37.9
5.6
34.4
4.1
14.1
1.8
12.02
2.2
1 2.62
2.1
15.3
Education
WI
-
4.0
18.8
5.6
18.7
5.5
21.9
Handedness
score’
3.7
3.73
7.7
1 8.33
6.7
12.03
-
Disease
duration
(yr)
’
-
7.9
106.0
9.1
106.7
7.5
107.4
NART
IQ
1. Based on a quantitative measure of handedness (Crovit! and Zener, 1962). Right-handers score between 14 and 31.
2. Age was significantly different from the control group, and education was significantly different from the remaining groups (analysis of variance, p <’ 0.001).
3. All comparisons were significant:frontal < schizophrenic < alcoholic (analysis of variance, p < 0.0001).
4. ” = 4.
Note. NART = National Adult Reading Test.
Frontal lobe (n = 7)
Alcoholic
Schizophrenic
SD
6.9
36.7
Normal control (n = 16)
Mean
Group
characteristics
Age
WI
Table 1. Demographic
4.8
10.0’
2.0
9.8
2.3
9.8
3.1
11.0
Vocabulary
sc+ll
score
180
The schizophrenic
patients underwent cognitive testing when they were considered by their
treatment
team to have likely achieved the maximum benefit from their medications.
All
schizophrenic
patients were receiving treatment with standard antipsychotic
medications with
the following three exceptions: (1) three patients were participating
in a randomized
doubleblind study of a new antipsychotic,
raclopride (Astra); (2) one patient was participating
in an
open label study of the novel antipsychotic
drug, (X-943 (Parke-Davis);
and (3) one patient
reported
that he had not taken any medication
given to him during a part of his
hospitalization,
including the time of his participation
in this study. In addition, the patients
were on a broad range of additional
psychopharmacologic
medications
during testing,
including anticholinergics,
benzodiazepines,
lithium, and tricyclic antidepressants,
as well as
chloral hydrate, propranolol,
amantadine,
hydroxzine, and buspirone.
The alcoholic group included 20 patients who were enrolled in an inpatient alcoholic
rehabilitation
program at the PADVAMC;
all had been abstinent for 3 to 4 weeks before
testing. The alcoholic patients. all right-handed
men, were recruited for this study as follows:
Consecutive
admissions
to the Alcohol Rehabilitation
Program at the PADVAMC
were
reviewed for eligibility. Patients entered into this study met RDC for alcoholism, had no past
history of hospitalization
for DSM-Ill-R
schizophrenia
or major affective disorders, and had
no history of medical or neurological
illness or trauma (e.g., loss of consciousness
> 30
minutes) that would have affected the central nervous system. Subjects who met RDC for
substance abuse other than alcohol within the past year were excluded. These criteria were
applied as follows: The hospital charts of all new admissions
were reviewed for readily
determinable
exclusion factors. Then, patients who passed the initial chart review were invited
to participate in the study and, if they consented, underwent additional screening. Psychiatric
screening involved an interview by a psychiatrist or a postdoctoral
fellow in psychiatry who
used the SADS-L to ensure the absence of current or past psychiatric and nonalcoholic
drug
use problems,
and to confirm the RDC diagnosis. Screening also included a medical and
psychiatric history. physical examination,
and a panel of blood tests (complete blood count,
SMA-20) administered
to all patients on admission.
The rzormal c,ontro/ group comprised
I6 community
volunteers.
recruited by newspaper
and word of mouth.
Other than the primary
diagnoses
of
advertisement,
posters,
schizophrenia
or chronic alcohol abuse, the control subjects met the same inclusion and
exclusion criteria as the schizophrenic
and the alcoholic patients. No schizophrenic
patient or
normal control subject met DSM-III-R
criteria for current substance dependence or for past
or current alcohol dependence.
The alcoholic and the control subjects were drawn tram a larger pool of research subjects to
match the schizophrenic
group. The schizophrenic
and alcoholic groups were matched in
years of formal education and duration of illness. Although the normal control group had
more years of formal education than the two patient groups (p < O.OOl), all three groups
performed
similarly on the National
Adult Reading
Test, an estimate
of premorbid
intelligence (Nelson, 1982) and on the Vocabulary subtest of the Wechsler Adult Intelligence
Scale-Revised
(Wechsler, 1981) an estimate of current intelligence (Table I). As discussed by
Zipursky et al. (1992). matching groups on the basis of premorbid intelligence may be more
meaningful than matching on morbid intelligence or years of education. Group differences in
years of education
may result from premature
dropout
from school due to prodromal
symptoms of the psychiatric disorder (cf. Meehl, 1971) and, in turn, could influence estimates
of current intelligence based on the Wechsler scales. which are highly correlated with years of
formal education.
The patients with frontal-lobe
lesions had been tested by one author, and their WCST data
were presented previously in a study of cortical contribution
to repetition priming (Kersteen,
1989; Kersteen-Tucker
and Knight, 1989). Their characteristics
are summarized
here. This
group comprised seven right-handed
men, aged 43 to 73 years (Table I). These patients were
selected from and tested at the Neurology Service of the Martinez Department
of Veterans
Affairs Medical Center on the basis of having unilateral focal lesions invading the dorsolateral
prefrontal cortex on the left or right side as determined
by lesion reconstruction
with com-
181
puted tomography
or magnetic resonance imaging (Frey et al., 1987; Kersteen, 1989). The
average lesion volume was estimated to be 40.7 cc (SD = 13.1, range = 25.9-59.5). Six men
had experienced unilateral cerebral vascular accidents (4 left-sided and 2 right-sided) and one
a coloid cyst of the third ventricle, which was resected via the dorsolateral frontal cortex; these
events or surgery had occurred 1 to 11 years before testing. Four patients had an expressive
aphasia, but none was demented as assessed by the Mini-Mental Status Examination (Folstein
et al., 1975) (mean = 27.0, SD = 1.3, range = 25-29).
Procedure. Four measures of executive function (including the WCST) and four of memory
function were administered
to the schizophrenic,
alcoholic, and control subjects. For the
patients with frontal-lobe lesions, only WCST scores were used in this analysis, and their data
were not used to derive the WCST factors.
Tests of executive function: WCST and ordering. The ordering tests were used to
assess the convergent validity of the WCST factors, derived from the principal components
analysis, as measures of executive function.
In the WCST (Grant and Berg, 1948; Milner, 1963), the subjects sorted 128 cards, each of
which displayed 1 to 4 colored symbols, according to one of three strategies: color, form, or
number. The subject attempted to arrive at the correct sorting strategy, which was controlled
by the examiner but never mentioned explicitly. The correct strategy was switched without
warning after 10 consecutive correct responses. In all cases except for the frontal-lobe
group,
testing continued for 128 cards; thus, the maximum number of categories possible to achieve
was 12. Testing of the frontal-lobe
patients was discontinued
in two cases after six categories
had been achieved, in three cases after most of the cards had been sorted (i.e., at least 110
cards), or in two cases when patients experienced extreme frustration (one patient completed
44 cards and another 64 cards) (Kersteen, 1989). To equate all groups for the number of cards
sorted, the scores of the frontal patients were prorated to 128 cards. The responses were scored
according to Heaton’s criteria (1981) and yielded 14 scores: number of categories achieved,
total errors, total correct responses, number of correct responses excluding those achieved in
successful categories, prior category perseverations,
all perseverative responses, perseverative
errors, nonperseverative
errors, percent perseverative errors, unique responses, trial to the first
category, percent conceptual responses, failure to maintain set, and learning-to-learn.
Of the
14 scores, 11 were used in principal components
analysis. The three excluded scores (prior
category perseverations,
trial to the first category, and learning-to-learn)
were not used because of missing values for subjects who achieved fewer than three categories.
The verbal self-ordering
test (Petrides and Milner, 1982) samples working memory by
requiring subjects to create, organize, and execute a sequence of acts. In the verbal version,
test stimuli consisted of 54 abstract words taken from Paivio et al. (1968). Following the
procedure of Petrides and Milner (1982), all words were rated below 3.2 out of 7 on a scale
where the lower the rating, the less capable a word was at evoking an image. In addition, these
words had medium to high Thorndike-Lorge
(1944) frequency ratings, thereby excluding rare
words, which could be either unknown or salient to subjects. Two tests were administered,
one
consisting of six words and one of 12 words. Three trials of each test were given. The test order
was the same from subject to subject: the six-word test preceded the 12-word test.
The self-ordering test proceeded as follows. For the six-word test, the subject saw a series of
six pages shown one at a time, each displaying the same six words but in different places in a 2
X 3 matrix of squares. The subject’s task was to point to a word on successive pages not
previously selected. Thus, on the first page, the subject was correct regardless of which word
he chose; on the second page, only five words were potentially correct, and so on. For the
12-word test, 12 words and pages were used.
One component
of the nonverbal temporal ordering test assessed ordering function and
another component assessed memory function; the function tested depended upon the type of
question asked, and each component
was scored separately. Subjects were shown a series of
colored wallpaper swatches glued to 5- X I-inch index cards and were instructed to remember
them in their order of appearance. At various intervals throughout the series, subjects received
182
one of two types of two-choice
recognition
tests: recency judgment
tests contained
two
previously
presented
designs and measured
ordering function;
design recognition
tests
contained a previously presented design and a foil, never before seen on the test and measured
memory function (see below). For recency judgments,
testing ordering, patients were asked,
“Which one have you seen more recently?” For design recognition,
testing memory, patients
were asked, “Which one did you see in this test?” In both cases, patients pointed to their
choice. This test was an abbreviated version of the one described by Sullivan and Sagar (1989).
Although
recency judgments
and design-recognition
responses
were made after varying
intervals, only the two total scores were used in this analysis and were expressed as percent
correct of 50 recency-judgment
trials and percent correct of 25 item-recognition
trials.
Tests of memory function. These tests assessed verbal and nonverbal recognition
and
recall. They were used to assess the discriminant
validity of the WCST factors as measures of
executive function.
Recognition of nonverbal material was assessed within the context of the temporal ordering
test. As described above, recognition of previously seen designs (i.e., memory function) was
tested in a two-choice forced-choice
paradigm.
A final and separate part of the verbal self-ordering
test involved an unannounced wordrecognition test, assessing verbal recognition memory. For each of the 54 words used in the
six-word and 12-word self-ordering
tests combined, a three-choice,
forced-choice
recognition
question was posed. The advantage of testing recognition memory in this paradigm was that
the examiner
had some degree of assurance
that all subjects, regardless
of attentional
incapacities, attended to all stimuli because of the multiple pointing opportunities.
In the test of delayed recall of verbal memory, patients were asked to recall the stories of the
Wechsler Memory Scale (WMS; Wechsler and Stone, 1945) 1 hour after immediate recall and
without warning (Milner, 1958; Corkin et al., 1985). The score was the average number of
points achieved for the two stories.
In the test of delayed recall of nonverbal material, patients were instructed to recall the three
drawings of the WMS 1 hour after immediate recall and without warning (Milner, 1958).
After the recall tests, the patients copied the drawings from the cards so that the recall scores
could be corrected for drawing ability (Cooper et al., 1991). The copy and delayed recall scores
were derived according to Wechsler and Stone’s (1945) criteria for immediate recall. The
delayed recall score used in this analysis was the delayed recall score divided by the copy score,
following a similar correction approach used by Jones-Gotman
(1986).
Statistical Analysis. Comparisons
of the frontal-lobe
patients with the control subjects used
t tests. One-way analyses of variance (ANOVAs) compared performance
across the four
groups on the 14 conventional
scores of the WCST, and across the three groups (control
subjects, schizophrenic
patients, and alcoholic patients) on composite scores of the ordering
and the memory tests. Post hoc comparisons
for significant
F values (a = 0.05) were
performed with Scheffi tests (a = 0.05).
To reduce the 11 WCST scores to a smaller set of nonredundant
scores, the WCST data
were subjected to a principal components
analysis with a varimax rotation. The three groups
(control subjects, schizophrenic
patients, and alcoholic patients) were pooled for the principal
components
analysis to identify the major dimensions along which the groups might differ, as
well as to generate factor scores that could be compared across groups. On the basis of the
factor solution, WCST factor scores were computed by summing the standard scores from the
scores with high loadings on the factors. These WCST factor scores were then used in the
subsequent analyses.
Two composite scores were formed from the ordering and the memory test scores. The
ordering composite was a single score derived from the three ordering measures: six-word and
12-word self-ordering
tests and total nonverbal
temporal
ordering
test. The memory
composite was a single score derived from the four memory tests: recognition of verbal and
nonverbal
material and delayed recall of verbal and nonverbal
material. The ordering
composite was formed by standardizing
(i.e., calculating z scores) and then summing the
183
scores from the three ordering tests; the memory composite was formed by standardizing
(z scores) and then summing the scores from the four memory tests. A constant was added to
each composite so as to keep all scores in the positive direction. For both the ordering and the
memory composites, high scores reflected good performance.
The convergent validity and the discriminant validity of the WCST factors were assessed
within each of the diagnostic groups using bivariate and then multivariate approaches. The
bivariate approach examined simple Pearson product-moment
correlations between the
WCST factors and the ordering and memory composite scores. In addition, the regression
equations reflecting convergent validity were directly compared across groups using
hierarchical regression analyses, which included the WCST factors, the ordering composite,
and “dummy-coded” categorical variables representing group membership (Pedhazur, 1982).
The multivariate approach used hierarchical regression analysis (Cohen and Cohen, 1985) to
control for overlap of variance between the ordering and memory composites while examining
their unique relationships with the WCST factors.
Results
WCST Conventional
Scores. Table 2 presents the results of the conventional
scoring of the WCST. The performance
of the frontal-lobe
group was similar to that
reported by others (e.g., Milner, 1963; Heaton, 1981), and this sample therefore
appears to be representative
of other such samples. Of the nine instances showing
nominally
significant
group differences when all four groups were compared,
six
were significant with Bonferroni
correction @ < 0.004). The number of categories
sorted was the only WCST measure on which all three patient groups were
significantly
impaired relative to the control group. The schizophrenic
and frontallobe groups shared further impairment
on the following scores: total errors, total
correct,
all perseverative
responses,
perseverative
errors, percent perseverative
errors, and percent conceptual responses. The alcoholic, but not the schizophrenic
or
frontal lobe groups, had significantly
worse scores than did the control group on
number correct minus IO/category and failure to maintain set. In no comparison
did
the patient groups differ significantly
from each other. Five scores showed no group
differences: prior category perseverations,
nonperseverative
errors, unique responses,
trials to first category, and learning-to-learn.
Principal Components Analysis of the WCST Conventional
Scores. The 11
conventional
scores for the WCST from all 58 subjects in the control, schizophrenic,
and alcoholic groups were entered into a principal
components
analysis. Using
varimax rotation, three factors emerged with Eigenvalues > 1, accounting for 91% of
the original variance; Table 3 presents the factor loadings. In addition, the threefactor solution was readily interpretable.
The first factor, accounting for 58% of the
variance and representing
the major dimension
underlying
most of the indexes of
WCST performance,
consisted of seven high loading variables: all perseverative
responses, total perseverative errors, percent perseverative
errors, total errors, total
correct, percent conceptual
responses,
and number of categories achieved. This
factor seemed to tap WCST errors due to perseverative responses, and we labeled it
Perseveration. The second factor, accounting for 19% of the variance, had its major
contributions
from two high loading variables, failure to maintain
set and total
correct minus 10 per category achieved. These indexes seemed to reflect inefficiency
errors
% Perseverative
responses
3.9
-1.6
0.6
SD
0.3
18.4
SD
Mean
70.4
Mean
SD
0.9
1.0
SD
11.5
0.6
Mean
SD
6.6
Mean
6.2
SD
8.5
-8.6
1.2
1.0
24.3
46.0
5.0
14.0
4.4
2.4
21.6
26.2
17.4
22.7
3.8
19.4
Mean
SD
28.0
32.9
36.6
39.6
8.5
7.9
SD
SD
9.3
10.9
Mean
18.1
22.2
12.3
29.7
24.7
75.7
24.7
51.5
3.3
4.6
10.1
-4.0
1.4
1.5
19.2
53.4
12.5
17.1
3.5
2.5
13.9
17.0
8.5
22.1
16.0
20.6
19.4
23.4
14.7
17.1
20.3
34.2
16.7
85.7
17.9
42.9
3.2
5.2
(n = 20)
(n = 22)
6.3
-6.4
1.2
1.1
24.7
39.9
4.1
15.0
2.2
1.0
19.3
30.3
9.7
20.0
24.7
38.8
34.7
47.6
17.8
21.5
9.5
33.0
24.5
69.1
24.5
56.9
2.9
3.6
1. Scores prorated to 128 cards. On average. the frontal-lobe group sorted 98 cards k 32.7.
2. One-way analysts of vanance.
3. Excluded from the principal components analysis.
l
2.37
2.96’
5.11 **
1.51
1.33
5.90’**
0.29
5.99”’
5.37”
2.28
l
4.01 *
6.49”’
6.02”’
6.32”’
F2
< 0.004. Bonferroni corrections
(n = 7)
Frontal lobe’
Note. Values tn boldface type were stgntfrcantly worse than those for the normal control group (Scheffe F test, a 5 0.05). ‘p < 0.05. “p < 0.01, “‘p
a 5 0.05 for 14 comparisons, p 5 0.004.
Learning-to-learn3
Failure to maintain set
% Conceptual
Trials to 1 st category3
Unique responses
errors
errors
responses
9.1
10.9
13.4
SD
SD
17.4
Mean
Mean
11.4
SD
100.7
11.4
Mean
27.3
SD
2.4
SD
Mean
6.3
Mean
Alcoholic
Schizophrenic
Card Sortina Test
Control
(n= 16)
scores on the Wisconsin
perseverations3
Nonperseverative
Perseverative
All perseverative
Prior category
No. correct-l O/category
Total correct
Total errors
Categories
Table 2. Conventional
g
185
Table 3. Principal components analysis of the WCST conventional scores
Factors:
Variance proportion:
Eigenvalues:
Perseveration
0.58
7.24
Nonperseverative
errors
0.14
1.12
Factor loadings
WCSTscores
Perseverative
Inefficient
sorting
0.19
1.62
0.987’
0.055
0.081
All perseverations
0.983’
0.024
0.061
Perseverations
0.977’
0.102
0.069
Total errors
0.927’
0.169
0.312
Total correct
-0.918’
-0.161
0.327
-0.899*
-0.24
0.324
-0.808*
-0.522
0.250
set
-0.049
0.984’
0.027
No. correct-l O/category
0.468
0.848’
0.068
Unique
0.106
0.007
0.839.
0.203
0.027
0.703’
l
errors
% Conceptual
responses
Categories
Failure to maintain
responses
Nonperseverative
errors
Note. Principal components analysis with orthogonal
nonoverlapping loading on the factor.
(varimax) rotation (n = 58). * score with high and relatively
in sorting strategy, and hence this factor was labeled ZneSJicient Sorting. The third
factor, Nonperseverative Errors, accounted
for 14% of the variance and had its
highest loadings from two indexes reflecting WCST errors of a nonperseverative
nature: nonperseverative
errors and unique responses.
On the basis of the results of the factor analysis, factor composites were computed
for each of the three factors by summing the z scores from only those conventional
scores with high loadings on the factor. Each score contributed
to only one factor.
To form the z scores, each conventional
score was standardized
against the mean and
standard deviation for the combined group of control, schizophrenic,
and alcoholic
subjects. Because the frontal-lobe
patients were not included in the factor analysis
and were used to provide evidence of criterion validity of the factors, these means
and standard deviations were also used to standardize the conventional
scores of the
frontal-lobe
patients. High factor composites reflected poor WCST performance.
Criterion Validity of WCST Factors Based on Performance by Patients With
Frontal-Lobe
Lesions. A comparison
of the factor scores of the frontal-lobe
patients with the control subjects provided a test of criterion validity for each WCST
factor derived from the principal components
analysis. Comparisons
with t tests
revealed that the frontal-lobe
group had significantly higher scores on the Perseveration (t = -4.404, df= 21, p = 0.002) and the Inefficient Sorting factors (t = -2.774,
df = 21, p = 0.0114) than did the control group, but that these two groups did not
differ significantly
on the Nonperseverative
Errors factor (t = -0.48, df = 21, NS)
(Fig. 1). Therefore, the performance
of the frontal group supported the Perseveration and Inefficient Sorting factors as measures of frontal-lobe
dysfunction,
and so
provided criterion validity to these factors.
The purpose of the next analysis was to test whether the schizophrenic
and the
alcoholic groups showed a similar pattern of deficits as that observed in the frontal-
186
Fig. 1. Wisconsin Card Sorting Test (WCST) factor scores
3
2Q-
.
15
0
.
.
10.
I!
$5
e
co,
k
e
e
t
II
o
i
;
8
f
COIla
Schizophrenic Alcoholic
Fmntal
c0nuo1
Schizophrenic Alcoholic
Frontal
8
0
8
4*
2
‘2
3,
2,
[
6
z
0
8
0
0
-
_:..
lri
i
-2,
-3-
0
COnoOl
0
0
Schizophrenic
0
Alcoholic
T
:
.
FlUltal
Scatterplots and means of the three WCST factor scores for the four study groups. On these measures, low scores are
good. The Perseveration factw scores of the schizophrenic and frontal-lobe groups were significantly greater than that
of the normal control group (p<O.OOOl),
whereas the Inefficient Sorting score of the alcoholic group was significantly
greater than that of the normal control group (p< 0.01). The Nonperseverative Errors score showed no significant
group differences.
lobe group. To this end, the pattern of performance
based on the three WCST
factors was compared across all four groups with ANOVAs and post hoc Scheffe
tests. The Perseveration
factor scores of the schizophrenic
and frontal groups were
significantly
greater than that of the normal control group (F = 6.22; df= 3, 64;
p < O.OOl),
whereas the Inefficient
Sorting score of the alcoholic group was
significantly
greater than that of the normal control group (F = 4.08; df = 3, 64;
p = 0.01). With the use of a Scheffe post hoc test, which incorporated
a protected
p value by taking into account the number of comparisons
made in an analysis, it
was determined
that the frontal-lobe
group did not differ significantly
from the
187
control group on Inefficient
Sorting; this result conflicts with that obtained with
a simple t test. The Nonperseverative
Errors score showed no group differences
(F = 1.11; df = 3, 64; NS). The correlations
among the WCST factors were high in
the normal control subjects, moderate to low in the schizophrenic
and the alcoholic
patients, and not significant in the frontal patients (Table 4).
Construct Validity of the WCST Factors Based on Convergent and Discriminant Analysis Exclusive of the Patients With Frontal-Lobe Lesions. To assess
the validity of the WCST factors for groups other than the criterion sample of
frontal
patients,
a construct-validation
approach
was taken. Accordingly,
the
three WCST factors were analyzed in parallel in the subsequent analyses to elucidate
the differences in their meaning with respect to the data from the other tests. Only
data from the normal control, schizophrenic,
and alcoholic subjects were considered
in these analyses. Bivariate and multivariate
analytical approaches were used.
Ordering and memory composite scores. Of the total group of 58 subjects, 17
schizophrenic,
19 alcoholic, and 15 normal control subjects had data for all tests; 16
normal control subjects had data for the WCST and ordering tests. The ordering
composite scores of the schizophrenic
and alcoholic groups were inferior to those of
the normal control group, but the differences
were not statistically
significant
(F = 1.49; df = 2, 50; NS) and may reflect low power due to the small number of
subjects. The memory composite score of the schizophrenic
group was significantly
Table 4. Factor and composite
groups
score correlational
Inefficient
sorting
Group
matrix for the four subject
Nonperseverative
errors
Ordering
Memory
Normal control
0.81***
Perseveration
Inefficient
sorting
Nonperseverative
-0.07
0.84”*
0.28
0.59”
0.41
0.18
0.05
-0.28
errors
0.70”
Schizophrenic
Perseveration
Inefficient sorting
Nonperseverative
0.10
0.41*
0.11
errors
-0.70***
0.04
-0.59**
-0.70”’
0.01
-0.75***
O&i’*
Alcoholic
0.52”
Perseveration
Inefficient
sorting
Nonperseverative
Ordering
0.50
0.31
0.27
-0.07
0.29
0.15
errors
Frontal lobe
Perseveration
Inefficient
0.14
-0.18
sorting
‘p < 0.05;“p < 0.01;"'p < 0.001.
0.46’
-0.33
0.12
-
-
0.03
-
-
188
worse than the memory composite scores of the normal control and alcoholic groups
(F= 8.56; df = 2, 50;~ < 0.001) (Fig. 2).
Bivariate Convergent
Validity.
Simple (bivariate)
correlations
among the
WCST factors and ordering composite
scores were calculated within each group
(Table 4). These correlations
bear upon the convergent validity of the WCST factors
as measures of executive function. The convergent validity of a given test is demonstrated to the extent that it is correlated with another measure of the same function
(Campbell and Fiske, 1959; Wiggins, 1980). Thus, convergent validity of the WCST
factors would be supported by negative correlations
between the WCST factors (low
scores reflect good performance)
and the ordering composite (high scores reflect
good performance).
For the Perseveration
and Nonperseverative
Errors factors, the correlations
supported convergent validity in the schizophrenic
subjects only (Table 4, column 3).
In the alcoholic subjects, the correlations
were either not significant or, in the case of
the Perseveration
factor, the correlation
was in the counterintuitive
direction (i.e., a
tendency to perseverate was associated with good ordering performance).
In the
normal control subjects, no correlations
were significant.
These findings strongly suggest that the convergent validity of the WCST factors
as measures of a common cognitive function
depend on diagnostic
status (i.e.,
schizophrenic,
alcoholic, or normal control). fiierarchical
regression analyses were
conducted
to address explicitly
the dependence
of the correlations
upon group
membership.
In predicting each WCST factor, the ordering composite was entered
into a regression analysis first, followed by two dummy-coded
variables representing
group membership.
Finally, the product of the ordering composite and the dummycoded variables was entered in step 3, allowing for a test of differences among the
groups in the slopes of the regression lines. A significant predictive increment at step
3 suggested that the degree and/or direction of the relationship
between the WCST’
factors and the ordering composite interacted with diagnostic status (Table 5).
In predicting the Perseveration
factor from the ordering composite, there was a
strong group interaction
(p < 0.0001) (Table 5). There was a strong and expected
Fig. 2. Ordering and memory composite
Ordering
scores
MelOrJ
Means and standard errors of the mean of the ordenng and memory composite scores. On these measures, high
scores are good. The ordering composite scores of the schizophrenic and alcoholic groups were inferior to those of the
normal control group, but the differences were not statistically significant. The memory composite score of the
schizophrenic group was significantly worse than the memory composite scores of the normal control and alcoholic
groups (p < 0.001).
189
Table 5. Hierarchical regression of Wisconsin Card Sorting Test (WCST)
factors on ordering composite and diagnostic group
I?2
R
chanae
WCST factor
Stet4
Predictor added
Perseveration
Inefficient sorting
Nonperseverative
*p< 0.01:
errors
1
Ordering
2&3
Diagnostic
composite
0.21
0.04
groups’
0.46
4&5
Ordering
X diagnostic
0.17’
0.71
0.28*’
composite
0.10
0.01
group
0.45
0.19*
1
Ordering
2&3
Diagnostic
4&5
Ordering
X diagnostic
1
Ordering
composite
2&3
Diagnostic
4&5
Orderina
group*
group
group
X diaanostic
arouD
0.47
0.02
0.22
0.05
0.30
0.04
0.53
0.19*
‘“p < 0.001.
1. Two dummy-coded variables were used to represent the three diagnostic groups.
2. Two product variables, obtained by multiplying the ordering composite with each dummy variable, were used to
represent group differences in the slopes of the regression line.
relationship
in the schizophrenic
subjects (r = -0.70) a moderate but counterintuitive relationship
in the alcoholic subjects (r = OSO), and a weak counterintuitive
relationship
in the normal control subjects (r = 0.28) (Fig. 3).
When the Nonperseverative
Errors factor was predicted
from the ordering
composite, a significant group interaction
effect emerged (p = 0.004) (Table 5). The
slopes differed among the groups in a pattern similar to that observed with Perseveration: the schizophrenic
subjects showed the expected relationship
(r = -0.60) the
alcoholic subjects exhibited a weak but counterintuitive
relationship
(r = 0.30), and
the normal control subjects showed virtually no relationship
(r = 0.05).
In predicting Inefficient Sorting, no interaction
was found (Table 5). There was,
however, a significant group effect at step 2 of the regression equation, indicating
that the groups showed mean differences in their tendency toward Inefficient Sorting
(see ANOVA results described earlier). At step 1, with all groups combined,
the
Fig. 3. Perseveration factor scores regressed on ordering composite scores
Schizophrenics
Normal Controls
20
f
%
1s
101
Uz
I
=
e
5
m
y.1.S2.371
y=-*.26+0.51x
r=m
.
.
0
.
.
.=I_
j”
-
Alcoholics
20
.
-10’
-6
-4
ordaia#
-2
.
0
cmpaitc
2
scm
l
’ -10’
4
-6
-4
-2
OlJaiqWb=
0
2
4
-4
-4
ordate
-2
0
2
4
CompMilc scare
Convergence of the Perseveration and the ordering composite scores. Only for the schizophrenia
correlation in the expected direction. The slopes differed significantly (p < 0.0001).
group was the
190
ordering composite was not a significant predictor of Inefficient Sorting.
Bivariate discriminant
validity. Simple (bivariate)
correlations
between the
WCST factors and the memory composite bear upon the discriminant
validity of the
WCST factors as measures of executive function. Discriminant
validity is concerned
with demonstrations
of what a test does not measure and is an indication
of a test’s
specificity (Campbell and Fiske, 1959; Wiggins, 1980). Accordingly,
one indication
of the discriminant
validity of the WCST factors is their lack of correlation
with
the presumably
unrelated memory composite. Because convergent validity was not
demonstrated
for the normal control subjects or the alcoholic subjects, discriminant
validity was examined
only for the schizophrenic
subjects. Table 5 presents the
correlations
for the alcoholic and normal control subjects for descriptive purposes
only. The results revealed that Perseveration
and Nonperseverative
Errors were
significantly
correlated
with the memory composite
(Table 5, column 4). These
correlations
represent an apparent failure to support the discriminant
validity of
these two WCST factors in the schizophrenic
subjects.
Multivariate
discriminant validity. The emergence of a correlation
between the
ordering composite and the memory composite complicates the interpretation
of the
bivariate analysis of discriminant
validity (Table 4). This correlation may have arisen
from global cortical pathology present in schizophrenia
(Zipursky et al., 1992). In
other words, ordering and memory may have been correlated
because a single
pathological
process produced impaired functioning
of both the frontal-lobe
and the
medial temporal-lobe
regions in schizophrenia.
The bivariate approach to discriminant validity fails to consider this ordering-memory
correlation.
Therefore, a multivariate approach was undertaken
to examine the discriminant
validity of the WCST
factors while controlling
for the shared variance between the ordering and memory
functions in schizophrenia.
In light of the shared variance between the ordering and memory composites, the
simple correlation
between the WCST factors and the memory composite can be
explained
by at least two alternatives:
Alternative 1. The WCST factors lack
discriminant
validity. In addition to the shared variance between the ordering and
memory composite scores, aspects of memory that are unrelated to ordering contribute to WCST performance.
The left panel of Fig. 4 depicts this pattern
of
overlapping
variance among the measures. Alternative 2. The WCST factors reflect
specific measures of executive function.
The specificity,
however, is masked by
correlated
ordering and memory test performance
in schizophrenia
that produce
overlap between memory and WCST performance.
The right panel of Fig. 4 depicts
this pattern of overlapping variance among the measures.
To distinguish
between these alternatives,
we used hierarchical regression analysis
in which the ordering and memory composite scores were entered as predictors for
each WCST factor. The ordering composite score was entered into the equation first.
Then, the change in R* was examined after the memory composite score was entered
at the second step. A significant R2 change at the second step supports Alternative
1
(i.e., supports lack of discriminant
validity), whereas an insignificant
increment in R2
supports Alternative 2 (i.e., supports discriminant
validity). In effect, the analysis at
the second step examined
the predictive
utility of the components
of memory
performance
that were unrelated to ordering performance.
Thus, at the second step,
191
Fig. 4. Alternative patterns of overlapping variance among the measures
Alternative 1. The Wisconsin Card Sorting Test (WCST) factors lack discriminant validity. In addition to the shared
variance between the ordering and memory composite scores, aspects of memory that are unrelated to ordering
contribute to WCST performance. The left panel depicts this pattern of overlapping variance among the measures.
Alternative 2. The WCSTfactors are specific measures of executive function. The specificity, however, is masked by
correlated ordering and memory test performance in schizophrenia that produces overlap between memory and
WCST performance. The right panel depicts this pattern of overlapping variance among the measures.
the contribution
of memory to WCST performance
could be evaluated
while
ordering was held constant. This analysis of discriminant
validity was conducted
only in instances
where convergent
validity
was supported-namely,
for the
Perseveration
and Nonperseverative
Errors factors in the schizophrenic
subjects.
The results for the Perseveration
factor showed that memory performance
did not
produce a significant
improvement
in the R2 when ordering performance
was
controlled
for (R2 change = 0.10, NS). This result is consistent with Alternative
2.
For the Nonperseverative
Errors factor, memory performance
did contribute
a
significant
increment in prediction
when ordering performance
was controlled
for
( R2 change = 0.23, p < 0.05). This result is consistent with Alternative
1 (Table 6).
Discussion
The results of this study were as follows: (1) the schizophrenic,
alcoholic, and
frontal-lobe
groups all showed impairment
on the conventional
scores of the WCST;
(2) the conventional
WCST scores could be reduced to three factors, Perseveration,
Inefficient
Sorting,
and Nonperseverative
Errors;
(3) of these three factors,
Perseveration
and, to a lesser extent, Inefficient Sorting were sensitive to dysfunction
in the frontal-lobe
group and therefore possessed criterion validity; and (4) the
Perseveration
factor, as an index of executive function,
demonstrated
construct
validity for the schizophrenic
group.
Factors of WCST Scores and Patterns of Impairment.
The 11 conventional
scores of the WCST could be reduced
to three factors,
which we named
Perseveration,
Inefficient
Sorting,
and Nonperseverative
Errors.
This factor
structure closely resembles the sources of difficulty common on this task: “inefficient
initial conceptualization,
perseveration,
failure to maintain
set, and inefficient
learning across the several stages of the test” (Heaton, 1981, p. 7). The emergence of
the three factors suggests that different components
of cognition
contribute
to
WCST performance
and provides an attractive alternative
to the redundancy
of
‘p < 0.05; “p < 0.01
Nonperseverative
Perseveration
WCST Factor
errors
Memory composite
Ordering
Memory
1
2
composite
composite
Ordering composite
1
Predictor added
2
Step
0.76
0.59
0.77
0.70
R
0.23’
0.35
0.10
0.4Y’
R 2 change
-0.59
-0.70
Step 1
-0.63
-0.19
-0.42
-0.43
Step 2
fl at each step
Table 6. Hierarchical regression: Wisconsin Card Sorting Test (WCST) factors on ordering and memory composite
scores for the schizophrenic patients
193
analyzing all scores. Indeed, use of redundant scores could obscure the assessment of
the major dimensions underlying WCST performance. Furthermore, choosing a
subset of the scoring methods without regard to the underlying major dimensions
would be arbitrary. Examining WCST performance in terms of a reduced number of
factor scores may afford several benefits, including increased reliability and a more
manageable number of WCST scores. The results of the present study suggest one
possible factor solution for the conventional scores of the WCST, though this factor
structure requires replication. It would also be important to test the power of this
factor structure in predicting performance patterns in new samples of schizophrenic,
alcoholic, and frontal-lobe patients.
The performance of the frontal-lobe patients suggested that Perseveration and
possibly Inefficient Sorting possess criterion validity for the assessment of
impairment in patients with focal DLPFC lesions. The Perseveration factor result is
in accord with previously published observations that perseveration commonly
accounts for the WCST impairment of patients with frontal-lobe lesions (e.g.,
Milner, 1963; Drewe, 1974; Heaton, 1981). The tendency of schizophrenic patients to
perseverate was also observed by Fey (1951) even before the introduction of
psychoactive medications, indicating that this tendency is not simply a result of
medication. Regarding Inefficient Sorting, alcoholics have been shown to have a
graded impairment in maintaining set, depending upon their length of drinking (i.e.,
< 10 years vs. > 10 years; Tarter, 1973). The patients of the present study would have
been considered long-term alcoholics by Tarter’s criteria and behaved as such.
Apparently, the tendency to make nonperseverative errors does not show sensitivity
to DLPFC pathology.
Construct
Validity
of WCST
Factors as Measures
of Executive
Function.
The validity pattern, based on the frontal-lobe patients, could not be assumed to
hold for both psychiatric groups. Thus, an attempt was made to corroborate the
criterion validity results with evidence of construct validity of the three factors.
Moreover, construct validation was assessed in each group separately since the
factors might be valid measures of executive function in some populations but not
others.
The bivariate correlations of the Perseveration and Nonperseverative Errors
factors with the independent ordering composite (executive function) supported
their convergent validity within the schizophrenic group, but not within the alcoholic
or normal control groups. The Inefficient Sorting factor was not supported as a valid
measure of executive function within any of the three groups considered. When
convergent validity was demonstrated, the bivariate correlations did not support the
discriminant validity of the WCST factors.
To assess the multivariate discriminant validity of the WCST factors as measures
of executive function, we used multivariate analysis to test whether these factors did
nor measure memory function in the schizophrenic group. With this approach,
which controlled for the correlation between the ordering and memory composites,
the memory composite showed a significant association with Nonperseverative
Errors, but not Perseveration. Thus, these multivariate results, along with the
convergent validity results, suggested that (1) the Perseveration factor was a valid
194
and specific measure of executive function
in schizophrenic
subjects but not in
alcoholic or normal control subjects, and (2) the Nonperseverative
Errors factor
reflected both executive and memory function in schizophrenic
subjects, whereas in
alcoholic and normal control subjects it was associated with neither function. The
Inefficient
Sorting
factor was not significantly
associated
with the ordering
composite in any group, and it therefore could not be supported
as a measure of
executive function despite its criterion validity in patients with frontal-lobe
lesions.
In construct validation studies, the failure to show convergence between two tests
that theoretically
measure an underlying
construct
raises questions
about the
construct validity of both tests. While this caveat applies to all of the relationships
evaluated
in the present study, it is particularly
salient in the alcoholic subjects,
where the counterintuitive
relationships
of the Perseveration
and Nonperseverative
Errors factors with the ordering composite emerged: the alcoholic subjects showing
relatively poor scores on the WCST factors were the same individuals scoring well on
the ordering composite. Which of these scores should be taken as the true indication
of DLPFC
function
in alcoholics?
Our results do not permit an answer to this
question; rather, the findings markedly reduce our confidence in both the ordering
composite tests and the WCST factors as valid and selective measures of executive
function in alcoholics. We might speculate from these results that alcoholics perform
these tests qualitatively
differently from schizophrenic
patients and normal control
subjects, perhaps because the pathology
responsible
for the impairment
in the
alcoholics may involve brain regions other than the dorsolateral
frontal convexity
and its connections;
one candidate region is the orbital frontal cortex (cf. Freedman
and Oscar-Berman,
1986).
The problem of showing that a test possesses discriminant
validity-that
is, that a
test is a measure of a specific function-is
complicated
in disorders that may exhibit
multiple localizable cognitive or neurological
impairments.
To the extent that the
multiple impairments
in a disorder are correlated, perhaps due to multiple localized
lesions arising from a unitary pathogenesis,
the examination
of simple correlations
among measures designed to assess distinct functions (i.e., the bivariate approach)
can produce misleading conclusions
about the discriminant
validity of the measures.
For example,
in the schizophrenic
subjects, the pattern
of simple correlations
observed
between the two WCST factors, Perseveration
and Nonperseverative
Errors, and the memory composite could have come about for at least two reasons:
(1) the WCST factors could lack discriminant
validity (i.e., they may be tapping both
executive
and memory function);
and (2) the WCST factors could be specific
measures
of executive function,
but because schizophrenia
may produce
both
a correlation
between ordering
(executive
executive
and memory impairments,
function)
and memory
might
be expected.
Yet, even when these multiple
impairments
coexist in a disorder, the correlation
between the functions is less than
perfect. In other words, there are overlapping
and nonoverlapping
(i.e., dissociable)
components
of executive and memory function
in schizophrenia.
An important
challenge for neuropsychology,
then, is to find a method, richer than bivariate
analysis, to distinguish
between performance
on a nonspecific
test and correlated
impairments
in evaluating the discriminant
validity of a test.
195
Hierarchical multiple regression models offer one promising multivariate
approach to meeting this challenge. In our regression model, we could view the
correlation between the ordering composite and the memory composite as a
reflection of the overlapping executive and memory dysfunctions in schizophrenia.
When this correlation was removed from the memory composite, the residual
memory composite was able to reflect the component of memory function that was
dissociable from executive function. In effect, we removed the shared aspects of
memory and executive function that coexist in a disorder exhibiting both executive
and memory dysfunction. If the residual memory composite was uncorrelated with
the WCST factor, then we could conclude that the simple correlation between the
WCST factor and the nonresidualized memory composite arose from overlap between the functions within the disorder (i.e., from the covarying aspects of multiple
cognitive impairments). This was our conclusion about the Perseveration factor in
the schizophrenic patients. On the other hand, if the residual memory composite
(i.e., the aspect of memory function that was dissociuble from executive function)
were correlated with the WCST factor, then this WCST factor would be shown to be
sensitive to aspects of memory function that were dissociable from executive
function. Therefore, the factor would lack specificity, or discriminant validity, as an
executive measure. This was our conclusion about the Nonperseverative Errors
factor in schizophrenia. Thus, the contribution of multivariate hierarchical regression was to provide evidence about the validity and specificity of a test designed to
measure a specific function when applied to a disorder that causes multiple cognitive
and neurological impairments. Indeed, using tests without discriminant validity
could lead to erroneous conclusions suggesting the presence of “global,““diffuse,” or
“nonlocalizable” impairment when, in reality, such impairment comprises a limited
number of discrete deficits (cf. Sagar and Sullivan, 1988).
Depending upon the WCST score or factor that was analyzed and the group that
was considered, the WCST reflected dysfunction either relatively specific (e.g., the
Perseveration factor) or nonspecific (e.g., categories achieved). Similarly, Kersteen
(1989) reported that patients with DLPFC lesions (7 patients used in the present
study), as well as patients with temporal-parietal lesions, were impaired in categories
achieved and total errors on the WCST, whereas the former but not the latter group
showed impairment, relative to control subjects, in perseverative errors. Teuber and
Liebert (1958) also illustrated this point with a different test. Their patients with
penetrating head injuries involving the anterior aspects of the brain were selectively
impaired in setting a luminous line to the vertical when their bodies were tilted. By
contrast, patients with posterior brain injuries were selectively impaired in this task
when their field of vision was limited to a striped background set at various tilts.
Interestingly, when the results were scored taking starting position of the line to be
adjusted to the vertical into account, both head injury groups were impaired in the
body tilt condition. The conclusion drawn was that “...one and the same task could
be shown to reveal general (nonlocalizable) and specific (relatively localizable) effect
of brain injury” (p. 407), depending upon the analytic approach used. Thus, even in
studies investigating patients with localizable lesions, the selective effects of the
lesions can be elusive.
196
Construct
Validity
in Neuropsychology.
The analysis
used in this study
demonstrates
one approach that can be used to establish the construct validity of
neuropsychological
tests in populations
without
localized
brain
lesions.
In
populations
of patients with structurally
observable
localized lesions, there is an
obvious criterion against which to evaluate the validity of a test designed to measure
a localized brain function: patients with lesions in one brain region show impaired
performance
on the test relative to intact individuals
as well as to patients with
lesions in a different brain region. In patient groups without clearly delineated
structural
lesions but suspected
of having abnormal
brain function,
no single
criterion can serve to validate a neuropsychological
test. In the absence of observable
structural
or functional
brain abnormalities,
the inference
of localized
brain
pathology
from impaired performance
on neuropsychological
tests requires the
accumulation
of supporting
evidence that is necessarily
of an indirect nature. In
effect, neuropsychological
impairment
in psychiatric
populations
without demonstrable lesions takes on the status of a hypothetical
construct (Cronbach and Meehl,
1955). Inasmuch as constructs are not directly observable, their existence is inferred
from the pattern
of interrelationships
among
their hypothesized
observable
manifestations.
Since there is no set of necessary or sufficient conditions
(short of
discovering a localized brain lesion) that can establish the validity of a test measuring
a construct,
the validity must be established
by the “nomological
network”
of
associations
and nonassociations
that the test shows with other theoretically relevant
and irrelevant
variables. This is what Cronbach
and Meehl (1955) referred to as
construct validity. Demonstrating
patterns of convergent
and discriminant
validity
within and across data domains (e.g., psychometric,
behavioral,
physiological,
and
anatomical)
is a fundamental
approach
to the validation
of “neuropsychological
constructs.”
Acknowledgments.
The authors wish to thank Jody Rawles, Chung Nim Ha, Linda Davis,
Stacie DeMent, Maria Stein, Hubert Chen, and Dorothy Talbert for their careful attention to
detail in test construction,
administration,
and scoring; and Robert Heaton, Marlene OscarBerman, Margaret Rosenbloom,
and Paula K. Shear for valuable discussions and comments
on the manuscript.
Results from this study were presented at the International
Congress on
Schizophrenia
Research (Sullivan et al., 1991). This work was supported
by NIMH (MH30854), NIAAA (AA-05965), NARSAD,
NINDS (NS-21135) the Department
of Veterans
Affairs, and the Meyer Foundation.
References
American Psychiatric Association.
DSM-III-R: Diagnostic and Statistical Manualof Mental
Disorders. 3rd ed., revised. Washington,
DC: American Psychiatric Press, 1987.
Anderson,
S.W.; Damasio, H.; Jones, R.D.; and Tranel, D. Wisconsin Card Sorting Test
performance
as a measure of frontal lobe damage. Journal of Clinical and Experimental
Neuropsychology, 13:909-922, 199 1.
D.R. Physiological
dysfunction
of
Berman,
K.F.; Illowsky,
B.P.; and Weinberger,
dorsolateral
prefrontal
cortex in schizophrenia:
IV. Further evidence for regional and
behavioral specificity. Archives of General Psychiatry, 45:616-622, 1988.
Bornstein,
R.A. Contribution
of various neuropsychological
measures to detection
of
frontal lobe impairment. International Journal of Clinical Neuropsychology, 8: 18-22, 1986.
197
Braff, D.L.; Heaton, R.; Kuch, J.; Cullum, M.; Moranville, J.; Grant, I.; and Zisook, S. The
generalized pattern of neuropsychological
deficits in outpatients with chronic schizophrenia
with heterogeneous
Wisconsin Card Sorting Test results. Archives of General Psychiatry,
48:891-898, 1991.
Campbell, D.T., and Fiske, D.W. Convergent and discriminant validation by the multitraitmultimethod
matrix. Psychological Bulletin, 56:81-105, 1959.
Cohen, J., and Cohen, P. Applied Multiple Regression/Correlation
Analysis for the
Behavioral Sciences. Hillsdale, NJ: Lawrence Erlbaum, 1985.
Cooper,
J.A.; Sagar, H.J.; Jordan,
N.; Harvey, N.; and Sullivan,
E.V. Cognitive
impairment in early, untreated Parkinson’s disease and its dissociation from motor disability.
Brain, 114:2095-2122, 1991.
Corkin, S.; Cohen, N.J.; Sullivan, E.V.; Clegg, R.A.; Rosen, T.J.; and Ackerman,
R.H.
Analyses of global memory impairments
of different etiologies. Annals of the New York
Academy of Sciences, 444: 10-40, 1985.
Cronbach,
L.J., and Meehl, P.E. Construct validity in psychological
tests. Psychological
Bulletin. 52:281-302, 1955.
Crovitz, H.F., and Zener, K.A. Group test for assessing hand and eye dominance. American
Journal of Psychology, 75:271-276, 1962.
Dehaene, S., and Changeux, J.-P. The Wisconsin Card Sorting Test: Theoretical analysis
and modeling in a neuronal network. Cerebral Cortex, 1:62-79, 1991.
Drewe, E.A. The effect of type and area of brain lesion on Wisconsin Card Sorting Test
performance.
Cortex, 10:159-170, 1974.
Endicott, J., and Spitzer, R.L. A diagnostic interview: The Schedule for Affective Disorders
and Schizophrenia.
Archives of General Psychiatry, 35:837-844, 1978.
Eslinger, P.J., and Damasio, A.R. Severe disturbance
of higher cognition after bilateral
frontal lobe ablation: Patient EVR. Neurology, 35:1731-1741, 1985.
Fey, E.T. The performance
of young schizophrenics
and young normals on the Wisconsin
Card Sorting Test. Journal of Consulting Psychology, 15:311-319, 1951.
Folstein, M.F.; Folstein, S.E.; and McHugh, P.R. Mini-Mental
State: A practical method
for grading the cognitive state of patients for the clinician. Journal of Psychiatric Research,
12:189-198, 1975.
Freedman,
M., and Oscar-Berman,
M. Bilateral frontal lobe disease and selective delayed
response deficits in humans. Behavioral Neuroscience, 100:337-342, 1986.
Freund, G., and Ballinger, W.E., Jr. Loss of cholinergic muscarinic receptors in the frontal
cortex of alcohol abusers. Alcoholism, 12:630-638, 1988.
Frey, R.T.; Woods, D.L.; Knight, R.T.; Scabini, D.; and Clayworth,
CC. Defining
functional areas with averaged CT scans. Society for Neuroscience Abstracts, 1987.
Goldberg, T.E.; Kelsoe, J.R.; Weinberger, D.R.; Pliskin, N.H.; Kirwin, P.D.; and Berman,
K.F. Performance
of schizophrenic
patients on putative neuropsychological
tests of frontal
lobe function. International Journal of Neuroscience, 42151-58, 1988.
Goldberg, T.E.; Weinberger, D.R., Berman, K.F.; Pliskin, N.H.; and Podd, M.H. Further
evidence for dementia of the prefrontal type in schizophrenia?
A controlled study of teaching
the Wisconsin Card Sorting Test. Archives of General Psychiatry, 44:1008-1014, 1987.
Grant, D.A., and Berg, E.A. A behavioral analysis of degree of reinforcement
and ease of
shifting to new responses in a Weigl-type card-sorting
problem. Journal of Experimental
Psychology, 38:404-411, 1948.
Gur, R.C.; Saykin, A.J.; Benton, A.; Kaplan, E.; Levin, H.; Kester, D.B.; and Gur, R.E.
“Behavioral
imaging”-111.
Interexpert
agreement
and
reliability
of weightings.
Neuropsychiatry, Neuropsychology, and Behavioral Neurology, 3: 113-124, 1990a.
Gur, R.C.; Saykin, A.J.; Muenz, L.R.; Trivedi, S.S.; and Gur, R.E. Response to Yeo et al.‘s
critique
of behavioral
imaging.
Neuropsychiatry,
Neuropsychology,
and Behavioral
Neurology, 3:304-312, 19906.
Harper, C.G., and Kril, J.J. Neuropathology
of alcoholism. Alcohol & Alcoholism, 25:207216, 1990.
198
Heaton, R.K. Wisconsin Card Sorting Test Manual. Odessa, FL: Psychological
Assessment
Resources, Inc., 198 1.
Heck, E.T., and Bryer, J.B. Superior sorting and categorizing
ability in a case of bilateral
frontal
atrophy:
An exception
to the rule. Journal of Clinical and Experimental
Neuropsychology, 8:313-3 16, 1986
Heinrichs,
R.W. Variable associated with Wisconsin Card Sorting Test performance
in
neuropsychiatric
patients
referred
for assessment.
Neuropsychiatry,
Neuropsychology,
Behavioral Neurology, 3: 107-l 12, 1990.
Jones-Gotman,
M. Memory for designs: The hippocampal
contribution.
Neuropsychologia,
24: 193-203, 1986.
Kersteen,
Z.A. Behavioral
and neuroanatomical
aspects of priming and motor skill
learning. Unpublished
doctoral dissertation,
1989.
Kersteen-Tucker,
Z., and Knight, R.T. Cortical lesions dissociate short and long-term
components
of repetition priming. Society for Neuroscience Abstracts, 1989.
Kolb, B., and Whishaw, I.Q. Performance
of schizophrenic
patients on tests sensitive to left
or right frontal, temporal, or parietal function in neurological
patients. Journal of Nervous
and Mental Disease, 17 1:435443, 1983.
Lishman, W.A. Alcohol and the brain. British Journal of Psychiatry, 156:635-644, 1990.
Malmo, H.P. On frontal lobe functions: Psychiatric patient controls. Cortex, 10:231-237,
1974.
Matarazzo,
J.D. Psychological
assessment versus psychological
testing: Validation from
Binet to the school, clinic, and courtroom. American Psychologist, 45:999-1017, 1990.
Meehl, P.E. High school yearbooks:
A reply to Schwartz.
Journal of Abnormal
Psychology, 77:143-148, 1971.
Milner, B. Psychological deficits produced by temporal-lobe
excision. Research Publication
of the Association for Research of Nervous and Mental Diseases, 361244-257, 1958.
Milner, B. Effects of different brain lesions on card sorting. Archives of Neurology, 9:90100, 1963.
Milner, B. Interhemispheric
differences in the localisation
of psychological
processes in
man. British Medical Bulletin, 271272-277, 197 1.
Milner, B.; Corsi, P.; and Leonard, G. Frontal-lobe
contribution
to recency judgments.
Neuropsychologia, 29:60 l-6 18, 199 1.
Milner, B.; Petrides, M.; and Smith, M.L. Frontal lobes and the temporal organization
of
memory. Human Neurobiology, 4: 137-142, 1985.
Nelson, H.E. National Adult Reading Test, NART Windsor: Nelson Publishing Company,
1982.
Paivio, A.; Yuille, J.C.; and Madigan, S.A. Concreteness,
imagery, and meaningfulness
values for 925 nouns. Journal of Experimental Psychology, 76: l-24, 1968.
Pakkenberg,
B. Post-mortem
study of chronic schizophrenic
brains. British Journal of
Psychiatry, 151~744-752, 1987.
Parsons,
O.A.; Butters, N.; and Nathan, P.E., eds. Neuropsychology of Alcoholism:
Implications for Diagnosis and Treatment. New York: Guilford Press, 1987.
Pedhazur, E.J. Multiple Regression in Behavioral Research. 2nd ed. Fort Worth, TX: Holt,
Rinehart and Winston, 1982.
Petrides, M., and Milner, B. Deficits on subject-ordered
tasks after frontal- and temporallobe lesions in man. NeuropsychoIogia, 20:249-262, 1982.
Pfefferbaum,
A.; Lim, K.O.; Zipursky, R.B.; Mathalon,
D.M.; Rosenbloom,
M.J.; and
Sullivan, E.V. Brain gray and white matter volume loss accelerates with aging in chronic
alcoholics:
A quantitative
MRI study. Alcoholism: Clinical and Experimental Research,
16:1078-1089, 1992.
Ron, M.A. The brain of alcoholics:
An overview. In: Parsons, O.A.; Butters, N.; and
Nathan,
P.E., eds. Nemopsychology
of Alcoholism, Implications for Diagnosis and
Treatment. New York: Guilford Press, 1987. pp. 1 I-20.
199
Ryan, C., and Butters, N. Cognitive deficits of alcoholics. In: Kissin, B., and Begleiter, H.,
eds. l?hhePathology of Alcoholism. New York: Plenum Publishing Corp., 1983. pp. 485-538.
Sagar, H.J., and Sullivan, E.V. Patterns of cognitive impairment in dementia. In: Kennard,
C., ed. Recent Advances in Clinical Neurology. Vol. 5. Edinburgh:
Churchill Livingstone,
1988. pp. 47-86.
Shimamura,
A.P.; Jernigan, T.L.; and Squire, L.R. Korsakoff’s syndrome: Radiological
(CT) findings and neuropsychological
correlates. Journal of Neuroscience, 8:4400-4410, 1988.
Spitzer, R.L.; Endicott, J.; and Robins, E. Research Diagnostic Criteria (RDC). New York:
New York State Psychiatric Institute, 1975.
Sullivan, E.V.; Mathalon, D.H.; Zipursky, R.B.; and Pfefferbaum,
A. The validity of the
Wisconsin Card Sorting Test factors as measure of dorsolateral prefrontal cortical function in
schizophrenia
and alcoholism. Schizophrenia Research, 41394, 1991.
Sullivan, E.V., and Sagar, H.J. Nonverbal recognition and recency discrimination
deficits
in Parkinson’s disease and Alzheimer’s disease. Brain, 112: 1503-1517, 1989.
Tarter, R.E. An analysis of cognitive deficits in chronic alcoholics. Journal of Nervous and
Mental Disease, 157:138-147, 1973.
Teuber, H.-L., and Liebert, R.S. Specific and general effects of brain injury in man.
Archives of Neurology & Psychiatry, 80:403407, 1958.
Thorndike,
E.L., and Lorge, I. 7’he Teacher’s Word Book of 30,000 Words. New York:
Teachers College, Columbia University, 1944.
Wechsler, D. Wechsler Adult Intelligence Scale-Revised: Manual. Cleveland, OH: The
Psychological
Corporation,
198 1.
Wechsler,
D., and Stone, C.P. Wechsler Memory Scale. New York: Psychological
Corporation,
1945.
Weinberger,
D.R.; Berman,
K.F.; and Illowsky,
B.P. Physiological
dysfunction
of
dorsolateral
prefrontal
cortex in schizophrenia:
III. A new cohort and evidence for a
monoaminergic
mechanism. Archives of General Psychiatry, 45:609-615, 1988.
Weinberger,
D.R.; Berman, K.F.; and Zec, R.F. Physiologic dysfunction
of dorsolateral
prefrontal
cortex in schizophrenia:
I. Regional cerebral blood flow evidence. Archives of
General Psychiatry, 43~114-125, 1986.
Wiggins, J.S. Personality and Prediction: Principles of Personality Assessment. Reading,
MA: Addison-Wesley
Publishing Company, 1980.
Yates, W.R.; Swayze, V.W. II; and Andreasen, N.C. Neuropsychological
effect of global
and focal cerebral
atrophy
in schizophrenia.
Neuropsychiatry,
Neuropsychoiogy,
and
Behavioral Neurology, 3:98-106, 1990.
Yeo, R.A.; Turkheimer,
E.; and Bigler, E.D. Neuropsychological
methods of localizing
brain dysfunction:
Clinical versus empirical approaches. Neuropsychiatry,
Neuropsychology,
and Behavioral Neurology, 3:290-303, 1990.
Zipursky, R.B.; Lim, K.O.; Sullivan, E.V.; Brown, B.W.; and Pfefferbaum,
A. Widespread
cerebral gray matter volume deficits in schizophrenia.
Archives of General Psychiatry, 49: 195205, 1992.