Time-Quality
Performance of Goodwill
Clients Evaluated By The JEVS
Work Sample Battery
DON
FLENNIKEN
January 1973,
the vocational evaluation unit of Goodwill Industries, Zanesville, Ohio adopted the Philadelphia Jewish Employment and Vocational Service (JEVS) work sample battery. The standardized procedures, as outlined in the •lanual for Use of ]EVS WorkSample Program (Jewish Employment and Vocational Service, 1970),
were used until July, 1973, when
a revision of the work sample program was made available." Since the administration, scoring, and norming procedures were different in the revised form, data obtained from
the revised system were not incorporated into the results of this study.
The purpose of this paper is to report the findings resulting from
the use of the JEVS System in terms of client performance on the
work sample components. Since the JEVS System has been revised,
the rosults of this study have historical value. In addition, these findings might assist in understanding similar variations that occur in the
System.
use of the revised JEVS
In
SUBJECTS
ninety-three clients were involved in the work sample
evaluation. Each was a referral from the Ohio Bureau of Vocational
Rehabilitation. Disabilities, recorded from the standard referral forms,
Listed in their order of frewere grouped into fourteen categories.
quency, with the number of clients indicated in parantheses, these
disability groups are mental retardation (36), psychotic and psychoneurotic disorder (31), epilepsy (13), impairment of one or more
upper or lower limbs (11), accident involving trunk or spine, or both
(8), behavioral disorder ( 6 ), borderline mental retardation (5), endocrine dysfunction (4), alcoholism (3), circulatory, including heart
A total of
Flenniken is the Supervisor of the Evaluation Department at Goodwill Industries in ZanesHe was previously employed as a high school science teacher in Ohio £or nine years. Mr.
Flenniken received a bachelor's degree from Marietta College in Marietta, Ohio in 1962 and a master's
degree from the University o£ Northern Iowa in Cedar Fails, Iowa in 1971.
Don
ville, Ohio.
VOLU•E
7, NUMBER
3 /
SEPTEMBER, 1974
3
ulcers (1), colostomy (1), blindness in one eye, other eye good
and arthritis and rheumatism (1). it is to be noted that some of
the clients were recorded as having multiple disabilities and as such
than once in the disability figures above. Of the
were counted more
total population of ninety-three, sixty were male and thirty-three were
female. Ages ranged from sixteen to sixty-three for males and from
sixteen to forty-four for females.
(9.),
(J),
METHOD
During the twenty-six weeks between January 22 and July 23,
1973, ninety-three clients were evaluated by the JEVS work sample
battery. Initially, all work samples in the battery were administered
to each client, unless otherwise precluded by low reading ability or
by physical disability (e.g. paraplegics were unable to perform the
TABLE
NUMBER
Name
Work
OF
CLIENTS
1
TA•KING EACH
WORK
Number of
Individuals
Of
Sample
Nut, Bolt, Washer Assembly
Stamping
Washer Threading
Budgette Assembly
Sign Making
Tile Sorting
Nut Packing
Collating Leather Samples
Grommet Assembly
Union Assembly
Belt Assembly
Ladder Assembly
Soldering
Hardware Assembly
Telephone Assembly
Lock Assembly
Filing Numbers
Proofreading
Filing Letters
Nail and Screw Sorting
Adding Machine
Payroll Computation
Computing Postage
Resistor Reading
Pipe Assembly
Blouse Making
Vest Making:
Condensing Principle
•Specific work sample for
•Specific work sample for
90
90
Rubber
87
75
89
92
90
91
87
84
84
47
27
80
60
49
77
46
51
69
50
45
17
27
32
7
16
10
•
4
VOCATIONAL
SAMPLE
female evaluees
male evaluees
EVALUATION
only.
only.
)d•rI)
WORK
/ixDJUSTMENT BULLETIN
Subsequently, however, administration of the work samples was made more selective. Only those
clients who possessed sufficient reading and physical ability and who
performed adequately on the more elementary work samples were
administered the more complex tasks. The results of such selectivity
is reflected in Table 1 which shows the total number of clients administered each specific work sample.
Two work samples were developed by ]EVS to be administered separately to male and female evaluees; Vest Making for males and Blouse
Making for females. This selectivity is also seen in Table 1.
For each client, ratings on time and quality were recorded in accordance with the procedures specified in the Manual for Use of 1EVS
Work-Sample Program (1970). In this manual, ratings for time are
based upon a scale from one ( 1 ) through five (5). Quality ratings are
based upon this same five-point scale with the exception of certain
work samples which are designated as PASS/FAIL. With respect to
the PASS/FAIL ratings, the IEVS Manual indicates that
grommet assembly work sample).
These work samples, if perform•d correctly, receive 5 or the
score; ff performed incorrectly; they are scored 1 or the lowest
(p. 139).
score
is
as
The five-point rating scale,
follows:
Legend
for
1.
acceptable
setting
Rating
in
in the
JEVS Manual
(1970)
Samples
(1) are
performance
rating of
highly a structured, onesupportive,
noncompetitive work
a
present evidence
with little
achieved.
specified
receiving
Variables
even
Work
as
highest
possible
un-
or
indication that
improvement
can
be
performances
receiving a rating of two (2) are
achieving
below
the
level
of a highly
participant
is
a
structured, supportive, noncompetitive work setting. However, there are
indications that with continued support he may be able to t?unction within
such a work setting.
performances char3. Variables receiving a three (3) rating are
acteristic of a participant functioning satisfactorily in a highly structured,
supportive, noncompetitive work setting.
4..Variables receiving a rating ot? four (4) are
performances
which indicate the potential to achieve at and approach the criteria of vocational training and/or competitive employment.
5. Varibales receiving a rating of five (5) are demonstrated
performances commensurate with
standards of
worker in competitive industry or a good candidate for vocational traininga (pp. 138-139).
In each instance, the evaluees" performance was contrasted with
2. Variables
which suggest that
that
a
expected
comparison
can
be
VOLUME
from a "'normal curve" model. The importance of such
is that any variation of data from a normal distribution
seen.
7,
NUMBER
3 /
SEPTEMBER, 1974
5
RESULTS
A total of 1659 work samples were administered to ninety-three
Bureau of Vocational Rehabilitation referred clients during the twentysix week period between January 22 and July 23, 1973.. Although not
all clients were administered the entire battery of twenty-eight work
samples, several general trends were noted in comparing actual performance with that expected on the basis of a normal curve model.
As shown in Figure 1, the number of total work samples on which
clients ratings of 5 and 4 was consistent with the number of 5 and 4
ratings expected by the normal curve model. The number of 1, 2, and
3 ratings were not in accordance with normality.
N==I2
56
RATING
Fig.
1.
Composite
Work
Time
Norms
for
all
Fig. 2.
Samples.
Composite Quality Norms •[or all
Work Samples Except those Bated as PASS/FAIL.
N----•03
Fig.
3.
Composite Quality NOrms
Work Samples Rated as
for
all
PASS/
FAIL.
The overall, composite quality ratings, as seen in Figure 2, show
There is a steady
over four times as many one ratings as two ratings.
upward progression of the ratings of three, four, and five. Certain of
the work samples are designated as either PASS or FAIL in respect to
quality standards. Overall, the ratio of PASS-FAIL was 3 1 (Figure
3.).
VOCATIONAL
EVALUATION
AND
WORK
ADJUSTMENT
BULLETIN
•
•-9•..
•,o
•o
•
•
"
,,.•
•o
"
•
30
•of
•
5
TIME
Fig.
4.
....
Fig.
TIME •TING
Rubber St•ping
Fig. 5.
etc.
"•
N--87
.-•
40
F
•
-.•
N=75
._--:
TIME
Fig.
TIME
•0
.
TIME •TING
6. Washer Threa•ng
Fig. 8.
RATING
Fig.
-• •o-
T/ME
RATING
Nut Packing
VOLUM-• 7, NU/VJ•BER 3 /
SEPTEMBER,
•TING
Bud•e•e •sembly
•.
TIME
Sign Making
N=9o
Fi•. 10.
3
•
RATING
Nuts, Bolts,
•o,
./,•'•"
L
•
9.
RATING
Tile Sorting
N=gz
T/ME
Fi•. II.
I974
RATING
Leather Samlfles
7
Fig.
TIME
TIME RATING
12.
Grommet Assembly
Fig,
RATING
Union
13,
Assembly
RATING
Ladder Assembly
TIME
Fig. 15,
Nffi80
TI]VIE RATIN(•
Fig. 17.
TI•E
Fig. 18,
8
RATING
Telephone Assembly
VOCATIONAL
Fig.
EVALUATION
AND
WOrK
Hardware Assembly
TIME RATING
19.
Lock Assembly
AD3USTMENT ]•ULLETIN
N=77
//
•o-
•
Fig.
#0
•
•o
•
20
•
"',•
TNE
20.
-
:••
Fig.
:--
•
N=50
• 10
o
•"
•o
• .•..•
/"" . ._..
•
•,
• "-
"X-•
.
•TING
•aehine
TI•E
Fig. •.
Proo•eading
•o•,
Adding
/
.o
•
•-••
..."
.Fig.
TIME •TING
Filing Numbers
22.
•0
• 20
•TING
T•E
Fig. 21.
•
.i
__--.___
,
RATING
o•-•.
•
•
FiI•g N•bers
r
• •o
N=46
•
•
40
•
20
••
•
io
,
TIME
•TING
N•4 5
•----•.,.-,,
__•:/_I.
•.•.•
"---•
TIME
Fig.
•TING
•a•oll Computation
25.
5
Fig.
VOL•nV•.
26.
RATING
Compuring Postage
7, NLrlVIBER 3 / SEPTEMBER, 1974
".
o
_...:)
TIME
So•ing
Nail/Screw
23.
•..•..•.
4
TIME
Fig. ET.
3
2
1
BATING
Resistor Beading
9
Fig.
• 12
•
6
TIME RATING
28.
Pipe Assembly
HATING
TIME
Fig. 29.
N=16.
-• 8
Blouse Making
F
•
5
TIME •TING
Fig, 30. Vest Mak•g
4
3
TIME
Fig. 31,
2
•TING
Condensin• PrinciDle
Examination of the time ratings •or each work sample and comparing these with the expected, or normal, distribution show that the
general trend seen in Figure 1 is also dominant in many of the individttal work samples (Figures 4-31 ). Only in Figures 5, 8, 9, 11, and
28 did the pattern approach the expected bell-shaped curve o5 a
normal distribution. The other patterns of distribution are similar to
the overall pattern o£ distribution seen in Figure 1.
10
VOCATIONAL
EVALUATION
AND
WORK
1
ADJUSTMENT BULLETIN
..So-
•
50
N=90
[•
•.o
•
,.o20
•o
• •.o
•3o
•20,•
1
Fig.
32.
50-
•40-
0
4
3
2
QUALITY
r
•o
/
0-
N=90
1
RATING
Nuts, Bolts,
50_-
etc.
N=87
Fig.
3
2
QUALITY
33,
4
5
RATING
Stamping
Rubber
N=75
•
• •0
•o
20-
o
20-
I0-
QU•ITY
RATING
Washer Threading
Fig. 34.
N=89
25-
2
1
QUALITY
Fig. 35.
25-
3.
4
5
RATING
Budge•e Assembly
Nffi90
• I0-
QUALITY RATING
36.
Sign Making
_Fig.
VOLUME
7,
NUMBER
3 /
SEPTEMBEII, 1974
QUALITY
Fig. 37.
RATING
Nut Packing
ii
N=84
N=47
• 30
•20
Z=
o
I0-
I0
Z
0-
5
1
QUALITY RATING
Fig. 38. Union Assembly
10-
•
Fig.
N=27
QUALITY RATING
39.
Ladder Assembly
N=80
m
40
.•-
30
[,
o-.
5
QUALITY
RATING
Fig. 40.
Soldering
QUALITY R•TING
Fig. 41. Hardware Assembly
5o
5o
•
•
N=60
40
N=77
• 30
"• 30
m
2
3
4
5
QUALITY RATING
Fig. 43. Filing Numbers
VOCATIONAL
EVALUATION
AND
WOI•K
ADJUSTMENT
BULLETIN
QUALITY RATING
Fig. 45. Filing Numbers
QUALITY RATING
Fig. 44. Proofreading
Nffi69
•o•
•
•
,
r
,
2
q
4
o-.
1
5
"t
QUALITY RATING
Fig.
2.
3
QUALITY RATING
Fig. 47.
Nail/Screw Sorting
46.
N=50
2o-
Adding
Machine
N=I7
2
1
Fig.
3
4
5
QUALITY RATING
Payroll Computation
QUALITY RATING
Fig. 49. Computing Postage
48.
VOLUlVIE 7, Ntr•B•.R 3 /
SEPTEMBER,
1974
13
•I•
-
.
• •0
• I0
QUALITY RATING
Fig. 51. Pipe Assembly
QU•IT• RATING
Fig. 50. Resistor Reading
5
N=I6
N---7
3.. •
2
1
5
]i
2
3
•5
QUALITY RATING
Fig. 53. Vest Making
QUALITY RATING
Fig. 52. Blouse Making
10-
N=I0
4-
•
I
2
3
4
5
QUALITY RATING
Condensing Principle
l•'ig. 54.
14
VOCATIONAL
EVALUATION
AND
WOBK
ADJUSTMENT BULLETIN
Quality ratings
for each work sample (Figures 32-54) show greatdid time ratings. In most instances there is still a
er variation than
bimodel distribution similar to that of Figure 2; the two largest groups
of clients performing at opposite ends of the rating scale, with a lesser
number falling between the extremes. This is particularly seen in
Figures 32, 35, 41, 42, 43, 44, 45, 46, and 49. The pattern is less
dominant in Figures 37, 38, 39, 50, and 54, and not seen in Figures
33, 34, 36, 40, 47, 48, 51, 52, or 53.
27.
Fall
Fig. 55.
Fig. 56.
Tile Sorting
Leather
Ssmples
N=87
Fig. 57.
Grommet
Assembly
Fail
Fig. 58.
Belt
VOLV'ME
7,
Fig. 59.
Assembly
N•]eR
S /
S•'-rT•'•E•S, 1974
Lock
Assembly
samples having a PASS-FAIL criteria, the pervaried from 0% (Figure 55) to 84% (Figure 59).
In the five work
centage of failures
CONCLUSIONS
This study indicates that actual performance ratings of time and
quality do not always follow the normal distribution patterns established by the Philadelphia Jewish Employment and Vocational Service for their work sample battery. In general, such clients do either
well or poorly in quality; few perform at the average level. Rather
than showing the normal inverted "'V'" distribution, the quality ratings
generally show a "'U'" pattern. Furthermore, instead of showing the
normal inverted "'V'" pattern, time ratings generally form an "'M'" pattern when such ratings are plotted with the numbers of individuals
along the x-axis and the time ratings along the }-axis of the graph.
Based upon the results of this study, it appears that bimodel distributions of ratings may be typical of clients who possess such varied
disabilities as alcoholism, arthritis, epilepsy, behavioral problems,
emotional instability, and mental retardation. Since the ratings of
these clients were not distributed on a normal curve, it appears that
the normative data which accompanies the JEVS System (which was
derived from the use of the System with a disadvantaged population)
may not directly be applicable to a disabled population. Conversely,
these findings may also raise questions about the applicability of other
vocational evaluation systems, that were designed for a disabled population, when such systems are applied with a disadvantaged population.
SUMMARY
the course of this study a total of ninety-three clients were
referred to Goodwill Industries, Zanesville, Ohio by the Ohio Bureau
of Vocational Rehabilitation for vocational evaluation by use of the
JEVS work sample battery. A total of 1659 work samples were administ.ered during the time period from January 22 until July 23, 1974.
A bim-•dal distribution of performance scores in both time and quality
evidenced by the results of this study. Clients generally did
was
either well or poorly in quality, with relatively few clients occupying
the average range. Clients usually followed normal distribution patterns for the above average norms for time, but fell short of the expected numbers in the average time range. Additionally, those clients
working slower than the expected below-average speeds were in
gr.eater numbers than expected on a normal curve. Such bimodal distributions might be expected when applying the JEVS System with
clients who have a variety of disabilities.
During
REFERENCE
JEWISH
EMPLOYMENT
VOCATIONAL
SERVICE.
Manual
for
of ]EVS work
use
sample program. AND(Manpower Administration Contract No.
82-40-67-40).
Philadelphia, Pa.: Jewish Employment and Vocational Service, 1970.
VOCATIONAL
EVALUATION
AND
WORK
ADJUSTMENT
BULLETIN