How to build a Situational Judgment
Test that delivers real value to the
recruiter and the candidate
Eugene Burke & Carly Vaughan
SIOPSA July 2011
©2011 SHL Group Limited
A story in three parts
The state of the science of SJTs
The practical and investment decisions in
implementing an SJT
Question & answer session framed around key issues
©2011 SHL Group Limited
2
The science of SJTs
©2011 SHL Group Limited
3
• Considerable debate around
• What SJTs measure
• Most effective methods for
building SJTs
• Lack of psychometric
properties for SJTs with
analogous case of the
assessment centre raised
• Differences in relationships
with other measures as well as
impacts such as ethnic and
gender differences
• Little information on the
generalisability of SJTs across
languages and cultures
©2011 SHL Group Limited
4
So in Part 1 we will address these issues
©2011 SHL Group Limited
5
Let us start with a brief history of SJTs
©2011 SHL Group Limited
6
History of SJTs (1)
• US Civil Service Exams (1870s)
• A banking company asks protection for a certain device, as a
trade-mark, which they propose to put upon their notes. What
action would you take in the application
• Binet (1905)
• When a person has offended you and comes to offer his
apologies, what would you do?
• Ansbacher (1941)
• Your sports club is planning a trip to Berlin to attend the
Germany-England football game, which will take place in 2 weeks.
You have been entrusted with the preparations and entire
management of the trip. What do you intend to do?
©2011 SHL Group Limited
7
History of SJTs (2)
• Closed ended response options started to be introduced:
• George Washington Social Intelligence Test
• Criticism that they were very highly correlated with general
intelligence
• Motowidlo, Dunnette & Carter (1990) re-opened the
discussion of SJTs
• Low fidelity simulation for entry-level managers in
telecommunications industry
• Hypothetical work behaviour can predict performance
(correlations ranged between .28 and .37)
• In the last 20 years, research published into SJTs has
grown
• Lievens quotes 2 publications per year 1990–2002 and 10
publications in 2006 alone
©2011 SHL Group Limited
8
Why are SJTs becoming more popular?
©2011 SHL Group Limited
9
SJT’s are becoming popular
• They are seen to have high relevance to the job/
organisation by stakeholders
• Rated highly as a preferred method of assessment in research
on candidate perceptions of assessment methods
• Seen as highly relevant by line managers
• Evidence shows that they are fair
• Small to zero differences between males and females, between
age groups and between ethnic groups
• Strong evidence showing that they are good
predictors of job performance
• Low fakability when using the answer format
developed for EPSO
©2011 SHL Group Limited
10
Is the candidate also a customer?
• Consider the case of a retailer or service provider
• Consider the case of recruitment to a front line
operational role and volume recruitment
• The applicant may be an existing or potential future
customer
Is the candidate experience going to be important?
Would a highly credible and realistic assessment help
manage the dual needs of a measure that is predictive
and promotes the company brand?
©2011 SHL Group Limited
11
But what is an SJT?
©2011 SHL Group Limited
12
• A candidate is presented with a variety of realistic
situations likely to be encountered ‘on the job’
• Typically gathered via a job analysis method like critical
incidents
• Each situation is accompanied by a number of different
responses that could be applied to the situation
• The candidate is asked to either
• Choose which is the most effective response
• The response they would most likely choose
• Rate response options on a Likert scale
©2011 SHL Group Limited
13
In addition to response formats, the
framing and scoring of the SJT needs
to be considered
©2011 SHL Group Limited
14
• Behavioural tendency (BT)
• The respondent is asked for their preferred response to the
situation
• This will tell you about their style but does it tell you whether
they understand what an effective response would be?
• Subject matter scoring (SMS)
• Subject matter experts (SMEs) are asked to rate which
responses are more effective or preferred
• The validity of this approach depends on the framing of the SME
task and the nature of those defined as SMEs
• Knowledge instructions (KI)
• Refers to a format the requires the respondent to declare the
most effective and/or least effective responses rather than any
prior job knowledge
©2011 SHL Group Limited
15
Possible response instructions
One response
scored
Two
responses
scored
All responses
scored
Behavioural
Tendency
(BT)
Choose your
preferred answer.
What would you
like to do?
What would you
most prefer to do?
What would you
least prefer to do?
Rate each response
option for how likely
you would respond
that way
Knowledge
Instructions
(KI)
Choose the most
effective answer.
What should you
do?
Pick the most and
the least effective
answer.
Rank the responses
from the most
effective to last
effective
©2011 SHL Group Limited
16
How do SJT scores relate
to other assessments?
©2011 SHL Group Limited
17
The answer partly depends on
how you build an SJT?
©2011 SHL Group Limited
18
Can-do and will-do correlations
Knowledge
Instructions (KI)
Behavioral
Tendency (BT)
Cognitive ability
0.35
0.19
Conscientiousness
0.24
0.34
Agreeableness
0.19
0.37
Emotional Stability
0.12
0.35
From McDaniel et al., 2007
©2011 SHL Group Limited
19
Other factors related to
instructions
(McDaniel, 2007)
Knowledge
Instructions (KI)
Behavioral
Tendency (BT)
Mean scores
Higher
Lower
Ethnic differences
Larger
Smaller
Less
More
ρ = .26
ρ = .26
Fakability
Validity
©2011 SHL Group Limited
20
We advocate the KI format
which we will describe in
more detail
©2011 SHL Group Limited
21
An example (KI) item
A customer has accidentally broken a bottle of oil.
There is broken glass and the oil is quickly spreading
across the floor. One of your colleagues is helping the
customer. You still have to re-fill a lot of shelves before
your break.
Watch your colleague for a moment to see if your help is
needed.
Help your colleague deal with the situation by calling the
cleaning staff.
Stay focused on completing your current task of re-filling
the shelves in time.
©2011 SHL Group Limited
22
A key question is what do SJTs
measure?
©2011 SHL Group Limited
23
Some common problems with SJTs
• No clear logic in the design and construction of content
• Based on a subjective view of the job or role
• Highly inductive and not generalizable
• Not clear what the constructs are
• Scoring is highly empirical
• Based the “best” answers on a sample of employees
• Short validity half-life as the scoring key reflects only a
snapshot in time (i.e. the specific sample the scoring key was
developed on)
• Not designed to cover key behaviours and to allow content to
be configured to meet changing organisational needs
• Not designed to meet localisation needs
• The need for versions in other languages is an after thought
• No definition of whether the content functions in an equivalent
way across languages
©2011 SHL Group Limited
24
But there is hope!
©2011 SHL Group Limited
25
The deductive (construct) driven
approach to designing SJTs
©2011 SHL Group Limited
26
Deductive approach to SJTs
• Founded on a theory of the job
• Uses a behavioural taxonomy to define the critical behaviours
that underpin effective performance
• Uses the links between behaviours and attributes of people to
define the constructs to be measured by the SJT
• This is the criterion-centric approach
• Evaluates the generalisability of the constructs
• Captured through deploying the criterion-centric model across
geographies and settings
• Incorporated in the localisation process through linguist and
subject matter expert reviews
• This is a combined emic-etic approach
©2011 SHL Group Limited
27
Assessing what drives a judgment
Competency can only be evaluated from
the evidence provided by an action
The action will be evidenced in the
form of an observed behaviour
Behaviours therefore provide evidence of judgments
SJTs can be designed to predict which
behaviours are most likely to influence
the judgments of the candidate
©2011 SHL Group Limited
28
An example item
You work as part of a technical support team that produces
work internally for an organisation. You have noticed that
often work is not performed correctly or a step has been
omitted from a procedure.
You are aware that some
individuals are more at fault than others as they do not make
the effort to produce high quality results and they work in a
disorganised way.
a.
Explain to your team why these procedures are important and
what the consequences are of not performing these correctly.
b.
Try to arrange for your team to observe another team in the
organisation who produce high quality work.
c.
Check your own work and that of everyone else in the team to
make sure any errors are found.
d.
Suggest that the team tries many different ways to approach
their work to see if they can find a method where fewer mistakes
are made.
©2011 SHL Group Limited
29
Situation presenting
candidate with a dilemma
to be resolved
©2011 SHL Group Limited
30
Multiple answer options
to capture judgment on
most versus least effective
responses
©2011 SHL Group Limited
31
Designed to tap into judgments
You work as part of a technical support team that produces
work internally for an organisation. You have noticed that
often work is not performed correctly or a step has been
omitted from a procedure.
You are aware that some
individuals are more at fault than others as they do not make
the effort to produce high quality results and they work in a
disorganised way.
• The situations follow scientific principles of judgment theory
• The focus in this situation is on the quality of work and how
effectively work is organised
• The dilemma for the candidate is how to engage in this
situation to address these issues
• The response alternatives are structured around a
behavioural model of choices for resolving this situation
©2011 SHL Group Limited
32
Constructing answer keys
• The answer alternatives are written to reflect different
behaviours
• Those behaviours have been defined to reflect key
competencies in the client development programme
• Answer choices provide data on whether judgments are
likely to be in line with the client’s competencies
a.
Explain to your team why these procedures are important and
what the consequences are of not performing these correctly.
b.
Try to arrange for your team to observe another team in the
organisation who produce high quality work.
c.
Check your own work and that of everyone else in the team to
make sure any errors are found.
d.
Suggest that the team tries many different ways to approach
their work to see if they can find a method where fewer mistakes
are made.
©2011 SHL Group Limited
33
Systematic process (1)
• Step 1: Start with a focus on the critical behaviours
– Mappings to competencies and specific behaviours ensure that there
is a clear theory of behaviour built into the assessment
• Step 2: Ensure that there is a clear measurement
model underpinning the content
– Target behaviours are mapped to specific qualities of people so we
know what is being measured and how
• Step 3: Make sure it makes sense to key
stakeholders
– Through a simple and efficient process, ensure that managers and
other stakeholders understand the relevance of the answer key to
effective performance
©2011 SHL Group Limited
34
Systematic process (2)
• Step 4: Check that the answer key fits the theory
– We test this thoroughly through subject matter expert (SME) reviews
early in the construction process
• Step 5: Validate the performance of the items and
their fairness
– The mathematical properties of items are evaluated empirically and
checked for equivalence across different demographic groups
(gender, age and ethnicity)
• Step 6: Address language and cultural issues from
the start
– Through our localisation processes and translation partners, we are
able to assess language and culture issues early in the programme
– Equivalent mathematical properties across languages is part of the
empirical evaluation process
©2011 SHL Group Limited
35
Developing an SJT
1.
Construct
Design
Learning about
the role
2.
Content
Creation
Generating the
scenarios
Establishing the
response options
3.
Trialling
Rating the
response options
Trialling the final
item models
4.
Data Analysis
5.
Quality
Assurance
6.
Handover
Analysis
Handover & QA
©2011 SHL Group Limited
36
Imagine you are asked to:
Design 1,625 SJT items
125 items delivered every 6 weeks
That map to five competencies
And have to operate in 27 languages
©2011 SHL Group Limited
37
Building blocks for the solution
Leading &
Deciding
Deciding &
Initiating
Action
Leading &
Supervising
Supporting &
Cooperating
Working with
People
Adhering to
Principles &
Values
Interacting &
Presenting
Relating &
Networking
Persuading &
Influencing
Presenting &
Communicating
Analysing &
Interpreting
Writing &
Reporting
Applying
Expertise &
Technology
Analysing
Creating &
Conceptualising
Learning &
Researching
Creating &
Innovating
Formulating
Concepts &
Strategies
Organising &
Executing
Planning &
Organising
Delivering &
Meeting
Expectations
Following
Instructions &
Procedures
Adapting &
Coping
Adapting &
Responding to
Change
Coping with
Pressure
Enterprising &
Performing
Achieving
Goals &
Objectives
Entrepreneurial
& Commercial
Thinking
Universal
Competency
Framework
©2011 SHL Group Limited
38
Competencies covered
• Analysis and problem solving: Identifies the critical facts in
complex issues and develops creative and practical solutions
• Prioritising and organising: Prioritises the most important
tasks, works flexibly and organises own workload effectively
• Delivering quality and results: Takes personal responsibility
and initiative for delivering work to a high standard of quality
within set procedures
• Resilience: Remains effective under heavy workload, handles
organisational frustrations positively and adapts to a changing
working environment
• Working with others: Works co-operatively with others in
teams and across organisational boundaries, and respects
differences between people
©2011 SHL Group Limited
39
Mapping to client’s competencies
Leading &
Deciding
Supporting &
Cooperating
Working With
Others
Interacting &
Presenting
Analysing &
Interpreting
Analysing & Problem
Solving
Creating &
Conceptualising
Organising &
Executing
Prioritising &
Organising
Adapting &
Coping
Resilience
Enterprising &
Performing
©2011 SHL Group Limited
Delivering Quality
& Results
Delivering Quality
& Results
40
Behavioural blueprint for items
Behaviour matrix
Analysing & Problem Solving
EPSO Competency
Making Judgements
UCF Behavioural
Component
Positive
indicator
Negative
indicator
Takes all available
information into account
when making judgements
Makes quick judgements
based upon incomplete
information
Passage with statements
©2011 SHL Group Limited
41
We will now look at an example
• Analysis and problem solving: Identifies the critical facts in
complex issues and develops creative and practical solutions
• Prioritising and organising: Prioritises the most important
tasks, works flexibly and organises own workload effectively
• Delivering quality and results: Takes personal responsibility
and initiative for delivering work to a high standard of quality
within set procedures
• Resilience: Remains effective under heavy workload, handles
organisational frustrations positively and adapts to a changing
working environment
• Working with others: Works co-operatively with others in
teams and across organisational boundaries, and respects
differences between people
©2011 SHL Group Limited
42
Delivering quality and results
You work as part of a technical support team that produces
work internally for an organisation. You have noticed that
often work is not performed correctly or a step has been
omitted from a procedure.
You are aware that some
individuals are more at fault than others as they do not make
the effort to produce high quality results and they work in a
disorganised way.
a.
Explain to your team why these procedures are important and
what the consequences are of not performing these correctly.
b.
Try to arrange for your team to observe another team in the
organisation who produce high quality work.
c.
Check your own work and that of everyone else in the team to
make sure any errors are found.
d.
Suggest that the team tries many different ways to approach
their work to see if they can find a method where fewer mistakes
are made.
©2011 SHL Group Limited
43
Qualities of SJTs deployed
• 40 equivalent forms deployed for graduate entry and
40 for administrative entry
• Test lengths of 20 items with 4 items per competency
• From data on 38,000 applicants processed in 2010
• Internal consistencies of 0.8 across all forms
• Means and standard deviations within standard errors
(i.e. equivalent)
• Overall, use of the SJTs seen to improve the fairness
of the selection process
• Improved gender balance is applicants called forward
for final assessment centre stage
• Positive reactions from applicants
©2011 SHL Group Limited
44
Applying this approach to
customer roles
©2011 SHL Group Limited
45
Of these competency dimensions
Leading &
Deciding
Deciding &
Initiating
Action
Leading &
Supervising
Supporting &
Cooperating
Working with
People
Adhering to
Principles &
Values
Interacting &
Presenting
Relating &
Networking
Persuading &
Influencing
Presenting &
Communicating
Analysing &
Interpreting
Writing &
Reporting
Applying
Expertise &
Technology
Analysing
Creating &
Conceptualising
Learning &
Researching
Creating &
Innovating
Formulating
Concepts &
Strategies
Organising &
Executing
Planning &
Organising
Delivering &
Meeting
Expectations
Following
Instructions &
Procedures
Adapting &
Coping
Adapting &
Responding to
Change
Coping with
Pressure
Enterprising &
Performing
Achieving
Goals &
Objectives
Entrepreneurial
& Commercial
Thinking
©2011 SHL Group Limited
46
These were selected
Supporting &
Cooperating
Working with
People
Interacting &
Presenting
Relating &
Networking
Delivering &
Meeting
Expectations
Organising &
Executing
Adapting &
Coping
©2011 SHL Group Limited
Adhering to
Principles &
Values
Adapting &
Responding to
Change
Coping with
Pressure
47
An example for customer roles
A customer has accidentally broken a bottle of oil.
There is broken glass and the oil is quickly spreading
across the floor. One of your colleagues is helping the
customer. You still have to re-fill a lot of shelves before
your break.
Watch your colleague for a moment to see if your help is
needed.
Help your colleague deal with the situation by calling the
cleaning staff.
Stay focused on completing your current task of re-filling
the shelves in time.
©2011 SHL Group Limited
48
Qualities of SJT deployed
• Single form with 17 items providing an omnibus
assessment across critical competencies
• Internal consistency of 0.72
• Deployed in online in UK (first deployments) and US,
and paper-and-pencil format in Turkey
• Positive reactions from applicants
• Client calculated that the SJT reduced the cost of
opening a new store in the UK by £1.2 million
• Reduced the number of interviews to successful hires
from 6:1 to 2:1
• This is an estimated reduction of 73,000 hours of
management time
©2011 SHL Group Limited
49
Differences in employee quality
70
60
Lowest
scorers on
the SJT
5 : 1 being
lowest
performers
on the job
50
40
Highest
scorers on
the SJT
12 : 1
being
highest
performers
on the job
30
20
10
0
Lowest Quartile SJT
Lower Quartile Employee
©2011 SHL Group Limited
Highest Quartile SJT
Upper Quartile Employee
50
Cross validation
UK
Concurrent
(N=78)
UK
Predictive
(N=70)
Turkey
Concurrent
(N=93)
US
Concurrent
(N=101)
Sample
Weighted
Average
(N=342)
Working with People
0.35
0.38
0.31
0.25
0.32
Adhering to Principles & Values
0.28
0.18
0.55
0.21
0.32
Relating and Networking
0.37
0.52
0.36
0.29
0.38
Meeting Customer Expectations
0.46
0.28
0.22
0.26
0.30
Adapting to Change
0.38
0.19
0.26
0.32
0.29
Coping with Pressure
0.38
0.18
0.43
0.10
0.28
Overall Job Performance
0.47
0.40
0.36
0.28
0.37
Criterion Measure
©2011 SHL Group Limited
51
The practical and investment
decisions in implementing an
SJT
©2011 SHL Group Limited
52
Why use an SJT?
©2011 SHL Group Limited
53
Is the candidate also a customer?
• Consider the case of a retailer or service provider
• Consider the case of recruitment to a front line
operational role and volume recruitment
• The applicant may be an existing or potential future
customer
• Is the candidate experience going to be important
• Would a highly credible and realistic assessment help
manage the dual needs of a measure that is predictive
and promotes the company brand?
©2011 SHL Group Limited
54
As we saw earlier …
Knowledge
Instructions (KI)
Behavioral
Tendency (BT)
Cognitive ability
0.35
0.19
Conscientiousness
0.24
0.34
Agreeableness
0.19
0.37
Emotional Stability
0.12
0.35
From McDaniel et al., 2007
Whether it is a KI or BT format, the correlations with
cognitive ability and personality are modest
©2011 SHL Group Limited
55
Global professional services firm
Cognitive
Ability
Can Do
Personality
Will Do
SJT
Will Fit
Use all three types of assessment to provide a
holistic and fair basis for graduate hire
©2011 SHL Group Limited
56
A “dialogue” with the candidate
• Think of a client who is competing for the very best
graduates…
• … who wants to engage a candidate…
• … and ensure they complete the selection process…
• Imagine being able to show a candidate what it is
like to work in your organisation…
• … what the values are of your organisation…
• … and it is closely aligned to your brand…
• An SJT can do this…and it can ensure the candidates
that end up in your organisation display the right
behaviours too…
©2011 SHL Group Limited
57
Where SJTs add specific value
• Makes the selection process more credible in terms of
the perceived link between the assessment and the
job
• Strengthen the read of a candidate by supplementing
other measures such as cognitive and personality
measures
• Extending the utility to a tool such as an RJP in order
to engage candidates and keep them motivated
throughout the process
• Tool to communicate organisational values, culture
and brand
©2011 SHL Group Limited
58
Creating a dialogue framed by
what you expect them to do
SJT approach applied to a Realistic Job Preview (RJP)
©2011 SHL Group Limited
59
Helping the candidate decide (1)
• A big 4 consultancy hires grads in to 5 main areas of
the business
• Audit
• Consulting
• Financial Advisory
• Tax
• Enterprise Risk Services
• Challenge they are facing is that graduates are
applying for the more ‘exciting’ areas and are more
suited to other areas
• It needs to be deployed via a mobile phone
• How can an RJP help?
©2011 SHL Group Limited
60
Helping the candidate decide (2)
• One RJP
• 4 items per business area, 20 overall
• Overall fit score that provides feedback to the
candidate on which area they are most suited to…
Good Fit
Great answers! Your responses to these initial questions suggest
that you are likely to be a relatively good fit to our Financial
Advisory business function! You seem to understand what is
important to perform well in roles in this business function and this
indicates that you have potential to do well in this area. Therefore,
we would be delighted if you continued your application to the
Financial Advisory teams. For more information about vacancies,
please visit (insert link). Thank you for your interest in (client)
careers.
©2011 SHL Group Limited
61
Which candidates are committed
to you?
No Shows At Interview Stage
100
90
80
70
60
50
50%
40
30
20
10
10%
0
No Shows Before RJP
©2011 SHL Group Limited
No Shows after RJP
62
Let’s go through some other issues to
consider and decisions to be made
©2011 SHL Group Limited
63
Consider the case …
• Consider the case of a multinational company
• Assessments to support consistent talent acquisition
need to be run in several countries, languages and
cultures
• One Size Will Fit All: Do they develop their SJT in one
country and language and then export it (force it) into
the other languages?
• Everyone Is Different: Do they develop different SJTs
in each language?
©2011 SHL Group Limited
64
And if you are going online, they need
to be fake resistant and secure as well
©2011 SHL Group Limited
65
Technology and SJTs
©2011 SHL Group Limited
66
Text based
A customer has accidentally broken a bottle of oil.
There is broken glass and the oil is quickly spreading
across the floor. One of your colleagues is helping the
customer. You still have to re-fill a lot of shelves before
your break.
Watch your colleague for a moment to see if your help is
needed.
Help your colleague deal with the situation by calling the
cleaning staff.
Stay focused on completing your current task of re-filling
the shelves in time.
The measurement model must work whatever the medium
©2011 SHL Group Limited
67
More immersive versions
Swarovski_5r2. wmv
Driver video for ppt. wmv
©2011 SHL Group Limited
68
©2011 SHL Group Limited
69
Level of Fidelity
How much investment?
Video
3D animation
2D animation
Text based
©2011 SHL Group Limited
70
Bang for your buck
Level of Fidelity
How much investment?
Video
3D animation
2D animation
Text based
©2011 SHL Group Limited
71
Let’s walk through two client projects
©2011 SHL Group Limited
72
©2011 SHL Group Limited
73
Case study 1 – Marks and Spencer
• Marks and Spencer are a global retailer
• Text based SJT existed as part of the original process
• Key challenge to them was to provide a candidate
experience that is engaging and promotes their
brand - candidates are customers too
• Key requirements for the solution:
– Easy to roll out in other languages
– Long term investment
– Consistent with and enhance their brand
©2011 SHL Group Limited
74
Case study 1 – Marks and Spencer
• Marks and Spencer are a global retailer
• Text based SJT existed as part of the original process
• Key challenge to them was to provide a candidate
experience that is engaging and promotes their
brand - candidates are customers too’
• Key requirements for the solution:
– Easy to roll out in other languages
– Long term investment
– Consistent with and enhance their brand
Decision = 2D animation
©2011 SHL Group Limited
75
Pros & cons of solution
• Pros
• Easy to localise
• Less likely to age over time
• Engaging experience for candidates
• Cons
• May not be seen by all clients to fit their brand image
©2011 SHL Group Limited
76
Case study 2 – global retail bank
• Key challenge – how to attract the best graduates in
a sector that is very competitive
• Requirements
• Engaging candidate experience
• Initial roll out in the UK
• Different to competitors
• What happened when the brand police were
engaged?
• Client wants to be portrayed as safe and reliable in the
current climate
• Animation deemed too ‘risky’
©2011 SHL Group Limited
77
Case study 2 – global retail bank
• Key challenge – how to attract the best graduates in
a sector that is very competitive
• Requirements
• Engaging candidate experience
• Initial roll out in the UK
• Different to competitors
• What happened when the brand police were
engaged?
• Client wants to be portrayed as safe and reliable in the
current climate
• Animation deemed too ‘risky’
Decision = Video
©2011 SHL Group Limited
78
Pros & cons of solution
• Pros
• Real people which present a more professional brand
• Higher fidelity
• Cons
• Higher investment to roll out in other languages
• Look and feel can date more quickly
©2011 SHL Group Limited
79
It is all about choice
Text
2D
Animation
3D
Animation
Video
Development Costs
♦
♦♦
♦♦♦
♦♦♦♦
Development Time
Requirements
Localization Costs
♦
♦♦
♦♦♦
♦♦♦♦
♦
♦♦
♦♦♦
♦♦♦♦
Level of Realism
♦
♦♦
♦♦♦
♦♦♦♦
Validity
♦♦♦
♦♦♦♦
♦♦♦♦
♦♦♦♦
Lack of Adverse Impact
♦♦♦
♦♦♦♦
♦♦♦♦
♦♦♦♦
♦♦
♦♦♦
♦♦♦♦
♦♦♦♦
Consideration
Applicant Reactions
©2011 SHL Group Limited
80
Considerations for the client
• What are we trying to achieve?
• This isn’t just about predicting job performance, this is about
brand and candidate experience
• What is your candidate demographic and how does
this need to be represented?
• What countries/languages will this need to be
deployed in?
• What is the infrastructure in all the countries involved?
• Which solution will best represent your organisation
• How branded does it need to be – how closely aligned to your
values and culture? Is this a key driver?
• Who are the key internal stakeholders that need to
be engaged and how do we engage them?
©2011 SHL Group Limited
81
Part 3
A walk through the key items in
designing, building and deploying
an SJT
©2011 SHL Group Limited
82
Why would an SJT be attractive
to your organisation or your client?
What would it offer over the
assessments already in place?
What would the SJT need to demonstrate
and achieve over and above psychometric
qualities (e.g. reliability, validity)?
©2011 SHL Group Limited
83
Who would the SJT be designed for?
What roles in the organisation need
to be covered?
What is the demographic of the target
applicant population?
©2011 SHL Group Limited
84
What are the key deployment issues
that would need to be addressed?
What level of fidelity would best support
the employer brand?
What infrastructure issues would need to
be taken into account?
©2011 SHL Group Limited
85
Who are the key stakeholders who
need to be engaged?
What is their stake in the programme?
What is their role in the decisions to be made?
©2011 SHL Group Limited
86
What would success look like?
What metrics would show value to the user
(efficiency and effectiveness)?
What other objectives would show that
the SJT has delivered value?
©2011 SHL Group Limited
87
What other issues need to be
taken into account?
©2011 SHL Group Limited
88
Thank you
[email protected]
[email protected]
You will also find us on
©2011 SHL Group Limited
89