Working goalbased with
IEC 61508-3
Tor Stålhane
NTNU, IDI
Introduction
There are several ways to approach software
development according to IEC 61508-3 in a goalbased manner:
• The common sense model
• Assurance case / Safety case
• The IMO model
• Interpreting the tables in annex A and B in a goal
based manner – according to
– IEC 61508-7 aims
– Company specific activities
Goal-based arguments in IEC 61508-3
Technique / Measure
Ref. SIL 1 SIL 2 SIL 3
-
R
R
SIL 4
HR
Goal – what do we
want to achieve?
Focus should be on what we
want to achieve, not on what
we will do to achieve it
This goal is
met because…
Why use a goal-based approach
The standard will be independent of changes in
technology – methods and techniques =>
• We can chose whatever approach we want as
long as we reach the defined goals.
• We can start using new methods as soon as
– they become available
– we have done the necessary testing and
evaluation
Introducing new methods
• The standard should contain a set of
requirements for the acceptance of new methods
– e.g. a “method proven in use” argument.
• Software development organizations need to
become more resilient:
– more focused on the goals
– less focused on
• rules and regulations
• the methods and techniques considered best practice now
• On the other hand – ignoring rules and
regulations is not so smart either
The common sense model
SUSS’ opinion:
All software development requirements in IEC
61508-3 can be described as a set of methods
and techniques used to realize a set of sound
software engineering practices
Common sense standard – example
Current standard: 6.1.3 [Test] Output documents:
•
•
•
•
•
•
•
•
Overall Software Test Specification
Overall Software Test Report
Software Integration Test Specification
Software Integration Test Report
Software/Hardware Integration Test Specification
Software/Hardware Integration Test Report
Software Component Test Specification
Software Component Test Report
Our point of view: 6.1.3 [Test] Output documents:
The output documents should have all the info needed
to show that sufficient testing has been done
Example of requirements – goal mapping
Table B.2 – Dynamic analysis and testing
(Referenced by Tables A.5 and A.9)
1
Test case execution from boundary value
analysis
4
Test case execution from model-based test
case generation
7a Structural test coverage (entry points) 100 %
7b Structural test coverage (statements) 100 %
7c Structural test coverage (branches) 100%
Suggested goal
Testing should be sufficient
for the designated SIL level
Common sense - Pros and cons
• Pros
– Easy to understand
– In line with the way most developers work – do as
good a job as time permits
• Cons
– The standard should decide what is necessary for each
SIL value.
– Who shall decide the sufficient amount of testing for a
given SIL value?
Assurance / Safety case based
In this approach we
• Build an assurance or safety case – what should
be done to assure that a set of requirements are
met
• Identify the tools, methods and techniques
identified by the assurance case
• Use the identified tools, methods and techniques
• Provide proof of compliance for the tools,
methods and techniques
Testing vs. Assurance / Safety Case – 1
Testing vs. Safety Case – 2
Assurance based – Pros and cons
• Pros: We can
– Create large parts of the assurance cases without describing
tools, methods and techniques
=> large opportunities for reuse
– Freely select the methods and techniques needed to satisfy the
goals
– Freely select tools as long as they satisfies the standards
requirements
• Cons
– A assurance case will be large and difficult to understand =>
difficult to use
– Need templates for this but it has been difficult to construct
good ones
– The assessor must be involved - at least to accept the assurance
case
The IMO model
IMO – International Maritime Organization – is
one of the organizations that has put
considerable effort into moving in a goaloriented direction.
Their ideas are described in the five-tier model.
The first three tiers make up the goal based:
• safety requirements
• functional requirements
• verification of compliance criteria
IMO goal-based
regulatory
framework
15
The assessor as a broker
The goal based standard
What we need to achieve
The assessor - broker
We want to do
this to meet goal X
Used to define
company activities
Developer company
Yes – that’s OK
or
No – this is not OK
The IMO approach – Pros and cons
• Pros
– Leave important decision to be sorted out in
negotiations between developers and assurance
organizations
– Once agreed on, the results can be reused
• Cons
– The IMO approach does not use SIL – it is either
safe or unsafe
=> needs to construct goals for each SIL
Based on aims from IEC 61508-7
IEC 61508-7 defines one or more aims for each
requirement in IEC 61508-3. Thus, we could
focus:
• More on the specified aims
• Less on the required techniques and methods
This will move us considerably nearer a goal
based standard without much ado.
Requirements and aims – example
Table A.7 – Software aspects of system safety validation
Activity
IEC 61508 – 7 aims
Probabilistic testing
Not applicable for SIL 3
Simulation
To test the function of a
software system, together
with its interface to the
outside world, without
allowing it to modify the real
world in any way.
Modelling
B5
Functional and black-box testing
B3
Forward traceability between the
Software Safety Requirements
Specification and the software
safety validation plan.
To maintain consistency
Backward traceability between the between lifecycle stages
software safety validation plan and
the Software Safety Requirements
Specification.
Company approach
Requirements and aims – Pros and cons
• Pros
– The goals are already available in IEC 61508-7
• Cons
– The descriptions are not always precise enough
– Needs interaction with assessor to accept possible
refinements of the goals
Discussion topics – 1
All suggestions requires more involvement from
the assessor:
• Common sense: agree on what is sufficient
• Assurance case: accept or reject the assurance
case
• IMO: act as broker between standard and
developers
• Aims in part 7: interact to refine/define more
precise goals
Discussion topics – 2
• Current problem – from the SASSUR workshop:
– assessment might be reduced to just filling in
checklists =>
– “dumbing down” assessment – FAA assessor’s opinion
• New problems:
– More work for the assessor – discussions and
meetings
– More difficult for new companies to enter the domain
of safety critical software
IEC 61508 SW development project
Part 3: Annex
A+B
Extract
SW aims
Company strategy:
System, product
Backlog
Decide which
T&M to be
applied
Aim
Part 7
Company aim (if
beneficial) and
implementation
To be discussed
with the
assessor
To be discussed
with the
assessor
Tool strategy
This information is often included in one single document
e.g. RAMS
responsible
What have we chosen – and why
Main requirements: must be easy to
• adapt for SafeScrum
• use by the company – testers and developers
• adapt to IEC 61508-3
Based on these requirements we recommend
The aims in IEC 61508-7 as goals for the process
requirements in annexes A and B in IEC 61508-3.