1. technology and computing
2. software
3. databases

Software Quality Journal manuscript No.
(will be inserted by the editor)
Accelerating Temporal Verification of Simulink
Diagrams Using Satisfiability Modulo Theories
Petr Bauch · Vojtˇ
ech Havel · Jiˇ
r´ı Barnat
Received: date / Accepted: date
Abstract Automatic verification of programs and computer systems with
input variables represents a significant and well-motivated challenge. The case
of Simulink diagrams is especially difficult, because there the inputs are read
iteratively and the number of input variables is in theory unbounded. We apply
the techniques of explicit model checking to account for the temporal (control)
aspects of verification and use set-based representation of data, thus handling
both sources of non-determinism present in the verification. Two different
representations of sets are evaluated in scalability with respect to the range of
input variables. Explicit (enumerating) sets are very fast for small ranges but
fail to scale. Symbolic sets, represented as first-order formulas in the bit-vector
theory and compared using satisfiability modulo theory solvers, scale well to
arbitrary (though still bounded) range of input variables.
To leverage the combined strengths of explicit and symbolic representations,
we have designed a hybrid representation which we showed to outperform both
pure representations. Thus the proposed method allows complete automatic
verification without the need to limit the non-determinism of input. Moreover,
the principle underlying the hybrid representation entails inferring knowledge
about the system under verification, which the developers did not explicitly
include in the system, and which can significantly accelerate the verification
process.
The research leading to these results has received funding from the European Union’s
Seventh Framework Program (FP7/2007-2013) for CRYSTAL – Critical System Engineering
Acceleration Joint Undertaking under grant agreement Number 332830 and from specific
national programs and/or funding authorities.
Petr Bauch, Vojtˇ
ech Havel, and Jiˇr´ı Barnat
Faculty of Informatics, Masaryk University,
Botanick´
a 68a, Brno, Czech Republic
Tel.: +420 549 491 869
E-mail: {xbauch,xhavel1,barnat}@fi.muni.cz
2
Petr Bauch et al.
Keywords Formal verification · model checking · circuit analysis · satisfiability
modulo theories
1 Introduction
The primary motivation of this paper is the verification of Simulink diagrams
against specifications given as linear temporal logic (LTL) formulas. The methodology we propose is more general and can accommodate a wider class of systems
and models, but the research is guided by this concrete application. Consider
the diagram in Figure 1.
A Simulink diagram is an oriented graph, which can be interpreted as
an abstract syntactic tree, except it may contain self-referencing cycles. The
expression associated with a node is described by the tree underneath it, e.g.
in our diagram the next value of the delay d1 is computed as the multiplication
of the two subtrees of the node labeled by ∗ to the left of the d1 node1 . Other
than the arithmetic and logical operators there are 4 types of nodes: delays
(box), inports (circle), outports (ellipse), and constants (plain). Only delays are
self-referential and the semantics is that of a single element memory storage:
whenever referred to, the delay node provides the stored value. That stored
value d is initiated to 0 and is replaced by d0 with every tick of the Simulink
global clock.
Since the inport nodes serve as inputs – providing a new, non-deterministic
value after every tick – a Simulink diagram generates a transition system of
the possible values stored in delays. Such a transition system represents the
possible evolution of the diagram, describing its behavior as it changes in time.
Similarly as with parallel and reactive programs, a developer may need to
verify that the behavior of a system complies with the specification. Consider
for example a specification requiring that: whenever the inport sv1 is negative
in three consecutive ticks, the value of pm2 will eventually be positive. Such a
specification cannot be expressed in terms of safety, i.e. as a reachability of bad
states, because if the system were incorrect the violating run would be infinite.
Consequently, verification procedures that do not allow temporal specification,
such as most implementations of symbolic execution by King (1976), static
analysis by Cousot and Cousot (1977) or deductive verification by Owicki and
Gries (1976), cannot be used in this case. Model checking, initiated by Clarke
et al (1986); Vardi and Wolper (1986), allows verification against temporal
properties, yet Simulink diagrams and similar systems with input variables
pose another obstacle in the form of data-flow non-determinism.
In order to answer temporal verification queries automatically, an algorithm
must allow evaluation of Peano arithmetic expressions that form subtrees in
the diagram. At the same time the input variables may be evaluated to an
arbitrary number (which in practice is bounded, e.g. by the number of bits used
to express the value) and each of these evaluations must be considered for the
1 Translated into a full equation, the value of d
1 in the next tick equals
d01 = min{c3 , (¬sv1 ∨ c1 > sd13 )?c4 : d1 } ∗ int(¬sv1 ∨ c1 > sd13 ).
+
ite
Title Suppressed Due to Excessive Length
pm1
≥
min
min
c3
≥
3
pm3
sd inport
+
ite
d delay
≥
pm2
min
ite
∗
d1
pm
c4
sm2
sv1
cnv
∨
¬
sd13
>
c1
outport
Fig. 1: A partial Simulink diagram for the VoterCore system: the complete diagram was
verified during the experimental evaluation (see Section 5.2). Delay nodes form the state
space, inport nodes generate new evaluations with every tick, and outport nodes can be
referred to in the specification requirements.
verification to be exhaustive. Standard symbolic model checking approaches are
either restricted to only a subset of Peano arithmetic, see Bultan et al (1997),
or are concerned with Boolean input variables, as demonstrated by McMillan
(1992). Standard explicit approaches to model checking already suffer from
the state space explosion caused by the control-flow non-determinism and
thus cannot scale to any considerable degree with the allowed ranges of input
variables, as we showed in Barnat et al (2012).
Hence the motivation for LTL verification of Simulink diagrams: it has
the potential to replace testing in the design phase of software (or systems)
development. In this sense, our work in Barnat et al (2012) succeeded only in
part. In a tool-chain, and preceded by a tool to formalize pattern-based, human
language requirements into LTL formulas, a model checker could provide similar
results as testing for considerably smaller expenses. Yet the severely limited
domains of input variables the standard model checking allows still enforce a
test-like approach to correctness validation. Effectively, the developer still has
to mechanically test (a smaller number of) instances of the same validation
query with different (intervals of) input values.
This paper meliorates the limitations of previous approaches by proposing
a model checking technique that scales to arbitrarily large domains of input
variables. Through the reduction of certain crucial operations required by the
model checking process to Satisfiability Modulo Theories (SMT) queries, we
have achieved a method for complete verification (for large but finite domains).
Though the experiments were conducted on a relatively small Simulink model,
they still convey the most important practical result: appropriately adapted
model checking may be used to replace testing of safety-critical units as early
as in the modeling phase of software development.
4
Petr Bauch et al.
2 Contribution
We have first formalized the process of model checking of Simulink diagrams,
building on the concept of set-based reduction of the underlying state space,
proposed in Barnat and Bauch (2013). This reduction can significantly decrease
the number of states, but does not lead to any reduction in the worst case.
Also the representation used for the sets of variable evaluations determines the
resulting complexity, which can differ exponentially for particular representations. We have implemented the set-based reduction using explicit sets as one
representation and using bit-vector formulas as another representation. Explicit
sets are exponential in both the number of variables represented and in their
ranges, but allow simple implementation of state matching, i.e. deciding if two
states represent the same evaluations. The representation using formulas is
exponentially more succinct, but the state matching is a complex operation
that requires deciding satisfiability (SAT) of quantified bit-vector formulas.
Finally, the results of this paper show that (1) in the case of Simulink
diagrams the set-based reduction is efficient, leading to exponentially smaller
states spaces; (2) the representation using explicit sets allows fast verification
but only for small ranges of variables; and (3) the representation using bit-vector
formulas scales to arbitrary (bounded) ranges, though the state matching is
resource expensive. In the terms of practical results, the proposed SAT-based
verification is fully automated. Provided that the LTL specification and the
Simulink model are available, the developer can demonstrate the correctness
without any additional manual effort.
This paper is an extended version of Barnat et al (2014) in the following
aspects. We have expanded the related work section, comparing our work with
the areas of module checking and symbolic transition graphs and we have also
provided an extensive list of other methods combining explicit and symbolic
approaches. Preliminaries to the theory of automata-based model checking
are now less dense, to ease understanding for wider audience. Similar reasons
lead to the addition of a Simulink diagram to Example 2: the reader can now
follow its formalization into bit-vector formulas. The proof of correctness of
the equality computation was added in full length, and there are also two new
paragraphs describing the complexity aspects of the proposed algorithm. In
order to further improve repeatability of our experiments we have extended
the section on implementation with two examples of the generation of SMT
files: the first is more general and leads to a query assessing the satisfiability
of the path condition, the second then demonstrated how state equality via a
quantified query can be encoded. Section 6 was added completely and presents
further insight into the SAT-based representation and suggests what is the
cause for its high computational complexity.
Most importantly, Section 4.3 contains detailed description of the hybrid
representation and the methodology behind combining explicit and SAT-based
representations, which was in no part included in the conference version. Consequently, Section 5.2.3 is also completely new, and describes the experimental
evaluation of the hybrid representation. Also Section 5.1 describing implemen-
Title Suppressed Due to Excessive Length
5
tation and the Conclusion have been extended to describe the aspects of the
hybrid representation relevant for these sections. The hybrid representation
incorporates both the explicit and the symbolic representations, thus allowing
to utilize the hidden limits of input values if these are present but can also
cope with systems without anyhow restricted inputs. We consider this novel
methodology to be the most important added value of this publication.
3 Preliminaries
We assume the theory of bit-vectors (BV), a first-order theory where a bit width
q is associated with every term. The universe of this theory is that of natural
numbers whose binary representation uses at most q bits; we will denote this
domain as bvq. The signature of BV subsumes Peano arithmetic – all operations
are computed modulo q, as is common in many programming languages – and
it also contains bit-level operations. The ability to precisely reason about
computer arithmetic and the existence of efficient decision procedures are the
two main reasons why BV is popular in automating both hardware and software
verification. In the following we will assume notation similar to that of Kroening
and Strichman (2010), i.e. a first-order evaluation ν maps a predicate p to a
Boolean value: ν |= p; a function f is evaluated to ν[f ]; finally, ν / {x 7→ y} is
ν except the variable x is now mapped to y.
Let X be a set of bvq variables. Then we can define the LTL recursively as
follows:
Ψ ::= pX | ¬Ψ | Ψ ∧ Ψ | X Ψ | Ψ U Ψ,
where pX is a BV predicate over the variables from X . A parallel program with t
threads and with inputs, bvq variables I, can be modeled as a transition system
M = (S, →, s0 ). There the states S are triplets ((l1 , . . . , lt ), ν : X → bvq, pX ∪I ),
li are program locations of individual threads, ν is a BV valuation of variables
and pX ∪I a path condition, i.e. a predicate stating the condition that would
lead the execution to this state. The transition relation →⊆ S × Act × Con × S,
where Act = {(x0 , fX ∪I ) | x ∈ X } (denoting an update to x) and Con is the
(x0 ,fX ∪I )
set of predicates over X , will be denoted as s −−−−−−→ s0 .
pX
Let πi be the standard projection functions, then a run of M is an in(x0 ,f )
finite sequence w = s0 , s1 , . . ., where for all i ∈ N: si −−−−→ si+1 and for
p
pˆ = p[x/π2 (si )(x)] it holds that π3 (si+1 ) = π3 (si ) ∧ pˆ is satisfiable; finally we
need to change the valuation of x: π2 (si+1 ) = π2 (si )/{x 7→ π2 (si )[f ](π2 (si )(x))}.
A run w satisfies an LTL formula Ψ , w |= Ψ :
w
w
w
w
w
|= p
|= ¬Ψ
|= Ψ1 ∧ Ψ2
|= X Ψ
|= Ψ1 U Ψ2
iff
iff
iff
iff
iff
π2 (w(0)) |= p,
w 6|= Ψ,
w |= Ψ1 and w |= Ψ2 ,
w1 |= Ψ,
∃i∀j < i : wj |= Ψ1 , wi |= Ψ2 .
6
Petr Bauch et al.
Where w(i) is the i-th state of w and wi = si , si+1 , . . . the i-th suffix. Every
LTL formula Ψ can be translated into a B¨
uchi automaton AΨ = (S, →, F ),
where →⊆ S × Con × S and F ⊆ S is the set of accepting states. For AΨ it
holds that a run w of M satisfies Ψ if there exists a run w0 = s01 , s02 , . . . in AΨ
such that ∀i ∈ N, s0i −
→ s0i+1 we have π2 (si ) |= p and infinitely many states of
p
w0 are in F . One can construct a product of M and AΨ , where the only runs
are those satisfying Ψ . (Then the states are still triples only the sequence of
program locations is longer by one, lt+1 denoting the state of AΨ .) It follows
that deciding whether M satisfies Ψ can be reduced to locating cycles with
accepting states in such a product.
Note that under this definition, which is very similar to that of Lin (1996),
both sources of non-determinism that occur in parallel programs – the interleaving of threads and the evaluation of input variables – are represented explicitly.
In symbolic execution, initiated by King (1976), the transition system is unfolded (acyclic and infinite), S is thus a multi-set and the second component of
a state is a set of possible evaluations, represented symbolically (every x ∈ X
is associated with a BV function over I). In Barnat and Bauch (2013) we have
proposed a set-based reduction of M, where two states are equal if they have the
same sequence of program locations and their sets of evaluations are the same.
Hence the set-based reduction extends the approach of symbolic execution by
allowing the detection of accepting cycles, and consequently, verification of
LTL properties.
Example 1 The figure bellow depicts a program with input on the left-hand
1:read a;
2:if(a>5)
3: a++;
4:else
5: a--;
(0, (a, 0), >)
(a0 , ι1 )
(1, (a, ι1 ), >)
a≤5
a>5
(2, (a, ι1 ), ι1 > 5)
(a , a + 1)
(3, (a, ι1 + 1), ι1 > 5)
0
(4, (a, ι1 ), ι1 ≤ 5)
(a , a − 1)
(5, (a, ι1 − 1), ι1 ≤ 5)
0
side and its associated transition system on the right-hand side. The set of
evaluations (of the one variable a) is represented symbolically, as a function
over input symbols from I, here only ι1 .
4
4 Methods
We start by formalizing the Simulink diagrams, presented in the introduction.
There are several specific details of Simulink diagrams with respect to the general programs with inputs presented above, which may simplify the verification
Title Suppressed Due to Excessive Length
7
and thus need to be pointed out in detail. Also, until specified otherwise, the
logical system used in our description is the quantifier-free fragment of BV
with the bit-width q, i.e. over sort bvq (although constants are written in the
decimal system).
A state of a Simulink system is described by the evaluation of delay variables,
X = {d1 , . . . , dm }, where their values for the next iteration are given by a
function fX ∪I , denoted as δi . There is a finite number of input variable
templates, I = {i1 , . . . , in }, which can be read iteratively, but for each there
are bit-vector constraints limiting its possible evaluation (represented by a
predicate pI , denoted as ιi ). Finally, the complete description of a system also
requires the outport functions O = {σ1 , . . . , σk } (each of which is fX ∪I ). On
the side of the specification, we will abstract the given B¨
uchi automaton to only
the predicates pX ∪I∪O labeling its transitions, Φ = {ϕ1 , . . . , ϕl }, and assume
that the transition system, the order in which the predicates occur, is given
implicitly. Hence the example below contains all the information needed for a
Simulink diagram verification query Q = (X , I, O, Φ).
Example 2 Let Q1 = (X , I, O, Φ), where
X = {d1 , d2 }
o2
δ1 : (i1 + d1 ) ∗ (i2 + d2 )
δ2 : d1
O = {σ1 , σ2 , σ3 }
ite
σ1 : max(i1 , i2 , i3 )
σ2 : ite (i1 > 7) d1 d2
σ3 : d1 + i3
d2
d1
>
7
I = {i1 , i2 , i3 }
+
∗
+
i1
i2
i3
max
o1
ι1 : 1 ≤ i1 ≤ 20
ι2 : 3 ≤ i2 ≤ 5
ι3 : 5 ≤ i3 ≤ 10
Φ = {ϕ1 , ϕ2 }
ϕ1 : σ2 > 3 ∨ i1 ≤ σ1 /2
ϕ2 : σ1 = σ2 ⇒ i2 ≤ 4
4
As mentioned above, the states of a Simulink transition system are sets of
evaluations of the variables from X . The initial state consists of a single evaluation {di 7→ 0}. Let I = {x := {ij 7→ y} | x |= ιj } be the |I|-dimensional polyhedron of allowed inport evaluations, for Q1 it would be {1..20} × {3..5} × {5..10}.
Then, computing every transition consists of three steps:
1. compute the Cartesian product (of this state) with I: results in a set of
evaluations of variables from X ∪ I;
8
Petr Bauch et al.
2. create a new state, initialized with the above set of evaluations, for every
available specification formula ϕ (there can be more that one, hence the
transition system may branch at this point) and remove those evaluations
that do not satisfy ϕ;
3. apply δ functions to compute the new evaluations of X .
Assuming that the computation is in state s, represented as the evaluation of
delay variables, and that the relevant transition is associated with a specification
ϕ, then s0 = {x := {di 7→ v[δi ](v(d), v(i))} | v ∈ s × I ∧ x |= ϕ} is the resulting
state. The system can reach a deadlock if there is no such v ∈ s × I for which
{di 7→ v[δi ](v(d), v(i))} |= ϕ. The model checking process computes the graph
of the Simulink computation, where identical states are unified, i.e. if a newly
generated state s00 (as a successor of s) is found equal to an already existing
state s0 , then s00 is discarded and s0 is acknowledged as a successor of s.
For the purpose of providing the user with counterexample traces, the
states also contain an |I|-long sequence of inport values that satisfies the
specification condition. But that is easily implementable, and will only be
mention if a significant change in the computation was necessary to incorporate
the counterexample traces.
Example 3 We continue with the query Q1 from Example 2 and simplify
the notation by establishing two transformations. First, ϕei maps a set of
evaluations of X ∪ I to that subset where every evaluation satisfies ϕi , formally
e maps a set of the same
A ⊆ bvqX ∪I 7→ {x ∈ A | x |= ϕi }. Second, ∆
evaluations to evaluations of X , after the δ functions have been applied, hence
A ⊆ bvqX ∪I 7→ {{di 7→ x[δi ]} | x ∈ A}. Hence a part of the transition system
of Q1 is as follows:
e◦ϕ
s01 = ∆
f1 (s0 × I)
s0 = {di 7→ 0}
e◦ϕ
e◦ϕ
s011 = ∆
f1 (∆
f1 (s0 × I) × I)
e◦ϕ
s02 = ∆
f2 (s0 × I)
e◦ϕ
s012 = ∆
f2 (s01 × I)
Concretely, the set s0 × I has 360 elements. ϕ
f1 (s0 × I) and ϕ
f2 (s0 × I)
have 63 and 315 elements, respectively; s01 has 12 elements and s02 has
45 elements. For example s01 = {(3, 0), (4, 0), (5, 0), (6, 0), (8, 0), (9, 0), (10, 0),
(12, 0), (15, 0), (16, 0), (20, 0), (25, 0)}.
4
The process described so far produces a transition system, where the states
are evaluations of delay variables and a location of the B¨
uchi automaton, and
the branching is given by that same automaton. Without going into details
with respect to the product of program and its specification – where we went in
the previous section – the answer to a verification query Q is equivalent to the
presence of accepting cycles in the produced transition system. The efficiency
Title Suppressed Due to Excessive Length
9
of answering a verification query, the complexity of Simulink model checking,
is most noticeably influenced by the choice of how to represent the sets of
evaluations. That is the biggest contributor to the state space explosion, since,
unlike in parallel programs, the control-flow non-determinism is negligible in
Simulink models.
4.1 Verification with Explicit Sets
The first representation considered in this paper stores the allowed evaluations
in an explicit set, enumerating all possible combinations of individual variable
evaluations. This approach is closely related to the one we proposed in Barnat
et al (2012) but improves on it in several aspects. Effectively, rather than
repetitive executions for every evaluation of both inport and delay variables,
we only represent the delay variables in an explicit evaluation; the inport
variables are stored purely symbolically, only as their constructing predicates
ιj , outside individual states. The above description of Simulink verification
semantics justifies this optimisation, since the inports are only used for the
evaluation of Φ predicates from the specification. They are discarded during
e transformation, and can be recovered using an inverse transformation if
the ∆
necessary, e.g. during counterexample generation.
The implementation itself is then a relatively straightforward extension of
the purely explicit approach. The states enumerate every allowed evaluation
and thus a separate access to each of these evaluations is permitted. Computing
the Cartesian product with I amounts to enumerating all combinations of
e transformations, one
delay and inport variable evaluations. For the ϕ
e and ∆
iteratively considers individual evaluations and removes those not satisfying ϕ
and applies the δ function, respectively. Most importantly, however, deciding
equality of states is possible on the level of syntax: two states are the same if
their representations in memory match.
Verification with explicit sets clearly retains the biggest limitation of the
purely explicit approach, i.e. the spacial complexity grows exponentially with the
ranges of inport variables. We will postpone the discussion of the comparison
between the explicit set approach and the purely explicit approach until
Section 5.2 and only remark that while the improvement is considerable, it does
not allow specification of ranges with respect to their bit-width. That ability
appears to require a symbolic representation and we continue our analysis
towards temporal verification of Simulink models with a method which uses
BV formulas and the satisfiability modulo theories (SMT) procedures for this
theory.
4.2 SAT-Based Verification
The need to iteratively read from inputs via the inport variables complicates the
otherwise relatively straightforward application of symbolic execution. Later we
10
Petr Bauch et al.
will show that the execution-based approach, where current values of variables
are represented as symbolic functions over input variables, entails another
complication in deciding state equality. In standard symbolic execution, the
variables are initialized to an arbitrary value only once, at the beginning. Thus
reading from inports requires to strengthen the standard symbolic execution
to allow computation with potentially infinite number of input variables.
Henceforth, only the inport variables are considered as first-order variables,
the delays and outports are only functions in the BV theory (over inport
variables). The inports are history-dependent and thus one may label the
symbols from I to stand for the inport variables, i.e. I 0 , I 1 , I 2 and so on.
Given that the only branching in the transition system is caused by the
particular choice of a specification transition, the states can be represented
as l-long sequences of numbers ρ = (r1 , . . . , rl ), where l is the length of this
computation. Each number in this sequence represents which specification
formula was selected in the respective branching of the computation. Indeed,
the functions representing delays δ l , outports σ l and the specification predicates
ϕl are unambiguously defined for each l recursively as follows:
δ 0 = δ[ij /i0j , dj /0],
σ 0 = σ[ij /ilj , dj /0],
ϕ0 = ϕ[σj /σj0 , ij /i0j , dj /0],
l
δ l+1 = δ[ij /il+1
j , dj /δj ];
l+1
l+1
σ
= σ[ij /ij , dj /δjl ];
l
ϕl+1 = ϕ[σj /σjl , ij /il+1
j , dj /δj ],
(1)
where f [x/y] is the formula f with every occurrence of x replaced by y. Hence
every formula at any point of the verification computation uses only the inport
variables decorated with history indices.
As mentioned above, the system reaches a deadlock if no value satisfies
the specification formula. Let the computation be in a state represented by ρ
and the outgoing transition labeled with formula ϕp . Then checking whether
this transition leads to a deadlock is equivalent to checking satisfiability of the
following recursive path condition formula, where is an empty sequence:
V
pc() = 1≤j≤|I| ι0j ,
V
|ρ|+1
|ρ|+1
pc(ρ.p) = pc(ρ) ∧ 1≤j≤|I| ιj
∧ ϕp
.
Also note that the model of this formula, the satisfying evaluation of inport
variables, is a counterexample trace (one of them to be precise) which the SMT
solver can generate while deciding the satisfiability.
Finally, to complete the verification process, one needs to be able to decide
if two states are the same. Let ρ and ρ0 be two states, of which it needs to be
assessed whether they are identical, i.e. if the sets of possible evaluations of
|ρ|
the delay variables are the same. Then the function δj represents the possible
0
values of dj in ρ, provided that |= pc(ρ) ∧ pc(ρ ), i.e. both path conditions are
satisfiable. However, the δ functions represent the current values of delays by
the means of inport variables, describing to what evaluation of delays does a
particular combination of inputs lead. The states ρ, ρ0 may not be of the same
length and thus may differ in the number of inputs. Consequently, comparing
two states amounts to comparing the images of the δ functions, i.e. comparing
Title Suppressed Due to Excessive Length
11
Algorithm 1: Successor generation
Input : storage of seen states S; state s; specification automaton AΦ
Output : S0 where all successors of s have been correctly added
ϕ
1
2
3
4
5
6
7
8
9
10
11
12
13
14
foreach ϕi , b0 such that exp(s) →i b0 ∈ AΦ do
ρ ← sym(s).i
if SAT QFBV(pc(ρ)) then
seen ← false
foreach s0 ∈ S such that exp(s0 ) = b0 do
ρ0 ← sym(s0 )
if SAT QBV(ψ(ρ, ρ0 )) then
s0 is a successor of s
seen ← true
break
if ¬seen then
exp(s00 ), sym(s00 ) ← b0 , ρ
S0 ← store(S, s00 )
s00 is a successor of s
the sets of possible evaluations. We will continue this argument in Section 6
and for now simply state the quantified version of ψ(ρ, ρ0 ) := Ψρρ0 ∨ Ψρ0 ρ – a
formula which is satisfiable iff ρ and ρ0 are two different states:
Ψρρ0 : pc(ρ) ∧
(∀y1 , . . . , y|ρ0 |∗|I| )(pc(ρ0 )[i0b
a /ya∗b ] ⇒
W|X | |ρ|
|ρ0 | 0b
δ
=
6
δ
[i
/y
]
.
a∗b
a
j
j=1 j
Put together the SAT-based verification of Simulink works as shown in
Algorithm 1 – we only describe generation of one successor of a state s, the
rest is the same as in standard model checking. That is, a particular accepting
cycle detection algorithm traverses the state space by the means of successor
generation. Given that the states already visited are known, a property that
has to be preserved by any implementation of successor generation, the model
checking process is indeed similar to the standard explicit model checking,
described for example by Vardi and Wolper (1986).
The functions in the teletype font have their expected meaning, i.e. sym
and exp return the symbolic and explicit part of the state given as their
parameter. As described in Section 3, the symbolic part is the set of evaluations
of input variables (here succinctly represented as the sequence of specification
formulas indices) and the explicit part is the control state (here only the state
of the B¨
uchi automaton is relevant). One interesting observation is that the
search for equivalent states (cycle on lines 5–10) is in standard model checking a
constant time operation, using a hashing function. But our approach prohibits
hashing because a representation using BV function is not canonical, leading
to identical states with different hashes.
The explicit part, which is canonical, is used (line 5) for a partial hashbased search, thus the body of the cycle only considers states with the same
12
Petr Bauch et al.
explicit part. Hence the inner cycle computes the linear search among states
with a common explicit part, where each comparison requires calling the state
matching procedures (line 7), i.e. deciding satisfiability of the quantified formula
ψ.
Correctness and Complexity : One thing that may not be obvious, and
which is crucial when arguing correctness is that the proposed equality of
states using the function ψ is correct with respect to the semantics of Simulink
diagrams, i.e.
∀ρ, ρ0 : ψ(ρ, ρ0 ) ⇔ sρ 6= sρ0 .
This can be easily reduced to Ψρρ0 ⇔ sρ 6⊆ sρ0 and similarly for the other
inclusion. Furthermore, Ψρρ0 ⇔ ∃x.x ∈ sρ ∧ x ∈
/ sρ0 and
|κ|
1
x ∈ sκ ⇔ ∃j11 , . . . , j|I|
, j12 , . . . , j|I| .x = d(κ, J1 , . . . , J|κ| ),
l
where Jl unrolls to j1l , . . . , j|I|
and
|X |
e◦ϕ
e g
d(<a1 , . . . , an>, J1 , . . . , Jn ) = ∆
g
, J1 }) . . . × {Jn }).
an (. . . ∆ ◦ ϕ
a1 ({0
|κ|
1
One then only needs to show that ∀j11 , . . . , j|I|
, j12 , . . . , j|I| if pc(κ)[iab /jba ] then
|κ|
|κ|
<ν[δ1 [iab /jba ]], . . . , ν[δ|I| [iab /jba ]]>= d(κ, J1 , . . . , J|κ| ).
That can be done by induction on the length of κ. For the base case we have
n+1
that for arbitrary interpretation ν, ν[δi0 ] = 0 = πi (d()). Assume j1n+1 , . . . , j|I|
are in some ν evaluated such that ϕk [ia /jan+1 ]. Then ∀1 ≤ l ≤ |X | it holds that
e◦ϕ
ν[δl [ia /jan+1 , da /πl (d(κ, J1 , . . . , Jn ))]] = πl (∆
fk (d(κ, J1 , . . . , Jn ) × {Jn+1 })),
which finally equals πl (d(κ.k, J1 , . . . , Jn+1 )).
Having established the correctness of the ψ function, i.e. equivalence among
states that represent sets of variable evaluations, the rest follows from the
correctness of the set-based reduction, which we proved in Barnat and Bauch
(2013).
The complexity of the process is much more difficult to establish. On the
one hand, the set-based reduction may, in the worst case, lead to no reduction
at all. This closely relates to computation of the fix points of operations on
program cycles, which is discussed in detail by Lin (1996); Hungar et al (1995)
among others. The reader should note, however, that in the case of the Simulink
diagram and the verified properties used in our experiments this phenomenon
did not occur. In fact the transition systems were exponentially smaller than
in the case when no reduction was used.
On the other hand, even with exponentially smaller state spaces, every
instance of state matching requires decision of satisfiability in the quantified BV
theory. That is asymptotically equivalent to satisfiability of quantified Boolean
formulas, and thus NEXPTIME-complete, as proved by Wintersteiger et al
(2013). Given the results of our experiments (Section 5.2), this high complexity
does not prevent from practical use, since many of these decision instances are
in fact trivial.
Title Suppressed Due to Excessive Length
13
4.3 Hybrid Representation
Although the SAT-based representation already achieves viable results for the
real-world Simulink model we have experimented on, there is still space for
further improvement. In some sense, both explicit and SAT-based representations carried out an excessive amount of work in testing all possible input
values. There may be a large number of values that the Simulink engineers
did not intend to use, yet this information was not included into the model:
either intentionally or because the language does not permit other than interval
constraints. Such tacit knowledge may be inferred and utilized to accelerate
the verification process.
Assume that we want to generate the successor ρ0 of a state ρ, represented
V|X | |ρ|
as a conjunction of BV terms j=1 δj and the accumulated path condition
pc(ρ), using a transition guarded by ϕ. A single evaluation of delay variables
belonging to the successor can be obtained by reading the satisfiability model
of the formula
|X |
^
|ρ|
dj = δj ∧ pc(ρ.ϕ).
j=1
A satisfiability model is an evaluation of all free variables that satisfies the
formula. The input variables can be ignored but we will store the evaluation
of delay variables as a sequence d1 =<d11 , . . . , d1|X |>. Now we can extend the
above formula to
|X |
^
j=1
|X |
dj =
|ρ|
δj
∧ pc(ρ.ϕ) ∧
_
dj 6= d1j ,
j=1
whose satisfiability model (provided the formula is still satisfiable) contains evaluation of delay variables d2 different from d1 . This processes can be repeated
until the resulting formula η (Formula 2 below) becomes unsatisfiable, at which
point we have all satisfying evaluation, and thus the explicit representation of
the successor ρ0 .
There are several notes to be made regarding the above procedure. It
is clearly impractical in cases when there are many evaluations forming ρ0 .
Yet in many cases the Simulink design permits only a very small number of
evaluations. The procedure is intended to be a heuristic: detecting that only few
evaluations are admissible or reverting to the classical, symbolic representation.
Such detection can be implemented by setting a user-specified threshold thr
on the number of evaluations. It immediately follows that using this heuristic
will require the ability to switch from symbolic to hybrid representation and
vice versa.
In the general case, the hybrid state comprises of a pair: the accumulated
path condition pc and a non-empty set of delay evaluations D = {d}i . We can
assume D to be non-empty because, initially, D = {<0, . . . , 0>} and with every
computation step the evaluations in D are either replaced with new evaluations
D0 or the path condition is extended (in which case D remains unmodified). The
14
Petr Bauch et al.
first case occurs when the successor contains less than thr values, |D0 | < thr
and η is unsatisfiable, and the hybrid successor state is < true, D0 >. The
second case occurs when |D0 | ≥ thr, the successor is <pc.ϕ, D>, where ϕ is
the transition guard. We are now ready to formulate the formula η, employed
in the generation of successor evaluations:
|D 0 | |X |
|X |
η:
^
dj =
|ρ|
δj
∧ pc(ρ.ϕ) ∧
j=1
^ _
dj 6= dkj .
(2)
k=1 j=1
The hybrid representation thus maintains invariant the property that a state
is formed by a certain concrete evaluation of delay variables D and a symbolic
path condition, consisting of n formulas. This state, however, represents the
result of a computation of n + m steps, where the effect of the first m steps is
precisely represented and stored in D. The only modification of the preceding
SAT-based verification required to incorporate hybrid representation is an
additional set of formulas restricting the initial delay evaluations. More precisely,
the set of recursive equations from Definition 1 has to be preceded by
|D| |X |
_ ^
d0 = dkj .
k=1 j=1
The size of D is initially 1 and never grows above thr: if the successor state
contains more than thr values of delays – a fact which is automatically deduced
– the verification proceeds with the symbolic representation. Yet whenever the
successor contains at most thr values, our hybrid representation becomes fully
explicit, succinctly representing the complete information about the system’s
computation up to this point.
5 Results
We report the results of the verification of the VoterCore diagram, which was
partly presented in Section 1: the figure shows one third of the whole diagram,
the other two thirds are connected via the c3 constant. This diagram was
selected for two reasons: (1) the operations that must be computed during
state space traversal are sufficiently complex to demonstrate the strength of
SMT, (2) the diagram itself is reasonably simple so that scalability compared
to other approaches can be demonstrated. Note that even though this diagram
comes from a real-world development, it does not represent the full model used
in the design phase. It was identified as a crucial unit and extracted manually
for verification purposes.
The above mentioned diagram was verified against three LTL specifications:
φ1 : G (sd23 > 1 ∧ ¬sv1 ) ⇒ sm1 ;
φ2 : G(pm1 ⇒ G pm1 );
φ3 : G(sm1 ∧ Xsm1 ∧ XXsm1 ∧ XXXsm1 ⇒ XXXpm1 ),
Title Suppressed Due to Excessive Length
15
where the model satisfied φ1 , and φ2 , but was incorrect with respect to φ3 . The
previous, purely explicit solution, which we described in Barnat et al (2012)
and refer to it here as expl, is compared with the two solutions proposed in
this paper: verification using explicit sets (set) and the SAT-based verification
(sat). Finally, the hybrid representation (hyb) is compared to sat on a set of
experiments using the same model but focusing also on a collection of extreme
cases, where sat behaved poorly.
5.1 Implementation
Both explicit-set and SAT-based approaches to verification of Simulink diagrams
were implemented2 in a clone of the DiVinE verification environment, which
already supported these diagrams as input, in the form of CESMI intermediate
language (see Barnat et al (2013) for details). The implementation of set is a
relatively straightforward extension of expl: instead of keeping the evaluations
in separate states there are fewer multi-states for every explicit control state.
Furthermore, only the delay variables are represented; the inport variables are
stored separately only once for the whole diagram.
The implementation of sat first needed the expressions represented by
Simulink blocks to be translated into BV formulas.
Algorithm 1 demonstrates how to implement the model checking process,
with a function for generating successors, using syntactic substitution. Initially,
the state matching was implemented exactly as described, where only the
inport variables were quantified. But the experiments revealed that much
better verification times can be achieved when all variables are quantified and
the substitution is not recursive, but uses only the previous history version of a
particular variable. At least for the Z3 de Moura and Bjørner (2008) solver, this
leads to better results. Communication with the SMT solver is facilitated by the
SMT2 Barrett et al (2010) format, and thus any solver of quantified BV theory
could have been used; to the best of our knowledge, however, Z3 Wintersteiger
et al (2013) is the only one at this time.
As mentioned in Section 3, the states comprise of two parts: explicit and
symbolic. The explicit part contains, most importantly, the state of the B¨
uchi
automaton and the model checker uses hashing on this part of a state to partially solve the state matching. The symbolic part contains the path condition
vector and must be excluded from hashing. During successor generation, the
path condition of the original state (in a conjunction with a formula labeling the
B¨
uchi automaton transition) is tested for satisfiability. The resulting formula
does not contain quantifiers, and thus a decision procedure for quantifier-free
BV is sufficient: line 3 of Algorithm 1. The SMT2 file communicating the
request would then be as follows:
(set-option :produce-models true)
(set-logic QF BV)
2
Code available at http://anna.fi.muni.cz/~xbauch/code.html#simulink.
16
Petr Bauch et al.
(declare-fun a0x0 () ( BitVec 4))
(declare-fun a1x1 () ( BitVec 4))
(declare-fun a12x1 () ( BitVec 4))
...
(assert (= a0x0 #x0))
(assert (and (bvuge a1x1 #x0)(bvule a1x1 #x3)))
(assert (= a12x1 (ite (bvuge (bvadd (ite (or (bvugt a1x1 #x1) (not (= a2x1 #x1)))
#x1 #x0) (ite (bvult (ite (or (bvugt a1x1 #x1) (not (= a2x1 #x1))) a0x0 #x0) #3)
(ite (or (bvugt a1x1 #x1) (not (= a2x1 #x1))) a0x0 #x0) #x3)) #x3) #x1 #x0)))
...
(assert (and(= a12x1 #x1)(= a12x1 #x0)))
(check-sat)
(get-value (a1x1 a2x1 a4x1 a5x1 a7x1 a8x1))
where the variables a0x0, a1x1, and a12x1 are examples of delay, inport, and
outport variables, respectively; also the xi suffix denotes the history index.
The third line from the end is the current path condition: the algorithm is
generating successors of the first state, thus only one specification formula is
present. Finally, the last line requests the model of the system of formulas: the
inport variables evaluation for counterexample generation.
State matching, on the other hand, leads to a formula in quantified BV, see
line 7 of Algorithm 1, and adequately powerful solver needs to be used. The
following fragment shows the interesting part of the communicated SMT2 file:
(assert (forall ((b0x0 Full)(b3x0 Full)(b6x0 Full)) (=> (and (= b0x0 #x0) (= b3x0
#x0) (= b6x0 #x0))
(or (distinct a0x1 b0x0) (distinct a3x1 b3x0) (distinct a6x1 b6x0)))))
where Full is the domain of delay variables (ai and bi refer to the same
variables in different states). Note also that the algorithm allows comparison
between delay variables with different history indices, i.e. those from states
represented by path conditions of unequal length.
Hybrid Representation Implementation: With regards to the hybrid
representation of Section 4.3, there are two interesting implementation details:
the generation of delay values using SMT queries and the modification of
searching among known states. Other than these two changes, both path
condition satisfiability and state matching have to be extended with the
enumeration of initial delay values:
(assert (or (and (= a6x0 #x0) (= a7x0 #x0) (= a8x0 #x0)) (and (= a6x0 #x0)
(= a7x0 #x0) (= a8x0 #x1)) (and (= a6x0 #x0) (= a7x0 #x0) (= a8x0 #x2)) ))
which are then referenced in expressions for other Simulink variables (and
possibly delays with higher history index). These initial delays are the only
variables with history index zero and they retain it throughout the computation.
This is justified because the length of the computation represented symbolically
is equal to the length of the pc sequence, thus we can always label the Simulink
variables with the correct index.
The generation of delay values is an extension of the path condition satisfiability checking: there as well has to be the enumeration of initial delay values
but instead of collecting the values of input variables from the satisfiability
Title Suppressed Due to Excessive Length
17
model, we are interested in the values of delays. These values are stored and
later used in subsequent queries so that the enumeration could be exhaustive.
The interesting part of the query would thus have the following structure:
(assert (and (or (distinct a6x2 #x0) (distinct a7x2 #x2) (distinct a8x2 #x7))
(or (distinct a6x2 #x3) (distinct a7x2 #x0) (distinct a8x2 #x0)) ))
(check-sat)
(get-value (a6x2) (a7x2) (a8x2))
which instruments the SMT solver to check satisfiability of the path condition
using only such inputs whose evaluation leads to any value of delays different
from those explicitly enumerated.
Storing delay evaluations explicitly in the hybrid representation provides
a very important property for model checking and that the representation
is canonical. Hence the slow linear search on lines 5–10 of Algorithm 1 may
be replaced with an effectively constant time hash-based search. Given that
the states are completely represented by the delay evaluations, moving these
evaluations into the explicit part of the state suffices to implement such hashbased search. Only those states with path conditions consisting of at least
one formula would need to be distinguished using the above SMT queries
and only among themselves. Two states <pc, D > and <pc0 , D0 > can only
be equal if |pc| = 0 ⇔ |pc0 | = 0, i.e. if either both have been exhaustively
concretized in the previous step or neither of them. Since two equal states
must contain the same number of possible delay evaluations, it follows that the
generating procedure would either report that there are more than thr values
or successfully enumerate both states.
5.2 Experiments
Similar execution conditions were chosen for the comparison between the purely
explicit approach (unmodified DiVinE) and the new hybrid approaches: the
codes were compiled with optimisation option -O2 using GCC version 4.7.2
and ran on a dedicated Linux workstation with quad core Intel Xeon 5130 @
2GHz and 16GB RAM. Given that DiVinE uses on-the-fly verification – the
traversal terminates when an accepting cycle is detected – the progression of
the verification process is prone to vary. Furthermore, parallel algorithms tend
to differ in the runtime between individual executions, hence we have executed
every experiment 10 times and report the average of those.
5.2.1 Purely Explicit versus Explicit Sets
The 6 plots in Figure 2 contain all the measured experiments: the left-hand
side reports time results and the right-hand side reports space results; each
row refers to experiments against different temporal specification φ1 , φ2 , and
φ3 described above. Each plot has the range of inport variables placed on the
linear x-axis and either time or space on the logarithmic y-axis. The crucial
observation to be made from the figures is that the set experiments scale
18
Petr Bauch et al.
1000
Space (kB)
100
Time (s)
1e+07
expl-seq
expl-par
set-seq
set-par
10
1
0.1
1
2
3
4
5
6
7
8
expl-seq
expl-par
set-seq
set-par
1e+06
100000
10000
9
1
2
3
Range of variables
4
5
6
7
8
9
7
8
9
7
8
9
Range of variables
(a) Verification complexity for property φ1
1000
Space (kB)
100
Time (s)
1e+07
expl-seq
expl-par
set-seq
set-par
10
1
0.1
1
2
3
4
5
6
7
8
expl-seq
expl-par
set-seq
set-par
1e+06
100000
10000
9
1
2
3
Range of variables
4
5
6
Range of variables
(b) Verification complexity for property φ2
1000
Space (kB)
100
Time (s)
1e+07
expl-seq
expl-par
set-seq
set-par
10
1
expl-seq
expl-par
set-seq
set-par
1e+06
100000
0.1
0.01
1
2
3
4
5
6
7
8
9
Range of variables
10000
1
2
3
4
5
6
Range of variables
(c) Verification complexity for property φ3
Fig. 2: Plots that report the results of the VoterCore Simulink diagram verification, comparing
the purely explicit approach (expl) with the approach that uses explicit sets of variable
evaluations (set). The sequential verification (-seq) is compared with the parallel verification
(-par) which used four processor cores. The range z of input variables on the x-axis corresponds
to the domain {0 . . . z}. The expl experiments had a time-out limit of 1000 seconds and a
memory limit of 10GB. For example the unreduced state space of range 5 has 375000 states
and 500 million transitions.
with the range of variables much more effectively than expl experiments. This
observation, however, is obfuscated by two phenomena: the difference between
parallel and sequential algorithms (treated in the next paragraph) and the
results for property φ3 . The VoterCore system is not a model of φ3 and hence
Title Suppressed Due to Excessive Length
19
there is an accepting cycle in the system. On-the-fly verification enables very
fast detection in some cases (when the cycle is near the initial state) and thus
the expl experiments are better on smaller ranges (though the time and space
are negligible for both approaches), because for set experiments the generation
of every successor involves nontrivial amount of computation.
The difference between parallel and sequential algorithms used in the
experiments deserves placing under closer scrutiny. Parallel algorithms (expl-par
and set-par in the plots) employ breadth-first search unlike sequential algorithms
(expl-seq and set-seq) that employ depth-first search. Given that the resulting
transition system is very shallow (the longest paths have approx. 20 edges)
and at the same time extremely wide (the data-flow explosion causes each
state to have many successors), the depth-first approach has a considerable
advantage with respect to the space complexity. This can be observed on all
the right-hand side figures for expl experiments, yet the set experiments cannot
profit from the same phenomenon: the data-flow explosion of the number of
successors was transformed into the complexity of their generation, which is
common to both parallel and sequential approaches. Hence while retaining
the parallel scalability (par experiments use all four processor cores) in time,
in space the set experiments demonstrate superiority only compared to par
experiments. (φ1 experiments are anomalous because the property transition
system has no branching and thus set experiments traverse a linear system and
cannot scale in parallel even in theory.)
5.2.2 Explicit Sets versus SAT-Based Verification
The next 6 plots in Figure 3 compare the two approaches proposed in this paper.
The setting is exactly the same as before, except this time we do not include
the results of parallel verification not to obfuscate the main message. Most
importantly, for the VoterCore diagram and for domains of input variables
larger than approximately 40, the SAT-based representation surpasses the one
using explicit sets. The plots only report a part of the experiments, the limit on
the x-axis is artificial so that the phenomena before the range of 40 remain clear.
In fact the sat experiments continued to a range corresponding to the domain
0 − 232 of standard unsigned integers, which degraded the time performance at
most twice compared to, for example, the range 10. Furthermore, the spacial
requirements of such verification were almost negligible for arbitrary range,
rarely higher than 500 MB.
For very small ranges of variables, strictly smaller than 40, the spacial
requirements of the set approach are tolerable and the time complexity is better
than that of the sat approach. The outstanding spikes in the complexity of sat
also deserve further explanation. They correspond to ranges 2n − 1 for some n.
We postulate that this relates to the specific procedure that Z3 uses to decide
satisfiability of quantified BV formulas. The precise explanation would probably
require a detailed exposition of the functioning of the particular decision
procedure employed in Z3, and we encourage the interested readers to follow it
in Wintersteiger et al (2013). The high level process in fact incorporates a model
20
Petr Bauch et al.
10000
1e+07
sat
set
sat
set
Space (kB)
Time (s)
1000
100
10
1
100 101 102 103 104 105 106 107 108 109
Range of variables
1e+06
100000
10000
100 101 102 103 104 105 106 107 108 109
Range of variables
(a) Verification complexity for property φ1
10000
1e+07
sat
set
sat
set
Space (kB)
Time (s)
1000
100
10
1
100 101 102 103 104 105 106 107 108 109
Range of variables
1e+06
100000
10000
100 101 102 103 104 105 106 107 108 109
Range of variables
(b) Verification complexity for property φ2
10000
1e+07
sat
set
sat
set
100
10
1
100 101 102 103 104 105 106 107 108 109
Range of variables
Space (kB)
Time (s)
1000
1e+06
100000
10000
100 101 102 103 104 105 106 107 108 109
Range of variables
(c) Verification complexity for property φ3
Fig. 3: Plots that report the results of VoterCore Simulink diagram verification, comparing
the representations based on explicit sets (set) and BV formulas (sat). The range z of input
variables on the x-axis corresponds to the domain {0 . . . z}. The set experiments had a
memory limit of 10GB.
checking of quantifier instantiations, guided by previously tried unsuccessful
counterexamples. It appears that limiting the range to a value very close to 2n
forces a larger number of wrong instantiations.
Title Suppressed Due to Excessive Length
21
5.2.3 Evaluation of the Hybrid Representation
As will become clear once the results pertaining to the verification using hybrid
representation (hyb) are presented, there is no need to compare it with the
explicit verification. Both time and space requirements of hyb are lower than
the respective requirements of expl, except on the most trivial instances. Yet
the theoretical comparison of the two approaches is particularly interesting
because the states themselves have exactly the same representation, i.e. explicit
enumeration of delay variable evaluations. The only difference is in the process
by which these evaluations were generated. Unlike in expl where we have to
test all evaluations of inputs to see which delay evaluations they produce, in
hyb we only employ those combinations of inputs for which we know that they
will produce different delays. This knowledge about the Simulink system is
obtained using further SMT queries, and may by itself help the designer in
validation that the modeled system is the one intended.
The plots in Figure 4 compare the purely symbolic and the hybrid representations with respect to the verification time. (We are not comparing the
spacial requirements because they were negligible in sat already and in hyb the
peak memory consumption never surpassed 6MB.) The timing results for sat
are exactly the same as in Figure 3, only here we intentionally pinpoint the
difficult domain sizes, i.e. the cases when input ranges are 0 − (2n − 1). The
SMT solver struggles in these cases if it must produce a proof of unsatisfiability
for a quantified query. We hypothesize that this phenomenon is caused by the
model-based instantiation of quantified variables, used as one of the techniques
in Z3. Setting the upper bound of input range this close to the limit posed by
the fixed bit width seems to force the instantiation to choose from a larger set
of values, before the solver can deduce unsatisfiability. It may thus be possible
to modify the solver to achieve better results, and we are presently investigating
this possibility.
What these plots immediate show is that the undesirable behavior on
difficult domain sizes was completely eliminated with the employment of hyb.
Even though it occurred in a logarithmic number of cases, it would still be
very unpleasant if the system designer chose such a case. Disregarding these
extreme cases, with hyb we have achieved 2.4, 6.7, and 2.5 speedup for the three
verified properties respectively. With the new representation, the verification
process scales even better and the complexity is in fact only dependent on
the number of steps needed to resolve the verification query. On the other
hand, the complexity of individual SMT queries in hyb does not grow with
the number of steps. It is not the case for sat, where the number of quantified
variables is directly related to the length of the current path condition. Hence
the difference in speedups for the three verified properties: checking ϕ2 required
building a state state space with the longest path twice the length as for the
other properties. We thus conclude that for more complex systems and in cases
when hyb is applicable, as discussed in Section 4.3, it would produce even
better speedup when compared to sat.
22
Petr Bauch et al.
Time (s)
1000
sat
hyb
100
10
1
20 22 24
28
216
Range of variables
232
(a) Verification complexity for property φ1
Time (s)
10000
sat
hyb
1000
100
10
20 22 24
28
216
Range of variables
232
(b) Verification complexity for property φ2
Time (s)
10000
sat
hyb
1000
100
10
20 22 24
28
216
Range of variables
232
(c) Verification complexity for property φ3
Fig. 4: Plots that report the results of VoterCore Simulink diagram verification, comparing
the representation based on BV formulas (sat) and the hybrid representation (hyb). The
range z of input variables on the x-axis corresponds to the domain {0 . . . z}.
6 Discussion
By reducing deadlock checking and state matching to SMT queries we have
managed to accelerate the model checking process to a point when the developer
Title Suppressed Due to Excessive Length
23
can omit specifying the unit testing input and verify that the design is correct
for all possible values. Given that the process still requires large amounts of
computational resources, the newly proposed hybrid representation accelerates
the method further, thus rendering it even more applicable for verifying critical
units in Simulink designs.
To summarize the findings of our experiments with verification of Simulink
diagrams, we propose a combination of the set and sat/hyb approaches. Such a
combining verification is possible, because the two approaches allow identical
models as the input. The result would correspond to the composition of (the
better of) the solid and dotted lines of Figure 3. The developer could decide
which representation to use, based on the range of variables they need to verify
against. Or – better yet, and fully automatically – the two approaches can be
run in parallel, the one finishing first reporting the results of the verification.
Arguably, the experimental results and the achieved empirical findings
remain largely academic. Even though the VoterCore model used in our experiments comes from a real-world design, it is only a part of a larger Simulink
model. Industrial application of the proposed method would equally require
manually selecting a safety-critical part of the verified model and verifying
that this part has the desired properties. Our experiments already show the
limits of using SMT solvers for LTL verification, even when the new hybrid
representation is employed. More importantly, however, the experiments also
show that if the input model is sufficiently small for the verification to finish
in reasonable time, then a complete coverage across all possible input values is
provided. Which is a quality that the established techniques based on testing
cannot provide.
6.1 Theoretical Limitations
Given that the proposed verification technique reduces the problem of LTL
model checking to a sequence of SMT-queries, the limitations of the technique
derives from the state-of-the-art of the SMT solvers. It follows that improvements in the efficiency of SMT solvers will also improve our technique and this
improvement will require no further modifications. On the other hand, solving
quantified SMT queries is a computationally demanding task. We will thus
discuss the state matching – the part of our technique that seems to require
solving quantified queries – in a greater detail.
The Figure 5 below shows a part of the verification process, where the
x-axis lists the individual instances of state matching as they appeared in time.
The y-axis represents the number of quantifier instantiations the solver needed
to employ before finding a counterexample or a proof. Finding counterexamples,
i.e. evaluations of delay variables that exist in one state but not in the other,
appears to be a considerably simpler task, in the figure corresponding to steps
of less than 40 instantiations. Finding proof of equality between the two states
was much more difficult, often requiring more than 400 instantiations with
Petr Bauch et al.
Instantiations
24
1000
100
10
1
10
20
30
40
50
60
70
80
90
100
Fig. 5: Time progression of the complexity of individual state matching instances.
equivalently higher time and space complexity. These higher spikes occur in
consecutive pairs, one for each subset inclusion.
6.2 State Matching by Comparing Function Images
Returning back to the claimed necessity of resorting to quantified formulas
0
for state matching, note that computing the equivalence between δ |ρ| and δ |ρ |
would be sufficient for state matching, if the inputs were read only once at the
beginning. The state space would be larger as some states with the same sets
of values would have semantically different representing functions. In Simulink,
where inputs are read iteratively, however, such an approach would lead to
infinite transition systems in all nontrivial instances.
Computing equivalence between the two δ functions does not yield a correct
result with respect to image comparison. For example on the Boolean domain
the functions negation and identity are not equivalent, yet their images are
the same. Consequently, stating the comparison as satisfiability in the BV
theory must be different than simple equality. Assume that the formula Ψρρ0 is
satisfiable iff ρ and ρ0 are two different states. Then the model of Ψρρ0 must
encode an element, current evaluation of delay variables, of either ρ or ρ0
that is not in the other state. But Ψρρ0 can only refer to the domains of δ
functions, the input variables, because in symbolic execution the current values
are represented as functions on input variables. It follows that the satisfying
evaluation of input variables in ρ must be such that for any evaluation of the
inputs in ρ0 the δ functions evaluate to different values.
It appears that stating functional image comparison within symbolic execution as a satisfiability decision requires the use of quantifiers. That much
seems to hold if it is the input variables that are free in Ψρρ0 as argued above.
The more general question, whether for bit-vector functions f : bvqn → bvq,
f 0 : bvqm → bvq, there is a quantifier-free bit-vector formula ψ such that
n
m
|ψ| ∈ poly(n + m) and ψ is satisfiable iff f (2bvq ) = f 0 (2bvq ), is outside the
scope of this work and we leave it open for future research.
The more interesting question, at least from the practical point of view, is
how to implement the state matching in set-based reduced transition system,
regardless of how the sets are represented. Consider for example that one
can represent the sets of evaluations using their characteristic functions, i.e.
Title Suppressed Due to Excessive Length
25
functions over state-forming variables that, when provided with an evaluation
of those variables, return either 1 or 0, whether or not this evaluation is part
of the set. (In other words representing evaluations of variables as the collected
post-conditions.) Then the state matching would reduce to deciding semantic
equivalence of these characteristic functions, or, if a canonical representation
was available, even to syntactic equivalence.
7 Related Work
Firstly, there are three practical papers, that we are aware of, describing
LTL verification of Simulink diagrams. Miller et al (2005) translate Simulink
models into the synchronous data-flow language Lustre, created by Halbwachs
et al (1991), and then into the native language of the symbolic model checker
NuSMV, created by Cimatti et al (2002). NuSMV was then used to verify
correctness of the model with respect to a set of specifications (mostly in CTL,
but NuSMV can verify full LTL as well) using BDD-based (Binary Decision
Diagrams: see McMillan (1992), for a classical approach) representation of
the state space. Although it was not a limitation in the input diagrams they
used, BDDs are not readily able to support complex data types with Peano
arithmetic, at least not while preserving their efficiency, as we will discuss later
in this subsection.
Similarly, Meenakshi et al (2006) translate Simulink diagrams for the
verification using NuSMV, but in this case the translation is direct, without
an intermediate language. This work is more relevant because the diagrams
used contained integer data types and full Peano arithmetic (NuSMV flattens
integer types into Boolean types, but represents the stored values precisely).
The authors do not mention explicitly if they used the BDD-based branch or
SAT-based branch of verification. Unfortunately, we could not compare that
approach with the one proposed in this paper, since the implementation is not
available. Its efficiency can be only assumed from the reported results: when
input variables were not bounded explicitly, i.e. allowed values between 0 and
232 , the verification time was almost a week; the bounds for reasonably fast
verification had to be relatively small: between 0 and 60.
Explicit model checking was also used for verification of Simulink diagrams
by Barnat et al (2012), but there the data-flow non-determinism in the form of
input variables had even greater detrimental effect. This is the only comparable
implementation presently available and we provide detailed comparison between
this purely explicit approach and the hybrid, control explicit—data symbolic
approaches described in this paper in Section 5.
From a theoretical point of view, the method proposed in this paper is
effectively equivalent to symbolic execution extended with LTL verification.
This combination was attempted before by Braione et al (2008), but classical
symbolic execution does not allow state matching and thus only a small subset
(safety properties) of LTL was supported. Without state matching, an infinite
violating run would cause the tool to diverge, never yielding an answer. On the
26
Petr Bauch et al.
other hand it is possible to implement state subsumption (a state represents
more heap configurations than another state) in symbolic execution, as shown
by Xie et al (2005), but subsumption is not correct with respect to LTL even
though it can significantly reduce the state space (and make it finite).
A considerable amount of work pertained to allowing symbolic model
checking to use more complex data types than Booleans. The problem is that
BDDs grow exponentially in the presence of integer multiplication, as proved
by Bryant (1991). Other symbolic representations were designed that represent
variables on the word level rather than binary level, such as Binary Moment
Diagrams, used by Bryant and Chen (1995), or Boolean Expression Diagrams,
used by Williams et al (2000), even though these are no longer canonical.
Similarly to our approach, symbolic model checking can also use first-order
formulas directly, leading to SAT-based and SMT-based approaches, invented
by Biere et al (1999) and Armando et al (2006), respectively, provided that
the procedure is bounded in depth, and consequently incomplete. Various
approaches to unbounding have been proposed, most recently the IC3 method
of Bradley (2011), but these are rather theoretical, without implementation,
or limited to weaker arithmetics, see e.g. the model checker Kind based on
k-induction and created by Hagen and Tinelli (2008).
Module checking, introduced by Kupferman and Vardi (1996) and detailed
by Godefroid (2003), allows verification of open systems. A system is open
in the sense that the environment may restrict the non-determinism and the
verification has to be robust with respect to arbitrary restrictions. Consequently,
the approach to verifying open systems also differs since only branching time
logics can distinguish open from closed systems, in the module checking sense.
For linear time logics, every path has to satisfy the property and thus open
and closed systems collapse into one source of non-determinism.
Much closer to our separation between control and data is the work initiated
by Lin (1996). Lin’s Symbolic Transition Graphs with Assignments represent
precisely parallel programs with input variables using a combination of firstorder logic and process algebra for communicating systems. Similarly as for
symbolic execution, the most complicated aspect of this representation is
the handling of loops. Lin’s solution computes the greatest fix point of a
predicate system representing the first-order term for each loop. Then two
transition graphs are bisimilar if the predicate systems representing their loops
are equivalent. The set-based reduction of transition systems used in this paper
effectively compute the fix points explicitly, represented as sets of variable
evaluations.
Finally, various combinations of different approaches and representations
have been devised and experimented with. When multiple representations
were combined, it was mostly to improve on weak aspects of either of the
representations, for example by Yang et al (2006) multiple symbolic representations for Boolean and integer variables were employed in combination.
Presburger constraints were combined with BDDs by Bultan et al (1998) to
allow, with some restrictions, the verification of infinite state spaces. Finally,
the two approaches to model checking, explicit and symbolic, were combined
Title Suppressed Due to Excessive Length
27
to improve solely upon control-flow non-determinism. Some improvement was
achieved by storing multiple explicit states in a single symbolic state, implemented by Duret-Lutz et al (2011), or by storing explicitly the property and
symbolically the system description, implemented by Sebastiani et al (2005).
The approach described by Hungar et al (1995) represents control using symbolic model checking and data by purely symbolic manipulation of first-order
formulas. There the symbolic data are limited in the sense that they cannot
influence the control flow.
8 Conclusion
This paper combines explicit model checking with a representation of data
variables based on sets of possible evaluations. The resulting verification method
allows temporal verification of systems that read from non-deterministic inputs.
When the sets of evaluations are represented symbolically – as bit-vector
formulas – the major obstacle that we faced and solved is the solution of
state matching. The state matching procedure proposed here requires deciding
satisfiability of quantified bit-vector formulas, which is a resource-intensive
process, but the experimental evaluation on Simulink diagrams indicates that
for such systems is the cost acceptable. Still, in order to improve the effectivity of
design supported by verification, we have proposed, implemented and evaluated
a combing approach using a hybrid representation of data. This representation
leverages the positive aspects of both the concrete and symbolic representations
while employing some knowledge about the Simulink diagram which was not
explicitly stated by the developer. We have demonstrated how SMT solvers
can be used to infer this tacit knowledge, which can subsequently enable the
hybrid representation to achieve considerably better performance.
For the use in verification of parallel programs or at least safety critical
segments of programs, the cost would probably be prohibitive. We discuss ideas
that could lead to expressing the state matching problem without the use of
quantifiers, but leave that direction of research to future work. Yet in the case
of Simulink diagrams, our approach scales with the ranges of input variables
and thus allows the developer to use the range limit of the used data type, e.g.
32 bits integers, where before the developers had to test for ranges that were
exponentially smaller. This improvement effectively automates the verification
process, replacing completely any manual testing, since the resulting confidence
in the correctness of the system with model checking is at least as high as it
was with classical testing.
References
Armando A, Mantovani J, Platania L (2006) Bounded Model Checking of Software Using
SMT Solvers Instead of SAT Solvers. In: SPIN, LNCS, vol 3925, Springer, pp 146–162,
DOI 10.1007/11691617 9
Barnat J, Bauch P (2013) Control Explicit—Data Symbolic Model Checking: An Introduction.
Tech. rep., Masaryk University, http://arxiv.org/abs/1303.7379
28
Petr Bauch et al.
Barnat J, Beran J, Brim L, Kratochv´ıla T, Roˇ
ckai P (2012) Tool Chain to Support Automated
Formal Verification of Avionics Simulink Designs. In: FMICS, LNCS, vol 7437, Springer,
pp 78–92, DOI 10.1007/978-3-642-32469-7 6
ˇ
Barnat J, Brim L, Havel V, Havl´ıˇ
cek J, Kriho J, Lenˇ
co M, Roˇ
ckai P, Still
V, Weiser J (2013)
DiVinE 3.0 – Explicit-state Model-checker for Multithreaded C/C++ Programs. In:
CAV, pp 863–868, DOI 10.1007/978-3-642-39799-8 60
Barnat J, Bauch P, Havel V (2014) Temporal Verification of Simulink Diagrams. In: Proc. of
HASE, pp 81–88, DOI http://dx.doi.org/10.1109/HASE.2014.20
Barrett C, Stump A, Tinelli C (2010) The SMT-LIB Standard: Version 2.0. Tech. rep., The
University of Iowa
Biere A, Cimatti A, Clarke E, Fujita M, Zhu Y (1999) Symbolic Model Checking Using SAT
Procedures instead of BDDs. In: Proc. of DAC, ACM, pp 317–320, DOI 10.1145/309847.
309942
Bradley A (2011) SAT-Based Model Checking without Unrolling. In: VMCAI, pp 70–87,
DOI 10.1007/978-3-642-18275-4 7
Braione P, Denaro G, Kˇrena B, Pezz`
e M (2008) Verifying LTL Properties of Bytecode with
Symbolic Execution. In: Proc. of Bytecode, Elsevier Science, pp 1–14
Bryant R (1991) On the Complexity of VLSI Implementations and Graph Representations
of Boolean Functions with Application to Integer Multiplication. IEEE Trans Comput
40(2):205–213, DOI 10.1109/12.73590
Bryant R, Chen YA (1995) Verification of Arithmetic Circuits with Binary Moment Diagrams.
In: Proc. of DAC, pp 535–541, DOI 10.1145/217474.217583
Bultan T, Gerber R, Pugh W (1997) Symbolic Model Checking of Infinite State Systems
Using Presburger Arithmetic. In: CAV, LNCS, vol 1254, Springer, pp 400–411, DOI
10.1007/3-540-63166-6 39
Bultan T, Gerber R, League C (1998) Verifying Systems with Integer Constraints and
Boolean Predicates: A Composite Approach. In: Proc. of ISSTA, pp 113–123, DOI
10.1145/271771.271799
Cimatti A, Clarke E, Giunchiglia E, Giunchiglia F, Pistore M, Roveri M, Sebastiani R,
Tacchella A (2002) NuSMV 2: An OpenSource Tool for Symbolic Model Checking. In:
CAV, pp 241–268, DOI 10.1007/3-540-45657-0 29
Clarke E, Emerson E, Sistla A (1986) Automatic Verification of Finite-State Concurrent
Systems Using Temporal Logic Specifications. ACM T Progr Lang Sys 8(2):244–263,
DOI 10.1145/5397.5399
Cousot P, Cousot R (1977) Abstract Interpretation: A Unified Lattice Model for Static
Analysis of Programs by Construction or Approximation of Fixpoints. In: Proc. of POPL,
ACM, pp 238–252, DOI 10.1145/512950.512973
Duret-Lutz A, Klai K, Poitrenaud D, Thierry-Mieg Y (2011) Self-Loop Aggregation Product
— A New Hybrid Approach to On-the-Fly LTL Model Checking. In: ATVA, LNCS, vol
6996, Springer, pp 336–350, DOI 10.1007/978-3-642-24372-1 24
Godefroid P (2003) Reasoning about Abstract Open Systems with Generalized Module Checking. In: EMSOFT, LNCS, vol 2855, Springer, pp 223–240, DOI 10.1007/
978-3-540-45212-6 15
Hagen G, Tinelli C (2008) Scaling Up the Formal Verification of Lustre Programs with SMTBased Techniques. In: Proc. of FMCAD, pp 1–9, DOI 10.1109/FMCAD.2008.ECP.19
Halbwachs N, Caspi P, Raymond P, Pilaud D (1991) The Synchronous Data Flow Programming Language LUSTRE. Proc IEEE 79(9):1305–1320, DOI 10.1109/5.97300
Hungar H, Grumberg O, Damm W (1995) What if Model Checking Must Be Truly Symbolic.
In: CHARME, LNCS, vol 987, Springer, pp 1–20, DOI 10.1007/3-540-60385-9 1
King J (1976) Symbolic Execution and Program Testing. Commun ACM 19(7):385–394,
DOI 10.1145/360248.360252
Kroening D, Strichman O (2010) Decision Procedures: An Algorithmic Point of View. Springer
Kupferman O, Vardi M (1996) Module Checking. In: CAV, LNCS, vol 1102, Springer, pp
75–86, DOI 10.1007/3-540-61474-5 59
Lin H (1996) Symbolic Transition Graph with Assignment. In: CONCUR, LNCS, vol 1119,
Springer, pp 50–65, DOI 10.1007/3-540-61604-7 47
McMillan K (1992) Symbolic Model Checking. PhD thesis, Carnegie Mellon University
Title Suppressed Due to Excessive Length
29
Meenakshi B, Bhatnagar A, Roy S (2006) Tool for Translating Simulink Models into Input
Language of a Model Checker. In: ICFEM, LNCS, vol 4260, Springer, pp 606–620,
DOI 10.1007/11901433 33
Miller S, Anderson E, Wagner L, Whalen M, Heimdahl M (2005) Formal Verification of
Flight Critical Software. In: Proc. of GNC, pp 1–16
de Moura L, Bjørner N (2008) Z3: An Efficient SMT Solver. In: TACAS, LNCS, vol 4963,
Springer, pp 337–340, DOI 10.1007/978-3-540-78800-3 24
Owicki S, Gries D (1976) An Axiomatic Proof Technique for Parallel Programs I. Acta
Inform 6:319–340, DOI 10.1007/BF00268134
Sebastiani R, Tonetta S, Vardi M (2005) Symbolic Systems, Explicit Properties: On Hybrid
Approaches for LTL Symbolic Model Checking. In: CAV, LNCS, vol 3576, Springer, pp
100–246, DOI 10.1007/11513988 35
Vardi M, Wolper P (1986) An Automata-Theoretic Approach to Automatic Program Verification. In: Proc. of LICS, IEEE Computer Society Press, pp 332–344
Williams P, Biere A, Clarke E, Gupta A (2000) Combining Decision Diagrams and SAT
Procedures for Efficient Symbolic Model Checking. In: CAV, LNCS, vol 1855, Springer,
pp 124–138, DOI 10.1007/10722167 13
Wintersteiger C, Hamadi Y, de Moura L (2013) Efficiently Solving Quantified Bit-Vector
Formulas. Form Method Syst Des 42(1):3–23, DOI 10.1007/s10703-012-0156-2
Xie T, Marinov D, Schulte W, Notkin D (2005) Symstra: A Framework for Generating
Object-Oriented Unit Tests Using Symbolic Execution. In: TACAS, LNCS, vol 3440,
Springer, pp 365–381, DOI 10.1007/978-3-540-31980-1 24
Yang Z, Wang C, Gupta A, Ivanˇ
ci´
c F (2006) Mixed Symbolic Representations for Model
Checking Software Programs. In: Proc. of MEMOCODE, pp 17–26, DOI 10.1109/
MEMCOD.2006.1695896