CSE 245: Computer Aided Circuit
Simulation and Verification
Fall 2004, Nov
Nonlinear Equation
Outline
 Nonlinear problems
 Iterative Methods
 Newton’s Method






Derivation of Newton
Quadratic Convergence
Examples
Convergence Testing



Basic Algorithm
Quadratic convergence
Application to circuits

Limiting Schemes
Multidimensonal Newton Method
Improve Convergence



Direction Corrupting
Non corrupting (Damped Newton)

Source stepping
Continuation Schemes
January 11, 2017
2
courtesy Alessandra Nardi UCB
Nonlinear Problems - Example
1
Vd
I1
I d  I s (e
Id
Ir
Vt
 1)  0
0
Need to Solve
I r  I d  I1  0
e1
Vt
1
e1  I s (e  1)  I1  0
R
January 11, 2017
g (e1 )  I1
3
courtesy Alessandra Nardi UCB
Nonlinear Equations
 Given g(V)=I
 It can be expressed as: f(V)=g(V)-I
 Solve g(V)=I equivalent to solve
f(V)=0
Hard to find analytical solution for f(x)=0
Solve iteratively
January 11, 2017
4
courtesy Alessandra Nardi UCB
Nonlinear Equations
– Iterative Methods
 Start from an initial value x0
 Generate a sequence of iterate xn-1,
xn, xn+1 which hopefully converges to
the solution x*
 Iterates are generated according to
an iteration function F: xn+1=F(xn)
Ask
• When does it converge to correct solution ?
• What is the convergence rate ?
January 11, 2017
5
courtesy Alessandra Nardi UCB
Newton-Raphson (NR) Method
Consists of linearizing the system.
Want to solve f(x)=0  Replace f(x) with its
linearized version and solve.
*
df
(
x
)
*
f ( x)  f ( x ) 
( x  x* )
dx
k
df ( x ) k 1 k
k 1
k
f (x )  f (x ) 
(x  x )
dx
x
k 1
1
 df ( x ) 
k
 x 
 f (x )
 dx 
k
Taylor Ser ies
k
Iteration function
Note: at each step need to evaluate f and f’
January 11, 2017
6
courtesy Alessandra Nardi UCB
Newton-Raphson Method
January 11, 2017
7
– Graphical View
courtesy Alessandra Nardi UCB
Newton-Raphson Method
Define iteration
Do k = 0 to ….
x
k 1
 df ( x ) 
 x 

 dx 
k
k
– Algorithm
1
k
f (x )
until convergence
 How about convergence?
 An iteration {x(k)} is said to converge with order q if
there exists a vector norm such that for each k  N:
January 11, 2017
x
k 1
 xˆ  8x  xˆ
k
q
courtesy Alessandra Nardi UCB
Newton-Raphson Method
– Convergence
~
df
(
x
)
d
f
(
x) *
*
k
*
k
k 2
0  f (x )  f (x ) 
(x  x ) 
(
x

x
)
2
dx
dx
k
2
Mean Value theorem
truncates Taylor series
But
k
df
(
x
) k 1
k
0  f (x ) 
(x  xk )
dx
by Newton
definition
January 11, 2017
9
courtesy Alessandra Nardi UCB
Newton-Raphson Method
Subtracting
– Convergence
df ( x k ) k 1 *
d 2 f (~
x) k
* 2
(x  x ) 
(
x

x
)
2
dx
dx
Dividing through
2
df
d
f
k 1
*
k 1
k
* 2
( x  x )  [ ( x )]
(
x
)(
x

x
)
2
dx
d x
df k 1 d 2 f
k
Let [ ( x )]
(
x
)

K
2
dx
d x
then x
k 1
x K x x
*
k
k
* 2
Convergence is quadratic
January 11, 2017
10
courtesy Alessandra Nardi UCB
Newton-Raphson Method
– Convergence
Local Convergence Theorem
If
df
a)
dx
d2 f
b)
2
dx

bounded away from zero 

 K is bounded

bounded

Then Newton’s method converges given a
sufficiently close initial guess (and
convergence is quadratic)
January 11, 2017
11
courtesy Alessandra Nardi UCB
Newton-Raphson Method
– Convergence
Example 1
f ( x)  x 2  1  0,
df k
( x )  2 xk
dx
find x ( x*  1)


2

k
k 1
x ) x
k
k 1
 x )  2x (x  x )   x
2x (x
2x (x
or ( x
k 1
k
*
k
k
1
k
*

k
 x  
2
*
2
1
 x )  k ( x k  x* ) 2
2x
Convergence is quadratic
January 11, 2017
*
12
courtesy Alessandra Nardi UCB
Newton-Raphson Method
– Convergence
Example 2
f ( x)  x  0, x  0
df k
( x )  2 xk
dx
k
k 1
k
2
 2 x ( x  0)  ( x  0)
2
*

1
 df 
 not bounded
 dx 
Note : 
away from zero

1 k
x 0  x 0
for x k  x*  0
2
1
*
*
or ( xk 1  x )  ( xk  x )
2
k 1
Convergence is linear
January 11, 2017
13
courtesy Alessandra Nardi UCB
Newton-Raphson Method
– Convergence
Example 1,2
January 11, 2017
14
courtesy Alessandra Nardi UCB
Newton-Raphson Method
– Convergence
x = Initial Guess, k  0
0
Repeat {
 
f x k
x


 
x k 1  x k   f x k
k  k 1
} Until ?
x
k 1

 x  threshold ?
k
January 11, 2017
f x
15
k 1
  threshold ?
courtesy Alessandra Nardi UCB
Newton-Raphson Method
– Convergence
Convergence Check
Need a "delta-x" check to avoid false convergence
f(x)
x
x
k 1
x
k 1
 x   xa   xr x
k
k
x

k 1
*

f x k 1   fa
January 11, 2017
16
courtesy Alessandra Nardi UCB
X
Newton-Raphson Method
– Convergence
Convergence Check
Also need an "f  x  " check to avoid false convergence
f(x)

f x
x
x
January 11, 2017
k 1
k 1

  fa
*
X
x k 1 x k
 x   xa   xr x
k
17
k 1
courtesy Alessandra Nardi UCB
Newton-Raphson Method
– Convergence
demo2
January 11, 2017
18
courtesy Alessandra Nardi UCB
Newton-Raphson Method
– Convergence
Local Convergence
Convergence Depends on a Good Initial Guess
f(x)
1
x
1
x
x
January 11, 2017
19
2
x
0
x
0
courtesy Alessandra Nardi UCB
X
Newton-Raphson Method
– Convergence
Local Convergence
Convergence Depends on a Good Initial Guess
January 11, 2017
20
courtesy Alessandra Nardi UCB
Nonlinear Problems – Multidimensional Example
v1
i2 + v2b -
v2
Nodal Analysis
i3
i1
+
+
b
1
b
3
v
v
-
-
At Node 1: i1  i2  0
 g  v1   g  v1  v2   0
At Node 2: i3  i2  0
 g  v3   g  v1  v2   0
Nonlinear
Resistors
Two coupled
nonlinear equations
in two unknowns
i  g v
January 11, 2017
21
courtesy Alessandra Nardi UCB
Outline
 Nonlinear problems
 Iterative Methods
 Newton’s Method






Derivation of Newton
Quadratic Convergence
Examples
Convergence Testing



Basic Algorithm
Quadratic convergence
Application to circuits

Limiting Schemes
Multidimensonal Newton Method
Improve Convergence



Direction Corrupting
Non corrupting (Damped Newton)

Source stepping
Continuation Schemes
January 11, 2017
22
courtesy Alessandra Nardi UCB
Multidimensional Newton Method
 
Problem: Find x such that F x  0
*
x 
*
N
*
and F :
F ( x)  F ( x )  J ( x )( x  x )
*
*
*
F1 ( x) 
 F1 ( x)

 x


x
1
N


J ( x)   

 
 FN ( x)  FN ( x) 
 x1
x N 


k 1
k
k 1
k
 x  x  J (x ) F (x )
January 11, 2017
23
N

N
Taylor Ser ies
Jacobian Matrix
Iteration function
courtesy Alessandra Nardi UCB
Multidimensional Newton Method
Computational Aspects
Iteration : x k 1  x k  J ( x k ) 1 F ( x k )
k 1
Do not compute J ( x ) (it is not sparse).
Instead solve :
J ( x k )( x k 1  x k )   F ( x k )
Each iteration requires:
1. Evaluation of F(xk)
2. Computation of J(xk)
3. Solution of a linear system of algebraic
equations whose coefficient matrix is J(xk)
and whose RHS is -F(xk)
January 11, 2017
24
courtesy Alessandra Nardi UCB
Multidimensional Newton Method
Algorithm
x = Initial Guess, k  0
0
Repeat {
   
 x  x  x    F  x  for x
Compute F x k , J F x k
Solve J F
k
k 1
k
k
k 1
k  k 1
} Until
January 11, 2017
x k 1  x k ,

f x k 1
25

small enough
courtesy Alessandra Nardi UCB
Multidimensional Newton Method
Convergence
Local Convergence Theorem
If
 
a) J F1 x k  
 Inverse is bounded 
b) J F  x   J F  y  
 Derivative is Lipschitz Cont 
x y
Then Newton’s method converges given a
sufficiently close initial guess (and
convergence is quadratic)
January 11, 2017
26
courtesy Alessandra Nardi UCB
Application of NR to Circuit Equations
Companion Network
 Applying NR to the system of equations we find
that at iteration k+1:
 all the coefficients of KCL, KVL and of BCE of the
linear elements remain unchanged with respect to
iteration k
 Nonlinear elements are represented by a linearization
of BCE around iteration k
 This system of equations can be interpreted as the
STA of a linear circuit (companion network) whose
elements are specified by the linearized BCE.
January 11, 2017
27
courtesy Alessandra Nardi UCB
Application of NR to Circuit Equations
Companion Network
 General procedure: the NR method applied
to a nonlinear circuit whose eqns are
formulated in the STA form produces at
each iteration the STA eqns of a linear
resistive circuit obtained by linearizing the
BCE of the nonlinear elements and leaving
all the other BCE unmodified
 After the linear circuit is produced, there is
no need to stick to STA, but other methods
(such as MNA) may be used to assemble
the circuit eqns
January 11, 2017
28
courtesy Alessandra Nardi UCB
Application of NR to Circuit Equations
Companion Network – MNA templates
Note: G0 and Id depend on the iteration count k
 G0=G0(k) and Id=Id(k)
January 11, 2017
29
courtesy Alessandra Nardi UCB
Application of NR to Circuit Equations
Companion Network – MNA templates
January 11, 2017
30
courtesy Alessandra Nardi UCB
Modeling a MOSFET
(MOS Level 1, linear regime)
d
January 11, 2017
31
courtesy Alessandra Nardi UCB
Modeling a MOSFET
January 11, 2017
(MOS Level 1, linear regime)
32
courtesy Alessandra Nardi UCB
DC Analysis Flow Diagram
For each state variable in the system
January 11, 2017
33
courtesy Alessandra Nardi UCB
Implications
 Device model equations must be continuous
with continuous derivatives (not all models do
this - - be sure models are decent - beware of
user-supplied models)
 Watch out for floating nodes (If a node
becomes disconnected, then J(x) is singular)
 Give good initial guess for x(0)
 Most model computations produce errors in
function values and derivatives. Want to have
convergence criteria || x(k+1) - x(k) || <  such
that  > than model errors.
January 11, 2017
34
courtesy Alessandra Nardi UCB
Outline
 Nonlinear problems
 Iterative Methods
 Newton’s Method






Derivation of Newton
Quadratic Convergence
Examples
Convergence Testing



Basic Algorithm
Quadratic convergence
Application to circuits

Limiting Schemes
Multidimensonal Newton Method
Improve Convergence



Direction Corrupting
Non corrupting (Damped Newton)

Source stepping
Continuation Schemes
January 11, 2017
35
courtesy Alessandra Nardi UCB
Improving convergence
 Improve Models (80% of problems)
 Improve Algorithms (20% of
problems)
Focus on new algorithms:
Limiting Schemes
Continuations Schemes
January 11, 2017
36
courtesy Alessandra Nardi UCB
Improve Convergence
 Limiting Schemes
 Direction Corrupting
 Non corrupting (Damped Newton)
 Globally Convergent if Jacobian is
Nonsingular
 Difficulty with Singular Jacobians
 Continuation Schemes
 Source stepping
January 11, 2017
37
courtesy Alessandra Nardi UCB
Multidimensional Newton Method
Convergence Problems – Local Minimum
Local
Minimum
f
At a local minimum,
0
x
Multidimensional Case: J F  x  is singular
January 11, 2017
38
courtesy Alessandra Nardi UCB
Multidimensional Newton Method
Convergence Problems – Nearly singular
f(x)
x1
x0
Must Somehow Limit the changes in X
January 11, 2017
39
courtesy Alessandra Nardi UCB
X
Multidimensional Newton Method
Convergence Problems - Overflow
f(x)
x
X
0
1
x
Must Somehow Limit the changes in X
January 11, 2017
40
courtesy Alessandra Nardi UCB
Newton Method with Limiting
Newton Algorithm for Solving F  x   0
x = Initial Guess, k  0
0
Repeat {
   
 x  x  F  x  for x
 limited  x 
Compute F x k , J F x k
Solve J F
x k 1  x k
}
k
k 1
k
k 1
k 1
k  k 1
k 1
k 1

x
,
F
x

 small enough
Until
January 11, 2017
41
courtesy Alessandra Nardi UCB
Newton Method with Limiting
Limiting Methods
• Direction Corrupting


limited x k 1 
i
xik 1 if xik 1  

limited x k 1
 sign  xik 1  otherwise

x k 1

• NonCorrupting


limited x k 1  x k 1


  min 1, k 1
 x

limited x



k 1


Heuristics, No Guarantee of Global Convergence
January 11, 2017
42
courtesy Alessandra Nardi UCB
x k 1
Newton Method with Limiting
Damped Newton Scheme
General Damping Scheme
 
 
Solve J F x k x k 1   F x k for x k 1
x k 1  x k   k x k 1
Key Idea: Line Search

Pick  to minimize F x   x
k

F x   x
k
k
k 1

2
2
k
k

k 1

 F x   x
k
k
2
2
k 1
 
T
F x k   k x k 1
Method Performs a one-dimensional search in
Newton Direction
January 11, 2017
43
courtesy Alessandra Nardi UCB

Newton Method with Limiting
Damped Newton – Convergence Theorem
If
 
a) J F1 x k  
 Inverse is bounded 
b) J F  x   J F  y  
 Derivative is Lipschitz Cont 
x y
Then
There exists a set of  k ' s   0,1 such that



F x k 1  F x k   k x k 1

 
  F xk
with  <1
Every Step reduces F-- Global Convergence!
January 11, 2017
44
courtesy Alessandra Nardi UCB
Newton Method with Limiting
Damped Newton – Nested Iteration
x = Initial Guess, k  0
Repeat {
0
   
Solve J  x  x   F  x  for x
Find    0,1 such that F  x   x 
Compute F x k , J F x k
k
k 1
k 1
k
F
k
k
k
k 1
is minimized
x k 1  x k   k x k 1
k  k 1
} Until
January 11, 2017
 
x k 1 , F x k 1
small enough
45
courtesy Alessandra Nardi UCB
Newton Method with Limiting
Damped Newton – Singular Jacobian Problem
x2
1
D
x
1
x
x
0
X
Damped Newton Methods “push” iterates to local minimums
Finds the points where Jacobian is Singular
January 11, 2017
46
courtesy Alessandra Nardi UCB
Newton with Continuation schemes
Basic Concepts - General setting
Newton converges given a close initial guess
 Idea: Generate a sequence of problems,
s.t. a problem is a good initial guess for the
following one
F (x)  0  Solve F  x    ,    0 where:
a) F  x  0  , 0   0 is easy to solve  Starts the continuation
b) F  x 1 ,1  F  x  Ends the continuation
c) x    is sufficiently smooth Hard to insure!
January 11, 2017
47
courtesy Alessandra Nardi UCB
Newton with Continuation schemes
Basic Concepts – Template Algorithm
Solve F  x  0  , 0  , x   prev   x  0 
 =0.01,   
While   1 {
x0     x   prev 
Try to Solve F  x    ,    0 with Newton
}
If Newton Converged
x   prev   x    ,      ,   2
Else
1
   ,    prev  
2
January 11, 2017
48
courtesy Alessandra Nardi UCB
Newton with Continuation schemes
Basic Concepts – Source Stepping Example
January 11, 2017
49
courtesy Alessandra Nardi UCB
Newton with Continuation schemes
Basic Concepts – Source Stepping Example
R
Vs
+
-
v
Diode
1
f  v    ,    idiode  v    v  Vs   0
R
f  v,  
v

idiode  v 
v
1

 Not  dependent!
R
Source Stepping Does Not Alter Jacobian
January 11, 2017
50
courtesy Alessandra Nardi UCB
Transient Analysis Flow Diagram
Predict values of variables at tl
Replace C and L with resistive elements via integration formula
Replace nonlinear elements with G and indep. sources via NR
Assemble linear circuit equations
Solve linear circuit equations
NO
Did NR converge?
YES
Test solution accuracy
Save solution if acceptable
Select new t and compute new integration formula coeff.
January 11, 2017
Done?
51
NO
courtesy Alessandra Nardi UCB