1. No category

How to apply the
simulation extrapolation
method in
errors-in-variables
models?
Silvelyn Zwanzig, Uppsala
[email protected]
Sunday August 29, 2004
1
OU T LIN E
• The main trick
• Variations of the method
• What can we reach in the best cases?
• What did we reach in linear EIV?
• Advantages of SIMEX ?
• Extension to nonlinear models
Cooperation with Jörg Polzehl WIAS, Berlin
2
T HE M AIN IDEA
• Set up
Y = (y1, ..., yn)T
yi ∼ Pθ,i independently , θ ∈ Θ
V ar (yi) = σ 2
• Wanted: θ ∈ Θ
• Start with an ''unpleasant'' estimator
b
θ = arg min Q (Y, θ)
θ∈Θ
• ''unpleasant'' because ¡ ¢
Eb
θ = θ + BIAS σ 2 + rest
but
BIAS (.) ↓ f or σ 2 ↓
3
AIM
• Simulate new observations with smaller variances.
BUT
• It is impossible to generate new observations
with smaller variances.
• We can simulate observations with a larger
error by adding a pseudo error δ ∗i
√ ∗
yi(λ) = yi + λ δ i
such that
V ar (yi(λ)) = σ 2 + λ
• OBS! λ ≥ 0
• DREAM: λ = −σ 2
4
T HE P ROCEDU RE
(1) For λ1, ..., λK positive fixed simulate
p
= yi + λk ε∗ik , i = 1, .., n
Y (λk ) = (y1∗ (λk ) , ..., yn (λk ))T
yi∗ (λk )
∗
(2) Calculate the estimators b
θ(k)
b
θ(k) = arg min Q (Y ∗ (λk ) , θ)
θ∈Θ
(3) Fit a parametric model to b
θ(1), .., b
θ(K)
K °
°2
X
°
°b
fb = arg min
°θ(k) − f (λk )°
f ∈F
k=1
(4) Extrapolate backwards
b
θSIM EX = fb(−σ 2)
5
V ARIAN T S
(1) Simulation step
different design for λ1, ..., λK
B repeated simulations for each λk
Add the pseudo errors on all observations?
(2) Calculate the estimators b
θ(k)
Which ''unpleasant'' estimators give a good
start?
(3) Model fit
Which model?
local linear
nonlinear model F,if BIAS(.) ∈ F
(4) Extrapolation step
variance unknown
determine λ∗ : b
θSIM EX = fb(λ∗)
6
ORIGIN AL SIM EX
EIV-Model:
yi = ξ Ti β + σ y εi
xi = ξ i + σ xδ i
Cook & Stefanski (1994)
bnaive.
• start with the naive estimator β
• Consider each component of the estimator
separately.
• BIAS is a function of σ 2x.
• Adding pseudo errors on the related component of xi only.
• Fit a parametric curve bb(.) for the component
of the naive estimator,
© a
ª
BIAS (λ) ∈ b+λ , a, b > 0 .
• Assume that σ 2x is known and
bj,SIM EX := bb(−σ 2 ), j = 1, .., p.
define β
x
7
SY M EX − symmetrized SIM EX
J. Polzehl and S.20 (2003)
• Use the implicit model formulation
Z = ζ + ε, αT ζ = 0, α = (−1, β 1, ..., β p)T
• Start with p + 1 different naive estimators
(l)
α
b naive, l = 0, ..., p with respect to the p + 1
different explicit models.
• Carry out the first steps SIMEX type procedure for each l.
Add pseudo errors to Z(−l) , l = 0, ..., p.
• Fit the model¡
¢−1
(l)
(l)
θ, αl (λ, θ) = 1
α(−l) (λ, θ) = M(−l)M(−l) + λI
θ p−dimensional
The model is linear in θ and nonlinear in λ.
All components of the estimator are fitted
simultaneously.
8
• Retransform to the parametrization of the explicit models
³ ´
(l)
1
(l)
b (λ) =
³ ´ α(−0)
λ, b
θ , l = 0, ..., p.
β
(l)
α0 λ, b
θ
• Extrapolation step
°2
XX°
(m)
°b(l)
∗
b (λ)°
λ = arg min
°β (λ) − β
°
λ
l
m
• Define the SYMEX estimator by
X (l)
1
b (λ∗)
bSY M EX =
β
β
p+1
l
For known σ 2x we have
¢
(0) ¡
2
b
b
β SIM EX = β
−σ x
9
ADAP T IV E SIM EX
J. Polzehl and S.20 (2003)
bnaive =
• Consider only one naive estimator β
(0)
b
β
naive
• Simulation step same as in SYMEX for l = 0
• Model fit same as in SYMEX for l = 0
• Extrapolation step:
Determine the variance estimator by σ
e2x =
λmin (MZZ ) and set
λ∗∗ = −λmin (MZZ )
• Define
(0)
b
b
β ASIM EX = β (λ∗∗)
10
W hat can we reach in the best cases?
E = simple EIV-Model:


T
T


Y = (y1, ..., yn) , X = (x1, x2..., xn) ,
yi ∼ N (βξ i, 1), xi ∼ N (ξ i, 1), independent


θ = (β, ξ 1, ..., ξ n) ∈ Θ
F = SIMEX experiment:


T
T
Y = (y1, ..., yn) , X = (x11, x21..., xnK ) ind.








yi ∼ N (βξ i, 1), xik ∼ N (ξ i, 1 + λk ),


Cov (Y ) = I, Cov(X) = In ⊗ Σ, Cov(X, Y ) = 0


T




Σ = 1K 1K + diag(λ1, ..., λK ),




θ = (β, ξ 1, ..., ξ n) ∈ Θ
11
Comparing chances − Comparing experiments
The deficiency δ (E, F) describes the loss of
information. We say
δ (E, F) = 0 iff F is less informative than E.
F is equivalent to E iff δ (E, F) = 0
and δ (F, E) = 0.
Le Cam (1986):
For E = {Pθ , θ ∈ Θ} and F = {Qθ , θ ∈ Θ}the
deficiency δ (E, F) is exactly
1
δ (E, F) = inf sup kQθ − KPθ k ,
K θ 2
where the infimum is taken over the sets of
all transitions K .
12
It holds
• The original experiment is better than the SIMEX
experiment
δ (E, F) = 0,
because Qθ = KPθ .
• The SIMEX experiment is really worse. But
asymptotically for K → ∞ they are equivalent.
1
0 < δ (F, E) < const.
K
13
SIM EX M ethods −
the best what we can reach!
σ 2x
σ 2y
known: For all n and for K → ∞
bSY M EX = β
bT LS + oP ∗ (1)
β
bASIM EX = β
bT LS + oP ∗ (1)
β
• SYMEX and ASIMEX are a methods for calculation of TLS.
• SYMEX and ASIMEX are approximately best.
• SYMEX and ASIMEX are consistent for fixed
σ 2.
σ 2x known: For all n and for K → ∞
bSIM EX = β
bM M E + oP ∗ (1) .
β
14
Why should we use SIMEX methods instead
of TLS?
• Multicollinarity?
• Computational advances?
• Is it only a theoretical result for justification
of SIMEX methods?
15
ASIM EX f or N ON LIN EAR M odels
∗ T
∗
) by adding
, ..., ynk
(1) Simulate Yk∗ = (y1k
pseudo errors
p
∗
yik = yi + λk ε∗ik
(2) Calculate
b
θ(k) = arg min Q (Yk∗, θ)
and
θ∈Θ
³
´
∗ b
Qk = Q Yk , θ(k)
(3) Fit a parametric model to b
θ(1), .., b
θ(K)
K °
°2
X
°
°
θ(k) − f (λk )°
fb = arg min
°b
f ∈F
k=1
(4) Fit a parametric model to Q1, ..., QK
K
X
qb = arg min
(Qk − q(λk ))2
q∈Q
k=1
(5) Extrapolate the criterium function to zero:
qb(λ∗∗) = 0
(6) Define the ASIMEX by : b
θASIM EX = fb(λ∗∗).
16
Man kan säga SIMEX är en trick för att uppfylla varje statistikers dröm om fler och exaktere observationer.
??????
17