1. education
2. graduate school

Largest eigenvalues of the sample covariance matrix
for p-variate time series with heavy-tails
Richard A. Davis, Columbia University
Thomas Mikosch, University of Copenhagen
Oliver Pfaffel, Munich Re
September 8–11, 2013
EVT2013 Extremes in Vimeiro Today
In Honour of Maria Ivette Gomes
Vimeiro, Portugal
Davis (Columbia University)
Extremes in Vimeiro Today
September 8–11, 2013
1
Motivation
Large dimensional data sets appear in many quantitative fields like
finance, environmental sciences, wireless communications, fMRI, and
genetics.
Structure in this data can often be analyzed via sample covariances.
PCA is used to transform data to a new set of variables, the principal
components, ordered s.t. the first few retain most of the variation of
the data.
This suggests the need for an eigenvalue decomposition of the sample
covariance matrix.
Davis (Columbia University)
Extremes in Vimeiro Today
September 8–11, 2013
2
Game Plan
The Setup
Background
Main result for linear time series with linear dependence between
rows
Corollaries and applications
Elements of the proof I
Elements of the proof II
Extensions to more general linear structure
Random coefficient models and hidden Markov models
Extension to nonlinear models–stochastic volatility and GARCH(1,1)
Davis (Columbia University)
Extremes in Vimeiro Today
September 8–11, 2013
3
The Setup
Data matrix: A p × n matrix X consisting of n observations of a
p-dimensional time series, i.e.,

 X11 X12 · · ·
 X
 21 X22 · · ·
X =  .
..
..
 ..
.
.

Xp1 Xp2 · · ·
X1n
X2n
..
.
Xpn




 .


Sample covariance matrix: the p × p sample covariance matrix
(normalized) is given by
 n
p
X

ˆ

XX = nΓ(0) = 
Xit Xjt 
T
t =1
.
i ,j =1
Objective: study the ordered eigenvalues
λ(1) ≥ λ(2) ≥ . . . ≥ λ(p )
of the p × p sample covariance matrix XX T .
Davis (Columbia University)
Extremes in Vimeiro Today
September 8–11, 2013
4
The Setup-continued
Data matrix and sample covariance matrix:

 X11 X12 · · ·
 X
 21 X22 · · ·
X =  .
..
..
 ..
.
.

Xp1 Xp2 · · ·
X1n
X2n
..
.
Xpn




ˆ 0)
 and XX T = nΓ(


Note that if the rows are independent and identically distributed
ergodic time series (with mean 0 and variance 1), then for p fixed,
P
ˆ 0) →
Γ(
Ip .
Relation to PCA: λ(1) is the empirical variance of the first principal
component, λ(2) of the second, and so on.
Davis (Columbia University)
Extremes in Vimeiro Today
September 8–11, 2013
5
Known results for the largest eigenvalue
Assume the entries of X are iid Gaussian (with mean zero and
variance one)
For n → ∞ and fixed p, Anderson [1963] proved that
r
n λ(1)
d
− 1 → N(0, 1) .
2 n
!
Johnstone [2001] showed that for p , n → ∞ s.t. p /n → γ ∈ (0, ∞)
√
n+
q
3
√1
n
√
+
Davis (Columbia University)



λ(1)
 d
 √
−
1
 → Tracy-Widom distribution
2
√

n+ p

p 
√1
p
Extremes in Vimeiro Today
September 8–11, 2013
6
Our objective
The assumption of Gaussianity in Johnstone’s result can be relaxed
to a moment condition (c.f. Four Moment Theorem by Tao and
¨ Johansson, Pech
´ e,
´ Schlein,
Vu [2011]; and work by Erdos,
Soshnikov, Yau and others).
BUT: in applications one often has neither independent observations,
nor Gaussianity or even the existence of sufficient moments.
This lead us to consider heavy-tailed random matrices with dependent
entries.
Davis (Columbia University)
Extremes in Vimeiro Today
September 8–11, 2013
7
Setting for our results
Suppose X = (Xit )i ,t , i = 1, . . . , p, t = 1, . . . , n, with
Xit =
∞ X
∞
X
cj θk Zi −k ,t −j .
j =0 k =0
The noise (Zi ,t ) is iid with regularly varying tails of index α ∈ (0, 4)
(infinite fourth moment), i.e.,
n P (|Z11 | > an x ) → x −α as n → ∞, for x > 0,
(an = L (n)n1/α ) and
lim
x →∞
P (Z11 > x )
= p+
P (|Z11 | > x )
and
lim
x →∞
P (Z11 ≤ −x )
= 1 − p+
P (|Z11 | > x )
|cj |δ < ∞,
0
k =0 |θk | < ∞ for some δ < min{1, α} and δ < min{1, α/2} .
Summability assumptions on the coefficients:
P∞
δ0
Davis (Columbia University)
Extremes in Vimeiro Today
P∞
j =0
September 8–11, 2013
8
Setting for our results (cont)
Let λ1 , . . . , λp be the eigenvalues of



if α ∈ (0, 2),
XX T ,


T
T
XX − E (XX ), if α ∈ (2, 4).
Let (Ds ) be the iid sequence given by
(n)
Ds = Ds
=
n X
∞
X
cl2 Zs2,t −l .
t =1 l =0
Note:
1
The Ds play a key role in determining the asymptotic properties of the
ordered eigenvalues λ(1) ≥ · · · ≥ λ(p ) .
2
Ds is the partial sum of a linear process
noise is regularly varying with index α/2.
P∞
Davis (Columbia University)
Extremes in Vimeiro Today
2
l =0 cl Zs ,t −l ,
t ∈ Z, whose
September 8–11, 2013
9
Theorem (First main result)
Let p = pn → ∞ be a sequence satisfying certain growth conditions (to be
specified later) and suppose k = kp → ∞ is any sequence such that
P
2
k 2 = o (p ). Set Θ = ∞
i =0 θi .
1
If α ∈ (0, 2), then
−2
anp
2
P
max λ(i ) − D(i ) Θ ∨ max λ(i ) → 0 ,
i =1,...,k
n → ∞.
k +1≤i ≤p
If α ∈ (2, 4), then
P
−2
anp
max e
λ(i ) − (D`i − ED ) Θ ∨ max |λ(i ) | → 0 ,
i =1,...,k
k +1≤i ≤p
n → ∞.
˜(1) , . . . , λ
˜(p ) are the ordered eigenvalues (λi ) according to their
Note: 1) λ
absolute values.
2) The `i ’s are defined via
˜ `1 ≥ · · · ≥ D
˜ `p ,
D
Davis (Columbia University)
˜ s = |Ds − ED | .
where D
Extremes in Vimeiro Today
September 8–11, 2013
10
Corollaries and applications
Theorem (Point process convergence)
Under the conditions of the previous theorem, we have the point process
convergence,
Np :=
p
X
d
anp
−2 λ → N =
i
∞
X
C ΘΓ−2/α ,
i =1
i =1
i
where Γi = E1 + . . . + Ei is the cumulative sum of iid standard (i.e., mean
one) exponentially distributed rv’s,
C=
∞
X
cj2 and Θ =
j =0
∞
X
θk2 .
k =0
(N is a Poisson process with E (N (dx )) = (C Θ)α/2 α/2x −α/2−1 dx.)
Davis (Columbia University)
Extremes in Vimeiro Today
September 8–11, 2013
11
The largest eigenvalues
The theorem implies the joint convergence of the m-largest
eigenvalues
d
−2
λ(1) , . . . , λ(m) → C Θ Γ−1 2/α , . . . , Γ−m2/α .
anp
(1)
For independent entries this was shown by Soshnikov [2006] for
´ e´ [2009] for 2 ≤ α < 4.
α < 2, and by Auffinger, Ben Arous and Pech
Davis (Columbia University)
Extremes in Vimeiro Today
September 8–11, 2013
12
The largest eigenvalues
The theorem implies the joint convergence of the m-largest
eigenvalues
d
−2
λ(1) , . . . , λ(m) → C Θ Γ−1 2/α , . . . , Γ−m2/α .
anp
(1)
For independent entries this was shown by Soshnikov [2006] for
´ e´ [2009] for 2 ≤ α < 4.
α < 2, and by Auffinger, Ben Arous and Pech
d
2
(λ(1) /(anp
C Θ))−α/2 → Γ1
Davis (Columbia University)
Extremes in Vimeiro Today
September 8–11, 2013
12
The largest eigenvalues
The theorem implies the joint convergence of the m-largest
eigenvalues
d
−2
λ(1) , . . . , λ(m) → C Θ Γ−1 2/α , . . . , Γ−m2/α .
anp
(1)
For independent entries this was shown by Soshnikov [2006] for
´ e´ [2009] for 2 ≤ α < 4.
α < 2, and by Auffinger, Ben Arous and Pech
d
2
(λ(1) /(anp
C Θ))−α/2 → Γ1
By the continuous mapping theorem,
λ(1)
d
λ(1) + · · · + λ(m)
Davis (Columbia University)
→
Γ−1 2/α
Γ−1 2/α + · · · + Γ−m2/α
Extremes in Vimeiro Today
,
n → ∞.
September 8–11, 2013
12
The largest eigenvalues
The theorem implies the joint convergence of the m-largest
eigenvalues
d
−2
λ(1) , . . . , λ(m) → C Θ Γ−1 2/α , . . . , Γ−m2/α .
anp
(1)
For independent entries this was shown by Soshnikov [2006] for
´ e´ [2009] for 2 ≤ α < 4.
α < 2, and by Auffinger, Ben Arous and Pech
d
2
(λ(1) /(anp
C Θ))−α/2 → Γ1
By the continuous mapping theorem,
λ(1)
d
λ(1) + · · · + λ(m)
→
Γ−1 2/α
Γ−1 2/α + · · · + Γ−m2/α
,
n → ∞.
In fact, we have more!!
Davis (Columbia University)
Extremes in Vimeiro Today
September 8–11, 2013
12
Corollaries and applications
Under the conditions of the theorem, the following limit results hold.
1
If α ∈ (0, 2), then
Γ−2/α
d
→ P∞ 1 −2/α ,
λ1 + · · · + λp
i =1 Γi
λ(1)
n → ∞,
where the limit is independent of (θk ) and (cl ).
2
If α ∈ (2, 4) then
λ(1)
d
λ1 + · · · + λp
→
Γ−1 2/α
,
ξα/2
n → ∞,
and the limit is independent of (θk ) and (cl ). The LHS is the
proportion of the sample variance explained by the variance of the
largest principal component.
Davis (Columbia University)
Extremes in Vimeiro Today
September 8–11, 2013
13
Application–consistency
For α ∈ (2, 4),
d
−2
anp
λ(1) → C ΘΓ−1 2/α ,
where λ(1) is the largest eigenvalue of Xn XnT − E (Xn XnT ).
Now λ(1) /n is the largest eigenvalue of n−1 Xn XnT − Γn (0), where
Γn (0) = [E (Xit Xjt )]pi,j =1 ,
Since n−1 Xn XnT − Γn (0) → 0 element-wise, one might expect consistency
of the eigenvalues, i.e.,
λ(1) /n → 0 .
−2 → 0 or when p = o (n (α−2)/2 ) .
But this is true if and only if nanp
Davis (Columbia University)
Extremes in Vimeiro Today
September 8–11, 2013
14
QQ−Plot via ratios of partial sums of the largest eigenvalue
Normalized partial sum of the transf. largest ev
1
0.9
Xt ∼ Pareto(α = 1.3) iid
0.8
0.7
n=p=1000
Number of replications: 200
0.6
0.5
0.4
0.3
0.2
0.1
0
0
0.2
0.4
0.6
0.8
1
i/100
Davis (Columbia University)
Extremes in Vimeiro Today
September 8–11, 2013
15
QQ−Plot via ratios of partial sums of the largest eigenvalue
Normalized partial sum of the transf. largest ev
1
0.9
0.8
0.7
Xt + 0.7Xt−1 = Zt
Zt ∼ Pareto(α = 1.3) iid
n=p=1000
Number of replications: 200
0.6
0.5
0.4
0.3
0.2
0.1
0
0
0.2
0.4
0.6
0.8
1
i/100
Davis (Columbia University)
Extremes in Vimeiro Today
September 8–11, 2013
16
Growth conditions on pn
Typical entry in XX T involves sums of terms involving squares Zs2 and
cross-products Zs1 Zs2 with s1 , s2 .
Davis (Columbia University)
Extremes in Vimeiro Today
September 8–11, 2013
17
Growth conditions on pn
Typical entry in XX T involves sums of terms involving squares Zs2 and
cross-products Zs1 Zs2 with s1 , s2 .
From Embrechts and Goldie (1980),
P (|Z1 | > x ) = L1 (x )x −α
and P (|Z1 Z2 | > x ) = L2 (x )x −α
where L1 and L2 are SV functions.
Davis (Columbia University)
Extremes in Vimeiro Today
September 8–11, 2013
17
Growth conditions on pn
Typical entry in XX T involves sums of terms involving squares Zs2 and
cross-products Zs1 Zs2 with s1 , s2 .
From Embrechts and Goldie (1980),
P (|Z1 | > x ) = L1 (x )x −α
and P (|Z1 Z2 | > x ) = L2 (x )x −α
where L1 and L2 are SV functions.
Precise conditions on pn rely on the asymptotic relationship between
L1 and L2 .
Davis (Columbia University)
Extremes in Vimeiro Today
September 8–11, 2013
17
Growth conditions on pn
Typical entry in XX T involves sums of terms involving squares Zs2 and
cross-products Zs1 Zs2 with s1 , s2 .
From Embrechts and Goldie (1980),
P (|Z1 | > x ) = L1 (x )x −α
and P (|Z1 Z2 | > x ) = L2 (x )x −α
where L1 and L2 are SV functions.
Precise conditions on pn rely on the asymptotic relationship between
L1 and L2 .
General conditions:
2
For α ∈ (0, 1), lim supn→∞ p [n p P (|Z1 Z2 | > anp
)] = 0 .
For α ∈ (1, 2), there exists γ ∈ (α, 2) but arbitrarily close to α such
2
that lim supn→∞ p γ [n p P (|Z1 Z2 | > anp
)] = 0 .
For α ∈ (2, 4), there exists γ ∈ (α, 4) arbitrarily close to α such that
2
lim supn→∞ nγ/2−1 p γ [n p P (|Z1 Z2 | > anp
)] = ∞ .
Davis (Columbia University)
Extremes in Vimeiro Today
September 8–11, 2013
17
Growth conditions on pn
Case P (Z1 > x ) ∼ cx −α : Here L2 (x ) = C log(x ).
For α ∈ (0, 2),
pn = O (nβ ) ,
for any β > 0 .
Can allow for a touch faster growth rate (pn = O (exp{cn }), where
cn2 /n → 0 in the α ∈ (0, 1) case.
For α ∈ (2, 4),
pn = O (nβ ) ,
β ∈ (0, (4 − α)/[2(α − 1)]) .
This excludes the case pn ∼ cn.
Davis (Columbia University)
Extremes in Vimeiro Today
September 8–11, 2013
18
Elements of the proof I:
Special case: Xi ,t = θ0 Zi ,t + θ1 Zi −1,t
n
X
Xit2
=
t =1
n
X
Xit Xi +1,t
t =1
n
X
θ02 Zi2,t + θ12 Zi2−1,t +2θ0 θ1
Zi ,t Zi −1,t
| {z }
|
{z
}
t =1
t =1
tail index α
tail index α/2
=
and
n
X
θ02 Di
= θ0 θ1
+
θ12 Di −1
n
X
+
2
nop (anp
)
2
Zi2,t + nop (anp
)
t =1
2
= θ0 θ1 Di + nop (anp
)
Davis (Columbia University)
Extremes in Vimeiro Today
September 8–11, 2013
19
Elements of the proof I:
Special case: Xi ,t = θ0 Zi ,t + θ1 Zi −1,t
n
X
Xit2
=
t =1
n
X
n
X
θ02 Zi2,t + θ12 Zi2−1,t +2θ0 θ1
Zi ,t Zi −1,t
| {z }
|
{z
}
t =1
t =1
tail index α
tail index α/2
=
and
n
X
Xit Xi +1,t
θ02 Di
= θ0 θ1
+
θ12 Di −1
n
X
+
2
nop (anp
)
2
Zi2,t + nop (anp
)
t =1
t =1
2
= θ0 θ1 Di + nop (anp
)
XTi Xi
XTi+1 Xi
XTi+1 Xi
XTi+1 Xi +1
θ02 θ0 θ1
θ0 θ1 θ12
!
!
≈
+
Davis (Columbia University)
0 0
0 θ02
!
Di +
θ12 0
0
0
!
Di −1
Di +1
Extremes in Vimeiro Today
September 8–11, 2013
19
The covariance matrix can be approximated by
T
XX =
p
X
2
Di Mi + op (anp
),
i =1
where Mi is the p × p matrix consisting of all zeros except for a 2 × 2 matrix,
θ02 θ0 θ1
θ0 θ1 θ12
M=
!
,
whose NW corner is pinned to the i th position on the diagonal. For
example,
 2
 θ0 θ0 θ1 0 · · ·

2
 θ0 θ1 θ1 0 · · ·
 0
0
0 ···
M1 = 
 .
..
.. . .
 ..
.
.
.

0
Davis (Columbia University)
0
0
0






.. 
. 

0
0
0
0





M2 = 



Extremes in Vimeiro Today
0
0
0 θ02
0 θ0 θ1
0
..
.
..
.
θ0 θ1
θ12
..
.
0
0
0






.. 
. 

···
···
···
..
.
0
0
0
0
0
September 8–11, 2013
20
Denote the order statistics of the Di ’s by D(1) ≥ D(2) ≥ · · · ≥ D(p ) and write
DLi = D(i ) .
Then,
XX T =
Pp
i =1
2
DLi MLi + op (anp
) in the sense that
−2
anp
kXX T −
p
X
P
DLi MLi k2 → 0 ,
i =1
where
q
kA k2 =
Davis (Columbia University)
largest eigenvalue of AA T (operator 2-norm).
Extremes in Vimeiro Today
September 8–11, 2013
21
Elements of the proof II
For k → ∞ sufficiently slow,
k
X
P
−2 XX T −
DLi MLi → 0 .
anp
i =1
2
Since the Ds are iid, (L1 , . . . , Lp ) is a random permutation of (1, . . . , p )
and hence the set Ak = {|Li − Li −1 | > 1, i = 1, . . . , k } has probability
converging to 1 provided k 2 = o (p ).
Davis (Columbia University)
Extremes in Vimeiro Today
September 8–11, 2013
22
Elements of the proof II
For k → ∞ sufficiently slow,
k
X
P
−2 XX T −
DLi MLi → 0 .
anp
i =1
2
Since the Ds are iid, (L1 , . . . , Lp ) is a random permutation of (1, . . . , p )
and hence the set Ak = {|Li − Li −1 | > 1, i = 1, . . . , k } has probability
converging to 1 provided k 2 = o (p ).
On the set Ak , the matrix ki=1 DLi MLi is block diagonal with nonzero
eigenvalues DLi Θ, i = 1, . . . , k . Here we used the fact that MLi is a
rank 1 matrix with nonzero ev equal to Θ = θ02 + θ12 .
P
Davis (Columbia University)
Extremes in Vimeiro Today
September 8–11, 2013
22
Elements of the proof II
For k → ∞ sufficiently slow,
k
X
P
−2 XX T −
DLi MLi → 0 .
anp
i =1
2
Since the Ds are iid, (L1 , . . . , Lp ) is a random permutation of (1, . . . , p )
and hence the set Ak = {|Li − Li −1 | > 1, i = 1, . . . , k } has probability
converging to 1 provided k 2 = o (p ).
On the set Ak , the matrix ki=1 DLi MLi is block diagonal with nonzero
eigenvalues DLi Θ, i = 1, . . . , k . Here we used the fact that MLi is a
rank 1 matrix with nonzero ev equal to Θ = θ02 + θ12 .
P
By Weyl’s inequality
k
X
P
−2
−2 XX T −
anp
max |λ(i ) − DLi Θ| ≤ anp
DLi MLi → 0 .
i =1,...,k
i =1
2
Davis (Columbia University)
Extremes in Vimeiro Today
September 8–11, 2013
22
Elements of the proof II (cont)
(n)
Large deviations: Ds
=
P∞
Pn
t =1
2 2
l =0 cl Zs ,t −l
.
∞
P (D > x ) X
1
α |
c
|
−
sup j → 0,
2
x >bn nP (Z > x )
1
j =0
where bn /an2 → ∞.
Classical EVT plus large deviations implies:
p
X
i =1
Davis (Columbia University)
anp
−2 λ ∼
i
p
X
d
anp
−2 Θ D → N =
i
i =1
Extremes in Vimeiro Today
∞
X
i =1
ΘC Γ−2/α .
i
September 8–11, 2013
23
Elements of the proof II
Important tool: kA k2 =
2-norm).
p
largest eigenvalue of AA T (operator
Define D ∈ Rp ×p by Dii = (XX T )ii and Dij = 0 for i , j. Then
P
−2 anp
XX T − D 2 → 0 as p , n → ∞.
By Weyl’s inequality
n
X
T
P
−2 2 −2 2 → 0 as p , n → ∞
anp
λ
−
max
X
≤
a
XX
−
D
(
1
)
np
it 1 ≤i ≤p
t =1
and likewise for λ(2) , λ(3) , . . .
Hence, we “only” have to derive the extremal behavior of the diagonal
P
elements ( nt=1 Xit2 )i of XX T .
Davis (Columbia University)
Extremes in Vimeiro Today
September 8–11, 2013
24
QQ−Plot via ratios of partial sums of the largest eigenvalue
Normalized partial sum of the transf. largest ev
1
0.9
0.8
0.7
Xit = Zit + θi Zi,t−1
Zit ∼ Pareto(α = 1.3) iid
θi = 0.8θi−1 + ξi
ξi ∼ Uniform(−1, 1) iid
n=p=1000
Number of replications: 200
0.6
0.5
0.4
0.3
0.2
0.1
0
0
0.2
0.4
0.6
0.8
1
i/100
Davis (Columbia University)
Extremes in Vimeiro Today
September 8–11, 2013
25
Extensions to more general linear structure
Preview example:
Xi ,t
Davis (Columbia University)
= Zi ,t − Zi ,t −1 − 2Zi −1,t + 2Zi −1,t −1
Extremes in Vimeiro Today
September 8–11, 2013
26
Extensions to more general linear structure
Preview example:
Xi ,t
= Zi ,t − Zi ,t −1 − 2Zi −1,t + 2Zi −1,t −1
Then,
Np :=
p
X
i =1
Davis (Columbia University)
d
anp
−2 λ → N =
i
∞ X
8Γ−2/α + 2Γ−2/α .
i =1
Extremes in Vimeiro Today
i
i
September 8–11, 2013
26
Extensions to more general linear structure
Preview example:
= Zi ,t − Zi ,t −1 − 2Zi −1,t + 2Zi −1,t −1
Xi ,t
Then,
Np :=
p
X
i =1
d
anp
−2 λ → N =
i
∞ X
8Γ−2/α + 2Γ−2/α .
i =1
i
i
Results:
d
−2/α
−2 λ
anp
(1) → 8Γ1
Davis (Columbia University)
Extremes in Vimeiro Today
September 8–11, 2013
26
Extensions to more general linear structure
Preview example:
= Zi ,t − Zi ,t −1 − 2Zi −1,t + 2Zi −1,t −1
Xi ,t
Then,
Np :=
p
X
i =1
d
anp
−2 λ → N =
i
∞ X
8Γ−2/α + 2Γ−2/α .
i =1
i
i
Results:
d
−2/α
−2 λ
anp
(1) → 8Γ1
d
−2 (λ , λ ) → (8Γ−2/α , 2Γ−2/α ∨ 8Γ−2/α )
anp
(1) (2)
1
1
2
Davis (Columbia University)
Extremes in Vimeiro Today
September 8–11, 2013
26
Extensions to more general linear structure
Preview example:
= Zi ,t − Zi ,t −1 − 2Zi −1,t + 2Zi −1,t −1
Xi ,t
Then,
Np :=
p
X
d
anp
−2 λ → N =
i
i =1
∞ X
8Γ−2/α + 2Γ−2/α .
i
i =1
i
Results:
d
−2/α
−2 λ
anp
(1) → 8Γ1
d
−2 (λ , λ ) → (8Γ−2/α , 2Γ−2/α ∨ 8Γ−2/α )
anp
(1) (2)
1
1
2
λ(1)
λ1 + · · · + λp
Davis (Columbia University)
d
→
−2/α
Γ1
8
,
P
∞
10
Γ−2/α
i =1
n → ∞.
i
Extremes in Vimeiro Today
September 8–11, 2013
26
Extensions to more general linear structure
Suppose X = (Xit )i ,t , i = 1, . . . , p, t = 1, . . . , n, with
Xit =
m
∞ X
X
h (k , l )Zi −k ,t −l ,
j =0 k =0
Summability assumptions on the coefficients:
m
X
|h (k , l )|δ < ∞ for some δ < min{1, α}
k =0
and
∞ X
∞
X
( |h (k , l )|)α/2 < ∞,
for k = 0, . . . , m.
t =0 l =t
Davis (Columbia University)
Extremes in Vimeiro Today
September 8–11, 2013
27
Extensions to more general linear structure
Set hi = (hi0 , hi1 , . . .)0 and define the matrix H = (h0 , . . . , hm ). Let
M = H0H .
i.e., the (i , j )th entry of M is
Mij = h0i hj =
∞
X
hil hjl ,
i , j = 0, . . . , m .
l =0
By construction, M is symmetric and non-negative definite and has
ordered eigenvalues
vm ≤ · · · ≤ v1 .
Let r be the rank of M so that vr > 0 while vr +1 = 0 if r < m.
This matrix plays the role of the M matrix in previous example.
Davis (Columbia University)
Extremes in Vimeiro Today
September 8–11, 2013
28
Extensions to more general linear structure
For previous numerical example,
1 −1 0 0 · · ·
−2 2 0 0 · · ·
T
H =
so that
T
M=H H=
2 0
0 8
!
!
which has eigenvalues 2 and 8.
Theorem (Second main result)
Np :=
p
X
i =1
Davis (Columbia University)
d
anp
−2 λ → N =
i
r X
∞
X
j =1 i =1
Extremes in Vimeiro Today
vj Γ−2/α .
i
September 8–11, 2013
29
Random coefficient models–dependence between rows
X = (Xit ) with
∞
X
Xit =
cj (θi )Zi ,t −j
j =−∞
where cj (·) is a family of functions s.t.
supθ |cj (θ)| ≤ c˜j with
∞
X
|c˜j |δ < ∞, δ < min{1, α}, and
j =−∞
(θi ) is a stationary ergodic sequence independent of (Zit )i ,t .
Conditionally on (θi ), each process (Xit )t in the rows of X has a
different set of coefficients (cj (θi ))j .
Unconditionally, the row processes are identically distributed with
regularly varying tail probabilities.
Rows of X are dependent if the θi ’s are dependent.
Davis (Columbia University)
Extremes in Vimeiro Today
September 8–11, 2013
30
Random coefficient models
Theorem (Random coefficient models)
Suppose pn , n → ∞ such that
lim sup
n→∞
pn
<∞
nβ
for some β > 0 satisfying
1
2
β < ∞ if α ∈ (0, 1], and
n
o
−α 1
β < max 2α−
,
1 2 if α ∈ (1, 2).
Then, conditionally on (θi ) as well as unconditionally,
p
X
d
→N=
anp
−2 λ
(i )
i =1
Davis (Columbia University)
∞
X
i =1
P
α/2 2/α
2
Γi−2/α E ∞
j =−∞ cj (θ1 )
Extremes in Vimeiro Today
.
September 8–11, 2013
31
The largest eigenvalue
In particular we have
α/2 −1
 

∞

 λ(1) −α/2 d  X

2




cj (θ1 )  Γ1
→ E 2

anp

j =−∞
= constant × standard exponential
as p , n → ∞
Also a good approximation for finite p and n?
Davis (Columbia University)
Extremes in Vimeiro Today
September 8–11, 2013
32
Hidden Markov models
Suppose that (θi ) is either
an irreducible Markov chain on a countable state space Θ, or
positive Harris in the sense of Meyn and Tweedie [2009].
If (θi ) has a stationary probability distribution π then, conditionally on
(θi ) as well as unconditionally,
p
X
i =1
Davis (Columbia University)
d
anp
→N=
−2 λ
(i )
∞
X
i =1
Γ−i 2/α
2/α
R P
| j cj2 (θ)|α/2 π(d θ)
Extremes in Vimeiro Today
.
September 8–11, 2013
33
Stochastic volatility models—special case
Suppose the rows are independent copies of the SV process given by
Xt = σt Zt
where (Zt ) is iid RV(α) and (ln σ2t ) is a purely nondeterministic stationary
Gaussian process (this can be weakened), independent of (Zt ).
Theorem
Suppose pn , n → ∞ such that
lim sup
n→∞
pn
< ∞ , for some β > 0 satisfying
nβ
1
β < ∞ if α ∈ (0, 1), and
2
β<
2−α
α−1
if α ∈ (1, 2).
Then, we have the point process convergence,
Np :=
p
X
d
anp
→N=
−2 λ
(i )
i =1
Davis (Columbia University)
∞
X
i =1
Extremes in Vimeiro Today
Γ−2/α .
i
September 8–11, 2013
34
Stochastic volatility models—special case
Point process convergence:
Np :=
p
X
d
anp
→N=
−2 λ
(i )
∞
X
Γ−2/α .
i =1
i =1
i
Remarks:
Proof uses a large deviation result of Davis and Hsing (1995); see
also Mikosch and Wintenberger (2012).
Likely that we can weaken the restriction on β
Similar results hold for GARCH processes if Xt is RV(α) with
α ∈ (0, 2).
Davis (Columbia University)
Extremes in Vimeiro Today
September 8–11, 2013
35
References
Richard A. Davis, Oliver Pfaffel and Robert Stelzer
Limit Theory for the largest eigenvalues of sample covariance matrices with
heavy-tails.
Stoch. Proc. Appl. 24, 2014, 18–50.
Ian M. Johnstone
On the Distribution of the Largest Eigenvalue in Principal Components
Analysis.
Ann. Statist., 2001, 29, 295-327.
Alexander Soshnikov
Poisson Statistics for the Largest Eigenvalues in Random Matrix Ensembles.
Lect. Notes in Phys., 2006, 690, 351-364.
´
´ e´
Antonio Auffinger, Gerard
Ben Arous, and Sandrine Pech
Poisson convergence for the largest eigenvalues of heavy tailed random
matrices.
Ann. Inst. H. Poincare´ Probab. Statist., 2009, 45, 589-610.
Davis (Columbia University)
Extremes in Vimeiro Today
September 8–11, 2013
36
Happy Birthday and Best Wishes Ivette!!!
Davis (Columbia University)
Extremes in Vimeiro Today
September 8–11, 2013
37