.
.
Time Series: Lecture 5
Giacomo Livan
[email protected]
March 24, 2015
Giacomo Livan
Time Series: Lecture 5
Recap of lecture 4
Yule-Walker estimators (3.30)-(3.31)
Method of moments for AR(1) and AR(2) models
Method of moments for MA(1) model (example 3.19)
Least squares estimation for AR(1) and AR(2) models
Forecasting (3.45)-(3.49)
Best linear prediction for AR(2) models (example 3.24)
Giacomo Livan
Time Series: Lecture 5
Spectral analysis
Spectral analysis: study of time series in the frequency domain
Discrete Fourier transform of a time series X1 , . . . , Xn
n
1 ∑
d(ωj ) = √
Xt e−2πiωj t ,
n t=1
where ωj = j/n (j = −(n − 1), . . . , 0, . . . , n − 1) are known as fundamental
frequencies
The Fourier transform “breaks” a signal (a time series in this case) into basic
sinusoids with frequencies ωj . The inverse transform reads:
n−1
1 ∑
Xt = √
d(ωj ) e2πiωj t
n j=0
Reminder: given a sinusoid A cos(2πωt + ϕ) we have the following
A: amplitude
ω: frequency (1/ω: period)
ϕ: phase
Giacomo Livan
Time Series: Lecture 5
Periodogram
Definition:
I(ωj ) = |d(ωj )|2
=
1
n
=
1
n
(
(
n
∑
)2
Xt cos (2πωj t)
t=1
+
1
n
( n
∑
)2
Xt sin (2πωj t)
t=1
( n
(
))2
(
))2
j
1 ∑
j
Xt cos 2π t
Xt sin 2π t
+
n
n t=1
n
t=1
n
∑
Symmetry around 0: I(ωj ) = I(−ωj )
Symmetry around 1/2 (folding frequency) when sampling time and natural time
coincide: I(ωj ) = I(1 − ωj )
Rescaled periodogram: P (ωj ) = 4/nI(ωj ) = 4/n|d(ωj )|2
For each sinusoid in the original signal (time series) the periodogram picks up the
corresponding amplitude
Giacomo Livan
Time Series: Lecture 5
Periodogram: example 1
Time series: Xt = A cos(2πωt) + ϵt , where ϵt is Gaussian white noise with variance σ 2
Time series
2
1.5
1
X(t)
0.5
0
-0.5
-1
-1.5
-2
1
2
3
4
5
6
7
8
9
10
0.8
1
t
Periodogram
60
50
I(ω)
40
30
20
10
0
-1
-0.8
-0.6
-0.4
-0.2
0
ω
0.2
0.4
0.6
In this example: A = 1, ω = 0.3, σ = 0.5
Giacomo Livan
Time Series: Lecture 5
Periodogram: example 2
Time series: Xt = A1 cos(2πω1 t) + A2 cos(2πω2 t) + ϵt , where ϵt is Gaussian white
noise with variance σ 2
Time series
2.5
2
1.5
1
X(t)
0.5
0
-0.5
-1
-1.5
-2
-2.5
1
2
3
4
5
t
6
7
8
9
10
0.9
1
Periodogram
60
50
I(ω)
40
30
20
10
0
0
0.1
0.2
0.3
0.4
0.5
ω
0.6
0.7
0.8
In this example: A1 = 0.75, A2 = 1, ω1 = 0.3, ω2 = 0.7, σ = 0.5
Giacomo Livan
Time Series: Lecture 5
Spectral representation
.
Lemma
.
The autocovariance function γ(h) of an ARMA(p, q) process satisfies the absolute
summability condition:
∞
∑
|γ(h)| < ∞
h=−∞
.
.
Spectral representation theorem
.
If the autocovariance function of a stationary process satisfies the absolute
summability condition, then it can be represented as
∫
1/2
γ(h) =
h = 0, ±1, ±2, . . . ,
e2πiωh f (ω)dω
−1/2
where f (ω) is the spectral density
f (ω) =
∞
∑
γ(h) e−2πiωh
− 1/2 ≤ ω ≤ 1/2
h=−∞
From its definition it is easy to verify that the spectral density is even
(f
. (ω) = f (−ω)), so that it is enough to study it in [0, 1/2]
Giacomo Livan
Time Series: Lecture 5
Spectral representation 2
.
Propositions
.
The periodogram I(ωj ) is an estimator of the spectral density f (ω)
I(ω) = fˆ(ω)
.
Increasing n does not guarantee that estimates will improve
Example 1: white noise
{
γ(h) =
σ2
0
for h = 0
for h =
̸ 0
=⇒ f (ω) = σ 2
Example 2: ARMA(p, q), Φ(B)Xt = Θ(B)ϵt
f (ω) = σ 2
Giacomo Livan
|Θ(e−2πiω )|2
|Φ(e−2πiω )|2
Time Series: Lecture 5
Least squares estimates of Fourier coefficients
From Fourier analysis we know that any time series can be written as
n−1
1 ∑
d(ωj ) e2πiωj t
Xt = √
n j=0
Taking into account the symmetry of the periodogram around 1/2 we can rewrite
the above (for n odd) as
∑(
n−1
2
Xt = β1 +
j=1
(
)
(
))
j
j
aj cos 2π t + bj sin 2π t
n
n
Least squares estimation of the parameters yields (recall problem set 2)
a
ˆj
=
ˆbj
=
)
(
n
2∑
j
Xt cos 2π t
n t=1
n
(
)
n
j
2∑
Xt sin 2π t
n t=1
n
The rescaled periodogram reads:
P (ωj ) = a2j + b2j
Giacomo Livan
Time Series: Lecture 5
Recap
Discrete Fourier transform and its inverse
Periodic processes / sinudoids (4.1)
Periodogram, rescaled periodogram, least squares estimation of Fourier
coefficients (example 4.2)
Spectral density and spectral representation (4.12)-(4.14)
Spectral density of white noise (example 4.4) and ARMA processes (property 4.2)
Giacomo Livan
Time Series: Lecture 5