Stat 761 Winter 2015
Stochastic Processes
Instructor: A. Swishchuk
Lecture 25: Stationary Processes (SP):
Definitions and Examples
Outline
⇒ SP: Definitions
⇒ SP: Examples
1
SP: Definitions
Def. 1. A stationary process is a stochastic process (X(t), t ∈ T ) with
the property that for any positive integer k and any points t1 , t2 , ..., tn , and
h in T, the joint distribution of (X(t1 ), ..., X(tk )) is the same as the joint
distribution of (X(t1 + h), ..., X(tk + h)).
Short Examples.
1. Electrical pulses in communication theory are SPs.
2. The spatial and/or planar distriutions of stars or galaxies are SPs.
3. Economic time series, such as unemployment, gross national product
(GNP), national income, etc., are SPs.
If the mean m(t) = E[X(t)] exists, it follows that this quantity must be
constant, m(t) = m for all t. Similarly, if the second moment E[X(t)]2 is
finite, then the variance σ 2 = E[(X(t) − m)]2 is a constant, independent of
time.
Let t and s be time points, and suppose, that t > s. Using the stationary
property, we compute the covariance
E[(X(t) − m)(X(s) − m)] = E[(X(t − s) − m)(X(0) − m)]
such that the right-hand side depends only on the time difference t − s. If we
define the covariance function,
R(h) = E[(X(h) − m)(X(0) − m)],
then
E[(X(t) − m)(X(s) − m)] = R(|t − s|).
Obviously, σ 2 = R(0).
To standardize the covariance we introduce the so-called the correlation
function or autocorrelation function, defined by
ρ(v) =
1
R(v) = R(v)/R(0).
σ2
Then ρ(0) = 1, and it can be shown (using Schwartz’ inequality) that −1 ≤
ρ(v) for all v.
Def. 2. A covariance stationary process is a stochastic process (X(t), t ∈
T ) havinf finite second moments, E[X(t)]2 < +∞, a constant mean m =
E[X(t)], and a covariance E[(X(t) − m)(X(s) − m)] that depends only on
the time difference |t − s|.
Other terms usedin the literature synonymously with covariance stationary are weakly stationary or wide-sense stationary, and what we have called
s tationary process is often termed strictly stationary to emphasize the distinction. Every SP with the second moments is covariance stationary, and
a covariance stationary process is not in general a stationary. Exception is
only for Gaussian process, because its distribution is defined by the first two
moments.
2
SP: Examples
1. Two Contrasting SPs
i) A sequence Yi of i.i.d.r.v.s is a stationary process. If the variance is σ 2
then the process is covariance stationary, and the covariance function is
2
σ , f or v = 0,
R(v) =
0, f or v 6= 0.
ii) Let Z be a single r.v. with known distribution and set Z0 = Z1 = Z2 =
... = Z. This process is stationary. If the r.v. Z has a finite variance σ 2 , the
the process is covarinace stationary, and the covariance function is
R(v) = σ 2 .
In this way, Yn and Zn are extremes and may be used toexemplify contrasting
behavior of SPs. Observing Y1 , Y2 , ..., Yn provides no information that could
be used to predict Yn+1 , while observing only Z0 enables Z1 , Z2 , ... to be
predicted exactly. Here is a second way that the processes are opposites.
Suppose the process Yn has a finite mean value function m. The by the LLN,
the sample average
Y0 + Y1 + ... + Yn
n
converges to the constant m = E[Y0 ]. No such convergence takes place in the
Zn process. Indeed
Z0 + Z1 + ... + Zn−1
= Z0 = Z
n
and there is just as much ’randomness’ in the nth sample average as there is
in the first observation.
2. Trigonometrical Polynomials Let A and B be i.d.r.v.s having a
mean of zero and variance σ 2 . We suppose that A and B are uncorrelated,
i.e., E[AB] = 0. Fix a frequency ω ∈ [0, π] and for n = 0, ±1, ±2, ... define
Xn = A cos(ωn) + B sin(ωn).
Then E[Xn ] = 0 and
E[Xn Xn+v ] = σ 2 cos(ωv).
The process is covariance stationary. If A and B have a normal distributon
with mean zero and variance σ 2 , the process is Gaussian and thus strictly
stationary.
3. Moving Average Processes Let ξn , n = 0, ±1, ±2, ..., be uncorrelated r.v. having a common mean µ and variance σ 2 . Let a1 , a2 , ..., am be
arbitrary real numbers and consider the process defined by
Xn = a1 ξn + a2 ξn−1 + ... + am ξn−m+1 .
We have
E[Xn ] = µ(a1 + ... + am ),
and
V ar[Xn ] = σ 2 (a21 + ... + a2m ).
The process is covariance stationary.
4. Stationary Markov Chains Let Xn be a stationary MC with stationary probability πi . This process is stationary process, as P (Xn = j) = πj
and P (Xn = i, Xn+1 = j) = P (Xn = i)P (Xn+1 = j|Xn = i) = πi Pij , and so
on. The joint distribution of (Xn , Xn+1 , ..., Xn+k ) does not depend on n.
5. Incremented Poisson Process Let X(t) := N (t + L) − N (t), t ≥
0, L > 0−fixed constant, and N (t) is a Poisson process having rate λ. The
stationary property follows from the stationary and independent increment
assumption of the Poisson process, which implies that the continuation of a
Poisson process at any time s remains a Poisson process. The same statement
valid for standard Wiener process.
Recommended Textbook: ’A First Course in Stochastic Processes’ by
S. Karlin and H. Taylor, Academic Press, 2nd ed., 1975, Chapter 9, Sec. 1.
Recommended Exercises: 1, p.525.