Probability
Adding and Multiplying Probabilities
Permutations
There is only 1 one-letter word containing the letter A:
A
There are 2 two-letter words containing the letters AB:
AB
BA
There are 6 three-letter words containing the letters AB:
ABC
BAC
CAB
ACB
BCA
CBA
There are 24 different four-letter words containing the four letters
ABCD:
1
ABCD
BACD
CABD
DABC
ABDC
BADC
CADB
DACB
ACBD
BCAD
CBAD
DBAC
ACDB
BCDA
CBDA
DBCA
ADBC
BDAC
CDAB
DCAB
ADCB
BDCA
CDBA
DCBA
For N -letter words, there are N ! possible permutations: N choices
for the first letter, N − 1 for the second, and so on:
N ! = N · (N − 1) · (N − 2) · · ·
The number of two-letter words that can be made with the four
letters in ABCD is written 4 P2 , which is 12:
ABCD
BACD
CABD
DABC
ABDC
BADC
CADB
DACB
In general
N Pk
N P2
ACBD
BCAD
CBAD
DBAC
ACDB
BCDA
CBDA
DBCA
ADBC
BDAC
CDAB
DCAB
ADCB
BDCA
CDBA
DCBA
→
→
→
→
AB
BA
CA
DA
AC
BC
CB
DB
AD
BD
CD
DC
= N · (N − 1), and
= N · (N − 1) · (N − 2) · · · (N − k + 1)
N · (N − 1) · (N − 2) · · · (N − k + 1) · (N − k) · (N − k − 1) · · · (1)
=
(N − k) · (N − k − 1) · · · (1)
N!
=
(N − k)!
2
If order is not important, we are looking for combinations rather
than permutations:
N Ck
=
N Pk
k Pk
=
N!
(N − k)!k!
This is the same as dividing N items into two groups, with N1
items in the first group, and N2 in the second (with, of course,
N1 + N2 = N .) Again with order irrelevant:
N CN1
=
N!
N!
=
(N − N1 )!N1 !
N1 !N2 !
If we have 5 objects, for example, that we want to divide into
groups of 3 and 2, we have 5 C3 = 5!/3!2! = 10 combinations :
abc:de
ade:bc
abd:ce
bcd:ae
abe:cd
bce:ad
Multinomial coefficients
(x + y)1
= x+y
1!
1!
=
x+
y
1!0!
0!1!
3
acd:be
bde:ac
ace:bd
cde:ab
Note that 5 C3 = 5 C2 .
=
1 C1 x
+ 1 C0 y
(x + y)2
= x2 + 2xy + y 2
2! 2
2!
2! 2
=
x +
xy +
y
2!0!
1!1!
0!2!
= 2 C2 x2 + 2 C1 xy + [2 C0 y 2
(x + y)3
= x3 + 3x2 y + 3xy 2 + y 3
3! 3
3! 2
3!
3! 3
=
x +
x y+
xy 2 +
y
3!0!
2!1!
1!2!
0!3!
= 3 C3 x2 + 3 C1 x2 y + 3 C2 xy 2 + 3 C0 y 2
In general, we can write
(x + y)N =
N
X
N CN1 x
N1 N −N1
y
N1 =0
Why does this work?
(x + y)6 =
x6
x5 y
x4 y 2
(x + y)·(x + y)·(x + y)·(x + y)·(x + y)·(x + y)
a · b · c · d · e · f
abcdef:
abcde:f, abcdf:e, abcef:d, etc. (6 C5 = 6 terms)
abcd:ef, abce:df, abcf:de, etc. (6 C4 = 15 terms)
We extend this to the task of dividing N items into three groups,
with N1 , N2 , and N3 elements, respectively. We already know that
there are N CN1 ways to form the first group; then, we have N − N1
elements left to apportion. There are thus N −N1 CN2 possible second groups, which, once formed, uniquely determine the remaining
N3 (= N −N1 −N2 ) elements in the third group. The total number
of possible combinations is
N CN1 ·N −N1 CN2
=
N!
(N − N1 )!
N!
·
=
(N − N1 )!N1 ! (N − N1 − N2 )!N2 !
N1 !N2 !N3 !
For r groups, we have
N!
N1 !N2 !N3 ! · · · Nr !
which turn out to be the coefficients in a multinomial expansion:
!∗
N
N
N
X
X
X
N!
N
r
(x1 + x2 + · · · + xr ) =
···
xN1 xN2 · · · xN
r
N1 !N2 !N3 ! · · · Nr ! 1 2
N1 =0 N2 =0
Nr =0
where the asterisk calls out the requirement that N1 + N2 + · · · +
Nr = N .
4
Stirling’s approximation
Stirling’s approximation for ln N !:
ln N !
=
ln [N · (N − 1) · (N − 2) · · ·]
=
ln N + ln(N − 1) + ln(N − 2) + · · · + ln 1
=
N
X
ln x
1
Z
≈
N
ln xdx
1
N
≈
(x ln x − x)|1
≈
(N ln N − N ) − (1 ln 1 − 1)
≈
N ln N − N + 1
≈
N ln N − N
This approximation, ln N ! = N ln N − N , is quite poor for small
values of N , but becomes increasingly accurate as N increases:
Means and Moments
The sum of all possible probabilities is always equal to 1:
X
pj = 1
j
Or, it is certain that some event is going to occur.
The mean value of a measurement is given by the sum of all of
the measurements, divided by the total number of measurements.
5
So
Aavg
X Nj
=
j
X
=
N
Aj
p j Aj
j
That is,
Aavg =
X
pj Aj = p19 · 19 + p20 · 20 + . . .
j
We rename the average age the expectation value of A, denoted
hAi.
If, for some reason, we wanted to tally up the average value of
the square of A, we would denote this hA2 i, and computing it is
straightforward:
hA2 i = p19 · 192 + p20 · 202 + . . .
6
In general, if we have any function of j, we can compute its expectation value using
X
hf (j)i =
f (j)pj
j
It is alway true that:
hAi2 ≤ hA2 i
The two are equal only when all values of A are the same. Any
difference between hxi2 and hx2 i gives a standard deviation:
D
E
2
σx2 =
(x − hxi)
X
2
(xj − hxi) pj
=
j
=
X
=
X
x2j − 2hxixj + hxi2 pj
j
j
x2j pj − 2
X
j
7
hxixj pj +
X
j
hxi2 pj
=
X
2
x − 2hxi2 + hxi2
pj
=
2
x − 2hxi2 + hxi2
2
x − hxi2
j
=
and the standard deviation is
p
p
σx = σx2 = hx2 i − hxi2
With continuous variables
Prob(x, x + dx) = ρ(x)dx
and we can extend to definite (finite-length) intervals by integrating
b
Z
Pab =
ρ(x)dx
a
We have
Z
+∞
ρ(x)dx = 1
−∞
and
+∞
Z
hxi =
hx2 i =
xρ(x)dx
−∞
Z +∞
x2 ρ(x)dx
−∞
Z +∞
hf (x)i =
f (x)ρ(x)dx
−∞
σx
=
p
hx2 i − hxi2
8