March 23rd , 2015
ECON 451 • Spring 2015
Econ 451 - Homework #4
Due: (Monday) March 30th , 2015
For all problems on this problem set, consider two players playing an infinitely repeated game with discount factor δ in
which they play the following Prisoner’s Dilemma game each period:
C
D
C
3 3
-1 4
D
4 -1
0 0
1. Represent each of the following strategies as an automaton,
a) 4 C then D - Play C in the first four periods, and then D after that, regardless of what the other player does.
b) Delayed Grim - Choose C in period 1 and after any history in which the other player chose C in every period
except, possibly, the previous period; choose D after any other history. (punishment is grim, but its initiation is
delayed by one period.)
c) One Lapse Grim - Choose C in period 1 and after any history in which the other player chose D in at most one
period; choose D after any other history. (punishment is grim, but a single lapse is forgiven.)
d) Win-Stay, Lose Shift - This strategy starts by playing C, and repeats their action from the previous period if the
other player played C in the last period, and switches actions if the other person played D in the last period.
Answer
Any
Start
C
Any
C
Any
C
Any
C
D
Any
Any
D
Start
C
C
D
Any
C
D
Start
D
C
C
D
C
C
Any
D
D
Start
C
D
C
C
2. Each player selects a strategy that can be represented with a finite automaton. In period 1 of the repeated game, player
1 plays the intended action with probability 1 − ε and makes a mistake (plays the incorrect action) with probability
ε. Determine the normalized discounted payoffs for each player (in terms of ε and δ) for each of the following pairs of
strategies (where the first strategy is player 1’s strategy),
a)
b)
c)
d)
e)
Tit-for-Tat vs. Tit-for-Tat
All D vs. Tit-for-Tat
Tit-for-Tat vs. All D
Win-Stay, Lose Shift vs. Win-Stay, Lose Shift
Win-Stay, Lose Shift vs. All D
Page 1/3
March 23rd , 2015
ECON 451 • Spring 2015
Answer
The table below lists the outcome when there is no error, and the corresponding payoffs. It also lists the outcome when
there is an error and the corresponding payoffs.
TFT v. TFT
AllD v. TFT
TFT v. AllD
WSLS v. WSLS
WSLS v. AllD
No Error Outcome
CC, CC, CC, CC, . . .
DC, DD, DD, DD . . .
CD, DD, DD, DD, . . .
CC, CC, CC, CC, . . .
CD, DD, CD, DD, . . .
Player 1 No Error Payoff
3
4 (1 − δ)
−1 (1 − δ)
3
−1
1+δ
Player 2 No Error Payoff
3
−1 (1 − δ)
4 (1 − δ)
3
4
1+δ
Error Outcome
DC, CD, DC, CD, . . .
CC, DC, DD, DD . . .
DD, DD, DD, DD, . . .
DC, CD, DD, CC, CC, CC, . . .
DD, DD, CD, DD, . . .
Player 1 Error Payoff
4−δ
1+δ
(3 + 4δ) (1 − δ)
0
(1 − δ) (4 − δ) + 3δ 3
−δ 2
1+δ
Player 2 Error Payoff
4δ−1
1+δ
(3 − δ) (1 − δ)
0
(1 − δ) (4δ − 1) + 3δ 3
4δ 2
1+δ
The total payoff for each player will just be (1 − ε) mulitplied by the no error payoff and ε multiplied by the error payoff. For example, the
payoff to player 1 if they play TFT is,
4−δ
(1 − ε) 3 + ε
1+δ
3. Find conditions on the discount factor δ under which the following strategy pairs are a Nash equilibrium of the infinitely repeated Prisoner’s
dilemma game.
a) Both play 4 C then D
b) Both play Delayed Grim
c) Both play One-Lapse Grim
d) Both play Win-Stay, Lose Shift
e) Both play Tit-for-Tat
f) Both play Limited Punishment sp (k) (with k − 1 punishments)
Answer
• Both players play 4 C then D (4CtD). The outcome is CC, CC, CC, CC, DD, DD, . . ..
U1 (4CtD, 4CtD) = (1 − δ) 3 + 3δ + 3δ 2 + 3δ 3
The best alternative is All D. The outcome is, DC, DC, DC, DC, DD, DD, . . ..
U1 (AllD, 4CtD) = (1 − δ) 4 + 4δ + 4δ 2 + 4δ 3
It is clear to see that U1 (AllD, 4CtD) > U1 (4CtD, 4CtD) for all δ between 0 and 1, so both playing 4CtD is not a Nash equilibrium
for any δ ∈ (0, 1).
• Both players play Delayed Grim (DG). The outcome is CC, CC, CC, CC, . . ..
U1 (DG, DG) = 3
The best alternative is All D. The outcome is, DC, DC, DD, DD, DD, DD, . . ..
U1 (AllD, DG) = (1 − δ) [4 + 4δ]
It is clear to see that,
U1 (DG, DG) ≥ U1 (AllD, DG) ⇐⇒ 3 ≥ (1 − δ) (4 + 4δ) ⇐⇒ δ ≥
So both player DG is a Nash equilibrium as long as δ ≥
1
2
1
.
2
• Both players play One Lapse Grim (OLG). The outcome is CC, CC, CC, CC, . . ..
U1 (OLG, OLG) = 3
The best alternative is one D then all C (1DtC). The outcome for 1DtC is, DC, CC, CC, CC, CC, CC, . . ..
U1 (1DtC, OLG) = (1 − δ) 4 + 3δ + 3δ 2 + 3δ 3 + · · · = 4 − δ
It is clear to see that,
U1 (OLG, OLG) ≥ U1 (1DtC, OLG) ⇐⇒ 3 ≥ 4 − δ ⇐⇒ δ ≥ 1
Since this holds for no δ ∈ (0, 1) so both playing OLG is not a Nash equilibrium for any δ ∈ (0, 1).
Page 2/3
March 23rd , 2015
ECON 451 • Spring 2015
• Both players play Win Stay, Lose Shift (WSLS). The outcome is CC, CC, CC, CC, . . ..
U1 (W SLS, W SLS) = 3
The best alternative is allD . The outcome for allD is, DC, DD, DC, DD, DC, DD, . . ..
4−δ
U1 (AllD, W SLS) = (1 − δ) 4 − δ + 4δ 2 − δ 3 + 4δ 4 + · · · =
1+δ
It is clear to see that,
U1 (W SLS, W SLS) ≥ U1 (AllD, W SLS) ⇐⇒ 3 ≥
So both playing WSLS is a Nash equilibrium for any δ ≥
4−δ
1
⇐⇒ δ ≥
1+δ
4
1
.
4
• Both players play Tit-for-Tat (TFT). The outcome is CC, CC, CC, CC, . . ..
U1 (T F T, T F T ) = 3
The best alternatives are allD and alternate between C and D (ALT). The outcome for allD is, DC, DD, DD, DD, . . ..
U1 (AllD, T F T ) = 4 (1 − δ)
The outcome for ALT is, DC, CD, DC, CD, . . ..
4−δ
U1 (ALT, T F T ) = (1 − δ) 4 − δ + 4δ 2 − δ 3 + · · · =
1+δ
It is clear to see that,
U1 (T F T, T F T ) ≥ U1 (AllD, T F T ) ⇐⇒ 3 ≥ 4 (1 − δ) ⇐⇒ δ ≥
U1 (T F T, T F T ) ≥ U1 (ALT, T F T ) ⇐⇒ 3 ≥
So both playing TFT is a Nash equilibrium for any δ ≥
• Both players play
sp (k).
1
4
4−δ
1
⇐⇒ δ ≥
1+δ
4
1
.
4
The outcome is CC, CC, CC, CC, . . ..
U1 (sp (k), sp (k)) = 3
The best alternative is allD. The outcome for allD is, DC, DD, DD, DD, DD, DD, DC, DD, DD, DD, DD . . ..
h
i
4 (1 − δ)
U1 (AllD, sp (k)) = (1 − δ) 4 + 4δ k + 4δ 2k + · · · =
1 − δk
It is clear to see that,
4 (1 − δ)
1 − δk
This is as far as we can simplify this. So both playing sp (k) is a NE if the above is satisfied.
U1 (sp (k), sp (k)) ≥ U1 (AllD, sp (k)) ⇐⇒ 3 ≥
Page 3/3