Outline Review & Motivation Single Decision Problem Sequential Decision Problem Examining the Effect of Uncertain Preferences upon Value of Sample Information Brett Houlding, Frank Coolen Department of Mathematical Sciences, Durham University, UK Brett Houlding, Frank Coolen Examining the Effect of Uncertain Preferences upon Value of Sample Information Outline Review & Motivation Single Decision Problem Outline: Review & Motivation Expected Utility Theory Value of Information Adaptive Utility Single Decision Problem Effect of Uncertain Preferences Example Information over Preferences Sequential Decision Problem Understanding the Value of a Decision Example Brett Houlding, Frank Coolen Examining the Effect of Uncertain Preferences upon Value of Sample Information Sequential Decision Problem Outline Review & Motivation Single Decision Problem Sequential Decision Problem Expected Utility Theory Expected Utility Theory I Let Ω be the set of possible states of nature ω. I Let D be a finite set of decisions mapping Ω to a reward set R. I Assume true state of nature ω uncertain and denote beliefs by Pω . I Let u be a cardinal utility function mapping the set R to R. I Under axioms of expected utility theory, a Decision Maker’s (DM’s) objective is to maximise expected utility return. I The DM should thus select decision d maximising Eω [u(d, ω)]. Brett Houlding, Frank Coolen Examining the Effect of Uncertain Preferences upon Value of Sample Information Outline Review & Motivation Single Decision Problem Sequential Decision Problem Value of Information Value of Information I Let X be unknown information about ω. I Without X DM picks d to maximise Eω [u(d, ω)]. I With X DM updates beliefs to Pω|X and picks d to maximise Eω|X [u(d, ω)]. I Not knowing X means Eω|X [u(d, ω)] is a random variable. I Let PX represent beliefs over what X will say. I Before X is known, DM expects that knowing X will permit a max expected utility return of EX [maxd Eω|X [u(d, ω)]]. Brett Houlding, Frank Coolen Examining the Effect of Uncertain Preferences upon Value of Sample Information Outline Review & Motivation Single Decision Problem Sequential Decision Problem Value of Information Continued... I Denote the value of X to the DM by Iω (X ). I Iω (X ) represents the fair price (in utility currency) to the DM for knowledge of X . I Iω (X ) is the max expected return knowing X minus the max expected return not knowing X . I Iω (X ) is an unknown random variable, its expectation, EX [Iω (X )], is the DM’s expected value of X . I EX [Iω (X )] = EX [maxd Eω|X [u(d, ω)]] − maxd Eω [u(d, ω)] Brett Houlding, Frank Coolen Examining the Effect of Uncertain Preferences upon Value of Sample Information Outline Review & Motivation Single Decision Problem Sequential Decision Problem Value of Information Example I A DM has decision d to either Buy (B) or Not Buy (NB) stock. I The state of nature ω is whether the stock will Rise (R) or Fall (F ), and prior beliefs are that Pω (R) = 0.5. I Utility function is such that u(B, R) = 1, u(B, F ) = −1 and u(NB, R) = u(NB, F ) = 0. I Max expected utility is thus 0. Brett Houlding, Frank Coolen Examining the Effect of Uncertain Preferences upon Value of Sample Information Outline Review & Motivation Single Decision Problem Sequential Decision Problem Value of Information Example Continued... I An expert offers to sell information X on how stock will perform. I X can be either Good (G ) or Poorly (P). I You believe the expert is right with chance 0.9, e.g., Pω|X (R|G ) = 0.9. I If X = G , max expected return is 0.8, and if X = P, then this is 0. I Calculate predictive to find PX (G ) = 0.5. I Thus EX [Iω (X )] = (0.5 × 0.8 + 0.5 × 0) − 0 = 0.4. Brett Houlding, Frank Coolen Examining the Effect of Uncertain Preferences upon Value of Sample Information Outline Review & Motivation Single Decision Problem Sequential Decision Problem Adaptive Utility Adaptive Utility I Assume DM uncertain of preference ranking over elements in R. I Let θ be the DM’s state of mind representing actual preferences. I A classical utility function is hence conditional on a value of θ, denoted u(r |θ). I An adaptive utility function is a function of both reward and θ, denoted u a (r , θ). I Once classical utility functions have been scaled to be commensurable, then u a (r , θ) = u(r |θ). I The DM’s objective is to maximse her adaptive utility function. Brett Houlding, Frank Coolen Examining the Effect of Uncertain Preferences upon Value of Sample Information Outline Review & Motivation Single Decision Problem Sequential Decision Problem Questions Questions If we start to permit uncertain preferences, then we may be interested in: I How to generalise EV X for uncertain utility? I How does EV of X informative of ω depend on beliefs over θ? I Can one consider EV of X informative of θ? I What happens when X only arises following decision selection in sequential problems? Brett Houlding, Frank Coolen Examining the Effect of Uncertain Preferences upon Value of Sample Information Outline Review & Motivation Single Decision Problem Sequential Decision Problem Effect of Uncertain Preferences Effect of Uncertain Preferences Assume beliefs over θ and ω independent. If θ uncertain, and if the objective is to max expected adaptive utility, then the value of X informative of ω depends on: I The set D of possible decisions. I The adaptive utility function u a . I Beliefs over ω as represented by Pω . I The likelihood PX |ω . I But also now on beliefs over θ as represented by Pθ . Brett Houlding, Frank Coolen Examining the Effect of Uncertain Preferences upon Value of Sample Information Outline Review & Motivation Single Decision Problem Sequential Decision Problem Effect of Uncertain Preferences Expected Value of Information I Now the DM is assumed to wish to max her expected adaptive utility function, Eω Eθ [u(d, ω, θ)]. I If X informative of ω is known then max Eω|X Eθ [u(d, ω, θ)]. I As before, the expected value of X is the expected max expected utility obtainable with X , minus the max expected utility without X . I To demonstrate use of adaptive utility, replace Iy (X ) notation by Iya (X ), where subscript y is parameter that X is informative of. I EX [Iωa (X )] = EX [maxd Eω|X Eθ [u(d, ω, θ)]] − maxd Eω Eθ [u(d, ω, θ)] Brett Houlding, Frank Coolen Examining the Effect of Uncertain Preferences upon Value of Sample Information Outline Review & Motivation Single Decision Problem Sequential Decision Problem Effect of Uncertain Preferences Properties Similar to case of classical value of information, EX [Iωa (X )] satisfies: I Non-Negativity: EX [Iωa (X )] ≥ 0 I Additivity: EX1 ,X2 [Iωa (X1 , X2 )] = EX1 ,X2 [Iωa (X2 |X1 )] + EX1 [Iωa (X1 )] Brett Houlding, Frank Coolen Examining the Effect of Uncertain Preferences upon Value of Sample Information Outline Review & Motivation Single Decision Problem Sequential Decision Problem Effect of Uncertain Preferences Relationship between Uncertainty and Value Less uncertainty over θ means the DM is more sure over preferences, so one may guess that more uncertainty decreases EX [Iωa (X )]. However, there appears to be no simple relationship between uncertainty over θ and EX [Iωa (X )]. In particular: I EX [Iωa (X )] is not necessarily increased as less values of θ are possible. I EX [Iωa (X )] is not necessarily increased as less probability is concentrated on a particular value of θ. I EX [Iωa (X )] is not necessarily increased as Vθ [θ] decreases. I EX [Iωa (X )] is not necessarily minimised in the case of equiprobable values of θ. Brett Houlding, Frank Coolen Examining the Effect of Uncertain Preferences upon Value of Sample Information Outline Review & Motivation Single Decision Problem Sequential Decision Problem Example Example The previous statements can be shown to be a result of the following counter example: I θ ∈ {θ1 , θ2 }, Pθ (θ1 ) = p I ω ∈ {ω1 , ω2 }, Pω (ω1 ) = 0.5 I X ∈ {x1 , x2 }, PX |ω (xj |ω1 ) = δij I d ∈ {d1 , d2 }, d1 (ω1 ) = d2 (ω2 ) = r1 , d1 (ω2 ) = d2 (ω1 ) = r2 I u a (r1 , θ1 ) = u a (r2 , θ1 ) = 1, u a (r1 , θ2 ) = 2, u a (r2 , θ2 ) = 0 Brett Houlding, Frank Coolen Examining the Effect of Uncertain Preferences upon Value of Sample Information Outline Review & Motivation Single Decision Problem Sequential Decision Problem Example Continued... We find that: I maxd Eω Eθ [u a (d, ω, θ)] = 1, ∀p ∈ [0, 1] I EX [maxd Eω|X Eθ [u a (d, ω, θ)]] = 2 − p I EX [Iωa (X )] = 1 − p I Setting θ1 = v1 and θ2 = v2 gives Vθ [θ] = p(1 − p)(v1 − v2 )2 . For a change in Pθ to increase EX [Iωa (X )] it must increase EX [maxd Eω|X Eθ [u a (d, ω, θ)]] more than maxd Eω Eθ [u a (d, ω, θ)]. Brett Houlding, Frank Coolen Examining the Effect of Uncertain Preferences upon Value of Sample Information Outline Review & Motivation Single Decision Problem Sequential Decision Problem Information over Preferences Dependencies & Information over θ I No problem in dealing with dependency of beliefs over θ and ω, and similar results and properties hold. I Information over θ could arise from interview with experts. I It could also arise through trial of rewards, e.g., test drive or sample tasters, or from observation of Advertisements. I Usually, however, we envisage X informative of θ will be observed following decision selection in a sequential problem. Brett Houlding, Frank Coolen Examining the Effect of Uncertain Preferences upon Value of Sample Information Outline Review & Motivation Single Decision Problem Sequential Decision Problem Understanding the Value of a Decision Understanding the Value of a Decision I Now consider period i that X is observed: Iya (X ; i). I Iya (X ; i) can not increase if i is increased. I If d i selected in period i influences X , then must consider EX [Iωa (X ; i)] in connection with d i . I Now decompose total expected value of decision d i into expected ‘pure’ value (resulting from the reward it entails), and expected ‘information’ value (resulting from value of X ). I This is why a DM in a sequential problems may select decisions not optimal under prior beliefs, and that such experimentation should occur at the earliest opportunity. Brett Houlding, Frank Coolen Examining the Effect of Uncertain Preferences upon Value of Sample Information Outline Review & Motivation Single Decision Problem Sequential Decision Problem Example Example I Must choose an Apple (A) or Banana (B) in each of 3 periods. I I Assume ω redundant (a decision to choose A will result in A). P3 P3 u a (d 1 , d 2 , d 3 , θ) = j=1 I{d i =A} + θ j=1 I{d i =B} I Thus choosing A increases utility by 1, B increases this by θ. I Assume θ ∈ {0.5, 1.5} and Pθ (0.5) = 0.6. I Let X = 1 if DM chooses A, whilst X = θ if DM chooses B. I Consider value of X observed after d 1 . Brett Houlding, Frank Coolen Examining the Effect of Uncertain Preferences upon Value of Sample Information Outline Review & Motivation Single Decision Problem Sequential Decision Problem Example Continued... I EX [Iθa (X ; 1)] = 0 for d 1 = A. No information can be gained! I EX [Iθa (X ; 1)] = 0.3 for d 1 = B. We will learn θ for sure! I Thus, even though selecting Apple gives sure reward of value 1, whilst prior expectation of value resulting from a Banana is Eθ [θ] = 0.9, d 1 = B has extra value of 0.3 because of its expected information. I Hence why a DM does best in expectation by first selecting d 1 = B. Brett Houlding, Frank Coolen Examining the Effect of Uncertain Preferences upon Value of Sample Information
© Copyright 2024