1. law, govt and politics
2. law enforcement
3. fire department

Risk Perceptions in the Virtual Wilderness
by
Glenn W. Harrison, Jared M. Johnson & E. Elisabet Rutström †
April 2015
ABSTRACT
In economic decision making most probabilities are formed in a compound manner through
the interaction of multiple attributes of events, each of which have likelihoods that are unknown to
various degrees. We consider how subjectively formed risk perceptions are affected by the
dispersion of the underlying objective, compound probability distribution. Our methodology relies
on virtual reality simulations of physical cues of the risk, allowing us to bring together the natural
stimuli of the field and the control of the lab. Our application is an important example of a risk with
serious economic consequences: the management of wild fire risk. This is an important natural
setting where the risk is compound, depending on many random physical processes and where the
formation of risk perceptions necessary for risk management is therefore complex. We find that
increasing the dispersion of the underlying objective risk leads to higher subjective probabilities of
the worst outcome occurring, consistent with increased pessimism. We compare the risk perceptions
of experts in this domain with non-expert residents that are affected by the risk, and conclude that
experts are not always better than non-experts at estimating the risks. Experts appear to be locked in
by their strong priors based on stimuli outside those presented in our naturalistic virtual reality.
With a global environment that produces increasingly extreme phenomena, training experts to be
less anchored on their prior experiences will become important.
†
Department of Risk Management & Insurance and Center for the Economic Analysis of Risk,
Robinson College of Business, Georgia State University, USA (Harrison); Google, USA (Johnson);
and Dean’s Behavioral Economics Lab, Robinson College of Business and Department of
Economics, Young School of Policy Studies, Georgia State University, USA (Rutström). E-mail
contacts: [email protected], [email protected], and [email protected]. Harrison and
Rutström are also affiliated with the School of Economics, University of Cape Town; Harrison is
also affiliated with IZA – the Institute of Labor; and Rutström is also affiliated with the Institute for
Simulation and Training, University of Central Florida. We thank the U.S. National Science
Foundation for research support under grants NSF/HSD 0527675 and NSF/SES 0616746. The
rendering software employed here was developed together with Remo Pillat and Charles Hughes at
the Institute for Simulation and Training, University of Central Florida.
Risk perceptions are rarely degenerate and precise, with the exception of events generated by
roulette wheels, dice or bingo cages. In economic decision making most probabilities are formed in a
compound manner through the interaction of multiple attributes of events, each of which have
likelihoods that are unknown to various degrees. In addition, underlying probabilities are not usually
known with any precision, but are based on a variety of “cues and clues.” We use wild fires as an
example of such complex uncertain events, but most natural settings of interest exhibit similar
complexity. For example, there can be no serious discussion of the risks of global warming without
recognition of the complexity and uncertainty of evaluating the precise probabilities involved. Even
the familiar problems of financing retirement poses complex questions with uncertain answers about
“longevity risk” for the individual, society, and re-insurance company.1
In the case of wild fires, the likelihood of a wild fire causing damage depends on the
probability of a fire getting started, such as the probability of a lightning strike, but also on the
weather and vegetation conditions. During hot and windy conditions a fire can spread very rapidly,
and if in addition there is a drought the effects can be devastating. When planning for a wild fire
season, and deciding what fire prevention actions to take, one must form some risk perception that
takes into account all of these aspects of the “data generating process” that causes wild fire damage.
Objectively, if all underlying probabilities were known, one could in principle form a
compound probability distribution, but even such objective distributions might be expected to have
some dispersion. We consider how subjectively formed risk perceptions in such cues and clues
environments are affected by both the mean and the dispersion of the underlying objective,
compound probability distribution. This modest step is important because most studies of
1
In many settings, one does not need precise probabilities to make better decisions, or even optimal
ones given the feasible choices available. Desvousges, Johnson and Banzhaf [1999] provide rich illustrations
from the field of environmental policy, in the specific domain of “benefit transfer” calculations.
-1-
perception and Bayesian updating have occurred in the contrived, artefactual setting of the
laboratory, or the uncontrolled setting of the field. The former settings involve transparent instances
of compounding, and the latter settings require powerful identifying assumptions for subjective
beliefs (e.g., rational expectations). One important aspect of our approach is to inform the
participant decision maker about the risk using simulated cues and clues in a way that mimics the
cues and clues in natural field settings. In order to do so we build a virtual reality 3D simulation of
an area prone to wild fires that is familiar to the participants, rendering the simulation as close to
nature as the technology allows.
A second motivation for our approach is an interest in the relationship between prior
experiences and perception formation. If a decision maker has experiences with a wide set of
circumstances, such as the case with experts, then the perception of the risk based on recently
observed circumstances should be relatively precise. Thus, we would expect experts to have more
accurate perceptions than non-experts. In the case of forest fires, the forest rangers who work at
suppressing fires at the fire line would be expected to have more accurate and precise perceptions
than residents who live in fire prone areas.
On the other hand, experience can sometimes be a hindrance to forming accurate
perceptions of situations that are out of the ordinary. This idea is consistent with the concept of
learned attention, defined by Kruschke and Hullinger [2010; p. 26] as “individual cue activations
[that] are amplified or attenuated depending on which other cues are present.” These learning
patterns include highlighting effects, reviewed by Kruschke [2009], where irrelevant cues can be
psychologically treated as predictive in new situations simply because they were present during
earlier experiences. In contrast to stylized risky decision settings, which are the most common ones
studied in controlled lab settings, we expect that natural risky decision settings include combinations
of both relevant and irrelevant cues. This can result in perception and decision errors that are
-2-
qualitatively and quantitatively different from those typically found in the lab.
We present data from an experiment designed to provide respondents with cues about the
likelihood of events that are naturalistic. We use the term “naturalistic” to distinguish these cues
from many artificial cues in experimental lab settings that are entirely context-neutral and artificial,
and from natural cues that arise in actual field settings. The cues we provide participants are
generated in a virtual reality (VR) fire simulation which is immersive and dynamic, and where the
participant has control over many aspects of the process that generates the cues. The validity of VR
as an environment that mimics natural contexts has been demonstrated in Fiore, Harrison, Hughes
and Rutström [2009]. They show that naturalistic cues of this kind can generate risk perceptions that
are closer to objective risks than contextual cues that are non-interactive and non-immersive.
We approach the issue of the natural dispersion of subjective risk by experimentally
manipulating the dispersion of the objective risk that decision-makers face. Our VR setting allows us
to undertake that manipulation in a naturalistic manner, without “clubbing” our subjects over the
head with it. We therefore expect heterogeneity in the manner in which individuals process these
manipulations, and a key role for experience with that process.
In order to study the impact of experience we sample both wild fire experts (forest rangers)
and non-experts (regular residents), and confront them with two types of risk settings. One risk
setting is where the damage probability is relatively low, and is expected to be more in line with the
majority of ranger experiences. The other risk setting is where the damage probability is very high,
and could therefore be less in line with past experience. In both cases the objective damage
probabilities are generated by a combination of historic climate and weather factors, and these drive
a computer simulation model of wild fires used by wild fire experts during live fire events. Our risk
generating mechanism is therefore as close to natural as possible, given the complex nature of wild
fire risks and the need to have something we can control for experimental purposes.
-3-
We assume Subjective Expected Utility (SEU) throughout, although our findings generalize
to many other approaches such as Rank Dependent Utility, Choquet Expected Utility and Prospect
Theory. SEU does not assume that decision makers know the true, objective process generating
stochastic outcomes. We find that estimated subjective probabilities deviate in varying degree from
those “actuarially” implied by the underlying compound objective probabilities, but we are agnostic
regarding the possible cognitive and psychological cause of such variations.
We find that the accuracy of the estimated subjective probabilities differs across three
treatments that vary two characteristics of the objective probability distribution: the mean and the
dispersion. We also find differences in the estimated subjective probabilities between experts and
non-experts, and these differences depend on whether the context is one where the likelihood of
damage is high, or one where it is low. Our findings are consistent with experts having homegrown
expectations (i.e., expectations based on their field experiences) regarding the risk that are more in
line with our low risk scenario, and that these homegrown expectations influence their choices when
we are inducing an extreme, unfamiliar, high risk context. In effect, from a Bayesian perspective the
rangers’ experiences have left them with relatively degenerate, tight priors compared to non-experts.
In section 1 we describe the field experiments. In section 2 we review the structural model
used to elicit subjective probabilities and risk attitudes. In section 3 we present estimates from our
experimental data and econometric model, and in section 4 draw some conclusions and discuss
extensions to consider uncertainty and ambiguity in the wild.
1. Field Experiments
A novel feature of our experiments is the use of virtual reality to provide individuals with a
rich set of choice stimuli while still maintaining the internal validity needed for experimental control.
We do not use abstract referents, such as one might appropriately use in many laboratory
-4-
experiments. Fiore, Harrison, Hughes and Rutström [2009] propose a new experimental
environment, the Virtual Experiment (VX), that has the potential of generating both the internal
validity of lab experiments and the external validity of field experiments. A VX is an experiment set
in a controlled lab-like environment, using either typical lab or field participants, that generates
synthetic field cues using Virtual Reality (VR) technology. The naturalism of the VR simulations can
vary across experiments, depending on how closely the simulated environment mimics the target
field environment.2 The experiment can be taken to typical field samples, such as experts in some
decision domain, or to typical lab samples, such as student participants. The VX environment can
generate internal validity since it is able to closely mimic explicit and implicit assumptions of
theoretical models, and thus provide tight tests of theory; it is also able to replicate conditions in past
experiments for robustness tests of auxiliary assumptions or empirically generated hypotheses. The
VX environment can generate external validity because observations can be made in an environment
with cues mimicking those occurring in the field. In addition, any dynamic scenarios can be
presented in a realistic and physically consistent manner, making the interaction seem natural for the
participant. Thus the VX builds a bridge between the lab and the field, allowing the researcher to
smoothly go from one to the other and see what features of each change behavior.
A. Experimental Setting and Subjects
The focus of our experiments is an area around Lake Diaz in Volusia County, Central
Florida. This area was selected because it has the type of vegetation that is prone to wild fire risk,
and there are also some settlements mixed in with the forests. It is representative of many other
2
For example, Dixit, Harrison and Rutström [2014] consider the probability of having a virtual
accident in a driving simulator with VR technology. While the participants sits in a cab with controls from an
actual car, the simulated environment is not based on an existing, natural environment.
-5-
areas in Central Florida that experience wild fires during active fire seasons. Florida has about 5,000
wild fires each year during the active fire season, which is between March and June. The most severe
wild fire in recent times in Florida was in 1998 in the Central Florida region, with a total damage
estimated at $550 million by Butry, Mercer, Prestemon, Pye and Holmes [2001]. We were interested
in understanding the risk mitigating decisions of residents and property owners who live in these
areas, and of forest rangers who work on the fire lines and conduct prescribed burns in the Central
Florida region.
Experts were recruited from the Central Florida Region of the Division of Forestry of the
Florida Department of Agriculture and Consumer Services. Letters of invitation were distributed to
all field offices, and interested forest rangers contacted us to schedule a meeting time. Meetings took
place in the field offices after official working hours. We had 49 experts participate in the sessions:
31 of these were rangers, 14 were senior rangers, and 4 were forest area supervisors. Our expert
subjects averaged 11 years on the job, and had typically seen 5 to 10 fires that burned more than 500
acres during their career, and 11 to 20 fires that burned between 10 and 500 acres. The highest Red
Card certification levels (5-6) was held by 14 rangers, 8 rangers had levels 3-4, and 10 rangers had
levels 1-2. Red Card Certification levels define which positions you may work in during a wild fire.
Residents were recruited from Volusia, Lake and Seminole counties of Central Florida, by
mail using addresses from the county property records. Addresses that were within urban
boundaries, defined as 1,000 or more residents per square mile, were excluded. Just over 9,000
letters were sent out to randomly selected addresses, resulting in 254 participants for a 3% response
rate.3
Table 1 summarizes the composition of our subject pool by expert and resident. Rows 1-8
3
An additional 51 persons responded and participated, raising the response rate to 3.4%. We do not
include data from these participants because there were inconsistencies in what instructions they were given.
-6-
summarize individual characteristics, rows 9-11 personal experience with property loss due to wild
fires, rows 12-16 how they experienced wild fires in the simulator. There were very few women in
the expert sample, and the average expert is younger than the average resident by 13 years. The
income distribution among experts is shifted toward the lower income side compared to our resident
participants.
B. Constructing the Virtual Experiment
The VX environment is built on a computer simulation model of wild fires used by wild fire
experts during live fire events. Finney [1998] documents this model, known as FARSITE. We run the
simulation model on GIS data from the Lake Diaz area, provided by the Division of Forestry,
coupled with weather and climate data from 1999, a drought year, and 2003, a non-drought year.
Official weather data were collected directly from the Florida Automated Weather Network (see
http://fawn.ifas.ufl.edu/data/). We use fuel models and fuel moisture data from the 2002 Fire Risk
Assessment System documented by the Division of Forestry [2002] and Anderson [1982]. These are all
important determinants of the speed and spread of the fire, and are used to generate our low risk
and high risk treatments. Our high risk treatment uses fuel models that are consistent with no
prescribed burn having taken place in the last seven years, and the low risk treatment uses fuel
models consistent with a prescribed burn being applied one year earlier. Figure 1 shows a map view
of the output of the FARSITE model, based on a simulation of the Lake Diaz area. Each cell in the
map has a particular fuel model and moisture value. The topological lines show the spread of the fire
over time, based on the time of arrival of the fire for each cell. These time of arrival values provide
input to our VR rendering.
The VR model was constructed to render the fire spread, following the time of arrival data
from FARSITE. The VR provides a 3D visualization, with trees rendered using SPEEDTREE, a
-7-
commercial software package popularly used in many games and animated movies (see
http://www.speedtree.com). The trees in the existing SPEEDTREE library were slightly adjusted to
better visualize the local vegetation in Central Florida. The vegetation included are Pines, low growth
Palmettos, and Oaks. Grasses are not modeled as 3D objects, but are rendered as textures. Flames
and smoke show the fire line, and vegetation is blackened after the fire moves through an area. A
significant amount of design work went into the rendering of flames and the smoke: the former
because of delicacy with respect to the transparency and colors of the flames, and the latter because
of demands on rendering speed.
The Lake Diaz area is 407 acres. To limit the density of the trees, so as to avoid problems
with rendering speed, we rendered each tree slightly larger than it would otherwise be. Thus, when
experienced in the VR simulation, the area seems smaller than 407 acres. The person interacting with
the VR simulation can experience it either at ground level, or in helicopter mode flying over it and
looking down. The user has full control of the altitude, as well as the direction and speed of motion.
Apart from the ability to move in the 3D space, the software also gives the user full control over
time. It is possible to jump forward or backward in time to investigate the progression of the fire
line.
A house is placed in the Lake Diaz simulation, as shown in Figure 2, which also shows the
six map locations where a fire may start. Figure 3 shows two images taken from the VR simulation.
In the first image we can see the house as a fire is approaching. In the second image we can see both
large flames and the dense smoke that is being rendered.
We generated a total of 960 simulations in FARSITE, varying the underlying background
variables that impact the spread of the fire and the likelihood that the house will burn. Table 2 gives
an overview of these background variables. We distinguish first between 1999, a drought year with
extremely low or very low fuel moisture conditions, implying extreme or high fire spread conditions,
-8-
and 2003, a non-drought year with medium or high fuel moisture conditions, implying moderate or
low fire spread conditions. We vary the duration of a fire from one day to two days, simulating the
differential availability of fire suppression resources. For each of these two years we randomly
selected 10 dates to model weather conditions from the active fire seasons, shown on the last row of
Table 2 using month and day. Weather conditions are historic and will therefore also reflect the
drought or non-drought conditions of the two years. The six lightning strike locations are also
shown in the map in Figure 2. All of these conditions are duplicated for our two fuel load
conditions: a low fuel load that models the forest one year after a prescribed burn, when the fire
spread is smaller and the risk of house damage smaller, and a high fuel load that models the forest
seven years after a prescribed burn, when the fire spread is larger and the likelihood of the house
being damage is higher. Of course, many of these background variables interact so the marginal
effects on the likelihood of the house burning in a random fire depend on these interactions.
Pooling over all these factors the likelihood of the house burning in the low fuel load case is 0.27,
with a 95% confidence interval between 0.23 and 0.31 assuming a binomial distribution. For the
high fuel load case the likelihood of the house burning is 0.65, and the 95% confidence interval is
between 0.61 and 0.69.
C. Experimental Tasks
Each subject encounters two scenarios of forest fire risk. One is a “high” objective risk using
the high fuel load and the other is a “low” objective risk using the load fuel load, in senses to be
made clear. The high risk scenario has a higher probability that the house will burn. Each subject
completes four tasks in the experiment, after experiencing the virtual environment and forming their
subjective beliefs about the risk of house damage. This experience is explained in §E below. Our
design has an objective distribution of risks that the house will burn down, known to the
-9-
researchers, but subjects only see cues and clues to that distribution, as explained below.
Table 3 outlines the experimental design. We have two tasks that involve both subjective
probabilities and risk attitudes, and two tasks that only involve risk attitudes. In effect, the last two
tasks “recursively” identify risk attitudes, which are used to infer subjective probabilities from the
first two tasks. The first two tasks elicit subjective probabilities conditional on the high or low risk
scenario. Subjects were paid for one of their choices in the first two tasks, and one of their choices
in the last two tasks, each selected at random.4
The first task is to bet on whether the house will burn or not in the low risk scenario.
Payoffs depend on the bets placed, and whether or not the house burns in one simulation randomly
selected from the set of simulations making up the low risk scenario. Maximum earnings in this task
are $150, and a Linear Scoring Rule (LSR) is used to elicit bets.5 The choice options are in $5
intervals, using a multiple price list representation of the bets shown in Table 4. This format allows
the subject to see all bet choices, and to select one row. Inspection of Table 4 shows that the payoffs
change in an easily understood way. If we “know” the risk attitudes of the decision maker we can
infer their subjective probability, with a narrow interval, from their response to this task. We discuss
the effect of risk attitudes and subjective beliefs on this choice, and our inferences about subjective
probability, later.
The subject is given this task with instructions that connect it to the beliefs they formed in
4
We assume that the experimental payment protocol led subjects to treat each choice as if it was
taken in isolation. This assumption is critically evaluated by Cox, Sadiraj and Schmidt [2015] and Harrison and
Swarthout [2014].
5
We used a LSR for this task since all payoffs were to be shown to the subject for a finite set of
possible reports, and we did not want the non-linearity of a Quadratic Scoring Rule (QSR) to generate
complicated payoff values. The LSR has one unfortunate feature in comparison with the QSR if the subject is
exactly risk-neutral: with the LSR they should report at the very top or bottom of the table, depending on
whether their subjective probability of the house burning is less than ½ or greater than ½, respectively. Our
subjects are not risk neutral on average, and laboratory comparisons of LSR with QSR by Andersen,
Fountain, Harrison and Rutström [2014] show that it generates non-degenerate behavior.
-10-
the initial phase of the experiment by viewing certain simulations:
“Your first task will be to make a decision about the house burning when the area
has been treated with a prescribed burn in the last year or two so there is not so
much fuel for the fire. What is different from your experiences with the earlier
simulations is that now you will not know for sure what the background variables
will be. The lightning may strike in this area in a drought year or in a normal year.
There may be plenty of fire fighting resources available or not. Etc.”
The subject is told that the background variables will be selected by die rolls, performed by the
subject, after the bet has been placed. This makes the bet a task based on a compound lottery, where
the first stage of the compound lottery is the selection of specific values for background variables,
with the probabilities of combinations of values is known, and where the second stage is the
simulation of the fire based on those variables. The subject does not know if the house will burn or
not for the various combinations of background variables. Subjects are not told how each factor
affects the spread of the fire, and therefore the risk of damage to their property. They form their
own beliefs based on the experience gained in the simulator, which is why we work hard to ensure
that the simulator is faithful to the underlying natural processes at work.
The second task was to bet on whether the house will burn in the high risk scenario.
Again, the payoff depends on the bet and whether the house burns in a simulation randomly
selected from the set of simulations making up the high risk scenario. The decision sheet is the same
as for the first task, shown in Table 4; maximum earnings are $150, and the choice options are
presented in a multiple price list in $5 intervals using an LSR.
In the third task the subject was presented with a series of binary lottery choices to elicit
risk attitudes over objective probabilities. This task is the multiple price list of Holt and Laury
[2002], with payoffs between $2 and $77, and is implemented with the decision sheet in Table 5. The
subject picks one lottery in each row, and one row is randomly selected for payment.
In the fourth task the subject was given the same choices as in the third task, except that the
-11-
final payment would be generated by the “double or nothing” device. That is, if the subject made a
selection from Table 5 and it generated a payoff of $x, we would flip a coin: with 50% chance the
payment would then be $2x, and with 50% chance it would be zero. This turns the choice of simple
lotteries in the third task into a choice of compound lotteries. In addition, it means that risk attitudes
have been elicited over the interval $0 to $154, which spans the possible payoffs in the first two
tasks. Hence, when we apply estimates of risk attitudes to infer the subjective probabilities in earlier
tasks, we are in effect “interpolating” rather than “extrapolating.”
Each subject separately completes a questionnaire with a basic set of demographic questions.
The questionnaire includes Raven’s Progressive Matrix test, a standard non-verbal intelligence and
pattern recognition test used in psychology.
D. Treatments
We construct three treatments by varying the mean and the dispersion of the objective risk
in the scenarios presented to subjects. We do this by changing which background variables that are
varied. In treatment A only three lightning strike areas are included: Central West, North, and South
West. In treatment B the fire duration is only one day in the low risk case, but it is one or two days
in the high risk case. Finally, in treatment C the low risk case has a fire duration of only one day, and
the high risk case has a fire duration of only two days. For both the high and low risk case, treatment
C only includes the drought year of 1999, and not the non-drought year of 2003. Figure 4 illustrates
the treatments by showing the posterior probability distributions for the average risk using the
Jeffreys [1946] diffuse prior for the true probability.6 Each subject is randomly assigned to one of
6
This prior is a beta(½,½) distribution, and the posterior distribution for k successes out of n trials
from a binomial data-generation process is then beta(k+½, n-k+½). The 95% confidence interval is the
0.025 and 0.975 quantiles of that posterior distribution, and the length of the 95% confidence interval is the
difference between these quantiles.
-12-
these three treatments.
In treatment A the low risk scenario is more certain than the high risk scenario. The mean
objective probability risk of the house burning increases from 5% to 57%, but the length of the 95%
confidence interval on that risk increases as well from 5.5% to 12.5%. In treatment B the low and
high risk scenarios have the same imprecision. The mean for the low risk scenario is 17%, and the
mean for the high risk is 65%, and each has a 95% confidence interval length of approximately 9%.
In treatment C the low risk scenario is less precise than the high risk scenario. The mean for the
low risk scenario is 23%, and the mean for the high risk scenario is 96%, but the 95% confidence
interval length drops from 16.3% for the low risk scenario to only 1.9% for the high risk scenario.
Each subject is randomly assigned to a treatment, and then undertakes both a low and a high
risk task for that treatment.
E. Experience and the Formation of Subjective Probabilities
Before being asked to do any of the tasks, the subjects were given experience in the fire
simulations and a chance to form their subjective beliefs about the likelihood that the house would
burn in the low risk and high risk case. There are two parts to this experience phase: an exogenous
set of two simulations that all subjects view, and a set of simulations that they compose themselves
by combining any of the background variables that they are curious about. The purpose of this
phase is to allow subjects to generate a personal experience that is as informative as possible. This
experience gives them the best shot at forming accurate perceptions.7
The first set consists of one simulation from the low risk case with the low fuel load, and
7
This is where the VR is superior to the field as an experience generator. The frequency of
experiencing burns is higher in the simulation, and the ability to observe several aspects of the spread pattern
is greater.
-13-
one simulation from the high risk case with the high fuel load. For the low risk case the background
variables are set to minimize the spread of the fire: low fuel load, one day duration, not a drought
year (2003), high fuel moisture so the spread is low, and a northern lightning strike location. For the
high risk case the background variables are set to maximize the spread of the fire: high fuel load, two
days duration, drought year (1999), low fuel moisture so the spread is extreme, and again a northern
lightning strike location. These two extreme cases are therefore common information to all subjects.
In the second set they could select to view simulations with any combination of background
variables and spend up to 15 minutes doing so. As shown in Table 1, both experts and non-experts
viewed on average of 8 simulations in this period. The smallest number viewed was 3 and the
highest number viewed was 9.
F. Procedures
Figure 5 provides an overview of the procedures. All subjects completed a demographic
questionnaire and the Raven’s Progressive Matrix test. Some did this online before arriving to the
session, some did this at the beginning of the session, and some did it at the end of the session. This
variation was implemented so that we could optimize the use of the VR simulators, the most time
consuming part of the session. All other tasks were done in the same order for all subjects. A
research assistant helped each subject on a one-on-one basis with all tasks. These assistants were
carefully trained to be helpful without suggesting what choices to make.
The VR simulator tasks were always completed before the lottery tasks. Subjects were paid
for one randomly selected simulator task and one of the two lottery tasks, also randomly selected.8 A
general introduction to wild fires, to the VR simulator, and to their tasks were given before they sat
8
There were four simulator tasks in the experiment. The random chance of payment for each of the
belief tasks was therefore 1 in 4, not 1 in 2. We do not analyze the last two simulator tasks here.
-14-
down in the simulator. The simulator consisted of a computer with a large flat screen monitor, a
keyboard, mouse and flight stick. Before getting the simulated fire experience subjects went through
a structured training task, intended to familiarize them with the controls of the simulator and with
the layout of the forest. Subjects used a flight stick, commonly used for flight simulator games, to
move around in the 3D environment, and could easily change the speed of movement. Movement
options included forward, backward and sideward movement, straight line or turning, up or down
angle, straight up or straight down. Speeds could be selected to suit the ability of each subject: those
who had a harder time controlling their movements could choose a slower speed and those who had
an easier time could choose a faster speed. By using the mouse subjects could also jump forward or
backward in time, to either investigate where the fire spread was heading or to review how it had
progressed. Each subject was walked through a series of exercises to become familiar with these
controls.
After this general practice each subject was given the experience of the two common,
extreme conditions, after which they completed 15 minutes of self-selected simulations. Subjects
were carefully monitored for nausea, and could take a break if needed. After completing their
simulated fire experiences they were instructed in the simulator tasks and completed them. Before
moving on to the lotteries the simulator earnings were actualized. The final paid task was the lottery
task. All subjects were first instructed on the task, then they did the two lottery tasks in the same
order. The session ended with payments in cash.
2. Theory
The choices that our subjects made can be modeled using SEU and a structural econometric
model estimated from that theory. We start with the risk aversion tasks by themselves, and then
extend the analysis to the tasks involving subjective probabilities as well.
-15-
A. Risk Attitudes
Although our approach is not restricted to a particular utility function or decision theory, we
adopt assumptions that are transparent and easy to analyze.9 We assume that utility of income is
defined by
U(y) = M(1!r)/(1!r)
(1)
where M is the lottery prize and r…1 is a parameter to be estimated. Thus r is the coefficient of
CRRA: r=0 corresponds to risk neutrality, r<0 to risk loving, and r>0 to risk aversion. There are
two possible outcomes in each lottery. Under EUT the probabilities for each outcome Mj, p(Mj), are
those that are induced by the experimenter, so expected utility is simply the probability weighted
utility of each outcome in each lottery i 0 {A, B}:
EUi = [ p(M1) × U(M1) ] + [ p(M2) × U(M2) ]
(2)
The EU for each lottery pair is calculated for a candidate estimate of r, and the index
LEU = [(EUB ! EUA)/ν]/μ
(3)
calculated, where EUA is lottery A and EUB is lottery B as presented to subjects in Table 5, ν is a
normalizing term to ensure that the difference in EU for any lottery pair is bounded between 0 and
1, and μ is a structural “noise parameter” used to allow some errors from the perspective of the
deterministic EUT model ( Hey and Orme [1994]).10 This latent index, based on latent preferences,
is then linked to observed choices using the cumulative logistic distribution function Λ(LEU). This
“logit” function takes any argument between ±4 and transforms it into a number between 0 and 1.
Thus we have the logit link function,
9
There are many options for analyzing these data. Each row in the lottery task defines an interval for
the relative risk aversion coefficient, r, and one could evaluate this using panel interval regression techniques
(Harrison, Lau and Rutström [2007]). Variations on the ML approach used here could use Expo-power utility
functions, probit instead of logit link functions, or ratio instead of difference index functions; these variations
are reviewed in Harrison and Rutström [2008]. The approach can also be extended to accommodate other
decision theories such as Rank Dependent Utility or Prospect Theory.
10
See Wilcox [2008][2001] for a review of the implications of alternative stochastic specifications.
-16-
prob(choose lottery B) = Λ(LEU)
(4)
The index defined by (3) is linked to the observed choices by specifying that lottery B is chosen
when Λ(LEU)>½, which is implied by (4).
The likelihood of the observed responses, conditional on the EUT and CRRA specifications
being true, depends on the estimates of r given the above statistical specification and the observed
choices. The conditional log-likelihood for the observed risk choices in tasks 3 and 4 is then
ln LRISK (r, μ; y, X) = 3i [ (ln Λ(LEU)×I(yi = 1)) + (ln (1-Λ(LEU))×I(yi = !1)) ]
(5)
where I(@) is the indicator function, yi =1(!1) denotes the choice of the Option B (A) lottery in risk
aversion task i, and X is a vector of observable characteristics of the individual or the task. The
parameter r is defined as a linear function of the characteristics in vector X.11
B. Subjective Probabilities
Let π denote the subjective probability that the house will burn in the low risk scenario in a
particular choice setting, and let Π denote the subjective probability that the house will burn in the
high risk scenario. Hence we expect to infer Π $ π. As the discussion of the experimental design
made clear, this setting varied considerably in terms of the naturalistic stimuli presented to subjects
in the VR environment, and differences in π and Π across these settings will be a natural focus of
analysis. For all binary choices involving the subjective probability, the earlier specification of
expected utility with objective probabilities,
EUi = [ p(M1) × U(M1) ] + [ p(M2) × U(M2) ],
(2)
is changed slightly. For task 1, with bets being made over the chance of the house burning conditional
11
Harrison and Rutström [2008a; Appendix F] review procedures and syntax from the popular
statistical package Stata that can be used to estimate structural models of this kind, as well as more complex
models. The effect of the specification used here is to change the slope of the logit link function. Larger
values of μ flatten the function.
-17-
on the low risk scenario, the subjective expected utility of each row i is
SEUi = [ π × U(Mburn) ] + [ (1-π) × U(Mnot burn) ],
(6)
where Mburn is the payoff if the house burns and Mnot burn is the payoff if it does not. Table 4 shows
the various values for the payoffs, depending on the outcome of the simulation and the choice of
the subject. For task 2, with bets being made over the chance of the house burning conditional on the
high risk scenario, the subjective expected utility of each row i is
SEUi = [ Π × U(Mburn) ] + [ (1-Π) × U(Mnot burn) ],
(6N)
where the payoffs are again shown in Table 4.
For each of these tasks the likelihood of the observed choice can be specified. If row k is
selected from K alternatives we evaluate
seuk = exp(SEUk)
(7)
for any row k, using (6) or (6N) as appropriate, and then
LSEU = seuk/(seu1 + seu2 + ÿ + seuK)
(8)
for the specific row k chosen. This is the traditional form of the multinomial logit specification, and
directly defines the probability of observing row k.12 We can accommodate contextual behavioral
errors by using
seuk = exp[(SEUk/ν)/ξ]
(7N)
instead of (7), where ν is a normalizing term for row k and ξ is the behavioral error term. Hence, the
log-likelihood for task 1 can be written
ln LBELIEF1 (r, π, ξ; y, X) = ln(LSEU)
(9)
where LSEU is evaluated for the chosen row k. For task 2 the log-likelihood is
12
It is well known, but useful to note, that in the binary choice between A and B lotteries, (8) is
equivalent to Λ(SEUB - SEUA) where Λ(@) is the logistic cumulative density function. Thus (8) also embodies a
statistical “link function,” and can be viewed as a natural analogue of the approach to characterizing binary
choice.
-18-
ln LBELIEF2 (r, Π, ξ; y, X) = ln(LSEU).
(10)
We extend this specification to accommodate observed choices of extreme probabilities by
some subjects. Specifically, we posit a two-stage “hurdle” specification in which beliefs are either at
one extreme or interior, and then if they are interior they are chosen using (9) or (10). The corner
probabilities are characterized with likelihood contributions
ln LBELIEF1c (α; y, X) = ln(Λ[α])
(9N)
ln LBELIEF2c (β; y, X) = ln(Λ[β])
(10N)
where α and β are latent linear indices, and Λ is the familiar inverse logistic cumulative density
function. We then modify (9) and (10) to apply only when observed choices y are not at the
appropriate extreme Y:
ln LBELIEF1i (r, π, ξ; y, X | y … Y) = ln(LSEU)
(9O)
ln LBELIEF2i (r, Π, ξ; y, X | y … Y) = ln(LSEU).
(10O)
C. Joint Likelihood
The joint estimation problem is to find values of the structural parameters r, μ, π, Π and ξ
that maximize the likelihood of all choices over all tasks.13 The overall log-likelihood can be written
ln L = ln LRISK + ln LBELIEF1c + ln LBELIEF1i + ln LBELIEF2c + ln LBELIEF2i
(11)
as the sum of the component likelihoods defined by (5N), (9N), (9O), (10N) and (10O) respectively.
3. Estimates
A. General Results
In general terms, we find that subjects are on average modestly risk averse over the monetary
13
The ν terms in (3) and (7N) are not parameters, but are defined by the data and other parameters.
-19-
stakes in these tasks. We find that subjective probabilities in the virtual wilderness can be
characterized as a multi-modal, two-stage hurdle process, reflecting a mixture of some individuals
that behave as if there are extreme probabilities of 0 and 1 and some individuals that behave as if
there are interior probabilities. We find that both the mean and the dispersion of the objective
probability distribution affect the elicited subjective probabilities. Dispersion tends to increase the
subjective probability for both low and high risk tasks. Deviations in the mean from ½ tend to
increase the subjective probability in the high risk case, but also to increase it in the low risk case
(moving it in the opposite direction of the objective distribution). We also find that experts do better
than residents for some characteristics of the objective probability distribution, but not for all.
Residents behave counter-intuitively in the treatment where both the mean and the dispersion of the
objective risk are low and experts behave counter intuitively in the treatment where the mean risk is
high but the dispersion is low.
B. Risk Attitudes
Risk attitudes are estimated using the observed behavior in tasks 3 and 4. Table 6 shows the
maximum likelihood estimates of the structural model introduced earlier. We include fixed-effect
dummies for treatments B and C (treatB and treatC), so the default treatment is A. These dummies
solely reflect sampling variability across treatments, since the lottery choice tasks were identical. It is
important when we come to inferring subjective probabilities that we correct for these differences in
risk attitudes across treatments. Relative risk aversion, r, is estimated to be 0.57 in treatment A, the
default, in the top panel of Table 6. It is not statistically or economically different in treatment B, but
is lower in treatment C by 0.1 and is significant with a two-sided p-value of 0.08.
We include in our specification of risk attitudes a control for whether or not the subject was
an expert. We do this in two ways, to better see the effects. In panel A of Table 6 we simply add a
-20-
binary dummy (expert) for choices made by experts, no matter what treatment they were in. In panel
B of Table 6 we include interaction dummies between treatments B and C and experts (expB and
expC). We find that our experts are less risk averse than our residents, although we can safely reject the
hypothesis that experts are risk-neutral.14 The difference in relative risk aversion for experts is -0.14
(p-value of 0.04) when we ignore the session interactions. When we include session interactions we
notice that the significant effects of experts and treatment C in panel A of Table 6 is primarily
reflecting the interaction effect of experts in treatment C. The coefficient on expC is -0.3 with a pvalue of 0.03, and no other covariate is significant.
C. Subjective Beliefs
Turning to the elicited beliefs, Figure 6 displays the raw responses to the belief elicitation
tasks 1 and 2, pooling over all subjects. Each row from Table 4 is represented along the bottom
axes. The 4 blocks on the left of Figure 6 are for the low risk cases, and the 4 blocks on the right of
Figure 6 are for the high risk cases. The key feature of these displays is the existence of at least two
significant modes: one at the extreme probability, and another in the interior. For the low risk cases
the extreme probability is zero, which would imply that one would pick the first row in Table 4; for
the high risk cases the extreme probability is one, and this implies the last row in Table 4. These
histograms point to the importance of a hurdle specification of beliefs. There is one process
generating extreme probabilities, and another generating interior probabilities, and if these two
processes are modeled as one it could easily generate a specification error. The hurdle model is one
14
The difference in relative risk aversion for experts is -0.14 when we ignore the session interactions,
and it is statistically significant with a p-value of 0.04. In treatment A of the top panel of Table 6 they are
estimated to have a relative risk aversion of 0.43, with 95% confidence intervals between 0.31 and 0.56, and a
p-value on the hypothesis of risk-neutrality of less than 0.001. When we include session interactions, a test of
the hypothesis that all three covariates reflecting the behavior of experts are equal to zero has a p-value of
0.044.
-21-
of the oldest, and most popular, mixture specifications which can accommodate such behavior.
Tables 7 and 8 collect the estimates of subjective probabilities from our first structural model
specification which includes fixed treatment effects. Tables 9 and 10, which we will discuss later,
contain the estimates of a structural model specification that is based on two characteristics of the
objective distribution of damage risk: deviation (how far the mean of the distribution is from a
probability of ½) and dispersion (how wide the 95 percentage confidence interval is). Our three
treatments, A, B and C, differ in these two dimensions. Tables 7 and 8 are intended as agnostic ways
to evaluate the treatments, whereas Tables 9 and 10 provide complementary insight by cardinally
characterizing two dimensions of the treatments.
The first three blocks, panel A of each table, contain estimates of the core structural
parameters.15 The second and third blocks of panel A contain the estimates of the hurdle model of
the probability of a corner subjective probability of zero or 1, πcorner or Πcorner, and then the interior
probability conditional on a non-corner response, πinterior or Πinterior. The coefficients for the corner
probability are in fact the marginal probabilities of a corner outcome, and not the raw coefficients
from the binary logit specification. Similarly, the coefficients for the interior probability are the
marginal probabilities themselves, and not the raw coefficients from the multinomial logit
specification.16
Panels B and C in Tables 7 and 8 show the corner probabilities, interior probabilities, and
the implied weighted average probabilities, separately for each treatment. These are not the marginal
probabilities, but the total probabilities and are derived in the obvious way from the marginal
15
The risk aversion coefficients, r, are very close to those found in Table 6 when we only used the
lottery choices. They differ slightly here because they are estimated jointly to help explain the responses to the
belief elicitation tasks as well as the lottery tasks.
16
These transformations are more informative than the raw coefficients, and were undertaken using
the “delta method” in statistics (Oehlert [1992]), to properly account for the non-linear nature of the
transformations when computing standard errors.
-22-
estimates in the second and third blocks. Panel B is for non-expert residents, and Panel C is for experts,
the forest fire rangers. The easiest way to read these blocks is to look at the weighted average of the
corner and interior probability first. The weighted average is closer to 0 in Table 7 and closer to 1 in
Table 8, consistent with the general change in risk in the environment. For example, the weighted
average for residents in the low risk treatment A (Table 7) is 0.31, which is 0.094 × 0 + (1-0.094) ×
0.35, since the probability of a corner response is 0.094 and the interior response is 0.35 conditional
on it being a non-corner response. Similarly, the weighted average of 0.65 for residents in the high
risk treatment A (Table 8) is just 0.11 × 1 + (1-0.11) × 0.61. The average probability for the other
treatments are calculated in a similar way. Again, a corner response in Table 7 corresponds to a zero
probability, while a corner response in Table 8 corresponds to a probability of one.
While our specification with fixed treatment effects include interactions between experts and
treatment, no such interactions are included in the alternative specification in Tables 9 and 10 due to
limitations of the smaller sample size we have for experts. While continuing to sample in order to
generate a larger expert sample is always desirable, we faced a natural limitation from the small size
of the population of forest rangers in Central Florida.
Figures 7 and 8 compare the estimated subjective distributions of our three treatments to the
objective distributions in the experimental design. The objective distributions reproduce Figure 4,
and the subjective distributions are generated from the implied weighted average probabilities in
panels B and C of Tables 7 and 8.
Figures 9 and 10 show the implied marginal effects on the estimated subjective probabilities
from variations in the two objective probability distribution characteristics: deviation from ½ and
dispersion. The top left graph shows the point estimate of the subjective probability in the low risk
task, the bottom left graph shows the same thing for the high risk task, and the two graphs to the
right show the respective standard errors.
-23-
The Low Risk Task
Figures 7 and 8 clearly show that both residents and experts overestimate the risk in the low
risk task, but that the pattern is more mixed in the high risk task. For treatment A, the subjective
distributions for both residents and experts somewhat overestimate the objective one, but for the
other treatments they underestimate the objective ones to varying degrees. The most salient features
of these figures is the degree of overestimation in treatment A for residents and the degree of
underestimation in treatment C for experts.
The apparent lack of shift across the treatments for the residents in the low risk task masks
significant effects that are revealed once we investigate the corner and interior responses separately,
as in the hurdle model. In Panel A of Table 7 we see a significant increase in the proportion of
corner responses (πcorner) by residents as we move from treatment A to treatment B. We also see a
significant increase in the interior probability by residents in treatment C compared to each of A and
B. Thus, as the objective distribution shifts up from B to C the subjective response by residents is as
expected, but as the objective distribution shifts up from A to B we see an unexpected increase in
corner responses (probabilities of zero) by residents.
Experts, on the other hand, appear to react as expected to the shifts in the objective
distributions across all treatments: the corner probability is significantly lower in B and C compared
to A, and in A it is also larger than for residents. In addition, experts are not significantly different
from residents in the changes in interior probabilities.
In our alternative specification of the low risk task, when replacing the fixed treatment
effects with continuous measures of deviation and dispersion, we do not see any significant effects
on the corner probability, only on the interior probability. This is shown in Panel A of Table 9. As
we move from treatment A to treatment B, the decrease in deviation generates an (insignificant)
-24-
increase in the corner probability, and the increase in the dispersion generates an (insignificant)
decrease in the corner probability. Thus, the sign of the deviation effect on the corner probability is
consistent with the effect on residents in the first specification, but it is not significant.
The interior probability shows a significant effect from both dispersion and deviation in
Table 9. The coefficients on both are positive, implying that as we move from treatment A to B and
C, the decrease in deviation generates a (counterintuitive) decrease in the probability and the increase
in dispersion generates an increase in the probability. The positive effect from the increased
dispersion is therefore consistent with the higher interior probability we saw in treatment C
compared to treatments A and B. The effect of dispersion implies that as the dispersion increases,
the interior as well as the unconditional probabilities move up, closer to ½, becoming more uniform.
While the marginal probability effects in the table may look large, the effects they imply
within the range of deviations and dispersions that our treatments cover are quite reasonable. For
example, an increase in dispersion from 0.056 to 0.163, the lowest and highest dispersion in our
design, would result in the interior probability increasing by 0.10.
The High Risk Task
In Panel A of Table 8 we see that neither the extreme nor the interior responses for residents
are significantly affected by the difference between treatment A and B, but treatment C has a
significantly higher proportion of extreme responses and also a significantly higher interior
subjective probability. Given the relative size of the shifts in the objective distributions, as seen in
Figure 7, these effects are quite reasonable. Experts are not significantly different from residents in
treatments A and B, but there is an important significant difference in treatment C. In this treatment
we see that both the extreme and the interior responses by experts imply a lower probability than in
treatments A and B, and therefore also lower than for residents. Thus, as the objective distribution
-25-
moves to an extreme high, the subjective distribution for experts shifts down.
In our alternative specification of the high risk task, as shown in Table 10, we see that an
increase in the deviation has the expected positive sign on both the extreme responses an on the
interior probability. That is, as the objective distribution shifts up, away from ½, the subjective
distribution also shifts up. We also see that increased dispersion leads to higher subjective
probabilities. Thus, we can reject any hypothesis that implies that increased dispersion leads subjects
to hold more uniform subjective probability distributions. Instead, what we see is an increasing
degree of pessimism, in the sense of overestimating the probabilities, as dispersion is increasing.
Summary of Beliefs
In summary, we find that residents strongly overestimate the low risk in treatment A,
expressed primarily through a much higher proportion of participants who hold extreme zero
subjective probabilities. Experts, on the other hand, show the expected reduction in subjective
probabilities as the deviation increases from treatments B and C to treatment A. We can therefore
conclude that in the low risk case experience is helpful in perceiving the changes in the risk that we
induce in the VR simulations.
In the high risk case, on the other hand, residents respond as expected to the increase in
deviation by increasing the subjective probabilities. It is the experts, in this case, that show a
counterintuitive response to the high risk in C by expressing lower subjective probabilities. One
possible explanation is that experts have very strong priors that wild fires rarely or never result in a
close to certain property damage, such as imposed in our treatment C. If so, the limited 15 minute
experience they get in the simulator may not be enough to overcome those priors. This therefore
looks like an example of when experience can be a hindrance to forming accurate perceptions of
situations that are out of the ordinary.
-26-
4. Conclusions
We simulate wild fires using Virtual Reality so as to provide our experimental subjects with
information cues about risk and uncertainty that are naturalistic. This methodology allows us to
present scenarios where subjects can form their perceptions in natural ways by directly experiencing
the outcomes, but also affords us control to allow experimental treatments in terms of the cues and
clues we present. We then confront both experts and non-experts with decision tasks to infer their
perception of these risky outcomes, varying the mean and the dispersion of the objective probability
distribution, while controlling for risk attitudes with a joint estimation approach to obtain unbiased
belief estimates. Both groups are risk averse, and experts less so than non-experts.
Both residents and experts overestimate the damage probabilities in the low risk case in all
treatments. A counterintuitive increase in the subjective probability, with an increase in the deviation
from ½ in the underlying objective distribution, explains the high bias in treatment A. However, as
the deviation gets smaller in B and C, overestimation is still present due to the increased pessimism
that is a result of the increased dispersion.
Residents display a particularly high estimate for the lowest risk case, A, both by there being
fewer residents who choose the extreme zero probability than in B and C, but also by estimating an
interior probability that is no lower than in the higher risk treatments. On the other hand, a large
proportion of experts do express an extreme zero probability for A, consistent with the very low
actual risk. However, those experts who report a belief in the interior overestimate the risk, just like
the residents.
Residents do relatively well in perceiving the high risks, even for the most extreme treatment.
They correctly respond to the substantial risk increase in treatment C over that in A or B. Experts,
on the other hand have some problems. We estimate interior probabilities for experts in treatment A
that are higher than those estimated for treatments B or C. Experts also end up expressing a lower
-27-
probability for the most risky case, C, than do residents, displaying a stronger tendency to
“stubbornly” underestimate high risks. We conclude that, at least in naturalistic environments such
as virtual reality environments, experts are not always better than non-experts at estimating
compound risks. It appears that experience can sometimes be a hindrance when forming risk
perceptions in new situations.
If this behavior also holds in field environments it would imply that experts would have a
harder time judging situations that are particularly out of the ordinary, compared to non-experts. As
the global environment increasingly produces extreme phenomena, whether it is wild fires, tsunamis,
storms, or even financial crises, training experts to be less anchored on their prior experiences will
become important. We also find that the increasing degree of pessimism among residents and
experts, resulting from increased dispersion in the objective probability distribution, could result in
risk management decisions that are sub-optimal as natural events become more variable due to
increasing global temperatures.
-28-
Table 1: Description of the Subject Pool
Proportion or
Mean
of Residents
Proportion or
Mean
of Experts
34%
12%
53
40
1
Female
2
Age
3
Poor (<$35K)
15%
27%
4
Middle Income
61%
65%
5
Rich (>$100K)
21%
6%
6
College education
72%
33%
7
Smokers
68%
8%
8
Play video games
41%
57%
9
Fire close to home (within 20 miles)
65%
82%
10
Personal relation lost property in wild fire
23%
39%
11
Action taken to prevent fire damage to your house
55%
78%
12
Number of simulations viewed
8
8
13
Fraction of viewed simulations with low risk
50%
48%
14
Fraction of viewed simulations lasted one day
51%
56%
15
Fraction of viewed simulations in drought year
52%
57%
16
Fraction of viewed simulations near simulated house
31%
29%
-29-
Figure 1: Example Output from FARSITE Simulation of Fire in the Lake Diaz Area
-30-
Figure 2: Lake Diaz Map Showing the House and the 6 Lightning Strike Locations
-31-
Figure 3: Images from VR Rendering of Wild Fires
-32-
Table 2: Background Variables for Fire Simulations
Drought year 1999
Non-drought year 2003
Extreme spread conditions
High spread conditions
Moderate spread conditions
Low spread conditions
One or Two day duration
One or two day duration
One or two day duration
One or two day duration
Lightning Strike Locations: Central, Central East, Central West,
North, South East, South West
3/5
3/28
4/2
5/18
5/23
5/30
6/3
6/8
6/12
Lightning Strike Locations: Central, Central East, Central West,
North, South East, South West
6/17
3/6
3/26
4/4
4/12
5/26
5/27
6/18
6/20
6/22
Table 3: Experimental Design
Task
Type of task
Subjective beliefs
Risk attitudes
1
Conditional bet on low risk outcome
U
U
2
Conditional bet on high risk outcome
U
U
3
Simple lottery choice
U
4
Compound lottery choice
U
-33-
6/25
Table 4: Typical Decision Sheet for Betting Task
Row number
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
Earnings if house burns
$0
$5
$10
$15
$20
$25
$30
$35
$40
$45
$50
$55
$60
$65
$70
$75
$80
$85
$90
$95
$100
$105
$110
$115
$120
$125
$130
$135
$140
$145
$150
-34-
Earnings if house doesn’t burn
$150
$145
$140
$135
$130
$125
$120
$115
$110
$105
$100
$95
$90
$85
$80
$75
$70
$65
$60
$55
$50
$45
$40
$35
$30
$25
$20
$15
$10
$5
$0
Table 5: Decision Sheet for Risk Attitudes Task
Your Choice
(Circle A or B)
Option A
Option B
1
$40.00 if throw of die is 1
$32.00 if throw of die is 2-10
$77 if throw of die is 1
$2.00 if throw of die is 2-10
A
B
2
$40.00 if throw of die is 1-2
$32.00 if throw of die is 3-10
$77 if throw of die is 1-2
$2.00 if throw of die is 3-10
A
B
3
$40.00 if throw of die is 1-3
$32.00 if throw of die is 4-10
$77 if throw of die is 1-3
$2.00 if throw of die is 4-10
A
B
4
$40.00 if throw of die is 1-4
$32.00 if throw of die is 5-10
$77 if throw of die is 1-4
$2.00 if throw of die is 5-10
A
B
5
$40.00 if throw of die is 1-5
$32.00 if throw of die is 6-10
$77 if throw of die is 1-5
$2.00 if throw of die is 6-10
A
B
6
$40.00 if throw of die is 1-6
$32.00 if throw of die is 7-10
$77 if throw of die is 1-6
$2.00 if throw of die is 7-10
A
B
7
$40.00 if throw of die is 1-7
$32.00 if throw of die is 8-10
$77 if throw of die is 1-7
$2.00 if throw of die is 8-10
A
B
8
$40.00 if throw of die is 1-8
$32.00 if throw of die is 9-10
$77 if throw of die is 1-8
$2.00 if throw of die is 9-10
A
B
9
$40.00 if throw of die is 1-9
$32.00 if throw of die is 10
$77 if throw of die is 1-9
$2.00 if throw of die is 10
A
B
$40.00 if throw of die is 1-10
$77 if throw of die is 1-10
A
B
Decision
10
-35-
Figure 4: Simulated Probability of Burn by Treatment
Treat me nt A High Risk
Density
Treatment A Low Risk
0
.1
.2
.3
.4
.5
.6
Posterior Probability of House B urning
.8
.9
1
.9
1
.9
1
Treat me nt B Hig h Risk
Density
Treatment B Low Risk
.7
0
.1
.2
.3
.4
.5
.6
Posterior Probability of House B urning
.8
Treat me nt C Hig h Risk
Density
Trea tme nt C Low Risk
.7
0
.1
.2
.3
.4
.5
.6
Posterior Probability of House B urning
-36-
.7
.8
Figure 5: Overview of Procedures
Online optional demographics in advance.
On location:
Questionnaires:
In order to optimize the use of the simulators we staggered the arrival of subjects, and we had some
of them do the two questionnaires at the beginning and some at the end. Those who had completed
these online simply skipped them in the session.
•
Demographic questionnaire
•
Raven’s Progressive Matrix test
Simulator Tasks:
•
Instructions about the simulator tasks
•
Practice in using the simulator, not involving fires.
•
Experience of fires in the simulator (2 common simulations and 15 minutes of elective
simulations)
•
•
•
2 betting tasks. (There were a total of 4 simulator tasks for each subject. The other two were
given after the betting tasks. They are not analysed here.)
Selection of which of the four simulator tasks to pay.
Payoff determination if a betting task was selected for pay: Using dice select background
variables. Run simulation to see if house burns or not.
Lottery Tasks:
•
Instructions for lottery task.
•
Make decisions in two lottery tasks. Select one of them to pay. Select row to pay for.
All die rolls were done by the subject.
-37-
Table 6: Estimates of Risk Attitudes
Parameter
Point
Estimate
Standard
Error
p-value
95% Confidence Interval
A. Simple Covariate for Experts
constant
0.572
0.035
<0.001
0.502
0.641
treatB
-0.012
0.058
0.83
-0.126
0.101
treatC
-0.119
0.067
0.08
-0.249
0.012
expert
-0.137
0.065
0.04
-0.265
-0.009
B. Interaction Covariates for Experts
constant
0.556
0.038
<0.001
0.482
0.631
treatB
0.002
0.063
0.97
-0.122
0.127
treatC
-0.070
0.075
0.36
-0.217
0.078
expert
-0.007
0.075
0.92
-0.155
0.140
expB
-0.123
0.153
0.42
-0.422
0.176
expC
-0.301
0.141
0.03
-0.578
-0.024
-38-
Figure 6: Histograms of Raw Responses
Low Risks
0
10
High Risks
A
B
A
B
C
Total
C
Total
20
30
0
10
20
30
Row Chosen in Condit ional B etting T ask
0
10
20
30
0
10
20
Row Chose n in Conditional B etting T ask
-39-
30
Table 7: Estimates of Low Risk Beliefs
Parameter
Point
Estimate
Standard
Error
p-value
95% Confidence Interval
A. Structural Model Estimates
r
constant
0.559
0.036
<0.001
0.488
0.631
treatB
-0.008
0.062
0.90
-0.128
0.113
treatC
-0.095
0.075
0.20
-0.241
0.051
expert
-0.005
0.071
0.94
-0.144
0.134
expB
-0.176
0.141
0.21
-0.451
0.100
expC
-0.312
0.157
0.05
-0.620
-0.005
πcorner constant
0.094
0.032
0.003
0.032
0.156
treatB
0.113
0.054
0.04
0.007
0.218
treatC
0.102
0.064
0.11
-0.024
0.228
expert
0.263
0.132
0.05
0.004
0.522
expB
-0.081
0.025
0.002
-0.131
-0.031
expC
-0.060
0.03
0.05
-0.119
-0.004
πinterior constant
0.347
0.028
<0.001
0.293
0.402
treatB
-0.005
0.037
0.89
-0.077
0.067
treatC
0.107
0.045
0.02
0.019
0.195
expert
-0.030
0.058
0.60
-0.143
0.083
expB
0.001
0.112
1.00
-0.219
0.220
expC
0.013
0.089
0.88
-0.161
0.188
B. Inferred Subjective Beliefs of Residents
A: corner
0.094
0.032
0.003
0.032
0.156
A: interior
0.347
0.028
<0.001
0.293
0.402
A: average
0.315
0.029
<0.001
0.259
0.371
A: objective
0.050
-40-
B: corner
0.207
0.044
<0.001
0.122
0.292
B: interior
0.342
0.028
<0.001
0.288
0.397
B: average
0.271
0.026
<0.001
0.220
0.323
B: objective
0.170
C: corner
0.196
0.056
<0.001
0.087
0.305
C: interior
0.454
0.034
<0.001
0.387
0.522
C: average
0.365
0.037
<0.001
0.293
0.438
C: objective
0.230
C. Inferred Subjective Beliefs of Experts
†
A: corner
0.357
0.128
0.005
0.106
0.609
A: interior
0.318
0.055
<0.001
0.210
0.425
A: average
0.204
0.052
<0.001
0.102
0.307
A: objective
0.050
B: corner
0.033
0.032
0.31
-0.031 †
0.097
B: interior
0.343
0.106
0.001
0.136
0.550
B: average
0.332
0.104
0.001
0.128
0.535
B: objective
0.170
C: corner
0.077
0.064
0.23
-0.048 †
0.202
C: interior
0.469
0.089
<0.001
0.294
0.644
C: average
0.433
0.088
<0.001
0.261
0.605
C: objective
0.230
Estimates calculated using the “delta” approximation method for non-linear combinations of
estimated structural parameters. The approximation involved may mean that some estimates
fall outside the unit interval.
-41-
Table 8: Estimates of High Risk Beliefs
Parameter
Point
Estimate
Standard
Error
p-value
95% Confidence Interval
A. Structural Model Estimates
r
constant
0.558
0.036
<0.001
0.488
0.628
treatB
-0.010
0.061
0.88
-0.130
0.110
treatC
-0.072
0.077
0.35
-0.223
0.079
expert
-0.0004
0.071
1.00
-0.138
0.139
expB
-0.144
0.164
0.38
-0.464
0.177
expC
-0.284
0.137
0.04
-0.553
-0.020
πcorner constant
0.105
0.033
0.002
0.040
0.169
treatB
-0.001
0.047
0.98
-0.092
0.090
treatC
0.307
0.077
<0.001
0.157
0.457
expert
0.110
0.115
0.34
-0.115
0.334
expB
0‡
expC
-0.074
0.030
0.02
-0.133
-0.014
πinterior constant
0.614
0.025
<0.001
0.565
0.664
treatB
-0.009
0.037
0.81
-0.082
0.064
treatC
0.156
0.067
0.02
0.025
0.288
expert
0.076
0.063
0.22
-0.047
0.199
expB
-0.093
0.105
0.38
-0.299
0.113
expC
-0.236
0.118
0.05
-0.467
-0.005
B. Inferred Subjective Beliefs of Residents
A: corner
0.105
0.033
0.002
0.040
0.169
A: interior
0.614
0.025
<0.001
0.565
0.664
A: average
0.655
0.026
<0.001
0.603
0.706
A: objective
0.570
-42-
B: corner
0.103
0.033
0.002
0.039
0.168
B: interior
0.606
0.028
<0.001
0.550
0.661
B: average
0.646
0.028
<0.001
0.592
0.701
B: objective
0.650
C: corner
0.412
0.069
<0.001
0.276
0.547
C: interior
0.771
0.066
<0.001
0.642
0.901
C: average
0.865
0.040
<0.001
0.787
0.944
C: objective
0.960
C. Inferred Subjective Beliefs of Experts
A: corner
0.214
0.110
0.05
-0.001 †
0.430
A: interior
0.691
0.062
<0.001
0.570
0.811
A: average
0.757
0.060
<0.001
0.640
0.874
A: objective
0.570
B: corner
0‡
B: interior
0.512
0.099
<0.001
0.319
0.705
B: average ¤
0.512
0.099
<0.001
0.319
0.705
B: objective
0.650
C: corner
0.160
0.128
0.21
-0.092 †
0.412
C: interior
0.562
0.087
<0.001
0.393
0.732
C: average
0.632
0.091
<0.001
0.455
0.810
C: objective
0.960
†
Estimates calculated using the “delta” approximation method for non-linear combinations of
estimated structural parameters. The approximation involved may mean that some estimates
fall outside the unit interval.
‡ Constrained to be zero, since numerical derivatives could not be calculated accurately and the
point estimate was converging to an extremely small value.
¤ These estimates are the same as the interior estimate, since the corner probability has zero mass.
-43-
Table 9: Additional Estimates of Low Risk Beliefs †
Point
Estimate
Standard
Error
p-value
constant
0.559
0.036
<0.001
0.488
0.631
treatB
-0.008
0.061
0.90
-0.127
0.112
treatC
-0.095
0.074
0.20
-0.241
0.050
expert
-0.007
0.069
0.92
-0.142
0.128
expB
-0.176
0.135
0.19
-0.441
-0.017
expC
-0.309
0.149
0.05
-0.601
-0.005
πcorner constant
0.714
0.676
0.29
-0.611
2.04
dispersion
-0.688
0.859
0.42
-2.371
0.995
deviation
-0.707
0.696
0.31
-2.072
0.657
expert
0.115
0.219
0.60
-0.316
0.545
πinterior constant
0.055
0.074
0.45
-0.089
0.199
dispersion
0.944
0.075
<0.001
0.770
1.093
deviation
0.636
0.349
0.07
-0.048
1.320
expert
-0.005
0.011
0.63
-0.026
0.016
Parameter
r
†
95% Confidence Interval
Estimates calculated using the “delta” approximation method for non-linear combinations of
estimated structural parameters. The approximation involved may mean that some estimates
fall outside the unit interval.
-44-
Table 10: Additional Estimates of High Risk Beliefs †
Point
Estimate
Standard
Error
p-value
constant
0.557
0.036
<0.001
0.488
0.627
treatB
-0.011
0.061
0.86
-0.131
0.109
treatC
-0.085
0.074
0.25
-0.231
0.061
expert
-0.017
0.070
0.81
-0.155
0.121
expB
-0.123
0.157
0.43
-0.432
0.185
expC
-0.224
0.131
0.09
-0.480
0.032
πcorner constant
0.006
0.0103
0.56
-0.014
0.026
dispersion
0.994
0.010
<0.001
0.973
1.014
deviation
0.796
0.132
<0.001
0.537
1.055
expert
-0.001
0.003
0.74
-0.008
0.005
πinterior constant
0.398
0.150
0.008
0.104
0.692
dispersion
0.600
0.161
<0.001
0.282
0.916
deviation
0.440
0.195
0.02
0.056
0.821
expert
-0.010
0.047
0.83
-0.102
0.083
Parameter
r
†
95% Confidence Interval
Estimates calculated using the “delta” approximation method for non-linear combinations of
estimated structural parameters. The approximation involved may mean that some estimates
fall outside the unit interval.
-45-
Figure 7: Residents’ Subjective Probabilities of a Burn
T reat me nt A H igh Risk
Density
T reatment A Low R isk
0
.1
.2
.3
.4
.5
.6
Subjectiv e Probability of House Burning
.8
.9
1
.9
1
.9
1
T reat me nt B Hig h R isk
Density
Tr eatment B Low R isk
.7
0
.1
.2
.3
.4
.5
.6
Subjectiv e Probability of House Burning
.8
T reat me nt C Hig h R isk
Density
T rea tme nt C Low Risk
.7
0
.1
.2
.3
.4
.5
.6
Subjectiv e Probability of House Burning
.7
.8
Figure 8: Experts’ Subjective Probabilities of a Burn
Treat me nt A High Risk
Density
Treatment A Low Risk
0
.1
.2
.3
.4
.5
.6
Subjectiv e Probability of House Burning
.8
.9
1
.9
1
.9
1
Treat me nt B Hig h Risk
Density
Treatment B Low Risk
.7
0
.1
.2
.3
.4
.5
.6
Subjectiv e Probability of House Burning
.8
Treat me nt C Hig h Risk
Density
Trea tme nt C Low Risk
.7
0
.1
.2
.3
.4
.5
.6
Subjectiv e Probability of House Burning
-46-
.7
.8
Figure 9: Effect of Deviation and Dispersion on Subjective Beliefs of Residents
A. Effects of Deviation
B. Effects of Disperson
Evaluated at Average Dispersion
.5 Low Risk
Low Risk
.5
.6
.5
.4
.4
Evaluated a t Avera ge Deviation
.5 Low Risk
Low Risk
.4
.3
.2
Standard
Error
.3
Point
Estimate
Standard
Error
Point
Estimate
.4
.3
.3
.2
.2
.2
.1
.1
0
.2
.3
.4
Deviati on
.5
.5
.3
.4
Deviati on
.5
0
0 . 02 .04 .06.08 .1 .12.14. 16
Dispersion
High Risk
1
0 . 02 .04.06 .08 .1 .12. 14 .16
Dispersion
High Risk
.5
.9
.4
.5
High Risk
.4
.8
.3
.3
.2
.7
Standard
Error
.4
Point
Estimate
Standard
Error
Point
Estimate
0
.2
High Risk
.6
.1
.1
.6
.3
.2
.5
.1
.2
.1
.1
0
0
.1
. 2 .3
Deviati on
.4
.5
0
0
.1
. 2 .3 . 4
Deviati on
.5
0 .02 . 04 .06 .08 .1 .12
Dispersion
0 . 02 .04 .06 . 08 .1 .12
Dispersion
Figure 10: Effect of Deviation and Dispersion on Subjective Beliefs of Experts
A. Effects of Deviation
B. Effects of Dispersion
Evaluated at Average Dispersion
Low Risk
.5
Evaluated at Average Deviation
Low Risk
.6
Low Risk
.4
Point
Estimate
Standard
Error
Point
Estimate
.3
.2
.2
Standard
Error
.4
.3
.3
.3
.2
.2
.1
.1
.5
High Risk
.5
Standard
Error
.8
.7
.6
0
.2
.3
.4
Devia ti on
.5
High Risk
1
.1
. 2 .3 . 4
Devia ti on
.5
High Risk
.5
.9
.4
.3
.8
.3
.2
.7
.6
0
0
0 . 02 .04.06 .08 .1 .12. 14 .16
Dispersion
.4
.1
.5
0
0 . 02 .04 .06.08 .1 .12.14. 16
Dispersion
Point
Estimate
.3
.4
Devia ti on
.1
.1
0
.2
Point
Estimate
.5
.5
.4
.9
Low Risk
Standard
Error
.4
.1
. 2 .3 . 4
Devia ti on
.5
0
0 .02 . 04 .06 .08 .1 .12
Dispersion
-47-
.2
.1
.5
0
High Risk
0 . 02 .04 .06 . 08 .1 .12
Dispersion
References
Andersen, Steffen; Fountain, John; Harrison, Glenn W., and Rutström, E. Elisabet, “Estimating
Subjective Probabilities,” Journal of Risk and Uncertainty, 48, 2014, 207-229.
Butry, D.T., Mercer E.D., Prestemon, J.P., Pye, J.M, and Holmes, T.P., “What Is the Price of
Catastrophic Wild Fire,” Journal of Forestry, 99(11), November 2001, 9-17.
Cox, James C.; Sadiraj, Vjollca, and Schmidt, Ulrich, “Paradoxes and Mechanisms for Choice under
Risk,” Experimental Economics, forthcoming 2015.
Desvousges, William H.; Johnson, F. Reed, and Banzhaf, H. Spencer, Environmental Policy Analysis
With Limited Information: Principles and Applications of the Transfer Method (New York: Edward
Elgar, 1999).
Dixit, Vinayak; Harrison, Glenn W, and Rutström, E. Elisabet, “Estimating the Subjective Risks of
Driving Simulator Accidents,” Accident Analysis and Prevention, 62, 2014, 63-78.
Finney, Mark A., “FARSITE: Fire Area Simulator – Model Development and Evaluation,” Research
Paper RMRS-RP-4, Rocky Mountain Research Station, Forest Service, United States
Department of Agriculture, March 1998.
Fiore, Stephen M.; Harrison, Glenn W.; Hughes, Charles E., and Rutström, Elisabet, “Virtual
Experiments and Environmental Policy,” Journal of Environmental Economics & Management,
57(1), January 2009, 65-86.
Harrison, Glenn W.; Lau, Morten I., and Rutström, E. Elisabet, “Estimating Risk Attitudes in
Denmark: A Field Experiment,” Scandinavian Journal of Economics, 109(2), 2007, 341-368.
Harrison, Glenn W., and Rutström, E. Elisabet, “Risk Aversion in the Laboratory,” in J.C. Cox and
G.W. Harrison (eds.), Risk Aversion in Experiments (Bingley, UK: Emerald, Research in
Experimental Economics, Volume 12, 2008).
Harrison, Glenn, and Swarthout, J. Todd, “Experimental Payment Protocols and the Bipolar
Behaviorist,” Theory and Decision, 77, 2014, 423-438.
Hey, John D., and Orme, Chris, “Investigating Generalizations of Expected Utility Theory Using
Experimental Data,” Econometrica, 62(6), November 1994, 1291-1326.
Holt, Charles A., and Laury, Susan K., “Risk Aversion and Incentive Effects,” American Economic
Review, 92(5), December 2002, 1644-1655.
Jeffreys, H., “An Invariant Form for the Prior Probability in Estimation Problems,” Proceedings of the
Royal Society of London, Series A, 186, 1946, 453-461.
Kruschke, John K., and Hullinger, Richard A., “Evolution of Attention in Learning,” in N.
Schmajuk (ed.), Computational Models of Conditioning (New York: Cambridge University Press,
2010, 10-52).
Kruschke, John K., “Highlighting: A Canonical Experiment,” in B. Ross (ed.), The Psychology of
Learning and Motivation (New York: Elsevier, volume 51, 153-185).
-48-
Oehlert, Gary W., “A Note on the Delta Method,” The American Statistician, 46(1), February 1992, 2729.
Wilcox, Nathaniel T., “Stochastic Models for Binary Discrete Choice Under Risk: A Critical Primer
and Econometric Comparison,” in J. Cox and G.W. Harrison (eds.), Risk Aversion in
Experiments (Bingley, UK: Emerald, Research in Experimental Economics, Volume 12,
2008).
Wilcox, Nathaniel T., “‘Stochastically More Risk Averse:’ A Contextual Theory of Stochastic
Discrete Choice Under Risk,” Journal of Econometrics, 162(1), May 2011, 89-104.
-49-