Download Report

Neurodynamics of Decision Making – a Computational Approach
Hans Liljenströmab and Azadeh Hassanejad Nazira
a
Div of Biometry and Systems Analysis, ET, SLU
P.O. Box 7032, SE-75007 Uppsala, Sweden
[email protected]
[email protected]
b
Agora for Biosystems
P.O. Box 57
SE-19322 Sigtuna,Sweden
www.sigtunastiftelsen.se/agoraforbiosystems
Abstract
Decision making is a complex process, that normally seems to involve several brain structures. In
particular, amygdala, orbitofrontal cortex (OFC) and lateral prefrontal cortex (LPFC) seem to be
essential in human decision making, where both emotional and cognitive aspects are taken into
consideration. In this paper, we present a stochastic population model representing the neural
information processing of decision making, from perception to behavioral activity. We model the
population dynamics of the three neural structures significant in the decision making process
(amygdala, OFC and LPFC), as well as their interaction. In our model, amygdala and OFC
represent the neural correlates of secondary emotion, while the activity of OFC neural populations
represents the outcome expectancy of alternatives, and the cognitive aspect of decision making is
controlled by LPFC. The results may have implications for how we make decisions for our
individual actions, as well as for societal choices, where we take examples from transport and its
impact on climate change.
Keywords: Decision making, emotion-cognition, neural network model, amygdala, prefrontal cortex
1. Introduction
Our actions are to a large extent a result of decision making, which can be categorized into three
phases: 1) Prevailing alternatives concerning the internal and external states are emotionally
evaluated and prioritized, 2) a cognitive assessment of the options and the selection of actions,
depending on needs (Damasio, 1996). 3) the execution of an action, and evaluation of the resulting
effect, which allows for a comparison between actual and expected value. Based on the “prediction
error”, the assigned values to the choices in the first step are revisited and learnt, possibly resulting
in a change of mind. (Gold & Shadlen, 2007; Doya, 2008; Bedia et al., 2012).
The decision making (DM) process may not be as rational as often believed, which has been
demonstrated by Kahneman and colleagues (Kahneman & Tversky, 1979, Kahneman, 2011).
According to their hypothesis, decision making is the result of an interplay between an
intuitive/emotional and a rational/cognitive system, represented as System 1 and System 2,
respectively. This dual process model posits the integration of a “bottom-up”, intuitive, fast,
implicit, emotional system (1) and a “top-down”, deliberative, slow, explicit, cognitive system (2).
Amygdala, as a part of the limbic system, has since long been associated with emotional processing,
1
related to sensory perception and learning, linking the stimulus provoking emotional response and
its emotional value (Whalen & Phelps, 2009). The functionality of the amygdala is realized through
its connection to the orbitofrontal cortex (OFC), which receives extensive neural afferents from
different sensory modalities. The bidirectional connections between these two structures are
supposedly embodied in an affective decision making process, where the perception and evaluation
of environmental stimuli constitute the first phase. (Roesch & Schoenbaum, 2006).
In the second phase, the emotionally assessed stimuli in the amygdala-OFC pathway would be
monitored cognitively and a final decision would be taken. Lateral prefrontal cortex (LPFC) mainly
participates in the cognitive evaluation of stimuli. In contrast to OFC, which pursues short term
rewards, LPFC primarily values future rewards. LPFC contributes to the prediction of the expected
cognitive optional values. The resultant value of emotional and cognitive evaluations guides the
action selection. (Gray et al, 2002; Krawczyk, 2002).
The outcome of the second phase is followed by the execution of an action. The actual value of the
selected action is evaluated and compared with the expected one by OFC and LPFC. The computed
prediction error provides the basis for learning. Through a recurrent pathway from LPFC to OFC a
feedback signal is transmitted and the actual value of the performed action is maintained in OFC.
Learning occurs in OFC by changing the neural firing frequencies, depending on the prediction
error. The learning of new cue-outcome association and formation of adaptive behavior is also
supported by OFC. The interconnection between amygdala and OFC associates the cue to its actual
value and to generate the outcome expectancy signal inducing the behavioral change. (Anderson &
Matessa, 1997; Dixon & Christoff, 2014).
These systems may correspond to “fast and slow thinking”, respectively (D. Kahneman). The
corresponding stuctures, as well as the connections between them and other brain areas are
illustrated in Fig. 1. Integration of emotional and rational values in LPFC leads to the selection of
one of several possible and available options. The output is transmitted to motor cortex so as to
execute an action.
Fig 1. The figure represents the interactivities of different of five neural structures in the decision making
process. Amygdala and Orbitofrontal Cortex (OFC) are considered as the main organizations in emotional
decision making and Lateral Prefrontal Cortex (LPFC) play crucial role in rational analysis. The inputs
comes from all sensory modalities and the taken action is executed through the connectivity of LPFC and
motor cortex. The flexibility of this process depends on the real value of the executed action computed by
VentroMedial Prefrontal Cortex (VMPC) and transmitted to LPFC and OFC to update the stored rational
and emotional attitudes toward the chosen alternative.
2
2. Model description
As was outlined in the previous section, several key mental processes are involved when making a
decision. The proposed model attempts to present an adaptive DM under varying internal and
external contexts. Experimental results indicate the involvement of different neural sections in
decision making process, but to simplify the model, we focus our attention on three crucial neural
structures: Amygdala, orbitofrontal cortex (OFC) and lateral prefrontal cortex (LPFC). The model
encompasses an input-procedure-action-feedback process. This flexible process can be followed in
both the emotional and rational/cognitive systems. The salient features of two systems, System 1
and 2, are illustrated in our model.
The decision making process is modeled at a mesoscopic level of cortical neurodynamics. The
structures and dynamics of the three brain areas are modelled with attractor neural networks (see
below). Oscillatory rhythms encode information related to perception, cognition and emotional
associations, and are the result of interactions between excitatory and inhibitory neural populations.
The network activity can be envisioned as local field potentials (LFP) or electroencephalogram
(EEG) readouts.
Our model is based on a previous developed cortical neural network model (Liljenström, 1991),
which has been extended and modified to mimic the structures of the amygdala, OFC and LPFC,
respectively. The general structures of these three systems and their interconnections are shown in
Fig 1. whereas their schematic network structure is given in Fig. 2.
Fig. 2. This figure represents three neural structures. The upper and lower networks are
composed of 25 inhibitory neurons and the network in the middle is composed of 100
excitatory neurons. The external inputs stimulate (subset of) excitatory neural network.
The stimulation of excitatory neurons is the start of activity of the system. Stimulated
excitatory neurons excite inhibitory neurons which result in the excitation-inhibition
balance.
The excitatory sub-layer of each model structure consists of a network of 100 neural units
(populations), while each of the two inhibitory networks is composed of 25 inhibitory units. The
excitatory network nodes are connected recurrently, while there is no internal connection among
inhibitory network nodes. An excitatory-inhibitory balance results from the bidirectional
connectivity of excitatory units with the two inhibitory networks on either side.
3
The function of the three neural structures requires the formation and update of cell assemblies. In
our model, some specific parameters affect the oscillatory activities of the cell assemblies. A cell
assembly, is expressed as synchronized network oscillation, s, is the representation of stored
experiences, attitudes and associated feelings towards the consequence of choosing any of the
present options (alternatives) as the ultimate decision.
A decision in each of the emotional and rational system separately is based on the “winning”
response activity. The competition between the stored pattern toward the cue can be determined
with the help of cosine similarity of the frequency vectors f. The highest value of V will win the
competition, and result in a decision to choose that option.
Vopt = |s| × f × A
The ultimate decision is the resultant of the emotional and rational decisions. The integration of
emotion and cognition can be computed by adding the emotional and cognitive values, while
considering the dominance of each structure over the other.
3. Simulation Example and Results
In this model, some parameters are determined by a person’s attitudes and priorities. Cell
assemblies in Amygdala, OFC and LPFC represent the feeling, expectancy value and rules for the
outcome of a decision. According to the three pillars of sustainable development, the rational and
emotional attitudes considered are: 1) Ecology (eco), 2) Social (soc), and 3) Economic (mon).
The choice depends on various external (weather, costs, traffic etc) and internal (mood, motivation,
attitude etc) factors. Hence, we suggest there are three options to choose between: car (C), bike (B),
and train (T), which all are considered to be available, albeit with different levels of convenience.
The value equation can be reformulated based on the scenario:
V (C) = Peco × U(eco) + Peco × U(soc)+ Pmon × U(mon) = Veco + Vsoc + Vmon
(12)
In this example we assume that the decision maker is pro-environment with his emotional and
rational priorities in transportation.
Emotional priorities: C > B > T ;
Rational priorities : B > T > C
In the absence of external/internal context, the priority should be followed and the option with the
highest priority should be chosen in each system. In this case, emotionally, C is the best choice
while rationally B is preferred.
In this case the cell assemblies that determine the association between the options and feelings and
the consequence of taking an option are stimulated with external and internal stimuli. In the
program the frequencies are just random number generated in the gamma range [30Hz, 80Hz]. As is
illustrated in below, the randomly generated frequencies of emotional memory and rational memory
are values in gamma range but different values. The difference between these frequencies are due to
different external stimuli stimulate these two systems.
4
Fig. 5 The oscillatory activity of neurons representing the association between the car as an option with highest priority
and emotion. In the upper picture, the red oscillations represent the mean membrane potential of the stimulated neurons
and the blue ones are non-stimulated neurons. In the lower picture, the red and green oscillations show the mean
membrane potential of the inhibitory networks and blue one indicate the oscillation of all excitatory neurons.
Fig. The oscillatory activity of assemblies in emotional and rational systems representing the significance
of car as one of the options during decision making process. As is illustrated the first 3 seconds is the
length of emotional processing and remained 7 seconds the option is analysed rationally. The higher
frequency of neurons during emotional processing and low frequency during rational processing indicate
the high emotional priority and low rational priority of car.
5
5. Discussion
In this paper we have modelled a major part of the neural system involved in decision making,
including the amygdala, the orbitofrontal cortex and the lateral prefrontal cortex. These systems
also represent emotional, as well as cognitive aspects of the decision making. With our model we
have demonstrated how different cell assemblies, representing different optional choices available
to the individual, may compete with respect to level of activity. This level was suggested to be a
combined measure of the size of the cell assembly (number of neural populations), and the
frequency and amplitude of the oscillatory activity of the neural populations. The winning assembly
was simply the one with the largest neural activity, measured as the product of the three assembly
characteristics (number, frequency, amplitude). The different options get different values,
depending on internal as well as external factors. Internal factors could be attitudes, values, mood
etc, while the external factors include weather, availability, distance etc, but also influence of other
individuals in a social context. In fact, context could be considered the combined internal and
external context, while a signal could also be either internal or external. A signal is typical specific,
having a specific information content, that usually is transmitted to the decision making systems via
our sensory systems, or via other brain areas.
Hence, for any particular input signal, the final decision made by an individual could shift
depending on internal and external context. The experience of our decisions/choices are also learnt
and may influence future decisions. For example, if we have decided to take the bus instead of our
car one morning, and the bus is delayed or maybe even does not show up, this experience will affect
our willingness to take the bus the next day. Similarly, if there I end up in traffic jams almost every
morning taking my car, I (eventually) may decide to try another means of transport. In addition to
these contextual influences, our neighbors in our social networks also influence our decisions,
depending on the psycho-social distance (trust). This influence may either be informational, through
direct communication, or by adopting the trusted ones’ decisions/behaviour.
Understanding the decision making process of an individual may be helpful not only to that
individual, but also to those interacting with that individual, as well as to policy makers and
businesses who want to influence our behavior in various ways. Apparently, our decisions are based
on biological (genetic, neural, physiological) factors, but also to social and environmental factors,
that constitute a complex web of causation, making it hard or even impossible to predict an action
or the behaviour for any given individual. Yet, taken en masse, the behaviour of hundreds or
thousands of individuals may be more or less predictable, and to some extent controllable,
depending on our knowledge of internal as well as external signals and contexts. This knowledge
can be used (and misused) for such things as spreading of information, nudging, advertisement etc.
There are of course many simplifications and assumptions made in our modeling that could be
discussed and re-examined, and which may be crucial for the behaviour of our model system. For
example, experiments on OFC-lesioned animals indicate the incapability of those animals in
devaluation of the reward (Winstanley et al., 2004). Intact OFC is devaluating the reward regarding
the time of delivery. Large delayed reward has a lower value, comparing to small short term reward.
This can be presented with the help of a hyperbolic discounting function. This function models the
exponential reduction of rewards in terms of time.
Acknowledgements
We are grateful for generous grants from the EC-FP7 for COMPLEX, Grant 308601, and from the
John F. Templeton Foundation, Grant 39987.
6
References
Anderson, J.R. Matessa, M. (1997) A production system theory of serial memory. Psychological
Review 104: 728-748.
Baddeley, A.D., & Hitch, G. (1974) Working memory. In G.H. Bower (Ed.), The psychology of
learning and motivation: Advances in research and theory. V. 8, pp. 47–89. New York: Academic
Press.
Barbas, H. (2007) Specialized elements of orbitofrontal cortex in primates. Annals of the New york
Academy of Sciences, 1121:10-32.
Barbas, H. Zikopoulos,B. (2006) Sequential and parallel circuits for emotional processing. In:
primate orbitofrontal cortex In Zald, D. (Ed.) Rauch, S. (Ed). The Orbitofrontal Cortex.
Oxford University Press ISBN: 9780198565741
Bedia, M.G, Di Paolo, E. (2012) Unreliable gut feelings can lead to correct decisions: the somatic
marker hypothesis in non-linear decision chains. Frontier in psychology. v.3:384. doi:
10.3389/fpsyg.2012.00384
Damasio, A.R. (1996) The Somatic marker hypothesis and the possible functions of the prefrontal
cortex. Phil. Trans. R. Soc. Lond. B 351 (1346): 1413–20.
Dixon, M.L., Christoff, K.E. (2014) The lateral prefrontal cortex and complex value-based learning
and decision making. Neurosci Biobehav Rev. 45C:9-18.
Doya, K., (2008). Modulators of decision making, Nature Neuroscience 11, 410-416.
Ernstemail, M., Paulus, M.P. (2005) Neurobiology of Decision Making: A Selective Review from a
Neurocognitive and Clinical Perspective. Psychiatry 58:597–604.
Frank, M.J., Cohen, M.X. Sanfey, A.G. (2009) Multiple Systems in Decision Making: A
Neurocomputational Perspective. Current Directions in Psychological Science, 18:73-77.
Ghashghaei, H.T., Barbas, H. (2000) Pathways for emotion: interactions of prefrontal and anterior
temporal pathways in the amygdala of the rhesus monkey. Neuroscience. 115(4):1261-79.
Gold. JI, Shadlen. MN, The Neural Basis of Decision Making, In Annual Review of
Neuroscience, Vol. 30, No. 1, 2007.
Goldman-Rakic PS. (1987) Circuitry of primate prefrontal cortex and regulation of behavior by
representational memory. In: Mountcastle, VB, Plum, F. Geiger, SR (Eds). Handbook of
Physiology. Vol. V. Bethesda, MD. 373–417.
Gray, J. R. Braver, T. S. Raichle, M. E. (2002). Integration of emotion and cognition in the lateral
prefrontal cortex. Proceedings of the National Academy of Sciences USA, 99, 4115 – 4120.
Kahneman, D. (2011). Thinking Fast and Slow. Farrar, Straus and Giroux, New York.
Kahneman, D. and Tversky, A. (1979) ‘Prospect theory: an analysis of decision under risk’,
Econometrica, 47:263–91.
Krawczyk, D.C. (2002) Contributions of the prefrontal cortex to the neural basis of human decision
making. Neurosci Biobehav Rev. 26(6):631-64.
Liljenström, H. (1991) Modeling the dynamics of olfactory cortex using simplified network units
and realistic architecture. International Journal of Neural Systems, Yol. 2, Nos. I & 2 l- 15.
Roesch,M. Schoenbaum,G. (2006) From associations to expectancies: orbitofrontal cortex as
gateway between the limbic system and representational memory. In: Zald,D. (Ed.)
Rauch,S.(Ed.) The Orbitofrontal Cortex. Oxford University Press ISBN: 9780198565741.
Whalen, P.J. and Phelps, E.A. (Eds.) (2009) The Human Amygdala. The Guilford Press. ISBN-13:
978-1606230336.
Winstanley, C.A. Theobald, D. E. H. Cardinal, R.N. Robbins,T.W (2004) Contrasting Roles of
Basolateral Amygdala and Orbitofrontal Cortex in Impulsive Choice The Journal of
Neuroscience, 24(20): 4718-4722.
7