Why Not Try Harder? Computational Approach To Motivation Deficits in Neuro-Psychiatric Diseases

doi:10.
1093/brain/awx278 BRAIN 2017: Page 1 of 22 | 1
REVIEW ARTICLE
Why not try harder? Computational approach
to motivation deficits in neuro-psychiatric
diseases
Mathias Pessiglione,1,2 Fabien Vinckier,1,2,3 Sébastien Bouret,1,2 Jean Daunizeau1,2 and
Raphaël Le Bouc1,2,4
Motivation deficits, such as apathy, are pervasive in both neurological and psychiatric diseases. Even when they are not the core
symptom, they reduce quality of life, compromise functional outcome and increase the burden for caregivers. They are currently
assessed with clinical scales that do not give any mechanistic insight susceptible to guide therapeutic intervention. Here, we present
another approach that consists of phenotyping the behaviour of patients in motivation tests, using computational models. These
formal models impose a precise and operational definition of motivation that is embedded in decision theory. Motivation can be
defined as the function that orients and activates the behaviour according to two attributes: a content (the goal) and a quantity (the
goal value). Decision theory offers a way to quantify motivation, as the cost that patients would accept to endure in order to get
the benefit of achieving their goal. We then review basic and clinical studies that have investigated the trade-off between the
expected cost entailed by potential actions and the expected benefit associated with potential rewards. These studies have shown
that the trade-off between effort and reward involves specific cortical, subcortical and neuromodulatory systems, such that it may
be shifted in particular clinical conditions, and reinstated by appropriate treatments. Finally, we emphasize the promises of
computational phenotyping for clinical purposes. Ideally, there would be a one-to-one mapping between specific neural compo-
nents and distinct computational variables and processes of the decision model. Thus, fitting computational models to patients’
behaviour would allow inferring of the dysfunctional mechanism in both cognitive terms (e.g. hyposensitivity to reward) and neural
terms (e.g. lack of dopamine). This computational approach may therefore not only give insight into the motivation deficit but also
help personalize treatment.
1 Motivation, Brain and Behaviour (MBB) Lab, Institut du Cerveau et de la Moelle (ICM), Hôpital de la Pitié-Salpêtrière, Paris,
France
2 Inserm U1127, CNRS U9225, Université Pierre et Marie Curie (UPMC – Paris 6), France
3 Service de Psychiatrie, Centre Hospitalier Sainte-Anne, Université Paris Descartes, Paris, France
4 Urgences cérébro-vasculaires, Hôpital de la Pitié-Salpêtrière, Université Pierre et Marie Curie, Paris, France
Correspondence to: Mathias Pessiglione
Motivation, Brain and Behaviour (MBB) Lab, Institut du Cerveau et de la Moelle (ICM), Hôpital de la Pitié-Salpêtrière, Paris, France
E-mail: mathias.pessiglione@gmail.com
Correspondence may also be addressed to: Raphaël Le Bouc

E-mail: raphael.lebouc@aphp.fr
Keywords: apathy; computational psychiatry; goal-directed behaviour; behavioural neurology; decision-making

Abbreviations: dACC = dorsal anterior cingulate cortex; vmPFC = ventromedial prefrontal cortex
Received February 10, 2017. Revised July 14, 2017. Accepted August 30, 2017.
ß The Author (2017). Published by Oxford University Press on behalf of the Guarantors of Brain. All rights reserved.
For Permissions, please email: journals.permissions@oup.com
Downloaded from https://academic.oup.com/brain/advance-article-abstract/doi/10.1093/brain/awx278/4675073

by Inserm/Disc user
on 04 December 2017
2 | BRAIN 2017: Page 2 of 22 M. Pessiglione et al.
This means estimating the values that the patient assigns to

Introduction standard goals that people may have in general, such as
In this review, we present testing and analytic tools that getting a good job position. It is usually done using ques-
may provide better quantification and discrimination of tionnaires with multiple answers that can be scored to quan-
motivation deficits. These tools are embedded in the con- tify the motivation deficit, i.e. apathy (Levy and Dubois,
ceptual framework of decision theory. We thus start by 2006; Drijgers et al., 2010). Different apathy rating scales
sketching briefly the conceptual framework, then we de- have been proposed that vary in the richness of clinical de-
scribe the principles of behavioural tests and computational tails and consequently in the duration of assessment (Marin,
models that can be used to assess motivation, and finally 1990; Starkstein et al., 1992; Robert et al., 2002; Sockeel
we expose the potential neural bases of motivational pro- et al., 2006; Radakovic and Abrahams, 2014). These scales
cesses and the typical manifestations of motivation deficits obviously help the patients to express their trouble with mo-
in the clinics. Note that our purpose is not to advocate a tivation. For this purpose, questions use words from
particular model but to present the interests of the compu- common language, such as ‘Do you have motivation?’
tational approach in general. (Question 7 in Starkstein’s apathy scale) (Table 1).
Although they help clinicians to get a rough idea of the
motivation deficit, these scales suffer from some limitations.
First, they depend on the patient’s insight, which may not
Conceptual framework be fine-grained in the case of diseases that affect cognitive
The etymology suggests that the term motivation originally functions, or on the insight of relatives and caregivers, who
refers to a force that sets the behaviour in motion. Yet can be absent or too distant from the patient to give valu-
when we say that someone is strongly motivated, we able information. Second, they assess entities (either motiv-
seem to imply that motivation is something that we can ation as a whole or subcategories such as ‘interests’ or
quantify in order to predict the behaviour. And when we ‘concerns’) that are not likely to have direct counterparts
ask about real motivations behind observed behaviours, we at the neural level. To overcome these limitations, we sug-
look for reasons expressed in terms of implicit goals. Thus, gest the following 2-fold alternative strategy: (i) adding be-
motivation can be construed as a concept with two attri- havioural tests to questionnaires; and (ii) characterizing
butes, content and quantity, that somehow determine the behavioural performance using a normative framework—
behaviour. The effects of motivation are the direction of namely, decision theory (Fig. 1).
behaviour, which is determined by the content (i.e. the
goal), and the intensity of behaviour, which is determined
by the quantity (i.e. the goal value).
The promises of decision theory
Elementary principles of decision theory assume that when
considering whether or not to take a course of action,
The issue with clinical scales agents contrast costs and benefits to get a net value. If
While neuroscientists investigate the processes through the action is compared to doing nothing, then it is engaged
which the brain selects and implements goals such that only when its net value is positive. If it is compared to a set
they can drive the behaviour, clinicians are mostly concerned of alternative actions, then it is engaged only when its net
with assessing the intensity of motivation in their patients. value is above the others.
Table 1 Starkstein’s apathy scale
1. Are you interested in learning new things? Not at all Slightly Some A lot
2. Does anything interest you? Not at all Slightly Some A lot
3. Are you concerned about your condition? Not at all Slightly Some A lot
4. Do you put much effort into things? Not at all Slightly Some A lot
5. Are you always looking for something to do? Not at all Slightly Some A lot
6. Do you have plans and goals for the future? Not at all Slightly Some A lot
7. Do you have motivation? Not at all Slightly Some A lot
8. Do you have the energy for daily activities? Not at all Slightly Some A lot
9. Does someone have to tell you what to do each day? Not at all Slightly Some A lot
10. Are you indifferent to things? Not at all Slightly Some A lot
11. Are you unconcerned with many things? Not at all Slightly Some A lot
12. Do you need a push to get started on things? Not at all Slightly Some A lot
13. Are you neither happy nor sad, just in between? Not at all Slightly Some A lot
14. Would you consider yourself apathetic? Not at all Slightly Some A lot
Each answer is scored between 0 and 3, with the highest score corresponding to lowest motivation (leftmost answer for Questions 1–8, rightmost answer for Questions 9-14). Based
on the bimodal distribution observed in patients with Parkinson’s disease, a score superior to 14 has been suggested as a marker of apathy.

by Inserm/Disc user
on 04 December 2017
Computational approach to motivation deficits BRAIN 2017: Page 3 of 22 | 3
Figure 1 A schematic view of motivation. The box-and-arrow schema illustrates goal-directed behaviour. The brain adjusts the direction
and intensity of behaviour so as to reduce the delay or increase the probability of goal attainment. Decision theoretic principles posit that agents
should maximize the net value, obtained by subtracting costs (effort and time involved by the behaviour) from benefits (how much rewarding
overcomes punishing aspects of the goal). To perform this optimization the brain needs an approximate anticipation of both costs and benefits. In
this framework, motivation can have three different meanings: motivation-as-content refers to the goal, motivation-as-quantity refers to the goal
value, motivation-as-process refers to behavioural adjustments toward the goal.
sorts: time and effort. Time is a cost because if time is

Net ValueðActioni Þ ¼ Benefit½GoalðActioni Þ Cost ðActioni Þ
spent on a given course of action it is no longer available
ð1Þ
for other actions that might be profitable. Of course, this
opportunity cost of time only matters for actions that cannot
The benefit term corresponds to the value of the action be executed simultaneously. Effort is a cost because effortful
goal, i.e. how good it is for the agent to engage this action. actions consume resources that might be needed later and
Importantly, values may be attached to actions in two dif- will have to be restored through some other costly actions.
ferent ways. On the one hand, the action may be inherently This is obvious in the case of physical effort, which con-
valuable, like when reading a book because it is fun. In this sumes energetic resources and fatigues the muscles such
case the goal is the action itself, and the motivation is said to that they may not operate efficiently for later purposes. It
be intrinsic. On the other hand, the action may be good is not that obvious in the case of mental effort, for which the
because it leads to a valuable outcome, like when reading idea of a biological resource being consumed is still debated
a book is required to pass an examination. In this case the (Inzlicht et al., 2014; Botvinick and Braver, 2015). Indeed
goal is the exam success, the motivation is extrinsic, and the most of, if not all, the energy consumed by the brain is used
behaviour qualified as instrumental. There is therefore no for maintaining spontaneous activity, i.e. activity that is not
major conceptual difference between extrinsic and intrinsic related to a particular cognitive task (Raichle and Gusnard,
motivation from the perspective of decision theory. Note 2005). This has led some authors to argue that people avoid
that the goal can be multidimensional and include both posi- mental effort because of the opportunity cost induced by
tive and negative elements, which may be called gains and cognitive resource allocation. The idea is that when we
losses or rewards and punishments. For instance, being the engage central cognitive modules in a given task, they are
captain of a team can be set as a goal because it has high no longer available for other possible beneficial tasks
positive value on the dimensions of power and self-esteem, (Kurzban et al., 2013; Boureau et al., 2015). In any case,
which may overcome the negative values related to duties people tend to avoid mental effort (Kool et al., 2010;
and responsibilities. Naturally, a given anticipated state of Westbrook et al., 2013), so we must assume that it does
the world can only be set as a goal if the positive values (of entail a cost, which still needs to be specified at the biolo-
rewards) exceed the negative values (of punishments). gical and/or functional level.
The action does not necessarily lead to the goal, i.e. the Action costs can be taken as measures of motivation in-
valuable state anticipated by the agent, in a direct and de- tensity. This is because the action is only engaged if the net
terministic way. For the benefit to be positive, actions only value is positive. Therefore, the highest cost that a person is
need to bring the agent closer to the goal, by decreasing willing to accept, in order to reach a particular goal, is
either the uncertainty or the delay attached to goal reach- equal to the goal value. In principle, motivation could be
ing. In fact, subjective estimates show that people indeed assessed by measuring the time that the person would be
discount the goal value by both delay and uncertainty. willing to invest to reach the goal. Yet in practice, re-
There is a huge literature on delay and uncertainty dis- searchers have focused on effort when developing behav-
counting, the latter being also linked to the notion of ioural paradigms to assess motivation.
risk, defined as the variance of possible outcomes
(Frederick et al., 2002; Green and Myerson, 2004).
Critically, however, delay and uncertainty are not action
costs: they are only modulators of goal value.
Behavioural tests
Action costs are induced by the allocation of resources By definition, it is impossible to vary intrinsic motivation
required for performing the action. They can be of two and assess the behavioural consequences without changing

by Inserm/Disc user
on 04 December 2017
the task to be performed, which may introduce a problem- Table 2 A synthetic view of motivational tests used in
atic confound. It is much simpler to manipulate motivation humans
extrinsically, by varying the outcome associated with a
Type of . . . Main options
given task or action, which can require more or less
effort. The possibility that effortful actions trigger intrinsic . . . task Selection within continuous range (Schmidt et al.,
motivation might be an issue in theory, but in practice 2008)
empirical studies have shown that animals and persons Binary choice (Treadway et al., 2012)
tend to follow the law of least effort (Hull, 1943; Zipf, Willingness to accept (Chong et al., 2015)
1949). . . . effort Physical effort:
Experimental paradigms have been originally developed to power grip peak (Pessiglione et al., 2007) or
test animals, mostly rodents, and were later adapted to duration (Meyniel et al., 2013)
humans. Two categories may be distinguished, depending number of button presses (Treadway et al.,
on whether a choice is explicitly implemented or not. 2009)
Interestingly, the processes targeted by the two sorts of Mental effort:
tasks have been named differently, with an emphasis either N-back (Kool et al., 2010)
on effort or reward (e.g. ‘effort-based decision’ and ‘incen- numerical Stroop (Schmidt et al., 2012)
tive motivation’), although paradigms invariably involve the
. . . reward Real reward such as money (Le Bouc and
manipulation of both reward and effort levels (Table 2).
Pessiglione, 2013)
Virtual reward such as apples (Bonnelle et al.,
Binary choice 2015)
In effort-based decision tasks, subjects have a choice between Subliminal reward (real or virtual money)
two options: producing little effort for small reward, or pro- (Schmidt et al., 2010)
ducing a higher effort for a bigger reward. This has been . . . model Hyperbolic or exponential discounting (Prevost
operationalized in rodents, for instance with a T-maze (Fig. et al., 2010)
2) where a big reward (more food pellets) is placed in one Parabolic (Hartmann et al., 2013) or sigmoidal
arm behind a barrier, whereas the barrier-free arm only leads discounting (Klein-Flugge et al., 2015)
to a small reward (Salamone et al., 2007). Another example Subtraction of supralinear cost function
would be an operant box with two levers, one requiring more (Le Bouc et al., 2016)
presses but providing more food pellets than the other
(Walton et al., 2006). The number of food pellets (benefit) The table lists the main options (for the type of task, effort, reward, and model) that
and the height of the barrier or the number of lever presses have been implemented in behavioural paradigms designed to assess motivation in
human participants. References suggest one paper in which the corresponding option
(cost) can be varied so as to determine the cost that the has been implemented. The advantages and drawbacks of the different options are
animal is willing to accept in order to get one food pellet, explained in the text. Note that the list is non-exhaustive and that references are
which gives a measure for the subjective reward value. somewhat arbitrary.
Equivalent paradigms have been developed for human

subjects, with effort being manipulated in terms of, for ex- and second they prevent the issue of satiety, which may
ample, the force to be exerted on a handgrip (Fig. 2) or the change the reward value during the course of the experi-
number of clicks on a mouse (Treadway et al., 2009; ment. As famously argued by economists, marginal utility
Prevost et al., 2010). Compared to all the other measures, might be a decreasing function of monetary amount, mean-
the grip task has the advantage of isolating effort from ing that a same objective gain would appears as less valuable
delay. This is because in standard designs, participants (subjectively), once significant money has been accumulated.
are instructed to produce short pulses, such that the However, this effect likely remains limited given the small
effort cost (linked to peak force) can vary while keeping incentive ranges spanned in standard experiments.
the duration roughly constant. This prevents situations in
which enhancing effort postpones the reward, and thus in-
creases temporal discounting of reward. Another potential
Willingness to accept
confound is risk, if participants are uncertain about their In incentive motivation (or instrumental behaviour activa-
ability to produce the required effort. This is generally tion) tasks, there is no explicit choice in the sense that the
avoided by restricting the range of forces to what partici- alternative options are not explicitly laid down. For ex-
pants can achieve with a 100% chance of success. ample, in paradigms using progressive ratio schedules, the
The rewards used in human studies are generally more number of lever presses required for a given number of
abstract than in animals: typically money or even points food pellets is increased from one trial block to the next,
(tokens). They are generally thought to involve the same until the animal ceases responding. As seen before, the
brain system as primary rewards, as implied by the notion number of lever presses at break point can be taken as a
of ‘common neural currency’ (Levy and Glimcher, 2012). direct measure of the motivation induced by the food
Their advantage is 2-fold: first they are easy to quantify reward (i.e. the incentive). This sort of task can also be

by Inserm/Disc user
on 04 December 2017
animal experiments, which may stop performing the task

at any time, for instance by running away or by breaking
fixation, such that these ‘errors’ can be interpreted as moti-
vated choices (Minamimoto et al., 2012).
This sort of choices, with a single action that may be
performed or not, can be made more explicit in humans,
in the form of ‘accept’ versus ‘decline’ alternatives. Such
tasks measure the willingness to accept performing a
unique type of action for various combinations of effort
and reward levels (Bonnelle et al., 2015). It could be
argued that such yes/no decisions are more natural for
the brain, because in the ancestral environment in which
it has evolved (before the emergence of stores and restaur-
ants), options do not appear simultaneously (Stephens and
Krebs, 1986). When a new option arises, the agent is likely
to be already pursuing a goal, such that the decision is
whether to keep with the current course of action or to
Figure 2 Behavioural tasks assessing motivation as a switch to the new alternative.
reward/effort trade-off. Motivation intensity can be quantified as
the amount of effort that the agent is willing to expend in order to
obtain a potential reward. This trade-off between effort and reward
Selection within a continuous range
has been operationalized in binary choice tasks (left) or free operant Another paradigm where choices are implicit are free oper-
tasks (right). In animals (top), binary choice is typically implemented ant tasks using fixed ratio schedules, meaning that the
in a T-maze, with one branch representing the small reward/low number of food pellets is proportional to the number of
effort option and the other branch the bigger reward (more food)/ lever presses (Fig. 2). From a decision-theoretic perspective,
higher effort (due to the barrier) option. Free operant behaviour is
animals are viewed as making decisions between pressing
classically assessed in a Skinner box where pressing the lever trig-
and not pressing at every time step, or choosing the time
gers food delivery. When the ratio between the number of lever
presses and the number of food pellets is fixed, the reward obtained interval between presses, which may give a continuous set
is simply proportional to the effort produced, such that the reward/ of numerous hidden options, depending on the resolution
effort levels are freely selected within a continuum. Note that the of time estimates and the maximal interval allowed (Niyogi
same sort of apparatus can be used to implement binary choice, et al., 2014). In a human version of this free operant para-
with two levers associated with different ratios between effort and digm (Fig. 2), participants are first shown a given incentive
reward levels (not shown). Thus, binary choice can be envisaged as a level (amount of money) and then asked to squeeze a hand-
special case of free operant behaviour, for which a large range of grip, knowing that payoff will be proportional to peak
options (and not just two) are available. This can also be seen in the force (Schmidt et al., 2008). More precisely, the payoff is
behavioural tests adapted to human participants (bottom), where
calculated as the fraction of the incentive corresponding to
effort is produced on a power grip and reward is given in monetary
the proportion of maximal force that subjects produce,
units. The figure shows one key screenshot of the visual animation
presented to participants in a task trial. The vertical orange bar
such that they get half the incentive if they produce half
provides a visual feedback on the force produced, within a grid that their maximal force. Here again, the task (sometimes called
is scaled to the participant’s maximal force. In the free operant incentive motivation task) can be viewed as a continuous
version (sometimes called incentive motivation task), all grip forces version of the binary choice task (sometimes called effort
between zero and maximal force may be selected, for a proportional discounting task), where the set of alternative options in-
reward that ranges from zero to full incentive (E1 in the illustrated cludes all possible forces between zero and maximal force.
example). In the binary choice version (sometimes called effort An interesting variant of this paradigm can be obtained
discounting task), only two reward-effort combinations are available by indexing the payoff not on peak force but on effort
(force levels are indicated by orange horizontal targets and reward duration (Meyniel et al., 2013). In this case, a target
levels by coin images). This behavioural paradigm can also be
force level is imposed and payoff accumulates, at a speed
adapted to measure directly the willingness-to-accept a given effort
proportional to incentive level, as long as the force being
for a given reward. In this case (not shown), a single option is dis-
played on the screen, as a target/coin association that the participant produced by the participant is above the target. The dur-
chooses to accept or decline. ation of the trial (typically 30 s) is too long for the force to
be maintained throughout the end, so subjects have to re-
lease the grip at some point, take a break to recover from
fatigue and then start squeezing again to get more money.
interpreted in the framework of decision theory, as there is Effort production in this task can be interpreted as resulting
an implicit decision to make, between pressing the lever or from decisions about the durations of effort and rest peri-
not. In other words, there is a hidden option that is just ods, which in principle can take any value from zero to
doing nothing. In fact this option is always present in trial length.

by Inserm/Disc user
on 04 December 2017
Specificities of testing humans presented (Pessiglione et al., 2007). This effect was none-
theless tiny compared to that observed in supraliminal con-
Testing humans instead of animals present a number of ditions, where strategic adaptations (saving resources for
advantages. One is that verbal instructions, associated when they matter) are likely to have an impact. In both
with some familiarization with the task, may permit to in- cases (subliminal and supraliminal), the incentive motiv-
vestigate steady performance (i.e. with limited learning ef- ation effect was associated with an increase in skin con-
fects), without overtraining the subjects, which might make ductance response, a measure of autonomic activity.
performance more like a habit. This is important because It is not entirely clear whether subliminal motivation ef-
producing habitual behaviour has been shown to involve fects correspond to a (subconscious) instrumental adjust-
brain circuits different from those underpinning goal-dir- ment of behaviour, or to a more basic appetitive reflex,
ected behaviour (O’Doherty, 2016). or even to emotional arousal. Indeed, when presenting
A second advantage is the possibility to use virtual emotional pictures incidentally and orthogonally to incen-
reward and effort. Indeed, consequential choices, where tive levels (Schmidt et al., 2009), emotional arousal was
subjects have to perform the effort and truly receive the found to increase effort production in a manner that was
reward, can be reduced to a random fraction or even sup- independent from motivational effects (no interaction).
pressed. This opens the possibility of presenting on every Note however that, although appetitive and emotional re-
trial new items that resemble the sort of effort and reward actions to incentives may affect effort production and
we encounter in everyday life (an offer could be, for in- related decisions in many paradigms, their effect size
stance: would you clean up your room if I get you an ice seems limited in comparison to instrumental effects.
cream?). Previous studies have shown that using virtual
compared to real reward make little difference on effort
production or intertemporal choice (Bickel et al., 2009;
Schmidt et al., 2010). Yet it might not be neutral for deci-
Computational modelling
sions to propose effort items that are never actually imple- Even in simple behavioural readouts of motivation, several
mented during the task. Some patients might have issues factors might play a role. For instance, a shift in the pro-
with anticipated costs, on which a priori decisions are pensity to favour high reward–high effort options could be
based, and others with experienced costs, which would in- due to an increased sensitivity to reward or a decreased
fluence decisions posterior to effort production. sensitivity to effort. Computational models may be helpful
Manipulating anticipated versus experienced costs has for disentangling between such alternative explanations of
indeed been shown to result in distinct behavioural patterns behavioural changes.
(Meyniel et al., 2014).
A third advantage is the accessibility of mental effort,
which may be much more amenable to experimental inves-
Modelling the cost/benefit trade-off
tigation. Mental effort generally means attentional effort Computational models are algorithms that perform the task
and has been manipulated by varying the difficulty of ex- imposed to subjects, meaning that they generate behav-
ecutive tasks such as Stroop, N-back, or task-switching. ioural outputs through a limited series of mathematical op-
Just as physical effort, mental effort can be implemented erations applied to the experimental factors. Decision
in decision-making or incentive motivation tasks, i.e. with theory provides a generic and normative framework for
explicit or implicit choices. In decision tasks, it has been deriving the computational models that can be used to
well-established that healthy subjects prefer to avoid diffi- simulate the behaviour in specific motivation tests. If in
cult executive tasks (e.g. 3-back compared to 1-back), and these experimental tests, goals can be reduced to the re-
only accept to perform them if the expected reward is sig- wards that actions provide, and if actions can be reduced
nificantly bigger than the one associated with an easy ver- to the amount of effort that they involve, then Equation (1)
sion of the same task (Kool et al., 2010; Westbrook et al., can be simplified to:
2013; Apps et al., 2015). However, incentive effects on
VðEi Þ ¼ RðEi Þ CðEi Þ ð2Þ
performance in tasks involving attentional effort appeared
to be weaker than those observed with physical effort with V(Ei) being the net value of producing effort Ei, R(Ei)
(Schmidt et al., 2012). This might mean that mental the reward associated to effort Ei and C(Ei) the cost of
effort is more difficult to adjust, or less costly compared producing effort Ei. The contingencies between reward
to physical effort. and effort are directly specified by the task, for example
Finally, testing humans enables the assessment of possible E10 can be associated with reaching 80% of maximal grip
dissociations between conscious and subconscious effects of force. With money, reward level is an objective amount
incentive levels. This can be done by masking the cues (such as E10) that can be directly entered in the equation,
indicating incentive levels to subjects, using standard sub- if we ignore distortions such as decreasing marginal utility.
liminal presentation procedures. It has been demonstrated The main difficulty in establishing such models is to specify
that subjects produce more effort in the grip task for higher the cost of the required effort (80% of maximal force in the
incentives, even if they cannot report which incentive was example).

by Inserm/Disc user
on 04 December 2017
Pioneering investigations of how rewards are subjectively force window is hit or missed. In this case, the benefit
devalued by effort were inspired by delay discounting models, term could be formalized as krR.P(Fi), with P(Fi) being
which account for how rewards are subjectively devalued by the probability of hitting the target given the considered
delay of delivery (Green and Myerson, 2004; Prevost et al., peak force and some motor noise. Similar formulations
2010). Non-linear functions such as exponential or hyper- can be applied to situations where the outcome is not a
bolic decay with time are the mathematical forms most com- potential reward but a potential punishment such as mon-
monly used for delay discounting. Note that these functions etary loss. The benefit term also includes a constant, which
can only give positive values. This is problematic in the case may account for the fact that producing more force has
of effort because it would mean that agents always prefer positive outcomes other than money, for instance it could
producing the effort (even climbing a mountain for a make an impression on the experimenter. This is consistent
peanut) than doing nothing. As this would be absurd, we with the repeated observation that subjects squeeze the grip
focused in this review on a subtractive form of the value in this task even for negligible monetary incentives
function, following on recent formalizations (Manohar (Schmidt et al., 2008; Le Bouc et al., 2016).
et al., 2015; Le Bouc et al., 2016). This is not anecdotal, The cost term includes an explosive cost function, kc.Fi/
since it opens the possibility of attaching negative values to (1-Fi), modulated by a fatigue function, (1 + kf.T). The cost
the production of efforts that are not rewarded enough. function is a simple approximation of the equation derived
Discounting can nevertheless be non-linear with this subtract- from motor control theory (Rigoux and Guigon, 2012;
ive form, depending on the shape of the cost function. Lebouc et al., 2016). Note that if F is expressed as a pro-
Cost functions have been a matter of debate in recent portion of maximal force, then cost is null when no force is
years. In the case of physical effort, a consensus seems to produced, and infinite when force approaches the theoret-
emerge around the notion that discounting should be con- ical maximum. This maximum may correspond to what
cave, as in parabolic functions (Hartmann et al., 2013), or can be expected at best from the musculature of the arm.
at least initially concave, as in sigmoidal functions (Klein- In the case of handgrip, previous studies (Forbes et al.,
Flugge et al., 2015). Interestingly, classical formalization in 1988; Hsu et al., 1993; Neu et al., 2002) have shown
motor control theory, where action cost is typically defined that maximal force can be approximated from simple meas-
as the integral of squared motor command, also predicts ures of the forearm length, circumference and skin width (a
that cost should be a supralinear function of the objective proxy for non-muscular tissues). Fatigue is modelled as a
muscle contraction (Rigoux and Guigon, 2012). These stu- linear increase with the number of trials achieved so far, for
dies therefore converge to the conclusion that physical the sake of simplicity. This fatigue function captures the
effort cost increases faster than a motor output such as phenomenon that a same force is more and more costly
grip force. The cost function might be different in the to produce as fatigue progressively kicks in. More complex
case of mental effort (Westbrook et al., 2013; Bonnelle functions of trial index could be envisaged, and cumulative
et al., 2015), for which the essence of effort cost remains effort over trials could be a better proxy than mere trial
controversial. index. We also note that a symmetrical function could be
As an illustration, we describe below a net value function included in the benefit term to account for the phenomenon
that could be integrated in a computational model to ac- of satiety, according to which incentive effects would di-
count for the behaviour in the incentive motivation test, minish with trial index or cumulative reward.
where the payoff is proportional to the reward at stake We do not imply that the simple formulation adopted in
and to the force produced: Equation (3) is the unique possibility or that it provides the
VðFi Þ ¼ ð1 þ kr :RÞ:Fi kc :ð1 þ kf :TÞ:Fi =ð1 Fi Þ ð3Þ best account of motivational processes. We chose this illus-
tration because it respects the fundamental principles of
with Fi being the considered peak force (in proportion to decision theory and because it is sufficient to understand
maximal force), R the potential reward (incentive level) and the model fitting approach and hence the interests of com-
T the trial index. Note that Fi is the controlled variable that putational phenotyping.
the agent must set (in order to maximize the net value),
while R and T are experimental factors imposed by the task
design. Simulations of costs and benefits generated by this
Fitting model free parameters
value function, and the associated behavioural patterns, are The net value function suggested in Equation (3) follows on
shown in Fig. 3. a normative principle, in the sense that it specifies what
The benefit term, (1 + kr.R).Fi, accounts for the contin- subjects should do in order to maximize benefits and min-
gency imposed by experimental design, according to imize costs. However, it does not discard the possibility
which exerting more force linearly increases the monetary that individuals may vary in their subjective attitude to-
payoff. Such contingency might be present in many real-life wards various dimensions such as reward, effort, fatigue
situations where exerting more effort would enhance the etc. This subjectivity is enabled by the constants (the k
probability or reduce the delay of reward delivery. Other parameters), which account for variations in how agents
contingencies could be envisaged, for instance an all-or- weight the various factors that impact cost and benefit
none payoff depending on whether a target such as a terms.

by Inserm/Disc user
on 04 December 2017
Figure 3 Simulation of a motivation model. Simulation results were obtained with a model that can be approximated by Equation (3). It
was applied to the incentive motivation task that is illustrated in Fig. 2 (bottom right) and used in a recent publication (Le Bouc et al., 2016), with six
reward levels randomly distributed over 60 trials. (A) Simulated hidden variables. In a given trial, the model anticipates for each possible force
peak (a proxy for effort level), the associated benefit (left) and the associated cost (right). The expected benefit is proportional to the force peak,
with a slope that depends on the reward level offered in the present trial, i.e. the payoff corresponding to maximal force production (only four
levels are illustrated). The expected cost is a supralinear function of force peak, with a slope that depends on fatigue level, i.e. the number of trials
completed so far (two fatigue levels are illustrated, with the red line and red dots showing the expected cost at the beginning and at the end of the
task, respectively). The net value is obtained by subtracting costs from benefits. The predicted behaviour is the optimal force peak (for which the
net value is maximal), as illustrated by the green arrows. Note that effort expenditure (predicted force peak) increases with reward level,
reproducing the incentive motivation effect. (B) Simulated behavioural patterns. Top and bottom panels show how the predicted behaviour varies
with the two task factors: reward level and trial index. The different columns show how the predicted behaviour varies when changing one single
free parameter (grey versus black lines). While changing the sensitivity to effort cost (Kc) globally shifts effort production up or down, changing
the sensitivity to reward (Kr) affects the impact of reward level, and changing the sensitivity to fatigue (Kf) affects the impact of trial index.
Constants in computational models are viewed as free design, such as a stair-case procedure, that increments
parameters because they can be adjusted to fit (as closely effort and/or rewards levels based on observed choices
as possible) the behaviour exhibited by a given subject in a (Westbrook et al., 2013; Klein-Flugge et al., 2015; Le
given test. Value functions can be fitted directly to behav- Bouc et al., 2016).
ioural measures in choice paradigms that allow inferring Another approach is to integrate in the model a function
indifference points, i.e. reward–effort combinations that that generates the behaviour, in accordance to a value
are selected in 50% of trials when confronted two-by- maximization principle. In the incentive motivation test,
two. The fitting procedure in this case consists of searching where any force can be chosen between zero and max-
the set of free parameters that yield a theoretical value imum, the predicted behaviour is simply the peak force
function whose distance from indifference points is min- that gives the maximal net value (for which the value de-
imal, as classically implemented in least squares methods. rivative is null): F* = argmax V(F). If noise is incorporated
Indifference points can be determined through an adaptive in the model, then a probability (or likelihood) can be

by Inserm/Disc user
on 04 December 2017
assigned to every observable peak force. Here, model fitting obtained from participants through likeability or desirabil-
consists of searching for the set of free parameters that ity ratings on analogue scales. Note, however, that decision
maximize the log likelihood of the peak force series pro- theory is agnostic about whether values have an affective
duced across trials. This is implemented in a variety of component or not. The expected value (at the time of
optimization algorithms, from basic grid search methods choice) and the experienced value (at the time of consump-
to Bayesian model inversion techniques. tion) do not necessarily match what could be subjectively
Similar computational modelling and fitting approaches felt as desire (or dread) and pleasure (or pain). What is
can be applied to binary choice, which can be seen as a important for natural selection is that behavioural policies
special case of the incentive motivation problem, where the maximize reproductive success. Desire and pleasure might
option set is reduced to just one pair. In this case, the two be just proxies for how good a given state is, i.e. how much
option values, which can be calculated through Equation (3), it favours eventual reproduction. They might partially
are generally entered into a softmax function, which gives the depart from the values that actually guide the behaviour,
likelihood of the observed choice: P(Fc) = 1/(1 + exp(b.(V(Fu)- through the maximization mechanisms that the brain has
V(Fc)))), with Fc and Fu being the chosen and unchosen evolved and which may be modelled by decision theory.
forces, respectively and b a free parameter termed ‘inverse Similar issues may arise when autonomic data, such as
temperature’ that captures the noise in the choice process. pupil diameter or skin conductance, are used to fit compu-
The shape of the softmax function is a sigmoid that gives a tational models, as was done in passive paradigms, i.e. in
probability of 0.5 when the two option values are equal, and the absence of behavioural data (Petrovic et al., 2008). The
converges to 1 or 0 when one option gets much better than question is to what extent autonomic responses represent
the other. Fitting the parameters of the net value function valid proxies for the values that guide the behaviour. It has
thus reduces to (non-linear) logistic regression, which is clas- been shown that skin conductance responses were corre-
sically employed to account for binary data. lated with the magnitude or probability of both reward
Eventually, adjusted parameters, or parameter estimates,
and punishment (LaBar et al., 1998; Critchley et al.,
obtained by fitting the model to the data, characterize the
2001; Delgado et al., 2006; Schmidt et al., 2009;
subject’s behaviour in a testing session. They provide a
Gergelyfi et al., 2015). Thus, skin conductance could be
computational phenotype that quantifies individual atti-
used as a proxy for the absolute value of reward or pun-
tudes such as sensitivity to reward attraction (kr), effort
ishment separately, but would be off for actions that com-
cost (kc), fatigue effect (kf), etc. These models are therefore
bine positive and negative outcomes. Besides, it remains
not purely normative, in the sense that they do not deter-
unknown whether skin conductance, which reflects arousal
mine what should be done once and for all: they instead
levels, strictly matches the value estimates that guide deci-
take into account interindividual differences. This does not
sions. Along the same lines, pupil dilation is generally con-
imply that they represent trait measures, as a same subject
sidered as a marker of effort, both physical and mental
in different states might produce different behaviours,
(Kahneman and Wright, 1971; Beatty and Wagoner,
which would be best explained by different parameters.
1978; Piquado et al., 2010; Alnaes et al., 2014; Zenon
Changes in parameter estimates can therefore be used to
evaluate the impact of a disease, or the effect of a treat- et al., 2014). Thus, pupil size could be taken as a proxy
ment, or even normal fluctuations (in mood for instance). for effort level, although it remains unclear how closely it
Disease and treatment effects can also be assessed in a would follow the effort cost estimate integrated in the net
subtly different way, through Bayesian model comparison, value equation that guides decisions. The alternative is that
which provides a metric to elect the most plausible model pupil size may reflect the amount of effort eventually in-
given the behavioural data. The idea here is to provide vested in the action when it is performed, and not the
qualitative, rather than quantitative, differences. For in- amount of effort that is anticipated during decision-
stance, one may want to compare models in which the making. It should also be stressed that the dissociation
disease affects sensitivity to reward versus sensitivity to between skin conductance and pupil size as reflecting
effort, to discriminate between different forms of apathy outcome value and effort cost is not clear-cut, as both
(see application to Parkinson’s disease in Fig. 4). measures may reflect a common form of autonomic
Although the sort of computational models exposed here arousal.
provide reasonable accounts for optimization processes, Finally, we have restricted our computational presenta-
they are silent about how values are generated. When the tion to the basics of decision theory, leaving aside learning
anticipated outcomes are expressed in objective metrics processes. It is nonetheless expected that learning should
such as a number of pellets or a sum in euros, a slight occur at many levels in the tests used to assess motivation,
subjective distortion might provide a good enough approxi- not only with respect to outcome value but also with re-
mation. But when they come to episodes embedded in gards to sensorimotor contingencies. Learning effects can
social contexts, such as family vacation projects, current be partially controlled through training sessions on the
models fall short of suggesting how values are constructed. task, or captured by relevant learning models. A presenta-
Nevertheless, decision-theoretic models can be used to option of these models would, however, go far beyond the
timize action selection if proxies for subjective values are scope of this review.

by Inserm/Disc user
on 04 December 2017
Figure 4 Computational characterization of motivation deficit. The figure illustrates the computational approach of motivation deficit,
taking the example of dopamine depletion in Parkinson’s disease (adapted from Le Bouc et al., 2016). Behavioural data were acquired in a group of
Parkinson’s disease patients tested ON and OFF dopaminergic medication, as well as in matched healthy controls, using the binary choice and free
operant tasks displayed in Fig. 2. The model fitted to behavioural data can be approximated by Equation (3), and includes the same free parameters
as in the simulations (Fig. 3). (A) Model-free results. Graphs show inter-subject means and standard errors for the force selected in the choice
task, or freely generated in the motivation task, as a function of incentive level (from 0.1 to 5E). The slopes appeared to differ between groups, an
effect that was captured by computational modelling. (B) Comparison of parameter estimates. Graphs show inter-subject means and standard
errors for posterior estimates of free parameters, after model fitting through Bayesian inversion. Parameter estimates were then compared
between groups using t-tests. Only parameter Kr differed significantly between Parkinson’s disease patients and controls, and between ON and
OFF patients. (C) Clinico-computational correlation. The effects of dopaminergic medication on the computational parameter Kr and on the
Starkstein apathy score were correlated across patients. This supports the idea that dopaminergic medication alleviates the motivation deficit by
increasing reward sensitivity (as opposed to decreasing effort cost or susceptibility to fatigue). (D) Bayesian model selection. If we only consider
the three parameters Kr, Kc and Kf, there are 23 = 8 possible models (each parameter can be affected or not by medication). Then for each
parameter we compare the four models in which medication has an effect to the four models in which there is no effect. Exceedance probability
suggests that a medication effect on Kr is highly plausible (compared to no effect), whereas an absence of effect was much more plausible for Kc
and Kf. Thus, model selection leads to the same conclusion as comparing parameters estimates: the effect of dopaminergic medication is a
selective modulation of Kr. PD = Parkinson’s disease.
Neural implementation and reward, typically. We focus on work in primates that

might have a direct translation to clinics, even if this work
In this section, we briefly review studies that investigated was largely inspired by earlier studies in rodents, for which
motivation as a trade-off between cost and benefit—effort we refer readers to a recent review (Salamone et al., 2016).

by Inserm/Disc user
on 04 December 2017
We first explore activation studies that link brain activity to it was an objective value such as monetary amount, or a
reward/effort optimization, in both humans and monkeys. subjective value such as likeability rating. Accordingly,
We then review the behavioural consequences of lesions single-unit activity in the monkey vmPFC was strongly
and pharmacological manipulations in animals. What can associated with stimulus value, integrating subjective as-
be learned from lesions and pharmacological treatments in pects such as factors related to the internal state (satiety)
human clinical conditions is discussed in the next section. or representations stored in memory (Bouret and
Meta-analysis of functional MRI studies (Fig. 5) helps Richmond, 2010; Strait et al., 2014; Abitbol et al.,
with delineating a set of brain regions involved in reward 2015). Single-unit recordings in monkeys have also estab-
and effort processing. We deliberately set a conservative lished correlations (both positive and negative) with stimu-
statistical threshold to focus on key nodes, so we do not lus value in more lateral parts of the orbitofrontal cortex
pretend to provide an exhaustive description. The reward (lOFC) (Padoa-Schioppa and Cai, 2011; Wallis, 2011). In
network includes the orbitofrontal cortex (mostly the contrast, physical effort levels associated to actions were
medial part), the ventral striatum bilaterally and the mid- positively correlated with the dorsal anterior cingulate
brain (around dopaminergic nuclei). The effort network in- cortex (dACC) haemodynamic response, which also corre-
cludes principally the anterior cingulate cortex and bilateral lated negatively with the reward levels associated to action
anterior insula. In the following we briefly consider the outcomes (Burke et al., 2013; Kurniawan et al., 2013;
cortical, subcortical and neuromodulatory components of Skvortsova et al., 2014; Bonnelle et al., 2015; Scholl
these networks. et al., 2015). These results are consistent with a role for
the dACC in integrating costs and benefits and computing
the net value of actions, whereas the role of the vmPFC
Cortical structures might be confined to the outcome space, ignoring action
The medial part of the orbitofrontal cortex (or ventro- costs. Physical effort levels were also found to be reflected
medial prefrontal cortex, vmPFC) has been repeatedly in the anterior insula, together with outcomes bearing
shown to represent reward values in humans (Peters and strong aversive valence such as pain or punishment
Buchel, 2010; Bartra et al., 2013; Clithero and Rangel, (Seymour et al., 2005; Pessiglione et al., 2006; Samanez-
2014). More precisely, vmPFC haemodynamic response Larkin et al., 2008; Prevost et al., 2010; Skvortsova et al.,
was positively correlated with the stimulus value, whether 2014). Interestingly, cognitive effort appeared to recruit the
Figure 5 Meta-analysis of reward/effort neural representation. Statistical maps show with colour-coded z-score the results of functional
MRI meta-analysis performed on the Neurosynth platform. As the number of functional MRI studies employing the keywords ‘reward’ and ‘effort’
was very different, we used reverse and forward inference, respectively. The slices were selected so as to display the most noticeable clusters, and
superimposed on canonical anatomical template. The [x, y, z] coordinates refer to the Montreal Neurological Institute (MNI) space. aI = anterior
insula; vS = ventral striatum. Note that less significant clusters were observed in the posterior and anterior cingulate cortex for reward, and in the
inferior parietal and inferior frontal cortex for effort.

by Inserm/Disc user
on 04 December 2017
same brain areas, notably the dACC (Schouppe et al., not with effort level (Gan et al., 2010; Hollon et al.,
2014; Massar et al., 2015). 2014; Hamid et al., 2016). This is in-line with single cell
The differential involvement of the ACC and vmPFC/ electrophysiological recordings in monkeys that showed a
lOFC in effort representations was also observed in primate relatively limited sensitivity of dopamine neurons to infor-
studies. Indeed single cell activities were more sensitive to mation about upcoming physical effort, compared to ex-
the amount of effort needed to obtain a reward in the ACC pected reward (Pasquereau and Turner, 2013; Varazzani
compared to the lOFC/vmPFC (Shidara and Richmond, et al., 2015). This relative lack of sensitivity to effort by
2002; Kennerley et al., 2009; San-Galli et al., 2016). dopamine neurons contrasts with their strong sensitivity to
Similar conclusions were reached from lesion studies in other forms of cost that relate to reward delivery, such as
rats: lesions of the ACC decreased the willingness to risk or delay (Kobayashi and Schultz, 2008; Lak et al.,
climb a barrier to get the reward (Walton et al., 2003). 2014). These results suggest that dopamine helps to cope
Furthermore, a double dissociation was found between with effortful actions by signalling reward availability,
ACC and OFC lesions, which impaired effort-based and without contributing to the estimation of effort cost. This
delay-based decisions, respectively (Rudebeck et al., is compatible with the energizing effects of dopamine en-
2006). The same dissociation has been observed in hancers that have been observed in humans (Wardle et al.,
humans, with the delay of reward delivery modulating 2011) (see also next section), or the reduced motivation
value representation in the vmPFC, and the effort asso- following acute phenylalanine/tyrosine depletion, which re-
ciated to the action requested to obtain the reward modu- duces dopamine level (Venugopalan et al., 2011).
lating value representation in the dACC (Prevost et al., If not dopamine, other neuromodulators might be
2010). involved in signalling effort cost, such as noradrenaline
and serotonin. Electrophysiological recordings showed that
Subcortical structures when monkeys initiate an action, the firing of neurons in the
locus coeruleus scales positively with the amount of force
In functional MRI studies using incentive motivation tasks, required (Bouret and Richmond, 2015; Varazzani et al.,
the ventral striatum and pallidum were found to activate 2015). This is compatible with the idea that noradrenaline
with the amount of reward at stake and to drive motor release signals the amount of effort required by upcoming
performance (Knutson et al., 2001; Schmidt et al., 2012; actions. However, locus coeruleus neurons displayed virtu-
Kroemer et al., 2014). This activation of the ventral ally no response to information about effort provided by a
striato-pallidum complex was observed in subliminal condi- visual cue, in contrast to what has been reported for ACC
tions, where subjects were unaware of incentive level neurons in similar tasks (Kennerley et al., 2009). One inter-
(Pessiglione et al., 2007), and was dissociated from emo- pretation is that noradrenaline is important at the time of
tional arousal (Schmidt et al., 2009). It was also shown to action, when the effort needs to be produced, but not in the
drive both physical and mental effort, depending on the task earlier decision to make the effort or not. Evidence for a
demand (Schmidt et al., 2012). The ventral striatum and role of serotonin in the processing of effort cost comes from
pallidum are part of the limbic circuit of the basal ganglia, pharmacological manipulation in healthy humans.
which receives projections from the orbitofrontal cortex Increasing serotonin level with antidepressants (SSRIs) pro-
(Brown and Pluck, 2000; Haber, 2003). Nevertheless, longed effort duration in an incentive motivation task, an
their role in incentive motivation seems different from that
effect that was computationally captured by a reduction of
of the orbitofrontal cortex, as they increase their activity
effort cost (Meyniel et al., 2016). This is in agreement with
when subjects produce more effort (to get more reward).
a more general role for serotonin in overcoming costs, as
Similar conclusions have been drawn from electrophysiolo-
was previously shown for the delay subjects have to wait
gical studies in monkeys (Tachibana and Hikosaka, 2012).
before obtaining the reward (Miyazaki et al., 2012; Fonseca
However, when reward and effort levels have to be inte-
et al., 2015). Finally, opioids might also play a role in
grated in order to make a choice (and not to drive perform-
moderating the impact of experienced or anticipated
ance), activity in the striatum appeared to signal the net
effort, which could be seen as a prolongation of their role
value, i.e. reward minus effort (Botvinick et al., 2009;
in attenuating pain experience.
Croxson et al., 2009; Kurniawan et al., 2010).
Another line of research examined the role of dopamine
in the nucleus accumbens, a structure homologous to the A tentative neuro-computational
ventral striatum in rodents (see Salamone et al., 2007 for a
review). Increasing and decreasing dopamine levels shifted
scenario
the choices toward high effort–large reward and low effort– It is tempting to establish links between the net value equa-
small reward options, respectively. This result could be ex- tion that adjusts the reward/effort trade-off and the findings
plained either by dopamine enhancing reward attractive- reviewed in this section. A speculative but reasonable sum-
ness, or by dopamine diminishing effort cost. mary might be that (i) some regions, such as the vmPFC,
Voltammetry studies in rodents suggested that dopamine compute the benefit term; (ii) other regions, such as the
release in the nucleus accumbens scales with reward but dACC, integrate the effort cost and signal the net value;

by Inserm/Disc user
on 04 December 2017
and (iii) neuromodulators adjust the weights of reward and @F @V

effort in the net value estimation. ¼ ð4Þ
@t @F
In Fig. 6, we go further and suggest a possible implemen-
tation of the computational mechanism that finds the opti- where the temporal derivative of force is obtained by multi-
mal behaviour by maximizing the net value equation. For plying the derivative of value with respect to force by a
illustration, we take the case of the incentive force task (Le parameter that controls the rate of convergence of the
Bouc et al., 2016), a subcase of free operant paradigms gradient ascent. By construction, the steady-state
(Fig. 2), to which Equation (3) can be applied. The problem limt!1 FðtÞ of this ordinary differential equation is a local
for the brain is to find the force F* that maximizes the net maximum F* of the value function V(F). Computationally
value function V(F), without explicitly computing the value speaking, Equation (4) requires the moment-to-moment cal-
of all admissible forces. One solution is adjusting force dy- culation of the gradient @V=@F ¼ kr @B=@F kc @C=@F of the
namically so as to operate a gradient ascent on the value value function V with respect to force F. This gradient is
function, as follows: decomposed into two terms: the gradient of benefits and the
Figure 6 A speculative implementation of motivation process in the human brain. The figure illustrates the mechanisms that the
brain might implement to generate behaviour in a typical incentive motivation task, where one behavioural output must be selected within a
continuous range. Under decision theory, the behavioural output must maximize the benefit B and minimize the cost C. In this example (see
incentive force task in Le Bouc et al., 2016), the behavioural output is a force F, the benefit is a fraction of the reward R corresponding to F
(relative to maximal force Fmax), and the cost is a supralinear function of F. The problem for the brain is therefore to find the force F that
maximizes the net value (V = B – C) (see explanation in the text). (A) Plausible anatomical locations for the representation of computational
variables involved in the model. The pivotal region would be the dACC, which would integrate the goal value conveyed by ventral fronto-striatal
circuits and the effort cost transmitted by the anterior insula. The dACC would then send the net value to premotor and motor regions, which
would elaborate a motor command for the muscles. Muscle contraction would on the one hand have an added value, and on the other entail an
effort cost, which would be integrated in the corresponding brain regions. Within a particular trial, the behaviour would be generated through
several loops during which all representations are updated. Note that the loop can be internally simulated (with no overt movement) if the motor
regions inform directly (by an efferent copy of the motor command) the perceptual regions, from which effort cost can be estimated. There is no
consensus about the role of neuromodulators in this machinery—here we illustrate a tentative functional repartition where dopamine modulates
the goal value, 5HT the effort cost and noradrenaline the net value. (B) Mechanistic derivation of optimal behaviour in a putative brain-scale
network. The key mechanism is a gradient ascent: the temporal derivative of force (generated in motor regions) would scale with the derivative of
net value with respect to force (aggregated in the dACC by subtracting cost derivative from benefit derivative). In the incentive motivation task,
the derivative of the benefit is proportional to reward R (i.e. incentive level). The effective connectivity in the network would correspond to the
weighting of the different factors influencing the net value, in particular reward (kr) and effort (kc). The dynamics at longer time scales (across
trials), which could give rise to fatigue or satiety effects, is not represented. 5HT = serotonin; B = expected benefit; C = expected cost;
DA = dopamine; dACC = dorsal anterior cingulate cortex; F = produced force; Fp = perceived force; M1 = primary motor cortex;
NA = noradrenaline; PM = premotor cortex; R = reward level; SI-II = primary and secondary somatosensory cortex; V = net value;
vmPFC = ventromedial prefrontal cortex; VS = ventral striatum.

by Inserm/Disc user
on 04 December 2017
gradient of costs with respect to force. The former measures functional recovery during rehabilitation after vascular le-
the efficiency of effort investment, in terms of the net benefit sions, and represents an independent predictor of poor
accrued by a unitary change in force. The latter measures functional outcome (Hama et al., 2007; Mayo et al.,
the susceptibility of costs, in terms of the net energy expend- 2009; Caeiro et al., 2013), as well as everyday life depend-
iture incurred when force is increased by a unitary amount. ency (van Almenkerk et al., 2015). Apathy has a negative
In turn, this scheme highlights the computational nodes that impact on quality of life and can also lead to significant
are critical for performing effort allocation in the brain. In burden for caregivers (Aarsland et al., 1999; Samus et al.,
brief, effort allocation requires brain systems specialized for 2005).
(i) evaluating the benefit and cost gradients; (ii) aggregating The condition in which reward and effort processing
those to form the net gradient; and (iii) transforming the have been the most investigated is Parkinson’s disease,
latter into the appropriate dynamical adjustment of instant- which is characterized by a progressive degeneration of
aneous force production. In addition, the scheme may in- dopaminergic neurons, prominent motor symptoms such
volve a feedback loop of proprioceptive and/or afferent as akinesia, and frequent non-motor symptoms such as
copies of sensorimotor signals that enable the evaluation apathy. In Parkinson’s disease, apathy might result from
of the cost gradient. Note that in this particular design, dopamine depletion in the mesocorticolimbic pathway,
the benefit gradient is constant and equal to the potential which connects the ventral tegmental area to ventral stri-
reward R (i.e. incentive level). atum, orbitofrontal and anterior cingulate cortex (Remy
These theoretical brain systems can be mapped onto pos- et al., 2005; Thobois et al., 2010; Brown et al., 2012;
sible anatomical locations based on the neuroscientific lit- Martinez-Horta et al., 2014). Comparing Parkinson’s dis-
erature on the reward/effort trade-off (Fig. 6). This neural ease patients with and without dopaminergic medication
implementation is of course highly speculative but may has brought some insight into the role of dopamine in
serve to derive predictions about the sort of behavioural the reward/effort trade-off. Dopaminergic medication has
deficits that should be induced by specific brain damage. been shown to increase the amount of effort that
Lastly, this computational perspective suggests that the cou- Parkinson’s disease patients were willing to produce for a
pling strength between the aggregation system and the bene- given reward in a progressive ratio schedule task that
fit and cost systems corresponds to the control parameters involved clicking on a keyboard to earn money (Porat
kr and kc. In other terms, induced changes in effective con- et al., 2014), and in an accept versus decline choice task
nectivity between these systems should induce concomitant that involved squeezing a handgrip to earn fictive rewards
variations in the way individuals trade costs against benefits. (Chong et al., 2016). Dopaminergic medication has also
This mechanistic interpretation is interesting because it ties been shown to bias choices toward high effort–big
together the known biological impact of neuromodulators reward option in a binary choice task that traded handgrip
onto synaptic plasticity and the behavioural effect of related force against monetary payoff, and to enhance effort pro-
pharmacological drugs onto effort allocation. duction in a free operant (or incentive motivation) task that
involved squeezing a handgrip to win monetary rewards,
knowing that payoff would be proportional to force at
peak (Le Bouc et al., 2016). Thus, dopaminergic enhancers
Clinical manifestation have yielded consistent effects on effort expenditure for re-
The different neural structures involved in the reward/effort wards across a variety of experimental paradigms, in line
trade-off can be dysfunctional in a number of pathological with what was observed in rodents (Salamone et al., 2007).
conditions, and/or targeted by therapeutic interventions. Similar effects have also been obtained with high-frequency
Some specific motivational dysfunctions have been identi- stimulation of the subthalamic nucleus in Parkinson’s dis-
fied, in both neurology and psychiatry, and distinguished ease patients (Palminteri et al., 2013; Zenon et al., 2016b).
from motor or cognitive disabilities that might also affect Computational modelling showed that the energizing
the behaviour. We review this work, with a special em- effect of dopamine enhancers was best captured by an
phasis on studies that rely upon computational modelling. increased sensitivity to reward magnitude, and not a
decreased sensitivity to effort cost (Le Bouc et al., 2016).
This computational effect of dopaminergic medication
Neurology could account for both the enhanced propensity to select
Motivation disorders have been observed following focal high effort–big reward option in the choice task and the
lesions (stroke or tumour) and degenerative diseases. The enhanced force production in the motivation task. Thus,
most frequent disorder is apathy, which is generally defined the same computational mechanism could explain the
as a reduction of goal-directed behaviour, and present in orientation (action selection) and the intensity (action
50% of cases after traumatic brain injury (Arnould et al., vigour) of the behaviour. Dopaminergic medication also
2013), 40% in Parkinson’s disease (Starkstein et al., 1992), had an effect on the motor activation rate, a computational
50% in pre-dementia Alzheimer’s disease (Landes et al., parameter used in optimal control theory that serves to
2005), and 35% in patients after vascular lesions (Caeiro control the speed with which force is produced, irrespective
et al., 2013; van Dalen et al., 2013). Apathy alters of reward level. Crucially, this motor effect was

by Inserm/Disc user
on 04 December 2017
independent (uncorrelated across patients) from the motiv- term was affected in the net value function. Further com-
ational effect on reward magnitude. Moreover, medication putational analyses might help specify the motivational dys-
effects on motor and motivational parameters were corre- function that characterizes the autoactivation deficit. It has
lated with improvement of motor symptoms and apathy, been shown that dopaminergic medication could help re-
respectively. Such dissociation shows that computational store motivational function in these patients (Adam et al.,
analysis can help disentangle motor and motivational dis- 2013), which might indicate that apathy in autoactivation
orders in conditions where it is not obvious to tell whether deficit is qualitatively similar, but quantitatively more
the patient cannot or simply does not want to produce a severe, than apathy in Parkinson’s disease.
behaviour. This result is compatible with other model- Finally, apathy has been described following lesions of
based studies of Parkinson’s disease patients’ behaviour, the medial wall, which can include the ACC and vmPFC.
which also pointed to decreased reward sensitivity, irre- Unfortunately, few studies have investigated the reward/
spective of costs (Manohar et al., 2015; Zenon et al., effort trade-off in these patients. It has been reported that
2016a). ACC lesions attenuate the conscious feeling of mental effort
We note that dopamine receptor agonists may not only (Naccache et al., 2005), without impairing cognitive con-
improve apathy but also induce impulse control disorders trol performance. This is consistent with the report that
(ICDs) in a significant proportion of Parkinson’s disease electrical stimulation of ACC induces the subjective impres-
patients (17% according to Voon et al., 2017). ICDs sion of an imminent challenge and the determined intention
can include gambling disorder, binge eating, compulsive to cope with it (Parvizi et al., 2013). Examination of a large
shopping, compulsive sexual behaviours and compulsive cohort of patients showed that frontal medial damage was
medication use (dopamine dysregulation syndrome). In de- associated with a reduced effect of reward on invigorating
cision-making tasks, ICDs have been associated with saccade velocity (Manohar and Husain, 2016). Somewhat
enhanced sensitivity to delay, resulting in more impulsive surprisingly, this study reported an increased sensitivity to
choices (Housden et al., 2010; Voon et al., 2010b; Joutsa reward following specific (subgenual) vmPFC lesions, and a
et al., 2015), or with higher propensity to make risky trend in the opposite direction following ventral striatum
choices (Djamshidian et al., 2010; Voon et al., 2011). lesions. Further studies using proper physical or mental
However, ICDs could also be explained by dysfunction of effort would be needed to precise the effect of such lesions
learning mechanisms, such as an imbalance between posi- on the net value function.
tive and negative reinforcement (Voon et al., 2010a; Piray
et al., 2014). As learning models go beyond our scope, we
refer readers to a recent review of putative mechanisms
Psychiatry
underlying ICDs (Voon et al., 2017). Disorders of motivation are frequently observed in psychi-
Another case where apathy has been explored using atric diseases. Even when they are not the most apparent
reward/effort paradigms is the so-called autoactivation def- symptoms, they may interfere with cognitive abilities and
icit. This syndrome is characterized by a severe form of impede functional outcomes. Indeed, two of the most fre-
apathy following bilateral lesions in the striato-pallidal quent and devastating psychiatric diseases, schizophrenia
complex (Laplane et al., 1989). Patients present a complete and depression, include lack of motivation in their defin-
lack of self-initiated behaviour that contrasts with the pre- ition. According to the DSM-5, one of the two main nega-
served motor and cognitive abilities that they can exhibit tive symptoms in schizophrenia is avolition—a decrease in
when solicited by another person (Laplane and Dubois, motivated self-initiated purposeful activities (American
2001). These patients have been explored in the first Psychiatric Association, 2013). Similarly, a diagnosis of
study that assessed reward/effort trade-off in neurological major depressive disorder requires either depressed mood
conditions (Schmidt et al., 2008). In the incentive motiv- or markedly diminished interest or pleasure in all, or
ation task, patients were free to produce any handgrip almost all, activities most of the day, nearly every day.
force knowing that (fictive) payoff would be in proportion, Conversely, one of the criteria of manic/hypomanic epi-
whereas in the instructed control task, patients were expli- sodes is an increase in goal-directed activity (even if psy-
citly told how much to squeeze the handgrip. Patients chomotor agitation, i.e. purposeless non-goal-directed
showed a preserved ability to modulate their force accord- activity, is most often observed as severity increases).
ing to the instructions, a preserved autonomic (skin con- Beyond motivation deficit and excessive motivation, several
ductance) response to incentive cues (coin images), but an psychiatric diseases are partly defined by atypical interest
inability to modulate their force according to monetary in- (e.g. autism spectrum disorders) or deviant motivation (e.g.
centives (Schmidt et al., 2008). These results suggested that addictive disorders). In this section, we will mainly focus on
striato-pallidal lesions do not impair motor control, nor do schizophrenia and depression, where reward/effort-based
they impair the affective evaluation of monetary incentive, decision-making was particularly investigated in the recent
but rather the process that translates potential incentives years.
into motor activation (Schmidt et al., 2008). Interestingly, From an historical point of view, the very first descrip-
patients still produced a force in the incentive motivation tions of schizophrenia by Kraepelin and Bleuler already
task, suggesting that only the financial part of the benefit emphasized the motivation deficit, which has been

by Inserm/Disc user
on 04 December 2017
considered a cardinal and crucial component of negative Green et al., 2015), no existing study attempted to discrim-
symptoms (Kirkpatrick et al., 2006; Foussias and inate between decreased sensitivity to reward and increased
Remington, 2010). However, negative symptoms, which sensitivity to effort, using computational modelling tools.
are by definition less spectacular and de facto less respon- The only step in this direction was taken in a study report-
sive to antipsychotics than delusions or hallucinations, were ing that patients with schizophrenia produced the same
largely understudied until quite recently. Nevertheless, amount of effort as controls on average, but that effort
recent research put forward the critical importance of avo- expenditure was less driven by reward, such that choices
lition, which has been shown to be one of the best pre- were suboptimal in terms of maximizing outcomes
dictors of functional outcome (Bowie et al., 2006; Foussias (Treadway et al., 2015). More quantitative analyses
et al., 2011; Konstantakopoulos et al., 2011; Evensen et al., would be required to disclose the potential distortions of
2012; Fervaha et al., 2013a, 2014a, b), not only at the the net value function in schizophrenia, as well as the
chronic stage of the disease but also during first episodes impact of treatments such as antidopaminergic medication.
(Faerden et al., 2009, 2010; Chang et al., 2016) or even in As in schizophrenia, motivation deficit is a frequent and
ultra-high risk patients (Lam et al., 2015). Moreover, the burdensome symptom of depression, which is highly pre-
lack of motivation could partly account for other dimen- dictive of functional impairment and subjective wellbeing
sions of the disease, such as cognitive dysfunction, which (Calabrese et al., 2014; Fervaha et al., 2016). Moreover,
are usually evaluated through neuropsychological tests that as for avolition in schizophrenia, motivation deficit is less
require participants to be motivated for a correct perform- responsive to conventional treatment, such as serotoniner-
ance (Fervaha et al., 2014c). gic antidepressant in unipolar depression or mood stabilizer
Regarding behavioural tests of motivation, many studies in bipolar disorder, than other dimensions of the disease
revealed a reward processing deficit using desirability/pleas- (Calabrese et al., 2014).
antness ratings, intertemporal choices or reinforcement Over the past decade, most behavioural investigations
learning tasks (see Gold et al., 2008 for a review). Effort focused on reward processing versus punishment processing
was introduced lately in a seminal study showing that pa- (Eshel and Roiser, 2010; Huys et al., 2013; Admon and
tients with schizophrenia are less likely to favour high Pizzagalli, 2015; Chen et al., 2015). Typical results in de-
effort, especially when paired with largest and most certain pressed patients are excessive processing of negative stimuli,
monetary rewards (Gold et al., 2013). This shift in the hyposensitivity to positive outcomes with decreased pleas-
reward/effort trade-off has been replicated by several stu- urable experience, and blunted response to reward in the
dies (Fervaha et al., 2013c; Barch et al., 2014; Hartmann ventral striatum. Effort was first introduced with an incen-
et al., 2015; Treadway et al., 2015; Wang et al., 2015; tive motivation test using a handgrip device (Cléry-Melin
McCarthy et al., 2016; Strauss et al., 2016), across various et al., 2011), which established that contrary to healthy
types of tasks (binary choice, willingness to accept) and controls, depressed patients would not produce more
efforts (key pressing, handgrip squeezing), with only one effort when more money is at stake. A follow-up study
negative result (Docx et al., 2015). It has also been ex- showed that a normal pattern of effort production in re-
tended to cognitive (rather than motor) effort (Wolf sponse to monetary incentives was restored after remission
et al., 2014; Culbreth et al., 2016), again with one negative from depressive episode (Mauras et al., 2016). The dimin-
finding (Gold et al., 2015a). Furthermore, most studies ished sensitivity of effort production to incentives has been
found a correlation between the willingness to produce ef- replicated many times, using different tasks (binary choice,
forts and the severity of negative symptoms assessed with willingness to accept) and efforts (key pressing, handgrip
standard clinical scales (Gold et al., 2015b). Interestingly, a squeezing, cognitive effort) (Sherdell et al., 2012; Treadway
recent study compared five different reward/effort trade-off et al., 2012; Yang et al., 2014; Hershenberg et al., 2016).
tasks in a large sample of 94 patients with schizophrenia, While the shift in the reward/effort trade-off was correlated
with a 4-week retest (Reddy et al., 2015). All tasks, with with Beck Depression Inventory (BDI) score in healthy sub-
the notable exception of the perceptual categorization task, jects (Treadway et al., 2009), and was found more pro-
discriminated between patients and controls, with good nounced in actual depression than in subsyndromal
enough reliability, but relatively modest correlation with depression, two studies failed to find a correlation with
negative symptoms (Horan et al., 2015). Surprisingly, no global BDI score in depressed patients (Yang et al., 2014;
correlation was found with treatment, although most medi- Hershenberg et al., 2016). Thus, abnormal effort allocation
cations used in schizophrenia target dopaminergic transmis- might represent a specific dimension of depression rather
sion. Indeed, other authors found a strong effect of drug than a consequence of depression severity. Indeed, the will-
type (particularly conventional antipsychotics) on the willingness to produce effort for a given reward was correlated
ingness to expend effort, but argued that different drugs with lack of motivation, as measured by apathy scales, in
generally mean different patients, and claimed for rando- both healthy participants and depressed patients (Bonnelle
mized trials (Gold et al., 2015b). Critically, and despite the et al., 2015; Mauras et al., 2016). The link between anhe-
growing amount of evidence pointing toward an altered donia and reward/effort-based decision-making is less clear
reward/effort trade-off in schizophrenia (Fervaha et al., and still a matter of debate (Treadway and Zald, 2011;
2013b; Strauss et al., 2014; Green and Horan, 2015; Der-Avakian and Markou, 2012; Rizvi et al., 2016).

by Inserm/Disc user
on 04 December 2017
Regarding diagnostic, only one study compared unipolar that computational approach would (i) provide unprece-
and bipolar depressed patients, and found no difference dented insight into the disease; and (ii) improve on differ-
between these two populations, nor between the medication ential diagnostic and therapeutic orientation (e.g. assuming
classes used to treat them (Hershenberg et al., 2016). that hyposensitivity to reward should not require the same
A decreased reward/effort ratio was also found in drug- treatment as hypersensitivity to effort).
naı̈ve patients (Yang et al., 2014), which is an important There are nonetheless limitations to this approach. First,
control since serotoninergic medication could impact the estimation of computational parameters may depend
reward or effort processing. upon the paradigms used to probe the behaviour. Instead
Again, while consistent evidence was found for an alter- of faithfully representing the patient’s state, parameters
ation of reward/effort-based decision-making in depression, might be susceptible to confounding details of the tasks.
no study could dissociate between the two candidate under- A related issue is that quantification of motivation deficit
lying mechanisms, namely increased sensitivity to effort or may depend upon the particular goals suggested in behav-
decreased sensitivity to reward. Thus, the question remains ioural tests. Goal values are subjective and assigning a low
to what extent these apparently similar reward/effort trade- value to one particular goal is not necessarily pathological,
off alterations in schizophrenia and depression are indeed as long as other goals receive high values. This may be
supported by the same cognitive distortion and neurobio- problematic because most tasks target a reduced set of
logical pathophysiology. Importantly, decreased reward/ interests, typically motivation for performance or payoff
effort ratio is not an unspecific feature that would be (the goal is to accumulate points or money). Yet patients
observed in any psychiatric disease. Indeed this ratio was may not be strongly motivated by that sort of goal but still
reported to be unaltered for instance in pathological gam- driven by other goals, such as making their relatives happy.
blers (Fauth-Buhler et al., 2014), or even increased in Note that decision theory is only useful to capture disorders
autism spectrum disorder (Damiano et al., 2012). characterized by abnormal motivation intensity, not those
Unfortunately, reward/effort-based decision-making has defined by abnormal motivation contents (e.g. atypical
not been explored in patients for whom an excessive mo- interests in autism spectrum disorder). Finally, because all
tivation would be predicted from the clinical presentation, behavioural tasks suggest goals (as in ‘would you make an
such as during hypomanic episodes of bipolar disorder. effort to get this reward?’), they might miss motivation
deficits in which the generation of goals (and not goal valu-
ation) is precisely the problem. One could envisage the case
Conclusions and perspectives of a patient who would like to engage in some activity but
the idea of doing this activity would simply not come to
Compared to psychometric scoring that assesses motivation
mind. Other types of tests would certainly be required in
deficit as a whole, computational phenotyping based on
behavioural tests is a promising avenue for clinical neuro- order to detect this type of motivation deficit.
science. Modelling the behaviour as generated from a value
function (borrowed from decision theory) enables a norma-
tive decomposition of motivation into subcomponents that
may possess anatomo-functional specificity at the neural
Funding
level. For instance, the benefit term has been associated No funding was received towards this work.
with cortical regions such as the vmPFC, whereas other
regions like the dACC would integrate the costs. Also,
the weights of costs and benefits might be determined by
different neuromodulators such as dopamine, noradrenaline References
and serotonin. Thus, computational phenotyping may help
Aarsland D, Larsen JP, Lim NG, Janvin C, Karlsen K, Tandberg E,
discriminate between distinct mechanisms underlying mo- et al. Range of neuropsychiatric disturbances in patients with
tivation deficits, such as hyposensitivity to benefits and Parkinson’s disease. J Neurol Neurosurg Psychiatry 1999; 67:
hypersensitivity to costs, or abnormal susceptibility to fa- 492–6.
tigue. It may also help dissociate motivation deficits from Abitbol R, Lebreton M, Hollard G, Richmond BJ, Bouret S,
motor disorders, which seems crucial in a number of neuro- Pessiglione M. Neural mechanisms underlying contextual depend-
ency of subjective values: converging evidence from monkeys and
psychiatric diseases.
humans. J Neurosci 2015; 35: 2308–20.
An overarching objective for computational neuropsych- Adam R, Leff A, Sinha N, Turner C, Bays P, Draganski B, et al.
iatry would be to build a quantitative model of motivation Dopamine reverses reward insensitivity in apathy following globus
that integrates the neural and algorithmic levels. From vir- pallidus lesions. Cortex 2013; 49: 1292–303.
tual lesions of this grand model, one could derive predic- Admon R, Pizzagalli DA. Dysfunctional reward processing in depres-
sion. Curr Opin Psychol 2015; 4: 114–18.
tions about specific behavioural patterns that should be
Alnaes D, Sneve MH, Espeseth T, Endestad T, van de Pavert SH,
observed in patients. Reciprocally, by fitting the model to Laeng B. Pupil size signals mental effort deployed during multiple
observed patients’ behaviour, one could make inferences object tracking and predicts brain activity in the dorsal attention
about the underlying pathophysiology. The hope here is network and the locus coeruleus. J Vis 2014; 14: 1–20.

by Inserm/Disc user
on 04 December 2017
American Psychiatric Association. Diagnostic and statistical manual of Cléry-Melin ML, Schmidt L, Lafargue G, Baup N, Fossati P,
mental disorders. 5th edn. Arlington, VA: American Psychiatric Pessiglione M. Why don’t you try harder? An investigation of
Publishing; 2013. effort production in major depression. PLoS One 2011; 6: e23178.
Apps MA, Grima LL, Manohar S, Husain M. The role of cognitive Clithero JA, Rangel A. Informatic parcellation of the network involved
effort in subjective reward devaluation and risky decision-making. in the computation of subjective value. Soc Cogn Affect Neurosci
Sci Rep 2015; 5: 16880. 2014; 9: 1289–302.
Arnould A, Rochat L, Azouvi P, Van der Linden M. A multidimen- Critchley HD, Mathias CJ, Dolan RJ. Neural activity in the human
sional approach to apathy after traumatic brain injury. brain relating to uncertainty and arousal during anticipation.
Neuropsychol Rev 2013; 23: 210–33. Neuron 2001; 29: 537–45.
Barch DM, Treadway MT, Schoen N. Effort, anhedonia, and function Croxson PL, Walton ME, O’Reilly JX, Behrens TE, Rushworth MF.
in schizophrenia: reduced effort allocation predicts amotivation and Effort-based cost-benefit valuation and the human brain. J Neurosci
functional impairment. J Abnorm Psychol 2014; 123: 387–97. 2009; 29: 4531–41.
Bartra O, McGuire JT, Kable JW. The valuation system: a coordinate- Culbreth A, Westbrook A, Barch D. Negative symptoms are associated
based meta-analysis of BOLD fMRI experiments examining neural with an increased subjective cost of cognitive effort. J Abnorm
correlates of subjective value. Neuroimage 2013; 76: 412–27. Psychol 2016; 125: 528–36.
Beatty J, Wagoner BL. Pupillometric signs of brain activation vary Damiano CR, Aloi J, Treadway M, Bodfish JW, Dichter GS. Adults
with level of cognitive processing. Science 1978; 199: 1216–18. with autism spectrum disorders exhibit decreased sensitivity to
Bickel WK, Pitcock JA, Yi R, Angtuaco EJ. Congruence of BOLD reward parameters when making effort-based decisions.
response across intertemporal choice conditions: fictive and real J Neurodev Disord 2012; 4: 13.
money gains and losses. J Neurosci 2009; 29: 8839–46. Delgado MR, Labouliere CD, Phelps EA. Fear of losing money?
Bonnelle V, Veromann KR, Burnett Heyes S, Lo Sterzo E, Manohar S, Aversive conditioning with secondary reinforcers. Soc Cogn Affect
Husain M. Characterization of reward and effort mechanisms in Neurosci 2006; 1: 250–9.
apathy. J Physiol Paris 2015; 109: 16–26. Der-Avakian A, Markou A. The neurobiology of anhedonia and other
Botvinick M, Braver T. Motivation and cognitive control: from behav- reward-related deficits. Trends Neurosci 2012; 35: 68–77.
ior to neural mechanism. Annu Rev Psychol 2015; 66: 83–113. Djamshidian A, Jha A, O’Sullivan SS, Silveira-Moriyama L, Jacobson
Botvinick MM, Huffstetler S, McGuire JT. Effort discounting in C, Brown P, et al. Risk and learning in impulsive and nonimpulsive
human nucleus accumbens. Cogn Affect Behav Neurosci 2009; 9: patients with Parkinson’s disease. Mov Disord 2010; 25: 2203–10.
16–27. Docx L, de la Asuncion J, Sabbe B, Hoste L, Baeten R, Warnaerts N,
Boureau YL, Sokol-Hessner P, Daw ND. Deciding how to decide: self- et al. Effort discounting and its association with negative symptoms
control and meta-decision making. Trends Cogn Sci 2015; 19: 700–10. in schizophrenia. Cogn Neuropsychiatry 2015; 20: 172–85.
Bouret S, Richmond BJ. Ventromedial and orbital prefrontal neurons Drijgers RL, Dujardin K, Reijnders JS, Defebvre L, Leentjens AF.
differentially encode internally and externally driven motivational Validation of diagnostic criteria for apathy in Parkinson’s disease.
values in monkeys. J Neurosci 2010; 30: 8591–601. Parkinsonism Relat Disord 2010; 16: 656–60.
Bouret S, Richmond BJ. Sensitivity of locus ceruleus neurons to reward Eshel N, Roiser JP. Reward and punishment processing in depression.
value for goal-directed actions. J Neurosci 2015; 35: 4005–14. Biol Psychiatry 2010; 68: 118–24.
Bowie CR, Reichenberg A, Patterson TL, Heaton RK, Harvey PD. Evensen J, Rossberg JI, Barder H, Haahr U, Hegelstad W, Joa I, et al.
Determinants of real-world functional performance in schizophrenia Apathy in first episode psychosis patients: a ten year longitudinal
subjects: correlations with cognition, functional capacity, and symp- follow-up study. Schizophr Res 2012; 136: 19–24.
toms. Am J Psychiatry 2006; 163: 418–25. Faerden A, Finset A, Friis S, Agartz I, Barrett EA, Nesvag R, et al.
Brown CA, Campbell MC, Karimi M, Tabbal SD, Loftin SK, Tian LL, Apathy in first episode psychosis patients: one year follow up.
et al. Dopamine pathway loss in nucleus accumbens and ventral Schizophr Res 2010; 116: 20–6.
tegmental area predicts apathetic behavior in MPTP-lesioned mon- Faerden A, Friis S, Agartz I, Barrett EA, Nesvag R, Finset A, et al.
keys. Exp Neurol 2012; 236: 190–7. Apathy and functioning in first-episode psychosis. Psychiatr Serv
Brown RG, Pluck G. Negative symptoms: the ‘pathology’ of motiv- 2009; 60: 1495–503.
ation and goal-directed behaviour. Trends Neurosci 2000; 23: 412– Fauth-Buhler M, Zois E, Vollstadt-Klein S, Lemenager T, Beutel M,
17. Mann K. Insula and striatum activity in effort-related monetary
Burke CJ, Brunger C, Kahnt T, Park SQ, Tobler PN. Neural integra- reward processing in gambling disorder: the role of depressive symp-
tion of risk and effort costs by the frontal pole: only upon request. tomatology. Neuroimage Clin 2014; 6: 243–51.
J Neurosci 2013; 33: 1706–13a. Fervaha G, Foussias G, Agid O, Remington G. Amotivation and func-
Caeiro L, Ferro JM, Costa J. Apathy secondary to stroke: a systematic tional outcomes in early schizophrenia. Psychiatry Res 2013a; 210:
review and meta-analysis. Cerebrovasc Dis 2013; 35: 23–39. 665–8.
Calabrese JR, Fava M, Garibaldi G, Grunze H, Krystal AD, Laughren Fervaha G, Foussias G, Agid O, Remington G. Neural substrates
T, et al. Methodological approaches and magnitude of the clinical underlying effort computation in schizophrenia. Neurosci Biobehav
unmet need associated with amotivation in mood disorders. J Affect Rev 2013b; 37 (10 Pt 2): 2649–65.
Disord 2014; 168: 439–51. Fervaha G, Foussias G, Agid O, Remington G. Impact of primary
Chang WC, Hui CL, Chan SK, Lee EH, Chen EY. Impact of avolition negative symptoms on functional outcomes in schizophrenia. Eur
and cognitive impairment on functional outcome in first-episode Psychiatry 2014a; 29: 449–55.
schizophrenia-spectrum disorder: a prospective one-year follow-up Fervaha G, Foussias G, Agid O, Remington G. Motivational and neuro-
study. Schizophr Res 2016; 170: 318–21. cognitive deficits are central to the prediction of longitudinal functional
Chen C, Takahashi T, Nakagawa S, Inoue T, Kusumi I. Reinforcement outcome in schizophrenia. Acta Psychiatr Scand 2014b; 130: 290–9.
learning in depression: a review of computational research. Neurosci Fervaha G, Foussias G, Takeuchi H, Agid O, Remington G.
Biobehav Rev 2015; 55: 247–67. Motivational deficits in major depressive disorder: cross-sectional
Chong TT, Bonnelle V, Husain M. Quantifying motivation with effort- and longitudinal relationships with functional impairment and sub-
based decision-making paradigms in health and disease. Prog Brain jective well-being. Compr Psychiatry 2016; 66: 31–8.
Res 2016; 229: 71–100. Fervaha G, Graff-Guerrero A, Zakzanis KK, Foussias G, Agid O,
Chong TT, Bonnelle V, Manohar S, Veromann KR, Muhammed K, Remington G. Incentive motivation deficits in schizophrenia reflect
Tofaris GK, et al. Dopamine enhances willingness to exert effort for effort computation impairments during cost-benefit decision-making.
reward in Parkinson’s disease. Cortex 2015; 69: 40–6. J Psychiatr Res 2013c; 47: 1590–6.

by Inserm/Disc user
on 04 December 2017
Fervaha G, Zakzanis KK, Foussias G, Graff-Guerrero A, Agid O, Housden CR, O’Sullivan SS, Joyce EM, Lees AJ, Roiser JP. Intact
Remington G. Motivational deficits and cognitive test performance reward learning but elevated delay discounting in Parkinson’s dis-
in schizophrenia. JAMA Psychiatry 2014c; 71: 1058–65. ease patients with impulsive-compulsive spectrum behaviors.
Fonseca MS, Murakami M, Mainen ZF. Activation of dorsal raphe Neuropsychopharmacology 2010; 35: 2155–64.
serotonergic neurons promotes waiting but is not reinforcing. Curr Hsu ES, Patwardhan AG, Meade KP, Light TR, Martin WR. Cross-
Biol 2015; 25: 306–15. sectional geometrical properties and bone mineral contents of the
Forbes GB, Brown MR, Griffiths HJ. Arm muscle plus bone area: human radius and ulna. J Biomech 1993; 26: 1307–18.
anthropometry and CAT scan compared. Am J Clin Nutr 1988; Hull CL. Principles of behavior, an introduction to behavior theory.
47: 929–31. New York, NY; London: Appleton-Century Company; 1943.
Foussias G, Mann S, Zakzanis KK, van Reekum R, Agid O, Huys QJ, Pizzagalli DA, Bogdan R, Dayan P. Mapping anhedonia
Remington G. Prediction of longitudinal functional outcomes in onto reinforcement learning: a behavioural meta-analysis. Biol
schizophrenia: the impact of baseline motivational deficits. Mood Anxiety Disord 2013; 3: 12.
Schizophr Res 2011; 132: 24–7. Inzlicht M, Schmeichel BJ, Macrae CN. Why self-control seems (but
Foussias G, Remington G. Negative symptoms in schizophrenia: avoli- may not be) limited. Trends Cogn Sci 2014; 18: 127–33.
tion and Occam’s razor. Schizophr Bull 2010; 36: 359–69. Joutsa J, Voon V, Johansson J, Niemela S, Bergman J, Kaasinen V.
Frederick S, Loewenstein G, O’Donoghue T. Time discounting and Dopaminergic function and intertemporal choice. Transl Psychiatry
time preference: a critical review. J Econ Lit 2002; 40: 351–401. 2015; 5: e491.
Gan JO, Walton ME, Phillips PE. Dissociable cost and benefit encod- Kahneman D, Wright P. Changes of pupil size and rehearsal strategies
ing of future rewards by mesolimbic dopamine. Nat Neurosci 2010; in a short-term memory task. Q J Exp Psychol 1971; 23: 187–96.
13: 25–7. Kennerley SW, Dahmubed AF, Lara AH, Wallis JD. Neurons in the
Gergelyfi M, Jacob B, Olivier E, Zenon A. Dissociation between frontal lobe encode the value of multiple decision variables. J Cogn
mental fatigue and motivational state during prolonged mental ac- Neurosci 2009; 21: 1162–78.
tivity. Front Behav Neurosci 2015; 9: 176. Kirkpatrick B, Fenton WS, Carpenter WT Jr, Marder SR. The NIMH-
Gold JM, Kool W, Botvinick MM, Hubzin L, August S, Waltz JA. MATRICS consensus statement on negative symptoms. Schizophr
Cognitive effort avoidance and detection in people with schizophre- Bull 2006; 32: 214–19.
nia. Cogn Affect Behav Neurosci 2015a; 15: 145–54. Klein-Flugge MC, Kennerley SW, Saraiva AC, Penny WD, Bestmann S.
Gold JM, Strauss GP, Waltz JA, Robinson BM, Brown JK, Frank MJ. Behavioral modeling of human choices reveals dissociable effects of
Negative symptoms of schizophrenia are associated with abnormal physical effort and temporal delay on reward devaluation. PLoS
effort-cost computations. Biol Psychiatry 2013; 74: 130–6. Comput Biol 2015; 11: e1004116.
Gold JM, Waltz JA, Frank MJ. Effort cost computation in schizophre- Knutson B, Fong GW, Adams CM, Varner JL, Hommer D.
nia: a commentary on the recent literature. Biol Psychiatry 2015b; Dissociation of reward anticipation and outcome with event-related
78: 747–53. fMRI. Neuroreport 2001; 12: 3683–7.
Gold JM, Waltz JA, Prentice KJ, Morris SE, Heerey EA. Reward pro- Kobayashi S, Schultz W. Influence of reward delays on responses of
cessing in schizophrenia: a deficit in the representation of value. dopamine neurons. J Neurosci 2008; 28: 7837–46.
Schizophr Bull 2008; 34: 835–47. Konstantakopoulos G, Ploumpidis D, Oulis P, Patrikelis P, Soumani A,
Green L, Myerson J. A discounting framework for choice with delayed Papadimitriou GN, et al. Apathy, cognitive deficits and functional
and probabilistic rewards. Psychol Bull 2004; 130: 769–92. impairment in schizophrenia. Schizophr Res 2011; 133: 193–8.
Green MF, Horan WP. Effort-based decision making in schizophrenia: Kool W, McGuire JT, Rosen ZB, Botvinick MM. Decision making and
evaluation of paradigms to measure motivational deficits. Schizophr
the avoidance of cognitive demand. J Exp Psychol Gen 2010; 139:
Bull 2015; 41: 1021–3.
665–82.
Green MF, Horan WP, Barch DM, Gold JM. Effort-based decision
Kroemer NB, Guevara A, Ciocanea Teodorescu I, Wuttig F, Kobiella
making: a novel approach for assessing motivation in schizophrenia.
A, Smolka MN. Balancing reward and work: anticipatory brain
Schizophr Bull 2015;41: 1035–44.
activation in NAcc and VTA predict effort differentially.
Haber SN. The primate basal ganglia: parallel and integrative net-
Neuroimage 2014; 102 Pt 2: 510–19.
works. J Chem Neuroanat 2003; 26: 317–30.
Kurniawan IT, Guitart-Masip M, Dayan P, Dolan RJ. Effort and valu-
Hama S, Yamashita H, Shigenobu M, Watanabe A, Hiramoto K,
ation in the brain: the effects of anticipation and execution.
Kurisu K, et al. Depression or apathy and functional recovery
J Neurosci 2013; 33: 6160–9.
after stroke. Int J Geriatr Psychiatry 2007; 22: 1046–51.
Kurniawan IT, Seymour B, Talmi D, Yoshida W, Chater N, Dolan RJ.
Hamid AA, Pettibone JR, Mabrouk OS, Hetrick VL, Schmidt R,
Choosing to make an effort: the role of striatum in signaling
Vander Weele CM, et al. Mesolimbic dopamine signals the value
physical effort of a chosen action. J Neurophysiol 2010; 104:
of work. Nat Neurosci 2016; 19: 117–26.
313–21.
Hartmann MN, Hager OM, Reimann AV, Chumbley JR, Kirschner
Kurzban R, Duckworth A, Kable JW, Myers J. An opportunity cost
M, Seifritz E, et al. Apathy but not diminished expression in schizo-
model of subjective effort and task performance. Behav Brain Sci
phrenia is associated with discounting of monetary rewards by phys-
2013; 36: 661–79.
ical effort. Schizophr Bull 2015; 41: 503–12.
LaBar KS, Gatenby JC, Gore JC, LeDoux JE, Phelps EA. Human
Hartmann MN, Hager OM, Tobler PN, Kaiser S. Parabolic discount-
amygdala activation during conditioned fear acquisition and extinc-
ing of monetary rewards by physical effort. Behav Processes 2013;
100: 192–6. tion: a mixed-trial fMRI study. Neuron 1998; 20: 937–45.
Hershenberg R, Satterthwaite TD, Daldal A, Katchmar N, Moore TM, Lak A, Stauffer WR, Schultz W. Dopamine prediction error responses
Kable JW, et al. Diminished effort on a progressive ratio task in both integrate subjective value from different reward dimensions. Proc
unipolar and bipolar depression. J Affect Disord 2016; 196: 97–100. Natl Acad Sci USA 2014; 111: 2343–8.
Hollon NG, Arnold MM, Gan JO, Walton ME, Phillips PE. Lam M, Abdul Rashid NA, Lee SA, Lim J, Foussias G, Fervaha G,
Dopamine-associated cached values are not sufficient as the basis et al. Baseline social amotivation predicts 1-year functioning in UHR
for action selection. Proc Natl Acad Sci USA 2014; 111: 18357–62. subjects: a validation and prospective investigation. Eur
Horan WP, Reddy LF, Barch DM, Buchanan RW, Dunayevich E, Neuropsychopharmacol 2015; 25: 2187–96.
Gold JM, et al. Effort-based decision-making paradigms for clinical Landes AM, Sperry SD, Strauss ME. Prevalence of apathy, dysphoria,
trials in schizophrenia: part 2-external validity and correlates. and depression in relation to dementia severity in Alzheimer’s dis-
Schizophr Bull 2015; 41: 1055–65. ease. J Neuropsychiatry Clin Neurosci 2005; 17: 342–9.

by Inserm/Disc user
on 04 December 2017
Laplane D, Dubois B. Auto-activation deficit: a basal ganglia related Padoa-Schioppa C, Cai X. The orbitofrontal cortex and the computa-
syndrome. Mov Disord 2001; 16: 810–14. tion of subjective value: consolidated concepts and new perspectives.
Laplane D, Levasseur M, Pillon B, Dubois B, Baulac M, Mazoyer B, Ann N Y Acad Sci 2011; 1239: 130–7.
et al. Obsessive-compulsive and other behavioural changes with bi- Palminteri S, Serra G, Buot A, Schmidt L, Welter ML, Pessiglione M.
lateral basal ganglia lesions. A neuropsychological, magnetic reson- Hemispheric dissociation of reward processing in humans: insights
ance imaging and positron tomography study. Brain 1989; 112 (Pt from deep brain stimulation. Cortex 2013; 49: 2834–44.
3): 699–725. Parvizi J, Rangarajan V, Shirer WR, Desai N, Greicius MD. The will
Le Bouc R, Pessiglione M. Imaging social motivation: distinct brain to persevere induced by electrical stimulation of the human cingulate
mechanisms drive effort production during collaboration versus gyrus. Neuron 2013; 80: 1359–67.
competition. J Neurosci 2013; 33: 15894–902. Pasquereau B, Turner RS. Limited encoding of effort by dopamine
Le Bouc R, Rigoux L, Schmidt L, Degos B, Welter ML, Vidailhet M, neurons in a cost-benefit trade-off task. J Neurosci 2013; 33:
et al. Computational dissection of dopamine motor and motiv- 8288–300.
ational functions in humans. J Neurosci 2016; 36: 6623–33. Pessiglione M, Schmidt L, Draganski B, Kalisch R, Lau H, Dolan RJ,
Levy DJ, Glimcher PW. The root of all value: a neural common cur- et al. How the brain translates money into force: a neuroimaging
rency for choice. Curr Opin Neurobiol 2012; 22: 1027–38. study of subliminal motivation. Science 2007; 316: 904–6.
Levy R, Dubois B. Apathy and the functional anatomy of the prefrontal Pessiglione M, Seymour B, Flandin G, Dolan RJ, Frith CD. Dopamine-
cortex-basal ganglia circuits. Cereb Cortex 2006; 16: 916–28. dependent prediction errors underpin reward-seeking behaviour in
Manohar SG, Chong TT, Apps MA, Batla A, Stamelou M, Jarman humans. Nature 2006; 442: 1042–5.
PR, et al. Reward pays the cost of noise reduction in motor and Peters J, Buchel C. Neural representations of subjective reward value.
cognitive control. Curr Biol 2015; 25: 1707–16. Behav Brain Res 2010; 213: 135–41.
Manohar SG, Husain M. Human ventromedial prefrontal lesions alter Petrovic P, Kalisch R, Pessiglione M, Singer T, Dolan RJ. Learning
incentivisation by reward. Cortex 2016; 76: 104–20. affective values for faces is expressed in amygdala and fusiform
Marin RS. Differential diagnosis and classification of apathy. Am J gyrus. Soc Cogn Affect Neurosci 2008; 3: 109–18.
Psychiatry 1990; 147: 22–30. Piquado T, Isaacowitz D, Wingfield A. Pupillometry as a measure of
Martinez-Horta S, Riba J, de Bobadilla RF, Pagonabarraga J, Pascual- cognitive effort in younger and older adults. Psychophysiology
Sedano B, Antonijoan RM, et al. Apathy in Parkinson’s disease: 2010; 47: 560–9.
neurophysiological evidence of impaired incentive processing. Piray P, Zeighami Y, Bahrami F, Eissa AM, Hewedi DH, Moustafa
J Neurosci 2014; 34: 5918–26. AA. Impulse control disorders in Parkinson’s disease are associated
Massar SA, Libedinsky C, Weiyan C, Huettel SA, Chee MW. Separate with dysfunction in stimulus valuation but not action valuation.
and overlapping brain areas encode subjective value during delay J Neurosci 2014; 34: 7814–24.
and effort discounting. Neuroimage 2015; 120: 104–13. Porat O, Hassin-Baer S, Cohen OS, Markus A, Tomer R. Asymmetric
Mauras T, Masson M, Fossati P, Pessiglione M. Incentive sensitivity as dopamine loss differentially affects effort to maximize gain or min-
a behavioral marker of clinical remission from major depressive imize loss. Cortex 2014; 51: 82–91.
episode. J Clin Psychiatry 2016; 77: e697–703. Prevost C, Pessiglione M, Metereau E, Clery-Melin ML, Dreher JC.
Mayo NE, Fellows LK, Scott SC, Cameron J, Wood-Dauphinee S. A Separate valuation subsystems for delay and effort decision costs.
longitudinal view of apathy and its impact after stroke. Stroke 2009; J Neurosci 2010; 30: 14080–90.
40: 3299–307. Radakovic R, Abrahams S. Developing a new apathy measure-
McCarthy JM, Treadway MT, Bennett ME, Blanchard JJ. Inefficient
ment scale: Dimensional Apathy Scale. Psychiatry Res 2014; 219:
effort allocation and negative symptoms in individuals with schizo-
658–63.
phrenia. Schizophr Res 2016; 170: 278–84.
Raichle ME, Gusnard DA. Intrinsic brain activity sets the stage for
Meyniel F, Goodwin GM, Deakin JW, Klinge C, MacFadyen C,
expression of motivated behavior. J Comp Neurol 2005; 493: 167–
Milligan H, et al. A specific role for serotonin in overcoming
76.
effort cost. Elife 2016; 5: e17282.
Reddy LF, Horan WP, Barch DM, Buchanan RW, Dunayevich E,
Meyniel F, Safra L, Pessiglione M. How the brain decides when to
Gold JM, et al. Effort-based decision-making paradigms for clinical
work and when to rest: dissociation of implicit-reactive from expli-
trials in schizophrenia: part 1—psychometric characteristics of 5
cit-predictive computational processes. PLoS Comput Biol 2014; 10:
paradigms. Schizophr Bull 2015;41: 1045–54.
e1003584.
Remy P, Doder M, Lees A, Turjanski N, Brooks D. Depression in
Meyniel F, Sergent C, Rigoux L, Daunizeau J, Pessiglione M.
Parkinson’s disease: loss of dopamine and noradrenaline innervation
Neurocomputational account of how the human brain decides
in the limbic system. Brain 2005; 128 (Pt 6): 1314–22.
when to have a break. Proc Natl Acad Sci USA 2013; 110: 2641–6.
Rigoux L, Guigon E. A model of reward- and effort-based optimal
Minamimoto T, Hori Y, Richmond BJ. Is working more costly than
decision making and motor control. PLoS Comput Biol 2012; 8:
waiting in monkeys? PLoS One 2012; 7: e48434.
e1002716.
Miyazaki K, Miyazaki KW, Doya K. The role of serotonin in the
Rizvi SJ, Pizzagalli DA, Sproule BA, Kennedy SH. Assessing anhedonia
regulation of patience and impulsivity. Mol Neurobiol 2012; 45:
in depression: potentials and pitfalls. Neurosci Biobehav Rev 2016;
213–24.
Naccache L, Dehaene S, Cohen L, Habert MO, Guichart-Gomez E, 65: 21–35.
Robert PH, Clairet S, Benoit M, Koutaich J, Bertogliati C, Tible O,
Galanaud D, et al. Effortless control: executive attention and con-
scious feeling of mental effort are dissociable. Neuropsychologia et al. The apathy inventory: assessment of apathy and awareness in
2005; 43: 1318–28. Alzheimer’s disease, Parkinson’s disease and mild cognitive impair-
Neu CM, Rauch F, Rittweger J, Manz F, Schoenau E. Influence of ment. Int J Geriatr Psychiatry 2002; 17: 1099–105.
puberty on muscle development at the forearm. Am J Physiol Rudebeck PH, Walton ME, Smyth AN, Bannerman DM, Rushworth
Endocrinol Metab 2002; 283: E103–7. MF. Separate neural pathways process different decision costs. Nat
Niyogi RK, Shizgal P, Dayan P. Some work and some play: micro- Neurosci 2006; 9: 1161–8.
scopic and macroscopic approaches to labor and leisure. PLoS Salamone JD, Correa M, Farrar A, Mingote SM. Effort-related func-
Comput Biol 2014; 10: e1003894. tions of nucleus accumbens dopamine and associated forebrain cir-
O’Doherty JP. Multiple systems for the motivational control of behav- cuits. Psychopharmacology 2007; 191: 461–82.
ior and associated neural substrates in humans. Curr Top Behav Salamone JD, Yohn SE, Lopez-Cruz L, San Miguel N, Correa M.
Neurosci 2016; 27: 291–312. Activational and effort-related aspects of motivation: neural

by Inserm/Disc user
on 04 December 2017
mechanisms and implications for psychopathology. Brain 2016; 139 Parkinson’s disease: predictors and underlying mesolimbic denerv-
(Pt 5): 1325–47. ation. Brain 2010; 133 (Pt 4): 1111–27.
Samanez-Larkin GR, Hollon NG, Carstensen LL, Knutson B. Treadway MT, Bossaller NA, Shelton RC, Zald DH. Effort-based de-
Individual differences in insular sensitivity during loss anticipation cision-making in major depressive disorder: a translational model of
predict avoidance learning. Psychol Sci 2008; 19: 320–3. motivational anhedonia. J Abnorm Psychol 2012; 121: 553–8.
Samus QM, Rosenblatt A, Steele C, Baker A, Harper M, Brandt J, Treadway MT, Buckholtz JW, Schwartzman AN, Lambert WE, Zald
et al. The association of neuropsychiatric symptoms and environ- DH. Worth the ‘EEfRT’? The effort expenditure for rewards task as
ment with quality of life in assisted living residents with dementia. an objective measure of motivation and anhedonia. PLoS One 2009;
Gerontologist 2005; 45 Spec No 1: 19–26. 4: e6598.
San-Galli A, Varazzani C, Abitbol R, Pessiglione M, Bouret S. Primate Treadway MT, Peterman JS, Zald DH, Park S. Impaired effort allo-
ventromedial prefrontal cortex neurons continuously encode the cation in patients with schizophrenia. Schizophr Res 2015; 161:
willingness to engage in reward-directed behavior. Cereb Cortex 382–5.
2016. doi: 10.1093/cercor/bhw351. Treadway MT, Zald DH. Reconsidering anhedonia in depression: les-
Schmidt L, Clery-Melin ML, Lafargue G, Valabregue R, Fossati P, sons from translational neuroscience. Neurosci Biobehav Rev 2011;
Dubois B, et al. Get aroused and be stronger: emotional facilitation 35: 537–55.
of physical effort in the human brain. J Neurosci 2009; 29: 9450–7. van Almenkerk S, Smalbrugge M, Depla MF, Eefsting JA, Hertogh
Schmidt L, d’Arc BF, Lafargue G, Galanaud D, Czernecki V, Grabli D, CM. Apathy among institutionalized stroke patients: prevalence
et al. Disconnecting force from money: effects of basal ganglia damage and clinical correlates. Am J Geriatr Psychiatry 2015; 23: 180–8.
on incentive motivation. Brain 2008; 131 (Pt 5): 1303–10. van Dalen JW, Moll van Charante EP, Nederkoorn PJ, van Gool WA,
Schmidt L, Lebreton M, Clery-Melin ML, Daunizeau J, Pessiglione M. Richard E. Poststroke apathy. Stroke 2013; 44: 851–60.
Neural mechanisms underlying motivation of mental versus physical Varazzani C, San-Galli A, Gilardeau S, Bouret S. Noradrenaline and
effort. PLoS Biol 2012; 10: e1001266. dopamine neurons in the reward/effort trade-off: a direct electro-
Schmidt L, Palminteri S, Lafargue G, Pessiglione M. Splitting motiv- physiological comparison in behaving monkeys. J Neurosci 2015;
ation: unilateral effects of subliminal incentives. Psychol Sci 2010; 35: 7866–77.
21: 977–83. Venugopalan VV, Casey KF, O’Hara C, O’Loughlin J, Benkelfat C,
Scholl J, Kolling N, Nelissen N, Wittmann MK, Harmer CJ, Fellows LK, et al. Acute phenylalanine/tyrosine depletion reduces
Rushworth MF. The good, the bad, and the irrelevant: neural mech- motivation to smoke cigarettes across stages of addiction.
anisms of learning real and hypothetical rewards and effort. Neuropsychopharmacology 2011; 36: 2469–76.
J Neurosci 2015; 35: 11233–51. Voon V, Gao J, Brezing C, Symmonds M, Ekanayake V, Fernandez H,
Schouppe N, Demanet J, Boehler CN, Ridderinkhof KR, Notebaert W. et al. Dopamine agonists and risk: impulse control disorders in
The role of the striatum in effort-based decision-making in the ab- Parkinson’s disease. Brain 2011; 134 (Pt 5): 1438–46.
sence of reward. J Neurosci 2014; 34: 2148–54. Voon V, Napier TC, Frank MJ, Sgambato-Faure V, Grace AA,
Seymour B, O’Doherty JP, Koltzenburg M, Wiech K, Frackowiak R, Rodriguez-Oroz M, et al. Impulse control disorders and levodopa-
Friston K, et al. Opponent appetitive-aversive neural processes induced dyskinesias in Parkinson’s disease: an update. Lancet
underlie predictive learning of pain relief. Nat Neurosci 2005; 8: Neurol 2017; 16: 238–50.
1234–40. Voon V, Pessiglione M, Brezing C, Gallea C, Fernandez HH, Dolan
Sherdell L, Waugh CE, Gotlib IH. Anticipatory pleasure predicts mo-
RJ, et al. Mechanisms underlying dopamine-mediated reward bias in
tivation for reward in major depression. J Abnorm Psychol 2012;
compulsive behaviors. Neuron 2010a; 65: 135–42.
121: 51–60.
Voon V, Reynolds B, Brezing C, Gallea C, Skaljic M, Ekanayake V,
Shidara M, Richmond BJ. Anterior cingulate: single neuronal signals
et al. Impulsive choice and response in dopamine agonist-related
related to degree of reward expectancy. Science 2002; 296: 1709–11.
impulse control behaviors. Psychopharmacology 2010b; 207: 645–
Skvortsova V, Palminteri S, Pessiglione M. Learning to minimize ef-
59.
forts versus maximizing rewards: computational principles and
Wallis JD. Cross-species studies of orbitofrontal cortex and value-
neural correlates. J Neurosci 2014; 34: 15621–30.
based decision-making. Nat Neurosci 2011; 15: 13–19.
Sockeel P, Dujardin K, Devos D, Deneve C, Destee A, Defebvre L. The
Walton ME, Bannerman DM, Alterescu K, Rushworth MF.
Lille apathy rating scale (LARS), a new instrument for detecting and
Functional specialization within medial frontal cortex of the anterior
quantifying apathy: validation in Parkinson’s disease. J Neurol
cingulate for evaluating effort-related decisions. J Neurosci 2003;
Neurosurg Psychiatry 2006; 77: 579–84.
23: 6475–9.
Starkstein SE, Mayberg HS, Preziosi TJ, Andrezejewski P, Leiguarda R,
Walton ME, Kennerley SW, Bannerman DM, Phillips PE, Rushworth
Robinson RG. Reliability, validity, and clinical correlates of apathy in
MF. Weighing up the benefits of work: behavioral and neural ana-
Parkinson’s disease. J Neuropsychiatry Clin Neurosci 1992; 4: 134–9.
lyses of effort-related decision making. Neural Netw 2006; 19:
Stephens DW, Krebs JR. Foraging theory. Princeton, NJ; Guildford:
1302–14.
Princeton University Press; 1986.
Wang J, Huang J, Yang XH, Lui SS, Cheung EF, Chan RC.
Strait CE, Blanchard TC, Hayden BY. Reward value comparison via
mutual inhibition in ventromedial prefrontal cortex. Neuron 2014; Anhedonia in schizophrenia: deficits in both motivation and hedonic
82: 1357–66. capacity. Schizophr Res 2015;168: 465–74.
Strauss GP, Waltz JA, Gold JM. A review of reward processing and Wardle MC, Treadway MT, Mayo LM, Zald DH, de Wit H. Amping
motivational impairment in schizophrenia. Schizophr Bull 2014; 40 up effort: effects of d-amphetamine on human effort-based decision-
(Suppl 2): S107–16. making. J Neurosci 2011; 31: 16597–602.
Strauss GP, Whearty KM, Morra LF, Sullivan SK, Ossenfort KL, Frost Westbrook A, Kester D, Braver TS. What is the subjective cost of
KH. Avolition in schizophrenia is associated with reduced willing- cognitive effort? Load, trait, and aging effects revealed by economic
ness to expend effort for reward on a progressive ratio task. preference. PLoS One 2013; 8: e68210.
Schizophr Res 2016; 170: 198–204. Wolf DH, Satterthwaite TD, Kantrowitz JJ, Katchmar N, Vandekar L,
Tachibana Y, Hikosaka O. The primate ventral pallidum encodes ex- Elliott MA, et al. Amotivation in schizophrenia: integrated assess-
pected reward value and regulates motor action. Neuron 2012; 76: ment with behavioral, clinical, and imaging measures. Schizophr
826–37. Bull 2014; 40: 1328–37.
Thobois S, Ardouin C, Lhommee E, Klinger H, Lagrange C, Xie J, Yang XH, Huang J, Zhu CY, Wang YF, Cheung EF, Chan RC, et al.
et al. Non-motor dopamine withdrawal syndrome after surgery for Motivational deficits in effort-based decision making in individuals

by Inserm/Disc user
on 04 December 2017
with subsyndromal depression, first-episode and remitted depression and the cost of effort during decision-making. Brain 2016b; 139 (Pt
patients. Psychiatry Res 2014; 220: 874–82. 6): 1830–43.
Zenon A, Devesse S, Olivier E. Dopamine manipulation affects re- Zenon A, Sidibe M, Olivier E. Pupil size variations correlate with
sponse vigor independently of opportunity cost. J Neurosci 2016a; physical effort perception. Front Behav Neurosci 2014; 8: 286.
36: 9516–25. Zipf GK. Human behavior and the principle of least effort: an intro-
Zenon A, Duclos Y, Carron R, Witjas T, Baunez C, Regis J, et al. The duction to human ecology. Cambridge, MA: Addison-Wesley Press;
human subthalamic nucleus encodes the subjective value of reward 1949.

by Inserm/Disc user
on 04 December 2017

Why Not Try Harder? Computational Approach To Motivation Deficits in Neuro-Psychiatric Diseases

Transféré par

Informations du document

Titre original

Copyright

Formats disponibles

Partager ce document

Partager ou intégrer le document

Options de partage

Avez-vous trouvé ce document utile ?

Ce contenu est-il inapproprié ?

Droits d'auteur :

Formats disponibles

Why Not Try Harder? Computational Approach To Motivation Deficits in Neuro-Psychiatric Diseases

Transféré par

Droits d'auteur :

Formats disponibles

doi:10.

1093/brain/awx278 BRAIN 2017: Page 1 of 22 | 1

Correspondence may also be addressed to: Raphaël Le Bouc

Keywords: apathy; computational psychiatry; goal-directed behaviour; behavioural neurology; decision-making

Downloaded from https://academic.oup.com/brain/advance-article-abstract/doi/10.1093/brain/awx278/4675073

This means estimating the values that the patient assigns to

Table 1 Starkstein’s apathy scale

Downloaded from https://academic.oup.com/brain/advance-article-abstract/doi/10.1093/brain/awx278/4675073

sorts: time and effort. Time is a cost because if time is

Downloaded from https://academic.oup.com/brain/advance-article-abstract/doi/10.1093/brain/awx278/4675073

Equivalent paradigms have been developed for human

Downloaded from https://academic.oup.com/brain/advance-article-abstract/doi/10.1093/brain/awx278/4675073

animal experiments, which may stop performing the task

Downloaded from https://academic.oup.com/brain/advance-article-abstract/doi/10.1093/brain/awx278/4675073

Downloaded from https://academic.oup.com/brain/advance-article-abstract/doi/10.1093/brain/awx278/4675073

Downloaded from https://academic.oup.com/brain/advance-article-abstract/doi/10.1093/brain/awx278/4675073

Downloaded from https://academic.oup.com/brain/advance-article-abstract/doi/10.1093/brain/awx278/4675073

Downloaded from https://academic.oup.com/brain/advance-article-abstract/doi/10.1093/brain/awx278/4675073

Neural implementation and reward, typically. We focus on work in primates that

Downloaded from https://academic.oup.com/brain/advance-article-abstract/doi/10.1093/brain/awx278/4675073

Downloaded from https://academic.oup.com/brain/advance-article-abstract/doi/10.1093/brain/awx278/4675073

Downloaded from https://academic.oup.com/brain/advance-article-abstract/doi/10.1093/brain/awx278/4675073

and (iii) neuromodulators adjust the weights of reward and @F @V

Downloaded from https://academic.oup.com/brain/advance-article-abstract/doi/10.1093/brain/awx278/4675073

Downloaded from https://academic.oup.com/brain/advance-article-abstract/doi/10.1093/brain/awx278/4675073

Downloaded from https://academic.oup.com/brain/advance-article-abstract/doi/10.1093/brain/awx278/4675073

Downloaded from https://academic.oup.com/brain/advance-article-abstract/doi/10.1093/brain/awx278/4675073

Downloaded from https://academic.oup.com/brain/advance-article-abstract/doi/10.1093/brain/awx278/4675073

Downloaded from https://academic.oup.com/brain/advance-article-abstract/doi/10.1093/brain/awx278/4675073

Downloaded from https://academic.oup.com/brain/advance-article-abstract/doi/10.1093/brain/awx278/4675073

Downloaded from https://academic.oup.com/brain/advance-article-abstract/doi/10.1093/brain/awx278/4675073

Downloaded from https://academic.oup.com/brain/advance-article-abstract/doi/10.1093/brain/awx278/4675073

Downloaded from https://academic.oup.com/brain/advance-article-abstract/doi/10.1093/brain/awx278/4675073

Vous aimerez peut-être aussi