For a PDF version of this article that can be printed or downloaded, click here:   

 

The Picoeconomics of Addiction

George Ainslie
Veterans Affairs Medical Center, Coatesville PA

Published in Pickard, Hanna and Ahmed, Serge, eds. (2018)  The Routledge Handbook of Philosophy and Science of Addiction. Routledge, pp. 34-44.

This material is the result of work supported with resources and the use of facilities at the Department of Veterans Affairs Medical Center, Coatesville, PA, USA, and is thus not subject to copyright.  The opinions expressed are not those of the Department of Veterans Affairs or of the US Government.

 

The core paradox of addiction is a persistent tendency of the addict to choose what she believes she doesn’t want.  Addiction is sometimes characterized by other attributes, such as dependence on a substance, withdrawal (sickness on quitting), a preoccupation that squeezes out “normal” activities, or tolerance (a progressive insensitivity to the relevant reward), but none of the first three is necessary for the core paradox (witness gambling, cocaine dependence, and smoking, respectively), and the last is not specific:  There is probably no rewarding activity that does not habituate with repetition.  Activities with one or more of the other attributes may properly be called addictions, of course— dependency without awareness, or even “willing addictions” despite a knowledge of negative consequences (Flanagan, 2016; Pickard & Ahmed, 2016); but these do not entail the puzzling feature of choosing what one consciously doesn’t want.   The nub of the problem is this apparent paradox of choice. For centuries it has remained a scientific puzzle and a moral/legal quandary.  The problem has become more urgent, for two reasons: not, perhaps, because the human susceptibility to addiction has increased, but because (1) it endures while other causes of premature death and disability have dropped away, and (2) human craft has developed addictive activities faster than it has developed protections against them.  There was a great acceleration when the growth of trade permitted worldwide sharing of opium, coca, tobacco and other natural substances, another when new techniques extracted or synthesized concentrated ingredients from them (Crocq 2007), and, arguably, a current expansion into fast-paying interactive patterns that do not depend on substances—video games, internet gambling, internet porn, and absorption in the internet itself. 
              What explains the paradox of unwanted behavior?  Theorists have long been drawn to the answer that thinking is divided into two kinds, a far-seeing process that plans consistently over time and a myopic process left over from our evolutionary roots.    Modern authors discern “visceral,” “hot,” or “type 1” thinking (Loewenstein 1996, Metcalf &Mischel 1999, Kahneman 2011, respectively) impinging on a deliberate, rational process (type 2, etc.), and have reported separate areas in the brain that might govern them (Luerssen et.al. 2015; McClure et.al. 2004).  The implication is that we have a steady, rational self that is occasionally attacked by an evolutionarily primitive process, which pressures us to choose costly short-term options before returning us to our rational state.  But although there are clearly divisions of labor in the brain, this is not an adequate explanation.
              Such dualism is intuitively appealing, but it has limitations.  For instance, there is sometimes an urge that is impulsive with respect to another impulsive urge, e.g. laziness in executing a plan that is itself short-sighted (see Ainslie 2009).  There are also familiar instances where we give in to short-sighted goals that do not arouse visceral or hot thinking, such as simple procrastination.  Sometimes the short-sighted goal is too distant to be based on arousal, as when we knowingly save too little for retirement over months or years.  Furthermore, the oft-reported individuals who prefer a hypothetical $50 now to $100 in 3 years, but do not prefer $50 in 6 years to $100 in 9 years, meet the definition of myopia but are unlikely to have been emotionally aroused by the immediate option—as demonstrated by the persistence of the pattern when a month is added to both earlier and later delivery times (Green et.al. 2005).  Appetites and emotions are certainly arousable, and increase the reward value of their objects when aroused; but many short term preferences cannot be attributed to such arousal.  There is evidence that a more general mechanism promotes addictive behavior, and that this behavior may be more widespread than is usually recognized.

A marketplace model of addiction

We are just beginning to learn how brain activity corresponds to the judgments involved in self-control, but two striking findings may help us frame the problem:  1) When human subjects face the prospect of getting real money after delays of days to weeks, the activity in their reward-sensitive regions is the inverse of the delays (Kable & Glimcher 2007), thus confirming the inverse relationship of value and delay found in many behavioral experiments.  2) There are many anatomical and functional connections in the brain that are associated with choice-making (Haber & Knutson 2010), but they do not converge on any region that is likely to house the faculty of “self,” as philosophers from Descartes to Fodor have imagined it.  Choice seems to be governed by a bidding process among any options that can replace each other, based on a common currency that is best called reward—a marketplace, not a homunculus.  As Daniel Dennett has said, “all the work done by the imaginary homunculus in the Cartesian Theater must be distributed in time and space in the brain” (2003, p. 123—his emphasis).
              (1) Psychophysical experiments have long found that changes in sensations such as brightness, loudness, and heat are proportional to the change in the strength and/or proximity of the physical stimulus.  Over the last forty years experiments have surprisingly found that the same phenomenon occurs with delay to expected rewards (Green & Myerson 2004).  Surprisingly because, although value has been known since Socrates to be discounted for delay, the form of the discounting has been assumed not to result in changes of preference over time.  That is, if you preferred a larger, later (LL) reward to a smaller, sooner (SS) one when both were distant, you were assumed to prefer it when both were close, if the lag between them stayed the same and you learned no new information.  The only formula that permits this constant relative preference is the one that banks use for interest, widely accepted after the economist Samuelson rather belatedly drew it (1937):

The surprise is that, in contrast to the bankers’ exponential formula, both humans and other animals tend to value delayed rewards in inverse proportion to that delay, in accordance with their perception of other kinds of stimuli, but not rationally for future planning. 
 

                                                                                    
Such hyperbolic delay discounting often leads to preference for an LL reward when a pair are distant, but an SS reward when they are close, as in the $50 vs. $100 example above.  People usually learn to discount the future like banks when dealing with  banks or with another smart dealer, because otherwise the dealer will money pump them—buy their winter coat cheap in the spring and sell it back to them higher in the fall, again and again until they learn consistent valuation.  But much human folly suggests that this learning is tenuous; and the presence of hyperbolic delay discounting even in rats and pigeons (albeit over periods of seconds-- Ainslie & Monterosso, 2003; Ainslie & Herrnstein 1981) shows that this pattern dates back to the evolutionary origin of psychophysical laws, probably too fundamental a part of animal design to have been selected out to facilitate human planning   Exponential discounting never becomes a basic process, but depends on learned self-control.
              (2) Whether there are one or two or ten brain centers that take part in the decision process, there must be a mechanism—a reward process-- by which any option that you can learn to substitute for any other option is weighed, one against the other (Shizgal & Conover 1996; Levy & Glimcher 2012).  Still more fundamentally, “you” comprise the substitution process, rather than an entity that stands outside of it and supervises it from above.   

Implications of hyperbolic delay discounting

If choice is entirely governed by the marketplace of reward, we might expect it to be the simple matter of calculation, the way utilitarians have long imagined.  But the finding of hyperbolic delay discounting literally throws a curve into this picture.  If you expect to change your preference away from an LL reward before you get to it, any plan for it must include a way to forestall this change—by modifying the expected SS or LL values or making the change impossible.  The ordinary process of learning paths to reward produces interests—the sets of paths that lead to and are rewarded by particular alternatives.  This is a trivial concept except to the extent that one interest has incentive to undermine another, for instance a long term interest in staying sober that must deal strategically with a short term interest in getting drunk.  On a different time scale, someone with post-traumatic stress disorder (PTSD) has a mid- to long term interest in avoiding intrusive memories, and a very short term interest in rehearsing them.  Short term interests are based on the power of rewards that are close; interests based on more distant rewards have the opportunity to act earlier, but must forestall the short term interests before they become stronger.  Neither interest will simply prevail as long as each has a prospect of succeeding, a situation that is apt to endure because hyperbolic discount curves give each interest a period of dominance (see the right hand pair of Figure 1).  Thus a marketplace model with hyperbolic delay discounting does result in dualisms of a sort, but not between different kinds of motivation.  Any alternative rewards that are available at different delays are apt to become the bases of long and short term interests, whether one, both, or neither evoke “type 1 thinking,” and over timelines ranging from split seconds to years.  Furthermore, the long range interest in one dualism can be the short range interest in another: A shoplifter in action needs to avoid distractions—short term interests with respect to her plan—but a still longer term interest is to forestall the shorter term interest in shoplifting itself (Ainslie 2009).  
              The first lesson to be drawn from hyperbolic delay discounting is that a pattern of temporary preferences and regrets is not an unusual process imposed on a person by some pathological factor.  Rather this is the normal baseline of motivation written over time.  We naturally overvalue the imminent future, and learn the ability to maintain long term plans only gradually and imperfectly.  Everyone struggles with bad habits.  If addiction is defined with a low threshold, half the people in America are addicted to something (Sussman et.al. 2011).  Those of us who have avoided the named addictive diagnoses are nevertheless apt to suffer from habitual overvaluation of the present moment, as in chronic procrastination, overuse of credit, or unrealistic future time commitment.  So the problem for the science of addiction is not an addict’s susceptibility to temptation, but why she fails to use her culture’s shared knowledge to counteract it in specific areas over part of her life—and then, usually, succeeds again.  (Most substance addicts eventually recover without treatment—Heyman 2009.) 
              The next lesson is that the mechanisms often proposed for addictions all depend on motivation (Ainslie 2016).  Here I describe the three prominent proposals for which hyperbolic discounting has the greatest implications.

Three explanations for addiction re-interpreted

1. Weak will. A universal tendency to form temporary preferences makes commitment necessary for consistent behavior.  Acting in one’s long term interest there are simple ways to forestall temptations: keep your attention away from them so they do not enter the marketplace, abandon the relevant appetite or emotion before it gets too strong, or find external influences (for instance Alcoholics Anonymous) or commitments (for instance disulfiram=Antabuse).  But all of these have serious drawbacks:  Attention is hard to divert for long; appetites and emotions are rewarding in their own right; other people have their own agendas, and neither they nor physical commitments may be available when you need them.
              The tendency of hyperbolic curves to level out at long delays permits a more sophisticated strategy: The influence of the LL rewards (height of their curves) grows faster than that of the SS rewards when a series of choices is summed, so an arrangement to make a whole series of SS/LL choices at once favors the LL options (Figure 1; details in Ainslie 2005).   Such an arrangement is apt to form spontaneously in anyone who notices that her current SS/LL choice predicts how she will make similar choices in the future: Then her expectation of getting LL outcomes in the set of similar choices will be somewhat determined by her current choice.  This perception is apt in turn to become a factor in making the current choice, and each subsequent choice that looks similar.  Each LL choice increases, or preserves, her expectation of getting the whole series—or bundle—of LL rewards, and each SS choice decreases this expectation, a process of recursive self-prediction.

              The stake of LL reward may be aggregated from the evident consequences of each choice, as in binges with hangovers, or the stake may be a more distant anticipated condition such as good health or adequate savings.  The necessary element for the self-prediction process is that the person’s expectation of getting a category of LL reward as a whole is put at stake when each opportunity within a definable set of SS rewards occurs.  This self-prediction process is recursive, in that each estimate of future self-control is fed back into the estimating process.
              Once someone has noticed this phenomenon she is apt to propose bundles deliberately: resolutions, or diets, or tests of character.  In effect she is defining a game resembling a repeated prisoner’s dilemma, intertemporal bargaining with her expected future selves, stabilized by self-enforcing contracts in which single defections (SS choices) have more impact than single cooperations (LL choices—modeled in Monterosso et.al. 2002).  The reason the dieter doesn’t eat the single chocolate éclair is not that it will cause a noticeable gain in weight, but that it will damage the credibility of her diet.  The struggle between impulse and control now turns not so much on how close she gets to a temptation—although this remains a factor—but on whether she expects a later self to see her current choice as a defection and thus have less reason in turn not to defect.  Her self-prediction has become the basis of personal rules.  The bundling tactic has the greatest impact when a person sees the stake as her belief about her character—whether she is “the kind of person who” abuses her children, is the slave of an addiction, or has a disgusting paraphilia.  The game theory of such high-stake “self-signaling” was formally worked out by Ronit Bodner and Drazen Prelec (2003). 
              Intertemporal bargaining forms willpower—a weapon against addictions—but also raises the stakes of choice in ways that can strengthen an addiction:

        “Loss of control.”  A person may have recruited great differential motivation in an important set of choices, so a single defection can cause a spectacular collapse—the “abstinence violation effect” (Curry et.al. 1987).  Then to save her expectation of controlling herself generally, she will be strongly motivated to find a line excluding the kind of choice where her will failed from her larger rule. This means attributing the lapse to a particular aspect of that situation, even though it will make self‑control more difficult when that aspect is present in the future.  She may decide that she can’t resist the urge to eat jelly donuts, or to smoke after meals, or accept cocaine when offered by a friend—or resist that particular modality of temptation at all.  The result is a circumscribed area of dyscontrol.

        Cognitive  blocks.  Since personal rules organ­ize great amounts of motivation, they create an incentive to suborn the self-perception process.  The potential damage from lapses creates an incentive not to see them, giving rise to a motivated unconscious: suppression, repression, denial.

          Compulsiveness.  Choices may become more important as test cases than for their own sakes, making it hard to live in the here-and-now.  Awareness that damaging a personal rule may threaten to de-stabilize a larger network of intertemporal bargains may lead to an unwillingness risk modifying the rule.  
              Thus intertemporal bargaining stabilizes not only long term plans but also ways around them.  It lays down a history of choices that have turned out either well or badly, leading to modifications in your personal rules.  The logic is the same as for how court decisions over time lay down the English and American common laws.  Most psychological therapies aim at unpicking the tangle of overgrown resolutions and lapses that individuals have developed in their various attempts at rule-making: “cognitive maps” (Gestalt), “conditions of worth” (client-centered), “musturbation” [sic] (rational-emotive), “overgeneralization” (cognitive behavioral), and of course the punitive superego (summarized in Corsini & Wedding 2011).  Unfortunately, these therapies have not had a good record against addictions. 
              Short term interests based on addictions are aggressive bargainers by definition.  They may make it necessary to cut the Gordian knot of intertemporal bargains and focus concretely on a single target.  The Anonymous organizations’ concept of helplessness should be understood not as a negation of reward bundling, but as a warning against the compromises that ordinary self-control invites (see Monterosso & Ainslie, 2007, p.S107).  How to maximize the effect of a bundle of LL choices while leaving open the idea of recovery from a possible abstinence violation effect is a conundrum for all addiction therapists—logically, anticipating the latter weakens the former.  For rebuilding trust AA cautiously adopts the insight of some religions in invoking the influence of a “higher power,” which cannot be presumed upon to grant grace and thus remains credible after failures (see the Saint X effect in Ainslie, 2001, pp. 107-108). 

2. Habit.  The persistence of addictions despite deteriorating reward has suggested the relevance of behavioral experiments on habit—the persistence of a choice after it is no longer rewarded.  Some kinds of brain lesions make rats unable to change overlearned responses when reward contingencies change (Everitt & Robbins 2013).  Addiction to stimulants in particular has been accompanied by suggestive changes in human brain activity.  However, although addicts have been observed in laboratory choice-making tasks to respond less well than nonaddicts to changed information, this difference has usually been moderate, as it has been even in patients with gross brain lesions (Fellows & Farah 2005).  Furthermore, overlearning is of questionable relevance to choices about whether or not to consume substances; addictive “habits” have very little to do with mindless repetition, but on the contrary require a high degree of flexible, goal-directed behavior to evade a hostile society. 
              In addition to the tangle of personal rules just described, hyperbolic discounting predicts a simple mechanism for apparent habit.  Devalued addictive reward does not lose its power evenly over a consumption episode; initial phases stay rewarding, at least relative to the alternatives still available, but no longer last as long.  Given its reduced average value the consumption may look like robotic repetition, while hyperbolic overvaluation of the near future keeps it the dominant choice.  A recurring urge followed by disappointment has the time pattern of an itch, but the path of least resistance is still to scratch the itch—light another cigarette, eat another snack—rather than tax a depleted willpower fighting recurrent urges.  The activity is still based on reward (see Ainslie, 2017a)..

3. Ignorance. Motivational science has necessarily studied the control of behavior by external rewards, and so has not permitted explanations that do without such rewards.  Addiction was long assumed to arise from the action of a drug.  The obvious exception of gambling addiction was attributed to delusions about how the laws of probability govern the ostensibly rational goal of getting money.  However, the power of internet-assisted and video gaming makes it hard not to notice how bets become rewarding in their own right—without promising to deliver anything but outcomes per se.  The importance that people attach to purely symbolic prizes is familiar, of course, but a motivational model in which learned (“secondary”) rewards must predict some innately rewarding (“primary”) event has seemed necessary, as has the location of that event beyond the subject’s control (Baum 2005, pp. 277-286).  However, such a mechanism does not credibly deal with the problem of activities that we do not expect to lead to an ultimate product.  This is a topic where the problem of addiction reveals the need to greatly broaden the conventional theory of reward, with implications for many non-addictive activities as well.  Mathematical modelers have begun to explore the notion that reward can be generated internally, but they still assume that such rewards come from innately programmed outcomes, “hard-wired from the start of the agent’s life” (Singh et.al 2010, p. 73). 
              The theoretical problem has been that coining your own reward might be expected to produce short circuits in which people reward themselves autistically—Indeed this is literally seen in the stereotyped behaviors of severe autism and, at a higher level, in the preoccupations of fantasy-prone personalities (Rhue & Lynn 1987).  However, the ability to reward oneself at will is familiar enough.  With a physical reward like food, availability at will makes its effect depend on appetite.  Appetite builds potential payoffs disproportionately to their delays, so although hyperbolic discount curves create urges to harvest reward as soon as possible, your long term interest is to cultivate the appetite— to “work it up,” or make a personal rule not to eat between meals.  The same principle is true of self-generated reward.   Fiction, games, even daydreams require an appetite factor such as suspense or curiosity to be satisfying.  You can harvest this appetite, as it were, at will—look ahead to the end, cheat at solitaire.  But the earlier you harvest the appetite, the less well it pays.  Appetite builds as a function of time, and is accelerated by challenges as in thrill-seeking, gambling, or video games.  The combination of challenge and rapid success might make any behavior addicting, as Foddy has pointed out (2016).   Just assuming for illustration that appetite builds linearly and is consumed linearly over time, the hyperbolically discounted values simply of unobstructed versus obstructed consumption patterns can take the familiar SS/LL form (Figure 2).  In this example, the obstructed pattern is worth waiting for from the perspective of distance, but requires commitment if it is to stay chosen when the unobstructed pattern becomes imminently available.

              Where the harvesting depends on physical objects such as books or cards, personal rules about it are easy to enforce.  Where the harvesting is entirely mental, ways to pace it are more conjectural.  Rules for such harvesting necessarily begin in discerning criteria—occasions-- for payoffs that are either external or hard to create.  Adoption of such criteria might best be called betting: on occasions in a book or card game, on whether a sports team will win on TV, or on whether you can master a task—which can be a mental task, as long as benchmarks for mastering it are clear.  The personal rule is to maintain the appetite until the occasions occur, instead of redefining the occasions, or withdrawing your attention to alternative bets or to the expectation of physical rewards.  A mental action that spoils a bet is less conspicuous than turning a page or card.  If you withdraw your investment in a movie when it gets too scary, you may or may not be conscious of saying to yourself, “it’s only a story;” but in either case you have reduced your potential to be rewarded by its subsequent events.  Likewise you may or may not be conscious of rooting less for a sports team, or valuing a friendship less, but you are apt to notice a change in your relevant payoffs.  As with other personal rules, seeing yourself pull investment from a movie will make you less likely to resist the urge to do so later, and not just in this movie but in other movies and perhaps other projects.
              The potential endogenous reward that depends on a bet could be called its hedonic importance, as opposed to an instrumental importance based on external reward.  The great weakness of hedonic bets is their arbitrariness, the fact that they can as well be based on any number of scenarios leading to any number of outcomes, as in daydreams.  This weakness can be overcome by betting on singular occasions—those that stand out from other possible occasions by being infrequent and easily distinguishable from those that are more frequent.  Such bets win out in the marketplace of choice.  Singularity may consist in being a remote coincidence, a round anniversary, a great exercise of skill, an event close to home or to the present moment, and so on (Ainslie 2013a).  Two sources of singularity make occasions especially effective in pacing endogenous reward: a history of your having bet on them, and, perhaps confusingly, their realistic prediction of external rewards. 
              A history of betting recruits importance recursively.  You find a problem or a book or a sports team important from having spent attention on them, and you make them more important by spending more attention on them.  This recursive process has obvious similarities to intertemporal bargaining, except that the alternatives do not necessarily serve long vs. short term interests; a withdrawal of importance may represent just a change in taste rather than a defection.  However, accumulated importance is an investment, “consumption capital” (Becker & Murphy 1988), which may bring satisfactions in art or philosophy but may also create obsessions or dares— absorption in gambling, increasingly risky shoplifting, building a video game score, self-destructive mountain climbing (Leamer 1999).  For want of a motivational explanation the role of hedonic importance in sustaining addictions has often been dismissed as habit.
              Goals that are objectively important are often singular as well, so their instrumental benchmarks may serve as occasions for hedonic reward in addition to being secondary external rewards-- “Getting there is half the fun.”  It is difficult to distinguish the two processes by observation (Lea & Webley 2006), which means that ostensibly productive activities that are good pacers of endogenous reward are apt to become potent process addictions in disguise, such as “skilled” gambling, day trading of stocks, or dealing in collectibles.  Arguably the crossed purposes of increasing instrumental productivity versus cultivating appetite underlie much misinterpretation of economic activity in general, but that is another topic (Ainslie 2013b).
              The implication of an endogenous reward process is that strong motivations can arise from nothing but a source of occasions, and, unlike secondary rewards, grow substantially with practice (see Ainslie, 2017b).  Sometimes the result is long term artistic pleasure or sublimation, but occasions can also be structured to invite hedonic importance onto faster-paying patterns that can become addictive.  This would seem to be the object of modern gambling technology and the video game industry (again, see Foddy, 2016).  The availability of big data  to evaluate patterns of occasions can only make this kind of addiction grow more inviting.

So is addiction a disease?

A third, but probably not final, lesson from hyperbolic delay discounting is that “motivated” does not mean “voluntary.”  Will is a learned executive function—intertemporal bargaining—that re-interprets patterns of expected reward to reveal incentive for consistent choice.  This is the skill that society holds us responsible for maintaining.  But sometimes a personal history of bargaining has left very little expected reward to be bargained with, raising the question of whether some addicts “can” stop their activity.  That is the crux of whether hopeless addiction can be called a disease.  I have proposed elsewhere that it is better seen as a bankruptcy (2011), although, unlike financial bankruptcy, it sometimes responds to sudden shifts in bargaining that re-focus motives (e.g. Miller & C’de Baca, 2001).  If we want to call it a disease, we need to recognize that it is a disease of motivation, that is, one that does not bypass the mechanism of choice.  Identifying it as such is only the start of identifying its incentives.

 

Notes

1. Hyperbolic discounting suggests how negative experiences can compete with positive ones: the negative experience must have a positive phase, experienced as an urge, long enough to attract attention; so intrusive memories, rages, fears, and even physical pains must lure you into participation.

2. Of course what literally governs the bidding in the internal marketplace is the prospect of reward, which does not itself occur until after the choice is made; but choosing according to prospects is also the process in literal markets. 

 

References