1
Addiction as Maladaptive Learning, with a Focus on Habit Learning

Theresa H. McKim and Charlotte A. Boettiger

Introduction

Addiction is a chronic, relapsing disorder in which individuals typically cycle between periods of sustained, compulsive drug use and abstinent periods of varying durations. Such relapse to drug use despite negative consequences is a key criterion for addiction and is one of the most troubling aspects of addictive disorders (APA, 2000). This disregard for action consequences is a feature of addiction that is thought to reflect, in part, maladaptive associative learning consequent to extensive exposure to the reinforcing properties of drugs of abuse (Ostlund & Balleine, 2008; Balleine & O'Doherty, 2010; Belin, Belin-Rauscent, Murray, & Everitt, 2013). Drug use behaviors recruit and engage the same neural circuits as those engaged during normal learning and memory processes (Koob & Volkow, 2010); however, the reinforcing properties of drugs of abuse are thought to dramatically enhance the representation of behaviors associated with drug use and the association of such behaviors with related stimuli. The normal processes of learning involve establishing circuits through which stimuli can come to drive heavily repeated, stereotyped actions (Dickinson, 1985). Such actions are habit-based rather than goal-directed, and establishing such habit circuits allows for efficient response selection, freeing cognitive resources for other processes. However, by their nature, habitual actions are under the control of triggering stimuli rather than determined by the outcome of those actions. As a consequence, it is difficult to suppress these actions even if their outcome turns negative. This process is thought to underlie habitual drug-seeking and drug-taking behaviors, which are very difficult to eradicate despite their increasingly negative outcomes. Compounding this issue is the fact that repeated exposure to drugs of abuse primes and potentiates reliance on habitual-action circuits and alters associative learning behaviors (Belin-Rauscent, Everitt, & Belin, 2012; Hogarth, Balleine, Corbit, & Killcross, 2013). Therefore addiction is appropriately considered a neurobehavioral disorder of maladaptive learning that results from chronic drug use.

Associative learning in the context of addiction has been most widely investigated in terms of the association between addiction-related stimuli and the initial, positively reinforcing, properties of drugs of abuse. According to the incentive sensitization theory proposed by Robinson and Berridge (1993), repeated pairings of such stimuli with the drug attaches "incentive salience" to previously neutral stimuli, which then enables these drug cues to elicit "wanting" of the drug that facilitates drug-seeking and drug-taking behaviors. This craving state comes to dominate drug use, whereas the hedonic value of the drug, or drug "liking," typically drives initial drug seeking but decreases over time with chronic drug abuse. Initial drug-seeking and -taking behaviors are thus goal-directed in nature, as they are motivated by expected positive reinforcement. Although the positively reinforcing properties of drugs of abuse tend to decline over time, drug seeking may remain goal-directed through negative reinforcement processes (e.g., to avoid the aversive consequences of drug withdrawal). However, in the context of addiction, a shift toward compulsive drug use that is impervious to the increasingly negative consequences of drug-seeking and -taking behaviors results in maladaptive habit-based responses.

The ability to establish habitual (or automatic) actions is highly adaptive outside of the addiction context. In fact a high degree of automaticity is required to successfully execute certain high-level motor procedures, such as musical or athletic performance. In the course of daily life, interactions with the environment engage associative learning processes that allow individuals to perform behaviors automatically. Such automaticity confers adaptive value through its efficiency: automatic responses to familiar stimuli free cognitive resources for application to more demanding conditions. For example, such resources are needed in contexts where response contingencies are uncertain due to novelty or to changing environments. In such cases, optimal responding must be adaptively adjusted, and cognitive flexibility is necessary to allow such response dynamics. During initial learning, associations between stimuli and responses are formed by goal-directed actions shaped by the contingent outcomes of behavioral responses (Balleine & Dickinson, 1998). Such behavioral responses are initially flexible, which allows individuals to adapt their behavior in the face of changing outcome values and to maximize positive outcomes. Repeated practice facilitates behavioral autonomy, ultimately enabling behavioral response selection driven by stimulus-response (S-R) associations instead of action-outcome associations. Thus S-R (or habitual) behaviors are no longer under the control of the response outcome or goal and are instead stimulus-bound actions (Dickinson, 1985). These more rigid habitual behaviors typically support efficient interaction with the environment; however, such habit-based actions also underlie pathological behavioral patterns that are difficult to change and, in the context of addiction, theoretically promote compulsive drug use and increased susceptibility to relapse.

The aim of this chapter is to provide an overview of the growing evidence to support the role of habit learning in perpetuating addictive behaviors despite diminished reinforcing properties of the drug and increasingly negative consequences of continued drug use. Most of the research for this framework to date stems from animal studies, which will be touched upon briefly to discuss translational methods for the study of habits. We will then discuss the challenges of translating findings from animal models to studies with humans, including substance-abusing populations, within the laboratory, and then we will review the small number of recently published studies of habit-based response in human substance abusers. Finally, we will propose the use of habit learning as an intermediate phenotype for study in various populations, both clinical and nonclinical, and will discuss the possible societal implications of better understanding habit-based behaviors.

Foundations for Studying Habit Behaviors: Animal Studies

To experimentally distinguish goal-directed from habit-based responding, two behavioral tests are standardly used in animal models: contingency degradation and outcome devaluation (Dickinson, 1985; Ostlund & Balleine, 2008). In contingency degradation tests, animals are first trained to execute two distinct actions (e.g. pressing two different levers) in order to learn the particular reward outcome associated with each action - for example, receiving a chocolate chip for pressing lever "A," and receiving a mini marshmallow for pressing lever "B." After lever pressing is well established, one lever retains the same probability of reinforced lever presses, while the outcome associated with the other lever is delivered noncontingently with lever pressing. If the animal stops pressing the lever previously associated with the now degraded outcome (which is delivered regardless of lever-pressing behavior) but continues to press the lever with intact reward contingency, the lever-pressing behavior is considered goal-directed. In contrast, continued pressing of the lever with the degraded response contingency is considered evidence of habit-based responding. An alternative test of habit-based responding is outcome devaluation, which changes the current motivational value of a reward through either sensory-specific satiety or conditioned taste aversion. Sensory-specific satiety temporarily devalues a specific reward by providing an individual with free access to consume that reward prior to a test of lever-pressing behavior. Conditioned taste aversion devalues a normally rewarding outcome by pairing consumption of that reward with injection of a sickness-inducing compound (e.g., lithium chloride, LiCl). These devaluation manipulations occur after initial training and before testing of lever-pressing behavior. During the test session, if animals suppress responding for the devalued outcome, the lever-pressing behavior is considered goal-directed, whereas continued, habitual behavioral responses persist due to a cached value representation that does not allow for updating changes in value.

Extensive research into the neural bases of goal-directed actions versus habitual responding based on S-R associations - research carried out in animal models - points to an interplay between the striatum and the prefrontal cortex (PFC) in these processes (Coutureau & Killcross, 2003; Killcross & Coutureau, 2003; Yin & Knowlton, 2004; Yin, Knowlton, & Balleine, 2004; Yin, Knowlton, & Balleine, 2005; Yin, Knowlton, & Balleine, 2006; Yin, Ostlund, & Balleine, 2008; Tran-Tu-Yen, Marchand, Pape, Di Scala, & Coutureau, 2009; Izquierdo & Jentsch, 2012; Smith, Virkud, Deisseroth, & Graybiel, 2012; Rhodes & Murray, 2013). Sensitivity...