What is it called when a neutral stimulus is paired with a previously conditioned stimulus to become a conditioned stimulus as well this?

4.1.3 Classical-conditioning procedures

Neutral stimuli can also come to elicit drug-like responses via classical conditioning. A series of studies in non-humans, for example, demonstrated that formerly neutral stimuli come to elicit signs of opioid withdrawal via pairing with the withdrawal syndrome (Wikler and Pescor, 1967; Goldberg and Schuster, 1970). This research was subsequently extended to humans (O’Brien et al., 1975, 1977). In that research, opioid-dependent volunteers received a low-dose intramuscular injection of naloxone, an opioid antagonist, which elicited a mild opioid-withdrawal syndrome. Across 12 training trials, this operation was paired with a specific tone and odor. When the tone and odor stimulus was presented on test trials, but with saline substituted for naloxone, subjects exhibited discernible signs of opioid withdrawal, although of a lower intensity than was elicited by naloxone injections. More recent developments in this area have tried to apply these conditioning principles in the treatment of drug dependence as relapse-prevention strategies (Childress et al., 1987).

Classical conditioning occurs when neutral stimuli become associated with a psychologically significant event. The main result is that the ‘neutral’ stimuli come to evoke responses or emotions that can contribute to many clinical disorders. Recent research emphasizes the fact that conditioned stimuli evoke whole systems of physiological, emotional, and behavioral responses that help the organism prepare for the significant event. Basic research on classical conditioning has many other implications for understanding the development of clinical disorders, including (but not limited to) anxiety disorders and drug dependence, as well as their therapy.

Pavlovian Conditioning

If a “neutral” stimulus (e.g., a bell) reliably precedes, usually in close temporal proximity, a stimulus that reliably and persistently elicits behavior (e.g., food in the mouth), then people and animals begin reacting during the neutral stimulus (e.g., by salivating) in way that prepares them for the impending stimulus. Although few if any stimuli are neutral in the sense that they do not elicit any behavior, behavioral psychologists consider a stimulus to be neutral when any behavior it elicits readily wanes with repeated presentations of the stimulus (i.e., responding habituates). A ringing bell, for example, might initially elicit an orienting response directed toward the sound, but this action will disappear with repeated ringing.

A stimulus that comes to elicit behavior after being paired with the stimulus that elicits behavior is a conditional stimulus (CS). A stimulus that reliably and persistently elicits behavior resistant to habituation is an unconditional stimulus (US). The responses elicited by the CS and the US are the conditional response (CR) and the unconditional response (UR), respectively. The procedure for changing behavior when two or more stimuli are paired is Pavlovian, classical, or respondent conditioning.

Although many examples of Pavlovian conditioning involve biologically significant USs such as food or water, the US does not have to be biologically significant. For example, imagine that when Jane's grandparents visit each week, they give her $20 when they arrive. After several pairings between these relatives and the money, how will Jane react when her grandparents visit? She will probably be happy and expecting money at the sight of her grandparents at the door because her grandparents (CS) precede the occurrence of money (US), which normally elicits a constellation of positive emotional responses (URs) when she receives it. By reliably giving Jane $20 when they arrive, the grandparents come to elicit a set of similar responses (CRs). However, if these relatives stop giving Jane the $20 gift when they visit, then she will become less happy and less likely to expect money when she sees them. That is, when the CS no longer predicts the US, the CR weakens and might eventually disappear. Presenting a CS without the US is termed extinction.

But just as the passage of time without a stimulus causes the spontaneous recovery of a habituated response, so too does a period of time following the extinction of a CR cause spontaneous recovery of that response. If Jane's grandparents stop visiting for a few weeks after extinction had occurred, and then following this break they show up again at Jane's door, their arrival might once again elicit positive emotional responses. Extinction itself wanes when the CS on which it is based stops occurring.

Stimulus generalization and discrimination also affect Pavlovian conditioned behavior. Generalization would be evident if Jane was happy to see her grandmother when she visited without her husband. If visits by the grandmother alone did not elicit positive responses, then stimulus discrimination had occurred. Thus, a CR elicited by a CS might also be elicited by other stimuli. The more similar a stimulus is to the CS, the more likely it will elicit a strong CR; the more a stimulus differs from the CS, the weaker the CR it elicits. Explicit stimulus discrimination training could also occur when visits by Jane's grandparents together predicted a gift of $20, but visits by either the grandmother or grandfather alone were never followed by a gift. The presence of either grandparent alone would not elicit as strong a reaction as when they visited together. A stimulus that is never followed by a US becomes an inhibitory CS or CS–, a stimulus that suppresses behavior. Differential stimulus training is yet another way in which stimuli come to control behavior.

Not only is the correlation between the CS and US important, humans' and other animals' evolutionary histories predispose them to learn some stimulus associations differently from other associations. Perhaps the most well known example is taste or flavor aversion. Flavor aversion learning occurs when people and animals avoid gustatory and olfactory cues associated with gastrointestinal illness. There are two interesting facts about flavor aversions: (1) the flavor elicits illness and avoidance of the flavor after just a single pairing and (2) flavor aversion occurs even when the flavor is separated from the illness by many hours. In most circumstances, Pavlovian conditioning works best when the neutral stimulus repeatedly precedes the US by just a few seconds. The unique characteristics of flavor aversion probably evolved because most gastrointestinal illnesses are caused by the ingestion of toxins, some of which are life-threatening and have effects that might not be experienced for several hours. An organism that could associate a flavor with illness even if many hours separate this pairing has the advantage of not repeating its mistake. In this manner, natural selection could have favored organisms that learned flavor–illness associations after a single trial and when the flavor and illness were separated by many hours.

1 Basic Terms

The pairing of an initially neutral stimulus (the conditioned stimulus—CS) with a biologically relevant stimulus (the unconditioned stimulus—US) comes to elicit a response (conditioned response—CR) that is usually but not always similar to the response previously associated with the unconditioned stimulus (the unconditioned response—UR). In fear conditioning, the US is an aversive fear-eliciting stimulus such as painful electric shock or loud noise, the CS is a neutral tone or light stimulus. The unconditioned and the conditioned response consist of changes on the subjective, the behavioral and the physiological level and include (in humans) enhanced subjective fear and responses such as freezing, changes in heart rate and skin conductance, the release of stress hormones, reduced pain sensitivity and startle reflex potentiation.

The development of the CR is based on the formation of an association between a neutral stimulus and a stimulus with innate biological significance (Rescorla 1988). Most studies involving fear conditioning have used cue rather than context conditioning, i.e., discrete CSs were presented rather than using the environment of the animal (e.g., the cage) as CS. In addition, delay conditioning where the CS terminates with the US rather than trace conditioning where the CS and US are separated in time were used in most studies. Fear can be viewed as a specific reaction to threatening stimuli. It can turn into an anxiety disorder when the fear becomes disproportionate to the stimulus that elicits it or when fear is experienced in inappropriate situations.

Classical Conditioning

Classical conditioning occurs when a previously neutral stimulus (e.g., a tone) is paired with a stimulus that elicits unconditionally a reflex of some kind (e.g., the suppression of respiration rate or heart rate). With multiple pairings of these stimuli, the previously neutral stimulus (conditioned stimulus) begins to elicit a response very much like the unconditioned reflex response. The occurrence of this response to the conditioned sound stimulus is an indication of hearing. Then the methods of psychophysics are applied to the conditioned stimulus. For example, sounds may be presented with decreasing intensity (e.g., sound pressure level) until the conditioned response fails to occur. The intensity level at which the response fails can be defined as the threshold for hearing that sound. Then, the frequency of the sound is changed and the experiment repeated. A plot of the thresholds as a function of the frequencies tested is the audiogram. It describes the weakest tones that are detectable, and the frequency range of hearing. An audiogram for the goldfish (Carassius auratus) is shown in Figure 1 using these methods.

The audiogram can be interpreted as follows: at all frequencies tested, sound pressures above the audiogram line are detectable, while those levels below the line cannot be heard. The frequency range for the goldfish can be described as from 30 Hz or below to about 2500 Hz. The best frequency of hearing is about 350 Hz. One micro-Pascal (1 Pa = 1 N m−2) is the unit of sound pressure against which all sound pressure thresholds are compared. We have confidence in this audiogram because many physiological and behavioral experiments on goldfish confirm that it is specialized for detecting sound pressure (it has Weberian ossicles), and sound pressures were manipulated and measured in this experiment. The greatest sensitivity for goldfish (about 60 dB re: 1 μPa underwater) is comparable to the best sensitivity for humans in air, and many other vertebrate animals. The goldfish is the most studied of fish species in terms of hearing experiments.

2.17.3.3 Emotional Stimuli Are Distinctive

Emotional stimuli stand out relative to neutral stimuli because they have unique attributes not shared by neutral stimuli (Schmidt, 1991). For example, compared to a picture of a child walking to school, a picture of an injured child has unique objective features (e.g., facial expression) because it triggers feelings and unique cognitive processes (e.g., reflexive attention and appraisal of goal relevance). Schmidt suggests that emotional stimuli are distinct in both an “absolute” and a “relative” sense, compared to other items in long-term and working memory, respectively. For example, a picture of a woman wearing a dress is not absolutely distinct, but it would be distinct relative to a set of pictures of nude models (Schmidt, 2002).

Distinctive items naturally capture encoding resources, but only relative distinctiveness reliably enhances memory (Schmidt, 2002, 1991). For example, memory for a picture of a clothed model was better than memory for a picture of a nude model when both were presented mixed within a set of nude models (Schmidt, 2002). Relative distinctiveness can capture attention at encoding, but even when it does not, it can enhance the competitive advantage of distinct stimuli at retrieval. Thus, for example, when common and bizarre sentences were studied together, bizarre sentences were only remembered better than common sentences when participants were asked to recall them together, but not when they were recalled separately (McDaniel et al., 2005). In the emotional memory literature, Dewhurst and Parry (2000) first demonstrated that primary distinctiveness modulates the EEM by manipulating list composition in a remember–know recognition paradigm. In their study, positive and negative words were remembered better than neutral words when they were all presented together in lists of “mixed” valence, but not when words of each valence were presented in separate “pure” lists. Therefore, even though emotional words were always distinct in an “absolute” sense, this did not produce a memory advantage in the pure lists, suggesting that only relative distinctiveness contributes to the EEM. This conclusion was supported by several studies where EEM in free recall was stronger when items were studied and recalled together than when they were studied and recalled separately (Hadley and MacKay, 2006; Schmidt and Saari, 2007; Talmi et al., 2007a; Talmi and McGarry, 2012).

Imaging the Acquisition of Emotion

In classical or Pavlovian conditioning, a neutral stimulus, through pairing with an emotional stimulus (e.g., an aversive noise in fear conditioning), acquires the ability to predict its future occurrence. Associative learning provides a highly conserved means by which organisms acquire knowledge regarding the causal structure of the environment. Such knowledge endows an organism with the ability to anticipate future events of value, such as the likelihood of food or danger, on the basis of predictive sensory cues. The amygdala plays a central role in this type of causal learning. Thus, amygdala activation is seen when a previously neutral item (the conditioned stimulus (CS+)) acquires predictive significance through its pairing with a biologically salient reinforcer (the unconditioned stimulus (UCS). This pattern of activation is seen both for aversive and appetitive events. Thus, across a wide range of experiments, an enhanced amygdala response is evident when subjects learn that a neutral stimulus predicts the occurrence of an emotion-inducing event, for example, an aversive event.

Theoretical accounts of associative learning based on the Rescorla–Wagner rule and their real-time extensions, such as temporal difference (TD) reinforcement models, provide plausible descriptions of the computational processes underling associative learning. Crucial to these models is the expression of a teaching signal, referred to as the prediction error. A prediction error is used to direct acquisition and refine expectations relating to predictive cues, and it records a change in expected affective outcomes, being expressed whenever predictions are generated,updated, or violated. Functional neuroimaging studies have demonstrated that such a prediction error is expressed in the human brain in structures such as amygdala, striatum, and orbital-prefrontal cortex (OFC) during aversive and appetitive associative emotional learning.

Fear Conditioning

Fear conditioning is the learning that a neutral stimulus predicts the appearance of an aversive event. The combination of a neutral (conditioned) stimulus and an aversive (unconditioned) stimulus renders the formerly neutral stimulus a frightful quality, so that even when it appears by itself, without the aversive stimulus, it will elicit a fearful conditioned response (Maren, 2001). Fear conditioning can be rapidly formed in humans and animals, even following a single conditioning trial, and is usually maintained for long periods (Maren, 2001). In rodents, the dominant behavioral fear response is freezing (complete immobility), and the most commonly used aversive stimulus is the delivery of a weak, short electrical shock (Fanselow, 2000). The conditioning process itself (i.e., the association between the neutral and aversive stimuli) is mediated primarily by the amygdala (Maren, 2001; Maren and Quirk, 2004), and by using different types of conditioned stimuli, the fear-conditioning paradigm enables differentiation between hippocampal-dependent and -independent functions: When a simple perceptual conditioned stimulus (such as a light or an auditory cue) is used, the hippocampus is not required. However, when the conditioned stimulus is a new environment, a mental representation of this new context has to be created so that the amygdala can associate this representation with the aversive stimulus; this function depends on hippocampal functioning (Fanselow, 2000; Maren and Holt, 2000). In the most commonly used version of the fear-conditioning paradigm, animals are placed in a novel conditioning cage, in which they hear an auditory tone, followed by a brief foot-shock. Thus, the animal can associate the aversive stimulus with the new context as well as with the tone. To test contextual fear conditioning, animals are placed again in the original conditioning cage, and freezing is measured. This task is hippocampal dependent, as it cannot be performed following hippocampal lesions (Fanselow, 2000). To test the hippocampal-independent auditory-cued fear conditioning, freezing is measured when the tone is sounded in a differently shaped context (Fanselow, 2000; Maren and Holt, 2000). In this case, hippocampal lesions have no detrimental effect on performance level (Fanselow, 2000). The fear-conditioning paradigm has been employed extensively in research on the neural basis of memory in various pharmacological, genetic, and environmental studies (Fanselow and Poulos, 2005).

3.14.5 Reward Timing in Primary Sensory Cortex

While perceptual learning can occur passively to neutral stimuli, plasticity can be both accelerated in rate and exaggerated in magnitude if conditioned stimuli are paired with reward or punishment (Edeline et al., 1993). Much of the work on this kind of associative learning in primary sensory cortices has focused on primary auditory cortex (A1) (Bakin and Weinberger, 1996; Froemke et al., 2007; Letzkus et al., 2011), but stimulus–reward associations also occur in V1. Such associations have perhaps most clearly been demonstrated in a series of experiments in which water rewards were delivered at a particular temporal interval after presentation of a light flash, through goggles, to one or other eye in rats. Recordings of action potentials elicited by individual neurons in binocular V1 revealed that the timing of activity driven by the visual stimulus could be strongly influenced by the repeated paired delivery of a reward (Fig. 2D–F; Shuler and Bear, 2006). Some neurons extended their “firing” to the moment of reward delivery, while others exhibited a second response to the reward delivery. These neurons continued to show a modified response to reward even in the absence of reward delivery, demonstrating that a process of memory was enabling the neuron to respond in a predictive manner to expected reward. Once again, an obvious query arising from this result is where the actual underlying plasticity has occurred: Local to V1 or at another locus that evokes echoes in V1. Tellingly, the temporal interval between visual stimulus and reward was varied for each eye in an individual animal, such that a 1-s delay would occur for one eye and a 2-s delay for the other eye. Importantly, neurons exhibited eye-specific responses appropriately timed to the delivery of reward for that particular eye. Thus, similar to some of the psychophysical evidence described elsewhere, it is likely that underlying plasticity must occur prior to integration of inputs from the two eyes (i.e., V1 or even more proximal to visual input).

Given a wealth of evidence from former work on reward timing (Hollerman and Schultz, 1998), one might expect the neurotransmitter dopamine to play a critical role in signaling reward in V1. However, there is remarkably little dopaminergic input to V1 in the adult animal (Papadopoulos et al., 1989), meaning that other systems are likely at play. Lesions of cholinergic inputs to the cortex using the retrograde neurotoxin Ig-saporin demonstrated that it is cholinergic input from the basal forebrain that mediates the visual reward timing described earlier (Chubykin et al., 2013). Two key points to be made here are, first, that this effective blockade of learning was selectively applied in V1, confirming through an interventional approach that observed changes in neural response reflected plasticity local to V1 and, second, that loss of cholinergic input through this method prevented learning of reward timing but did not disrupt already established timed responses in V1 neurons. Thus, the cholinergic input to V1 is required for induction but not maintenance of plasticity. Moreover, timed optogenetic stimulation of cholinergic inputs to V1 could be used to mimic delivery of reward and thereby condition the timing of visually evoked neural response in V1, implying that the cholinergic system is the signal that governs reward timing (Liu et al., 2015). Remarkably, further work in ex vivo slices of V1 established that timed delivery of a cholinergic agonist, carbachol, could mimic the delivery of reward in the whole animal, so that the response of neurons to a brief electrical stimulus could be extended out to a delayed delivery of carbachol (Chubykin et al., 2013). Again, this is further evidence that self-contained systems for associative learning exist within V1, but it also confirms that cholinergic input to V1 is capable of signaling the delivery of a reward, just as dopamine appears to do elsewhere (Schultz, 1998). Finally, it is important to note that this reward timing in V1 actually supports behaviorally manifest learning because much the same timed responses to reward are seen in V1 neurons of rats as they learn to lick at the appropriate time relative to visual stimulus delivery to gain maximal reward. This timed behavior was disrupted if activity of neurons in V1 was perturbed using local optogenetic stimulation during the interval between visual stimulus and reward, demonstrating the importance of appropriately timed neural activity in V1 to the performance of learned behavior (Namboodiri et al., 2015).

Organisms largely determine causality through observance of repeatedly correlated events (Hume, 1896), which is the basis of associative learning (Pavlov, 1927; Skinner, 1974; Thorndike, 1911). The temporal credit assignment problem is a description of the difficulties that exist for the brain in reinforcing only the appropriate stimulus (Pavlov, 1927) or behavior (Skinner, 1974; Thorndike, 1911) among all of those that occur within an extended period prior to reward delivery. Gratification is, after all, usually delayed in the natural world. Trial-and-error learning across a large number of episodes enables the reinforcement of only those cues or behaviors that, respectively, reliably signal or trigger reward, but there is still a requirement for a synaptic or neural mechanism that can bridge the temporal interval between stimulus or behavior and the resulting pay-off. In the case of reward timing in V1, a delay of 2 s can be learned for one eye and 1 s for the other eye even in binocularly responsive cells, implying an underlying synaptic mechanism (Shuler and Bear, 2006). The involvement of synaptic mechanisms, as opposed to a change in the intrinsic firing properties of individual neurons, is confirmed by experimental findings from ex vivo V1 slices that are artificially conditioned with carbachol (Chubykin et al., 2013). Models of this process have introduced the concept of a protoweight, which serves as a biophysical mechanism to demarcate those synapses responsive to the relevant visual input and remain active for a long enough latency to contribute to fully fledged Hebbian synaptic plasticity once reward is finally delivered. A population of protoweights across lateral intracortical connections can serve as a temporary memory of reverberations through V1 ensembles that last up until the point of reward delivery (Gavornik et al., 2009; Namboodiri et al., 2016). Exactly what the molecular players are in this process has yet to be determined, but the study of reward timing in V1 will allow for experimental constraint that is more challenging to attain in higher-order cortex.

Pavlovian Conditioning

In a Pavlovian conditioning procedure, an initially neutral stimulus (the (to-be-)conditioned stimulus, or CS) is presented to the subject paired with a motivationally significant unconditioned stimulus (US) that, on its own, elicits an unconditioned behavioral response (UR). This CS–US pairing can create associations between internal representations of the CS and the US (stimulus–stimulus associations); it can also create associations between the CS and the UR (stimulus–response associations), and it can create associations between the CS and a representation of value or affect (‘good’ or ‘bad’ emotional states). Pavlovian conditioning therefore creates several psychological routes to action.

Of note, while some conditioned responses are concerned with direct physical responses to a predicted stimulus (e.g. blinking, salivation), other so-called ‘preparatory’ conditioned responses are directly concerned with motivated action. For example, subjects may approach appetitive CSs; this Pavlovian conditioned approach response brings the subject directly towards stimuli that are likely to be behavioral goals.