What the previously neutral stimulus is called once it has been paired with an unconditioned stimulus and now elicits a conditioned response?

The Discovery of the Conditioned Reflex

The Nobel laureate physiologist Ivan Petrovich Pavlov discovered the conditioned reflex in the first decade of the twentieth century. The initial work involved the autonomic nervous system with Pavlov’s use of the salivating reflex in dogs. He discovered that presenting a neutral stimulus such as a bell that never normally elicited salivation could come to elicit salivation if presented repeatedly before giving the dog meat powder. The learned response (called the ‘conditioned response’) was the dog’s salivation to the bell. Presentation of the bell followed by presentation of the meat powder led to the formation of an association. Throughout the century following the discovery of classical (Pavlovian) conditioning, it has become one of the most widely employed paradigms for the investigation of associative learning and memory and its neurobiological substrates. Pavlov was the first scientist to observe that old dogs condition more slowly than young dogs. Thus, research on classical conditioning in normal aging has a long and illustrious history.

C Classical Conditioning

Classical conditioning resembles sensitization in that the response to a stimulus to one pathway is enhanced by activity in another. In classical conditioning an initially weak or ineffective conditioned stimulus (CS) becomes highly effective in producing a behavioral response after it has been paired temporally with a strong unconditioned stimulus (US). Often a reflex can be modified by both sensitization and classical conditioning. In such cases, the response enhancement produced by classical conditioning (paired presentation of the CS and US) is greater and/or lasts longer than the enhancement produced by sensitization (presentation of the US alone). Moreover, whereas the consequences of sensitization are broad and affect defensive responses to a range of stimuli, the effects of classical conditioning are specific and enhance only responses to stimuli that are paired with the US.

In conditioning of the Aplysia withdrawal response, the unconditioned stimulus is a strong shock to the tail and produces a powerful set of defensive responses; the conditioned stimulus is a weak stimulus to the siphon and produces a feeble response. After repeated pairing of the CS and US, the CS becomes more effective and elicits a strong gill and siphon withdrawal reflex. Enhancement of this reflex is acquired within 15 trials, is retained for days, extinquishes with repeated presentation of the CS alone, and recovers with rest (Carew et al., 1981). The siphon withdrawal reflex can also be differentially conditioned using stimuli to the siphon and mantle shelf as the discriminative stimuli. Using this procedure, we have found that a single training trial is sufficient to produce significant learning, and that the learning becomes progressively more robust with more training trials (Carew et al., 1983). We also found significant conditioning when the onset of the CS preceded the onset of the US by 0.5 sec, and marginally significant conditioning when the interval between the CS and the US was extended to 1.0 sec. In contrast, no significant learning occurred when the CS preceded the US by 2 sec or more, when the two stimuli were simultaneous, or, in backward conditioning, when US onset preceded the CS by 0.5 sec or more (Hawkins et al., 1986). Thus, conditioning in Aplysia resembles conditioning in vertebrates in having a steep ISI function, with optimal learning when the CS precedes the US by approximately 0.5 sec (e.g., Gormezano, 1972).

What cellular processes give classical conditioning this characteristic stimulus and temporal specificity? Evidence obtained over the past several years indicates that classical conditioning of the withdrawal reflex involves a pairing-specific enhancement of presynaptic facilitation. In classical conditioning the sensory neurons of the CS pathway fire action potentials just before the facilitator neurons of the US pathway become active. Using a reduced preparation we have found that if action potentials are generated in a sensory neuron just before the US is delivered, the US produces substantially more facilitation of the synaptic potential from the sensory neuron to a motor neuron than if the US is not paired with activity in the sensory neuron. Pairing spike activity in a sensory neuron with the US also produces greater broadening of the action potential in the sensory neuron than unpaired stimulation, indicating that the enhancement of facilitation occurs presynaptically. Thus, at least some aspects of the mechanism for classical conditioning occur within the sensory neuron itself. We have called this type of enhancement activity-dependent amplification of presynaptic facilitation (Hawkins et al., 1983). Similar cellular results have been obtained independently by Walters and Byrne (1983), who have found activity-dependent synaptic facilitation in identified sensory neurons that innervate the tail of Aplysia. By contrast, Carew et al. (1984) have found that a different type of synaptic plasticity first postulated by Hebb (1949), which has often been thought to underlie learning (see Sejnowski and Tesauro, Chapter 6, this volume), does not occur at the sensory neurnon–motor neuron synapses in the siphon withdrawal circuit.

These experiments indicate that a mechanism of classical conditioning of the withdrawal reflex is an elaboration of the mechanism of sensitization of the reflex: presynaptic facilitation caused by an increase in action potential duration and Ca2+ influx in the sensory neurons. The pairing specificity characteristic of classical conditioning results because the presynaptic facilitation is augmented or amplified by temporally paired spike activity in the sensory neurons. We do not yet know which aspect of the action potential in a sensory neuron interacts with the process of presynaptic facilitation to amplify it, nor which step in the biochemical cascade leading to presynaptic facilitation is sensitive to the action potential. Preliminary results suggest that the influx of Ca2+ with each action potential provides the signal for activity, and that it interacts with the cAMP cascade so that serotonin produces more cAMP (Fig. 2C). Thus, brief application of serotonin to the sensory cells can substitute for tail shock as the US in the cellular experiments, and Ca2+ must be present in the external medium for paired spike activity to enhance the effect of the serotonin (Abrams, 1985; Abrams et al., 1983). Furthermore, serotonin produces a greater increase in cAMP levels in siphon sensory cells if it is preceded by spike activity in the sensory cells than if it is not (Kandel et al., 1983; see also Occor et al., 1985, for a similar result in Aplysia tail sensory neurons). Finally, experiments on a cell-free membrane homogenate preparation have shown that the adenyl cyclase is stimulated by both Ca2+ and serotonin, consistent with the idea that the cyclase is a point of convergence of the CS and US inputs (Abrams et al., 1985).

The conditioned response (CR) that develops during classical conditioning of the autonomic nervous system has been characterized as either a discrete response or a nonspecific response to the conditioned stimulus (CS). A discrete CR is a learned response that has been elaborated from an unconditioned reflexive response to a highly specific unconditioned stimulus (CS), whereas a nonspecific autonomic CR is one of a cluster of concurrent responses elicited by a CS. The unconditioned autonomic response (UR) is viewed as a component of a reactive homeostatic mechanism that services to return a controlled variable to its preset reference level in response to a regulatory challenge—the unconditioned stimulus (US). In contrast, the autonomic CR is seen as an element of a predictive homeostatic mechanism that is engaged prior to the onset of a noxious US. Viewed from this perspective, a discrete CR is a preemptive response that mitigates or totally nullifies the impact of the regulatory challenge (the US). Similarly, the development of nonspecific autonomic CRs may be thought of as the initial stage of a predictive homeostatic mechanism that is followed by the development of somatic CRs (e.g., eye blink) or an integrated somatic motor response pattern, learned through instrumental conditioning, that mitigates (or totally nullifies) a noxious US; the autonomic CRs are thought to reflect a change in the CNS, referred to as conditioned fear. Alternatively, the autonomic CR may be interpreted solely in terms of autonomic responses involved in predictive homeostasis. For example, a bradycardia CR develops in an aversive conditioning experiment in which the animal is restrained, thereby attenuating the blood pressure increase elicited by the US. Although there is ample evidence for instrumental conditioning of autonomic responses, the level of specificity of operant control seems limited. It appears that the CNS programs for somatomotor and cardiorespiratory responses are coupled and these functionally related systems are influenced in parallel by the same process.

C. Fear Conditioning

Conditioned responses, such as freezing, can be observed in relationship to environmental cues that were previously paired with an aversive stimulus, such as a brief foot shock. Such paradigms are referred to as fear conditioning. The main manipulation is the interval between the aversive conditioning and the observation of the CR. Environmental cues can also be manipulated to examine the impact of context on the response. The main dependent variable is the time spent freezing. Several procedural advantages can identified for this paradigm. As in conditioned taste adversion conditioning, typically only one pairing of the CS (the environment) and the US (the shock) is conducted. The behavioral variable (freezing) is easily observable and quantifiable. The equipment needed is minimal. The disadvantage is the relevance to human applications as noted for the conditioned taste adversion paradigm.

Recently this paradigm has come into extensive general use in mouse studies, but its application to gerontological studies is still rather limited. Oler and Markus (1998) and Houston et al. (1999) reported an age-related decrease in the retention of fear conditioning as a function of the delay interval between conditioning and test in F344 rats. Typically freezing behavior strengthens with time presumably because of the “memory incubation.” This phenomenon was observed in young rats, whereas, after 20 days reduced freezing behavior was reported in the aged rats (Houston et al., 1999).

2.3 Extinction

Conditioned reflexes and operant responses can be weakened by extinction. Operant behavior maintained by reinforcement can be reduced in frequency by terminating the reinforcement contingency or by delivering reinforcers independently of responses. Discontinuing a punishment contingency almost always results in the recovery of the punished behavior. A CS can be extinguished by presenting it without the US or by removing the correlation with the US. Extinction procedures do not erase the effects of contingencies but instead produce additional learning. In the case of operant behavior, organisms stop responding because they learn that the operant contingency is broken—extinguished responses reappear without additional training when a reinforcement contingency is reinstated. In the case of Pavlovian extinction, the organism learns that the CS now signals the absence of the US.

INITIAL STUDIES ESTABLISHING THE CONDITIONABILITY OF BLOOD FLOW

Classical conditioning of vasomotor responses in human subjects has been performed for some time, especially in the U.S.S.R. (for summary of this work, see Razran, 1961). Some of the experiments carried out in the United States include those by Baer and Fuhrer (1970), Shmavonian (1959), and Teichner and Levine (1968). The basic paradigm involves the presentation of some neutral stimulus (the conditioned stimulus) such as a light, which does not normally have a discernible effect on vasomotor responses, followed by an unconditioned stimulus which typically elicits a marked change in peripheral blood flow (the unconditioned response). After a number of pairings of conditioned and unconditioned stimuli, the vasomotor change occurs on presentation of the conditioned stimulus alone. Most of the studies involve the use of vasoconstriction rather than vasodilatation, undoubtedly because of the comparative ease with which the former can be elicited in controlled and reproducible fashion by such stimuli as electric shock, loud noises, or the application of cold substances. With respect to practical applications of vasomotor conditioning, the classical conditioning paradigm, of itself, has certain limitations in that it is best suited for the production of episodic rather than sustained effects, and the desired change in blood flow is tied to the repetition of the conditioned stimulus, which would tend to restrict the generality of its use.

The operant training of vasomotor responses was first successfully carried out by a Russian investigator, Lisina, in 1965. This investigator first elicited vasomotor responses reflexly, but then obtained operant control of them through the use of “additional afferentation” or biofeedback. This technique of training has been termed “operant-respondent overlap” (Keller & Schoenfeld, 1950). In 1968, Snyder and Noble were able to increase the frequency of vasoconstrictive events by wholly operant methods involving presentation of a light when finger pulse-volume amplitude fell below a criterion value. DiCara and Miller in a well-known study (1968b) showed that rats paralyzed with d-tubocurarine could be trained in a single session to differentially constrict the vasculature of one ear while dilating the vasculature of the other ear. Electrical stimulation of a site within the hypothalamus was used both as a reinforcement and as a source of immediate feedback concerning the performance of desired responses. Miller and DiCara (1968) also demonstrated that curarized rats can learn to effect large changes (increases or decreases) in urine output by altering glomerular filtration rate and–of particular interest here–renal blood flow. In a third study (DiCara & Miller, 1968a), paralyzed rats were trained to change blood flow in the vessels of the tail.

Alcohol

Conditioned responses to cues for administration of ethanol have been extensively investigated.55 These have been motivated by the search for how environmental stimuli may contribute to intoxication, and by a search for how environmental stimuli may affect ethanol tolerance.56

The findings of Gundersen et al55 are of particular interest. They used BOLD fMRI to investigate the effect of alcohol on brain activation in social drinkers, and used a balanced placebo design to control for alcohol-related expectations. Thus, one half of the participants consumed a soft-drink without alcohol before the scanning, and half of these were informed correctly about the drink’s content; the other half were informed incorrectly that an alcoholic beverage was consumed. The other half of the participants consumed the soft-drink with alcohol before the scanning, and half of these were informed correctly about the drink’s content; the other half were informed incorrectly that a soft-drink was consumed. Interestingly, alcohol decreased neuronal activation, mostly in the dorsal anterior cingulate cortex and in prefrontal areas, while expectancy, i.e. when subjects were told that they got alcohol but received placebo, increased neuronal activation in the same areas. Thus, alcohol intoxication and expectancy had opposite effects on neuronal activation. While the response to placebo is not a conditioned compensatory response, it may still be interpreted as a compensatory response to the effects of alcohol.

MATERIALS AND METHODS

Classical conditioning of fixed bees to an odor stimulus (Kuwabara, 1957) has proved to be a very useful way to study the physiological and pharmacological aspects of learning. Bees extend their probosces (tongues) reflexively if the antennae are touched with a drop of sucrose solution. The sucrose solution acts as an US when presented to the proboscis. Association between the CS (olfactory odor or mechanical stimulation of antennae) and the US is established within 3 sec. The memory trace of a single learning trial is erased by electroconvulsive shock, cooling, or narcosis within the following 3 min. Later, the memory trace is resistent to experimental manipulations (Menzel et al, 1974). The transfer from short-term to long-term memory does not occur in the chemo-sensory neuropil (antennal lobes) nor in the motor-output region of the brain, but in a special associative region of the insect brain, the mushroom bodies (Menzel et al, 1974; Erber et al, 1980). The mushroom bodies are two structures, one in each half of the protocerebrum (the most frontal region of the insect brain), which consist of 160.000 very fine neurons (Kenyon cells). The inputs are the two cup shaped, dorsally located calyces; the output regions are two lobes, the α-lobe pointing frontally and the ß-lobe pointing towards the midline, deep within the protocerebrum. Thus, the Kenyon cells arborize their dendrites in the calyces, bifurcate midway in the pedunculus and project to both α - and ß-lobe. Local cooling at different times after single trial learning revealed that the α-lobe is a particularly important structure for the consolidation of a long lasting memory trace (Menzel et al, 1974; Erber et al, 1980).

PAVLOVIAN CONDITIONING AND PROBLEMS IN HUMAN BEHAVIOR

The CR work begun by Pavlov has found clinical applications far beyond those ever thought possible by Pavlov himself. In his later life Pavlov attempted to apply his research findings to hospitalized psychiatric patients. He thought his observations on neuroses experimentally induced in animals would help him to understand and treat acutely disturbed psychiatric patients. His therapeutic attempts, however, met with little success. His failure stems from three sources. First, Pavlov's original observations were made on animals totally isolated from all social stimuli. Therefore Pavlov overlooked the entire realm of interpersonal relationships in the formation of neurotic and psychotic processes. Second, whether neuroses are strictly phenomena of conditioning (i.e., difficult discriminations) independent of emotional factors and genetic predisposition was opened to question. Third, and perhaps most important, is the question of whether schizophrenic processes are exclusively a human phenomenon.

One of the clinically relevant uses of Pavlov's techniques involves the assessment of cortical function in patients. Work with the CR has been applied by Gantt and others in determining the status of a patient with respect to his ability to learn, to establish rapport with those around him, to differentiate psychogenic from organic bases for mental disease, and to assess the damage of electroconvulsive treatment (ECT), leukotomies, and so forth. Some of the procedures used have involved an eye blink CR to a tap at the root of the nose, avoidance responses to a faradic shock applied to the finger, and the psychogalvanic response. The procedure found most useful requests the patient to identify the various stimuli by name, tests the threshold for perception and pain in the hand, and then repeats each pair of signals five times before reinforcing one signal with an appropriate level of shock stimulation. The patient is then told to press a lever when he expects a shock. If the patient learns the difference between excitatory and inhibitory signals, he is graded A; some impairment is graded B, and marked impairment with an inability to differentiate the stimuli is graded C. In addition to the motor, respiratory, and cardiac recordings, the psychogalvanic response can be measured. It is generally parallel to the other measures but is usually grossly impaired in organic cases and less so in functional disorders [28].

Clinical approaches today that use Pavlovian techniques to treat behavioral disorders are generally encompassed in the field of behavior therapy. Beginning with the observations of Pavlov and John B. Watson, investigators Wolpe, Salter, Reyner, John Paul Brady, and many other investigators have been using both operant and Pavlovian conditioning techniques in successful attempts to alleviate a broad range of behavioral and psychosomatic problems [2,3,14,35,36]. They have been especially successful with disorders labeled as neurotic or psychosomatic in character and less so with more acutely disturbed types of disorders, such as schizophrenia.

It has been a general assumption of the behavioral approach that many of the pathological responses and symptoms seen in patients have been CRs reinforced by fear and anxiety. Such CRs do not extinguish because of the internal nature of the UR, which does not depend on external events for its elicitation. Through the concepts of reciprocal inhibition and systematic desensitization, competing responses that are not compatible with anxiety may be paired with repeated presentations or visualizations of a disturbing stimulus constellation, effecting an extinction of the symptomatic CR.

The clinical field of behavior therapy has become a highly refined discipline and is attempting to develop systematized research methods to evaluate the therapeutic efficacy of its techniques. Questions still remain, however, about the extent to which behavioral changes are brought about primarily by the presence of the therapist.

One of the areas Pavlov identified that needs further exploration in terms of understanding psychopathology is the question of the interaction of conditioning processes with genetic predisposition to produce abnormal behavior. In recent years the work of Scott and Fuller at Bar Harbor, Maine, as well as the work of Oddist Murphree, Newton, Dykman, and their colleagues at Little Rock, Arkansas, has suggested that both genetic inheritance and conditioning are important in producing abnormal behavior [26,30,34]. Murphree and his colleagues, for example, have developed a strain of normal and severely abnormal dogs, bred from the same original litter, which showed markedly different patterns of behavior. Among the more interesting abnormal behavior patterns they have developed is an analog of catatonic stupor genetically bred in these dogs [27]. These scientists have also observed that the autonomic nervous system of these dogs is also quite different from that of normal dogs. Perhaps of greatest interest is the fact that the dogs’ autonomic reactions to social contact are markedly disturbed [27].

Another aspect of work developed from Pavlov's model is the recognition of the biological significance of social contact. Pavlov isolated his animals from all social contact because he noted that the dog was responsive to all types of contact, especially to human contact. This was a serious empirical problem for Pavlov, and it initially prevented him from studying conditioning in dogs because they were always being distracted by the human. Consquently he developed sophisticated experimental chambers to isolate the animal from all uncontrolled environmental and social stimuli. This development was refined and rigidly adhered to by almost all psychologists and physiologists. Much current research continues to look upon social isolation as a necessary prerequisite; the Skinner box, for example, enables the rat to be placed in a box all alone.

Social contact was shown to be important in the Pavlovian conditioning paradigm when attempts were made to condition the cardiovascular system. It was observed in this research that the cardiovascular system of dogs was remarkably responsive to human contact, in fact, so remarkably reactive that it became difficult to ignore this biological phenomenon. Therefore, in an ironic fashion, Pavlov's isolation chamber was used to study the very thing it was developed to control: human contact. The observation that social contact is a powerful cardiovascular stimulus has led to the conviction that a great deal of physical illness, especially cardiovascular disease, may relate not only to early conditioning phenomena but, more importantly, to disruptions in social relationships both at an early age and in later life [20]. Even within clinical settings such as coronary care units and other intensive care units, human contact has clinically important effects on the human heart [22,24].

One final aspect of Pavlovian conditioning needs to be briefly described: the conditioning of drug reactions. This description could be accomplished by referring to drug addiction. In Pavlovian terminology, any stimulus that acts through the central nervous system will elicit physiological reactions that can be conditioned if certain preconditions are present. Like food, shock, or any other physiological reinforcer, pharmacological agents are also unconditional stimuli that elicit unconditional reactions. If such reactions are evoked by the central nervous system, then they can be quickly conditioned. Such a process has more than empirical interest, however; it also has direct clinical ramifications. Alcohol, for example, is an unconditional stimulus that elicits unconditional physiological responses. The question then is, Can such alcohol reactions be classically conditioned? The evidence is overwhelmingly positive. Thus the next question becomes, How and under what circumstances will such reactions be extinguished once the alcohol is removed? It is our belief that for the alcoholic a blinking barroom light or a glass or a bottle can become conditional stimuli which can elicit conditional responses in the body. It has been our experience that such reactions, once conditioned, are quite difficult to extinguish and therefore potentially addictive. What then is conditioned? It is our belief that the bodily reactions (autonomic neurophysiological and biochemical) evoked by drugs such as alcohol and morphine can be elicited by conditional stimuli that are paired with these drugs. This entire area needs more study, but if indeed conditional reactions can duplicate unconditional reactions, then it is at least theoretically possible that the body manufactures its own addictive agents through conditional processes [21,23].

Assuming for the moment that such a situation does exist, then the question becomes, How does one help extinguish such addictive conditional reactions once they have become established? Since the conditional reactions are occurring at the autonomic and perhaps cellular level and therefore are out of the person's conscious awareness, then exclusively conscious attempts to control such reactions are not likely to meet with success. The control or extinction of such reactions will have to involve extinction that is also not entirely in one's consciousness.

4.3.4 Multi-dimensional response classes

A conditioned response does not occur in only a single location. Variability in response location seems inevitable and has been considered important enough to warrant advances in automated techniques that quantify this aspect of behavior in great detail. For example, a response-reinforcer contingency generates more than the response topography necessary for reinforcer delivery. A given response is rarely repeated with the exact same force, duration and location. Yet, the typical key peck or lever press is treated as if the same response is repeated over and over as when we speak of rate of responding. The apparent inconsistency here stems from the fact that the repeated responses have several elements in common such as striking a key, yet differ in some other properties such as location or duration. The response class is defined by some of the common elements, usually consequences such as activating a certain detection device. Studying only one kind of element of the response may lead to conclusions that do not pertain to other elements. The variability in response location portrayed above provides visual aids to an understanding of the difference between descriptive and functional operants (cf. Section 4.1). Consider the data in Fig. 6 from Hori and Watanabe's (1987) experiment. The peck-food contingency generated pecks not only within but also outside the target area. These pecks were never reinforced, yet persisted. The class of functional operants thus encompasses not only pecks to the target area but also pecks outside. In addition, other elements of pecking, such as duration or force, may be functionally related to the peck-food contingency and would similarly be part of the functional operant. Thus, the operant response has many dimensions only few of which are studied customarily. The technology seems to be available for mapping of response classes and a conceptual step forward from the operant as being a one-dimensional response class to the operant as a set of response classes or a multi-dimensional response class.