Sometimes Opponent Process (SOP) Model, in Conditioning

views updated

SOMETIMES OPPONENT PROCESS (SOP) MODEL, IN CONDITIONING

Sometimes opponent process (SOP) is an associative, real-time, quantitative theory of Pavlovian conditioning. As such, it describes basic principles from which the behavioral regularities of Pavlonian conditioning can be deduced, and it makes predictions about yetto-be observed Pavlovian phenomena. It specifies rules for stimulus representation, how learning occurs, and how learning that cannot be observed directly in translated into performance. This article does not present the equations that describe these principles, but they are available in related articles (Mazur and Wagner, 1982; Wagner, 1981).

SOP's Basic Principles

Theories of learning assume that experiences are recorded in a theoretical memory system. They describe how that memory system is conceptualized, how experiences come to be represented, and how memories affect behavior. As for any theory, these basic principles are the assumptions the theory makes from which the predictions will follow. A good theory strives to explain the observable phenomena through a priori assumptions that are as few and as simple as possible.

SOP assumes that experiences activate corresponding theoretical representations in the memory system. For example, a cat hears the sound of a can opener and then is fed; these events may activate corresponding sound and food representations. The cat does not have to learn how to represent these incoming events. What it does learn is how to associate one event with another; that is, the cat may learn to associate the sound of the can opener with the food, as a function of experiencing that sound and the food together in time and space. As is common in psychological theories, SOP assumes that memories—or, representations—become associated with each other as a function of their temporal and spatial contiguity, and that what is learned are changes in connections (associations) among representations. SOP further proposes that observable behavior reflects which representations are being processed, and how strong their processing is.

The central principle of SOP is that stimulus representation consists of a large but finite set of theoretical elements that are in one of three dynamic states. This is illustrated in Figure 1a. To use the cat-food example: When the cat encounters the food, elements in the corresponding cat-food representation are activated. The course of activation is determined: there is a brief period of focal processing during which the elements are in a state designated A1. These elements do not stay long in this focal activation state, however, but decay passively into a secondary, more peripheral state designated A2. Eventually, when the food is no longer present, the elements passively decay further into inactivity (I). How long an element might be in each dynamic state depends on three parameters: p₁, which reflects the perceived salience of the incoming stimulus and is assumed to increase monotonically with the intensity of the stimulus; p_d1, which determines how likely an element is to go from A1 to A2; and p_d2, which determines how likely an element is to go from A2 to I. It is assumed that p_d1p_d2.

Stimulus Representation

Figure 2 depicts the application of these rules for elemental dynamics to the instance of exposure to a stimulus that occurs in real time. The figure shows the proportion of total stimulus representation elements that are active across time. In each moment in which the stimulus is presented, p1 of the inactive elements are activated to the A1 state. However, pd1 of those elements in A1 also decay to A2. The result is an A1 function that shows an increase followed by a decrease and relative flattening, until the stimulus is withdrawn and there is decay to inactivity. The form of the A1 function, which is SOP's stimulus representation, is similar to that which has been observed in psychophysical experiments with human subjects who are asked to estimate the perceived intensity of a stimulus (e.g., light, sound, or smell) across time. They report an initial growth in the intensity of sensation followed by a decrease—an adaptation to the stimulus—before it is turned off (Marks, 1974).

Rules for Learning

In SOP, learning—the acquisition of associations between stimulus representations—is determined by the conjunction of active A1 and A2 elements. If the sound of the can opener is considered a conditioned stimulus (CS), and the food an unconditioned stimulus (US), then an excitatory association between that CS and US will come about to the extent that the elements that represent the CS and the elements that represent the US are currently in the A1 state. The likelihood of such concurrent CS_A1 and US_A1 activation increases with the degree of contiguity of the two events. An excitatory association is one that may activate the representations that are linked; hearing the opener now may elicit a memory of food.

SOP captures the difference between the experience of an event and its memory by assuming that associatively activated representations reach only the A2 processing level. Thus, once an association between a CS and US is established, a CS may activate its associated US representation to its A2 processing state, i.e., US_A2. This is shown in Figure 1b as the direct link from I to A2. The likelihood of such activation is determined by the parameter p2, which is a function of the strength of the CS trace and the strength of the association.

Associative strength can also be inhibitory, where an inhibitory association may make it harder for one representation to activate another to which it is linked. Inhibitory associative strength grows when the elements of one representation are in A1 and elements of another are in A2. An effective way to produce an inhibitory CS-US association is to present the US first and then the CS. This should be effective to the extent that it ensures the concurrent processing of CS_A1 with US_A2. Excitatory tendencies (V⁺) and inhibitory tendencies (V^-) are assumed to summate in a single associative connection (σV) that may have a net excitatory or net inhibitory strength.

Rules for Performance

The performance rule in SOP follows from its assumptions about dynamic states and the nature of the learned associations: a conditioned response (CR) is functionally the same as the behavior that is elicited by the A2 processing state of the stimulus with which it is associated. The behavior that is identified as a CR, therefore, can be elicited by presenting the associated US and looking for its secondary (A2) response.

Acquisition and Extinction

Figure 3a shows the expected change in net associative strength and the corresponding stimulus traces and increments in V⁺ and V^- for a first CS-US pairing, a fiftieth CS-US pairing when learning is asymptotic, and an initial extinction trial (Figure 3b). On Acquisition, Trial 1, there is a considerable overlap of CS_A1 and US_A1, along with smaller overlap of CS_A1 and US_A2. (There is overlap of CS_A2 and US_A1 and US_A2 also, but the associations thus formed are presumed to have little behavioral effect.) The contiguous CS_A1/US_A1 processing produces a relatively large increase in V⁺, and the smaller CS_A1 and US_A2 overlap produces relatively little V^-, resulting in a relatively large gain in σV CS-US. After many pairings (Acquisition, Trial 50), the onset of the CS elicits US_A2 processing as a function of the associative connection that is established; that is, the CS now results in a CR (recall that the CR reflects US_A2 processing). There also is a reduction in US_A1 when the US is presented because of the effect of the CS to put elements into US_A2, making these elements unavailable for activation to US_A1. This reduction in US_A1 processing can be seen in a reduced unconditioned response (UR) to the US, where the primary UR is assumed to reflect US_A1 processing. Finally, because the US is not present until the end of the CS period, there also is considerable overlap of CS_A1 and US_A2, which results in an increment in V^-. At asymptote, the V⁺ and V^- cancel each other out, and there is no net change in associative strength. The negatively accelerated course of learning, the elicitation of a CR by a CS, and the so-called conditioned diminution of the UR to that US, are fundamental behavioral characteristics of Pavlovian conditioning.

Extinction occurs (Extinction, Trial 1) with CS-alone trials. Now, because the CS has the capacity to elicit US_A2 processing, there is an increment in V^- and no counteracting increment in V⁺, since the US is not presented. With continued presentations of the CS alone, there is a gradual decrease in σV CS-US and less and less of a tendency for the CR to be elicited. The latter also is a behavioral characteristic of Pavlovian conditioning.

Cue-Competition Effects

SOP explains cue-competition effects in Pavlovian conditioning with the same principles as those that explain acquisition and extinction. One powerful cue competition effect is blocking, a condition in which learning a CS-US association fails in spite of the good temporal contiguity of the two stimuli. For example, suppose that the cat of our previous example had previously learned to associate the turning on of the kitchen light with food and that only then did we present the sound of the can opener and the light together preceding the food. Now the cat would be unlikely to learn to make a conditioned response to the sound. It appears that previously learning to associate the light with food makes the normally effective pairing of sound and food ineffective for association.

The blocking effect can be understood with reference to the functions in Figure 3 for Acquisition Trial 50. Here, the previously trained CS (the light) regularly elicits a CR, reflecting the US_A2 processing elicited by that CS. The addition of the novel CS (the sound), will result in the overlap of CS A1-sound with the same US_A1 and subsequent US_A2 processing that is elicited already by the conjunction of CS A1-light and the US. That is, there will be an initial increment in V^-, followed by a comparable increment in V⁺, that will accrue equally for both CSs. Since V^- and V⁺ balance out, there is no net increment in either association. The CS sound US association, starting at 0, thereby fails to acquire any σV. That is, the cat will not learn a sound-food association.

Conclusion

SOP is able to explain much of what is known about Pavlovian conditioning, including cue-competition effects. By rationalizing excitatory and inhibitory learning in terms of A1/A1 and A1/A2 conjunctions in time, it allows us to understand how the outcome of a conditioning trial depends on the order of the CS and US and the difference between simple conditioning and conditioned inhibition training. By rationalizing priming effects in terms of A1 and A2 effects over time, it allows us to understand blocking (as shown above), as well as short-term habituation and pre-trial CS and US exposure effects. By rationalizing the relationship of a CR and UR as the CR reflecting only US_A2 processing, without the US_A1 processing additionally elicited by the US, we can understand why the CR sometimes mimics the UR and sometimes does not.

Extensions of the SOP model have been developed to increase its theoretical power, to allow for an understanding of occasion setting (Brandon and Wagner, 1998), CR timing (Brandon, Vogel, and Wagner, 2002), and various differences in Pavlovian conditioning involving skeletal versus emotional responses (Wagner and Brandon, 1989). Wagner and Donegan (1989) have further indicated how it may relate to the known neural circuitry for eyeblink conditioning.

Bibliography

Brandon, S. E., Vogel, E. H., and Wagner, A. R. (2002). Computational theories of classical conditioning. In J. W. Moore, ed., A neuroscientist's guide to classical conditioning. London: Cambridge University Press.

Brandon, S. E., and Wagner, A. R. (1998). Occasion setting: Influences of conditioned emotional responses and configural cues. In N. Schmajuk and P. Holland, eds., Occasion setting: Associative learning and cognition in animals. Washington, DC: American Psychological Association

Marks, L. E. (1964). Sensory processes. New York: Academic Press.

Mazur, J. E., and Wagner, A. R. (1982). An episodic model of associative learning, in M. L. Commons, R. J. Herrnstein, and A. R. Wagner, eds., Quantitative analyses of behavior, Vol. 3: Acquisition. Cambridge, MA: Ballinger.

Wagner, A. R. (1981). SOP: A model of automatic memory processing in animal behavior. In N. E. Spear and R. R. Miller, eds., Information processing in animals: Memory mechanisms. Hillsdale, NJ: Erlbaum.

Wagner, A. R., and Brandon, S. E. (1989). Evolution of a structured connectionist model of Pavlovian conditioning (AESOP). In S. B. Klein and R. R. Mowrer, eds., Contemporay learning theories: Pavlovian conditioning and the status of traditional learning theory. Hillsdale, NJ: Erlbaum.

—— (2001). A componential theory of Pavlovian conditioning. In R. R. Mowrer and S. B. Klein, eds., Handbook of contemporary Learning Theory. Mahway, NJ: Erlbaum.

Wagner, A. R., and Donegan, N. H. (1989). Some relationships between a computational model (SOP) and an essential neural circuit for Pavlovian (rabbit eyeblink) conditioning. In R. D. Hawkins and G. H. Bower, eds., The psychology lf learning and motivation, Vol. 23: Computational models of learning in simple neural systems. Orlando, FL: Academic Press.

Susan E.Brandon

Allan R.Wagner

Learning and Memory