A number of studies have demonstrated that neurons show a response decrement under stimulus repetition:
Neuroimaging studies of implicit memory retrieval in humans (e.g. Buckner et al., 1995; Demb et al., 1995; Schacter et al., 1996)
Extracellular single-unit neuron recordings in behaving primates (e.g. Miller, Gochin, & Gross, 1991; Miller, Li, & Desimone, 1991; Ringo, 1996)
Recordings from inferotemporal cortex of a macaque monkey during repeated presentation of common objects at different ISI's (Miller, Gochin, & Gross, 1991).
Several properties of this decrement (often called "repetition suppression" in the primate literature) have been characterized (Desimone, 1996; Wiggs & Martin, 1998):
the decrement is stimulus-specific
it can be relatively long-lasting (up to 24 hours)
it is graded with the number of repetitions, asymptoting after 6 to 8 repetitions to around 30-40% of initital response
it occurs during passive fixation, under anesthesia, and after cholinergic blockade, suggesting that it is automatic and an intrinsic property of cortical neurons
it occurs regardless of the behavioral significance of a stimulus Some researchers have noted that these properties make repetition suppression a good candidate for the neural basis of priming effects (Desimone, 1996).
However, the seemingly automatic nature of repetition suppression and the fact that it is reducing the ability of neurons to code information by firing rate leads one to ask:
1) What is the underlying mechanism of repetition suppression?
2) How is it that activity can be sustained and integrated in circumstances that require it?
We suggest that this response decrement is largely due to short-term plasticity which is present at individual synapses ("synaptic depression") - a mechanism which matches the characteristics of repetition suppression well and appears to be quite widespread in the neocortex (Shaw & Teyler, 1982; Thomson, Deuchars, & West, 1993).
In circumstances which require activity to be maintained, we propose that neurons have to learn to cope with the presence of synaptic depression, building up strong functional connections between cortical areas responsible for maintaining and integrating semantic information over time (e.g. between frontal and more posterior areas).
We will attempt to provide support for this hypothesis by showing that a connectionist model which incorporates synaptic depression and learns to maintain semantic information in the face of it is capable of accounting for two sets of phenomena from the domain of semantic memory:
semantic satiation in normal subjects
access & degraded-store semantic impairments
"Semantic satiation" is the apparent attenuation of the meaning of a word as the result of massed repetition.
Smith (1984; Exp. 2)
Participants:
16 female and 16 male undergraduates
Task:
Participants were asked to repeat the name of a semantic category either 3 or 30 times. They were then asked to decide whether a target word was a member of the repeated category.
Design:
Number of Repetitions (3 vs. 30 reps) and Membership (Member vs. Non-member) were manipulated within subject. Target words served as members and non-members equally often.
Hypothesis:
In the course of normal language processing, the brain has learned to maintain semantic information in the face of synaptic depression by building up strong connections between posterior semantic areas and more frontal ones. However, it is rare that one must process rapid strings of identical repetitions. Therefore, semantic satiation may be due to synaptic depression of peripheral information (e.g. phonological), rather than semantic information. The initial priming effect following 3 repetitions may be due to residual activity - with category members sharing many of the same neurons.
Many neurophysiological studies have implicated a pre-synaptic locus of synaptic depression (e.g. Varela et al., 1997; Thomson et al., 1993; Klein et al., 1980). This would tend to limit its role as a general-purpose learning mechanism. Thus, our model includes both a relatively transient synaptic depression and a learning algorithm responsible for long-term weight changes.
Synaptic depression is implemented by adding a short-term bias to the netinput of each unit in the network:
stbiasj(t) = -c * aj(t-1) + lambda * stbiasj(t-1)
where c is the "buildup" parameter, and lambda is the "decay" parameter. The values of c and lambda were constrained to be the same for all units in the network.
Input Patterns:
a "word is a temporal sequence of 2 phonological patterns or "syllables"
8 unique syllables (4/12) units on for each)
32 total words were constructed from the 8 syllables
Target Patterns:
32 semantic patterns (5/20 units on for each) forming 2 non-overlapping categories (16 members each)
In each category, 8 patterns are "closely" related (on average, share 3.875 "on" units with prototype), and 8 are "distantly" related (share 2.375 "on" units).
Each semantic pattern is paired at random with an input "word" The network was trained with a continuous, temporal version of backpropagation discussed in Pearlmutter (1989).
Half of the 32 training patterns were assigned to be high frequency (presented twice as often during training) and half low frequency.
Time Course of a Single Training Pattern:
ITI's were biased to be short if members from the same category followed each other.
Results:
After approximately 30,000 passes through the training set, the network performed at 91% correct.
The trained network was repeatedly presented with a "category name" (prototype of one of the two categories) for either a small or large number of repetitions (2 vs. 10).
The network was then presented with a target word.
Category decision was modeled by computing the similarity (ndp) of the resultant semantic pattern with the centroids of both categories throughout the presentation of the target. The closest category was the network's choice ("member" decision = choosing category 1 when the category 1 name had been repeated). Reaction Time was modeled by number of processing cycles needed to reach a certain confidence threshold.
While the means for member decisions appeared to move in the correct direction, Repetition was not significant (Member: p > .2; Non-member:n.s.).
The "Access/Degraded-Store" Distinction is a proposal for two different types of semantic impairment: Damage to "access" processes vs. damage to semantic representations themselves (Warrington & Shallice, 1979). These two different types of impairment are predicted to yield a contrasting pattern of behavioral effects.
Warrington & Cipolotti (1996)
Patients:
Two global aphasics (A1, A2) with wide-spread left hemisphere damage, largely sparing temporal cortex.
Four "semantic dementia" patients (S1-S4), all with focal atrophy of the left temporal lobe.
Task:
Spoken-word picture matching (point to a picture in an array of 4 that matches the word spoken by the experimenter)
Design:
Inter-Trial Interval (ITI; 1 sec vs. 15 secs), Semantic Relatedness (Close vs. Distant), and Lexical Frequency (High vs. Low) were completely crossed. Blocks consisted of 3 repetitions of all 4 pictures in an array, probed in pseudorandom order.
Comparison of observed consistency with random distribution (significant difference = consistent)
For the first two within-block repetitions at a fast rate, a comparison to determine if there was a trend to get the first probe right and then the second probe wrong.
Hypothesis:
In the semantic satiation simulation, the network learned to deal with synaptic depression at semantics by building up strong connections with the cleanup layer.
If this compensation were removed by damaging the connections between semantics and cleanup, the "access" pattern should be produced: Semantic information would decrement strongly under a fast rate, high semantic relatedness, and following repetition. Frequency might be expected to have less impact because it is relatively unrelated to these factors.
Under damage to semantic units, the "degraded-store" pattern should be produced. Mild to moderate damage would leave the semantic<->cleanup interactions largely spared, preventing a rate effect. However, impaired interactions of semantics with the bottom-up pathway (as well as with the cleanup pathway to a certain degree) might be expected to give rise to a larger frequency effect.
The same network used in the semantic satiation simulation was damaged in order to simulate "access/refractory" and "degraded-store" semantic impairments.
"Access/refractory" damage:
randomly removed connections between cleanup and semantics
"Degraded-Store" damage:
randomly removed semantic units
Damage was sampled to produce a comparable level of performance to that of the patients and matched performance for the two types of damage. Each lesion severity was repeated 20 times to insure a stable estimate of performance.
Rather than modeling spoken-word/picture matching, per se, we modeled only the auditory word comprehension aspects of the task. This is partially motivated by the observation that "access/refractory" patients on record tend to perform at ceiling on picture/picture matching.
Each lesioned network was presented with 4 types of arrays of 4 words each, as in Warrington & Cipolotti (1996): Close/HF, Close/LF, Distant/HF, Distant/LF.
In each testing block, all 4 words in one of the arrays were probed 3 times in a pseudorandom order and at a fixed ITI (either 2 or 30 time units).
Each array was presented at both a fast rate and a slow rate.
The pattern of semantic activity generated by an input word was compared with the target semantic patterns of all the words in the array. The best match was taken to be the network's response (chance performance = 25% correct).
Comparison of observed consistency with random distribution (significant difference = consistent)
For the first two within-block repetitions at a fast rate, a comparison to determine if there was a trend to get the first probe right and then the second probe wrong.
1) What is the underlying mechanism of repetition suppression?
We have argued that "synaptic depression", an intrinsic mechanism of neocortical neurons, underlies repetition suppression.
This view suggests that response decrement may be counterproductive when activity needs to be sustained.
1) How is it that activity can be sustained and integrated in circumstances that require it?
Our proposal is that neurons must learn to maintain information in the face of synaptic depression by building up strong functional connections between cortical areas.
The results of our simulations provide some basic support for these ideas and point to a special role of task demands in shaping representations.