E-mail address: [email protected] (S. Fusi).

Neurocomputing 38}40 (2001) 1223}1228

Long term memory:Encoding and storing strategies of the brain

Stefano FusiInstitute of Physiology, University of Bern, Bu( hlplatz 5, 3012 Bern, Switzerland

Abstract

Plastic material devices, either arti"cial or biological, should be capable of rapidly modifyingtheir internal state to acquire information and, at the same time, preserve it for long periods (thestability}plasticity dilemma). Here we compare, in a simple and intuitive way, memory stabilityagainst noise of two di!erent strategies based, respectively, on fully analog devices thataccumulate linearly small changes and on systems with a limited number of stable states andthreshold mechanisms. We show that the discrete systems are more stable, even with shortinherent time constants, and can easily exploit the noise in the input to control the learning rate.We "nally demonstrate the strategy by discussing a model of a biologically plausible spike-driven synapse. � 2001 Elsevier Science B.V. All rights reserved.

Keywords: Synaptic plasticity; Long term memory; Learning

1. Introduction

Material (arti"cial or biological) learning devices, like the synapses, have thecapability of changing their internal states in order to acquire (learn) and store(memorize) information about the statistics of the incoming #ux of stimulations. Ina realistic situation, the stimulations carrying relevant information are separatedby long time intervals of noisy input which tends to erase the memory of thepreviously acquired information.Moreover the interference of novel stimulations withalready acquired older &memories' may give rise to memory loss (e.g. the oldeststimulations are forgotten to make room for the new ones). This is also known as the

0925-2312/01/$ – see front matter � 2001 Elsevier Science B.V. All rights reserved.PII: S 0 9 2 5 – 2 3 1 2 ( 0 1 ) 0 0 5 7 1 – 9

stability}plasticity dilemma: the memory should be stable against irrelevant inputs(e.g. noise) for long periods and, at the same time, the internal state should be rapidlymodi"ed to acquire the information conveyed by the relevant inputs. This dilemmabecomes particularly arduous when dealing with material memory devices that do notallow arbitrarily large time constants or parameters "ne tuning, especially if thedevices are small (e.g. it is reasonable to assume that permanent changes can not bearbitrarily small).Here we show one possible encoding and storing strategy that solves this dilemma

and we exemplify it by discussing a model of a spike-driven learning synapse. Thestrategy is based on the assumption that information to be coded is redundant: e.g. forthe synapses this means that many cells on the dendritic tree carry similar informa-tion.We compare two possible scenarios: in the "rst each synapse is described in termsof one continuous internal variable x. In the absence of any stimulation, the valueencoded by x is preserved forever. In the second, the synapse is discrete on long timescales: it has only a limited number of attracting stable states: when x drifts away fromone of them, a recalling force drives it back to the closest stable state. To makea change permanent, the internal variable should cross some threshold, to be thenattracted towards a di!erent stable state. Let K be the number of stable states and �xthe minimal distance between two stable states.

2. Preserving information: The stability problem

We now consider the current generated by the synaptic inputs as the relevantvariable. We assume that it is approximately the linear sum of many input neuronalactivities a

�multiplied by the corresponding weights J

�, which, in turn, depend on the

internal state of the synapses. Let I�be the current induced by N neurons that encode

the same information, i.e. that are activated in the same way by a generic stimulus(a

�"a for i"1,2,N):

I�

"

1

N

�����

J�a�"

a

N

�����

J�.

If we start from the fully analog synaptic values and we clamp them to the closest

stable states (see Fig. 1), the error on I�goes as &1/(K�N). If N is large enough (the

code is redundant), the error becomes negligible and there is no relevant loss ofinformation, which would be the only disadvantage of the discrete code. This isa known property of some neural networks (see e.g. [6]).However, memory preservation is much more stable in the case of discrete synapses

since the e!ects of noise do not accumulate. Let �t be the typical `responsea time of thesynaptic device, i.e. the time interval during which any change of an internal variableis established: the noise induces small jumps �x with probability p, either upwards ordownwards, once every �t. The ratio p/�t can also be seen as the rate of events that caninduce permanent changes (e.g. the spikes). Let � be the time constant of the recallingforce: no matter how far x gets from one stable state, in a time of the order of �, it is

1224 S. Fusi / Neurocomputing 38}40 (2001) 1223}1228

Fig. 1. Clipping synaptic e$cacies: passing from fully analog synapses (left) to three-state synaptic e$cacies(right) does not degrade much of the memory. The input neurons (below) are arranged in such a way thatthe "rst N neurons are driven by a generic stimulus to the same activity level. These neurons carry the sameinformation (redundancy) for that speci"c stimulus. The e$cacies are di!erent because other uncorrelatedstimuli, activating di!erent subsets of neurons, had been previously encoded. When clipped to the closeststable state, the synapses are pushed up and down and the "nal `errora on the a!erent current I

�, generated

by N neurons, is equivalent to a noise whose amplitude scales as 1/�N.

driven back to the closest stable state. For the fully analog synapse, after time ¹, the

mean displacement is of the order of �x�p¹/�t. Hence, to have an error of �x, onehas to wait a time of the order of:

¹��&�t��x

�x��p��.

If the internal variable x hits one of the boundaries, this time is even shorter [4]. Forthe discrete synapse, the same error �x is produced when a #uctuation drives theinternal variable across the threshold. This happens with a probability &(p�/�t)� per� where h"�/�x is the number of jumps required to reach �. Hence

¹��&�t

��t�

p��t�

��, (1)

which can be much longer than the time of the fully analog synapse, especially if p issmall. It can be so long, that x practically never hits the boundaries (see Section 4). Thebest case is when h is maximal, i.e. when the synapse is binary. The same behaviorcould be obtained in the analog case by adding an extra device that triggers perma-nent modi"cations only if some threshold is crossed. However, there is accumulatingexperimental evidence that the single synaptic contacts are actually binary on longtime scales [5].

3. Acquiring information: The plasticity problem

It was rather intuitive and well known that discreteness can increase stabilitywithout necessarily degrading memory performance. What was less clear is whetherthis is still true in case of on-line learning, when discrete synapses are updated afterevery stimulus presentation. Actually discreteness can be advantageous also in this

S. Fusi / Neurocomputing 38}40 (2001) 1223}1228 1225

Fig. 2. Updating synaptic e$cacies. The scheme is described in Fig. 1. Upon the presentation of a genericstimulus, the analog synapses (left) are potentiated by �"�x/4. Since theN synapses see the same pre- andpost-synaptic activity they are all updated in the same way. The same change in I

�can be obtained in the

discrete case by modifying only a fourth of the N synapses (synapse �2 in the "gure). This can be obtainedwith a stochastic selection mechanism that updates each synapse with probability q"1/4. Interestingly thepresentation of a generic pattern interferes with the memory of other uncorrelated patterns in the same wayin the two scenarios. Indeed, if f is the fraction of neurons activated by a di!erent stimulus, the "nal changein its current would be fN� in the analog case and fqN�x in the discrete case. For a more general analyticalstudy see [1].

case. Since the code is redundant, there is no need to modify all the synapses. If thefraction of synapses that are changed following each stimulation is small (slowlearning), it is possible to better redistribute the synaptic &memory' resources amongthe di!erent patterns of stimulation and actually recover the optimal storage capacityeven with binary synapses [1]. Slow learning is usually di$cult because it is ratherunlikely that the minimal change � inducible by the input is arbitrarily small. AfterM repetitions of the same signal, the minimal change of I

�would be M�. In the

discrete case, the noise superposed to the stimulations can turn in our favor byproviding a triggering signal which selects in a local and unbiased way a small fractionof synapses to be changed. With the threshold mechanism of the discrete case, theinput, at parity of signal, can induce or not a permanent change, i.e. a transition toa di!erent stable state. In this case the minimal change in Iwould beMq�x, where q isthe transition probability for each synapse. q�x can be much smaller than � and theaverage number of synapses changed after each repetition can be even(1 (see Fig. 2).This scheme has the very attractive feature that it transfers part of the updatingprocess outside the device (e.g. embedded in the input): q is not necessarily related tothe intrinsic dynamics of the system. This can be a much better strategy, especially forsmall devices with short time constants.

4. Spike-driven synaptic plasticity

To demonstrate how the load of generating low probability events can be transfer-red outside the device, we discuss a model of a bistable (K"2) spike-driven learningsynapse which has been recently introduced [3]. The transitions between the twostates are activity dependent and stochastic, even without any intrinsic noise source inthe synaptic device. The synapse exploits the #uctuations in the inter-spike intervals,

1226 S. Fusi / Neurocomputing 38}40 (2001) 1223}1228

Fig. 3. Simulation of stochastic LTP. Pre- and post-synaptic neurons have the same mean rate and thesynapse starts from the same initial value. At parity of activity (signal), the "nal state is di!erent in the twocases.

Fig. 4. Contour plots of LTP and LTD probabilities (q) on log scale vs pre- and post-synaptic neuron ratesfor a 500 ms stimulation. LTP occurs when pre- and post-synaptic rates are both high. Around the whiteplateau, P

���drops sharply and becomes negligible for spontaneous rates. The strong non-linearity allows

to discriminate easily between relevant signals and background noise.

which are the results of the collective dynamics of the network. This noise is alwayssuperposed to the signal (pre- and post-synaptic mean frequencies) during the stimula-tion and is di!erent from synapse to synapse. Each pre-synaptic spike drives theinternal state x either up or down depending on whether the post-synaptic depolariz-ation is above or below the threhsold �

�. LTP/LTD might occur or not at parity of

mean pre-synaptic and post-synaptic activities (see Fig. 3). In this case p (see Eq. (1)) isthe probability of coincidence of two events (e.g. a pre-synaptic spike and highdepolarization) and hence can be very small. In Fig. 4 we show that the stochastictransitions between stable states are easily manipulable. In the presence of noise (low,spontaneous activity), the time to wait for a transition can be of the order of years,even if the longest time constant � is of the order of 100 ms, whereas under stimulation(higher frequencies) the transition probabilities are easily controllable in the range

S. Fusi / Neurocomputing 38}40 (2001) 1223}1228 1227

10��}10��, as expected from Eq. (1). Extensive simulations of the learning process innetworks of integrate-and-"re neurons connected by the proposed synapse are pre-sented in [2].We believe that this strategy based on the combination of discreteness and external

stochasticity is a good general strategy for storing variables on long time scales and itis likely to underlie the basic mechanisms of many other biological small systems.Moreover this analysis shows that synaptic models in which single events (e.g. singlespikes) modify permanently the synaptic e$cacy can be hardly used as long termmemory devices since the information acquired during the stimulation would beerased in a short time by the spontaneous activity.

References

[1] D.J. Amit, S. Fusi, Learning in neural networks with material synapses, Neural Comput. 6 (1994)957}982.

[2] P. Del Giudice, M. Mattia, Long and short term synaptic plasticity and the formation of workingmemory: a case study, Neurocomputing 38}40 (2001) 1175}1180, this issue.

[3] S. Fusi, M. Annunziato, D. Badoni, A. Salamon, D.J. Amit, Spike-driven synaptic plasticity: theory,simulation, VLSI implementation, Neural Comput. 12 (2000) 2227}2258.

[4] G. Parisi, A memory which forgets, J. Phys. A 19 (1986) L617.[5] C.C.H. Petersen, R.C. Malenka, R.A. Nicoll, J.J. Hop"eld, All-or-none potentiation at CA3-CA1

synapses, Proc.Natl.Acad.Sci. 95 (1998) 4732.[6] H. Sompolinsky, The theory of neural networks: the Hebb rule and beyond, in: L. van Hemmen, I.

Morgenstern (Eds.), Heidelberg Colloquium on Glassy Dynamics, Springer, 1987.

Stefano Fusi was born in 1968 in Florence, Italy. He received his master degree inphysics from the university of Roma in 1992. He had been working as a researcherin the National Institute of Nuclear Physics (INFN, Roma) from 1993 to 1999 andreceived a Ph.D. in physics from the HebrewUniversity of Jerusalem in 1999. He iscurrently working in the Institute of Physiology of Bern. His research interestsinclude long-term synaptic plasticity, in vivo experiments on behaving monkeys,neuromorphic VLSI hardware and analytical studies of networks of spikingneurons.

1228 S. Fusi / Neurocomputing 38}40 (2001) 1223}1228